Point cloud data encoding device, point cloud data encoding method, point cloud data decoding device, and point cloud data decoding method

Abstract

A method according to embodiments may comprise the steps of: encoding geometry data of point cloud data; and encoding attribute data of the point cloud data. A method according to embodiments may comprise the steps of: decoding geometry data of point cloud data in a bitstream; and decoding attribute data of the point cloud data.

Description

Point cloud data encoding device, point cloud data encoding method, point cloud data decoding device and point cloud data decoding method

[0001] The embodiments relate to a method and apparatus for processing point cloud content.

[0002] Point cloud content is content represented as a point cloud, which is a set of points belonging to a coordinate system that represents three-dimensional space. Point cloud content can represent three-dimensional media and is used to provide various services such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and autonomous driving services. However, representing point cloud content requires tens of thousands to hundreds of thousands of point data points. Therefore, a method is required to efficiently process a vast amount of point data.

[0003] The embodiments provide an apparatus and a method for efficiently processing point cloud data. The embodiments provide a method and apparatus for processing point cloud data to address latency and encoding / decoding complexity.

[0004] However, the scope of rights of the embodiments is not limited to the technical problems described above, and may be extended to other technical problems that a person skilled in the art can infer based on the entire content described.

[0005] To achieve the above-described purpose and other advantages, the method according to the embodiments may include the step of decoding geometry data of point cloud data within a bitstream; and the step of decoding attribute data of the point cloud data.

[0006] A method according to the embodiments may include the step of encoding geometry data of point cloud data; and the step of encoding attribute data of the point cloud data.

[0007] The device and method according to the embodiments can process point cloud data with high efficiency.

[0008] The device and method according to the embodiments can provide a high-quality point cloud service.

[0009] The device and method according to the embodiments can provide point cloud content for providing general-purpose services such as VR services and autonomous driving services.

[0010] Drawings are included to further understand the embodiments, and the drawings illustrate the embodiments along with descriptions related to the embodiments. For a better understanding of the various embodiments described below, one must refer to the description of the embodiments below in relation to the following drawings, which include parts corresponding to similar reference numerals throughout the drawings.

[0011] FIG. 1 shows an example of a point cloud content provision system according to embodiments.

[0012] FIG. 2 is a block diagram illustrating a point cloud content provision operation according to embodiments.

[0013] FIG. 3 shows an example of a point cloud encoder according to embodiments.

[0014] FIG. 4 shows examples of octree and occupancy codes according to embodiments.

[0015] Figure 5 shows an example of a point configuration by LOD according to embodiments.

[0016] Figure 6 shows an example of a point configuration by LOD according to embodiments.

[0017] FIG. 7 shows an example of a point cloud decoder according to embodiments.

[0018] FIG. 8 is an example of a transmission device according to embodiments.

[0019] FIG. 9 is an example of a receiving device according to embodiments.

[0020] FIG. 10 shows an example of a structure that can be linked with a point cloud data transmission / reception method / device according to embodiments.

[0021] FIGS. 11 and 12 illustrate a conventional partial encoding and partial decoding process.

[0022] FIG. 13 shows an example of a point cloud data configuration consisting of layers according to embodiments.

[0023] FIG. 14 shows a bitstream segment according to embodiments.

[0024] FIG. 15 shows a bitstream fragment according to the embodiments.

[0025] FIG. 16 illustrates a bitstream alignment method according to embodiments.

[0026] FIG. 17 illustrates a bitstream alignment method according to embodiments.

[0027] FIG. 18 shows an example of symmetrically selecting geometry attributes according to embodiments.

[0028] FIG. 19 shows an example of asymmetrically selecting geometry attributes according to embodiments.

[0029] FIG. 20 shows an example of a slice configuration according to embodiments.

[0030] FIG. 21 shows an example of a geometry tree structure included in a slice structure according to embodiments.

[0031] FIG. 22 shows a layer group structure according to embodiments.

[0032] FIG. 23 shows the structure of a layer group, subgroup, and bounding box according to embodiments.

[0033] FIG. 24 shows a context reference of a layer group slicing structure according to embodiments.

[0034] FIG. 25 shows an example of a context reference structure of a layer group according to embodiments.

[0035] FIG. 26 shows a context buffer change of a parent subgroup reference according to embodiments.

[0036] FIG. 27 shows a context buffer change of a root layer group reference according to embodiments.

[0037] Figure 28 shows an example of buffer management using context reference indicators.

[0038] FIG. 29 is a comparison table of memory usage and compression loss for context reference methods according to embodiments.

[0039] FIG. 30 is a comparison table of memory usage and compression loss for context reference methods according to embodiments.

[0040] FIG. 31 is a comparison table of average bit rates between a root layer group reference and a parent subgroup reference according to embodiments.

[0041] FIG. 32 is a bitdule comparison table based on whether a context reference indicator is used according to the embodiments.

[0042] FIG. 33 illustrates a list-based context management method according to embodiments.

[0043] FIG. 34 illustrates context buffer management according to embodiments.

[0044] FIG. 35 shows the layer group coding order of the breadth-first search method and the depth-first search method according to the embodiments.

[0045] FIG. 36 shows a bitstream structure according to embodiments.

[0046] FIG. 37 shows a set of sequence parameters according to the embodiments.

[0047] FIG. 38 shows a dependent geometry data unit header according to embodiments.

[0048] FIG. 39 shows a dependent attribute data unit header according to embodiments.

[0049] FIG. 40 shows a layer group structure inventory according to embodiments.

[0050] FIG. 41 shows an encoder according to embodiments.

[0051] FIG. 42 shows a decoder according to embodiments.

[0052] FIG. 43 shows a sub-bitstream classifier flowchart according to embodiments.

[0053] FIG. 44 shows an encoder operation flowchart according to embodiments.

[0054] FIG. 45 shows a decoder operation flowchart according to embodiments.

[0055] Figure 46 illustrates a context buffer management method of a conventional layer group slicing method.

[0056] FIG. 47 illustrates a context buffer management method according to embodiments.

[0057] FIG. 48 illustrates a context buffer management method according to embodiments.

[0058] FIG. 49 shows a geometry data unit header according to embodiments.

[0059] FIG. 50 shows a dependent geometry data unit header according to embodiments.

[0060] FIG. 51 shows an attribute data unit header according to embodiments.

[0061] FIG. 52 shows a dependent attribute data unit header according to embodiments.

[0062] FIG. 53 shows an experimental result table of width priority according to the embodiments.

[0063] FIG. 54 shows an experimental result table of depth priority according to the embodiments.

[0064] FIG. 55 shows a set of sequence parameters according to the embodiments.

[0065] FIG. 56 shows a dependent geometry data unit header according to embodiments.

[0066] FIG. 57 shows an attribute data unit header according to embodiments.

[0067] FIG. 58 shows a dependent attribute data unit header according to embodiments.

[0068] FIG. 59 shows a geometry data unit header according to embodiments.

[0069] FIG. 60 illustrates a partial decoding context buffer management method according to embodiments.

[0070] FIG. 61 shows a set of sequence parameters according to the embodiments.

[0071] FIG. 62 shows a geometry data unit header according to embodiments.

[0072] FIG. 63 shows a dependent geometry data unit header according to embodiments.

[0073] FIG. 64 shows ROI partial decoding according to embodiments.

[0074] FIG. 65 shows an ROI bounding box according to embodiments.

[0075] FIG. 66 shows an ROI bounding box according to embodiments.

[0076] FIG. 67 shows a partial decoding flowchart according to embodiments.

[0077] FIG. 68 shows a pseudocode for a loosely overlapped condition according to the embodiments.

[0078] FIG. 69 shows a pseudocode for a strictly overlapped condition according to the embodiments.

[0079] FIGS. 70a and 70b show pseudocode for the process of releasing nodes and context memory based on whether the ROI and child group are referenced according to the embodiments.

[0080] FIGS. 71a, FIGS. 71b, and FIGS. 71c show pseudocode for a method of integrally releasing attribute decoder resources according to embodiments.

[0081] FIGS. 72a, FIGS. 72b, and FIGS. 72c represent pseudocode for a process for releasing a stored context state according to the embodiments.

[0082] FIGS. 73a and FIGS. 73b show pseudocode for the process of releasing a stored parent node according to the embodiments.

[0083] FIGS. 74a and FIGS. 74b show pseudocode for the process of generating an ROI sub-region list according to the embodiments.

[0084] FIG. 75 shows an encoder flowchart according to embodiments.

[0085] FIG. 76 shows pseudocode for the process of recalculating empty FGS geometry according to embodiments.

[0086] FIG. 77 illustrates a method for controlling the ROI bounding box according to embodiments.

[0087] FIG. 78 illustrates a method for controlling the ROI bounding box according to embodiments.

[0088] FIGS. 79a, FIGS. 79b, FIGS. 79c, and FIGS. 79d represent pseudocode for a process of determining whether a point exists within an ROI according to embodiments and determining whether to perform decoding or release memory based thereon.

[0089] FIG. 80 illustrates a method for controlling the ROI bounding box according to embodiments.

[0090] FIG. 81 illustrates a method for controlling the ROI bounding box according to embodiments.

[0091] FIG. 82 shows a dependent geometry data unit header according to embodiments.

[0092] FIG. 83 shows the node distribution by layer group according to the embodiments.

[0093] FIG. 84 shows the alignment of subgroup unit nodes according to embodiments.

[0094] Fig. 85 shows conventional partial encoding / decoding.

[0095] FIG. 86 shows a partial PCC bitstream according to embodiments.

[0096] Figure 87 shows conventional partial encoding / decoding.

[0097] FIG. 88 shows a layer group slicing structure according to embodiments.

[0098] FIG. 89 illustrates a encoding method according to embodiments.

[0099] FIG. 90 illustrates a decoding method according to embodiments.

[0100]

[0101] Preferred embodiments of the embodiments are described in detail, and examples thereof are shown in the accompanying drawings. The following detailed description, with reference to the accompanying drawings, is intended to describe preferred embodiments of the embodiments rather than merely embodiments that may be implemented according to the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it is obvious to those skilled in the art that the embodiments may be practiced without these details.

[0102] Most terms used in the embodiments are selected from those commonly used in the field, but some terms are chosen at the applicant's discretion, and their meanings are described in detail in the following description as necessary. Accordingly, the embodiments should be understood based on the intended meaning of the terms, rather than their mere names or meanings.

[0103] FIG. 1 shows an example of a point cloud content provision system according to embodiments.

[0104] The point cloud content providing system illustrated in FIG. 1 may include a transmission device (10000) and a reception device (10004). The transmission device (10000) and the reception device (10004) can communicate via wired or wireless means to transmit and receive point cloud data.

[0105] A transmission device (10000) according to embodiments can acquire, process, and transmit point cloud video (or point cloud content). According to embodiments, the transmission device (10000) may include a fixed station, a base transceiver system (BTS), a network, an Artificial Intelligence (AI) device and / or system, a robot, an AR / VR / XR device and / or server, etc. Additionally, according to embodiments, the transmission device (10000) may include a device that communicates with a base station and / or other wireless devices using wireless access technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)), a robot, a vehicle, an AR / VR / XR device, a mobile device, a home appliance, an Internet of Things (IoT) device, an AI device / server, etc.

[0106] A transmission device (10000) according to embodiments includes a point cloud video acquisition unit (10001), a point cloud video encoder (10002), and / or a transmitter (or communication module), 10003.

[0107] A point cloud video acquisition unit (10001) according to the embodiments acquires a point cloud video through processing steps such as capture, synthesis, or generation. The point cloud video is a point cloud content represented as a point cloud, which is a set of points located in a three-dimensional space, and may be referred to as point cloud video data, etc. The point cloud video according to the embodiments may include one or more frames. A frame represents a still image / picture. Accordingly, the point cloud video may include a point cloud image / frame / picture and may be referred to as any one of a point cloud image, a frame, and a picture.

[0108] A point cloud video encoder (10002) according to the embodiments encodes the obtained point cloud video data. The point cloud video encoder (10002) can encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to the embodiments may include Geometry-based Point Cloud Compression (G-PCC) coding and / or Video-based Point Cloud Compression (V-PCC) coding or next-generation coding. Furthermore, the point cloud compression coding according to the embodiments is not limited to the embodiments described above. The point cloud video encoder (10002) can output a bitstream containing the encoded point cloud video data. The bitstream may include not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0109] A transmitter (10003) according to the embodiments transmits a bitstream containing encoded point cloud video data. The bitstream according to the embodiments is encapsulated into a file or segment (e.g., a streaming segment) and transmitted through various networks such as a broadcast network and / or a broadband network. Although not illustrated in the drawings, the transmission device (10000) may include an encapsulation unit (or encapsulation module) that performs an encapsulation operation. Additionally, according to the embodiments, the encapsulation unit may be included in the transmitter (10003). According to the embodiments, the file or segment may be transmitted to a receiving device (10004) via a network or stored on a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitter (10003) according to the embodiments can communicate wired or wirelessly with the receiving device (10004) (or receiver (10005)) via a network such as 4G, 5G, or 6G. Additionally, the transmitter (10003) can perform necessary data processing operations according to a network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). Additionally, the transmission device (10000) can transmit encapsulated data according to an on-demand method.

[0110] A receiving device (10004) according to embodiments includes a receiver (10005), a point cloud video decoder (10006), and / or a renderer (10007). According to embodiments, the receiving device (10004) may include a device, robot, vehicle, AR / VR / XR device, mobile device, home appliance, IoT (Internet of Thing) device, AI device / server, etc., that communicates with a base station and / or other wireless device using wireless access technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)).

[0111] A receiver (10005) according to the embodiments receives a bitstream containing point cloud video data or a file / segment containing the bitstream from a network or a storage medium. The receiver (10005) can perform necessary data processing operations according to a network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). The receiver (10005) according to the embodiments can decapsulate the received file / segment and output a bitstream. Additionally, according to the embodiments, the receiver (10005) may include a decapsulation unit (or decapsulation module) for performing a decapsulation operation. Additionally, the decapsulation unit may be implemented as an element (or component) separate from the receiver (10005).

[0112] A point cloud video decoder (10006) decodes a bitstream containing point cloud video data. The point cloud video decoder (10006) can decode the point cloud video data according to the way the point cloud video data is encoded (e.g., the reverse process of the operation of a point cloud video encoder (10002)). Accordingly, the point cloud video decoder (10006) can decode the point cloud video data by performing point cloud decompression coding, which is the reverse process of point cloud compression. Point cloud decompression coding includes G-PCC coding.

[0113] The renderer (10007) renders the decoded point cloud video data. The renderer (10007) can render not only the point cloud video data but also audio data to output point cloud content. According to embodiments, the renderer (10007) may include a display for displaying the point cloud content. According to embodiments, the display may not be included in the renderer (10007) but may be implemented as a separate device or component.

[0114] The arrows indicated by dotted lines in the drawing represent the transmission path of feedback information obtained from the receiving device (10004). The feedback information is information intended to reflect interaction with a user consuming point cloud content, and includes user information (e.g., head orientation information), viewport information, etc. In particular, if the point cloud content is content for a service requiring interaction with a user (e.g., autonomous driving service, etc.), the feedback information may be transmitted to the content transmission side (e.g., transmission device (10000)) and / or the service provider. Depending on the embodiments, the feedback information may be used in the receiving device (10004) as well as the transmission device (10000), or it may not be provided.

[0115] Head orientation information according to the embodiments is information regarding the user's head position, direction, angle, movement, etc. The receiving device (10004) according to the embodiments can calculate viewport information based on the head orientation information. Viewport information is information about the area of ​​the point cloud video that the user is looking at. The viewpoint refers to the point where the user is looking at the point cloud video, and may mean the exact center point of the viewport area. That is, the viewport is an area centered on the viewpoint, and the size and shape of the area can be determined by the Field Of View (FOV). Therefore, the receiving device (10004) can extract viewport information based on the vertical or horizontal FOV supported by the device in addition to the head orientation information. In addition, the receiving device (10004) performs gaze analysis, etc., to check the user's point cloud consumption method, the point cloud video area the user is looking at, the gaze time, etc. According to embodiments, the receiving device (10004) may transmit feedback information including gaze analysis results to the transmitting device (10000). According to embodiments, the feedback information may be obtained during the rendering and / or display process. According to embodiments, the feedback information may be obtained by one or more sensors included in the receiving device (10004). Also, according to embodiments, the feedback information may be obtained by the renderer (10007) or a separate external element (or device, component, etc.). The dotted line in FIG. 1 indicates the process of transmitting the feedback information obtained from the renderer (10007). The point cloud content providing system may process (encode / decode) point cloud data based on the feedback information. Accordingly, the point cloud video data decoder (10006) may perform a decoding operation based on the feedback information.Additionally, the receiving device (10004) can transmit feedback information to the transmitting device (10000). The transmitting device (10000) (or the point cloud video data encoder (10002)) can perform an encoding operation based on the feedback information. Thus, the point cloud content providing system can efficiently process necessary data (e.g., point cloud data corresponding to the user's head position) based on the feedback information without processing (encoding / decoding) all point cloud data, and provide point cloud content to the user.

[0116] According to embodiments, the transmission device (10000) may be referred to as an encoder, transmission device, transmitter, etc., and the receiving device (10004) may be referred to as a decoder, receiving device, receiver, etc.

[0117] Point cloud data processed in the point cloud content providing system of FIG. 1 according to embodiments (processed through a series of processes of acquisition / encoding / transmission / decoding / rendering) may be referred to as point cloud content data or point cloud video data. According to embodiments, point cloud content data may be used as a concept including metadata or signaling information related to point cloud data.

[0118] The elements of the point cloud content delivery system illustrated in FIG. 1 can be implemented in hardware, software, processors, and / or combinations thereof.

[0119] FIG. 2 is a block diagram illustrating a point cloud content provision operation according to embodiments.

[0120] The block diagram of FIG. 2 illustrates the operation of the point cloud content provision system described in FIG. 1. As described above, the point cloud content provision system can process point cloud data based on point cloud compression coding (e.g., G-PCC).

[0121] A point cloud content providing system according to the embodiments (e.g., a point cloud transmission device (10000) or a point cloud video acquisition unit (10001)) can acquire a point cloud video (20000). The point cloud video is represented as a point cloud belonging to a coordinate system representing a three-dimensional space. The point cloud video according to the embodiments may include a Ply (Polygon File format or the Stanford Triangle format) file. If the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as the geometry and / or attributes of the points. The geometry includes the positions of the points. The position of each point may be represented by parameters (e.g., values ​​of the X-axis, Y-axis, and Z-axis, respectively) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of XYZ axes). Attributes include attributes of points (e.g., texture information, color (YCbCr or RGB), reflectance (r), transparency, etc. of each point). A point has one or more attributes (or properties). For example, a point may have one attribute which is color, or two attributes which are color and reflectance. According to embodiments, geometry may be referred to as positions, geometry information, geometry data, etc., and attributes may be referred to as attributes, attribute information, attribute data, etc.In addition, a point cloud content provision system (e.g., a point cloud transmission device (10000) or a point cloud video acquisition unit (10001)) can obtain point cloud data from information related to the acquisition process of point cloud video (e.g., depth information, color information, etc.).

[0122] A point cloud content providing system according to embodiments (e.g., a transmission device (10000) or a point cloud video encoder (10002)) can encode point cloud data (20001). The point cloud content providing system can encode point cloud data based on point cloud compression coding. As described above, point cloud data may include geometry and attributes of points. Accordingly, the point cloud content providing system can output a geometry bitstream by performing geometry encoding to encode geometry. The point cloud content providing system can output an attribute bitstream by performing attribute encoding to encode attributes. According to embodiments, the point cloud content providing system can perform attribute encoding based on geometry encoding. The geometry bitstream and attribute bitstream according to embodiments can be multiplexed and output as a single bitstream. The bitstream according to the embodiments may further include signaling information related to geometry encoding and attribute encoding.

[0123] A point cloud content providing system according to embodiments (e.g., a transmission device (10000) or a transmitter (10003)) can transmit encoded point cloud data (20002). As described in FIG. 1, the encoded point cloud data can be represented as a geometry bitstream and an attribute bitstream. Additionally, the encoded point cloud data can be transmitted in the form of a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometry encoding and attribute encoding). Additionally, the point cloud content providing system can encapsulate the bitstream transmitting the encoded point cloud data and transmit it in the form of a file or segment.

[0124] A point cloud content providing system according to embodiments (e.g., a receiving device (10004) or a receiver (10005)) can receive a bitstream containing encoded point cloud data. Additionally, the point cloud content providing system (e.g., a receiving device (10004) or a receiver (10005)) can demultiplex the bitstream.

[0125] A point cloud content providing system (e.g., a receiving device (10004) or a point cloud video decoder (10005)) can decode encoded point cloud data (e.g., a geometry bitstream, an attribute bitstream) transmitted as a bitstream. A point cloud content providing system (e.g., a receiving device (10004) or a point cloud video decoder (10005)) can decode point cloud video data based on signaling information related to the encoding of point cloud video data included in the bitstream. A point cloud content providing system (e.g., a receiving device (10004) or a point cloud video decoder (10005)) can decode the geometry bitstream to restore the positions (geometry) of the points. A point cloud content providing system can decode the attribute bitstream based on the restored geometry to restore the attributes of the points. A point cloud content delivery system (e.g., a receiving device (10004) or a point cloud video decoder (10005)) can restore a point cloud video based on positions according to the restored geometry and decoded attributes.

[0126] A point cloud content providing system according to embodiments (e.g., a receiving device (10004) or a renderer (10007)) can render decoded point cloud data (20004). The point cloud content providing system (e.g., a receiving device (10004) or a renderer (10007)) can render geometry and attributes decoded through a decoding process according to various rendering methods. Points of the point cloud content may be rendered as vertices having a certain thickness, cubes having a specific minimum size with the vertex location as the center, or circles with the vertex location as the center, etc. All or part of the rendered point cloud content is provided to a user through a display (e.g., a VR / AR display, a general display, etc.).

[0127] A point cloud content providing system (e.g., a receiving device (10004)) according to the embodiments can obtain feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. Since the feedback information and the operation of the point cloud content providing system according to the embodiments are the same as the feedback information and operation described in FIG. 1, a detailed description is omitted.

[0128] FIG. 3 shows an example of a point cloud encoder according to embodiments.

[0129] FIG. 3 shows an example of the point cloud video encoder (10002) of FIG. 1. The point cloud encoder reconstructs point cloud data (e.g., positions and / or attributes of points) and performs encoding operations to adjust the quality of point cloud content (e.g., lossless, lossy, near-lossless) according to network conditions or applications. If the total size of the point cloud content is large (e.g., point cloud content of 60 Gbps in the case of 30 fps), the point cloud content delivery system may not be able to stream the content in real time. Therefore, the point cloud content delivery system may reconstruct the point cloud content based on a maximum target bitrate to provide it according to the network environment.

[0130] As described in FIGS. 1 and 2, the point cloud encoder can perform geometry encoding and attribute encoding. Geometry encoding is performed before attribute encoding.

[0131] The point cloud encoder according to the embodiments comprises a coordinate system transformation unit (Transformation Coordinates, 30000), a quantization unit (Quantize and Remove Points (Voxelize), 30001), an octree analysis unit (Analyze Octree, 30002), a surface approximation analysis unit (Analyze Surface Approximation, 30003), an arithmetic encoder (Arithmetic Encode, 30004), a geometry reconstruction unit (Reconstruct Geometry, 30005), a color transformation unit (Transform Colors, 30006), an attribute transformation unit (Transfer Attributes, 30007), a RAHT transformation unit (30008), an LOD generation unit (Generated LOD, 30009), a lifting transformation unit (Lifting) (30010), and a coefficient quantization unit (Quantize Coefficients, 30011). Includes an and / or arithmetic encoder (Arithmetic Encode, 30012).

[0132] The coordinate system transformation unit (30000), quantization unit (30001), octree analysis unit (30002), surface approximation analysis unit (30003), arismetic encoder (30004), and geometry reconstruction unit (30005) can perform geometry encoding. Geometry encoding according to the embodiments may include octree geometry coding, direct coding, trisoup geometry encoding, and entropy encoding. Direct coding and trisoup geometry encoding are applied optionally or in combination. Additionally, geometry encoding is not limited to the above examples.

[0133] As illustrated in the drawings, the coordinate system conversion unit (30000) according to the embodiments receives positions and converts them into a coordinate system. For example, the positions can be converted into position information in a three-dimensional space (e.g., a three-dimensional space expressed in an XYZ coordinate system). The position information in the three-dimensional space according to the embodiments may be referred to as geometry information.

[0134] The quantization unit (30001) according to the embodiments quantizes the geometry. For example, the quantization unit (30001) can quantize points based on the minimum position values ​​of all points (e.g., minimum values ​​on each axis for the X-axis, Y-axis, and Z-axis). The quantization unit (30001) performs a quantization operation to find the nearest integer value by multiplying the difference between the minimum position value and the position value of each point by a preset quantization scale value and then performing rounding down or rounding up. Thus, one or more points may have the same quantized position (or position value). The quantization unit (30001) according to the embodiments performs voxelization based on the quantized positions to reconstruct the quantized points. Just as the minimum unit containing 2D image / video information is a pixel, the points of the point cloud content (or 3D point cloud video) according to the embodiments may be contained in one or more voxels. A voxel is a combination of volume and pixel, and refers to a three-dimensional cubic space that is generated when a three-dimensional space is divided into units (unit=1.0) based on axes representing the three-dimensional space (e.g., X-axis, Y-axis, Z-axis). The quantization unit (40001) can match groups of points in the three-dimensional space to voxels. According to embodiments, a single voxel may contain only one point. According to embodiments, a single voxel may contain one or more points. In addition, to represent a single voxel as a single point, the position of the center of the voxel can be set based on the positions of one or more points included in the voxel. In this case, the attributes of all positions included in the voxel can be combined and assigned to the voxel.

[0135] The octree analysis unit (30002) according to the embodiments performs octree geometry coding (or octree coding) to represent the voxels in an octree structure. The octree structure represents points matched to the voxels based on an octree structure.

[0136] The surface approximation analysis unit (30003) according to the embodiments can analyze and approximate an octree. The octree analysis and approximation according to the embodiments is a process of analyzing to voxelize an area containing multiple points in order to efficiently provide octree and voxelization.

[0137] An arithmetic encoder (30004) according to the embodiments entropy-encodes an octree and / or an approximated octree. For example, the encoding method includes an arithmetic encoding method. As a result of the encoding, a geometry bitstream is generated.

[0138] The color conversion unit (30006), attribute conversion unit (30007), RAHT conversion unit (30008), LOD generation unit (30009), lifting conversion unit (30010), coefficient quantization unit (30011) and / or arismetic encoder (30012) perform attribute encoding. As described above, a point may have one or more attributes. The attribute encoding according to the embodiments is applied equally to the attributes of a point. However, if a single attribute (e.g., color) includes one or more elements, independent attribute encoding is applied to each element. The attribute encoding according to the embodiments may include color conversion coding, attribute conversion coding, Region Adaptive Hierarchical Transform (RAHT) coding, prediction transformation (Interpolaration-based hierarchical nearest-neighbour prediction-Prediction Transform) coding, and lifting transformation (interpolation-based hierarchical nearest-neighbour prediction with an update / lifting step (Lifting Transform)) coding. Depending on the point cloud content, the above-described RAHT coding, prediction transformation coding, and lifting transformation coding may be used optionally, or a combination of one or more of the codings may be used. Furthermore, the attribute encoding according to the embodiments is not limited to the examples described above.

[0139] The color conversion unit (30006) according to the embodiments performs color conversion coding that converts color values ​​(or textures) included in attributes. For example, the color conversion unit (30006) can convert the format of color information (e.g., convert from RGB to YCbCr). The operation of the color conversion unit (30006) according to the embodiments may be applied optionally depending on the color values ​​included in attributes.

[0140] The geometry reconstruction unit (30005) according to the embodiments reconstructs (decompresses) an octree and / or an approximated octree. The geometry reconstruction unit (30005) reconstructs an octree / voxel based on the results of analyzing the distribution of points. The reconstructed octree / voxel may be referred to as the reconstructed geometry (or restored geometry).

[0141] The attribute transformation unit (30007) according to the embodiments performs attribute transformation that transforms attributes based on positions where geometry encoding has not been performed and / or reconstructed geometry. As described above, since attributes are dependent on geometry, the attribute transformation unit (30007) can transform attributes based on reconstructed geometry information. For example, the attribute transformation unit (30007) can transform the attributes of a point at a position based on the position value of a point included in a voxel. As described above, when the position of the center point of a voxel is set based on the positions of one or more points included in a voxel, the attribute transformation unit (30007) transforms the attributes of one or more points. When trisoop geometry encoding is performed, the attribute conversion unit (30007) can convert attributes based on the trisoop geometry encoding.

[0142] The attribute transformation unit (30007) can perform attribute transformation by calculating the average value of attributes or attribute values ​​(e.g., the color or reflectance of each point) of neighboring points within a specific location / radius from the position (or position value) of the center point of each voxel. The attribute transformation unit (30007) can apply a weight based on the distance from the center point to each point when calculating the average value. Thus, each voxel has a position and a calculated attribute (or attribute value).

[0143] The attribute conversion unit (30007) can search for neighboring points within a specific location / radius from the position of the center point of each voxel based on a KD tree or a Molton code. A KD tree is a binary search tree that supports a data structure capable of managing points based on their positions to enable rapid Nearest Neighbor Search (NNS). A Molton code is generated by representing the coordinate values ​​(e.g., (x, y, z)) representing the 3D positions of all points as bit values ​​and mixing the bits. For example, if the coordinate values ​​representing the position of a point are (5, 9, 1), the bit values ​​of the coordinate values ​​are (0101, 1001, 0001). When the bit values ​​are mixed according to the bit indices in the order of z, y, and x, it becomes 010001000111. When this value is represented in decimal, it becomes 1095. That is, the Molton code value of the point with coordinates (5, 9, 1) is 1095. The attribute transformation unit (30007) sorts the points based on the Molton code value and can perform shortest neighbor search (NNS) through a depth-first traversal process. After the attribute transformation operation, if shortest neighbor search (NNS) is required in other transformation processes for attribute coding, a KD tree or Molton code is utilized.

[0144] As shown in the drawing, the converted attributes are input to the RAHT conversion unit (30008) and / or LOD generation unit (30009).

[0145] The RAHT transformation unit (30008) according to the embodiments performs RAHT coding to predict attribute information based on reconstructed geometry information. For example, the RAHT transformation unit (30008) can predict attribute information of a node at an upper level of the octree based on attribute information associated with a node at a lower level of the octree.

[0146] The LOD generation unit (30009) according to the embodiments generates a Level of Detail (LOD) to perform predictive transformation coding. The LOD according to the embodiments represents the degree of detail of the point cloud content, and indicates that the smaller the LOD value, the lower the detail of the point cloud content, and the larger the LOD value, the higher the detail of the point cloud content. Points can be classified according to the LOD.

[0147] The lifting transformation unit (30010) according to the embodiments performs lifting transformation coding that transforms the attributes of the point cloud based on weights. As described above, the lifting transformation coding may be applied optionally.

[0148] The coefficient quantization unit (30011) according to the embodiments quantizes attribute-coded attributes based on coefficients.

[0149] An arismetic encoder (30012) according to the embodiments encodes quantized attributes based on arismetic coding.

[0150] The elements of the point cloud encoder of FIG. 3 may be implemented in hardware, software, firmware, or a combination thereof, comprising one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device, although not illustrated in the drawing. One or more processors may perform at least one of the operations and / or functions of the elements of the point cloud encoder of FIG. 3 described above. Additionally, one or more processors may operate or execute a set of software programs and / or instructions for performing the operations and / or functions of the elements of the point cloud encoder of FIG. 3. One or more memories according to the embodiments may include high-speed random access memory and may include non-volatile memory (e.g., one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0151] FIG. 4 shows examples of octree and occupancy codes according to embodiments.

[0152] As described in FIGS. 1 to 3, a point cloud content providing system (point cloud video encoder (10002)) or a point cloud encoder (e.g., an octree analysis unit (30002)) performs octree geometry coding based on an octree structure (or octree coding) to efficiently manage the area and / or position of a voxel.

[0153] The top of FIG. 4 shows an octree structure. The three-dimensional space of the point cloud content according to the embodiments is represented by the axes of the coordinate system (e.g., X-axis, Y-axis, Z-axis). The octree structure has two poles (0,0,0) and (2 d , 2 d , 2 d It is generated by recursively subdividing the bounding box (cubical axis-aligned bounding box) defined by ). 2d can be set to the value that constitutes the smallest bounding box enclosing all points of the point cloud content (or point cloud video). d represents the depth of the octree. The value of d is determined according to the following equation. In the equation below, (x int n , y int n , z int n ) represents the positions (or position values) of quantized points.

[0154]

[0155] As illustrated in the middle of the top of Fig. 4, the entire three-dimensional space can be divided into eight spaces according to the division. Each divided space is represented as a cube having six faces. As illustrated in the right of the top of Fig. 4, each of the eight spaces is further divided based on the axes of the coordinate system (e.g., X-axis, Y-axis, Z-axis). Thus, each space is again divided into eight smaller spaces. The divided smaller spaces are also represented as cubes having six faces. This division method is applied until the leaf nodes of the octree become voxels.

[0156] The bottom of Fig. 4 shows the occupancy code of an octree. The occupancy code of an octree is generated to indicate whether each of the eight partitioned spaces resulting from the partitioning of a single space contains at least one point. Therefore, one occupancy code is represented by eight child nodes. Each child node represents the occupancy of the partitioned space, and the child node has a value of 1 bit. Thus, the occupancy code is represented as an 8-bit code. That is, if the space corresponding to the child node contains at least one point, the node has a value of 1. If the space corresponding to the child node does not contain a point (empty), the node has a value of 0. Since the occupancy code shown in Fig. 4 is 00100001, it indicates that the spaces corresponding to the 3rd and 8th child nodes among the eight child nodes each contain at least one point. As illustrated in the drawing, the 3rd child node and the 8th child node each have 8 child nodes, and each child node is represented by an 8-bit Occupancy code. The drawing indicates that the Occupancy code of the 3rd child node is 10000111 and the Occupancy code of the 8th child node is 01001111. A point cloud encoder according to the embodiments (e.g., an arismetic encoder (30004)) can entropy-encode the Occupancy code. Additionally, to increase compression efficiency, the point cloud encoder can intra- / inter-encode the Occupancy code. A receiving device according to the embodiments (e.g., a receiving device (10004) or a point cloud video decoder (10006)) reconstructs the octree based on the Occupancy code.

[0157] A point cloud encoder according to the embodiments (e.g., the point cloud encoder of FIG. 3, or the octree analysis unit (30002)) can perform voxelization and octree coding to store the positions of the points. However, since points in a three-dimensional space are not always evenly distributed, there may be specific areas where few points exist. Therefore, performing voxelization on the entire three-dimensional space is inefficient. For example, if there are almost no points in a specific area, there is no need to perform voxelization up to that area.

[0158] Accordingly, the point cloud encoder according to the embodiments can perform direct coding, which directly codes the positions of points included in a specific region (or nodes excluding leaf nodes of an octree) without performing voxelization on the aforementioned specific region. The coordinates of the points directly coded according to the embodiments are referred to as the Direct Coding Mode (DCM). Additionally, the point cloud encoder according to the embodiments can perform trisoup geometry encoding, which reconstructs the positions of points within a specific region (or node) based on voxels using a surface model. Trisoup geometry encoding is a geometry encoding that represents an object as a series of triangle meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct coding and trisoup geometry encoding according to the embodiments may be performed optionally. In addition, direct coding and trisoop geometry encoding according to the embodiments can be performed in combination with octree geometry coding (or octree coding).

[0159] To perform direct coding, the option to use direct mode for applying direct coding must be enabled, the node to which direct coding is to be applied must not be a leaf node, and there must be points within a specific node that are below a threshold. In addition, the total number of points subject to direct coding must not exceed a preset threshold. If the above conditions are satisfied, the point cloud encoder (or arismetic encoder (30004)) according to the embodiments can entropy-code the positions (or position values) of the points.

[0160] A point cloud encoder according to the embodiments (e.g., a surface approximation analysis unit (30003)) can determine a specific level of an octree (where the level is smaller than the depth d of the octree) and, starting from that level, perform trisoop geometry encoding to reconstruct the position of points within a node region based on voxels using a surface model (trisoop mode). The point cloud encoder according to the embodiments can specify the level to which trisoop geometry encoding is applied. For example, if the specified level is equal to the depth of the octree, the point cloud encoder does not operate in trisoop mode. That is, the point cloud encoder according to the embodiments can operate in trisoop mode only when the specified level is smaller than the depth value of the octree. A three-dimensional cubic region of nodes at a specified level according to the embodiments is referred to as a block. A block may contain one or more voxels. A block or a voxel may correspond to a brick. Within each block, geometry is represented as a surface. A surface according to the embodiments may intersect each edge of the block at most once.

[0161] Since one block has 12 edges, there are at least 12 intersection points within one block. Each intersection point is referred to as a vertex. A vertex along an edge is detected if there is at least one occupied voxel adjacent to that edge among all blocks sharing that edge. An occupied voxel according to the embodiments means a voxel containing a point. The position of a vertex detected along an edge is the average position along the edge of all voxels adjacent to that edge among all blocks sharing that edge.

[0162] When a vertex is detected, the point cloud encoder according to the embodiments can entropy-code the edge start point (x, y, z), edge direction vector (Δx, Δy, Δz), and vertex position value (relative position value within the edge). When trisoop geometry encoding is applied, the point cloud encoder according to the embodiments (e.g., geometry reconstruction unit (30005)) can generate restored geometry (reconstructed geometry) by performing triangle reconstruction, up-sampling, and voxelization processes.

[0163] The vertices located on the edges of the block determine the surface passing through the block. The surface according to the embodiments is a non-planar polygon. The triangle reconstruction process reconstructs the surface represented by triangles based on the edge start point, the edge direction vector, and the vertex position value. The triangle reconstruction process is as follows: ① calculate the centroid value of each vertex, ② subtract the centroid value from each vertex value, ③ square the result, and add all the result together.

[0164]

[0165] The minimum sum is calculated, and a projection process is performed along the axis where the minimum value is located. For example, if the x-element is at its minimum, each vertex is projected along the x-axis relative to the center of the block and onto the (y, z) plane. If the value obtained from projecting onto the (y, z) plane is (ai, bi), the θ value is calculated using atan2(bi, ai), and the vertices are aligned based on the θ value. The table below shows the combinations of vertices to generate triangles depending on the number of vertices. The vertices are aligned in order from 1 to n. The table below indicates that for four vertices, two triangles can be formed depending on the combination of vertices. The first triangle can be formed from the 1st, 2nd, and 3rd vertices among the aligned vertices, and the second triangle can be formed from the 3rd, 4th, and 1st vertices among the aligned vertices.

[0166] Table 2-1. Triangles formed from vertices ordered 1,… ,n

[0167] n triangles

[0168] 3 (1,2,3)

[0169] 4 (1,2,3), (3,4,1)

[0170] 5 (1,2,3), (3,4,5), (5,1,3)

[0171] 6 (1,2,3), (3,4,5), (5,6,1), (1,3,5)

[0172] 7 (1,2,3), (3,4,5), (5,6,7), (7,1,3), (3,5,7)

[0173] 8 (1,2,3), (3,4,5), (5,6,7), (7,8,1), (1,3,5), (5,7,1)

[0174] 9 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,1,3), (3,5,7), (7,9,3)

[0175] 10 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,1), (1,3,5), (5,7,9), (9,1,5)

[0176] 11 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,11), (11,1,3), (3,5,7), (7,9,11), (11,3,7)

[0177] 12 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,11), (11,12,1), (1,3,5), (5,7,9), (9,11,1), (1,5,9)

[0178] The upsampling process is performed to voxelize by adding intermediate points along the edges of the triangle. Additional points are generated based on the upsampling factor value and the width of the block. The additional points are referred to as refined vertices. A point cloud encoder according to the embodiments can voxelize the refined vertices. Additionally, the point cloud encoder can perform attribute encoding based on the voxelized positions (or position values).

[0179] Figure 5 shows an example of a point configuration by LOD according to embodiments.

[0180] As described in FIGS. 1 to 4, the encoded geometry is reconstructed (decompressed) before attribute encoding is performed. When direct coding is applied, the geometry reconstruction operation may include changing the arrangement of the direct-coded points (e.g., placing the direct-coded points at the front of the point cloud data). When trisoop geometry encoding is applied, the geometry reconstruction process involves triangle reconstruction, upsampling, and voxelization. Since attributes depend on geometry, attribute encoding is performed based on the reconstructed geometry.

[0181] A point cloud encoder (e.g., an LOD generation unit (30009)) can reorganize points by LOD. The drawing shows point cloud content corresponding to the LOD. The left side of the drawing shows the original point cloud content. The second figure from the left of the drawing shows the distribution of points of the lowest LOD, and the rightmost figure of the drawing shows the distribution of points of the highest LOD. That is, the points of the lowest LOD are sparsely distributed, while the points of the highest LOD are densely distributed. In other words, according to the direction of the arrow indicated at the bottom of the drawing, as the LOD increases, the spacing (or distance) between points becomes shorter.

[0182] Figure 6 shows an example of a point configuration by LOD according to embodiments.

[0183] As described in FIGS. 1 to 5, a point cloud content providing system or a point cloud encoder (e.g., a point cloud video encoder (10002), the point cloud encoder of FIG. 3, or an LOD generation unit (30009)) can generate an LOD. The LOD is generated by reorganizing points into a set of refinement levels according to a set LOD distance value (or a set of Euclidean distances). The LOD generation process is performed in a point cloud decoder as well as a point cloud encoder.

[0184] The top of Fig. 6 shows examples of points (P0 to P9) of point cloud content distributed in three-dimensional space. The Original Order in Fig. 6 represents the order of points P0 to P9 prior to LOD generation. The LOD-based Order in Fig. 6 represents the order of points following LOD generation. Points are rearranged by LOD. Additionally, higher LODs include points belonging to lower LODs. As illustrated in Fig. 6, LOD0 includes P0, P5, P4, and P2. LOD1 includes the points of LOD0 and P1, P6, and P3. LOD2 includes the points of LOD0, the points of LOD1, and P9, P8, and P7.

[0185] As described in FIG. 3, the point cloud encoder according to the embodiments can perform predictive transform coding, lifting transform coding, and RAHT transform coding selectively or in combination.

[0186] The point cloud encoder according to the embodiments can generate predictors for points and perform predictive transformation coding to set the predicted attribute (or predicted attribute value) of each point. That is, N predictors can be generated for N points. The predictor according to the embodiments can calculate a weight (=1 / distance) value based on the LOD value of each point, indexing information for neighboring points within a set distance per LOD, and the distance value to the neighboring points.

[0187] According to the embodiments, the predicted attribute (or attribute value) is set as the average value of the values ​​obtained by multiplying the attributes (or attribute values, e.g., color, reflectance, etc.) of neighboring points set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder according to the embodiments (e.g., coefficient quantization unit (30011)) can quantize and inverse quantize the residual values ​​(which may be referred to as residual attributes, residual attribute values, attribute prediction residual values, etc.) obtained by subtracting the predicted attribute (attribute value) from the attribute (attribute value) of each point. The quantization process is as shown in the following table.

[0188] graph. Attribute prediction residuals quantization pseudo code

[0189] int PCCQuantization(int value, int quantStep) {

[0190] if( value >=0) {

[0191] return floor(value / quantStep + 1.0 / 3.0);

[0192] } else {

[0193] return -floor(-value / quantStep + 1.0 / 3.0);

[0194] }

[0195] }

[0196] graph. Attribute prediction residuals inverse quantization pseudo code

[0197] int PCCInverseQuantization(int value, int quantStep) {

[0198] if( quantStep ==0) {

[0199] return value;

[0200] } else {

[0201] return value * quantStep;

[0202] }

[0203] }

[0204] A point cloud encoder according to the embodiments (e.g., an arismetic encoder (30012)) can entropy-code the quantized and inversely quantized residual values ​​as described above when there are neighboring points in the predictor of each point. A point cloud encoder according to the embodiments (e.g., an arismetic encoder (30012)) can entropy-code the attributes of the corresponding point without performing the process described above when there are no neighboring points in the predictor of each point.

[0205] A point cloud encoder according to the embodiments (e.g., a lifting transformation unit (30010)) can perform lifting transformation coding by generating a predictor for each point, setting the LOD calculated in the predictor, registering neighboring points, and setting weights based on the distance to neighboring points. The lifting transformation coding according to the embodiments is similar to the prediction transformation coding described above, but differs in that weights are cumulatively applied to attribute values. The process of cumulatively applying weights to attribute values ​​according to the embodiments is as follows.

[0206] 1) Create an array QW (QuantizationWight) to store the weight values ​​of each point. The initial value of all elements in QW is 1.0. Add the value obtained by multiplying the current point's predictor weight by the QW value of the predictor index of the neighboring node registered in the predictor.

[0207] 2) Lift prediction process: To calculate the predicted attribute value, the value obtained by multiplying the point's attribute value by a weight is subtracted from the existing attribute value.

[0208] 3) Create temporary arrays named updateweight and update, and initialize the temporary arrays to 0.

[0209] 4) For all predictors, the calculated weight is additionally multiplied by the weight stored in the QW corresponding to the predictor index, and the resulting weight is accumulated in the update weight array with the neighbor node index. In the update array, the value obtained by multiplying the attribute value of the neighbor node index by the calculated weight is accumulated.

[0210] 5) Lift update process: For all predictors, the attribute value of the update array is divided by the weight value of the update weight array at the predictor index, and the original attribute value is added back to the divided value.

[0211] 6) For all predictors, the predicted attribute value is calculated by additionally multiplying the attribute value updated through the lift update process by the weight (stored in QW) updated through the lift prediction process. A point cloud encoder according to the embodiments (e.g., coefficient quantizer (30011)) quantizes the predicted attribute value. Additionally, a point cloud encoder (e.g., arismetic encoder (30012)) entropies the quantized attribute value.

[0212] A point cloud encoder according to the embodiments (e.g., a RAHT transform unit (30008)) can perform RAHT transform coding to predict attributes of upper-level nodes using attributes associated with nodes at lower levels of the octree. RAHT transform coding is an example of attribute intra-coding through octree backward scanning. A point cloud encoder according to the embodiments scans from voxels to the entire region and repeats the merging process up to the root node, merging voxels into larger blocks at each step. The merging process according to the embodiments is performed only on occupied nodes. The merging process is not performed on empty nodes, and the merging process is performed on the node immediately above the empty node.

[0213] The following equation represents the RAHT transformation matrix. g l x, y, z represents the average attribute value of the voxels at level l. g l x, y, z can be calculated from gl+1 2x, y, z and gl+1 2x+1, y, z. The weights of gl 2x, y, z and gl 2x+1, y, z are w1=wl 2x, y, z and w2=wl 2x+1, y, z.

[0214]

[0215] gl-1 x, y, z are low-pass values ​​used in the merging process at the next higher level. hl-1 x, y, z are high-pass coefficients, and the high-pass coefficients at each step are quantized and entropy-coded (e.g., encoding of an arismetic encoder (300012)). The weights are calculated as wl-1 x, y, z = wl 2x, y, z + wl 2x + 1, y, z. The root node is the last g 1 0, 0, 0 and g 1 0, 0, 1 It is generated as follows through.

[0216]

[0217] The gDC value is also quantized and entropy-coded, just like the high-pass coefficient.

[0218] FIG. 7 shows an example of a point cloud decoder according to embodiments.

[0219] The point cloud decoder illustrated in FIG. 7 is an example of a point cloud decoder and can perform a decoding operation, which is the reverse process of the encoding operation of the point cloud encoder described in FIG. 1 to 6.

[0220] As described in Fig. 1, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0221] A point cloud decoder according to the embodiments comprises an arithmetic decoder (7000), a synthesize octree (7001), a synthesize surface approximation (7002), a reconstruct geometry (7003), an inverse transform coordinates (7004), an arithmetic decoder (7005), an inverse quantize (7006), a RAHT transform (7007), an LOD generater (7008), an inverse lifting (7009), and / or an inverse transform colors (7010).

[0222] An arismetic decoder (7000), an octree composite unit (7001), a surface offset composite unit (7002), a geometry reconstruction unit (7003), and a coordinate system inverse transformation unit (7004) can perform geometry decoding. Geometry decoding according to the embodiments may include direct coding and trisoup geometry decoding. Direct coding and trisoup geometry decoding are applied optionally. Additionally, geometry decoding is not limited to the above examples and is performed as the reverse process of geometry encoding described in FIGS. 1 through 6.

[0223] The arismetic decoder (7000) according to the embodiments decodes the received geometry bitstream based on arismetic coding. The operation of the arismetic decoder (7000) corresponds to the reverse process of the arismetic encoder (30004).

[0224] The octree synthesis unit (7001) according to the embodiments can generate an octree by obtaining an Occupancy code from a decoded geometry bitstream (or information regarding the geometry obtained as a result of decoding). A specific description of the Occupancy code is as described in FIGS. 1 to 6.

[0225] The surface off-relation synthesis unit (7002) according to the embodiments can synthesize a surface based on the decoded geometry and / or the generated octree when trisoop geometry encoding is applied.

[0226] The geometry reconstruction unit (7003) according to the embodiments can regenerate geometry based on a surface and / or decoded geometry. As described in FIGS. 1 through 6, direct coding and trisoop geometry encoding are applied optionally. Accordingly, the geometry reconstruction unit (7003) directly retrieves and adds position information of points to which direct coding has been applied. In addition, when trisoop geometry encoding is applied, the geometry reconstruction unit (7003) can restore geometry by performing reconstruction operations of the geometry reconstruction unit (30005), such as triangle reconstruction, up-sampling, and voxelization operations. Specific details are omitted as they are the same as those described in FIG. 4. The restored geometry may include a point cloud picture or frame that does not contain attributes.

[0227] The coordinate system inverse transformation unit (7004) according to the embodiments can obtain the positions of the points by transforming the coordinate system based on the restored geometry.

[0228] The arismetic decoder (7005), inverse quantization unit (7006), RAHT transformation unit (7007), LOD generation unit (7008), inverse lifting unit (7009), and / or color inverse transformation unit (7010) can perform attribute decoding as described in FIG. 10. Attribute decoding according to the embodiments may include Region Adaptive Hierarchial Transform (RAHT) decoding, Interpolaration-based hierarchical nearest-neighbour prediction-Prediction Transform) decoding, and interpolation-based hierarchical nearest-neighbour prediction with an update / lifting step (Lifting Transform) decoding. The three decodings described above may be used optionally, or a combination of one or more decodings may be used. Furthermore, attribute decoding according to the embodiments is not limited to the examples described above.

[0229] The arismetic decoder (7005) according to the embodiments decodes the attribute bitstream into arismetic coding.

[0230] The inverse quantization unit (7006) according to the embodiments inverse quantizes information about the decoded attribute bitstream or the attribute obtained as a result of decoding and outputs the inverse quantized attributes (or attribute values). Inverse quantization may be optionally applied based on the attribute encoding of the point cloud encoder.

[0231] According to embodiments, the RAHT transformation unit (7007), LOD generation unit (7008), and / or inverse lifting unit (7009) can process the reconstructed geometry and inverse quantized attributes. As described above, the RAHT transformation unit (7007), LOD generation unit (7008), and / or inverse lifting unit (7009) can optionally perform a corresponding decoding operation according to the encoding of the point cloud encoder.

[0232] The color inverse conversion unit (7010) according to the embodiments performs inverse conversion coding to inversely convert the color value (or texture) included in the decoded attributes. The operation of the color inverse conversion unit (7010) may be selectively performed based on the operation of the color conversion unit (30006) of the point cloud encoder.

[0233] The elements of the point cloud decoder of FIG. 7 may be implemented in hardware, software, firmware, or a combination thereof, comprising one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device, although not illustrated in the drawing. One or more processors may perform at least one of the operations and / or functions of the elements of the point cloud decoder of FIG. 7 described above. Additionally, one or more processors may operate or execute a set of software programs and / or instructions for performing the operations and / or functions of the elements of the point cloud decoder of FIG. 7.

[0234] FIG. 8 is an example of a transmission device according to embodiments.

[0235] The transmission device illustrated in FIG. 8 is an example of the transmission device (10000) of FIG. 1 (or the point cloud encoder of FIG. 3). The transmission device illustrated in FIG. 8 can perform at least one of the same or similar operations and methods as the operations and encoding methods of the point cloud encoder described in FIG. 1 to 6. A transmission device according to embodiments may include a data input unit (8000), a quantization processing unit (8001), a voxelization processing unit (8002), an octree occupancy code generation unit (8003), a surface model processing unit (8004), an intra / inter coding processing unit (8005), an arithmetic coder (8006), a metadata processing unit (8007), a color conversion processing unit (8008), an attribute conversion processing unit (or attribute conversion processing unit) (8009), a prediction / lifting / RAHT conversion processing unit (8010), an arithmetic coder (8011) and / or a transmission processing unit (8012).

[0236] The data input unit (8000) according to the embodiments receives or acquires point cloud data. The data input unit (8000) may perform an operation and / or acquisition method identical or similar to the operation and / or acquisition method of the point cloud video acquisition unit (10001) (or the acquisition process (20000) described in FIG. 2).

[0237] The data input unit (8000), quantization processing unit (8001), voxelization processing unit (8002), octree occupancy code generation unit (8003), surface model processing unit (8004), intra / inter coding processing unit (8005), and arithmetic coder (8006) perform geometry encoding. Since the geometry encoding according to the embodiments is identical or similar to the geometry encoding described in FIGS. 1 to 6, a detailed description is omitted.

[0238] The quantization processing unit (8001) according to the embodiments quantizes geometry (e.g., location values ​​of points, or position values). The operation and / or quantization of the quantization processing unit (8001) is the same or similar to the operation and / or quantization of the quantization unit (30001) described in FIG. 3. The specific description is the same as that described in FIG. 1 through 6.

[0239] The voxelization processing unit (8002) according to the embodiments voxelizes the position values ​​of the quantized points. The voxelization processing unit (80002) may perform the same or similar operation and / or process as the operation and / or voxelization process of the quantization unit (30001) described in FIG. 3. The specific description is the same as that described in FIG. 1 to 6.

[0240] The octree occupancy code generation unit (8003) according to the embodiments performs octree coding on the positions of voxelized points based on an octree structure. The octree occupancy code generation unit (8003) can generate an occupancy code. The octree occupancy code generation unit (8003) can perform operations and / or methods identical or similar to the operations and / or methods of the point cloud encoder (or octree analysis unit (30002)) described in FIGS. 3 and 4. The specific description is the same as that described in FIGS. 1 through 6.

[0241] The surface model processing unit (8004) according to the embodiments can perform trisup geometry encoding that reconstructs the positions of points within a specific region (or node) based on a voxel based on a surface model. The surface model processing unit (8004) can perform operations and / or methods identical or similar to the operations and / or methods of the point cloud encoder (e.g., surface approximation analysis unit (30003)) described in FIG. 3. The specific description is the same as that described in FIG. 1 through 6.

[0242] According to the embodiments, the intra / inter coding processing unit (8005) can intra / inter code point cloud data. The intra / inter coding processing unit (8005) can perform coding identical or similar to the intra / inter coding described in FIG. 7. The specific description is the same as that described in FIG. 7. According to the embodiments, the intra / inter coding processing unit (8005) may be included in an arismetic coder (8006).

[0243] An arismetic coder (8006) according to the embodiments entropy-encodes an octree and / or approximated octree of point cloud data. For example, the encoding method includes an arismetic encoding method. The arismetic coder (8006) performs the same or similar operation and / or method as the arismetic encoder (30004).

[0244] A metadata processing unit (8007) according to the embodiments processes metadata regarding point cloud data, such as setting values, and provides it to necessary processing processes such as geometry encoding and / or attribute encoding. Additionally, a metadata processing unit (8007) according to the embodiments may generate and / or process signaling information related to geometry encoding and / or attribute encoding. The signaling information according to the embodiments may be encoded separately from geometry encoding and / or attribute encoding. Additionally, the signaling information according to the embodiments may be interleaved.

[0245] The color conversion processing unit (8008), attribute conversion processing unit (8009), prediction / lifting / RAHT conversion processing unit (8010), and arithmetic coder (8011) perform attribute encoding. Since the attribute encoding according to the embodiments is identical or similar to the attribute encoding described in FIGS. 1 to 6, a detailed description is omitted.

[0246] The color conversion processing unit (8008) according to the embodiments performs color conversion coding that converts color values ​​included in attributes. The color conversion processing unit (8008) may perform color conversion coding based on reconstructed geometry. The description of the reconstructed geometry is the same as that described in FIGS. 1 through 6. In addition, it performs the same or similar operation and / or method as the operation and / or method of the color conversion unit (30006) described in FIG. 3. A detailed description is omitted.

[0247] The attribute transformation processing unit (8009) according to the embodiments performs attribute transformation that transforms attributes based on positions where geometry encoding has not been performed and / or reconstructed geometry. The attribute transformation processing unit (8009) performs operations and / or methods identical or similar to the operations and / or methods of the attribute transformation unit (30007) described in FIG. 3. A detailed description is omitted. The prediction / lifting / RAHT transformation processing unit (8010) according to the embodiments may code the transformed attributes by RAHT coding, prediction transformation coding, and lifting transformation coding, or a combination thereof. The prediction / lifting / RAHT transformation processing unit (8010) performs at least one of operations identical or similar to the operations of the RAHT transformation unit (30008), LOD generation unit (30009), and lifting transformation unit (30010) described in FIG. 3. In addition, the descriptions of predictive transformation coding, lifting transformation coding, and RAHT transformation coding are the same as those described in Figures 1 to 6, so a detailed description is omitted.

[0248] The arismetic coder (8011) according to the embodiments can encode coded attributes based on arismetic coding. The arismetic coder (8011) performs the same or similar operation and / or method as the operation and / or method of the arismetic encoder (300012).

[0249] A transmission processing unit (8012) according to embodiments may transmit each bitstream containing encoded geometry and / or encoded attributes and metadata information, or may transmit the encoded geometry and / or encoded attributes and metadata information by configuring them into a single bitstream. When the encoded geometry and / or encoded attributes and metadata information according to embodiments is configured into a single bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to embodiments may include signaling information and slice data, including SPS (Sequence Parameter Set) for sequence-level signaling, GPS (Geometry Parameter Set) for signaling of geometry information coding, APS (Attribute Parameter Set) for signaling of attribute information coding, and TPS (Tile Parameter Set) for tile-level signaling. The slice data may include information about one or more slices. One slice according to embodiments is one geometry bitstream (Geom0 0 ) and one or more attribute bitstreams (Attr0 0 , Attr1 0 It may include ).

[0250] A slice refers to a series of syntax elements representing all or part of a coded point cloud frame.

[0251] According to the embodiments, the TPS may include information regarding each tile (e.g., information on the coordinate values ​​of a bounding box and height / size information, etc.) for one or more tiles. The geometry bitstream may include a header and a payload. The header of the geometry bitstream according to the embodiments may include identification information of a parameter set included in the GPS (geom_parameter_set_id), a tile identifier (geom_tile_id), a slice identifier (geom_slice_id), and information regarding data included in the payload, etc. As described above, the metadata processing unit (8007) according to the embodiments may generate and / or process signaling information and transmit it to the transmission processing unit (8012). According to the embodiments, the elements performing geometry encoding and the elements performing attribute encoding may share data / information with each other as indicated by the dotted lines. The transmission processing unit (8012) according to the embodiments may perform an operation and / or transmission method identical or similar to the operation and / or transmission method of the transmitter (10003). A detailed explanation is omitted as it is the same as that described in FIGS. 1 and 2.

[0252] FIG. 9 is an example of a receiving device according to embodiments.

[0253] The receiving device illustrated in FIG. 9 is an example of the receiving device (10004) of FIG. 1 (or the point cloud decoder of FIG. 10 and FIG. 11). The receiving device illustrated in FIG. 9 can perform at least one of the same or similar operations and methods as the operations and decoding methods of the point cloud decoder described in FIG. 1 to FIG. 11.

[0254] A receiving device according to the embodiments may include a receiving unit (9000), a receiving processing unit (9001), an arithmetic decoder (9002), an occupancy code-based octree reconstruction processing unit (9003), a surface model processing unit (triangle reconstruction, up-sampling, voxelization) (9004), an inverse quantization processing unit (9005), a metadata parser (9006), an arithmetic decoder (9007), an inverse quantization processing unit (9008), a prediction / lifting / RAHT inverse transformation processing unit (9009), a color inverse transformation processing unit (9010), and / or a renderer (9011). Each component of the decoding according to the embodiments may perform the inverse process of the components of the encoding according to the embodiments.

[0255] A receiver (9000) according to the embodiments receives point cloud data. The receiver (9000) may perform an operation and / or a receiving method identical or similar to the operation and / or receiving method of the receiver (10005) of FIG. 1. A detailed description is omitted.

[0256] A receiving processing unit (9001) according to the embodiments can obtain a geometry bitstream and / or an attribute bitstream from the received data. The receiving processing unit (9001) may be included in the receiving unit (9000).

[0257] The arismetic decoder (9002), the Occupancy code-based octree reconstruction processing unit (9003), the surface model processing unit (9004), and the inverse quantization processing unit (9005) can perform geometry decoding. Since the geometry decoding according to the embodiments is identical or similar to the geometry decoding described in FIGS. 1 to 10, a detailed description is omitted.

[0258] The arismetic decoder (9002) according to the embodiments can decode a geometry bitstream based on arismetic coding. The arismetic decoder (9002) performs the same or similar operation and / or coding as the operation and / or coding of the arismetic decoder (7000).

[0259] According to the embodiments, the Occupancy code-based octree reconstruction processing unit (9003) can reconstruct an octree by obtaining an Occupancy code from a decoded geometry bitstream (or information regarding geometry obtained as a result of decoding). The Occupancy code-based octree reconstruction processing unit (9003) performs the same or similar operations and / or methods as the octree synthesis unit (7001) and / or octree generation method. According to the embodiments, the surface model processing unit (9004) can perform trisup geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, up-sampling, voxelization) based on the surface model method when trisup geometry encoding is applied. The surface model processing unit (9004) performs the same or similar operations as the surface offset synthesis unit (7002) and / or geometry reconstruction unit (7003).

[0260] The inverse quantization processing unit (9005) according to the embodiments can inverse quantize the decoded geometry.

[0261] A metadata parser (9006) according to the embodiments can parse metadata included in the received point cloud data, such as setting values, etc. The metadata parser (9006) can pass the metadata to geometry decoding and / or attribute decoding. A specific description of the metadata is omitted as it is the same as described in FIG. 8.

[0262] The arismetic decoder (9007), inverse quantization processing unit (9008), prediction / lifting / RAHT inverse transformation processing unit (9009), and color inverse transformation processing unit (9010) perform attribute decoding. Since attribute decoding is identical or similar to the attribute decoding described in FIGS. 1 to 10, a detailed description is omitted.

[0263] The arismetic decoder (9007) according to the embodiments can decode an attribute bitstream into arismetic coding. The arismetic decoder (9007) can perform decoding of the attribute bitstream based on reconstructed geometry. The arismetic decoder (9007) performs the same or similar operation and / or coding as the operation and / or coding of the arismetic decoder (7005).

[0264] The inverse quantization processing unit (9008) according to the embodiments can inverse quantize the decoded attribute bitstream. The inverse quantization processing unit (9008) performs the same or similar operation and / or method as the operation and / or inverse quantization method of the inverse quantization unit (7006).

[0265] According to the embodiments, the prediction / lifting / RAHT inverse transformation processing unit (9009) can process the reconstructed geometry and inverse quantized attributes. The prediction / lifting / RAHT inverse transformation processing unit (9009) performs at least one of the same or similar operations and / or decodings as the operations and / or decodings of the RAHT transformation unit (7007), LOD generation unit (7008), and / or inverse lifting unit (7009). According to the embodiments, the color inverse transformation processing unit (9010) performs inverse transformation coding to inversely transform the color values ​​(or textures) included in the decoded attributes. The color inverse transformation processing unit (9010) performs the same or similar operations and / or inverse transformation coding as the operations and / or inverse transformation coding of the color inverse transformation unit (7010). A renderer (9011) according to the embodiments can render point cloud data.

[0266] FIG. 10 shows an example of a structure that can be linked with a point cloud data transmission / reception method / device according to embodiments.

[0267] The structure of FIG. 10 represents a configuration in which at least one of a server (1060), a robot (1010), an autonomous vehicle (1020), an XR device (1030), a smartphone (1040), a home appliance (1050) and / or an HMD (1070) is connected to a cloud network (1010). The robot (1010), the autonomous vehicle (1020), the XR device (1030), the smartphone (1040), or the home appliance (1050) are referred to as devices. Additionally, the XR device (1030) may correspond to a point cloud data (PCC) device according to the embodiments or may be linked with a PCC device.

[0268] The cloud network (1000) may refer to a network that constitutes part of the cloud computing infrastructure or exists within the cloud computing infrastructure. Here, the cloud network (1000) may be configured using a 3G network, a 4G or LTE (Long Term Evolution) network, or a 5G network, etc.

[0269] The server (1060) is connected to at least one of a robot (1010), an autonomous vehicle (1020), an XR device (1030), a smartphone (1040), a home appliance (1050) and / or an HMD (1070) via a cloud network (1000) and can assist in at least some of the processing of the connected devices (1010 to 1070).

[0270] The HMD (Head-Mount Display) (1070) represents one of the types in which an XR device and / or PCC device according to the embodiments may be implemented. A device of the HMD type according to the embodiments includes a communication unit, a control unit, a memory unit, an I / O unit, a sensor unit, and a power supply unit, etc.

[0271] Hereinafter, various embodiments of the device (1010 to 1050) to which the above-described technology is applied are described. Here, the device (1010 to 1050) illustrated in FIG. 10 may be linked / coupled with a point cloud data transmission / reception device according to the above-described embodiments.

[0272] <PCC+XR>

[0273] The XR / PCC device (1030) may be implemented as a Head-Mount Display (HMD), a Head-Up Display (HUD) equipped in a vehicle, a television, a mobile phone, a smartphone, a computer, a wearable device, a home appliance, digital signage, a vehicle, a stationary robot, or a mobile robot by applying PCC and / or XR (AR+VR) technology.

[0274] The XR / PCC device (1030) can obtain information about surrounding space or real objects by analyzing 3D point cloud data or image data obtained through various sensors or from an external device to generate position data and attribute data for 3D points, and can render and output an XR object to be output. For example, the XR / PCC device (1030) can output an XR object containing additional information about a recognized object by associating it with the recognized object.

[0275] <PCC+XR+모바일폰>

[0276] The XR / PCC device (1030) can be implemented as a mobile phone (1040) or the like by applying PCC technology.

[0277] The mobile phone (1040) can decode and display point cloud content based on PCC technology.

[0278] <PCC+자율주행+XR>

[0279] The autonomous vehicle (1020) can be implemented as a mobile robot, vehicle, unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0280] An autonomous vehicle (1020) equipped with XR / PCC technology may refer to an autonomous vehicle equipped with means for providing XR images, or an autonomous vehicle that is the subject of control / interaction within the XR images. In particular, the autonomous vehicle (1020) that is the subject of control / interaction within the XR images is distinguished from the XR device (1030) and can be interconnected with it.

[0281] An autonomous vehicle (1020) equipped with means for providing XR / PCC images can acquire sensor information from sensors including cameras and output XR / PCC images generated based on the acquired sensor information. For example, the autonomous vehicle (1020) can provide an XR / PCC object corresponding to a real object or an object in the screen to the occupant by providing an XR / PCC object by outputting an XR / PCC image with a HUD.

[0282] At this time, when the XR / PCC object is displayed on the HUD, at least a portion of the XR / PCC object may be displayed so as to overlap with the actual object to which the occupant's gaze is directed. On the other hand, when the XR / PCC object is displayed on a display provided inside the autonomous vehicle, at least a portion of the XR / PCC object may be displayed so as to overlap with an object on the screen. For example, the autonomous vehicle (1220) may display XR / PCC objects corresponding to objects such as lanes, other vehicles, traffic lights, traffic signs, motorcycles, pedestrians, buildings, etc.

[0283] VR (Virtual Reality) technology, AR (Augmented Reality) technology, MR (Mixed Reality) technology and / or PCC (Point Cloud Compression) technology according to the embodiments can be applied to various devices.

[0284] In other words, VR technology is a display technology that provides real-world objects or backgrounds solely as CG images. On the other hand, AR technology refers to a technology that displays virtual CG images alongside images of real objects. Furthermore, MR technology is similar to the aforementioned AR technology in that it mixes and combines virtual objects with the real world. However, it is distinguished from AR technology in that while AR technology maintains a clear distinction between real-world objects and virtual objects created from CG images, using virtual objects to complement real-world objects, MR technology regards virtual objects as having the same nature as real-world objects. To give a more specific example, the aforementioned MR technology is applied in hologram services.

[0285] However, recently, rather than clearly distinguishing between VR, AR, and MR technologies, they are also referred to as XR (extended Reality) technology. Therefore, embodiments of the present invention are applicable to all VR, AR, MR, and XR technologies. These technologies may utilize encoding / decoding based on PCC, V-PCC, and G-PCC technologies.

[0286] The PCC method / device according to the embodiments can be applied to a vehicle providing autonomous driving services.

[0287] Vehicles providing autonomous driving services are connected to PCC devices to enable wired / wireless communication.

[0288] When a point cloud data (PCC) transceiver according to the embodiments is connected to a vehicle for wired or wireless communication, it can receive and process content data related to AR / VR / PCC services that can be provided along with an autonomous driving service, and transmit it to the vehicle. Additionally, when the point cloud data transceiver is mounted on a vehicle, the point cloud transceiver can receive and process content data related to AR / VR / PCC services according to a user input signal received through a user interface device and provide it to the user. A vehicle or a user interface device according to the embodiments can receive a user input signal. The user input signal according to the embodiments may include a signal indicating an autonomous driving service.

[0289] A point cloud transmission method / device (or encoding method and device) according to embodiments comprises a transmission device (10000) of FIG. 1, a point cloud video encoder (10002), a transmitter (10003), an acquisition-encoding-transmission (20000-20001-20002) of FIG. 2, an encoding process of FIG. 3, a transmission device of FIG. 8, a device of FIG. 10, encoding based on a layer structure of FIG. 13, bitstream segments of FIG. 14 to 15, bitstream alignment methods of FIG. 16 to 17, geometry-attribute selection of FIG. 18 to 19, encoding based on a slice configuration of FIG. 20, encoding based on a layer group of FIG. 21 to 23, context references of FIG. 24 to 30, FIG. 33 to 34, FIG. 60, encoding based on a buffer structure, bitstreams of FIG. 36 to 40, and The method may include and perform the generation of parameter information (syntax elements), the encoder of FIG. 41, the encoding process of FIG. 44, encoding according to the context buffer management method of FIG. 47 to 48, bitstream and parameter information (syntax elements) generation of FIG. 49 to 52, FIG. 55 to 59, FIG. 61 to 63, FIG. 82, the encoding process of FIG. 75, encoding based on ROI bounding box control of FIG. 77 to 78, FIG. 80 to 81, encoding according to node distribution by layer group of FIG. 83 and subgroup unit node alignment of FIG. 84, partial encoding of FIG. 85 to 87, encoding based on layer group slicing of FIG. 88, and encoding method of FIG. 89.

[0290] A point cloud receiving method / device (or decoding method and device) according to embodiments comprises a receiving device (10004) of FIG. 1, a receiver (10005), a point cloud video decoder (10006), a transmission-decoding-rendering (20002-20003-20004) of FIG. 2, a decoder of FIG. 7, a receiving device of FIG. 9, a device of FIG. 10, decoding based on a layer structure of FIG. 13, bitstream segments of FIG. 14 to 15, bitstream alignment methods of FIG. 16 to 17, geometry-attribute selection of FIG. 18 to 19, decoding based on slice configuration of FIG. 20, decoding based on layer group of FIG. 21 to 23, context references of FIG. 24 to 30, FIG. 33 to 34, FIG. 60, decoding based on buffer structure, FIG. 36 to The method may include and perform the acquisition of bitstream and parameter information (syntax elements) of FIG. 40, the decoder of FIG. 42, the sub-bitstream classifier flowchart of FIG. 43, the decoding process of FIG. 45, decoding according to the context buffer management method of FIG. 47 to 48, the acquisition of bitstream and parameter information (syntax elements) of FIG. 49 to 52, FIG. 55 to 59, and FIG. 82, ROI partial decoding of FIG. 64, the partial decoding process of FIG. 67, decoding based on ROI bounding box control of FIG. 77 to 78 and FIG. 80 to 81, decoding according to node distribution by layer group of FIG. 83 and sub-group unit node alignment of FIG. 84, partial decoding of FIG. 85 to 87, decoding based on layer group slicing of FIG. 88, and the decoding method of FIG. 90.

[0291] Additionally, the point cloud data transmission / reception method / device (or encoding / decoding method / device) according to the embodiments may be referred to simply as the method / device according to the embodiments.

[0292] According to the embodiments, geometry data, geometry information, location information, etc., constituting the point cloud data are interpreted as having the same meaning. Attribute data, attribute information, attribute information, etc., constituting the point cloud data are interpreted as having the same meaning. In addition, occupancy trees, octrees, etc., used when decoding / encoding geometry data are interpreted as having the same meaning.

[0293] The contents of this specification may be understood based on G-PCC standard specification documents, including ISO / IEC 23090-38, which were disclosed at the time of the priority date or filing date of this application.

[0294] Embodiments according to the present invention may include a method for efficiently supporting selective decoding of a portion of data when such decoding is required due to receiver performance or transmission speed when transmitting and receiving point cloud data. The embodiments may include a method for selecting necessary information or removing unnecessary information at the bitstream level by dividing geometry data and attribute data, which are conventionally transmitted as data units, into semantic units such as geometry octree and LoD (Level of Detail).

[0295] Embodiments according to the present invention may include techniques for constructing a data structure composed of a point cloud. Specifically, they may include packing and signaling methods for effectively transmitting PCC data configured based on layers, and methods for applying this to a scalable PCC-based service. Embodiments may include methods for constructing and transmitting / receiving slice segments to be more suitable for scalable PCC services when a direct compression mode is used for location compression. Embodiments may generate or acquire a compression structure for efficient storage and transmission of large-capacity point cloud data with a wide distribution and high point density.

[0296] Referring to FIGS. 3 and FIGS. 7, point cloud data consists of the location (geometry: e.g., XYZ coordinates) and attributes (attributes: e.g., color, reflectance, intensity, grayscale, opacity, etc.) of each data point. Point Cloud Compression (PCC) performs octree-based compression to efficiently compress distribution characteristics that are non-uniformly distributed in three-dimensional space, and compresses attribute information based on this.

[0297] FIGS. 11 and 12 illustrate a conventional partial encoding and partial decoding process.

[0298] Point cloud data is compressed and transmitted by dividing location information of data points and feature information such as color, brightness, and reflectivity into geometry and attribute information. At this time, PCC data can be configured in an octree structure with layers depending on the level of detail or according to the Level of Detail (LoD), and based on this, scalable point cloud data coding and representation are possible. However, depending on the performance of the receiver or the transmission speed, it is possible to decode or represent only a part of the point cloud data, and conventionally, there is no method to remove unnecessary data in advance.

[0299] In other words, when only a portion of a scalable PCC bitstream needs to be transmitted (when only some layers of scalable decoding are decoded), it is not possible to select and send only the necessary parts; therefore, as shown in Fig. 11, the necessary parts must be re-encoded after decoding, or as shown in Fig. 12, the entire stream must be transmitted and then selectively applied at the receiver. However, in the case of Fig. 11, delay may occur due to the time required for decoding and re-encoding, and in the case of Fig. 12, bandwidth efficiency is reduced because unnecessary data is transmitted, and there is a disadvantage that data quality must be lowered when using a fixed bandwidth.

[0300] In this case, for octree-based geometry compression, entropy-based compression methods and direct coding can be used together, and in this case, a slice configuration is required to efficiently utilize scalability.

[0301] In addition, for large-scale point clouds with a wide distribution and high point density, latency issues may occur due to the large number of bitstreams that must be processed to access the region of interest (ROI).

[0302] The embodiments can support partial decoding, spatial random access, and progressive decoding through layer-group slicing, and when using context continuation, context buffer management may be required for efficient memory usage.

[0303] The embodiments may include an efficient and accurate bounding box information signaling method and a boundary information representation method based on an index on a subgroup unit node grid.

[0304] Modifications and / or combinations between embodiments of the present invention are possible, and terms used herein may be understood based on their intended meanings within the scope of their widespread use in the field.

[0305] FIG. 13 shows an example of a point cloud data configuration consisting of layers according to embodiments.

[0306] The embodiments aim for efficient transmission and decoding by selectively transmitting and decoding at the bitstream level for layered point cloud data.

[0307] Referring to Fig. 13, the layering of point cloud data can have layer structures in various aspects such as SNR, sparial resolution, color, temporal frequency, and bit depth depending on the application field, and can form layers in a direction in which the density of data increases based on an octree structure or a LoD structure.

[0308] FIG. 14 shows a bitstream segment according to embodiments.

[0309] Referring to Fig. 14, the bitstream obtained through point cloud compression can be divided into a geometry data bitstream and an attribute data bitstream and transmitted according to the type of data. At this time, each bitstream can be transmitted by configuring it into slices, and the geometry data bitstream and the attribute data bitstream can each be configured into a single slice and transmitted regardless of layer information or LoD information. In this case, if only a part of the layer or LoD is to be used, the following steps must be taken: 1) decoding the bitstream, 2) selecting only the parts to be used and removing unnecessary parts, and 3) re-encoding based only on the necessary information.

[0310] FIG. 15 shows a bitstream fragment according to the embodiments.

[0311] The embodiments may include a method of dividing the bitstream into layers (or LoDs) to avoid these unnecessary intermediate processes.

[0312] For example, considering the case of LoD-based PCC technology, it has a structure in which a low LoD (or LoD for a coarse level) is included in a high LoD (or LoD for a fine level). If we define R as information that is included in the current LoD but not in the previous LoD, that is, information newly included for each LoD, then as shown in Fig. 15, the initial LoD information and the information R newly included in each LoD can be divided and transmitted as independent units.

[0313] In the embodiments described above, the low level LoD was explained as an example meaning the LoD for a coarse level, and the high level LoD was explained as the LoD for a fine level. However, this is not limited thereto, and according to the embodiments, the high level LoD may mean the LoD for a coarse level, and the low level LoD may mean the LoD for a fine level. Hereinafter, the explanation will be based on the premise that the low level LoD means the LoD for a coarse level and the high level LoD means the LoD for a fine level, as described above.

[0314] Bitstream alignment method

[0315] FIG. 16 illustrates a bitstream alignment method according to embodiments.

[0316] When transmitting a bitstream, geometry and attributes can be transmitted serially as shown in FIG. 16. In this case, depending on the type of data, the entire geometry information can be sent first, followed by the attribute information. In this case, there is an advantage that geometry information can be quickly restored based on the transmitted information.

[0317] FIG. 17 illustrates a bitstream alignment method according to embodiments.

[0318] In another method according to the embodiments, bitstreams constituting the same layer may be collected and transmitted as shown in FIG. 17. In this case, if a compression technique capable of parallel decoding of geometry and attributes is used, the decoding time can be reduced. Information that needs to be processed first (small LoD or LoD for coarse levels, geometry must precede attributes) can be placed first.

[0319] Bitstream selection

[0320] When transmitting a bitstream based on the method according to the embodiments, the desired layer (or LoD) in the application field can be selected at the bitstream level. When geometry information is collected and transmitted as shown in FIG. 16, there may be gaps in the middle after selecting the bitstream level, and in this case, the bitstream may need to be rearranged. When geometry and attributes are bundled and transmitted according to the layer as shown in FIG. 17, unnecessary information can be selectively removed as follows depending on the application field.

[0321] Symmetric Geometry-Attribute Selection

[0322] FIG. 18 shows an example of symmetrically selecting geometry attributes according to embodiments.

[0323] Referring to Fig. 18, this shows a case where only up to LoD1 is selected for transmission or decoding, and information about R2 corresponding to the upper layer can be removed and transmission / decoding can be performed.

[0324] Asymmetric Geometry-Attribute Selection

[0325] FIG. 19 shows an example of asymmetrically selecting geometry attributes according to embodiments.

[0326] Referring to Fig. 19, when geometry and attributes are transmitted asymmetrically, only the attributes of the upper layer can be removed and the entire geometry (gray area in the octree structure of triangles) can be selected for transmission / decoding.

[0327] Slice-level scalability vs. Octree-level scalability

[0328] The LoD defined in the embodiments of the present invention can be used as a unit to represent a set of one or more octree layers, and may also have the meaning of a bundle of octree layers to be configured in slice units. That is, while spatial scalability by actual octree layers (or scalable attribute layers) can be provided for each octree layer, when configuring scalability in slice units prior to bitstream parsing, it can be selected from the LoD unit defined in the embodiments of the present invention.

[0329] That is, as shown in Fig. 13, when utilizing scalability at the slice level, such as in scalable transmission, the provided scalable stages are three stages, LoD0, LoD1, and LoD2, and the scalable stages that can be provided in the decoding stage by the octree structure are eight stages from the root to the leaf.

[0330] Referring to FIG. 13, in the case where LoD0 to LoD2 are each composed of slices according to the embodiments, the transcoder of the receiver or transmitter may 1) select only LoD0, 2) select LoD0 and LoD1, or 3) select LoD0, LoD1, and LoD2.

[0331] 1) When only LoD0 is selected, the maximum octree level becomes 4, and one scalable layer among the octree layers from 0 to 4 can be selected during the decoding process. In this case, the receiver can consider the node size obtainable through the maximum octree depth as a leaf node, and signal the node size at that time.

[0332] 2) When LoD0 and LoD1 are selected, Layer 5 is added, making the maximum octree level 5, and one scalable layer among the octree layers from 0 to 5 can be selected during the decoding process. In this case, the receiver can consider the node size obtainable through the maximum octree depth as a leaf node, and signal the node size at that time.

[0333] 3) When LoD0, LoD1, and LoD2 are selected, layers 6 and 7 are added, making the maximum octree level 7, and one scalable layer among the octree layers from 0 to 7 can be selected during the decoding process. In this case, the receiver can consider the node size obtainable through the maximum octree depth as a leaf node and signal the node size at that time.

[0334] Slice composition

[0335] FIG. 20 shows an example of a slice configuration according to embodiments.

[0336] As a technique for dividing a G-PCC bitstream into a slice structure, slices can be configured in finer units. For example, not only can one or more octree layers be matched in a single slice, but some nodes of an octree layer may also be included in a single slice, as shown in FIG. 20(a). Alternatively, as shown in FIG. 20(b) and FIG. 20(c), when multiple octree layers are matched in a single slice, only some nodes of each layer may be included. In such cases, when multiple slices constitute a geometry / attribute frame, information necessary to configure the layers can be transmitted to the receiver. This may include layer information included in each slice, node information included in each layer, etc.

[0337] scalable transmission

[0338] When a structure like that shown in Fig. 20 is used for scalable transmission, it can transmit information for selecting the slice required by the receiver. Scalable transmission may mean supporting the transmission or decoding of only a portion of the bitstream rather than decoding the entire bitstream, and the result may be low-resolution point cloud data.

[0339] When applying scalable transmission to octree-based geometry bitstreams, it must be possible to construct point cloud data using only information up to a specific octree layer for the bitstreams of each octree layer from the root node to the leaf node. To achieve this, the target octree layer must not have any dependency on information from lower octree layers. This can be a constraint that applies commonly to geometry / attribute coding.

[0340] In addition, when transmitting scalable data, it is necessary to provide a scalable structure for selecting scalable layers at the transmitter and receiver. Considering the octree structure of FIG. 20, all octree layers may support scalable transmission, but scalable transmission may be enabled only for specific octree layers or lower. If some of the octree layers are included, by indicating which scalable layer the slice belongs to, it is possible to determine whether the slice is necessary or unnecessary at the bitstream stage.

[0341] For example, scalable transmission may not be supported for (1) starting from the root node in FIG. 20(a), and a single scalable layer may be configured, and the subsequent octree layers may be configured to have a one-to-one match with the scalable layer. Generally, scalability may be supported for parts corresponding to leaf nodes, and as in FIG. 20(c), when multiple octree layers are included within a slice, a single scalable layer may be defined for those layers.

[0342] In this case, scalable transmission and scalable decoding can be distinguished and used depending on the purpose. Scalable transmission can be used to select information up to a specific layer without passing through a decoder at the transmitting and receiving ends, while scalable decoding is intended to select a specific layer during the coding process. In other words, scalable transmission supports the selection of necessary information in a compressed state (at the bitstream stage) without passing through a decoder, enabling identification at the transmitting or receiving end. On the other hand, scalable decoding supports encoding or decoding only up to the necessary parts during the encoding / decoding process, allowing it to be used in cases such as scalable representation.

[0343] In this case, the layer configuration for scalable transmission and the layer configuration for scalable decoding may differ. For example, the lower three octree layers including the leaf node may be configured as a single layer from the perspective of scalable transmission, but from the perspective of scalable decoding, if they include all layer information, scalable decoding may be possible for the leaf node layer, leaf node layer-1, and leaf node layer-2, respectively.

[0344] FIG. 21 shows an example of a geometry tree structure included in a slice structure according to embodiments.

[0345] According to the specifications in the current G-PCC standard document, the entire encoded bitstream is contained in a single slice. In contrast, for multiple slices, each slice may contain sub-bitstreams, and the order of the slices is the same as the order of the sub-bitstreams. Since the bitstreams are accumulated in breadth-first order of the geometry tree, each slice can be matched with a group of tree layers, as shown in FIG. 21(b).

[0346] Since split slices inherit the layering structure of the G-PCC bitstream, subsequent slices may not affect preceding slices, just as the upper layer of a geometry tree does not affect the lower layer. This characteristic can provide split slices with advantages in terms of error robustness, efficient transmission, and region of interest (ROI) support.

[0347] Error resilience

[0348] Compared to the single-slice structure defined in the G-PCC standard, segmented slices can be more robust against errors. If a slice contains the entire frame bitstream, data loss can affect the entire frame data. On the other hand, if the bitstream is divided into multiple slices, some slices that are unaffected by the loss can be decoded.

[0349] Scalable transmission

[0350] Consider an application that supports multiple decoders with different capabilities. If the encoded data exists in a single slice, the Level of Detail (LoD) of the encoded point cloud must be determined prior to encoding. Consequently, multiple pre-encoded bitstreams containing point cloud data of different resolutions are transmitted independently, which is inefficient in terms of large bandwidth or storage space. If a PCC bitstream is generated and included in the divided slices, different levels of decoders can be supported with a single bitstream. At the decoder side (receiver), target layers can be selected and the partially selected bitstream can be passed to the decoder. Similarly, at the transmitter side, partial PCC bitstreams can be efficiently generated using a single PCC bitstream without parting the entire bitstream.

[0351] Region-based spatial scalability

[0352] In the G-PCC requirements, domain-based spatial scalability can be defined as follows:

[0353] The compressed bitstream must be structured into one or more layers so that a specific region of interest (ROI) can have higher density through additional layers; where the layers can be predicted from the lower layer(s).

[0354] To support these requirements, it is necessary to support different detailed representations for each region. For example, in VR / AR applications, it is desirable to represent nearby objects with high precision and distant objects with low precision. Alternatively, the decoder can increase the resolution of the region of interest upon request. This can be realized using scalable structures of G-PCC, such as geometry octree and scalable attribute coding techniques. In current slice structures that include the entire geometry or attributes, the decoder must access the entire bitstream, which can lead to reduced bandwidth, memory, and decoder efficiency. On the other hand, if the bitstream is divided into multiple slices and each slice contains sub-bitstreams according to the scalable hierarchy, the decoder can efficiently select the necessary slices before parsing the bitstream.

[0355] The embodiments may include a method for splitting geometry and attribute bitstreams contained in different slices. Additionally, the methods may include a method using a coding tree structure of geometry and attribute encoding, and a method in which each slice contains partial tree information in terms of tree depth.

[0356] FIG. 22 shows a geometry tree structure and a slice segment structure according to embodiments.

[0357] Referring to FIG. 22, there may be eight layers in the octree, and five slices may be used to contain sub-bitstreams of one or more layers. A group represents a group of geometry tree layers. According to embodiments, a group may mean a layer group. For example, group 1 may include layers 0 through 4 of the octree, group 2 may include layer 5, and group 3 may include layers 6 and 7.

[0358] Additionally, one group can be divided into three sub-groups, where parent and child pairs must exist within each sub-group. Referring to FIG. 22, groups 3-1 through 3-3 are sub-groups of group 3.

[0359] If scalable attribute coding is used and the tree structure is identical to the geometry tree structure, the same octree-slice mapping can be used to generate attribute slice segments, as shown in FIG. 22(b).

[0360] According to the embodiments, a layer group may refer to a group of layer structure units occurring in G-PCC coding, such as an octree layer, a LoD layer, etc.

[0361] According to the embodiments, a subgroup can be represented as a set of adjacent nodes based on geometry information for a layer group. Alternatively, the grouping can be formed based on the lowest layer within the layer group. Here, the lowest layer may refer to the layer closest to the root direction, and for example, may be layer 6 in group 3 of FIG. 22.

[0362] In addition, according to the embodiments, groups of adjacent nodes may be formed by Morton code order, by distance-based adjacent nodes, or by coding order. Additionally, nodes in a parent-child relationship may be defined to exist within a single subgroup.

[0363] When defining subgroups, boundaries occur in the middle of the layer, and regarding whether to maintain continuity at the boundary, continuity with the previous slice can be maintained by indicating whether to use entropy continuously and the ref_slice_id, such as sps_entropy_continuation_enabled_flag, gsh_entropy_continuation_flag.

[0364] Specifically, according to the embodiments, a layer-group is a group of consecutive tree levels of an octree, and each tree level may belong to only one layer-group. The minimum depth of a subgroup may be the maximum depth of its parent subgroup plus 1, or 0 in the case of a root layer-group. The maximum depth of a subgroup may be the minimum depth of its child subgroup minus 1, or the maximum depth of the octree for the last layer-group. A layer-group may be identified by a layer-group index (layer_group_id).

[0365] A subgroup is a spatial subset of a layer group, and a node at a single tree level can belong to only one subgroup within a single layer group. The location range of nodes within a subgroup must be described by bounding boxes, and these bounding boxes must not overlap with the bounding boxes of other subgroups within the same layer group. The set of nodes contained in all subgroups within a single layer group must be identical to the set of nodes within that layer group. For the root layer group, there is only one subgroup. Subgroups within a layer group can be identified by a subgroup index (subgroup_id). Each subgroup can correspond to a partial octree.

[0366] FIG. 23 shows the structure of a layer group, subgroup, and bounding box according to embodiments.

[0367] FIG. 23 is an example of slice selection based on subgroup bounding box information, and according to the embodiments, a slice matching the region of interest (ROI) can be determined and selected based on subgroup bounding box information.

[0368] According to the embodiments, when a bitstream compressed for a full coding layer is divided into slices and transmitted, receivers with different performance capabilities can be supported. When selectively decoding slices based on a region of interest (ROI) or receiver performance, the selection can be done directly at the receiver or at the transcoder. When the selection is done at the transcoder, information regarding full decoding (e.g., full coding layer depth, total number of layer groups, total number of subgroups, etc.) is not available. Since such information may be required during the decoding process at the receiver, such information can be transmitted directly or inferred as num_skipped_layer_groups and num_skipped_layers.

[0369] As described above, the embodiments can divide and transmit the compressed bitforce for the entire coding layer into slices, each slice may be referred to as a Fine Granularity Slice (FGS), and multiple FGSs may be defined as being included in a single parent slice. Each FGS may be mapped one-to-one to a subgroup within a layer group and may be identified by a pair consisting of a layer group index (layer_group_id) and a subgroup index (subgroup_id).

[0370] Each FGS may contain FGS geometry and / or FGS attributes. All FGS geometry includes a GDU or DGDU encoding partial slice geometry. All FGS attributes include an ADU or DADU encoding partial slice attributes.

[0371] The first FGS geometry within a slice must be a GDU, and this FGS geometry may be followed by a DGDU that depends on a decoded GDU and a DGDU. Additionally, the first FGS attribute within a slice must be an ADU, and this FGS attribute may be followed by a DADU that depends on a decoded ADU and a DADU. The FGS attribute is located after the FGS geometry identified by the same pair of layer group index and subgroup index.

[0372] context inheritance

[0373] FIG. 24 shows a context reference of a layer group slicing structure according to embodiments.

[0374] Referring to FIG. 24, each FGS is matched to a subgroup, and subgroups located in the same row belong to the same layer group. An arrow pointing from one slice (FGS) to another slice indicates a context referencing relationship between the two slices. The context reference target of the current slice may be one of the slices decoded prior to the current slice.

[0375] When considering the spatial random access use case, referencing a parent subgroup can guarantee independence among subgroups. However, as the number of layer groups and subgroups increases, the number of context buffers may increase. When considering the number of context buffers as the number of referenced slices, the number of context buffers can be the sum of all subgroups excluding those belonging to the first and last layer groups. This can be expressed by the following formula, where N represents the number.

[0376]

[0377] FIG. 25 shows an example of a context reference structure of a layer group according to embodiments.

[0378] To mitigate the issue of context buffer size, the number of subgroups referenced by following slices can be reduced. An extreme case of this is a method of referencing the root slice, as shown in FIG. 25. Since the number of referenced subgroup slices becomes zero based on the aforementioned number of context buffers, an approach like FIG. 25 can result in requiring only a single context buffer.

[0379]

[0380] That is, in the case of Fig. 25, all slices refer to the first slice, and compared to Fig. 24, since all dependent slices refer to the first slice, the number of stored states is reduced to one.

[0381] FIG. 26 shows a context buffer change of a parent subgroup reference according to embodiments.

[0382] Referring to FIG. 26, when a bitstream within a slice is decoded, the context state can be stored in a context buffer as in FIG. 26(a). As in FIG. 26(b), when context inheritance is used, the context states of subsequent slices can be initialized by one of the stored states of preceding slices indicated by ref_layer_group_id and ref_subgroup_id.

[0383] Accordingly, the context state of a slice belonging to the last layer group can be initialized by the stored context state of previous slices, as shown in FIG. 26(c). However, if there is a constraint that does not refer to subgroups within the same layer group, the decoders may decide not to store the context of FGS N+1 to 2N. Through prior information, the smart decoder can save the context buffer.

[0384] FIG. 27 shows a context buffer change of a root layer group reference according to embodiments.

[0385] When flexible context references are allowed, there exist context states that are not used by subsequent slices. For example, as shown in FIG. 27, one can consider root layer-group referencing where all context states of dependent slices are initialized by the context state stored from the first slice. This is because decoders do not know the entire reference structure and there is a possibility that subsequent slices will use the current context. However, as expected, the stored context states (1, 0) through (1, N-1) are not used by any slices. Such inefficiency is due to a lack of information on the decoder side.

[0386] To improve context buffer management efficiency, embodiments may include a new signal to help decoders indicate whether to use the current context in subsequent slices and to determine whether to save the context.

[0387] To improve context buffer management on the decoder side, embodiments may provide context reference information in terms of the current slice. In other words, embodiments may include an indication for context state reference by subsequent slices.

[0388] Figure 28 shows an example of buffer management using context reference indicators.

[0389] Referring to FIG. 28(b), storing the context state in the context buffer can be determined by a context reference indicator called context_reference_indication_flag. If the context reference indicator is on, the decoder may store the current context state in the context buffer to allow the use of the context state by subsequent slices. Conversely, if the context reference indicator is off, the decoder may not store the current context state in the context buffer to conserve context memory. Comparing FIG. 28(c) with FIG. 27(c), the context buffer for the root layer reference case has only one context compared to the context buffer for the parent subgroup reference case. The size of the stored context memory is N units, where N represents the number of subgroups.

[0390] FIG. 29 is a comparison table of memory usage and compression loss for context reference methods according to embodiments.

[0391] FIG. 29 is a comparison table of memory usage and compression loss for fixed and flexible context reference schemes to examine the effects of using flexible context inheritance. As described above, ref-parent, a context reference of a parent subgroup, and ref-root, a context reference of a root layer group, can be considered as representative examples of fixed and flexible context reference schemes.

[0392] For each method, the amount of memory used to store the GeometryOctreeContext during each decoding process of the slices is estimated. In this experiment, Statue_Klimt_vox12.ply was used as input, which generates 10 subgroups under specific test conditions. Memory usage at each step was estimated using the diagnostic tool of Visual Studio 15 2017. Referring to Figure 29, the memory usage of each method is plotted in bytes and the number of contexts. The number of contexts is derived using the fact that the size of one GeometryOctreeContext is 18,240 bytes.

[0393] In the case of parent referencing, the number of stored contexts increases monotonically from slice 0 to slice 5. For slice 10, the expected number of contexts is 11 (= 1 + 10), but the number of contexts estimated by the actual size is 13, which is due to the compiler's memory allocation strategy. Additionally, for the slices of the last layer group, namely slices 11 through 20, the size of memory for storing contexts does not change. On the other hand, in the ref-root case, the number of context memories does not change, which means that only one context is used for all slices from 1 to 20.

[0394] FIG. 30 is a comparison table of memory usage and compression loss for context reference methods according to embodiments.

[0395] To verify the behavior of memory usage according to the number of slices, the same memory check was performed on the same content with an increased number of layer groups. The layer group structure was changed from 8-3-1 to 8-1-1-1-1. In Fig. 30, the number of contexts stored for ref-parent increases, except for slices 31 to 40 corresponding to layer group 4. On the other hand, for ref-root, the number of context memories for all slices does not change.

[0396] FIG. 31 is a comparison table of average bit rates between a root layer group reference and a parent subgroup reference according to embodiments.

[0397] Referring to Fig. 31, the average bitrates of the two methods can be compared to investigate the compression loss caused by changes in context reference. As shown in Fig. 31, the average compression loss for C2 and CW conditions is 0.1%. In terms of time consumption, root layer group reference exhibits 3% and 4% less decoding time than parent subgroup reference.

[0398] FIG. 32 is a bitdule comparison table based on whether a context reference indicator is used according to the embodiments.

[0399] Referring to Fig. 32, the overall compression difference between using and not using a context reference indicator is summarized. Consequently, the effect of the additional signal appears to be minimal. Furthermore, in some cases where the 1-bit signal does not increase (the total size) due to bit alignment at the end of the dependent data unit header, the bit rate may not change.

[0400] Referring to the pseudocode in Table 1 below, the receiver can use context, phi buffer, planar context, and buffer continuity between slices to prevent coding efficiency degradation caused by dividing slices, and the receiver can determine whether to store the context, phi buffer, and planar context based on whether the context is reused.

[0401] if (_dep_gbh.context_reuse_flag) {_refIdxToSavedArrayIdx[curLayerGroup][_dep_gbh.subgroup_id] = _ctxtMemSaved.size();_ctxtMemSaved.push_back(cur_ctxtMem);if(_gps->geom_angular_mode_enabled_flag)_phiBuffer Saved.push_back(cur_phiBuffer);if(_gps->geom_planar_mode_enabled_flag)_planarSaved.push_back(cur_planar);int idx = _refIdxToSavedArrayIdx[curLayerGroup][_dep_gbh.subgroup_id];}

[0402] Indication of decoder context buffer release time

[0403] As described above, when indicating whether to reuse the context, by indicating whether to store the context used in a specific slice in the context buffer, the context buffer memory can be used efficiently by selectively storing only the context required when encoding / decoding a subsequent slice.

[0404] However, even in such cases, if the context remains stored in the context buffer until the encoding / decoding of the frame is finished, the burden on the context buffer may increase as the number of stored contexts increases. Therefore, to use the context buffer more efficiently, a method is required to release or delete the context from the buffer when the stored context is no longer in use.

[0405] When managing context memory based on a list

[0406] FIG. 33 illustrates a list-based context management method according to embodiments.

[0407] The embodiments may include a method for managing a list of slices or subgroups that use context information stored in a buffer to effectively manage context memory.

[0408] Referring to FIG. 33, the context_reference_indication_flag can indicate whether the context used to encode the current slice is stored. That is, if context_reference_indication_flag = 1, it can indicate that the context of the current slice / subgroup can be used in the slice / subgroup passed thereafter, and this can be stored in the context buffer.

[0409] To use the context later, the context index can be set to be the same as the subgroup index. In this case, if information about the slice / subgroup using the current context is provided as a list, the target list can be stored and compared with the list of cases actually used.

[0410] A list of slices / subgroups using context can be pre-investigated and passed by the encoder, or a list can be generated by the decoder estimating context reference relationships based on a predetermined layer-group structure. For example, if the method of referencing the context of a parent subgroup is fixed, a list of subgroups in a child relationship with respect to the current subgroup can be investigated, and this is referred to as List A in FIG. 33.

[0411] When a new slice / subgroup is passed, the context used to decode the slice / subgroup can be specified based on the context reference ID. In FIG. 33, the context corresponding to (0,0) is used, and (1,1), which is the index of the current slice / subgroup, can be added to the list of used subgroups. In FIG. 33, the list of slices / subgroups that use a specific context during the encoding process is referred to as List B.

[0412] If context_reference_indication_flag is 1, it means that the context of the current slice / subgroup will be used later, so it can be stored in the context buffer, and List B (1,1) of the context buffer (1,1) can be initialized.

[0413] Memory can be managed by releasing the context buffer when List A and List B are the same for a specific context buffer. As shown in FIG. 33, when the context reference ID is (1,1), encoding can be performed using the context state (1,1) in the context buffer. At this time, the index (2,1) of the current slice / subgroup can be added to List B. Since List B is identical to List A = (2,1), it can be seen that the context state (1,1) will not be used in the future; in this case, the memory of the context buffer can be effectively managed by releasing the memory storing the context state (1,1).

[0414] As described above, the embodiments may include a method for specifying an index of the context based on a layer group index and a subgroup index. However, if a unique slice index is assigned to each slice, the context buffer list may be managed based on the slice index.

[0415] When managing context memory based on the number of references

[0416] FIG. 34 illustrates context buffer management according to embodiments.

[0417] Implementations can efficiently use the context buffer by managing context states that are no longer in use in real time based on the number of times each context state is used. Implementations can manage the context buffer through a target number and a counter for each context state in the context buffer.

[0418] According to the embodiments, the target number stores the number of times each context state is used, and a counter can update the number of times the corresponding context state is used. FIG. 34(a) shows the case of the first slice / subgroup, where context_reference_indication_flag is 1 or layer-group slicing is used, and the context state (0,0) can be stored in the context buffer. Additionally, the target number may store N, which is the number of times the context state is used, and this can be specified as the same value as N, which is the number of subgroups belonging to layer group 1, depending on the case where the context state of the parent subgroup is used. In other words, the list of child subgroups of FGS 0 can be obtained as FGS 1 (1,0), FGS 2 (1,1), ..., FGS N (1, N-1), and the number of elements N in the above list can be specified as the target number.

[0419] When coding a new slice / subgroup, the context reference of the slice / subgroup can be identified in the context buffer through the context reference ID. Referring to FIG. 34, the context state (0,0) is used, and the counter number can be increased by 1. Since the context state (0,0) was already used to code FGS 1 (1,0), the counter can have a value of 2. Additionally, since context_reference_indication_flag being 1 means that it can be used as a context reference in a subsequent slice / subgroup, the context state of FGS 2 (1,1) can be stored in the context buffer. At this time, the number of child slices / subgroups using the context state (1,1) can be stored in the target number.

[0420] For context states where the number of targets managed in the context buffer and the counter are the same, the memory usage of the context buffer can be managed through memory release. Referring to FIG. 34, the context state (1,1) corresponding to the context reference ID of FGS N+2 can be used, and the counter (1,1) can be increased by 1. In this case, the number of targets and the counter of the context state (1,1) become the same, which means that the context state (1,1) is no longer used. This means that even if the context state (1,1) is deleted from the context buffer, there is no effect on the encoding of subsequent slices / subgroups. In the case of context states that are no longer used, the memory usage of the context buffer can be minimized by deleting them from the context buffer.

[0421] Table 2 below shows the pseudocodes of the decoder according to the embodiments.

[0422] if (_dep_gbh.context_reference_indication_flag) {_refIdxToSavedArrayIdx[curLayerGroup][_dep_gbh.subgroup_id] = _ctxtMemSaved.size();_ctxtMemSaved.push_back(cur_ctxtMem);_numSubsequentSubgroups.push_back(_dep_gbh.numSubsequentSubgroups);if (_gps->geom_angular_mode_enabled_flag)_phiBufferSaved.push_back(cur_phiBuffer);if (_gps->geom_planar_mode_enabled_flag)_planarSaved.push_back(cur_planar);}_numSubsequentSubgroups[refArrayIdx]--;if (_numSubsequentSubgroups[refArrayIdx] == 0) {_ctxtMemSaved[refArrayIdx].resetMap();_ctxtMemSaved[refArrayIdx].reset();}

[0423] In the embodiments, the number of targets for a given context state, i.e., the number of subsequent subgroups, can be investigated and transmitted by the encoder. If necessary, by additionally transmitting a list of slices / subgroups used as context references, it is possible to verify whether the received information on the number of subsequent subgroups is accurate and to determine whether to delete the context state.

[0424] When managing context memory based on a data unit coding structure

[0425] FIG. 35 shows the layer group coding order of the breadth-first search method and the depth-first search method according to the embodiments.

[0426] The embodiments may pass slices / subgroups based on a specific order, such as Breadth First Search (BFS) or Depth First Search (DFS). Breadth First Search is a method of encoding subgroups belonging to the same layer group and then encoding subgroups belonging to the child layer group. In contrast, Depth First Search is a method of encoding children belonging to the same parent first after reaching a subgroup corresponding to the maximum depth.

[0427] Referring to FIG. 35, when each node is considered as an FGS slice index, it can be assumed that slice 0 belongs to layer group 0, slices 1 and 2 belong to layer group 1, and slices 3, 4, 5, and 6 belong to layer group 2. Additionally, slices connected by solid lines can be assumed to represent a pair of slices belonging to a parent-child relationship. When encoded based on breadth-first search, they can be encoded in the order 0, 1, 2, 3, 4, 5, 6, and when encoded based on depth-first search, they can be encoded in the order 0, 1, 3, 4, 2, 5, 6.

[0428] When managing context buffer memory based on slice order, it can operate as follows. In this case, it is assumed that the parent context state is used as the reference context.

[0429] If encoding is based on Breadth-First Search (BFS), the context state of the parent layer group is no longer used when the encoding of the subgroups belonging to each layer group ends. That is, context state 0 is used when encoding slices 1 and 2, but context state 0 is no longer used when slice 2 is encoded. In this case, the context state of the parent subgroup (context state of slice 0) can be removed from the context buffer at the time the layer group is switched (slice 2).

[0430] If encoding is performed based on Depth-First Search (DFS), the context state can be removed at the point when encoding for a child subgroup ends or when a layer group is switched (switching from bottom to root). Referring to Fig. 35, after encoding of slices 3 and 4 ends, the process moves on to slice 2. At this time, slices 3 and 4 belong to layer group 2, and slice 2 belongs to layer group 1. Slices 3 and 4 are encoded using the context state of slice 1, and since context state 1 is no longer used, it can be removed from the context buffer.

[0431] In such cases, context memory can be used efficiently without passing additional information, such as a subsequent subgroup list or the number of subsequent subgroups.

[0432] FIG. 36 shows a bitstream structure according to embodiments.

[0433] Transmitting device / method or encoding method / device according to embodiments (transmitting device (10000) of FIG. 1, point cloud video encoder (10002), transmitter (10003), acquisition-encoding-transmission (20000-20001-20002) of FIG. 2, encoding process of FIG. 3, transmission device of FIG. 8, device of FIG. 10, encoding based on a layer structure of FIG. 13, bitstream segments of FIG. 14 to 15, bitstream alignment method of FIG. 16 to 17, geometry-attribute selection of FIG. 18 to 19, encoding based on a slice configuration of FIG. 20, encoding based on a layer group of FIG. 21 to 23, context reference of FIG. 24 to 30, FIG. 33 to 34, and FIG. 60, encoding based on a buffer structure, bitstream and parameters of FIG. 36 to 40 The generation of information (syntax elements), the encoder of FIG. 41, the encoding process of FIG. 44, the encoding according to the context buffer management method of FIG. 47 to FIG. 48, the generation of bitstream and parameter information (syntax elements) of FIG. 49 to FIG. 52, FIG. 55 to FIG. 59, FIG. 61 to FIG. 63, FIG. 82, the encoding process of FIG. 75, the encoding based on ROI bounding box control of FIG. 77 to FIG. 78, FIG. 80 to FIG. 81, the encoding based on node distribution by layer group of FIG. 83 and subgroup unit node alignment of FIG. 84, partial encoding of FIG. 85 to FIG. 87, encoding based on layer group slicing of FIG. 88, the encoding method of FIG. 89, etc.) can encode point cloud data, generate the bitstream of FIG. 36, encapsulate a file containing the bitstream, and transmit it to a decoder.

[0434] A receiving device / method or a decoding method / device according to embodiments (receiving device (10004) of FIG. 1, receiver (10005), point cloud video decoder (10006), transmission-decoding-rendering (20002-20003-20004) of FIG. 2, decoder of FIG. 7, receiving device of FIG. 9, device of FIG. 10, decoding based on a layer structure of FIG. 13, bitstream segment of FIG. 14 to 15, bitstream alignment method of FIG. 16 to 17, geometry-attribute selection of FIG. 18 to 19, decoding based on a slice configuration of FIG. 20, decoding based on a layer group of FIG. 21 to 23, context reference of FIG. 24 to 30, FIG. 33 to 34, FIG. 60, decoding based on a buffer structure, FIG. 36 to 40 Acquisition of bitstream and parameter information (syntax elements), the decoder of FIG. 42, the sub-bitstream classifier flowchart of FIG. 43, the decoding process of FIG. 45, decoding according to the context buffer management method of FIG. 47 to 48, acquisition of bitstream and parameter information (syntax elements) of FIG. 49 to 52, FIG. 55 to 59, and FIG. 82, ROI partial decoding of FIG. 64, partial decoding process of FIG. 67, decoding based on ROI bounding box adjustment of FIG. 77 to 78 and FIG. 80 to 81, decoding according to node distribution by layer group of FIG. 83 and sub-group unit node alignment of FIG. 84, partial decoding of FIG. 85 to 87, decoding based on layer group slicing of FIG. 88, decoding method of FIG. 90, etc.) receives a file including the bitstream of FIG. 36, and Point cloud data can be decoded based on parameter information included in the bitstream.

[0435] Referring to FIG. 36, each abbreviation signifies the following. Each abbreviation may be referred to by other terms within the scope of equivalent meaning. SPS: Sequence Parameter Set, GPS: Geometry Parameter Set, APS: Attribute Parameter Set, TPS: Tile Parameter Set, Geom: Geometry bitstream = geometry slice header + geometry slice data, Attr: Attribute bitstream = attribute brick header + attribute brick data

[0436] The embodiments may define information regarding separated slices in a parameter set and an SEI message. Additionally, the embodiments may define information regarding separated slices in a sequence parameter set (SPS), a geometry parameter set (GPS), an attribute parameter set (APS), a geometry slice header, and an attribute slice header; depending on the application or system, the scope and method of application may be used differently by defining them in corresponding locations or separate locations. That is, the signal may have different meanings depending on where it is transmitted; if defined in the SPS, it may be applied uniformly to the entire sequence, and if defined in the GPS, it may indicate that it is used for location restoration. If defined in the APS, it may indicate that it is applied to attribute restoration, and if defined in the TPS, it may indicate that the corresponding signaling is applied only to points within a tile. If transmitted at the slice level, it may indicate that the signaling is applied only to that slice.

[0437] In addition, depending on the application or system, the scope of application, the method of application, etc., may be used differently by defining them in a corresponding location or a separate location. Also, if the syntax elements of FIGS. 37 to 39, FIGS. 49 to 52, FIGS. 55 to 59, and FIG. 82 can be applied not only to the current point cloud data stream but also to multiple point cloud data streams, they may be transmitted through a higher-level concept parameter set, etc.

[0438] In the embodiments, the information is described as being defined independently of the encoding technique, but it can be defined in conjunction with the encoding method and can be defined in a TPS (Tile Parameter Set) to support regionally different scalability. Additionally, if the syntax elements of FIGS. 37 to 39, FIGS. 49 to 52, FIGS. 55 to 59, and FIG. 82 can be applied not only to the current point cloud data stream but also to multiple point cloud data streams, they can be transmitted through a higher-level parameter set, etc.

[0439] Alternatively, bitstreams can be selected at the system level by defining a NAL (Network abstract layer) unit and passing relevant information that allows selecting a layer, such as a layer_id.

[0440] FIG. 37 shows a set of sequence parameters according to the embodiments.

[0441] The sequence parameter set of Fig. 37 can be included in the bitstream of Fig. 36.

[0442] A layer_group_enabled_flag of 1 indicates that the geometry and / or attribute bitstreams of a frame or tile are contained in multiple slices that match a group of coding layers or a subgroup thereof. A layer_group_enabled_flag of 0 indicates that the geometry bitstreams of a frame or tile are contained in a single slice.

[0443] The layer_group_slice_order_type indicates the ordering type of the layer group slices. A layer_group_slice_order_type of 0 indicates the breadth-first search order of the layer group slices. A layer_group_slice_order_type of 1 indicates the depth-first search order of the layer group slices. A layer_group_slice_order_type of 2 indicates that no ordering type is specified.

[0444] FIG. 38 shows a dependent geometry data unit header according to embodiments, and FIG. 39 shows a dependent attribute data unit header according to embodiments.

[0445] The dependent geometry data unit header of FIG. 38 and the dependent attribute data unit header of FIG. 39 may be included in the bitstream of FIG. 36.

[0446] A context_reference_indication_flag of 1 indicates that the context state of the current dependent slice will be inherited by one or more subsequent dependent slices. A context_reference_indication_flag of 0 indicates that the context state of the current dependent slice will not be inherited by subsequent dependent slices.

[0447] Decoders can manage the context buffer using context_reference_indication_flag. If context_reference_indication_flag is 1, the context state of the current dependent slice is stored in the context buffer at the end of decoding. If context_reference_indication_flag is 0, the context state of the current dependent slice is not stored in the context buffer.

[0448] The number of subsequent data units (num_subsequent_data_units) represents the number of subsequent dependent data units that use the context state of the current data unit.

[0449] A subsequent_data_unit_list_present_flag of 1 indicates that a list of subsequent data units exists. A subsequent_data_unit_list_present_flag of 0 indicates that a list of subsequent data units does not exist.

[0450] The number of layer groups (number_of_layer_groups) represents the number of layer groups present in the list of subsequent data units.

[0451] The subsequent layer group index (subsequent_layer_group_id) represents the layer group index of the subsequent data unit.

[0452] The number of subgroups indicates the number of subgroups existing within one layer group of the list of subsequent data units.

[0453] The subsequent subgroup index (subsequent_subgroup_id) represents the subgroup index within the layer group of the subsequent data unit.

[0454] FIG. 40 shows a layer group structure inventory according to embodiments.

[0455] The layer-group structure inventory of Fig. 40 can be included in the bitstream of Fig. 36.

[0456] The sequence parameter set ID (lgsi_seq_parameter_set_id) represents the value of sps_seq_parameter_set_id. It is a bitstream conformance requirement that lgsi_seq_parameter_set_id be equal to 0.

[0457] The frame counter lsb bits (lgsi_frame_ctr_lsb_bits) represent the bit length of the lgsi_frame_ctr_lsb syntax element.

[0458] Frame counter lsb(lgsi_frame_ctr_lsb) represents the least significant bits (LSB) of lgsi_frame_ctr_lsb_bits of a FrameCtr for which the group structure inventory is valid. The layer group structure inventory remains valid until it is replaced by another layer group structure inventory.

[0459] The value obtained by adding 1 to the number of slices (lgsi_num_slice_ids_minus1) represents the number of slices existing in the layer group structure inventory.

[0460] The slice index (lgsi_slice_id[sid]) represents the slice ID of the sid-th slice within the layer group structure inventory. It is a bitstream conformance requirement that all lgsi_slice_id values ​​must be unique within the layer group structure inventory.

[0461] The value obtained by adding 1 to the number of layer groups (lgsi_num_layer_groups_minus1) represents the number of layer groups.

[0462] The layer group ID (lgsi_layer_group_id) represents the indicator of the layer group. The range of lgsi_layer_group_id must be from 0 to lgsi_num_layer_groups_minus1.

[0463] The value obtained by adding 1 to the number of layers (lgsi_num_layers_minus1[sid]) represents the number of coded layers in the i-th layer group within the sid-th slice. The total number of coded layers required to decode the n-th layer group is equal to the sum of lgsi_num_layers_minus1[sid][i] + 1 when i is from 0 to n.

[0464] The value obtained by adding 1 to the number of subgroups (lgsi_num_subgroups_minus1[sid]) represents the number of subgroups in the i-th layer group within the sid-th slice.

[0465] The subgroup ID (lgsi_subgroup_id) represents the subgroup identifier. The range of lgsi_subgroup_id must be from 0 to lgsi_num_subgroups_minus1.

[0466] The parent subgroup ID (lgsi_parent_subgroup_id) represents the indicator of a subgroup within the layer group indicated by lgsi_subgroup_id. The range of lgsi_parent_subgroup_id must be from 0 to gi_num_subgroups_minus1 within the layer group indicated by lgsi_subgroup_id.

[0467] The subgroup bounding box origin (lgsi_subgroup_bbox_origin) and subgroup bounding box size (lgsi_subgroup_bbox_size) indicate the bounding box of the current subgroup.

[0468] lgsi_subgroup_bbox_origin represents the origin of the subgroup bounding box of the subgroup indicated by lgsi_subgroup_id among the layer groups indicated by lgsi_layer_group_id.

[0469] lgsi_subgroup_bbox_size represents the size of the subgroup bounding box of the subgroup indicated by lgsi_subgroup_id among the layer groups indicated by lgsi_layer_group_id.

[0470] The origin bit (lgsi_origin_xyz) indicates the origin of all partitions. The value of lgsi_origin_xyz[ k ] must be equal to sps_bounding_box_offset[ k ].

[0471] The origin log scale (lgsi_origin_log2_scale) specifies the scaling factor for scaling the components of lgsi_origin_xyz. The value of lgsi_origin_log2_scale must be equal to sps_bounding_box_offset_log2_scale.

[0472] In the following, components of the transmitting end and the receiving end according to embodiments are described. Each component may correspond to a processor, software, or hardware, etc. Additionally, the following components may be combined with a PCC transmitting and receiving end structure and / or signaling information.

[0473] FIG. 41 shows an encoder according to embodiments.

[0474] FIG. 41 shows the detailed functional configuration of an encoder for encoding / transmitting PCC data according to embodiments.

[0475] Referring to Fig. 41, when point cloud data is input, the encoder can encode position information (geometry data: e.g., XYZ coordinates, phi-theta coordinates, etc.) and attribute information (attribute data: e.g., color, reflectance, intensity, grayscale, opacity, medium, material, glossiness, etc.). The compressed data is divided into units for transmission, and through a sub-bitstream generator, the necessary information at the bitstream level can be divided into units suitable for selection and packed according to layering structure information.

[0476] According to the embodiments, when different types of bitstreams are included in a single slice, the encoder may separate the generated bitstreams (AEC bitstream and / or DC bitstream) according to the purpose. Then, each slice or adjacent information may be included in a single slice according to layer group information. At this time, through a metadata generator, information such as bitstream type, bitstream offset, bitstream length, and bitstream direction may be transmitted along with layer group information, layer information included in the layer group, number of nodes, layer depth information, and number of nodes included in the subgroup, according to each slice ID.

[0477] The encoder of FIG. 41 is the transmitting device (10000) of FIG. 1, a point cloud video encoder (10002), a transmitter (10003), the acquisition-encoding-transmission (20000-20001-20002) of FIG. 2, the encoding process of FIG. 3, the transmitting device of FIG. 8, the device of FIG. 10, encoding based on the layer structure of FIG. 13, bitstream segments of FIG. 14 to 15, bitstream alignment methods of FIG. 16 to 17, geometry-attribute selection of FIG. 18 to 19, encoding based on slice configuration of FIG. 20, encoding based on layer group of FIG. 21 to 23, context references of FIG. 24 to 30, FIG. 33 to 34, FIG. 60, encoding based on buffer structure, bitstream and parameter information (syntax elements) of FIG. 36 to 40 It may correspond to generation, encoding process of FIG. 44, encoding according to the context buffer management method of FIG. 47 to 48, generation of bitstream and parameter information (syntax element) of FIG. 49 to 52, FIG. 55 to 59, FIG. 61 to 63, FIG. 82, encoding process of FIG. 75, encoding based on ROI bounding box control of FIG. 77 to 78, FIG. 80 to 81, encoding according to node distribution by layer group of FIG. 83 and subgroup unit node alignment of FIG. 84, partial encoding of FIG. 85 to 87, encoding based on layer group slicing of FIG. 88, encoding method of FIG. 89, etc.

[0478] FIG. 42 shows a decoder according to embodiments.

[0479] FIG. 42 shows the detailed functional configuration of a decoder for receiving / decoding PCC data according to embodiments.

[0480] Referring to FIG. 42, when a bitstream is input, the receiver can process the bitstream for position information and the bitstream for attribute information by separating them. At this time, the sub-bitstream classifier can pass the bitstream to an appropriate decoder based on the information in the bitstream header. In this process, the receiver may also select the required layer. Depending on the characteristics of the data, the classified bitstream can be restored into geometry data and attribute data in the geometry decoder and attribute decoder, respectively, and then converted into a format for final output in the renderer.

[0481] If different types of geometry bitstreams are included, each bitstream can be decoded separately through a bitstream splitter. Embodiments can process octree-coding-based arithmetic entropy-coded bitstreams and direct-coded bitstreams by distinguishing them in a geometry decoder. In this case, they can be separated based on information regarding bitstream type, bitstream offset, bitstream length, and bitstream direction.

[0482] For separated bitstreams, a process of concatenating bitstream segments of the same type may be included. This can be included as a process to process bitstreams separated by layer groups into a continuous bitstream, and the bitstreams can be sorted in order based on layer group information. Bitstreams capable of parallel processing can be processed in the decoder without a concatenation process.

[0483] The decoder of FIG. 42 includes the receiving device (10004) of FIG. 1, the receiver (10005), the point cloud video decoder (10006), the transmission-decoding-rendering (20002-20003-20004) of FIG. 2, the decoder of FIG. 7, the receiving device of FIG. 9, the device of FIG. 10, decoding based on the layer structure of FIG. 13, the bitstream segment of FIG. 14 to 15, the bitstream alignment method of FIG. 16 to 17, the geometry-attribute selection of FIG. 18 to 19, decoding based on the slice configuration of FIG. 20, decoding based on the layer group of FIG. 21 to 23, the context reference of FIG. 24 to 30, FIG. 33 to 34, FIG. 60, decoding based on the buffer structure, the bitstream and parameters of FIG. 36 to 40 It may correspond to the acquisition of information (syntax elements), the sub-bitstream classifier flowchart of FIG. 43, the decoding process of FIG. 45, decoding according to the context buffer management method of FIG. 47 to 48, the acquisition of bitstream and parameter information (syntax elements) of FIG. 49 to 52, FIG. 55 to 59, FIG. 82, ROI partial decoding of FIG. 64, the partial decoding process of FIG. 67, decoding based on ROI bounding box control of FIG. 77 to 78, FIG. 80 to 81, decoding according to node distribution by layer group of FIG. 83 and sub-group unit node alignment of FIG. 84, partial decoding of FIG. 85 to 87, decoding based on layer group slicing of FIG. 88, the decoding method of FIG. 90, etc.

[0484] FIG. 43 shows a sub-bitstream classifier flowchart according to embodiments.

[0485] When received data is input in slice units, the metadata parser transmits parameter set information such as SPS, GPS, APS, and TPS. Based on the transmitted information, it can be determined whether scalability is possible, and if scalability is possible, the slice structure for scalable transmission can be identified according to the flowchart of FIG. 43.

[0486] First, the geometry slice structure can be determined based on information transmitted via GPS, such as num_scalable_layers, scalable_layer_id, tree_depth_start, tree_depth_end, node_size, num_nodes, num_slices_in_scalable_layer, and slice_id. If aligned_slice_structure_enabled_flag = 1, for example, if the geometry is octree-based and the attributes are encoded based on scalable LoD or scalable RAHT, and geometry / attribute slice pairs generated through the same slice partitioning have the same number of nodes for the same octree layer, the attribute slice structure can be determined in the same way as the geometry slice structure.

[0487] In this case, the range of the geometry slice ID is determined according to the target scalable layer, the range of the attribute slice ID is determined through the slice_id_offset, and the geometry / attribute slice is selected according to the determined range.

[0488] If aligned_slice_structure_enabled_flag = 0, the attribute slice structure can be identified separately based on information such as num_scalable_layers, scalable_layer_id, tree_depth_start, tree_depth_end, node_size, num_nodes, num_slices_in_scalable_layer, and slice_id passed through APS, and the range of attribute slice IDs required for scalability purposes can be limited, and based on this, the slices required can be selected through each slice ID before reconstruction.

[0489] The geometry / attribute slice selected through the above-described process can be passed as input to the decoder.

[0490] In the embodiments described above, the decoding process according to the slice structure was explained based on scalable transmission or scalable selection of the receiver, but it is not limited thereto. The embodiments can also be used in non-scalable processes by selecting the entire slice and omitting the process of ranging geom / attr slice id when scalable_transmission_enabled_flag is 0. In this case, information about the preceding slice (e.g., slice information belonging to the upper layer or slice information specified through ref_slice_id) can be used through slice structure information transmitted via parameter sets such as SPS, GPS, APS, TPS, etc.

[0491] According to the embodiments, when different types of geometry bitstreams exist, all slices included within the range for different bitstreams can be selected during the slice selection process. If different types of bitstreams are included within a single slice, each bitstream can be separated based on offset and length information, and the separated bitstreams can be rearranged according to the layer group order for decoding.

[0492] FIG. 44 shows an encoder operation flowchart according to embodiments.

[0493] Referring to FIG. 44, when point cloud data is input into the encoder, the encoder can form a layer-group structure and obtain related parameters. Based on the layer-group structure (or by external input), the encoder can establish reference relationships between subgroups. The encoder can perform encoding based on the layer-group structure and the reference structure.

[0494] The encoder checks for each subgroup / slice whether it is used as a reference, and can set context_reference_indication_flag = 1 if it is used. If it is not used, it can set context_reference_indication_flag = 0. Whether it is used as a reference can be determined after encoding or can be obtained directly through the reference structure.

[0495] When used as a reference, the encoder can signal the number of times the slice / subgroup is used as a reference via num_subsequenct_data_units. Additionally, if a specific list of slices / subgroups used as references is provided, subsequent_subgroup_list_present_flag can be set to 1 and the list of subsequent data units can be passed as layer group index and subgroup index. The parameters required for decoding can be included in the data unit header, and the encoded compressed bitstream can be included in the data unit to generate the bitstream for each slice; this process can be performed for every data unit / slice / subgroup.

[0496] FIG. 45 shows a decoder operation flowchart according to embodiments.

[0497] Referring to FIG. 45, the decoder can prepare for decoding by analyzing the data unit header for each slice. At this time, the context state used for decoding can be retrieved from the context buffer through ref_layer_group_id and ref_subgroup_id. The decoder can perform decoding after initializing the decoder based on the context state.

[0498] The decoder can determine whether to save new context states generated during the decoding process based on the context_reference_indication_flag included in the data unit header. If the context state is used for decoding a subsequent slice / subgroup / data unit, context_reference_indication_flag is signaled as 1. In this case, the target number of subsequent data units can be set to num_subsequent_data_units for memory management of the context. Additionally, if subsequent_subgroup_list_present_flag = 1, the layer group indices and subgroup indices of the subsequent data units can be saved in the subsequent data unit list.

[0499] The decoder can update the context state counter and the list of used data units for the context state being used. In this case, if the target count and counter of subsequent data units of that context state are the same, or if the subsequent data unit list (list_given) and the updated list of used data units (lilst_updated) are the same, the memory allocated to that context state can be released from the context buffer.

[0500] The context buffer management method described above can be applied equally to encoders as well as decoders. In addition, it can be applied not only to slices based on layer group slicing but also to cases where context buffers are referenced within a regular frame or between slices between frames.

[0501] Figure 46 illustrates a context buffer management method of a conventional layer group slicing method.

[0502] In Fine Granularity Slicing, context inheritance between slices is used to mitigate encoding loss caused by discontinuities between adjacent nodes or encoding layers. However, as the number of slices or layer groups increases, the number of context states in memory increases. To help decoders manage context buffers, an indication for the future use of the current context by subsequent slices is signaled.

[0503] Referring to FIG. 46, when a bitstream within a slice is decoded and context_reference_indication_flag is enabled, the context state of the decoder output is stored in the context buffer as illustrated in FIG. 46(a). Referring to FIG. 46(b), when context inheritance is used, the context state of a subsequent slice may be initialized by one of the stored context states of the previous subgroups indicated by ref_layer_group_id and ref_subgroup_id. Referring to FIG. 46(c), the context state of a slice belonging to the last layer group is initialized by the stored context state of the parent slice. However, the output context state is not stored in the context buffer because context_reference_indication_flag is disabled.

[0504] By using the context_reference_indication_flag, the overall size of the context buffer can be reduced by selecting context states known to be used by subsequent slices. However, there is a limitation in that decoders cannot know the release timing of each stored context. Therefore, all context states must be stored in the buffer until the decoding of all subgroups is finished. To address this problem, embodiments may include a method for releasing context states.

[0505] Context buffer management

[0506] FIG. 47 illustrates a context buffer management method according to embodiments.

[0507] The embodiments may include a method for signaling the number of subgroups referencing the current subgroup so that decoders can know the timing to release stored context states.

[0508] FIG. 47 illustrates a context buffer release method using signaling according to embodiments. Compared to FIG. 46, the context buffer of FIG. 47 has two additional columns: a column for indicating the number of subsequent subgroups referencing the current subgroup, and a column for counting the number of subgroups that have already used the context state in subgroup decoding. When context_reference_indication_flag is enabled, num_subsequent_subgroups is signaled, and the number is stored along with the context state, as shown in FIG. 47(a). In FIG. 47(b), three context states are stored in the context buffer, which may be referenced N times or 1 time.

[0509] Since the context state of FGS 0 (0, 0) is referenced twice by FGS 1 (1, 0) and FGS 2 (1, 1), the counter for the context state (0, 0) has a value of 2. In FIG. 46(c), the context state of (1, 1) is released after being used by FGS N+2 (2, 1). The context states (0, 0) and (1, 0) can be released beforehand because there are no subsequent subgroups referencing them.

[0510] FIG. 48 illustrates a context buffer management method according to embodiments.

[0511] When a different ordering is used for Fine Granularity Slices (FGSs), the method described above can be effective in the same way. Referring to FIG. 49, the method described above can also be applied to the depth-first ordering case. For comparison with the breadth-first ordering case of FIG. 48, the names of each slice are identical, and the order of delivery is changed from FGS 0, FGS 1, FGS 2, FGS 3, ... to FGS 0, FGS 1, FGS N+1, FGS 2, ... as shown in FIG. 48(b). When the decoding of FGS 2 is complete, the output context state (1, 0) is stored in the context buffer, and the number of subsequent subgroups is stored. When the next slice FGS N+2 is decoded, the context state is initialized to the context state (1, 1), and the corresponding counter in the context buffer is incremented by 1. Since the number of counters and the stored value of the number of subsequent subgroups are the same, the context state (1, 1) can be released from this point onward. Referring to FIG. 48, the maximum number of context states in the context buffer is 2, which is the number of layer groups minus 1.

[0512] According to the embodiments, the size of the context buffer can be estimated in advance at the receiver. For example, assuming there are N layer groups and the size of a subgroup within the nth layer group is S[n], and considering the case of referencing a parent subgroup, the number of context states to be stored in memory can be estimated as shown in the formula below.

[0513]

[0514] As an extreme opposite case, consider the case where the initial slice / subgroup is referenced; in this case, the number of context states to be stored in memory can be estimated as shown in the formula below.

[0515] (number of subgroups in the root layer) = 1

[0516] Referencing the parent subgroup may be the case that uses the most context state, while referencing the first slice / subgroup may be the case that uses the least context state. Therefore, when using layer group slicing with different reference relationships, one may consider being between the maximum and minimum values.

[0517] When using the method described above according to the embodiments, space may be required to store fewer context states than when dynamically releasing memory. For example, when FGS generated by layer group slicing is generated / delivered based on breadth-first order and only parent subgroups are referenced, context states belonging to the parent layer group can be released when encoding for one layer group is finished. Therefore, the number of context states to be stored in memory can be estimated as shown in the following formula.

[0518]

[0519] If FGS generated by layer group slicing is generated / transmitted in depth first order, the context memory of the parent subgroup can be released when the encoding of the child subgroup is finished by the method described above according to the embodiments. In this case, only the context of the parent subgroup with remaining children needs to be stored, and since leaf layer groups are excluded, the number of context states to be stored in memory can be expected to be N-1.

[0520] If it is necessary to estimate the size of memory for storing the context buffer at the receiver, related information (slice encoding order type - depth-first order, breadth-first order, etc., number of layer groups, number of subgroups belonging to each layer group, context reference method - parent reference, root reference, etc.) can be transmitted, and the number of context states can be estimated as described above.

[0521] Based on the context memory management method described above, embodiments may include a method of signaling num_subsequent_data_units to a geometry data unit header, a dependent geometry data unit header, an attribute data unit header, and a dependent attribute data unit header.

[0522] FIG. 49 shows a geometry data unit header according to embodiments, FIG. 50 shows a dependent geometry data unit header according to embodiments, FIG. 51 shows an attribute data unit header according to embodiments, and FIG. 52 shows a dependent attribute data unit header according to embodiments.

[0523] The geometry data unit header of FIG. 49, the dependent geometry data unit header of FIG. 50, the attribute data unit header of FIG. 51, and the dependent attribute data unit header of FIG. 52 may be included in the bitstream of FIG. 36.

[0524] The number of subsequent data units (num_subsequent_data_units) specifies the number of subsequent dependent data units that reference the current data unit or dependent data unit.

[0525] FIG. 53 shows an experimental result table of width priority according to the embodiments, and FIG. 54 shows an experimental result table of depth priority according to the embodiments.

[0526] Referring to FIGS. 53 and 54, experiments according to the embodiments were performed for both breadth first order and depth first order of FGSs (fine granularity slices) by setting "depth1stSubgroupSearch" in the encoder configuration.

[0527] As shown in FIGS. 53 and 54, additional signaling does not affect coding performance in terms of coding gain and computational complexity.

[0528] When the Resident Set Size (RSS) of the decoder is estimated, the average RSS for breadth-first order is reduced to 94% for both lossless and lossy conditions. When depth-first order is used, there is an additional reduction in the RSS of the decoder, resulting in 90% for both lossless and lossy conditions.

[0529] In conclusion, the embodiments may include a context buffer management method by releasing no-longer-used context states from the context buffer. To this end, the number of subsequent subgroups referencing the current subgroup may be signaled, and experimental results show that the memory reduction is, on average, 10% for depth-first order.

[0530] Context memory control considering partial decoding

[0531] The context memory control method described in FIGS. 47 to 54 can efficiently manage the context buffer in a situation where the entire slice is decoded for a bitstream composed of multiple slices. However, in the case of partial decoding, which is one of the main use cases of layer group slicing (when only a portion of the slice is decoded when there is a region of interest or a resolution of interest), the reference count (num_subsequent_data_units) transmitted by the encoder may not be fully filled.

[0532] For example, considering the case of a layer group structure divided into three layer groups, the decoder may decide not to use the last layer group. In this case, if the number of references in the second layer group is known, there is no need to store the context in the context buffer to decode the last layer group.

[0533] To this end, embodiments may enable the decoder to release the context memory after a specified number of times by signaling the number of times the context is used in units of data units. In this case, the number of times the context is used for each data unit may have a relationship with the total number of times the context is used as shown in the formula below.

[0534]

[0535] subsequent layer-groups represents layer groups containing subgroups that reference the context of the current data unit, and num_sdu_per_layer_group may represent the number of times subgroups belonging to each layer group reference the context of the current subgroup.

[0536] The embodiments can early release context memory in partial decoding situations through the following method.

[0537] 1) Method of passing the entire context reference structure (e.g., SPS)

[0538] It can convey actual relationships that allow the decoder to identify the reference structure, such as information about the layer group structure and the reference subgroup ID / number of subsequent subgroups.

[0539] In the decoder, the context can be quickly released in the case of partial decoding based on the entire identified reference structure.

[0540] However, since the above method requires signaling the entire structure, the SPS can become bloated and inefficient.

[0541] 2) Method of directly passing the subsequent subgroup ID (Data Unit Header)

[0542] By passing the context reference structure in units of data units, the decoder checks whether a request comes from the corresponding subgroup ID in the buffer, and can quickly release the context when all of the contexts used in the subgroups used in partial decoding are used.

[0543] 3) A method of dividing and transmitting the number of subsequent subgroups (e.g., data unit header)

[0544] For the number of subsequent subgroups N, it can be passed by dividing N = N1 (number referenced in layer group 1) + N2 (number referenced in layer group 2).

[0545] In a receiver that does not decode layer group 2, if the corresponding context is used N1 times, the context can be quickly released.

[0546] In the case of full decoding, the context can be released after being used for N1+N2 times.

[0547] As described above, the embodiments may include a method of transmitting the number of times each data unit is referenced within each layer group and a subgroup ID. In cases where multiple child subgroups exist for a single parent subgroup, by adding bounding box information of the referenced subgroup, the number of context references in the case of partial decoding according to the region of interest (ROI) can be transmitted more accurately, thereby allowing the decoder to release the context memory at the correct time.

[0548] FIG. 55 shows a sequence parameter set according to embodiments, FIG. 56 shows a dependent geometry data unit header according to embodiments, FIG. 57 shows an attribute data unit header according to embodiments, and FIG. 58 shows a dependent attribute data unit header according to embodiments.

[0549] The sequence parameter set (SPS) of FIG. 55, the dependent geometry data unit header of FIG. 56, the attribute data unit header of FIG. 57, and the dependent attribute data unit header of FIG. 58 may be included in the bitstream of FIG. 36.

[0550] If layer_group_enabled_flag is 1, it indicates that the geometry bitstream of a slice is contained in multiple slices that match a group of coding layers or a subgroup thereof. If layer_group_enabled_flag is 0, it indicates that the geometry bitstream is contained in a single slice.

[0551] The value of (num_layer_groups_minus1) plus 1 represents the number of layer groups, where a layer group represents a group of consecutive tree layers that are part of a geometry coding tree structure. num_layer_groups_minus1 must be within the range of 0 to the number of coding tree layers.

[0552] The layer group ID (layer_group_id) represents the layer group indicator of the slice. The range of layer_group_id must be from 0 to num_layer_groups_minus1.

[0553] The value obtained by adding 1 to the number of layers (num_layers_minus1) represents the number of coding layers included in the i-th layer group. The total number of layer groups can be derived by adding all (num_layers_minus1[i] + 1) from i 0 to num_layer_groups_minus1.

[0554] If the subgroup_enabled_flag is 1, it indicates that the i-th layer group is divided into two or more subgroups, wherein the set of points within the layer group's subgroups is identical to the set of points within the layer group. If the subgroup_enabled_flag of the i-th layer group is 1, then when j is greater than or equal to i, the subgroup_enabled_flag of the j-th layer group must be 1. If the subgroup_enabled_flag is 0, it indicates that the current layer group is not subdivided into multiple subgroups and is contained in a single slice.

[0555] The value obtained by adding 1 to the subgroup bounding box origin bit length (subgroup_bbox_origin_bits_minus1) represents the bit length of the syntax element subgroup_bbox_origin. The value obtained by adding 1 to subgroup_bbox_size_bits_minus1 represents the bit length of the syntax element subgroup_bbox_size.

[0556] The value obtained by adding 1 to the number of subgroups (num_subgroups_minus1) represents the number of subgroups within the i-th layer group.

[0557] The number of subsequent data units (num_subsequent_data_units) represents the number of subsequent dependent data units that reference the i-th data unit or the i-th dependent data unit.

[0558] The subsequent_data_unit_id represents the index of a subsequent data unit that references a data unit of the i-th layer group and the j-th subgroup.

[0559] In fine granularity slicing, context inheritance between slices is used to mitigate encoding loss caused by discontinuities between adjacent nodes or encoding layers. However, as the number of slices or layer groups increases, the number of context states in memory increases.

[0560] FIG. 59 shows a geometry data unit header according to embodiments.

[0561] The geometry data unit header of FIG. 59 can be included in the bitstream of FIG. 36.

[0562] The number of subsequent data units (num_subsequent_data_units) indicates the number of subsequent dependent data units that reference the current data unit or dependent data unit.

[0563] A flag (num_sdu_per_layer_group_present_flag) of 1 indicates that there are subsequent data units within each layer group. If num_sdu_per_layer_group_present_flag is 0, there are no subsequent data units within each layer group.

[0564] A successor data unit presence flag (sdu_present_flag) of 1 indicates that the successor data unit of the current data unit exists in the i-th layer group. If sdu_present_flag is 0, the successor data unit of the current data unit does not exist in the i-th layer group.

[0565] The number of subsequent data units per layer group (num_sdu_per_layer_group) represents the number of subsequent dependent data units within the i-th layer group that reference the current data unit or dependent data unit.

[0566] The subsequent data unit ID (subsequent_data_unit_id) represents the index of the subsequent data unit that references the current data unit or a dependent data unit.

[0567] Referring to FIG. 48, to help decoders manage the context buffer, an indication of future use of the current context by subsequent slices is signaled. By using this, only context states known to be used by subsequent slices are stored, thereby reducing the overall size of the context buffer.

[0568] In addition, stored context states are released using count information of subgroups referencing the current subgroup. However, if a partial layer or partial region of the bitstream is decoded, subsequent subgroups outside the Region of Interest (ROI) cannot be counted, so the number of encoded subsequent subgroups cannot reach the signaled count. Consequently, the stored contexts may not be released.

[0569] FIG. 60 illustrates a partial decoding context buffer management method according to embodiments.

[0570] The embodiments may include a method for managing a list of subsequent subgroups that overlap with a region of interest (ROI) for each layer group to account for partial decoding cases.

[0571] Referring to FIG. 60, the context buffer has two lists (a list of subsequent subgroups that reference the current subgroup and a list of related encoded subgroups using the context state of the current subgroup). The first list can be provided by each data unit, while the second list can be updated by decoders.

[0572] When an ROI is given for partial region decoding, the regions of the subgroups in the list are compared with the ROI, and the subgroups with regions overlapping with the ROI are retained in the list. Additionally, when the number of skipped layer-groups is set for partial layer-group decoding, a list of subsequent subgroups within the non-skipped layer-groups is retained in the list.

[0573] In FIG. 60(a), the ROI is depicted as a red line over the FGSs having one skipped layer group. With this information, the FGS decoder will decode FGS 0 for layer group 0, FGS 1 and 2 for layer group 1, and nothing for layer group 2. Based on this, a list of selected subsequent subgroups (List A) is generated as a subset of the given list of subsequent subgroups. When FGS 1 is decoded, the context state is initialized by the stored context state of FGS 0 and updated in the list of encoded subsequent subgroups (List B) as shown in FIG. 60(b). Referring to FIG. 60(c), when FGS 2 is decoded, List B is updated to FGS 2 (1, 1), and since this is identical to List A, the context state (0, 0) can be released under the condition that the decoder will not decode subgroups outside the ROI.

[0574] Based on the context memory management method described above in FIG. 60, embodiments may include a method for signaling a subsequent subgroup index and bounding box information for each layer group.

[0575] FIG. 61 shows a set of sequence parameters according to the embodiments.

[0576] The sequence parameter set (SPS) of Fig. 61 may be included in the bitstream of Fig. 36.

[0577] A subsequent_subgroups_info_present_flag of 1 indicates that additional information for subsequent subgroups is delivered. A subsequent_subgroups_info_present_flag of 0 indicates that additional information for subsequent subgroups is not delivered.

[0578] FIG. 62 shows a geometry data unit header according to embodiments, and FIG. 63 shows a dependent geometry data unit header according to embodiments.

[0579] The geometry data unit header of FIG. 62 and the dependent geometry data unit header of FIG. 63 may be included in the bitstream of FIG. 36.

[0580] The number of subsequent subgroups (num_subsequent_subgroups) indicates the number of subsequent dependent data units that reference the current data unit or dependent data unit.

[0581] The subsequent subgroup ID (subsequent_subgroup_id) represents the subgroup index of a subsequent dependent data unit that references the current data unit or dependent data unit within the i-th layer group.

[0582] The subsequent_subgroup_bbox_origin represents the origin of the subgroup bounding box of the subsequent subgroup, which references the context state of the current subgroup.

[0583] The subsequent_subgroup_bbox_size indicates the size of the subgroup bounding box of the subsequent subgroup, which references the context state of the current subgroup.

[0584] FIG. 64 shows ROI partial decoding according to embodiments.

[0585] Referring to FIG. 64, embodiments can perform an early release of context memory without additional signaling when performing partial decoding. As an example of partial decoding for a region of interest (ROI), it can be assumed that skip layer-group is 1 and decoding is performed for the region of interest of the decoder.

[0586] In the case where context_reference_indication_flag is 1, or for all cases, the context buffer may store the context state for the subgroup index that matches the FGS being decoded. In the case of partial decoding, num_subsequent_subgroups may not be stored because decoding may finish before being referenced by the number specified in num_subsequent_subgroups. Instead, a list (a list of encoded subgroups associated with each context) may be created to store the indices of subgroups that reference the context of the current subgroup.

[0587] When a referenced subsequent FGS that references the context of a specific FGS is decoded, the index of the referenced subsequent FGS can be stored in a list corresponding to the specific FGS. FIG. 64 shows the case where it is stored as a pair of layer group index and subgroup index, such as (1, 0) or (1, 1), and when the index of the FGS is defined, a value of 1 or 2 corresponding to fgs_id can be stored.

[0588] FIG. 65 shows an ROI bounding box according to embodiments.

[0589] When a list corresponding to a specific FGS is updated for the context buffer array, the regions defined by the Region of Interest (ROI) (roiBboxMin, roiBboxMax) can be compared with the regions covered by the list. This is a step to verify whether the ROI is fully contained within the regions defined by the bounding boxes (bboxes) of the subgroups included in the list.

[0590] Referring to Fig. 65, the ROI can cover part of the ROI_bbox as shown on the left side of Fig. 65. When the list is updated and additional subgroups with bounding boxes as shown on the right side are decoded, the entire ROI can be covered by the subgroups within the list.

[0591] For a context state referenced across multiple layer groups, the area of ​​the sub-list corresponding to each layer group can be compared to see if it covers the area of ​​the ROI, and then it can be determined whether it is fully covered. In this case, the corresponding context state (e.g., context state (0,0)) can be released.

[0592] In this case, the context state can be released only when the context is no longer in use. That is, if additional partial decoding is performed on another region after progressive decoding or partial decoding, the context can be released after decoding all FGS referenced within the slice using the number of subsequent subgroups (num_subsequent_subgroups), rather than by comparing the ROI with the region of the list.

[0593] FIG. 66 shows an ROI bounding box according to embodiments.

[0594] However, due to the nature of point cloud data, there may be areas where no points exist. Since subgroups may not be transmitted for areas where no points exist, the ROI area and the list area may not completely overlap, as shown on the left side of Fig. 66.

[0595] In this case, it is possible to determine whether the ROI is covered by the list area only for the area where the actual point exists. An occupancy map can be used to determine whether each voxel is occupied based on the location information of the subgroup nodes of the subgroup where the context state is stored.

[0596] The right side of Fig. 66 shows an example comparing the occupancy map of subgroup (0, 0), the ROI bounding box (ROI_bbox), and the bounding box of subgroups in the list when the referenced context state is (0, 0).

[0597] When calculating the overlapping area (hatched area) between the ROI_bbox and the bounding boxes of the subgroups in the list, there are parts of the ROI_bbox that are not covered; however, if the occupancy map is considered simultaneously, it can be confirmed that the ROI_bbox covers all occupied nodes. In this case, assuming there is no subsequent decoding using that context state, the context state can be released.

[0598] The following cases can be considered for a method of releasing the stored context state to manage the context buffer through the method described above.

[0599] Case 1: Full decoding - when operating based on num_subsequent_data_units (sum of num_sdu_per_layer_group).

[0600] Case 2: Partial Depth - When releasing based on num_sdu_per_layer_group

[0601] Case 3: Partial Region - (a) When the list of subsequent subgroups is signaled, (b) When the decoder generates the list of subsequent subgroups without signaling

[0602] FIG. 67 shows a partial decoding flowchart according to embodiments.

[0603] For Case 3 above, the context state can be released for the case of partial decoding based on the flowchart of Fig. 67.

[0604] Referring to FIG. 67(a), embodiments may store the signaled list in the storage space listOfSubregionsForROI. When the referenced subgroup is decoded, the corresponding subgroup may be deleted from listOfSubregionsForROI. When the subgroup index in listOfSubregionsForROI is no longer available, the context state may be released.

[0605] Referring to FIG. 67(b), embodiments may initialize listOfSubregionsForROI as the current subgroup region. In this case, listOfSubregionsForROI may consist of a single region or may contain multiple sub-regions in the list. Whenever a subsequent subgroup is received, the bounding box of that subgroup may be cleared from listOfSubregionsForROI. If there are no regions in listOfSubregionsForROI, or if the regions remaining in listOfSubregionsForROI are non-occupied regions, the context state may be released.

[0606] Unlike the above, embodiments may include a method of checking whether the ROI is covered by accumulating the area as the list is updated, rather than a method of erasing the area.

[0607] Method for efficient data unit selection in partial encoding situations

[0608] FIG. 68 shows a pseudocode for a loosely overlapped condition according to the embodiments.

[0609] Referring to FIG. 68, the embodiments can selectively decode FGS based on a region of interest (ROI) by comparing the area of ​​a subgroup bounding box with the area of ​​the ROI and selecting and decoding FGS if there is an overlapping area.

[0610] FIG. 69 shows a pseudocode for a strictly overlapped condition according to the embodiments.

[0611] Referring to FIG. 69, the embodiments may choose not to decode if there are no actual nodes or points in the overlapping area, determining that it is not a data unit actually needed from the perspective of the ROI. That is, for each subgroup bounding box, decoding may be performed only when the area overlaps with the ROI and there are points / nodes in the overlapping area, determining that it is a data unit related to the ROI.

[0612] Based on the above, the decoder can selectively decode FGS for partial decoding situations as follows.

[0613] Partial density decoding

[0614] General

[0615] The decoder must generate a lower-density slice point cloud.

[0616] Low-density FGS point clouds are defined in terms of the following variables:

[0617] The variable SkippedLayerGroup represents the number of application-specific skipped layer-groups for partial decoding in the density direction. The value of SkippedLayerGroup must be within the range of 0 to num_layer_groups_minus1.

[0618] The variable MinNodeSizeLog2 represents the minimum occupancy tree node size specified by SkippedLayerGroup.

[0619] The array SubgroupNodePos[ layerGroupIdx ][ subgroupIdx ][ ptIdx ][ k ] represents the subgroup output nodes of layer group index layerGroupIdx and subgroup index subgroupIdx.

[0620] The array SubgroupNodeCnt[ layerGroupIdx ][ subgroupIdx ] represents the number of nodes in the subgroup output nodes of layer group index layerGroupIdx and subgroup index subgroupIdx.

[0621] FGS Selection

[0622] As shown in Table 3 below, if SkippedLayerGroup is greater than 0, layer groups with indices ranging from 0 to OutLayerGroup are selected for decoding. The maximum value of the partial decoding layer group index, OutLayerGroup, is set to the total number of layer groups minus SkippedLayerGroup.

[0623] OutLayerGroup := num_layer_groups_minus1 - SkippedLayerGroup

[0624] if (layer_group_id == 0)decode GDU or ADUelse if (layer_group_id OutLayerGroup)decode DGDU or DADUelseskip DGDU or DADU

[0625] As a result, as shown in Table 4 below, the geometry occupancy tree depth of partial decoding PartialDepth is inferred as the sum of the number of layers in each layer group with indices ranging from 0 to OutLayerGroup.

[0626] PartialDepth = 0for (i=0; I OutLayerGroup; i++)PartialDepth += num_layers_minus1[i] + 1

[0627] Geometry position compensation

[0628] As shown in Table 5 below, the maximum depth of the geometry occupancy tree when decoding all layer groups is inferred as the sum of the number of layers in each layer group with indices ranging from 0 to num_layer_groups_minus1.

[0629] TotalDepth = 0for (i=0; i < num_layer_groups_minus1; i++)TotalDepth += num_layers_minus1[i] + 1

[0630] MinNodeSizeLog2 is inferred to be the difference between occtreeMaxDepthMinus1 and PartialDepth.

[0631] MinNodeSizeLog2 = occtreeMaxDepthMinus1 + 1 - PartialDepth

[0632] As shown in Table 6 below, if MinNodeSizeLog2 is greater than 1, the points must be located in the center of their corresponding blocks.

[0633] for (ptIdx = 0; ptIdx < SubgroupNodeCnt[ layerGroupIdx ][ subgroupIdx ]; ptIdx++)for (k = 0; k < 3; k++)SubgroupNode[ layerGroupIdx ][ subgroupIdx ][ ptIdx ][ k ] |= (MinNodeSizeLog2 > 1) << (MinNodeSizeLog2 - 1)

[0634] Partial region decoding

[0635] General

[0636] The decoder must generate a point cloud of the partial region of the slice.

[0637] The partial region FGS point cloud can be specified in terms of the following variables:

[0638] The arrays RoiBBoxMin and RoiBBoxMax represent application-specific arrays that specify the region of interest (ROI) as the minimum and maximum positions of the bounding box.

[0639] The array SubgroupNodePos[ layerGroupIdx ][ subgroupIdx ] represents the subgroup output nodes of layer group index layerGroupIdx and subgroup index subgroupIdx.

[0640] The array SubgroupNodeCnt[ layerGroupIdx ][ subgroupIdx ] represents the number of nodes in the subgroup output nodes of layer group index layerGroupIdx and subgroup index subgroupIdx.

[0641] Selection of FGS

[0642] As shown in Table 7 below, if RoiBBoxMin and RoiBBoxMax exist, subgroup bounding boxes are selected to be decoded so that the subgroups whose bounding boxes overlap with the bounding box of the region of interest.

[0643] if (layerGroupIdx == 0)decode GDU or ADUelse if ((RoiBBoxMin[0] < SubgroupBBoxMax[layerGroupIdx][subgroupIdx][0] &&RoiBBoxMin[1] < SubgroupBBoxMax[layerGroupIdx][subgroupIdx][1] &&RoiBBoxMin[2] < SubgroupBBoxMax[layerGroupIdx][subgroupIdx][2]) &&(RoiBBoxMax[0] > SubgroupBBoxMin[layerGroupIdx][subgroupIdx][0] &&RoiBBoxMax[1] > SubgroupBBoxMin[layerGroupIdx][subgroupIdx][1] &&RoiBBoxMax[2] > SubgroupBBoxMin[layerGroupIdx][subgroupIdx][2]) && occupied)decode DGDU or DADUelseskip DGDU or DADU

[0644] Here, occupied represents the occupancy of the ROI overlapping area within the subgroup bounding box, which is estimated as shown in Table 8 below.

[0645] occupied = falsefor(i=0; i< SubgroupNodeCnt[layerGroupIdx][subgroupIdx]; i++) {if (RoiBBoxMin[0] <= SubgroupNodePos[layerGroupIdx][subgroupIdx][i][0] &&RoiBBoxMin[1] <= SubgroupNodePos[layerGroupIdx][subgroupIdx][i][1] &&RoiBBoxMin[2] <= SubgroupNodePos[layerGroupIdx][subgroupIdx][i][2] &&RoiBBoxMax[0] > SubgroupNodePos[layerGroupIdx][subgroupIdx][i][0] &&RoiBBoxMax[1] > SubgroupNodePos[layerGroupIdx][subgroupIdx][i][1] &&RoiBBoxMax[2] > SubgroupNodePos[layerGroupIdx][subgroupIdx][i][2]) {occupied = truebreak}}

[0646] Memory Management for Partial Coding - Point / Node Based Early Release

[0647] FIGS. 70a and 70b show pseudocode for the process of releasing nodes and context memory based on whether the ROI and child group are referenced according to the embodiments.

[0648] Referring to FIGS. 70a and 70b, context buffer management (or stored node management) can be effectively performed based on the strictly overlapped condition of FIG. 69. Referring to FIGS. 70a and 70b, as shown in Table 9 below, if there are no nodes or points in the area overlapping with the ROI for the subgroup bounding box of the decoded FGS (e.g., a direct coding node with no children), it can be expected that the child subgroups of that subgroup will not be selected. In this case, since it can be assumed that the context state and nodes of that subgroup will no longer be used, they can be released immediately without being stored.

[0649] if (_roi_enabled_flag && !checkRoiHasPoint(groupIndex, subgroupIndex, _bboxMinVector[curArrayIdx], _bboxMaxVector[curArrayIdx], subgroupPointCloud)) {releaseNodes(curArrayIdx);releaseCtxForGeometry(curArrayIdx);}

[0650] Referring to FIG. 70a and FIG. 70b, as shown in Table 10, the nodes used in the process of decoding a child subgroup can be tracked, and if there are no more nodes to be used for decoding the child subgroup, the memory for storing the nodes of that subgroup can be released.

[0651] if (_numRamainingNodesForChildSubgroups[refArrayIdx4Parent] == ​​0)releaseNodes(refArrayIdx4Parent);

[0652] FIGS. 71a, FIGS. 71b, and FIGS. 71c show pseudocode for a method of integrally releasing attribute decoder resources according to embodiments.

[0653] Referring to FIGS. 71a, 71b, and 71c, in the case of attribute decoding, multiple attributes may exist for a single subgroup, and in such cases, the geometry node and attribute context can be released after all related attributes have been decoded. At this time, the variable _numAttrsForReleaseAttrSubgroups may indicate how many of the attributes for the current subgroup have finished processing. The flag allChildAttrsDecodedFlag indicates whether the configuration information of each attribute data is consistent with the geometry information for all child subgroups associated with the subgroup currently being processed.

[0654] FIGS. 72a, FIGS. 72b, and FIGS. 72c represent pseudocode for a process for releasing a stored context state according to the embodiments.

[0655] Referring to FIG. 72a, FIG. 72b, and FIG. 72c, embodiments may first consider the area where the bounding box of the context reference and the ROI overlap as one sub-region when the ROI area list for the context reference is empty, and store it in the ROI area list (_listOfSubregionsForRoi or _listOfSubregionsForRoi_attr).

[0656] The embodiments can delete a sub-region from the ROI area list if the bounding box of the decoded subgroup (curArrayIdx) matches the sub-region in the list.

[0657] The embodiments may, if there is an overlapping area between a sub-area in the list and the bounding box of a decoded subgroup (curArrayIdx), divide the sub-area with the overlapping area into sub-areas, remove the overlapping area, and include the remaining sub-areas in the list.

[0658] The embodiments may assume that if all regions in the list are erased, the context state for the corresponding subgroup is no longer used.

[0659] FIGS. 73a and FIGS. 73b show pseudocode for the process of releasing a stored parent node according to the embodiments.

[0660] Referring to FIG. 73a and FIG. 73b, embodiments may treat the overlapping area between the bounding box of the parent subgroup and the ROI as one sub-region when the area list for the parent node is empty, and store it in the ROI area list (_listOfSubregionsForRoi_parent or _listOfSubregionsForRoi_parent_attr).

[0661] The embodiments can delete a sub-region from the list if the bounding box (bboxMin, bboxMax) of the decoded subgroup matches the sub-region in the list.

[0662] If there is an overlapping area between the sub-area in the list and the bounding box (bboxMin, bboxMax) of the decoded sub-group, the sub-area with the overlapping area can be divided into sub-areas, the overlapping area can be removed, and the remaining sub-areas can be included in the list.

[0663] The embodiments can assume that when all regions in the list are erased, the corresponding parent node is no longer used.

[0664] FIGS. 74a and FIGS. 74b show pseudocode for the process of generating an ROI sub-region list according to the embodiments.

[0665] Referring to FIG. 74a and FIG. 74b, the embodiments, in generating a list of sub-regions for areas overlapping with the ROI, additionally consider whether there is a point / node in each area and may include it in the list only if there is at least one point / node.

[0666] FIG. 75 shows an encoder flowchart according to embodiments.

[0667] In the encoder, if there are no nodes within the output geometry FGS, the FGS is not written to the bitstream. In this case, since the value of numSubsequentSubgroups is determined before the decision to skip the FGS is made, a discrepancy may occur between that value and the actual number of subsequent subgroups.

[0668] Referring to FIG. 75, embodiments may include a method to solve this problem by modifying numSubsequentSubgroups when an empty FGS exists, and then creating data unit headers and data units.

[0669] FIG. 76 shows pseudocode for the process of recalculating empty FGS geometry according to embodiments.

[0670] In the embodiments, if empty FGS geometry exists, numSubsequentSubgroups is recalculated as shown in FIG. 76.

[0671] FIG. 77 illustrates a method for controlling the ROI bounding box according to embodiments.

[0672] In partial decoding of FGS-based G-PCC bitstreams, an FGS belonging to the ROI can be selected based on whether each FGS overlaps with the ROI. In this case, when examining the region where the bounding boxes of the ROI and the FGS overlap (ROI overlapped region), a determination can be made based on whether a node of the parent subgroup exists within the ROI overlapped region.

[0673] In this case, a discrepancy may arise between the geometry resolution (or unit geometry node size) used to set the ROI and the actual resolution of the parent node. As shown on the left side of Fig. 77, this can lead to the child node being mistakenly identified as not being included in the ROI even though it actually is. Consequently, a problem may occur where FGS data containing the child node is skipped by determining that there are no nodes within the ROI overlap area with the parent subgroup bounding box (i.e., a problem where points within the ROI cannot be decoded).

[0674] To solve the above problem, embodiments may include a method of matching the resolution of the ROI range to the actual resolution of the parent node as shown on the right side of FIG. 77.

[0675] FIG. 78 illustrates a method for controlling the ROI bounding box according to embodiments.

[0676] The size of the ROI bounding box can vary depending on the node size of the layer group being considered. Referring to Fig. 77, when the parent subgroup is layer group 0, the size can change to match the voxel size of the parent node by performing a right shift followed by a left shift according to nodeSizeLog2. Referring to Fig. 78, when the parent subgroup is in layer group 1, the size of the ROI bounding box can change more finely than in Fig. 77 because nodeSizeLog2 is 2.

[0677] FGS Selection

[0678] FIGS. 79a, FIGS. 79b, FIGS. 79c, and FIGS. 79d represent pseudocode for a process of determining whether a point exists within an ROI according to embodiments and determining whether to perform decoding or release memory based thereon.

[0679] Referring to FIGS. 79a, 79b, 79c, and 79d, when RoiBBoxMin and RoiBBoxMax exist, the subgroup bounding box overlaps with the bounding box of the region of interest (ROI), and the subgroups that occupy the overlapped area can be selected as decoding targets.

[0680] Referring to FIGS. 79a, 79b, 79c, and 79d, as shown in Table 11 below, the partial depth (PrtDepth) is initialized to 0 and can be calculated by accumulating num_layers_minus1[i] + 1 for each layer group from index 0 to PrtLayerGroupIdx.

[0681] PrtDepth = 0 for (i=0; i <= PrtLayerGroupIdx; i++) PrtDepth += num_layers_minus1[i] + 1

[0682] Referring to FIGS. 79a, 79b, 79c, and 79d, as shown in Table 12 below, the partial node size log value (PrtNodeSizeLog2) can be inferred as the value obtained by subtracting PrtDepth from occtreeMaxDepthMinus1 + 1. Subsequently, the minimum and maximum coordinates of the ROI can be converted into coordinates (AdjustedRoiMin, AdjustedRoiMax) adjusted to the current node size.

[0683] PrtNodeSizeLog2 = occtreeMaxDepthMinus1 + 1 - PrtDepthfor(k = 0; k < 3; k++) {AdjustedRoiMin[k] = (RoiBBoxMin[k] >> PrtNodeSizeLog2) << PrtNodeSizeLog2AdjustedRoiMin[k] |= (PrtNodeSizeLog2 > 1) << PrtNodeSizeLog2 - 1AdjustedRoiMax[k] = (RoiBBoxMax[k] >> PrtNodeSizeLog2) << PrtNodeSizeLog2AdjustedRoiMax[k] |= (PrtNodeSizeLog2 > 1) << PrtNodeSizeLog2 - 1}

[0684] Referring to FIGS. 79a, 79b, 79c, and 79d, as shown in Table 13 below, the occupied status is set to an initial value of false. Each point position (pos) within the subgroup is adjusted by shifting it by PrtNodeSizeLog2, and it can be checked whether the adjusted pos exists between AdjustedRoiMin and AdjustedRoiMax.

[0685] occupied = falsefor(i=0; i< SubgroupNodeCnt[PrtLayerGroupIdx][PrtSubgroupIdx]; i++) {for(k = 0; k < 3; k++)pos[k] = SubgroupNodePos[ PrtLayerGroupIdx ][ PrtSubgroupIdx ][i][k] << PrtNodeSizeLog2pos[k] |= (PrtNodeSizeLog2 > 1) << PrtNodeSizeLog2 - 1if (AdjustedRoiMin[0] <= pos[0] && AdjustedRoiMax[0] > pos[0]AdjustedRoiMin[1] <= pos[1] && AdjustedRoiMax[1] > pos[1]AdjustedRoiMin[2] <= pos[2] && AdjustedRoiMax[2] > pos[2]) {occupied = truebreak}}

[0686] Referring to FIGS. 79a, 79b, 79c, and 79d, as shown in Table 14 below, the embodiments can determine whether to perform decoding based on the result of the judgment on possession.

[0687] if (layerGroupIdx == 0)decode GDU or ADUelse if (RoiBBoxMin < SubgroupBBoxMax[layerGroupIdx][subgroupIdx] &&RoiBBoxMax > SubgroupBBoxMin[layerGroupIdx][subgroupIdx] && occupied)decode DGDU or DADUelseskip DGDU or DADU

[0688] Here, num_layers_minus1 + 1 represents the number of partial occupancy tree depths for each layer group. occtreeMaxDepthMinus1 represents the depth of the encoded full occupancy tree. RoiBBoxMin and RoiBBoxMax represent the minimum and maximum values ​​indicating the range of the ROI. SubgroupNodePos represents the position information of the nodes belonging to the subgroup.

[0689] The method for determining an ROI overlap region according to the embodiments can be used for context memory release or parent node release. In this case, the decoded current subgroup can be considered as the parent subgroup for the child subgroup. Therefore, if there is no node within the ROI overlap region for the current subgroup, it can be inferred that the child subgroup is not decoded, and it can be assumed that the current subgroup no longer needs to be used.

[0690] Referring to Figs. 79a, 79b, 79c, and 79d, and referring to Table 15 below, ROI coordinates can be adjusted based on the current depth (CurDepth) and the current node size log value (CurNodeSizeLog2), and whether it is occupied can be determined.

[0691] CurDepth = 0for (i=0; i <= CurLayerGroupIdx; i++)CurDepth += num_layers_minus1[i] + 1CurNodeSizeLog2 = occtreeMaxDepthMinus1 + 1 - CurDepthfor(k = 0; k < 3; k++) { AdjustedRoiMin[k] = (RoiBBoxMin[k] >> CurNodeSizeLog2) << CurNodeSizeLog2AdjustedRoiMin[k] |= (CurNodeSizeLog2 > 1) << CurNodeSizeLog2 - 1AdjustedRoiMax[k] = (RoiBBoxMax[k] >> CurNodeSizeLog2) << CurNodeSizeLog2AdjustedRoiMax[k] |= (CurNodeSizeLog2 > 1) << CurNodeSizeLog2 - 1} occupied = falsefor(i=0; i< SubgroupNodeCnt[CurLayerGroupIdx][CurSubgroupIdx]; i++) { for(k = 0; k < 3; k++)pos[k] = SubgroupNodePos[ CurLayerGroupIdx ][ CurSubgroupIdx ][i][k] << CurNodeSizeLog2pos[k] |= (CurNodeSizeLog2 > 1) << CurNodeSizeLog2 - 1if (AdjustedRoiMin[0] <= pos[0] && AdjustedRoiMax[0] > pos[0]AdjustedRoiMin[1] <= pos[1] && AdjustedRoiMax[1] > pos[1]AdjustedRoiMin[2] <= pos[2] && AdjustedRoiMax[2] > pos[2]) { occupied = truebreak}}

[0692] Fig. 80 illustrates a method of ROI control box adjustment according to embodiments.

[0693] In partial decoding of FGS-based G-PCC bitstreams, an FGS belonging to the ROI can be selected based on whether each FGS overlaps with the ROI. In this case, when examining the region where the bounding boxes of the ROI and the FGS overlap (ROI overlapped region), a determination can be made based on whether a node of the parent subgroup exists within the ROI overlapped region.

[0694] In this case, a discrepancy may arise between the geometry resolution (or unit geometry node size) used to set the ROI and the actual resolution of the parent node. As shown on the left side of Fig. 80, this can lead to the child node being mistakenly identified as not being included in the ROI even though it actually is. Consequently, a problem may occur where FGS data containing the child node is skipped by determining that there are no nodes within the ROI overlap area with the parent subgroup bounding box (i.e., a problem where points within the ROI cannot be decoded).

[0695] To solve the above problem, embodiments may include a method of matching the resolution of the ROI range to the actual resolution of the parent node as shown on the right side of FIG. 80.

[0696] FIG. 81 illustrates a method for controlling an ROI bounding box according to embodiments.

[0697] The size of the ROI bounding box can vary depending on the node size of the layer group being considered. Referring to FIG. 80, when the parent subgroup is layer group 0, the size can change to match the voxel size of the parent node by performing a right shift followed by a left shift according to nodeSizeLog2. Referring to FIG. 81, when the parent subgroup is in layer group 1, the size of the ROI bounding box can change more finely than in FIG. 80 because nodeSizeLog2 is 2.

[0698] FGS Selection

[0699] If RoiBBoxMin and RoiBBoxMax exist, the subgroup bounding boxes overlap with the bounding boxes of the region of interest (ROI), and the subgroups that occupy the overlapped area can be selected as decoding targets.

[0700] As shown in Table 16 below, the partial depth (PrtDepth) and partial node size log value (PrtNodeSizeLog2) can be calculated based on the current partial layer group index (PrtLayerGroupIdx). Then, RoiBBoxMin and RoiBBoxMax can be converted into adjusted coordinates (AdjustedRoiMin, AdjustedRoiMax) using PrtNodeSizeLog2.

[0701] PrtDepth = 0for (i=0; i <= PrtLayerGroupIdx; i++)PrtDepth += num_layers_minus1[i] + 1PrtNodeSizeLog2 = occtreeMaxDepthMinus1 + 1 - PrtDepthfor(k = 0; k < 3; k++) {AdjustedRoiMin[k] = (RoiBBoxMin[k] >> PrtNodeSizeLog2) << PrtNodeSizeLog2AdjustedRoiMax[k] = ((RoiBBoxMax[k] >> PrtNodeSizeLog2) + 1) << PrtNodeSizeLog2}

[0702] As shown in Table 17 below, you can iterate through the node positions (SubgroupNodePos) within the subgroup and shift and adjust each node's position (pos) by PrtNodeSizeLog2 to check if it is included within the AdjustedRoi area.

[0703] occupied = falsefor(i=0; i< SubgroupNodeCnt[PrtLayerGroupIdx][PrtSubgroupIdx]; i++) {for(k = 0; k < 3; k++) {pos[k] = SubgroupNodePos[ PrtLayerGroupIdx ][ PrtSubgroupIdx ][i][k] << PrtNodeSizeLog2pos[k] |= (PrtNodeSizeLog2 > 1) << PrtNodeSizeLog2 - 1}if (AdjustedRoiMin[0] <= pos[0] && AdjustedRoiMax[0] > pos[0]&& AdjustedRoiMin[1] <= pos[1] && AdjustedRoiMax[1] > pos[1]&& AdjustedRoiMin[2] <= pos[2] && AdjustedRoiMax[2] > pos[2]) {occupied = truebreak}}

[0704] As shown in Table 18 below, whether to perform decoding for each layer group can be determined based on whether it is occupied and whether the bounding box overlaps.

[0705] if (layerGroupIdx == 0)decode GDU or ADUelse if (RoiBBoxMin < SubgroupBBoxMax[layerGroupIdx][subgroupIdx] &&RoiBBoxMax > SubgroupBBoxMin[layerGroupIdx][subgroupIdx] && occupied)decode DGDU or DADUelseskip DGDU or DADU

[0706] At this time, SubgroupBBoxMax and SubgroupBBoxMin can also be adjusted for the maximum depth (or partial node size) of the corresponding layer group, just like ROI.

[0707] SubgroupBBoxMin[ layerGroupIdx ][ subgroupIdx ] and SubgroupBBoxMax[ layerGroupIdx ][ subgroupIdx ] are aliases for the minimum and maximum point positions of the bounding boxes of the subgroups identified by layerGroupIdx and subgroupIdx, respectively.

[0708] These can be adjusted through bit shift operations using the partial node size log value (PrtNodeSizeLog2), as shown in Table 19 below.

[0709] SubgroupBBoxMin[layerGroupIdx][subgroupIdx][k] = (SubgroupBBoxMin[layerGroupIdx][subgroupIdx][k] >> PrtNodeSizeLog2) << PrtNodeSizeLog2SubgroupBBoxMax[layerGroupIdx][subgroupIdx][k] = ((SubgroupBBoxMax[layerGroupIdx][subgroupIdx][k]) >> PrtNodeSizeLog2) << PrtNodeSizeLog2

[0710] Meanwhile, the initial value of the subgroup bounding box can be defined using the subgroup origin (subgroup_bbox_origin_xyz) and size (subgroup_bbox_size) as shown in Table 20 below.

[0711] SubgroupBBoxMin[layerGroupIdx][subgroupIdx][k] := subgroup_bbox_origin_xyz[k]SubgroupBBoxMax[layerGroupIdx][subgroupIdx][k] := subgroup_bbox_origin_xyz[k] + subgroup_bbox_size[k]

[0712] FIG. 82 shows a dependent geometry data unit header according to embodiments.

[0713] The value obtained by adding 1 to the number of layer groups (num_layer_groups_minus1) represents the number of layer groups, where a layer group represents a group of consecutive tree levels within an occupancy tree. num_layer_groups_minus1 must be within the range of 0 to the number of coding tree layers.

[0714] The value obtained by adding 1 to the number of layers (num_layers_minus1[ i ]) represents the number of tree levels within the i-th layer group, where i represents the layer group index of the i-th layer group. The total number of layer groups should be derived by adding all (num_layers_minus1[ i ] + 1) when i ranges from 0 to num_layer_groups_minus1[ i ] (in context, num_layer_groups_minus1).

[0715] A subgroup_enabled[i] of 1 indicates that the i-th layer group contains two or more subgroups, where i represents the layer group index of the i-th layer group. A subgroup_enabled[i] of 0 indicates that the i-th layer group contains one subgroup.

[0716] If subgroup_enabled[ i ] is 1, the set of nodes (aggregation) in each subgroup within the i-th layer group must be the same as the set of nodes in that layer group.

[0717] If subgroup_enabled[ i ] is 1, then when j is greater than i, subgroup_enabled[ j ] must be 1.

[0718] FGS subgroup enable (fgs_subgroup_enabled) being 1 indicates that any layer group within the FGS contains two or more subgroups. fgs_subgroup_enabled being 0 indicates that all layer groups contain one subgroup.

[0719] fgs_subgroup_enabled = 0

[0720] for (i := 0; i <= num_layer_groups_minus1; i++)

[0721] fgs_subgroup_enabled |= subgroup_enabled[i]

[0722] The value obtained by adding 1 to the number of bits of the subgroup bounding box origin (subgroup_bbox_origin_bits_minus1) represents the bit length of the syntax element subgroup_bbox_origin.

[0723] The value of the number of bits for the subgroup bounding box size (subgroup_bbox_size_bits_minus1) plus 1 is the length in bits of the syntax element subgroup_bbox_size.

[0724] The root subgroup bounding box size (root_subgroup_bbox_size_log2[ k ]) represents the size of the bounding box of the root subgroup of the encoded occupancy tree. MaxVec(root_subgroup_bbox_size_log2) must be the number of encoded tree layers from the root to the leaf layer.

[0725] If fgs_layer_group_enabled is 1, the value of occtreeMaxDepthMinus1 is set as follows.

[0726] occtreeMaxDepthMinus1 = MaxVec(root_subgroup_bbox_size_log2) - 1

[0727] A subgroup_context_reference_indication_enabled of 1 indicates that the context state of the current data unit will be used to initialize one or more subsequent data units. A subgroup_context_reference_indication_enabled of 0 indicates that the context state of the current data unit will not be used to initialize subsequent data units. If not present, subgroup_context_reference_indication_enabled is inferred to be 1.

[0728] The number of missing layers in a subgroup should be the difference between the value of occtreeMaxDepthMinus1 plus 1 and the number of levels of the decoded occupancy tree. The number of missing layers in a subgroup should be used to derive the sampling direction in subgroup LoD generation (ISO / IEC 23090-38 clause E.6.3.3.1) or to compensate for the geometry location of nodes in intermediate layers (ISO / IEC 23090-38 E.8.2.3).

[0729] The number of subsequent subgroups (num_subsequent_subgroups[ i ]) represents the number of subsequent dependent data units belonging to the i-th layer group that refer to the context state of the current data unit. If none exist, the value of num_subsequent_subgroups[ i ] is inferred to be 0.

[0730] According to the embodiments, when subgroup_context_reference_indication_enabled is 1, the value of num_subsequent_subgroups[ i ] can be obtained for i in the range (inclusive) from layer_group_id + 1 to num_layer_groups_minus1.

[0731] A subgroup_planar_eligibility_by_density[ i ] value of 1 indicates that planar eligibility is enabled for the (i + startDepth)th depth of the current subgroup. A subgroup_planar_eligibility_by_density[ i ] value of 0 indicates that planar eligibility is disabled for the (i + startDepth)th depth of the current subgroup. If not present, subgroup_planar_eligibility_by_density[ i ] is inferred to be 0.

[0732] The embodiments may include the following three methods to match the geometry resolution (or unit geometry node size) for setting the ROI with the actual resolution of the parent node.

[0733] 1) The expressions SubgroupBBoxMin[ layerGroupIdx ][ subgroupIdx ] and SubgroupBBoxMax[ layerGroupIdx ][ subgroupIdx ] are aliases for the minimum and maximum point positions of the bounding box of a subgroup identified by the layer group index layerGroupIdx and the subgroup index subgroupIdx, respectively. As shown in Table 21 below, the embodiments can adjust the resolution by defining SubgroupBBoxMin[ layerGroupIdx ][ subgroupIdx ] and SubgroupBBoxMax[ layerGroupIdx ][ subgroupIdx ] using the partial node size log value (PrtNodeSizeLog2).

[0734] SubgroupBBoxMin[layerGroupIdx][subgroupIdx][k] := (subgroup_bbox_origin_xyz[k] >> PrtNodeSizeLog2) << PrtNodeSizeLog2SubgroupBBoxMax[layerGroupIdx][subgroupIdx][k] := ((subgroup_bbox_origin_xyz[k] + subgroup_bbox_size[k]) >> PrtNodeSizeLog2) << PrtNodeSizeLog2

[0735] 2) The embodiments may add conditions to the subgroup_bbox_origin_xyz[ k ] and subgroup_bbox_size_xyz[ k ] semantics as follows.

[0736] The subgroup bounding box origin (subgroup_bbox_origin_xyz[ k ]) specifies the minimum value of the k-th XYZ component of the subgroup bounding box of the subgroup indicated by the pair of layer_group_id and subgroup_id.

[0737] The subgroup bounding box size (subgroup_bbox_size_xyz[ k ]) specifies the k-th XYZ size component of the subgroup bounding box of the subgroup indicated by the pair of layer_group_id and subgroup_id.

[0738] The values ​​of subgroup_bbox_origin_xyz[ k ] and subgroup_bbox_size_xyz[ k ] must be greater than or equal to 0 and within a range less than a power of 2 of the maximum depth (curMaxDepth) of the current layer group. It is a bitstream constraint that the values ​​of subgroup_bbox_origin_xyz[ k ] and subgroup_bbox_size_xyz[ k ] must be the same as the values ​​calculated using the pseudocode in Table 22 below.

[0739] curMaxDepth = 0for (i = 0; i <= layerGroupIdx; i++)curMaxDepth += num_layers_minus1[i] + 1shiftBits = occtreeMaxDepthMinus1 + 1 - curMaxDepthfor (i = 0; i <3; i++) {shiftedSubgroupBboxOrigin[k] = (subgroup_bbox_origin_xyz[k] >> shiftBits) << shiftBitsshiftedSubgroupBboxSize[k] = (subgroup_bbox_size_xyz[k] >> shiftBits) << shiftBits}

[0740] 3) The embodiments represent subgroup_bbox_origin_xyz and subgroup_bbox_size_xyz as ranges for the maximum depth layer of each layer group, and can correct them to ranges for the maximum depth layer of the full occupancy tree at the time of decoding.

[0741] In this case, there is an advantage that the values ​​of subgroup_bbox_origin_xyz and subgroup_bbox_size_xyz can clearly indicate the boundaries of the subgroups, and additionally, there is the effect of reducing the number of bits used to pass subgroup_bbox_origin_xyz and subgroup_bbox_size_xyz.

[0742] FIG. 83 shows the node distribution by layer group according to the embodiments.

[0743] Referring to Fig. 83, for each layer group, the node size at the maximum occupancy depth can be represented as the minimum node size, and for each layer group, the nodeSizeLog2 value will be 3, 2, or 0.

[0744]

[0745] Referring to Fig. 83, the location of each node is indicated by a filled circle when it is occupied, and the occupied node of the parent subgroup can be divided into child nodes through decoding to find the detailed location.

[0746] FIG. 84 shows the alignment of subgroup unit nodes according to embodiments.

[0747] Referring to Fig. 83, if the origin and size of the subgroup bounding box in each layer group are represented in leaf node units, i.e., nodeSizeLog2 = 0, then, unless otherwise constrained, a not-aligned bounding box range may be set in the subgroup unit node.

[0748] In this case, as shown in Fig. 84, it may be possible to include not only points that should be included in the current subgroup but also children of neighboring subgroups. At this time, the dependency between the parent subgroup and the child subgroup is broken, which causes the FGS required in partial decoding situations to be skipped, and thus a decoding error may occur.

[0749] To prevent this, the embodiments may align the subgroup bounding box to each subgroup unit node. In particular, subgroup_bbox_origin_xyz and subgroup_bbox_size_xyz may be passed to each subgroup unit node, in which case each value may be passed as a concept of an index on the subgroup unit node grid.

[0750] As described above, in order to align the subgroup bounding box to the subgroup unit node grid for signaling, the semantics of the relevant syntax elements can be defined as follows.

[0751] The subgroup bounding box origin (subgroup_bbox_origin_xyz[ k ]) represents the position of the minimum k-th XYZ component of the subgroup bounding box of the subgroup indicated by the pair of layer group index layer_group_id and subgroup index subgroup_id. The value of subgroup_bbox_origin_xyz[ k ] must be expressed in units of minimum subgroup nodes. A minimum subgroup node may refer to the minimum node of the partial occupancy tree for the current subgroup.

[0752] The subgroup bounding box size XYZ (subgroup_bbox_size_xyz[ k ]) represents the k-th XYZ size component of the subgroup bounding box of the subgroup indicated by the pair of layer group index layer_group_id and subgroup index subgroup_id. The value of subgroup_bbox_size_xyz[ k ] must be expressed in units of the minimum subgroup node. The minimum subgroup node may refer to the minimum node of the partial occupancy tree for the current subgroup.

[0753] The values ​​of subgroup_bbox_origin_xyz[ k ] and subgroup_bbox_size_xyz[ k ] must be greater than or equal to 0 and within a range less than a power of 2 of the maximum depth of the current layer group.

[0754] The expressions SubgroupBBoxMin[ layerGroupIdx ][ subgroupIdx ] and SubgroupBBoxMax[ layerGroupIdx ][ subgroupIdx ] are aliases for the minimum and maximum point positions of the bounding boxes of the subgroups identified by layerGroupIdx and subgroupIdx, respectively. As shown in Table 23 below, to represent the subgroup bounding boxes with the same precision, the signaled origin and size can be left-shifted by minSubgroupNodeSizeLog2.

[0755] minSubgroupNodeSizeLog2 represents the minimum node size of the partial occupancy tree for the current subgroup.

[0756] SubgroupBBoxMin[layerGroupIdx][subgroupIdx][k] := subgroup_bbox_origin_xyz[k] << minSubgroupNodeSizeLog2SubgroupBBoxMax[layerGroupIdx][subgroupIdx][k] := (subgroup_bbox_origin_xyz[k] + subgroup_bbox_size[k]) << minSubgroupNodeSizeLog2

[0757] The minSubgroupNodeSizeLog2 value is set to the difference between occtreeMaxDepthMinus1 + 1 and the maximum occupancy tree depth of the current subgroup, as shown in Table 24 below.

[0758] minSubgroupNodeSizeLog2= occtreeMaxDepthMinus1 + 1 - subgroupMaxDepthsubgroupMaxDepth = 0for (i = 0; i <= layerGroupIdx; i++)subgroupMaxDepth += num_layers_minus1[i] + 1

[0759] Embodiments may include a method for signaling subgroup_bbox_origin_xyz and subgroup_bbox_size_xyz with fewer bits by dividing the origin and size by the minimum node size of the parent subgroup, taking into account that the boundary of the subgroup is expressed in units of the minimum node size of the parent subgroup.

[0760] The subgroup bounding box origin (subgroup_bbox_origin_xyz[ k ]) represents the position of the minimum k-th XYZ component of the subgroup bounding box of the subgroup indicated by the pair of layer group index layer_group_id and subgroup index subgroup_id.

[0761] According to the embodiments, the value of subgroup_bbox_origin_xyz[ k ] may be expressed in units of the minimum node size of the partial occupancy tree associated with the current subgroup. Specifically, the value of subgroup_bbox_origin_xyz[ k ] should be expressed in units of the minimum node size of the partial occupancy tree within the parent subgroup for the current subgroup.

[0762] The value of subgroup_bbox_origin_xyz[ k ] must be greater than or equal to 0 and less than the power of 2 (subgroup_bbox_origin_bits_minus1 + 1).

[0763] The subgroup bounding box size (subgroup_bbox_size_xyz[ k ]) represents the k-th XYZ size component of the subgroup bounding box of the subgroup indicated by the pair of layer group index layer_group_id and subgroup index subgroup_id.

[0764] According to the embodiments, the value of subgroup_bbox_size_xyz[ k ] may be expressed in units of the minimum node size of the partial occupancy tree associated with the current subgroup. Specifically, the value of subgroup_bbox_size_xyz[ k ] should be expressed in units of the minimum node size of the partial occupancy tree within the parent subgroup for the current subgroup.

[0765] The value of subgroup_bbox_size_xyz[ k ] must be greater than 0 and less than the power of 2 (subgroup_bbox_size_bits_minus1 + 1).

[0766] FGS geometry decoding process

[0767] In the decoding process, embodiments may include a method of representing all bounding boxes on a leaf node grid. Thus, when the minimum and maximum point locations of the subgroup bounding box are derived, the received signals subgroup_bbox_origin_xyz and subgroup_bbox_size_xyz are multiplied by the minimum node size of the parent subgroup.

[0768] The expressions SubgroupBBoxMin[ layerGroupIdx ][ subgroupIdx ] and SubgroupBBoxMax[ layerGroupIdx ][ subgroupIdx ] are aliases for the minimum and maximum point locations of the bounding boxes of the subgroups identified by layerGroupIdx and subgroupIdx, respectively. As shown in Table 25 below, to represent the subgroup bounding boxes in units of the leaf node size of the occupancy tree, the signaled origin and size are left-shifted by minPrtSubgroupNodeSizeLog2. minPrtSubgroupNodeSizeLog2 represents the minimum node size of the partial occupancy tree associated with the subgroup. Specifically, minPrtSubgroupNodeSizeLog2 represents the minimum node size of the partial occupancy tree of the parent subgroup associated with the subgroup.

[0769] SubgroupBBoxMin[layerGroupIdx][subgroupIdx][k] := subgroup_bbox_origin_xyz[k] << minPrtSubgroupNodeSizeLog2SubgroupBBoxMax[layerGroupIdx][subgroupIdx][k] := (subgroup_bbox_origin_xyz[k] + subgroup_bbox_size[k]) << minPrtSubgroupNodeSizeLog2

[0770] Here, minPrtSubgroupNodeSizeLog2 is defined as follows.

[0771] minPrtSubgroupNodeSizeLog2 = occtreeMaxDepthMinus1 + 1 - startDepth

[0772] startDepth can represent the cumulative sum of the number of occupation tree levels from the root layer group (wayer_group_id 0) up to the parent layer group of the layer group to which the current subgroup belongs.

[0773] The embodiments include a method for dividing and transmitting compressed data according to a certain standard for point cloud data. When using layered coding, the embodiments can divide and transmit compressed data according to the layer, in which case the storage and transmission efficiency of the transmitting end is increased.

[0774] Fig. 85 shows conventional partial encoding / decoding.

[0775] Referring to Fig. 85, in a PCC-based service, the compression rate or the number of data can be adjusted and sent depending on the receiver performance or transmission environment. However, if point cloud data is bundled into a single slice unit as in the conventional method, when the receiver performance or transmission environment changes, it is necessary to 1) convert the bitstream suitable for each environment in advance, store it separately, and select it when transmitting, or 2) perform a conversion process (transcoding) prior to transmission. In this case, if the number of receiver environments to be supported increases or the transmission environment changes frequently, storage space issues or delays caused by conversion may become a problem.

[0776] FIG. 86 shows a partial PCC bitstream according to embodiments.

[0777] As shown in Fig. 86, when compressed data is divided and transmitted according to layers, there is an advantage in that only the necessary parts of the pre-compressed data can be selectively transmitted at the bitstream stage without a separate conversion process. This is efficient in terms of storage space as only one storage space is required per stream, and efficient transmission is possible in terms of bandwidth as only the necessary layers are selectively transmitted before transmission (bitstream selector).

[0778] The embodiments include a method for dividing and transmitting compressed data according to a certain standard for point cloud data. When using layered coding, the embodiments can divide and transmit compressed data according to the layer, in which case the efficiency of the receiving end is increased.

[0779] Figure 87 shows conventional partial encoding / decoding.

[0780] Referring to Fig. 87, when receiving point cloud data consisting of layers, if information is transmitted that can restore the entire PCC data regardless of the receiver's performance, the receiver needs to perform a process (data selection or sub-sampling) to select only the data corresponding to the required layer after restoring the point cloud data through decoding. In this case, since the entire transmitted bitstream is decoded, a delay may occur in a receiver aiming for low latency, or decoding may not be possible depending on the receiver's performance.

[0781] In the embodiments, when the bitstream is divided into slices and transmitted, the receiver can selectively transmit the bitstream to the decoder based on the density of the point cloud data to be represented, depending on the decoder performance or the application. In this case, since the selection is made before decoding, decoder efficiency is increased, and there is an advantage of being able to support decoders of various performance levels.

[0782] Various elements according to the embodiments may be implemented by hardware, software, firmware, or a combination thereof. Various elements of the embodiments may be implemented on a single chip, such as a hardware circuit. Additionally, according to the embodiments, said elements may optionally be implemented on individual chips. According to the embodiments, at least one of the elements of the embodiments may be implemented in one or more processors that include instructions for performing operations according to the embodiments of the present disclosure.

[0783] Operations according to the embodiments may be performed by a transmitting device and / or a receiving device according to the embodiments. The transmitting and receiving device may include a transmitting and receiving unit for transmitting and receiving media data, a memory for storing instructions (program code, algorithm, flowchart and / or data) for a process according to the embodiments, and a processor for controlling the operations of the transmitting and receiving device.

[0784] The processor may be referred to as a controller, etc., and may correspond, for example, to hardware, software, and / or a combination thereof. The operation according to the embodiments described above may be performed by the processor. Additionally, the processor may be implemented as an encoder and / or decoder, etc., for performing the operation of the embodiments described above.

[0785] FIG. 88 shows a layer group slicing structure according to embodiments.

[0786] Multi-resolution ROIs can be supported by the scalability and spatial accessibility of hierarchical slicing.

[0787] Referring to FIG. 88, the encoder generates bitstream slices for octree layer-groups or spatial subgroups of each layer group. If requested, a slice matching the ROI of each resolution can be selected and transmitted.

[0788]

[0789] In this case, since the bitstream does not include details other than the requested ROI, the size of the total bitstream is smaller than that of tile-based approaches.

[0790] At the receiving end, the decoder can combine the slices to generate the following three outputs:

[0791] 1) High-level view output: Created from layer group slice 1.

[0792] 2) Mid-level view output: Generated from selected subgroups of layer group slice 1 and layer group 2.

[0793] 3) Low-level view, which is a fine detail output: is generated from layer group 1 and selected subgroups of layer groups 2 and 3.

[0794] Since the above outputs can be generated progressively, the receiver can provide a viewing experience such as zooming from a high-level view to a low-level view with a gradual increase in resolution.

[0795] FIG. 89 illustrates a encoding method according to embodiments.

[0796] The encoding method according to the embodiments may include the step of encoding geometry data of point cloud data (S8900); and / or the step of encoding attribute data of point cloud data (S8910).

[0797] The encoding method according to the embodiments may include the method described in the aforementioned FIGS. 1 to 88. Specifically, the step of encoding geometry data (S8900) may include the method described in FIGS. 1 to 4, FIG. 8, FIG. 10, FIGS. 13 to 38, FIGS. 40 to 41, FIG. 44, FIGS. 47 to 50, FIGS. 53 to 56, FIG. 59, FIGS. 61 to 63, FIGS. 65 to 66, FIGS. 68 to 84, FIGS. 86, FIGS. 88, etc. The step (S8910) of encoding attribute data of point cloud data may include the method described in FIGS. 1 to 3, FIGS. 5 to 6, FIGS. 8, FIG. 10, FIGS. 13 to 37, FIGS. 39 to 41, FIGS. 44, FIGS. 47 to 48, FIGS. 51 to 55, FIGS. 57 to 58, FIGS. 61, FIGS. 65 to 66, FIGS. 68 to 84, FIGS. 86, FIGS. 88, etc.

[0798] Referring together with FIG. 22, the encoding method according to the embodiments may include the geometric data being encoded based on a layer group structure related to tree levels of an occupancy tree, wherein the layer group structure includes a layer group identified by a layer group index, wherein the layer group includes subgroups, and wherein the geometric data includes a Fine Granularity Slice (FGS) geometry for a subgroup identified by a subgroup index in the subgroups for a partial occupancy tree, and the step of encoding the geometry may include the step of encoding the FGS geometry.

[0799] Referring together with FIG. 82, the encoding method according to the embodiments comprises that the FGS geometry includes a dependent geometry data unit header, and the dependent geometry data unit header includes information for the layer group index; information for the subgroup index; first information for the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes; and second information for the size component of the bounding box of the subgroup for each of the X, Y, and Z axes, wherein the value of the first information for each of the X, Y, and Z axes is expressed based on the minimum node size unit of the partial occupancy tree for the subgroup, and the value of the second information for each of the X, Y, and Z axes can be expressed based on the minimum node size unit of the partial occupancy tree for the subgroup.

[0800] The encoding method according to the embodiments includes the step of encoding the FGS geometry, which comprises: generating first information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes from a dependent geometry data unit header within the FGS geometry; and generating second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes from the dependent geometry data unit header, wherein the first variable regarding the minimum value of the bounding box of the subgroup is derived by shifting the value of the first information to the left by the value of the third variable regarding the minimum node size of the partial occupancy tree within the parent subgroup for the subgroup, and the second variable regarding the maximum value of the bounding box of the subgroup can be derived by shifting the value of the first information plus the value of the second information to the left by the value of the third variable.

[0801] The encoding method according to the embodiments may derive the value of the third variable based on the difference between the maximum depth of the occupancy tree for the layer group structure and the cumulative sum of the number of tree levels included from the root layer group to the layer group to which the subgroup belongs.

[0802] The encoding method is performed by an encoding device. The encoding device includes a memory; and at least one processor connected to the memory; and the at least one processor may be configured to: encode geometry data of point cloud data; and encode attribute data of point cloud data. The at least one processor may be configured to perform the operations of the above-described method.

[0803] The embodiments further include a computer-readable storage medium for storing a bitstream generated by the method according to FIG. 89.

[0804] The embodiments further include a method comprising the steps of: acquiring a bitstream for point cloud data; generating the bitstream based on the steps of encoding geometry data of the point cloud data and encoding attribute data of the point cloud data; and transmitting data including the bitstream.

[0805] FIG. 90 illustrates a decoding method according to embodiments.

[0806] The decoding method according to the embodiments may include the step of decoding geometry data of point cloud data within a bitstream (S9000); and / or the step of decoding attribute data of point cloud data (S9010).

[0807] The decoding method according to the embodiments may include the method described in the aforementioned FIGS. 1 to 88. Specifically, the step (S9000) of decoding geometry data of point cloud data within a bitstream may include the method described in FIGS. 1 to 4, FIG. 7, FIG. 9, FIG. 10, FIG. 13 to 38, FIG. 40, FIG. 42 to 43, FIG. 45, FIG. 47 to 50, FIG. 53 to 56, FIG. 59 to 74, FIG. 76 to 84, FIG. 86, FIG. 88, etc. The step (S9010) of decoding attribute data of point cloud data may include the method described in FIGS. 1 to 3, FIGS. 5 to 7, FIGS. 9 to 10, FIGS. 13 to 37, FIGS. 39 to 40, FIGS. 42 to 43, FIGS. 45, FIGS. 47 to 48, FIGS. 51 to 55, FIGS. 57 to 58, FIGS. 60 to 61, FIGS. 64 to 74, FIGS. 76 to 81, FIGS. 83 to 84, FIGS. 86, FIGS. 88, etc.

[0808] Referring together with FIG. 22, the decoding method according to the embodiments is such that the geometry data is decoded based on a layer group structure related to tree levels of an occupancy tree, the layer group structure includes a layer group identified by a layer group index, the layer group includes subgroups, the geometry data includes a Fine Granularity Slice (FGS) geometry for a subgroup identified by a subgroup index in the subgroups for a partial occupancy tree, and the step of decoding the geometry may include the step of decoding the FGS geometry.

[0809] Referring together with FIG. 82, the decoding method according to the embodiments includes a dependent geometry data unit header for the FGS geometry, wherein the dependent geometry data unit header includes information for the layer group index; information for the subgroup index; first information for the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes; and second information for the size component of the bounding box of the subgroup for each of the X, Y, and Z axes, wherein the value of the first information for each of the X, Y, and Z axes is expressed based on the minimum node size unit of the partial occupancy tree for the subgroup, and the value of the second information for each of the X, Y, and Z axes can be expressed based on the minimum node size unit of the partial occupancy tree for the subgroup.

[0810] The decoding method according to the embodiments includes the step of decoding the FGS geometry, wherein the step of obtaining first information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes from a dependent geometry data unit header within the FGS geometry; and the step of obtaining second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes from the dependent geometry data unit header, wherein the first variable regarding the minimum value of the bounding box of the subgroup is obtained by shifting the value of the first information to the left by the value of the third variable regarding the minimum node size of the partial occupancy tree for the subgroup, and the second variable regarding the maximum value of the bounding box of the subgroup can be obtained by shifting the value of the first information plus the value of the second information to the left by the value of the third variable.

[0811] The decoding method according to the embodiments includes the FGS geometry including a dependent geometry data unit header, wherein the dependent geometry data unit header includes information regarding the layer group index; information regarding the subgroup index; first information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes; and second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes, wherein the value of the first information for each of the X, Y, and Z axes is expressed based on the minimum node size unit of the partial occupancy tree associated with the subgroup, and the value of the second information for each of the X, Y, and Z axes can be expressed based on the minimum node size unit of the partial occupancy tree associated with the subgroup.

[0812] The decoding method according to the embodiments includes the step of decoding the FGS geometry, wherein the step of obtaining first information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes from a dependent geometry data unit header within the FGS geometry; and the step of obtaining second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes from the dependent geometry data unit header, wherein the first variable regarding the minimum value of the bounding box of the subgroup is obtained by shifting the value of the first information to the left by the value of the third variable regarding the minimum node size of the partial occupancy tree related to the subgroup, and the second variable regarding the maximum value of the bounding box of the subgroup can be obtained by shifting the value of the first information plus the value of the second information to the left by the value of the third variable.

[0813] The decoding method according to the embodiments can obtain the value of the third variable based on the difference between the maximum depth of the occupancy tree for the layer group structure and the cumulative sum of the number of tree levels included from the root layer group to the layer group to which the subgroup belongs.

[0814] The decoding method is performed by a decoding device. The decoding device includes a memory; and at least one processor connected to the memory; and the at least one processor may be configured to: decode geometry data of point cloud data within a bitstream; and decode attribute data of point cloud data. The at least one processor may be configured to perform the operations of the aforementioned method.

[0815] The embodiments have been described in terms of methods and / or devices, and the description of the methods and the description of the devices may be applied complementarily.

[0816] Although the drawings have been described separately for the convenience of explanation, it is also possible to design a new embodiment by combining the embodiments described in each drawing. Furthermore, designing a computer-readable recording medium containing a program for executing the previously described embodiments, as required by a person skilled in the art, falls within the scope of the claims of the embodiments. The apparatus and method according to the embodiments are not limited to the configuration and method of the embodiments described above; rather, the embodiments may be configured by selectively combining all or part of each embodiment to allow for various modifications. Although preferred embodiments have been illustrated and described, the embodiments are not limited to the specific embodiments described above. It is not only possible for a person skilled in the art to make various modifications without departing from the essence of the embodiments claimed in the claims, but such modifications should not be understood individually from the technical concept or perspective of the embodiments.

[0817] Various components of the device of the embodiments may be implemented by hardware, software, firmware, or a combination thereof. Various components of the embodiments may be implemented as a single chip, for example, a single hardware circuit. Depending on the embodiments, the components according to the embodiments may each be implemented as separate chips. Depending on the embodiments, at least one of the components of the device according to the embodiments may be composed of one or more processors capable of executing one or more programs, and one or more programs may include instructions for performing or executing any one or more of the operations / methods according to the embodiments. Executable instructions for performing the methods / operations of the device according to the embodiments may be stored in non-transient CRMs or other computer program products configured to be executed by one or more processors, or may be stored in transient CRMs or other computer program products configured to be executed by one or more processors. Additionally, memory according to the embodiments may be used as a concept that includes not only volatile memory (e.g., RAM, etc.) but also non-volatile memory, flash memory, PROM, etc. In addition, it may also include implementation in the form of carrier waves, such as transmission over the Internet. Furthermore, processor-readable recording media are distributed across networked computer systems, allowing processor-readable code to be stored and executed in a distributed manner.

[0818] In this document, “ / ” and “,” are interpreted as “and / or.” For example, “A / B” is interpreted as “A and / or B,” and “A, B” is interpreted as “A and / or B.” Additionally, “A / B / C” means “at least one of A, B and / or C.” Also, “A, B, C” means “at least one of A, B and / or C.” Additionally, in this document, “or” is interpreted as “and / or.” For example, “A or B” may mean 1) “A” alone, 2) “B” alone, or 3) “A and B.” In other words, “or” in this document may mean “additionally or alternatively.”

[0819] Terms such as "first," "second," etc., may be used to describe various components of the embodiments. However, the interpretation of the various components according to the embodiments should not be limited by these terms. These terms are merely used to distinguish one component from another. For example, the first user input signal may be referred to as the second user input signal. Similarly, the second user input signal may be referred to as the first user input signal. The use of these terms should be interpreted as not departing from the scope of the various embodiments. Although the first user input signal and the second user input signal are both user input signals, they do not imply the same user input signals unless clearly indicated in the context.

[0820] The terms used to describe the embodiments are intended for the purpose of describing specific embodiments and are not intended to limit the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless explicitly indicated in the context. Expressions of and / or are used to mean including all possible combinations between the terms. Expressions of include describe the presence of features, numbers, steps, elements, and / or components and do not imply the exclusion of additional features, numbers, steps, elements, and / or components. Conditional expressions such as "if" or "when" used to describe the embodiments are not limited to being optional. It is intended to be interpreted as "when a specific condition is satisfied," "when a related action is performed in response to a specific condition," or "when a related definition is interpreted."

[0821] Additionally, operations according to the embodiments described herein may be performed by a transmitting and receiving device including memory and / or a processor, depending on the embodiments. The memory may store programs for processing / controlling operations according to the embodiments, and the processor may control various operations described in this document. The processor may be referred to as a controller, etc. Operations in the embodiments may be performed by firmware, software, and / or a combination thereof, and the firmware, software, and / or a combination thereof may be stored in the processor or in memory.

[0822] Meanwhile, the operation according to the embodiments described above may be performed by a transmitting device and / or a receiving device according to the embodiments. The transmitting and receiving device may include a transmitting and receiving unit for transmitting and receiving media data, a memory for storing instructions (program code, algorithm, flowchart and / or data) for a process according to the embodiments, and a processor for controlling the operations of the transmitting and receiving devices.

[0823] The processor may be referred to as a controller, etc., and may correspond, for example, to hardware, software, and / or a combination thereof. The operation according to the embodiments described above may be performed by the processor. Additionally, the processor may be implemented as an encoder / decoder, etc., for the operation of the embodiments described above.

[0824]

[0825] As described above, the relevant details have been explained in the best mode for carrying out the embodiments.

[0826]

[0827] As described above, the embodiments may be applied wholly or partially to point cloud data transmission and reception devices and systems.

[0828] Those skilled in the art may make various changes or modifications to the embodiments within the scope of the embodiments.

[0829] The embodiments may include modifications / variations, and such modifications / variations do not exceed the scope of the claims and their equivalents.

Claims

A step of decoding geometry data of point cloud data within a bitstream; and A step of decoding attribute data of the above point cloud data; comprising Decryption method. In paragraph 1, The above geometry data is decoded based on a layer group structure related to the tree levels of an occupancy tree, and The above layer group structure includes a layer group identified by a layer group index, and The above layer group includes subgroups, and The above geometry data includes Fine Granularity Slice (FGS) geometry for a subgroup identified by a subgroup index in the subgroups for a partial occupancy tree, The step of decoding the geometry includes the step of decoding the FGS geometry. Decryption method. In paragraph 2, The above FGS geometry includes a dependent geometry data unit header, and The above dependent geometry data header is, Information regarding the above layer group index; Information regarding the above subgroup index; First information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes; and It includes second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes, and The value of the first information for each of the X, Y, and Z axes is expressed based on the minimum node size unit of the partial occupancy tree for the subgroup, and The value of the second information for each of the X, Y, and Z axes is expressed based on the minimum node size unit of the partial occupancy tree for the subgroup, Decryption method. In paragraph 2, The step of decoding the above FGS geometry is, A step of obtaining first information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes from a dependent geometry data unit header within the FGS geometry; and The method includes the step of obtaining second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes from the dependent geometry data header. The first variable for the minimum value of the bounding box of the above subgroup is obtained by shifting the value of the first information to the left by the value of the third variable for the minimum node size of the partial occupancy tree for the above subgroup, and The second variable for the maximum value of the bounding box of the above subgroup is obtained by shifting the value of the first information and the value of the second information to the left by the value of the third variable. Decryption method. In paragraph 2, The above FGS geometry includes a dependent geometry data unit header, and The above dependent geometry data header is, Information regarding the above layer group index; Information regarding the above subgroup index; First information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes; and It includes second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes, and The value of the first information for each of the X, Y, and Z axes is expressed based on the minimum node size unit of the partial occupancy tree related to the subgroup, and The value of the second information for each of the X, Y, and Z axes is expressed based on the minimum node size unit of the partial occupancy tree associated with the subgroup, Decryption method. In paragraph 2, The step of decoding the above FGS geometry is, A step of obtaining first information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes from a dependent geometry data unit header within the FGS geometry; and The method includes the step of obtaining second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes from the dependent geometry data header. The first variable for the minimum value of the bounding box of the above subgroup is obtained by shifting the value of the first information to the left by the value of the third variable for the minimum node size of the partial occupancy tree related to the above subgroup, and The second variable for the maximum value of the bounding box of the above subgroup is obtained by shifting the value of the first information and the value of the second information to the left by the value of the third variable. Decryption method. In paragraph 4, The value of the above third variable is obtained based on the difference between the maximum depth of the occupancy tree for the above layer group structure and the cumulative sum of the number of tree levels included from the root layer group to the layer group to which the above subgroup belongs, Decryption method. Memory; and At least one processor connected to the memory; comprising, wherein the at least one processor: Decoding geometry data of point cloud data within a bitstream; and Decoding attribute data of the above point cloud data; configured to do so, Decoding device. A step of encoding the geometry data of the point cloud data; and A step of encoding attribute data of the above point cloud data; comprising Encoding method. In Paragraph 9, The above geometry data is encoded based on a layer group structure related to the tree levels of an occupancy tree, and The above layer group structure includes a layer group identified by a layer group index, and The above layer group includes subgroups, and The above geometry data includes Fine Granularity Slice (FGS) geometry for a subgroup identified by a subgroup index in the subgroups for a partial occupancy tree, The step of encoding the geometry includes the step of encoding the FGS geometry. Encoding method. In Paragraph 10, The above FGS geometry includes a dependent geometry data unit header, and The above dependent geometry data header is, Information regarding the above layer group index; Information regarding the above subgroup index; First information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes; and It includes second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes, and The value of the first information for each of the X, Y, and Z axes is expressed based on the minimum node size unit of the partial occupancy tree for the subgroup, and The value of the second information for each of the X, Y, and Z axes is expressed based on the minimum node size unit of the partial occupancy tree for the subgroup, Encoding method. In Paragraph 11, The step of encoding the above FGS geometry is, A step of generating first information regarding the minimum component position of the bounding box of the subgroup for each of the X, Y, and Z axes from a dependent geometry data unit header within the FGS geometry; and The method includes the step of generating second information regarding the size component of the bounding box of the subgroup for each of the X, Y, and Z axes from the dependent geometry data header. The first variable for the minimum value of the bounding box of the above subgroup is derived by shifting the value of the first information to the left by the value of the third variable for the minimum node size of the partial occupancy tree within the parent subgroup for the above subgroup, and The second variable for the maximum value of the bounding box of the above subgroup is derived by shifting the value of the first information and the value of the second information to the left by the value of the third variable. Encoding method. In Paragraph 12, The value of the third variable is derived based on the difference between the maximum depth of the occupancy tree for the layer group structure and the cumulative sum of the number of tree levels included from the root layer group to the layer group to which the subgroup belongs, Encoding method. Memory; and At least one processor connected to the memory; comprising, wherein the at least one processor: Encoding the geometry data of the point cloud data; and Configured to encode the attribute data of the above point cloud data; Decoding device. A computer-readable storage medium for storing a bitstream generated by the method according to paragraph 9. Step of acquiring a bitstream for point cloud data, The bitstream is generated based on the step of encoding geometry data of the point cloud data; and the step of encoding attribute data of the point cloud data; and A method comprising the step of transmitting data including the bitstream above.

Images