Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Abstract

Disclosed are a method and apparatus for decoding point cloud data. The decoding method according to an embodiment includes the steps of receiving point cloud data and signaling information; and decoding the point cloud data, wherein the point cloud data includes geometry data and attribute data, and the point cloud data can be decoded based on layer groups and subgroup of an occupancy tree.

Description

Technical Field

[0001] The embodiments relate to methods and apparatus for processing point cloud content. Background Technology

[0002] Point cloud content is content represented by point clouds, which are collections of points belonging to a coordinate system representing three-dimensional space (or volume). Point cloud content can represent media configured in three dimensions and is used to provide various services such as Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), XR (Extended Reality), and autonomous driving services. However, tens of thousands to hundreds of thousands of point data points are required to represent point cloud content. Therefore, an efficient method for processing large amounts of point data is needed.

[0003] In other words, high throughput is required to send and receive point cloud data. Therefore, the computational operations are complex and time-consuming during the sending and receiving of point cloud data (in which encoding for compression and decoding for decompression are performed), due to the large volume of point cloud data. Summary of the Invention

[0004] Technical issues

[0005] The purpose of the implementation methods designed to solve the above problems is to provide apparatus and methods for efficiently sending / receiving point clouds.

[0006] Another objective of the implementation is to provide apparatus and methods for addressing the transmission latency and encoding / decoding complexity of point cloud data.

[0007] Another objective of the implementation is to provide an apparatus and method for efficiently transmitting and receiving point cloud data so as to enable partial decoding of the point cloud data.

[0008] The implementation methods are not limited to the above objectives, and the scope of the implementation methods can be extended to cover other objectives that can be inferred by those skilled in the art based on the entire disclosure.

[0009] Technical solution

[0010] To achieve these objectives and other advantages and in accordance with the purposes of this disclosure, as implemented and broadly described herein, the decoding method may include receiving point cloud data and signaling information, and decoding the point cloud data, wherein the point cloud data may include geometric data and attribute data.

[0011] According to the implementation method, point cloud data can be decoded based on layers and subgroups of the occupancy tree. Each layer can be a group of consecutive tree levels of the occupancy tree, and each subgroup can be a spatial subset of the layer group.

[0012] According to the implementation, geometric data can be included in multiple geometric data units, and attribute data can be included in multiple attribute data units. Multiple geometric data units and multiple attribute data units can be packaged into subgroup-based coding units. Each coding unit can include both geometric data units and attribute data units from the same subgroup.

[0013] According to an implementation, the signaling information may include identification information for identifying each of the coding units. The step of decoding the point cloud data may include decoding the geometric data units and attribute data units included in the coding units identified by the identification information.

[0014] According to the implementation method, the identification information may include a pair of layer group identification information and subgroup identification information, wherein the layer group identification information is used to identify the layer group and the subgroup identification information is used to identify the subgroup.

[0015] According to the implementation method, the identification information may include encoding unit ID information.

[0016] According to the implementation method, the encoding method may include encoding point cloud data and sending the encoded point cloud data and signaling information, wherein the point cloud data may include geometric data and attribute data.

[0017] According to the implementation method, the geometric data and attribute data of point cloud data can be encoded based on layers and subgroups of the occupancy tree. Each layer can be a group of consecutive tree levels of the occupancy tree, and each subgroup can be a spatial subset of the layer group.

[0018] According to the implementation, the encoded geometric data can be included in multiple geometric data units, and the encoded attribute data can be included in multiple attribute data units. The multiple geometric data units and multiple attribute data units can be packaged into a subgroup-based encoding unit and transmitted. Each of the encoding units can include both geometric data units and attribute data units from the same subgroup.

[0019] According to the implementation method, multiple geometric data units and multiple attribute data units can be rearranged based on subgroups and then packaged into encoding units.

[0020] According to the implementation method, the signaling information may include identification information for identifying each of the coding units.

[0021] According to the implementation method, the identification information may include a pair of layer group identification information and subgroup identification information, wherein the layer group identification information is used to identify the layer group and the subgroup identification information is used to identify the subgroup.

[0022] According to the implementation method, the identification information may include encoding unit ID information.

[0023] According to an implementation, the transmission method may include encoding point cloud data and transmitting a bit stream containing the encoded point cloud data and signaling information, wherein the point cloud data may include geometric data and attribute data.

[0024] According to the implementation method, the geometric data and attribute data of point cloud data can be encoded based on layers and subgroups of the occupancy tree. Each layer can be a group of consecutive tree levels of the occupancy tree, and each subgroup can be a spatial subset of the layer group.

[0025] According to the implementation, the encoded geometric data can be included in multiple geometric data units, and the encoded attribute data can be included in multiple attribute data units. The multiple geometric data units and multiple attribute data units can be packaged into a subgroup-based encoding unit and transmitted. Each of the encoding units can include both geometric data units and attribute data units from the same subgroup.

[0026] Beneficial effects

[0027] The apparatus and method according to the embodiments can provide high-quality point cloud services.

[0028] Various video codec schemes can be implemented using the apparatus and method described in the embodiments.

[0029] The apparatus and method according to the embodiments can provide general point cloud content such as for autonomous driving services.

[0030] The apparatus and method according to the embodiments can improve parallel processing and provide scalability by performing spatial adaptive segmentation of point cloud data for independent encoding and decoding of point cloud data.

[0031] The apparatus and method according to the embodiments can improve the encoding and decoding performance of point cloud data by partitioning point cloud data into tiles and / or slices for encoding and decoding and signaling the required data.

[0032] The apparatus and method according to the embodiments can divide the compressed point cloud data to be transmitted according to predetermined criteria. Furthermore, when layered encoding is used, compressed data can be divided and transmitted according to layers. Therefore, the storage and transmission efficiency of the transmitting apparatus can be improved.

[0033] The apparatus and method according to the embodiments can rearrange geometric data units and attribute data units for each subgroup, and package geometric data units and one or more attribute data units of the same subgroup into a single FGS, thereby allowing the decoder to selectively perform partial decoding only on the FGS corresponding to a specific subgroup. As a result, decoding latency can be significantly reduced, and efficient data reconstruction can be achieved even in real-time decoding environments. Attached Figure Description

[0034] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this application. They illustrate embodiments of the present disclosure and, together with the description, serve to illustrate the principles of the present disclosure.

[0035] Figure 1 An example of a point cloud content provision system according to an implementation method is shown.

[0036] Figure 2 This is a block diagram illustrating the operation of providing point cloud content according to an implementation method.

[0037] Figure 3 An example of a point cloud encoder according to an implementation method is shown.

[0038] Figure 4 An example of an octree and occupancy code according to an implementation method is shown.

[0039] Figure 5 An example of point configuration for each LOD according to the implementation method is shown.

[0040] Figure 6 Another example of point configuration for each LOD according to the implementation method is illustrated.

[0041] Figure 7 An example of a point cloud decoder according to an implementation method is shown.

[0042] Figure 8 An example of a transmitting device according to an embodiment is shown.

[0043] Figure 9 An example of a receiving device according to an embodiment is shown.

[0044] Figure 10 Examples of structures that can interoperate with point cloud data transmission / reception methods / apparatus according to embodiments are illustrated.

[0045] Figure 11 (a) and Figure 11 (b) illustrates the geometric coding layer structure according to the implementation method; Figure 12 The layer and subgroup structure according to the implementation method is illustrated; Figure 13 (a) to Figure 13 (c) illustrates a representation of layer-based point cloud data according to an implementation method; Figure 14 An example of a point cloud data transmission / reception apparatus / method according to an embodiment is shown; Figure 15 This is a flowchart illustrating an example of an encoding method of an encoder according to an embodiment; Figure 16 This is a flowchart illustrating an example of a decoding method of a decoder according to an embodiment; Figure 17 Another example of a point cloud transmitting apparatus according to an embodiment is illustrated; Figure 18 Another example of a point cloud receiving device according to an embodiment is illustrated; Figure 19 (a) to Figure 19 (d) illustrates different examples of the slicing and segmentation methods according to the implementation method; Figure 20 (a) to Figure 20 (f) illustrates another example of the slicing and segmenting method according to the implementation method; Figure 21 A bitstream configuration according to an implementation method is illustrated; Figure 22a and Figure 22b An example of the syntax structure of the sequence parameter set according to the implementation method is shown; Figure 23 An example of the syntax structure of the header of the relevant geometric data unit according to the implementation method is shown; Figure 24 An example of storing a sampling index in a geometric octree encoding according to an implementation method is illustrated; Figure 25 An example of the syntax structure of the attribute parameter set according to the implementation method is shown; Figure 26 Another example of the syntax structure of the attribute parameter set according to the implementation method is shown; Figure 27 An example of the syntax structure of the header of the relevant attribute data unit according to the implementation method is shown; Figure 28 (a) and Figure 28 (b) illustrates an example of the relationship between layers, subgroups and FGS according to the implementation method; Figure 29a and Figure 29b Another example of the syntactic structure of the Sequence Parameter Set (SPS) according to the implementation method is illustrated; Figure 30 An example of the syntax structure of a geometric data unit according to an implementation method is shown; Figure 31 An example of the syntax structure of the geometric data unit header according to the implementation method is shown; Figure 32 Examples of the syntax structure of the relevant geometric data units according to the implementation method are shown; Figure 33Another example of the syntax structure of the header of the relevant geometric data unit according to the implementation method is shown; Figure 34 An example of providing services by compressing the geometry and attributes of point cloud data according to an implementation method is illustrated; Figure 35 Another example of providing services by compressing the geometry and attributes of point cloud data according to an implementation method is illustrated; Figure 36 This illustrates yet another example of providing services by compressing the geometry and attributes of point cloud data according to an implementation method; Figure 37 This illustrates yet another example of providing services by compressing the geometry and attributes of point cloud data according to an implementation method; Figure 38 Another example of providing services by compressing the geometry and attributes of point cloud data according to an implementation method is illustrated; Figure 39 This is a flowchart illustrating a method for transmitting point cloud data according to an embodiment; and Figure 40 This is a flowchart illustrating a method for receiving point cloud data according to an embodiment. Detailed Implementation

[0046] Preferred embodiments of the present disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. The detailed description given below with reference to the drawings is intended to illustrate exemplary embodiments of the present disclosure, and not to show only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0047] Although most of the terms used in this disclosure are selected from commonly used terms in the art, some terms have been arbitrarily chosen by the applicant and their meanings are explained in detail in the following description as needed. Therefore, this disclosure should be understood based on the intended meaning of the terms rather than their simple names or meanings.

[0048] Figure 1 An exemplary point cloud content delivery system according to an implementation is shown.

[0049] Figure 1 The point cloud content providing system shown may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 are capable of transmitting and receiving point cloud data via wired or wireless communication.

[0050] The point cloud data transmission device 10000 according to an embodiment can secure and process point cloud video (or point cloud content) and transmit the point cloud video. According to an embodiment, the transmission device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device, and / or a server. According to an embodiment, the transmission device 10000 may include devices configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers.

[0051] The transmitting device 10000 according to the embodiment includes a point cloud video acquisition unit 10001, a point cloud video encoder 10002 and / or a transmitter (or communication module) 10003.

[0052] The point cloud video acquisition unit 10001 according to an embodiment acquires point cloud video through processing procedures such as capture, synthesis, or generation. Point cloud video is point cloud content represented by a point cloud, which is a set of points located in 3D space, and may be referred to as point cloud video data, point cloud data, etc. The point cloud video according to an embodiment may include one or more frames. A frame represents a still image / picture. Therefore, point cloud video may include point cloud images / frames / pictures, and may be referred to as point cloud images, frames, or pictures.

[0053] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 can encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to an embodiment may include geometry-based point cloud compression (G-PCC) coding and / or video-based point cloud compression (V-PCC) coding or next-generation coding. The point cloud compression coding according to an embodiment is not limited to the above embodiments. The point cloud video encoder 10002 can output a bitstream containing the encoded point cloud video data. The bitstream may contain not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0054] According to an embodiment, transmitter 10003 transmits a bitstream containing encoded point cloud video data. The bitstream, according to an embodiment, is encapsulated in a file or segment (e.g., a streaming segment) and transmitted via various networks such as broadcast networks and / or broadband networks. Although not shown in the figures, transmitting device 10000 may include an encapsulator (or encapsulation module) configured to perform encapsulation operations. According to an embodiment, the encapsulator may be included in transmitter 10003. According to an embodiment, the file or segment may be transmitted via a network to receiving device 10004 or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). Transmitter 10003 according to an embodiment is capable of wired / wireless communication with receiving device 10004 (or receiver 10005) via networks such as 4G, 5G, and 6G. Additionally, the transmitter may perform necessary data processing operations depending on the network system (e.g., a 4G, 5G, or 6G communication network system). Transmitting device 10000 can transmit encapsulated data on demand.

[0055] The receiving device 10004 according to an embodiment includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include devices, robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)).

[0056] According to an embodiment, receiver 10005 receives a bitstream containing point cloud video data or a file / segment encapsulated with a bitstream from a network or storage medium. Receiver 10005 can perform necessary data processing according to the network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). According to an embodiment, receiver 10005 can decapsulate the received file / segment and output a bitstream. According to an embodiment, receiver 10005 may include a decapsulator (or decapsulator module) configured to perform a decapsulation operation. The decapsulator may be implemented as a separate element (or component) from receiver 10005.

[0057] The point cloud video decoder 10006 decodes the bitstream containing point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to the method in which the point cloud video data is encoded (e.g., the reverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding (the reverse process of point cloud compression). Point cloud decompression encoding includes G-PCC encoding.

[0058] Renderer 10007 renders decoded point cloud video data. In one embodiment, renderer 10007 can render decoded point cloud video data according to a viewport, etc. Renderer 10007 can render not only point cloud video data but also audio data to output point cloud content. According to an embodiment, renderer 10007 may include a display configured to display point cloud content. According to an embodiment, the display may be implemented as a separate device or component rather than included in renderer 10007.

[0059] The arrows indicated by dashed lines in the diagram represent the transmission paths of the feedback information acquired by the receiving device 10004. The feedback information reflects the interactivity of the user consuming the point cloud content and includes information about the user (e.g., head orientation information, viewport information, etc.). Specifically, when the point cloud content is for a service requiring user interaction (e.g., autonomous driving services, etc.), the feedback information can be provided to the content sender (e.g., the sending device 10000) and / or the service provider. Depending on the implementation, the feedback information may be used in both the receiving device 10004 and the sending device 10000, or it may not be provided.

[0060] According to the embodiment, head orientation information can represent information about the position, orientation, angle, and movement of the user's head. The receiving device 10004 according to the embodiment can calculate viewport information based on the head orientation information. Viewport information is information about the area of ​​the point cloud video that the user is viewing (that is, the area the user is currently viewing). In other words, viewport information is information about the area the user is currently viewing in the point cloud video. In other words, the viewport or viewport area can represent the area the user is viewing in the point cloud video. The viewpoint is the point the user is viewing in the point cloud video and can represent the center point of the viewport area. That is, the viewport is the area centered on the viewpoint, and the size and shape of the area can be determined by the field of view (FOV). Therefore, the receiving device 10004 can extract viewport information based on the vertical or horizontal FOV supported by the device and the head orientation information. Furthermore, the receiving device 10004 can perform gaze analysis, etc., based on the head orientation information and / or viewport information to determine how the user consumes the point cloud video, the area the user is gazing at in the point cloud video, and the gaze duration. According to one embodiment, the receiving device 10004 can send feedback information, including gaze analysis results, to the transmitting device 10000. According to another embodiment, a device such as a VR / XR / AR / MR display can extract the viewport region based on the user's head position / orientation and a vertical or horizontal field of view (FOV) supported by the device. According to another embodiment, head orientation information and viewport information can be referred to as feedback information, signaling information, or metadata.

[0061] According to the embodiments, feedback information can be obtained during rendering and / or display. The feedback information can be ensured by one or more sensors included in the receiving device 10004. According to the embodiments, the feedback information can be ensured by the renderer 10007 or by separate external components (or devices, components, etc.). Figure 1 The dashed lines in the diagram represent the process of sending feedback information ensured by renderer 10007. This feedback information can be sent not only to the sending side but also consumed at the receiving side. In other words, the point cloud content providing system can process (encode / decode / render) point cloud data based on the feedback information. For example, point cloud video decoder 10006 and renderer 10007 can prioritize decoding and rendering point cloud video only for the area currently being viewed by the user, based on the feedback information (i.e., head orientation information and / or viewport information).

[0062] Furthermore, the receiving device 10004 can send feedback information to the transmitting device 10000. The transmitting device 10000 (or the point cloud video encoder 10002) can perform encoding operations based on the feedback information. Therefore, the point cloud content providing system can efficiently process necessary data (e.g., point cloud data corresponding to the user's head position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide the point cloud content to the user.

[0063] According to the implementation method, the transmitting device 10000 can be called an encoder, transmitting device, transmitter, transmitting system, etc., and the receiving device 10004 can be called a decoder, receiving device, receiver, receiving system, etc.

[0064] According to the implementation method Figure 1 Point cloud data processed in a point cloud content provision system (through a series of processes including acquisition, encoding, transmission, decoding, and rendering) can be referred to as point cloud content data or point cloud video data. Depending on the implementation, point cloud content data can be used to encompass concepts related to metadata or signaling information associated with point cloud data.

[0065] Figure 1 The components of the point cloud content providing system shown can be implemented by hardware, software, processors, and / or combinations thereof.

[0066] Figure 2 This is a block diagram illustrating the operation of providing point cloud content according to an implementation method.

[0067] Figure 2 The block diagram shows Figure 1 The operation of the point cloud content providing system described herein. As mentioned above, the point cloud content providing system can process point cloud data based on point cloud compression encoding (e.g., G-PCC).

[0068] A point cloud content providing system according to an embodiment (e.g., point cloud sending device 10000 or point cloud video acquisition unit 10001) can acquire point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system used to represent 3D space. The point cloud video according to an embodiment may include Ply (Polygon file format or Stanford Triangle format) files. When the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of the points. The position of each point can be represented by parameters (e.g., values ​​of the X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include the attributes of the points (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc., of each point). A point has one or more attributes. For example, a point may have a color attribute or two attributes: color and reflectivity. According to the implementation, geometry can be referred to as location, geometric information, geometric data, location information, location data, etc., and attributes can be referred to as attributes, attribute information, attribute data, etc. The point cloud content providing system (e.g., point cloud sending device 10000 or point cloud video acquisition unit 10001) can secure point cloud data from information related to the point cloud video acquisition process (e.g., depth information, color information, etc.).

[0069] A point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) according to an embodiment can encode point cloud data (20001). The point cloud content providing system can encode point cloud data based on point cloud compression encoding. As described above, point cloud data can include geometric information and attribute information about points. Therefore, the point cloud content providing system can perform geometric encoding to encode geometry and output a geometric bitstream. The point cloud content providing system can perform attribute encoding to encode attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system can perform attribute encoding based on geometric encoding. The geometric bitstream and attribute bitstream according to an embodiment can be multiplexed and output as a single bitstream. The bitstream according to an embodiment may also contain signaling information related to geometric encoding and attribute encoding.

[0070] A point cloud content providing system according to an embodiment (e.g., transmitting device 10000 or transmitter 10003) can transmit encoded point cloud data (20002). For example... Figure 1As shown, encoded point cloud data can be represented by geometric bitstreams and attribute bitstreams. Additionally, the encoded point cloud data can be transmitted as a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometric encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and transmit it as a file or fragment.

[0071] The point cloud content providing system according to the embodiment (e.g., receiving device 10004 or receiver 10005) can receive a bitstream containing encoded point cloud data. Additionally, the point cloud content providing system (e.g., receiving device 10004 or receiver 10005) can demultiplex the bitstream.

[0072] A point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode encoded point cloud data (e.g., geometric bitstream, attribute bitstream) transmitted in a bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode point cloud video data based on signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode the geometric bitstream to reconstruct the location (geometry) of the points. The point cloud content providing system can reconstruct the attributes of the points by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can reconstruct point cloud video based on location according to the reconstructed geometry and the decoded attributes.

[0073] A point cloud content providing system (e.g., receiving device 10004 or renderer 10007) according to an embodiment can render decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) can use various rendering methods to render the geometry and attributes decoded through the decoding process. Points in the point cloud content can be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to the user through a display (e.g., a VR / AR display, a general display, etc.).

[0074] The point cloud content providing system according to the embodiment (e.g., receiving device 10004) can ensure feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. The feedback information and operation / reference of the point cloud content providing system according to the embodiment... Figure 1 The feedback information and operation described are the same, so their detailed description is omitted.

[0075] Figure 3 An exemplary point cloud encoder according to an implementation method is illustrated.

[0076] Figure 3 Show Figure 1 An example of a point cloud video encoder 10002. The point cloud encoder reconstructs and encodes point cloud data (e.g., point locations and / or attributes) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) based on network conditions or applications. When the total size of the point cloud content is large (e.g., providing 60 Gbps of point cloud content for 30 fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system can reconstruct the point cloud content based on a maximum target bitrate to provide the point cloud content according to network conditions, etc.

[0077] For reference Figure 1 and Figure 2 As described, the point cloud encoder can perform both geometric encoding and attribute encoding. Geometric encoding is performed before attribute encoding.

[0078] The point cloud video encoder according to the implementation includes a coordinate transformer (transform coordinates) 30000, a quantizer (quantizes and removes points (voxarization)) 30001, an octree analyzer (analyzes octrees) 30002, a surface approximation analyzer (analyzes surface approximations) 30003, an arithmetic encoder (arithmetic encoding) 30004, a geometry reconstructor (reconstructs geometry) 30005, a color transformer (transforms colors) 30006, an attribute transformer (transforms attributes) 30007, a RAHT transformer 30008, a LOD generator (generates LODs) 30009, a lift transformer (lifts) 30010, a coefficient quantizer (quantizes coefficients) 30011, and / or an arithmetic encoder (arithmetic encoding) 30012. Figure 3 In the point cloud encoder, coordinate transformer 30000, quantizer 30001, octree analyzer 30002, surface approximation analyzer 30003, arithmetic encoder 30004, and geometric reconstructor 30005 can be grouped together and called the geometric encoder. Color transformer 30006, attribute transformer 30007, RAHT transformer 30008, LOD generator 30009, lift transformer 30010, coefficient quantizer 30011, and / or arithmetic encoder 30012 can be grouped together and called the attribute encoder.

[0079] Coordinate transformer 30000, quantizer 30001, octree analyzer 30002, surface approximation analyzer 30003, arithmetic encoder 30004, and geometric reconstructor 30005 can perform geometric coding. Geometric coding according to the implementation may include octree geometric coding, prediction tree geometric coding, direct coding, triplet geometric coding, and entropy coding. Direct coding and triplet geometric coding are applied selectively or in combination. Geometric coding is not limited to the examples described above.

[0080] As shown in the figure, the coordinate transformer 30000 according to the embodiment receives the position and transforms it into coordinates. For example, the position can be transformed into position information in three-dimensional space (e.g., three-dimensional space represented by the XYZ coordinate system). The position information in three-dimensional space according to the embodiment can be referred to as geometric information.

[0081] The quantizer 30001 according to the embodiment quantizes geometry. For example, the quantizer 30001 may quantize points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). The quantizer 30001 performs a quantization operation: multiplying the difference between the minimum position value and the position value of each point by a preset quantization scaling value, and then finding the nearest integer value by rounding the value obtained by the multiplication. Thus, one or more points may have the same quantized position (or position value). The quantizer 30001 according to the embodiment performs voxelization based on the quantized position to reconstruct the quantized points. As in the case of pixels (the smallest unit containing 2D image / video information), points in the point cloud content (or 3D point cloud video) according to the embodiment may be included in one or more voxels. As a combination of volume and pixel, the term voxel refers to the 3D cubic space generated when 3D space is divided into units (unit = 1.0) based on axes representing 3D space (e.g., X-axis, Y-axis, and Z-axis). Quantizer 30001 can match a group of points in 3D space to voxels. According to one embodiment, a voxel may include only one point. According to another embodiment, a voxel may include one or more points. To represent a voxel as a point, the location of the voxel's center can be set based on the locations of one or more points included in the voxel. In this case, attributes of all locations included in a voxel can be combined and assigned to the voxel.

[0082] The octree analyzer 30002 according to the implementation performs octree geometric encoding (or octree coding) to represent voxels in an octree structure. The octree structure represents points based on the matching of octree structures with voxels.

[0083] The surface approximation analyzer 30003 according to the embodiment can analyze and approximate an octree. The octree analysis and approximation according to the embodiment is a process of analyzing a region containing multiple points to efficiently provide an octree and voxelization.

[0084] The arithmetic encoder 30004 according to the embodiment performs entropy encoding on octrees and / or approximate octrees. For example, the encoding scheme includes arithmetic encoding. As a result of the encoding, a geometric bitstream is generated.

[0085] The attribute encoding is performed by a color transformer 30006, an attribute transformer 30007, a RAHT transformer 30008, a LOD generator 30009, a boosting transformer 30010, a coefficient quantizer 30011, and / or an arithmetic encoder 30012. As described above, a point can have one or more attributes. The attribute encoding according to the embodiment is also applied to the attributes that a point has. However, when an attribute (e.g., color) includes one or more elements, attribute encoding is applied independently to each element. The attribute encoding according to the embodiment includes color transformation encoding, attribute transformation encoding, region adaptive hierarchical transformation (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) encoding, and interpolation-based hierarchical nearest neighbor prediction (boosting transformation) encoding with update / boosting steps. Depending on the point cloud content, the above-described RAHT encoding, prediction transformation encoding, and boosting transformation encoding can be used selectively, or a combination of one or more encoding schemes can be used. The attribute encoding according to the embodiment is not limited to the examples described above.

[0086] The color converter 30006 according to the embodiment performs color transformation encoding to transform the color values ​​(or textures) included in the attributes. For example, the color converter 30006 can transform the format of color information (e.g., from RGB to YCbCr). Optionally, the operation of the color converter 30006 according to the embodiment can be applied based on the color values ​​included in the attributes.

[0087] The geometry reconstructor 30005, according to the implementation method, reconstructs (decompresses) octrees and / or approximate octrees. The geometry reconstructor 30005 reconstructs the octree / voxel based on the results of analyzing the point distribution. The reconstructed octree / voxel can be referred to as the reconstructed geometry (restored geometry).

[0088] The attribute transformer 30007 according to the embodiment performs attribute transformation to transform attributes based on reconstructed geometry and / or positions without performing geometric encoding. As described above, since attributes depend on geometry, the attribute transformer 30007 can transform attributes based on reconstructed geometric information. For example, based on the position value of a point included in a voxel, the attribute transformer 30007 can transform the attribute of the point at that position. As described above, when the center position of a voxel is set based on the positions of one or more points included in the voxel, the attribute transformer 30007 transforms the attribute of one or more points. When performing triadic geometric encoding, the attribute transformer 30007 can transform attributes based on the triadic geometric encoding.

[0089] The attribute transformer 30007 performs attribute transformation by calculating the average of the attributes or attribute values ​​(e.g., color or reflectivity of each point) of neighboring points within a specific location / radius from the center (or position value) of each voxel. The attribute transformer 30007 can apply weights based on the distance from the center to each point when calculating the average. Therefore, each voxel has a position and a calculated attribute (or attribute value).

[0090] The attribute transformer 30007 can search for nearest neighbors within a specific location / radius of the center of each voxel based on a KD-tree or Morton code. A KD-tree is a binary search tree and supports a data structure that allows points to be managed based on location, enabling fast nearest neighbor search (NNS). Morton codes are generated by representing the coordinates (e.g., (x, y, z)) of the 3D location of all points as bit values ​​and mixing the bits. For example, when the coordinates representing the point location are (5, 9, 1), the bit values ​​are (0101, 1001, 0001). Mixing the bit values ​​according to the bit index in the order of z, y, and x produces 010001000111. This value is represented as the decimal number 1095. That is, the Morton code value for the point with coordinates (5, 9, 1) is 1095. The attribute transformer 30007 can sort the points based on their Morton code values ​​and perform NNS using a depth-first traversal process. After an attribute transformation operation, use a KD tree or Morton code when an NNS is needed in another transformation process used for attribute encoding.

[0091] As shown in the figure, the transformation properties are input to the RAHT transformer 30008 and / or the LOD generator 30009.

[0092] According to the implementation, the RAHT transformer 30008 performs RAHT encoding for predicting attribute information based on the reconstructed geometric information. For example, the RAHT transformer 30008 can predict the attribute information of higher-level nodes in an octree based on the attribute information associated with lower-level nodes in the octree.

[0093] The LOD generator 30009 according to the embodiment generates a Level of Detail (LOD) to perform predictive transform coding. The LOD according to the embodiment represents the level of detail of the point cloud content. As the LOD value decreases, it indicates a deterioration in the detail of the point cloud content. As the LOD value increases, it indicates an enhancement in the detail of the point cloud content. Points can be classified according to LOD.

[0094] The lift transformer 30010 according to the embodiment performs lift transform coding to transform point cloud attributes based on weights. As described above, lift transform coding may optionally be applied.

[0095] According to the implementation method, the coefficient quantizer 30011 quantizes the attribute encoding based on the coefficient.

[0096] According to the implementation method, the arithmetic encoder 30012 encodes quantized attributes based on arithmetic coding.

[0097] Although not shown in the figure, Figure 3 The elements of the point cloud encoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above-described... Figure 3 At least one of the operations and / or functions of the elements of the point cloud encoder. Additionally, one or more processors can operate or perform operations for executing... Figure 3 The software program and / or instruction set for the operation and / or function of the elements of the point cloud encoder. One or more memories according to the embodiments may include high-speed random access memory, or include non-volatile memory (e.g., one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices).

[0098] Figure 4 An example of an octree and occupancy code according to an implementation is shown.

[0099] For reference Figure 1 to Figure 3 As described, the point cloud content delivery system (point cloud video encoder 10002) or point cloud encoder (e.g., octree analyzer 30002) performs octree geometric encoding (or octree encoding) based on an octree structure to efficiently manage the regions and / or locations of voxels.

[0100] Figure 4 The upper part shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes of a coordinate system (e.g., the X, Y, and Z axes). This is achieved by two poles (0, 0, 0) and (2... d , 2 d , 2 dAn octree structure is created by recursively subdividing a bounding box aligned to the cubic axis. Here, 2d can be set as the value of the minimum bounding box that constitutes all points surrounding the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following formula. In the following formula, (x int n , y int n , z int n ) indicates the position (or position value) of the quantized point.

[0101]

[0102] like Figure 4 As shown in the upper center, the entire 3D space can be divided into eight spaces according to partitions. Each partitioned space is represented by a cube with six faces. For example... Figure 4 As shown in the upper right, each of the eight spaces is further subdivided based on a coordinate system axis (e.g., the X, Y, and Z axes). Thus, each space is divided into eight smaller spaces. These smaller spaces are also represented by cubes with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0103] Figure 4 The lower part shows the octet occupancy code. The occupancy code generates the octet to indicate whether each of the eight partitions generated by dividing a space contains at least one node. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of a partitioned space, and each child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to a child node contains at least one node, the node is assigned a value of 1. When the space corresponding to a child node does not contain a node (the space is empty), the node is assigned a value of 0. Since... Figure 4 The occupancy code shown is 00100001, thus indicating that the spaces corresponding to the third and eighth child nodes among the eight child nodes each contain at least one point. As shown, each of the third and eighth child nodes has eight child nodes, and the child nodes are represented by an 8-bit occupancy code. The accompanying figure shows that the occupancy code for the third child node is 10000111, and the occupancy code for the eighth child node is 01001111. A point cloud encoder (e.g., an arithmetic encoder 30004) according to an embodiment can perform entropy encoding on the occupancy code. To increase compression efficiency, the point cloud encoder can perform intra-frame / inter-frame encoding on the occupancy code. A receiving device (e.g., receiving device 10004 or point cloud video decoder 10006) according to an embodiment reconstructs the octree based on the occupancy code.

[0104] A point cloud encoder according to an implementation method (e.g.,Figure 4 A point cloud encoder or octree analyzer (30002) can perform voxelization and octree encoding to store point locations. However, points are not always uniformly distributed in 3D space, so there may be specific regions with fewer points. Therefore, performing voxelization over the entire 3D space is inefficient. For example, when a specific region contains very few points, voxelization is not necessary in that specific region.

[0105] Therefore, for the aforementioned specific region (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment can skip voxelization and perform direct encoding to directly encode the point positions included in the specific region. The coordinates of the directly encoded points according to the embodiment are called the Direct Encoding Mode (DCM). The point cloud encoder according to the embodiment can also perform triadic geometry encoding based on the surface model, which reconstructs the point positions in the specific region (or node) based on voxels. Triadic geometry encoding is a geometric encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct encoding and triadic geometry encoding according to the embodiment can be performed selectively. In addition, direct encoding and triadic geometry encoding according to the embodiment can be performed in combination with octree geometry encoding (or octree encoding).

[0106] To perform direct encoding, the option to apply direct encoding using direct mode should be enabled. The node to which direct encoding is applied must not be a leaf node, and there should be fewer than a threshold number of points within that node. Furthermore, the total number of points to which direct encoding is applied should not exceed a preset threshold. When the above conditions are met, the point cloud encoder (or arithmetic encoder 30004) according to the implementation can perform entropy encoding on the point locations (or location values).

[0107] A point cloud encoder according to an embodiment (e.g., a surface approximation analyzer 30003) can determine a specific level of the octree (a level less than the depth d of the octree) and can start from that level using a surface model to perform triadic geometry encoding to reconstruct the point positions in the node region based on voxels (triadic mode). The point cloud encoder according to an embodiment can specify the level to which triadic geometry encoding is to be applied. For example, when the specific level is equal to the depth of the octree, the point cloud encoder does not operate in triadic mode. In other words, the point cloud encoder according to an embodiment can only operate in triadic mode when the specified level is less than the depth value of the octree. The 3D cubic region of a node at a specified level according to an embodiment is called a block. A block may include one or more voxels. A block or voxel may correspond to a cube. Geometry is represented as surfaces within each block. A surface according to an embodiment may intersect each edge of a block at most once.

[0108] A block has 12 edges, therefore a block contains at least 12 intersections. Each intersection is called a vertex. Vertices along an edge are detected when there is at least one occupied voxel adjacent to the edge in all blocks sharing the edge. An occupied voxel, according to the implementation, refers to a voxel containing a point. The vertex position detected along an edge is the average position of all voxels adjacent to the edge in all blocks sharing the edge along that edge.

[0109] Once a vertex is detected, the point cloud encoder according to the implementation can perform entropy encoding on the edge's origin (x, y, z), the edge's direction vector (Δx, Δy, Δz), and the vertex position value (relative position value within the edge). When applying triad geometry encoding, the point cloud encoder according to the implementation (e.g., geometry reconstructor 30005) can generate the restored geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization processes.

[0110] Vertices located at the edges of a block determine the surface passing through the block. The surface, according to the implementation, is a non-planar polygon. During triangle reconstruction, the surface represented by triangles is reconstructed based on the origin of the edges, the direction vectors of the edges, and the position values ​​of the vertices. The triangle reconstruction process is performed as follows: i) calculating the centroid value of each vertex, ii) subtracting the centroid value from each vertex value, and iii) estimating the sum of squares of the values ​​obtained through the subtraction.

[0111]

[0112] The minimum value of the sum is estimated, and a projection process is performed based on the axis with the minimum value. For example, when element x is minimum, each vertex is projected onto the x-axis relative to the center of the block, and onto the (y, z) plane. When the value obtained by projection onto the (y, z) plane is (ai, bi), the value of θ is estimated by atan2(bi, ai), and the vertices are sorted based on the value of θ. Table 1 below shows the vertex combinations for creating triangles based on the number of vertices. Vertices are sorted from 1 to n. Table 1 below shows that for four vertices, two triangles can be constructed based on vertex combinations. The first triangle can be composed of vertices 1, 2, and 3 from the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 from the sorted vertices.

[0113] [Table 1]

[0114] [Table 1] Triangles formed from vertices sorted by 1, ..., n

[0115] An upsampling process is performed to add points along the edges of the triangle at the center, and voxelization is then performed. The added points are generated based on the upsampling factor and the width of the block. The added points are called thinned vertices. A point cloud encoder according to an implementation can voxelize the thinned vertices. Additionally, the point cloud encoder can perform attribute encoding based on the voxelized positions (or position values).

[0116] Figure 5 Examples of point configurations in various Levels of Detail (LODs) according to the implementation method are illustrated.

[0117] For reference Figure 1 to Figure 4 The described approach involves reconstructing (decompressing) the encoded geometry before performing attribute encoding. When direct encoding is applied, the geometry reconstruction operation may include altering the placement of directly encoded points (e.g., placing directly encoded points in front of the point cloud data). When triadic geometry encoding is applied, the geometry reconstruction process is performed through triangle reconstruction, upsampling, and voxelization. Since attributes depend on geometry, attribute encoding is performed based on the reconstructed geometry.

[0118] Point cloud encoders (e.g., LOD generator 30009) can classify (or reorganize) points according to LOD. The figure shows the point cloud content corresponding to LOD. The leftmost view in the figure represents the original point cloud content. The second view from the left in the figure represents the point distribution in the lowest LOD, and the rightmost view represents the point distribution in the highest LOD. That is, points are sparsely distributed in the lowest LOD and densely distributed in the highest LOD. In other words, as the LOD increases in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between points narrows.

[0119] Figure 6 Examples of point configurations in various Levels of Detail (LODs) according to the implementation method are shown.

[0120] For reference Figure 1 to Figure 5 As described, a point cloud content providing system or point cloud encoder (e.g., point cloud video encoder 10002, Figure 3 A point cloud encoder or LOD generator (30009) can generate LODs. LODs are generated by reorganizing points into a set of refined levels based on a set of LOD distance values ​​(or a set of Euclidean distances). The LOD generation process is performed not only by the point cloud encoder but also by the point cloud decoder.

[0121] Figure 6 The upper part shows examples of points (P0 to P9) of point cloud content distributed in 3D space. Figure 6 In this context, the original order represents the order of points P0 to P9 before LOD generation. Figure 6In this context, LOD-based order represents the order in which points are generated according to their LOD. Points are reorganized by LOD. Additionally, higher LODs include points belonging to lower LODs. For example... Figure 6 As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, the points of LOD1, P9, P8, and P7.

[0122] For reference Figure 3 As described, the point cloud encoder according to the implementation can selectively or in combination perform predictive transform coding, lifting transform coding, and RAHT transform coding.

[0123] The point cloud encoder according to the embodiment can generate predictors for points to perform predictive transformation encoding for setting the predictive attributes (or predictive attribute values) of each point. That is, N predictors can be generated for N points. The predictors according to the embodiment can calculate weights (=1 / distance) based on the LOD value of each point, index information of neighboring points existing within a set distance of each LOD, and the distance to the neighboring points.

[0124] According to the implementation, the predicted attribute (or attribute value) is set as the average of the values ​​obtained by multiplying the attributes (or attribute values) of neighboring points (e.g., color, reflectivity, etc.) set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 30011) according to the implementation can quantize and inverse quantize the residual (which may be referred to as residual attribute, residual attribute value, attribute prediction residual, etc.) obtained by subtracting the predicted attribute (or attribute value) of each point from the attribute (attribute value) of each point. Tables 2 and 3 below illustrate the quantization process.

[0125] [Table 2]

[0126] [Table 3]

[0127] When the predictors of each point have neighboring points, the point cloud encoder (e.g., arithmetic encoder 30012) according to the embodiment can perform entropy encoding on the residual values ​​of quantization and inverse quantization as described above. When the predictors of each point do not have neighboring points, the point cloud encoder (e.g., arithmetic encoder 30012) according to the embodiment can perform entropy encoding on the attributes of the corresponding point without performing the above operations.

[0128] The point cloud encoder (e.g., lift transformer 30010) according to the embodiment can generate predictors for each point, set the calculated LOD and register neighboring points in the predictors, and set weights based on the distance to the neighboring points to perform lift transform coding. The lift transform coding according to the embodiment is similar to the prediction transform coding described above, but the difference is that weights are applied cumulatively to the attribute values. The process of cumulatively applying weights to the attribute values ​​according to the embodiment is configured as follows.

[0129] 1) Create an array quantized weights (QW) to store the weight values ​​of each point. The initial value of all elements of QW is 1.0. Multiply the QW value of the predictor index of the neighboring nodes registered in the predictor by the weight of the current point's predictor, and add the values ​​obtained by multiplication.

[0130] 2) Improve the prediction process: Subtract the value obtained by multiplying the attribute value of the point by the weight from the existing attribute value to calculate the predicted attribute value.

[0131] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0132] 4) The weights calculated by multiplying the weights computed for all predictors by the weights stored in the QW corresponding to the predictor index are summed with the updateweight array to serve as the index of the neighbor node. The values ​​obtained by multiplying the attribute values ​​of the neighbor node indexes by the calculated weights are summed with the update array.

[0133] 5) Improve the update process: Divide the attribute values ​​of the update array of all predictors by the weight values ​​of the updateweight array of the predictor index, and add the existing attribute values ​​to the values ​​obtained by division.

[0134] 6) For all predictors, the predicted attribute is calculated by multiplying the attribute value updated through the boosting update process by the weight updated through the boosting prediction process (stored in QW). The predicted attribute value is quantized by a point cloud encoder (e.g., coefficient quantizer 30011) according to the implementation. In addition, the point cloud encoder (e.g., arithmetic encoder 30012) performs entropy encoding on the quantized attribute value.

[0135] A point cloud encoder according to an embodiment (e.g., RAHT transform 30008) can perform RAHT transform coding, where attributes associated with lower-level nodes in an octree are used to predict attributes of higher-level nodes. RAHT transform coding is an example of intra-frame attribute coding performed by scanning backward through an octree. The point cloud encoder according to an embodiment scans the entire region starting from voxels and repeats a merging process of merging voxels into larger blocks at each step until the root node is reached. The merging process according to the embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on the node directly above an empty node.

[0136] The following equation represents the RAHT transformation matrix. In this equation, This represents the average attribute value of a voxel at level l. Based on and To calculate. and The weight is and .

[0137]

[0138] here, It is a low-pass value and is used in the next higher level of merging. This represents the high-pass coefficient. The high-pass coefficient at each step is quantized and subjected to entropy encoding (e.g., encoded by an arithmetic encoder 30012). Weights are calculated as follows: .pass and Create the root node as follows.

[0139]

[0140] The value of gDC is also quantized and entropy-encoded, just like the high-pass coefficient.

[0141] Figure 7 An example of a point cloud decoder according to an implementation method is shown.

[0142] Figure 7 The point cloud decoder shown is an example of a point cloud decoder and can perform decoding operations. Figure 1 to Figure 6 The reverse process of the encoding operation of the point cloud encoder is shown.

[0143] For reference Figure 1 and Figure 6 As described, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0144] The point cloud decoder according to the implementation includes an arithmetic decoder (arithmetic decoding) 7000, an octree synthesizer (synthesized octree) 7001, a surface approximation synthesizer (synthesized surface approximation) 7002, a geometry reconstructor (reconstructed geometry) 7003, an inverse coordinate transformer (inverse coordinate transformation) 7004, an arithmetic decoder (arithmetic decoding) 7005, an inverse quantizer (inverse quantization) 7006, a RAHT transformer 7007, an LOD generator (generated LOD) 7008, an inverse lifter (inverse lift) 7009, and / or a color inverse transformer (inverse color transformation) 7010.

[0145] An arithmetic decoder 7000, an octree synthesizer 7001, a surface approximation synthesizer 7002, a geometry reconstructor 7003, and a coordinate inverse transformer 7004 can perform geometric decoding. Geometric decoding according to the embodiment may include direct decoding and triplet geometric decoding. Direct encoding and triplet geometric decoding are selectively applied. Geometric decoding is not limited to the examples described above and is provided as a reference. Figure 1 to Figure 6 The reverse process of the described geometric encoding is executed.

[0146] The arithmetic decoder 7000 according to the embodiment decodes the received geometric bitstream based on arithmetic coding. The operation of the arithmetic decoder 7000 corresponds to the inverse process of the arithmetic encoder 30004.

[0147] The octree synthesizer 7001 according to the embodiment can generate an octree by obtaining occupancy codes (or information about the geometry obtained as a decoding result) from the decoded geometry bitstream. The occupancy codes are as follows: Figure 1 to Figure 6 Please describe that configuration in detail.

[0148] When applying triplet geometry encoding, the surface approximation synthesizer 7002 according to the implementation can synthesize the surface based on the decoded geometry and / or the generated octree.

[0149] The geometry reconstructor 7003 according to the embodiment can regenerate geometry based on surface and / or decoded geometry. See also... Figure 1 to Figure 9 As described, direct encoding and triadic geometric encoding are selectively applied. Therefore, the geometry reconstructor 7003 directly imports and sums the positional information of points for which direct encoding has been applied. When triadic geometric encoding is applied, the geometry reconstructor 7003 can reconstruct the geometry by performing the reconstruction operations (e.g., triangle reconstruction, upsampling, and voxelization) of the geometry reconstructor 30005. Details and references Figure 6 The descriptions are the same for all of them, so their descriptions are omitted. The reconstructed geometry may include point cloud images or frames that do not contain attributes.

[0150] According to the implementation method, the inverse coordinate transformer 7004 can obtain the point position based on the reconstructed geometric transformation coordinates.

[0151] Arithmetic decoder 7005, inverse quantizer 7006, RAHT transformer 7007, LOD generator 7008, inverse lifter 7009, and / or color inverse transformer 7010 can perform attribute decoding. Attribute decoding according to embodiments includes Region Adaptive Hierarchical Transformation (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) decoding, and interpolation-based hierarchical nearest neighbor prediction (lift transform) decoding with update / lifting steps. The above three decoding schemes can be used selectively, or a combination of one or more decoding schemes can be used. Attribute decoding according to embodiments is not limited to the examples described above.

[0152] According to the implementation method, the arithmetic decoder 7005 decodes the attribute bitstream by arithmetic encoding.

[0153] The inverse quantizer 7006 according to the implementation dequantizes information about the decoded attribute bitstream or the attributes obtained as a decoding result, and outputs the dequantized attributes (or attribute values). Inverse quantization can be selectively applied based on the attribute encoding of the point cloud encoder.

[0154] According to the implementation, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 can process the reconstructed geometry and inverse quantization attributes. As described above, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 can selectively perform decoding operations corresponding to the encoding of the point cloud encoder.

[0155] According to the implementation, the color inverse transformer 7010 performs inverse transformation encoding to inversely transform the color values ​​(or textures) included in the decoded attributes. The operation of the color inverse transformer 7010 can be selectively performed based on the operation of the color transformer 30006 of the point cloud encoder.

[0156] Although not shown in the figure, Figure 7 The elements of the point cloud decoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above-described... Figure 7 The point cloud decoder's components include at least one or more operations and / or functions. Additionally, one or more processors can operate or perform operations for executing... Figure 7 The software program and / or instruction set for the operation and / or function of the elements of the point cloud decoder.

[0157] Figure 8 An example of a transmitting device according to an embodiment is shown.

[0158] Figure 8 The transmitting device shown is Figure 1 The transmitting device 10000 (or Figure 3 Example of a point cloud encoder. Figure 8 The transmitting device shown can perform the same operation as the reference. Figure 1 to Figure 6 The described point cloud encoder includes one or more of the same or similar operations and methods. The transmitting apparatus according to the embodiment may include a data input unit 8000, a quantization processor 8001, a voxelization processor 8002, an octree occupancy code generator 8003, a surface model processor 8004, an intra / inter-frame coding processor 8005, an arithmetic encoder 8006, a metadata processor 8007, a color transformation processor 8008, an attribute transformation processor 8009, a prediction / boosting / RAHT transformation processor 8010, an arithmetic encoder 8011, and / or a transmission processor 8012.

[0159] According to the embodiment, the data input unit 8000 receives or acquires point cloud data. The data input unit 8000 can perform the same operation and / or acquisition method as the point cloud video acquisition unit 10001 (or refer to...). Figure 2 The described acquisition process (20000) is the same or similar operation and / or acquisition method.

[0160] The data input unit 8000, quantization processor 8001, voxelization processor 8002, octree occupancy code generator 8003, surface model processor 8004, intra / inter-frame coding processor 8005, and arithmetic encoder 8006 perform geometric coding. Geometric coding according to the implementation method and reference... Figure 1 to Figure 9 The geometric codes described are the same or similar, so their detailed descriptions are omitted.

[0161] The quantization processor 8001 according to the implementation quantizes geometry (e.g., point position values). The operation of the quantization processor 8001 and / or quantization with reference... Figure 3 The operation and / or quantization of the described quantizer 30001 are the same or similar. Details and references Figure 1 to Figure 9 The descriptions are the same.

[0162] According to the embodiment, the voxelization processor 8002 voxels the quantized position values ​​of points. The voxelization processor 8002 can perform operations similar to those described above. Figure 3 The operation and / or voxelization process of the quantizer 30001 described are the same as or similar to the operation and / or process. Details and references Figure 1 to Figure 6 The descriptions are the same.

[0163] According to the implementation method, the octree occupancy code generator 8003 performs octree encoding based on the voxelized positions of points in the octree structure. The octree occupancy code generator 8003 can generate occupancy codes. The octree occupancy code generator 8003 can perform encoding with reference to... Figure 3 and Figure 4 The operations and / or methods described are the same as or similar to those of the point cloud encoder (or octree analyzer 30002). Details and references Figure 1 to Figure 6 The descriptions are the same.

[0164] According to the implementation, the surface model processor 8004 can perform triadic geometry encoding based on a surface model to reconstruct point positions in a specific region (or node) based on voxels. The surface model processor 8004 can perform operations with reference to... Figure 3 The operations and / or methods described are the same as or similar to those of the point cloud encoder (e.g., surface approximation analyzer 30003). Details and references are available. Figure 1 to Figure 6 The descriptions are the same.

[0165] According to an embodiment, the intra / inter-frame coding processor 8005 can perform intra / inter-frame coding on point cloud data. The intra / inter-frame coding processor 8005 can perform coding that is the same as or similar to intra / inter-frame coding. According to an embodiment, the intra / inter-frame coding processor 8005 may be included in an arithmetic encoder 8006.

[0166] The arithmetic encoder 8006 according to the embodiment performs entropy encoding on the octree and / or approximate octree of point cloud data. For example, the encoding scheme includes arithmetic encoding. The arithmetic encoder 8006 performs the same or similar operations and / or methods as the arithmetic encoder 30004.

[0167] The metadata processor 8007 according to an embodiment processes metadata (e.g., set values) about point cloud data and provides it to necessary processing procedures such as geometric encoding and / or attribute encoding. Additionally, the metadata processor 8007 according to an embodiment can generate and / or process signaling information related to geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be encoded separately from the geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be interleaved.

[0168] The color transformation processor 8008, attribute transformation processor 8009, prediction / boosting / RAHT transformation processor 8010, and arithmetic encoder 8011 perform attribute encoding. Attribute encoding and reference according to the implementation method. Figure 1 to Figure 6 The attribute codes described are the same or similar, so their detailed descriptions are omitted.

[0169] According to the implementation, the color transformation processor 8008 performs color transformation encoding to transform color values ​​included in attributes. The color transformation processor 8008 can perform color transformation encoding based on reconstructed geometry. The reconstructed geometry and reference... Figure 1 to Figure 9 The description is the same. Furthermore, its execution is the same as the reference. Figure 3 The operation and / or methods of the described color converter 30006 are the same as or similar to those described. Detailed descriptions are omitted.

[0170] According to the implementation, the attribute transformation processor 8009 performs attribute transformation to transform attributes based on reconstructed geometry and / or locations where geometric encoding is not performed. The attribute transformation processor 8009 performs transformations with reference to... Figure 3 The operation and / or method of the described attribute transformer 30007 are the same as or similar to those described. Detailed descriptions are omitted. The prediction / boosting / RAHT transformation processor 8010 according to the embodiment can encode the transformed attributes using any one or a combination of RAHT encoding, prediction transformation encoding, and boosting transformation encoding. The prediction / boosting / RAHT transformation processor 8010 performs and references... Figure 3 The RAHT transformer 30008, LOD generator 30009, and boost transformer 30010 described herein operate at least one of the same or similar operations. Furthermore, the predictive transform coding, boost transform coding, and RAHT transform coding are similar to those of the reference transformer. Figure 1 to Figure 9 The descriptions are the same, so their detailed descriptions are omitted.

[0171] The arithmetic encoder 8011 according to the embodiment can encode the attributes of the code based on arithmetic encoding. The arithmetic encoder 8011 performs the same or similar operations and / or methods as the arithmetic encoder 300012.

[0172] The transmission processor 8012 according to an embodiment can transmit individual bitstreams containing encoded geometric and / or encoded attribute and metadata information, or transmit a single bitstream containing encoded geometric and / or encoded attribute and metadata information. When the encoded geometric and / or encoded attribute and metadata information according to an embodiment is configured as a single bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to an embodiment may include signaling information and slice data. The signaling information includes a sequence parameter set (SPS) for sequence-level signaling, a geometric parameter set (GPS) for geometric information encoded signaling, an attribute parameter set (APS) for attribute information encoded signaling, and a tile parameter set (TPS) for tile-level signaling. Slice data may include information about one or more slices. A slice according to an embodiment may include a geometric bitstream Geom0. 0 and one or more attribute bitstreams Attr00 and Attr1 0 .

[0173] A slice is a series of syntax elements that represent all or part of a coded point cloud frame.

[0174] The TPS according to an embodiment may include information about individual tiles in one or more tiles (e.g., coordinate information and height / size information about the bounding box). The geometric bitstream may include a header and a payload. The header of the geometric bitstream according to an embodiment may include a geom_parameter_set_id, a geom_tile_id, and a geom_slice_id included in the GPS, as well as information about the data included in the payload. As described above, the metadata processor 8007 according to an embodiment may generate and / or process signaling information and send it to the transmission processor 8012. According to an embodiment, the element performing geometry encoding and the element performing attribute encoding may share data / information with each other, as shown by the dashed lines. The transmission processor 8012 according to an embodiment may perform the same or similar operations and / or transmission methods as the transmitter 10003. Details and References Figure 1 and Figure 2 The descriptions are the same, so their descriptions are omitted.

[0175] Figure 9 An example of a receiving device according to an embodiment is shown.

[0176] Figure 9 The receiving device shown is Figure 1 Example of receiving device 10004. Figure 9 The receiving device shown can perform the same operation as the reference. Figure 1 to Figure 8 The same or similar one or more operations and methods described in the point cloud decoder.

[0177] The receiving apparatus according to the embodiments may include a receiver 9000, a receiving processor 9001, an arithmetic decoder 9002, an octree reconstruction processor based on occupancy codes 9003, a surface model processor (triangle reconstruction, upsampling, voxelization) 9004, an inverse quantization processor 9005, a metadata parser 9006, an arithmetic decoder 9007, an inverse quantization processor 9008, a prediction / boost / RAHT inverse transform processor 9009, a color inverse transform processor 9010, and / or a renderer 9011. Each decoding element according to the embodiments can perform the inverse process of the operation of the corresponding encoding element according to the embodiments.

[0178] Receiver 9000, according to the embodiment, receives point cloud data. Receiver 9000 can perform operations related to...Figure 1 The operation and / or receiving method of the receiver 10005 are the same as or similar to those of the receiver. Detailed description omitted.

[0179] The receiving processor 9001 according to the embodiment can acquire geometric bitstreams and / or attribute bitstreams from the received data. The receiving processor 9001 may be included in the receiver 9000.

[0180] Arithmetic decoder 9002, octet-based octree reconstruction processor 9003, surface model processor 9004, and inverse quantization processor 9005 can perform geometric decoding. Geometric decoding and reference according to the implementation method. Figure 1 to Figure 8 The described geometric decodings are the same or similar, so their detailed descriptions are omitted.

[0181] The arithmetic decoder 9002 according to the embodiment can decode geometric bitstreams based on arithmetic coding. The arithmetic decoder 9002 performs the same or similar operations and / or encodings as the arithmetic decoder 7000.

[0182] According to an embodiment, the octree reconstruction processor 9003 based on occupancy codes can reconstruct an octree by obtaining occupancy codes from the decoded geometric bitstream (or information about the geometry obtained as a decoding result). The octree reconstruction processor 9003 performs operations and / or methods identical or similar to those of the octree synthesizer 7001 and / or the octree generation method. When applying triad geometry encoding, the surface model processor 9004 according to an embodiment can perform triad geometry decoding and related geometric reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on surface model methods. The surface model processor 9004 performs operations identical or similar to those of the surface approximation synthesizer 7002 and / or the geometry reconstructor 7003.

[0183] The inverse quantization processor 9005 according to the implementation method can inverse quantize the decoded geometry.

[0184] The metadata parser 9006 according to the implementation can parse metadata (e.g., setting values) contained in the received point cloud data. The metadata parser 9006 can pass the metadata to geometry decoding and / or attribute decoding. Metadata and reference Figure 8 The metadata described is the same, so its detailed description is omitted.

[0185] Arithmetic decoder 9007, inverse quantization processor 9008, prediction / boost / RAHT inverse transform processor 9009, and color inverse transform processor 9010 perform attribute decoding. Attribute decoding and reference Figure 1 to Figure 8 The attributes described by at least one of them are identical or similar, so their detailed descriptions are omitted.

[0186] The arithmetic decoder 9007 according to the embodiment can decode the attribute bitstream via arithmetic coding. The arithmetic decoder 9007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 9007 performs the same or similar operations and / or encodings as the arithmetic decoder 7005.

[0187] The inverse quantization processor 9008 according to the embodiment can inverse quantize the decoded attribute bitstream. The inverse quantization processor 9008 performs the same or similar operations and / or methods as the inverse quantizer 7006.

[0188] The predictive / boost / RAHT inverse transform processor 9009, according to the implementation, can handle the reconstructed geometry and inverse quantization properties. The predictive / boost / RAHT inverse transform processor 9009 performs operations related to... Figure 7 The operation and / or decoding of the RAHT transformer 7007, LOD generator 7008, and / or inverse booster 7009 are the same as or similar to one or more operations and / or decodings. The color inverse transform processor 9010 according to the embodiment performs inverse transform encoding to inverse transform the color values ​​(or textures) included in the decoded attributes. The color inverse transform processor 9010 performs operations similar to... Figure 7 The operation and / or inverse transform encoding of the color inverse transformer 7010 are the same as or similar to the operation and / or inverse transform encoding. The renderer 9011 according to the embodiment can render point cloud data.

[0189] Figure 10 An exemplary structure according to an embodiment is shown that can be operated in conjunction with a point cloud data transmission / reception method / apparatus.

[0190] Figure 10 The structure represents a configuration in which at least one of the following components—server 1060, robot 1010, autonomous vehicle 1020, XR device 1030, smartphone 1040, home appliance 1050, and / or head-mounted display (HMD) 1070—is connected to cloud network 1000. Robot 1010, autonomous vehicle 1020, XR device 1030, smartphone 1040, or home appliance 1050 are referred to as devices. Furthermore, XR device 1030 may correspond to a point cloud data (PCC) device according to an embodiment or be operatively connected to a PCC device.

[0191] Cloud Network 1000 can refer to a network that forms part of or exists within a cloud computing infrastructure. Here, Cloud Network 1000 can be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

[0192] Server 1060 can be connected via cloud network 1000 to at least one of robot 1010, self-driving vehicle 1020, XR device 1030, smartphone 1040, home appliance 1050 and / or HMD 1070, and can assist at least a portion of the processing of the connected devices 1010 to 1070.

[0193] HMD 1070 represents one of the implementation types of an XR device and / or PCC device according to an embodiment. An HMD-type device according to an embodiment includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0194] Hereinafter, various embodiments of the apparatus 1010 to 1050 that apply the above-described technology will be described. Figure 10 The devices 1010 to 1050 shown can be operatively connected to / coupled to the point cloud data transmitting and receiving devices according to the above embodiments.

[0195] <PCC+XR>

[0196] The XR / PCC device 1030 can employ PCC technology and / or XR (AR+VR) technology, and can be implemented as an HMD, a head-up display (HUD) installed in a vehicle, a television, a mobile phone, a smartphone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a stationary robot, or a mobile robot.

[0197] The XR / PCC device 1030 can analyze 3D point cloud data or image data acquired through various sensors or from external devices and generate positional and attribute data about 3D points. Thus, the XR / PCC device 1030 can acquire information about the surrounding space or real-world objects and render and output XR objects. For example, the XR / PCC device 1030 can match an XR object, including auxiliary information about the identified object, with the identified object and output the matched XR object.

[0198] <PCC+Mobile Phone>

[0199] The XR / PCC device 1030 can be implemented as a smartphone 1040 by applying PCC technology.

[0200] The 1040 smartphone can decode and display point cloud content based on PCC technology.

[0201] <PCC+Autonomous Driving+XR>

[0202] The self-driving vehicle 1020 can be realized as a mobile robot, vehicle, unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0203] The self-driving vehicle 1020 employing XR / PCC technology can refer to a self-driving vehicle equipped with means for providing XR images, or a self-driving vehicle serving as a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, the self-driving vehicle 1020 can be distinguished from and operatively connected to the XR device 1030.

[0204] The autonomous vehicle 1020, equipped with means for providing XR / PCC images, can acquire sensor information from sensors including cameras and output generated XR / PCC images based on the acquired sensor information. For example, the autonomous vehicle 1020 may have a HUD and output XR / PCC images to it, thereby providing passengers with XR / PCC objects corresponding to real objects or objects presented on the screen.

[0205] When an XR / PCC object is output to a HUD, at least a portion of the XR / PCC object can be output to overlap with the actual object being pointed at by the passenger's eyes. Conversely, when an XR / PCC object is output to a display installed within the autonomous vehicle, at least a portion of the XR / PCC object can be output to overlap with objects on the screen. For example, the autonomous vehicle 1220 can output XR / PCC objects corresponding to objects such as roads, other vehicles, traffic lights, traffic signs, two-wheeled vehicles, pedestrians, and buildings.

[0206] Virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or point cloud compression (PCC) technologies according to the implementation methods are applicable to various devices.

[0207] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. AR technology, on the other hand, refers to the technology of displaying virtually created CG images on top of images of real objects. MR technology is similar to AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology differs from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with the same characteristics as real objects. More specifically, an example of MR technology application is holographic services.

[0208] Recently, VR, AR, and MR technologies have often been referred to as extended reality (XR) technologies rather than being explicitly distinguished from each other. Therefore, embodiments of this disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are suitable for such technologies.

[0209] The PCC method / apparatus according to the embodiments can be applied to vehicles that provide autonomous driving services.

[0210] Vehicles providing autonomous driving services connect to the PCC device for wired / wireless communication.

[0211] When the point cloud data (PCC) transmitting / receiving device according to the embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services (which may be provided together with autonomous driving services) and transmit it to the vehicle. When the PCC transmitting / receiving device is installed in the vehicle, it can receive / process content data related to AR / VR / PCC services based on user input signals input through a user interface device and provide it to the user. The vehicle or user interface device according to the embodiment can receive user input signals. User input signals according to the embodiment may include signals indicating autonomous driving services.

[0212] The point cloud data transmission method / apparatus according to the implementation method can be interpreted as referring to... Figure 1 The transmitting device 10000, the point cloud video encoder 10002, and the transmitter 10003, Figure 2 Acquisition 20000 - Encoding 20001 - Transmission 20002 Figure 3 Point cloud video encoder, Figure 8 The transmitting device Figure 10 The device Figure 15 Encoding methods Figure 17 The transmitting device Figure 39 Terms related to sending methods, etc.

[0213] The point cloud data receiving method / apparatus according to the embodiments can be interpreted as referring to... Figure 1 The receiving device 10004, receiver 10005, and point cloud video decoder 10006. Figure 2 Transmission 20002 - Decoding 20003 - Rendering 20004 Figure 7 Point cloud video decoder, Figure 9 The receiving device Figure 10 The device Figure 16 Decoding methods Figure 18 The receiving device Figure 40 Terms related to receiving methods, etc.

[0214] The point cloud data transmission / reception method / apparatus according to the embodiments can be simply referred to as the method / apparatus according to the embodiments.

[0215] According to the implementation method, geometric data, geometric information, and location information constituting point cloud data are interpreted as having the same meaning. Attribute data and attribute information constituting point cloud data are also interpreted as having the same meaning.

[0216] The method / apparatus according to the implementation can process point cloud data while taking scalable transmission into account.

[0217] Regarding the method / apparatus according to the embodiments, this document discloses a method for efficiently supporting selective decoding of partial data based on receiver performance or transmission speed when transmitting / receiving point cloud data. Specifically, this disclosure proposes techniques for improving the efficiency of scalable coding, wherein the encoder at the transmitting end can selectively deliver information required by the decoder at the receiving end for compressed data, and the decoder can decode it.

[0218] Specifically, this disclosure proposes a method for improving the efficiency of scalable coding in point cloud data compression methods. Here, scalable coding is a technique for gradually changing the resolution of data based on receiver requests, processing speed, performance, transmission bandwidth, etc., so that compressed data can be efficiently delivered on the sending side and decoded on the receiving side. Therefore, it can be applied in conjunction with the techniques of this disclosure for efficiently delivering layer-configured point cloud data packets.

[0219] Reference Figure 3 and Figure 7 The illustrated implementation of a point cloud data transmitting / receiving apparatus (which may be simply referred to as an encoder / decoder) shows point cloud data composed of a set of points. Each point includes geometric information (or geometry or geometric data) and attribute information (or attributes or attribute data). The geometric information is the three-dimensional position information (x, y, z) of each point. That is, the position of each point is represented by parameters in a coordinate system representing three-dimensional space (e.g., parameters (x, y, z) representing the three axes of space (e.g., the X-axis, Y-axis, and Z-axis)). The attribute information represents the point's color (RGB, YUV, etc.), reflectivity, normal vector, transparency, etc. In point cloud compression (PCC), octree-based compression is performed to efficiently compress non-uniform distributions in three-dimensional space, and the attribute information is compressed according to the octree-based compression. Figure 3 and Figure 7 The point cloud video encoder and point cloud video decoder shown can be processed by corresponding components according to the implementation method.

[0220] In this context, scalable transmission and scalable decoding can be used separately, depending on the purpose. According to one implementation, scalable transmission can be used so that the transmitting / receiving device can select information up to a specific layer without involving the decoder. According to another implementation, scalable decoding can be used to select a specific layer during encoding. That is, scalable transmission can support the selection of necessary information in a compressed state without involving the decoder (i.e., at the bitstream stage), allowing the transmitting or receiving device to determine a specific layer. On the other hand, in the case of scalable decoding, during encoding / decoding, encoding / decoding can be supported only for information up to the necessary portion. Therefore, scalable decoding can be used for operations such as scalable representation.

[0221] In this case, the layer configuration for scalable transmission can differ from the layer configuration for scalable decoding.

[0222] Figure 11 (a) and Figure 11 (b) illustrates a geometric coding layer structure according to an implementation method. Specifically, Figure 11 (a) illustrates an example of three slices generated by the layer group structure at the encoder on the transmitter side, and Figure 11 (b) illustrates an example of the output of partial decoding using two slices at the decoder on the receiving side.

[0223] When fine-grained slicing (FGS) is enabled, a G-PCC bitstream can be sliced ​​into multiple sub-bitstreams. Here, fine-grained slicing can be referred to as layer-group-based slicing. To effectively utilize the layered structure of G-PCC, each slice can include encoded data from a portion of the coding layer or a portion of the region. By using segmentation or partitioning of slices paired with the coding layer structure, scalable transmission or spatial random access use cases can be supported in an efficient manner.

[0224] Layer group based slice segment

[0225] In fine-grained slicing, each slice fragment can contain data encoded from a layer group defined below.

[0226] A layer group can be defined as a set of consecutive tree layers, wherein the start depth and end depth of the tree layer group can be any number in the tree depth, and the start depth can be less than the end depth. The order of data encoded in a slice fragment can be the same as the order of data encoded in a single slice. In this disclosure, a tree layer can be referred to as a tree level.

[0227] For example, consider as Figure 11The geometric coding layer structure shown in (a) has eight coding layers, with three layer groups, each matching a different slice. More specifically, layer group 1 matches slice 1 for coding layers 0 through 4, layer group 2 matches slice 2 for coding layer 5, and layer group 3 matches slice 3 for coding layers 6 and 7. When the first two slices (i.e., slice 1 and slice 2) are sent or selected, the decoded output will be a portion of layers 0 through 5, as shown in (a). Figure 11 As shown in (b). By using slices in the layer group structure, partial decoding of the coding layer can be supported without accessing the entire bitstream.

[0228] According to the embodiments, bitstream and point cloud data can be generated from slices based on coding layers. By slicing the bitstream at the ends of the coding layers during the encoding process, the method / apparatus according to the embodiments can select relevant slices, thereby supporting scalable transmission or partial decoding.

[0229] Figure 11 (a) illustrates a geometric coding layer structure with eight layers (also called levels), where each slice corresponds to a layer group. Layer group 1 includes coding layers 0 through 4. Layer group 2 includes coding layer 5. Layer group 3 is a group for coding layers 6 and 7. When the geometry (or attribute) has a tree structure comprising eight levels (depths), the bitstream can be configured hierarchically by grouping data corresponding to one or more levels (depths). Each group can be included in a slice.

[0230] Figure 11 (b) shows the output of the decoded data when two slices are selected from three slices. When the decoder selects group 1 and group 2, it selects partial layers of the tree (depth) from 0 to 5. That is, partial decoding of the coding layers can be supported by using slices of the layer group structure, even without accessing the entire bitstream.

[0231] For the partial decoding process according to the embodiment, the encoder can generate three slices based on the layer group structure. The decoder according to the embodiment can select two slices from the three slices and perform partial decoding.

[0232] The bitstream according to an embodiment may include layer-based slices. Each slice may include a header containing signaling information related to the point cloud data (i.e., geometric data and / or attribute data) included in the slice. The receiving method / apparatus according to an embodiment may select slices and decode the point cloud data contained in the payload of a slice based on the header included in the selected slice.

[0233] In addition to the layer group structure, the method / apparatus according to the implementation can also divide the layer group into multiple subgroups, taking into account the use case of spatial random access. Subgroups are mutually exclusive, and the set of subgroups can be the same as the layer group. Since the points of each subgroup form the boundary in the spatial domain, the subgroup can be represented by subgroup bounding box information. Based on spatial information, the layer group and subgroup structure can support access to the Region of Interest (ROI) by selecting slices that cover the ROI. By efficiently comparing the ROI with the bounding box information about each slice, spatial random access within a frame or tile can be supported.

[0234] The method / apparatus according to the embodiments can be as follows: Figure 11 The configuration shown in (a) is used to deliver slices of point cloud data.

[0235] According to the implementation method, the entire encoded bitstream can be included in a single slice. For multiple slices, each slice can contain sub-bitstreams. The order of the slices can be the same as the order of the sub-bitstreams. Furthermore, each slice can match a layer group in a tree structure.

[0236] Furthermore, slicing may not affect previous slices, just as higher levels in a geometric tree do not affect lower levels.

[0237] The segmented slicing method described in this implementation is efficient in terms of error robustness, effective transmission, and support for regions of interest.

[0238] 1) Error recovery

[0239] Compared to a single-slice structure, segmented slicing may be more robust to errors. In other words, when a slice contains the entire bitstream of a frame, data loss can affect the entire frame's data. On the other hand, when the bitstream is segmented into multiple slices, it is possible to decode at least one slice that is unaffected by the loss even if at least one slice is lost.

[0240] 2) Scalable transmission

[0241] This disclosure can support multiple decoders with different capabilities.

[0242] When encoded point cloud data (i.e., point cloud compression (PCC) bitstream) is contained in a single slice, the Level of Detail (LOD) of the encoded point cloud data can be determined before encoding. Therefore, multiple pre-coded bitstreams of point cloud data with different resolutions can be sent independently, which may be inefficient in terms of large bandwidth or storage space.

[0243] When encoded point cloud data (i.e., PCC bitstream) is contained in segmented slices, a single bitstream can support different levels of decoders. From the decoder's perspective, the receiving device can select the target layer and deliver a portion of the selected bitstream to the decoder. Similarly, by using a single bitstream without splitting the entire bitstream, the transmitting device can efficiently generate partial bitstreams.

[0244] 3) Region-based spatial scalability

[0245] In terms of the G-PCC requirements according to the implementation method, region-based spatial scalability can be defined as follows: the compressed bitstream consists of one or more layers, so a particular ROI can have a higher density with additional layers, and layers can be predicted from lower layers.

[0246] To support this requirement, it is necessary to support region-by-region detailed representations. For example, in VR / AR applications, distant objects can be represented with lower precision, while closer objects can be represented with higher precision. Furthermore, the decoder can increase the resolution of the region of interest upon request. This operation can be implemented using geometric octrees and scalable structures of G-PCC (e.g., scalable attribute encoding schemes).

[0247] According to one implementation, the decoder should access the entire bitstream based on the current slice structure containing the entire geometry or attributes, which may lead to inefficiencies in bandwidth, memory, and decoder performance. On the other hand, when the bitstream is segmented into multiple slices and each slice contains sub-bitstreams according to scalable layers, the decoder according to one implementation can efficiently select slices as needed before parsing the bitstream.

[0248] The method / apparatus according to the implementation can use a tree structure (or layer structure) of point cloud data to generate layer groups.

[0249] Reference Figure 11 (a) serves as an example. In a geometrically encoded hierarchical structure (e.g., an octree structure), there are eight layers (also called levels), and three slices can be used to contain one or more layers. A group represents a group of layers. When using scalable attribute encoding, the tree structure is the same as the geometric tree structure. The same octree-slice mapping can be used to create attribute slice fragments.

[0250] According to the implementation method, a layer group represents a bundle of layer structure units generated in G-PCC encoding, such as an octree layer or a LoD layer.

[0251] A subgroup can be represented as a set of neighboring nodes within a layer group. For example, it can be configured as a set of nodes that are adjacent to each other in Morton code order, a set of neighboring nodes based on distance, or a set of neighboring nodes based on coding order. Nodes in a parent-child relationship can also exist within a subgroup.

[0252] When defining subgroups, the boundary appears in the middle of the layer, and parameters such as `entropy_continuation_enabled_flag` can be signaled to indicate whether entropy continuation should be maintained at the boundary. Continuation can also be maintained by referencing a previous slice using `ref_slice_id`.

[0253] The tree structure according to the embodiments may be an octree structure, and the attribute layer structure or attribute encoding tree according to the embodiments may include a level of detail (LOD) structure. In other words, the tree structure for point cloud data may include layers corresponding to depth or level, and the layers may be grouped. This disclosure refers to this as a layer group.

[0254] The method / apparatus according to the implementation (e.g., Figure 3 Octree analyzer 30002 or LOD generator 30009 Figure 7 The octree synthesizer 7002 or LOD generator 7008 can generate octree structures of geometry or LOD tree structures of attributes. Furthermore, point cloud data can be grouped based on layers of the tree structure.

[0255] Reference Figure 11 (a) Multiple layers (also called levels) are divided into first to third groups (also called layer groups). Groups can be further divided to form subgroups.

[0256] According to the implementation, a slice may contain encoded data from a layer group. Here, a layer group is defined as a set of consecutive tree layers. The start depth and end depth of a tree layer can be specific numbers in the tree depth, where the number for the start is less than the number for the end. In this case, the tree layer can be referred to as a tree level or tree depth.

[0257] Although Figure 11 (a) illustrates a geometric coding layer structure as an example of a tree structure, but coding layer structures for attributes can also be created in a similar manner.

[0258] Figure 12 This is a diagram illustrating the layer and subgroup structure according to the implementation method.

[0259] Reference Figure 12 Point cloud data and bit streams can be distinguished and represented by bounding boxes.

[0260] exist Figure 12The diagram illustrates the subgroup structure and the corresponding bounding boxes for each subgroup. Layer group 2 is divided into two subgroups (group 2-1 and group 2-2) contained in different slices, and layer group 3 is divided into four subgroups (group 3-1, group 3-2, group 3-3, and group 3-4) contained in different slices. Given the slice and bounding box information for the layer groups and subgroups, spatial access can be performed by: 1) comparing the bounding box of each slice with the ROI, 2) selecting slices where the subgroup bounding boxes overlap with the ROIs, and 3) decoding the selected slices.

[0261] When considering the ROI in region 3-3, slices 1, 3, and 6 are selected as layer group 1 covering the ROI, and the subgroup bounding boxes of subgroups 2-2 and 3-3. For efficient spatial access, it is assumed that there are no dependencies between subgroups from the same layer group. In real-time streaming or low-latency use cases, time efficiency can be improved by performing selection and decoding upon receiving each slice fragment.

[0262] When encoding geometry and / or attributes, the method / apparatus according to the implementation can present the data as a tree 2200 consisting of layers (which may be referred to as depth, level, etc.). Point cloud data corresponding to layers (depth / level) can be grouped into layer groups (or groups) 45000. Layer group 2 can be further divided (segmented) into two subgroups 45002, and layer group 3 can be further divided (segmented) into four subgroups 45003. Each subgroup can be configured as a slice to generate a bitstream.

[0263] The receiving apparatus according to the embodiment can receive a bitstream, select a specific slice from the bitstream, and decode the bounding boxes corresponding to the subgroups included in the selected slice. For example, when slice 1 is selected, the bounding box 45004 corresponding to layer group 1 can be decoded. Layer group 1 may be data corresponding to the largest region. When additionally displaying detailed regions for layer group 1, the method / apparatus according to the embodiment can select slice 3 and / or slice 6, and access the bounding boxes (point cloud data) of subgroups 2-2 and / or subgroups 3-3 of detailed regions included in the region of layer group 1 in a layered and partial manner.

[0264] use Figure 12 The encoding and decoding of point cloud data of layers and subgroups can be performed by Figure 1 Transmitting / receiving devices Figure 2 Encoding and decoding, Figure 3 Transmission device / method Figure 7 Receiving device / method Figure 8 and Figure 9 Transmitting / receiving device / method Figure 10 The device Figure 14 ,Figure 15 or Figure 17 Sending / receiving methods Figure 16 and Figure 18 Transmitting / receiving devices and Figure 39 and Figure 40 The send / receive methods are executed.

[0265] Figure 13 (a) to Figure 13 (c) illustrates a representation of layer-based point cloud data according to an implementation method.

[0266] The apparatus / method according to the embodiments can provide efficient access to large-scale or dense point cloud data through layer slicing based on scalability and spatial access capabilities. Due to the large number of points and the large data size, point cloud data can take a significant amount of time to render or display content. Therefore, as an alternative approach, the level of detail can be adjusted based on the viewer's interest. For example, structural or global region information is more important than local details when the viewer moves away from the scene or object. On the other hand, detailed information about the ROI is needed when the viewer moves closer to a specific region or object. Using an adaptive approach, the renderer according to the embodiments can efficiently provide the viewer with data of sufficient quality. Figure 13 (a) to Figure 13 (c) illustrates an example of increased detail across three viewing distance levels based on changes in ROI.

[0267] Figure 13 The high-level view in (a) shows coarse details. Figure 13 The medium-level view in (b) shows medium-level details, and Figure 13 The low-level view in (c) shows fine-grained details.

[0268] Figure 14 An example of a point cloud data transmission / reception apparatus / method according to an embodiment is shown.

[0269] When layer group slices are used to generate G-PCC bitstreams, multi-resolution ROIs can be supported.

[0270] Reference Figure 14 Multi-resolution ROIs can be supported through the scalability and spatial accessibility of layered tiling. Figure 14In the transmitting side, encoder 47001 can generate bitstream slices of spatial subgroups or octree layer groups for each layer group. Upon request, slices matching the ROI at each resolution are selected and sent to the receiving side. Compared to a tile-based method, the overall bitstream size is reduced because it does not include details beyond the requested ROI. On the receiving side, decoder 47004 can combine slices to produce three outputs, for example: 1) a high-level view output from the layer group; 2) a mid-level view output from selected subgroups of layer groups 1 and 2; and 3) a low-level view output with good quality detail from selected subgroups of layer groups 2 and 3, and layer group 1. The outputs can be generated progressively, thus allowing the receiver to provide a viewing experience such as scaling, which progressively increases the resolution from the high-level view to the low-level view.

[0271] Encoder 47001 may correspond to a geometric encoder and / or an attribute encoder as a point cloud encoder according to an embodiment. The encoder may slice the point cloud data based on layer groups (or subgroups). Layers may be referred to as the depth of a tree, the level of LOD, etc. As shown in section 47002, the depth of the geometric octree and / or the level of the attribute layer may be divided into layer groups (or subgroups).

[0272] The slice selector 47003, in conjunction with the encoder 47001, can select segmented slices (or sub-slices) and selectively and partially send data such as layer group 1 to layer group 3.

[0273] Decoder 47004 can decode point cloud data that is selectively and partially transmitted. For example, decoder 47004 can decode layer group 1 (which has a high depth / layer / level or index 0, or is close to the root) for a high-level view. Furthermore, decoding can be performed for a mid-level view by incrementing the depth / level index only on layer group 1 based on layer group 1 and layer group 2. Additionally, decoding can be performed based on layer groups 1 through 3 for a low-level view.

[0274] Reference Figure 14The encoder 47001 according to the embodiment can receive point cloud data as input and slice it into layer groups. That is, the point cloud data can be structured hierarchically and divided into layer groups. The hierarchical structure can refer to an octree structure or a level of detail (LoD). Part 47002 represents the point cloud data divided into layer groups. The slice selector 47003 can select a layer group (or corresponding slice) and send the selected slice to the decoder 47004 on the receiving side. The decoder 47004 can reconstruct only layer group 1, reconstruct layer group 1 and layer group 2, or reconstruct all received layer groups by combining the received slices according to the user's needs. The layer groups are hierarchical and have different levels of detail. Reconstructing only layer group 1 can result in a wider range of reconstruction and coarse detail. Reconstructing all layer groups 1 to 3 can result in a narrower range of reconstruction and fine detail.

[0275] As described above, the input to the encoder used for layer group slicing consists of point cloud data and parameter information describing the structural information about the layer group slices (e.g., Sequence Parameter Set (SPS), Geometric Parameter Set (GPS), and a list of layer group slices or a list of layer group structures (LGSI)). At the beginning of each tree depth, the layer group is determined using the layer group structure parameters to determine the layer group for the target tree depth. When using the layer group index, the subgroup bounding box is used to determine the subgroup index for each node. When the subgroup for a node changes, the context state and buffers used in the previous subgroup encoder are stored, and the context state and buffers of the current subgroup encoder are loaded. Using a separate encoder for each subgroup allows the context state to persist within the subgroup. Furthermore, to restrict neighboring nodes to belong to the same subgroup as the current node, the geometric occupancy map is updated by considering subgroup boundaries above the map boundaries. These two methods enable each encoded bitstream to be decoded independently by the decoder without node information from neighboring subgroups. This process is performed recursively for all nodes at all tree depths. Upon reaching the end of the node at the target tree depth, a fine-grained slice is generated that matches the subgroups of each layer group one-to-one. In this disclosure, fine-grained slicing may be referred to as layer-group-based slicing.

[0276] In other words, parameters describing the layer group structure are signaled at different levels. Common structural information is described in the SPS, while details of each layer group or subgroup are signaled in the slice header. Additionally, a Layer Group Structure List (LGSI) and associated slice headers are introduced to describe the overall layer group structure. The definition of G-PCC slices and the signaling methods used for fine-grained slicing are given below.

[0277] A slice can be defined as a set of points encoded by an independent FGS and zero or more associated FGS.

[0278] A related FGS can be defined as a slice data unit that depends on a previous data unit within the same slice.

[0279] An independent FGS can be defined as the first geometric data unit of a slice.

[0280] Additionally, FGS is enabled in SPS.

[0281] The basic information used for decoding the relevant FGS is carried in the relevant data unit header. The relevant data unit header contains context inheritance information and slice-specific bounding box information.

[0282] Additionally, a Layer Group Structure List (LGSI) can be defined to describe the relationships between FGS.

[0283] The encoder's encoding operation can be performed by Figure 1 The transmitting device Figure 2 encoding, Figure 3 Transmission device / method Figure 8 and Figure 9 Transmission device / method Figure 10 The device Figure 15 Encoding methods Figure 17 The transmitting device or Figure 39 It is performed by at least one or a combination of the sending methods.

[0284] The decoding operation of the decoder can be performed by Figure 1 The receiving device Figure 2 Decoding Figure 7 Receiving device / method Figure 8 and Figure 9 Receiving device / method Figure 10 The device Figure 16 Decoding methods Figure 18 The receiving device or Figure 40 It is performed by at least one or a combination of the receiving methods.

[0285] Figure 15 This is a flowchart illustrating an example of an encoding method of an encoder according to an implementation method. That is, Figure 15 An example of an encoding process for layer group slicing is shown, which enables the encoded bitstream to be decoded independently by the decoder without the need for node information from neighboring subgroups. Figure 15 The encoding method can be determined by Figure 1 The transmitting device Figure 2 encoding, Figure 3 Transmission device / method Figure 8 Transmission device / method Figure 10 The device Figure 17 The transmitting device or a combination thereof is used to perform this action.

[0286] The encoder, according to the implementation method, generates parameter information such as SPS, GPS, and LGSI, and determines layer groups and subgroups (S2501, S2502). Then, it checks if the subgroup has changed (S2503), and whenever a subgroup changes, it stores information for encoding the point cloud data within the subgroup and loads necessary information to perform efficient encoding (S2504). For example, context information can be stored and then loaded for use during the next encoding. That is, when the subgroup for a node changes, the context state and buffer used by the previous subgroup encoder are stored, and the context state and buffer of the current subgroup encoder are loaded. Additionally, the geometric occupancy map is updated according to the encoding (S2505). Specifically, to restrict neighboring nodes to belong to the same subgroup as the current node, the geometric occupancy map is updated by considering subgroup boundaries and map boundaries. Then, the nodes of the subgroup are encoded (S2506). When this process is repeated and all nodes at all depths within the occupancy tree are encoded (S2507, S2508), that is, when the end of a node at the target tree depth is reached, a geometric data unit header is generated, and a geometric bitstream containing parameter information, the geometric data unit header, geometric data units, etc., is generated. In other words, fine-grained slices that match the subgroups of each layer group are produced. In this disclosure, the terms geometric occupancy map, geometric map, and map are used interchangeably and have the same meaning.

[0287] Figure 16 This is a flowchart illustrating an example of a decoding method of a decoder according to an implementation method. That is, Figure 16 A decoder process for layer group slicing according to an embodiment is illustrated.

[0288] Figure 16 You can follow Figure 15 The reverse process.

[0289] Figure 16 The decoding method can be derived from Figure 1 The receiving device Figure 2 Decoding Figure 7 Receiving device / method Figure 9 Receiving device / method Figure 10 The device Figure 18 The decoder or a combination thereof is used to execute.

[0290] According to the implementation, the decoder parses parameter information (e.g., SPS, GPS, LGSI, etc.), geometric data unit headers, and geometric data units from the received geometric bitstream. The decoding process for the layer group slice reference SW used for the first fine-grained slice (FGS) is the same as that for conventional geometric slice decoding (parameter set parsing, data unit header parsing, and data unit decoding). In this disclosure, parameter information, parameters, and parameter sets are used interchangeably and have the same meaning.

[0291] After the parsing process is executed, when layer group slicing is enabled (S2601), the next relevant geometric data unit is considered to be the FGS of the first slice (i.e., if they have the same Slice_id). Considering the context reference and node inheritance between parent (i.e., higher) child groups and sub-child groups, it is assumed that the order of FGS is sorted in ascending order according to layer_group_id and subgroup_id.

[0292] Before decoding the relevant data unit, the context state, output node, and layer group parameters of the previous slice are stored in a buffer for use in the next slice (S2602). That is, the buffer and layer group parameters are updated. After parsing the header of the relevant data unit (S2603), the parent-child group of the current subgroup is detected (S2604). According to the implementation, the parent-child group of the current subgroup is detected by finding a subgroup whose subgroup bounding box is a superset of the current subgroup bounding box. Once the parent-child group is determined, the parent (i.e., higher) node of the current relevant data unit is selected (S2605). When the parent (i.e., higher) node of the current relevant data unit is selected in operation S2605, the relevant data unit is parsed (S2606). Then, by using the selected node as the initial node of the decoding process, the current relevant data unit is decoded up to the tree layer covered by the current layer group. The decoding process of the relevant data unit is repeated until the end of the geometric bitstream (S2607). When the decoding of the relevant data unit is completed, the output point cloud is generated and the decoding process ends (S2608).

[0293] The following describes the additional process of finding the parent-child group and the parent node in more detail.

[0294] The decoder according to the implementation parses parameter information such as SPS, GPS, and LGSI contained in the bitstream. It parses the signaling information in the headers of data units included in slices such as FGS. Based on the header information, it parses the point cloud data contained in the data units. When a layer group slice is activated, the decoder updates the buffer and layer group parameters, and parses the header information of the data units related to the higher-level data units. It detects parent-child groups and selects a parent node. Then, it parses the relevant data units. The following is a detailed description of the parent-child group detection operation (S2604).

[0295] In decoding fine-grained slices (i.e., layer-group-based slices), nodes from parent-child groups are used as inputs to child-subgroups to provide continuous decoding at layer group boundaries. This can be derived using parent-child spatial relationships due to the hierarchical structure of layer group slices. That is, a child-subgroup is a subset of a parent-child group, where the bounding boxes of a child-subgroup are spatially exclusive of the bounding boxes of other child-subgroups within the same layer group.

[0296] Based on this relationship, slices with parent-child groups can be detected by finding the spatial superset of the current slice. By utilizing the subgroup_bbox_origin and subgroup_bbox_size, which are signaled in the data cell header, parent-child groups can be identified by comparing this information with the bounding box information associated with the subgroup at the previous layer group level.

[0297] The pseudocode below is an example of the parent-child group detection process described above.

[0298] pseudocode: parentLayerGroup = curLayerGroup - 1; for (i = 0; i < numSubgroups[parentLayerGroup]; i++) { if (_bboxMin[parentLayerGroup][i] <= curBboxMin && _bboxMax[parentLayerGroup][i] > curBboxMin) { parentSubgroup = i; break } } In other words, as described above, this disclosure enables the inference of parent-child groups based on spatial relationships between them without the need for additional signaling.

[0299] From the encoder's perspective, subgroups can be defined by partitioning the bounding boxes of parent and child groups. Since a subgroup is a child region of a parent group, the encoder is allowed to infer the parent-child group index using the method described above.

[0300] From the decoder's perspective, if the bounding box of the parent-child group is a superset of the bounding box of the child-child group, then the corresponding child group index is determined as the parent-child group index.

[0301] In the pseudocode above, parentLayerGroup is the index of the higher-level group associated with the current layer group.

[0302] `parentSubgroup` is the subgroup index of subgroups that have a parent-child relationship with the current subgroup.

[0303] curLayerGroup is the current layer group.

[0304] numSubgroups is the number of subgroups belonging to the layer group.

[0305] _bboxMin and _bboxMax are the minimum and maximum values ​​of the bounding box position for a specific layer group and subgroup.

[0306] curBboxMin is the minimum bounding box position of the currently coded subgroup.

[0307] The following is a detailed description of the input parent node selection operation (S2605).

[0308] In the decoding of a relevant slice, the output nodes of the parent-child group are used as the input for decoding the child-child group. If the child-child group and the child-child group have the same child bounding box, all nodes generated in the parent-child group are used. Conversely, if the child-child group's child bounding box is a subset of the parent-child group, the decoder selects the actual parent node. To identify the parent node, this disclosure compares each node of the parent-child group with the bounding box boundary of the child-child group.

[0309] The pseudocode below is an example of the parent node selection process described above. That is, only nodes within a specific range (bbox_min and bbox_max) are selected as parent nodes. For example, a node is added as a parent node only if its position (node.Pos) is greater than or equal to the minimum value (bbox_min) and less than the maximum value (bbox_max) of the bounding box.

[0310] pseudocode: for (node ​​= inNodes.begin(); node != inNodes.end(); node++) { if (node.Pos >= bbox_min && node.Pos < bbox_max) fifo.emplace_back(node); else continue? } Figure 17 Another example of a point cloud transmitting apparatus according to an embodiment is shown. Figure 17The elements of the transmitting device shown can be implemented as hardware, software, a processor connected to memory, and / or a combination thereof. That is, although not shown in the figure, Figure 17 The elements of the transmitting device can be implemented as hardware, software, firmware, or a combination thereof, comprising one or more processors or integrated circuits configured to communicate with one or more memories. One or more processors can perform the above-described... Figure 17 At least one of the operation and / or functions of the elements of the transmitting device. Additionally, one or more processors can operate or perform functions for executing... Figure 17 The software program and / or instruction set for the operation and / or function of the elements of the transmitting device. Figure 17 In this context, you can change the execution order of blocks, omit some blocks, and add new blocks.

[0311] According to an embodiment, the point cloud transmission device may include a data input unit 51001, a signaling processor 51002, a geometric encoder 51003, an attribute encoder 51004, and a transmission processor 51005. In this disclosure, the signaling processor 51002 may be referred to as a metadata generator.

[0312] The geometry encoder 51003 and the attribute encoder 51004 can perform operations related to... Figure 1 Point cloud video encoder 10002 Figure 2 The code 20001 Figure 3 Point cloud video encoder, Figure 8 Point cloud video encoder, Figure 15 encoding and Figure 39 The encoding describes some or all of the operations.

[0313] According to the embodiment, the data input unit 51001 receives or acquires point cloud data. The data input unit 51001 can perform... Figure 1 Point cloud video acquisition unit 10001 or Figure 8 Some or all of the operations in the data input unit 8000.

[0314] The data input unit 51001 outputs the positions of points in the point cloud data to the geometry encoder 51003 and the attributes of the points to the attribute encoder 51004. Additionally, it outputs parameters to the signaling processor 51002. According to the implementation, the parameters can be provided to both the geometry encoder 51003 and the attribute encoder 51004.

[0315] The geometry encoder 51003 uses the positions of the input points to perform geometric compression on a per-layer-group basis. The geometry encoder 51003 performs entropy encoding on the compressed geometric information and outputs the encoded information to the transmission processor 51005 in the form of a geometric bitstream.

[0316] The geometry encoder 51003 reconstructs geometric information based on the position changed by compression and outputs the reconstructed (or decoded) geometric information to the attribute encoder 51004.

[0317] According to an implementation, the geometry encoder 51003 uses the locations of input points to construct an octree, performs layer-group-based slicing on the octree, selects one or more slices, and then compresses the geometric information associated with the selected one or more slices. The layer-group-based slicing and slice-level geometric compression according to the implementation has been referenced... Figure 11 to Figure 15 The description has been described in detail, so it will be omitted below to avoid redundancy.

[0318] The attribute encoder 51004 compresses attribute information based on positional and / or reconstructed geometric information that has not yet undergone geometric encoding. In one implementation, the attribute information can be encoded using any one or a combination of RAHT coding, LOD-based predictive transform coding, and lifting transform coding. The attribute encoder 51004 performs entropy coding on the compressed attribute information and outputs the encoded information to the transmission processor 51005 in the form of an attribute bitstream.

[0319] Signaling processor 51002 generates and / or processes signaling information required for encoding / decoding / rendering geometric and attribute information, and provides it to geometry encoder 51003, attribute encoder 51004, and / or transmission processor 51005. Alternatively, signaling processor 51002 may receive signaling information generated by geometry encoder 51003, attribute encoder 51004, and / or transmission processor 51005. Signaling processor 51002 may provide information fed back from receiving device (e.g., head orientation information and / or viewport information) to geometry encoder 51003, attribute encoder 51004, and / or transmission processor 51005.

[0320] In this disclosure, signaling information including slice-related information based on layer groups (Layer Group Structure List (LGSI)) can be signaled and transmitted on a per-parameter-set basis (SPS: Sequence Parameter Set, GPS: Geometric Parameter Set, APS: Attribute Parameter Set, TPS: Patch Parameter Set (also known as Patch List) etc.) and / or-data-unit (i.e., slice). That is, it can be signaled and transmitted per-image-encoded-unit (or compression-unit or prediction-unit) (such as slice or patch).

[0321] The transmission processor 51005 can perform operations with Figure 8 The transmission processor 8012 operates in the same or similar manner and / or transmits using the same method, or performs the same operation as the transmission processor 8012. Figure 1The operation and / or transmission method are the same as or similar to those of transmitter 10003. For details (omitted below), please refer to... Figure 1 or Figure 8 The description.

[0322] The transmission processor 51005 can multiplex the geometric bitstream output from the geometric encoder 51003, the attribute bitstream output from the attribute encoder 51004, and the signaling bitstream output from the signaling processor 51002 into a single bitstream and transmit the bitstream, or it can encapsulate the bitstream into a file or fragment for transmission. In one embodiment of this disclosure, the file may be in ISOBMFF file format.

[0323] According to the embodiments, files or fragments can be sent to a receiving device or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmission processor 51005 according to the embodiments is capable of wired / wireless communication with the receiving device via a 4G, 5G, or 6G network. Furthermore, the transmission processor 51005 can perform necessary data processing operations according to the network system (e.g., a 4G, 5G, or 6G communication network system). In addition, the transmission processor 51005 can transmit encapsulated data on demand.

[0324] According to another embodiment, the transmission processor 51005 can divide the geometric bitstream compressed by the geometric encoder 51003 and the attribute bitstream compressed by the attribute encoder 51004 into units for transmission. In this case, it can pack the bitstream into NAL units through a packing module based on hierarchical structure information (or slice-related information based on layer groups).

[0325] Figure 18 Another example of a point cloud receiving device according to an embodiment is illustrated. Figure 18 The elements of the receiving device shown can be implemented as hardware, software, a processor connected to memory, and / or a combination thereof. That is, although not shown in the figure, Figure 18 The elements of the receiving device can be implemented as hardware, software, firmware, or a combination thereof, comprising one or more processors or integrated circuits configured to communicate with one or more memories. One or more processors can perform the above-described... Figure 18 At least one of the operation and / or functions of the elements of the receiving device. Additionally, one or more processors can operate or perform functions for executing... Figure 18 The software program and / or instruction set for the operation and / or function of the elements of the receiving device. Figure 18 In this context, you can change the execution order of blocks, omit some blocks, and add new blocks.

[0326] According to an embodiment, the point cloud receiving device may include a receiving processor 61001, a signaling processor 61002, a geometry decoder 61003, an attribute decoder 61004, a geometry buffer 61005, an attribute buffer 61006, and a post-processor 61007. In this disclosure, the signaling processor 61002 may be referred to as a metadata parser.

[0327] The receiving processor 61001 according to the embodiment can receive a single bit stream, or it can receive a geometric bit stream, an attribute bit stream, and a signaling bit stream separately. When a file and / or fragment is received, the receiving processor 61001 according to the embodiment can decapsulate the received file and / or fragment and output a bit stream for it.

[0328] When a single bitstream is received (or decapsulated), the receiving processor 61001 according to the embodiment can demultiplex the geometric bitstream, attribute bitstream and / or signaling bitstream from the single bitstream, and output the demultiplexed signaling bitstream to the signaling processor 61002, the geometric bitstream to the geometric decoder 61003, and the attribute bitstream to the attribute decoder 61004.

[0329] When the geometry bit stream, attribute bit stream and / or signaling bit stream are received (or decapsulated) respectively, the receiving processor 61001 according to the embodiment can deliver the signaling bit stream to the signaling processor 61002, the geometry bit stream to the geometry decoder 61003, and the attribute bit stream to the attribute decoder 61004.

[0330] According to the embodiment, the receiving processor 61001 can distinguish geometric bitstreams and attribute bitstreams by slice, layer group, or subgroup based on the signaling information processed by the signaling processor 61002. It can deliver the geometric bitstream by slice / layer group / subgroup to the geometric decoder 61003, and the attribute bitstream by slice / layer group / subgroup to the attribute decoder 61004. For example, the input to the geometric decoder 61003 may include fine-grained slice bitstreams (or layer group-based slice bitstreams) and layer group structures.

[0331] Signaling processor 61002 can parse and process information included in signaling information (such as SPS, GPS, APS, TPS, LGSI, and metadata) from the input signaling bitstream, and can provide this information to receiving processor 61001, geometry decoder 61003, attribute decoder 61004, geometry buffer 61005, attribute buffer 61006, and post-processor 61007. In another embodiment, the signaling information included in the geometry data unit header and / or attribute data unit header can also be pre-parsed by signaling processor 61002 before decoding the corresponding slice (or subgroup) data. In this disclosure, geometry buffer 61005 can store nodes belonging to the current subgroup and / or nodes belonging to the parent-child groups of the current subgroup for geometry decoding. Furthermore, before the relevant data unit is decoded, the context state, output node, and layer group parameters of the previous slice are stored in geometry buffer 61005 for subsequent slices (S2602). Additionally, the attribute buffer 61006 may include nodes belonging to the current subgroup and / or nodes belonging to the parent-child groups of the current subgroup for attribute decoding.

[0332] According to the implementation, the signaling processor 61002 can also parse and process the information that signals notifications in the sequence parameter set, geometric parameter set and / or geometric data unit header (e.g., information related to slices based on layer groups), and provide the processed information to the geometric decoder 61003 and the geometric buffer 61005.

[0333] According to the implementation, the signaling processor 61002 can also parse and process the information signaled in the sequence parameter set, attribute parameter set and / or attribute data unit header (e.g., information related to slices based on layer groups), and provide the processed information to the attribute decoder 61004 and the attribute buffer 61006.

[0334] According to an implementation, the geometry decoder 61003 can perform operations on the compressed geometry bitstream based on signaling information (e.g., slice-related information based on layer groups). Figure 17 The geometry is reconstructed by reversing the process of the geometry encoder 51003. The geometry decoding performed by the geometry decoder 61003 when layer group slicing is enabled has already referenced... Figure 11 to Figure 13 and Figure 16 The description has been described in detail, so it will be omitted below to avoid redundancy.

[0335] The geometric information recovered (or reconstructed) by the geometry decoder 61003 is provided to the attribute decoder 61004.

[0336] The attribute decoder 61004, based on signaling information and reconstructed geometric information, performs [processing] on the compressed attribute bitstream. Figure 17The property encoder 51004 reverses the process to recover the property.

[0337] According to the implementation, the attribute decoder 61004 can perform LoD generation and NN search on a per-subgroup basis.

[0338] According to the implementation, the post-processor 61007 can reconstruct and display / render point cloud data by matching the geometric information (i.e., position) reconstructed and output by the geometry decoder 61003 with the attribute information reconstructed and output by the attribute decoder 61004.

[0339] According to another embodiment, when compressed geometric bitstreams and attribute bitstreams have been packaged into NAL units by the transmitting device and then transmitted, a NAL unit parser (not shown) can distinguish non-coded layer (non-CL) information such as signaling information (e.g., SPS, GPS, and APS) from coded layer (CL) information such as geometric NAL units / attribute NAL units. The signaling processor 61002 can identify the geometric and attribute layer structure of the point cloud data based on information delivered via SPS, GPS, or APS, and can specify the target layer to be decoded. During the NAL unit selection operation performed by the NAL unit parser, layers excluded from decoding can be discarded on a per-NAL-unit basis based on the nal_spatial_id contained in the NAL unit header. If a target geometric layer is specified, geometric NAL units with a value greater than the nal_spatial_id corresponding to the target layer can be discarded. Furthermore, geometric data and attribute data can be reconstructed from the bitstream classified based on data characteristics by the NAL unit parser by the geometric decoder 61003 and the attribute decoder 61004, respectively, and then transformed into a format for the final output by the post-processor 61007.

[0340] As described above, the fine-grained slicing according to this disclosure relates to techniques for further segmenting existing slices into multiple slices, which can provide advantages in terms of error robustness, partial decoding, progressive decoding, scalable transmission, etc.

[0341] In this case, the existing slice consists of slice geometry and zero or more slice attributes. In this disclosure, a different method than the one described above can be used to segment slices with this structure.

[0342] Figure 19 (a) to Figure 19 (d) illustrates different examples of the slicing and segmentation methods according to the implementation method.

[0343] like Figure 19 The slice shown in (a) includes slice geometry and zero or more slice attributes (e.g., two slice attributes), such asFigure 19 As illustrated in (b). In other words, a slice geometry and zero or more slice attributes (e.g., two slice attributes) together constitute a slice. Conventionally, slice geometry and slice attributes are each considered as data units (e.g., geometric data unit and attribute data unit).

[0344] According to embodiments of this disclosure, a slice geometry can be divided into multiple geometric data units (GDUs), and each slice attribute can be divided into multiple attribute data units (ADUs). Then, each data unit can be packaged into a corresponding FGS for transmission to a receiving device. For example, see... Figure 19 In (c), the slice geometry is segmented into five GDUs, the first slice attribute (slice attribute 0) is segmented into five ADUs, and the second slice attribute (slice attribute 1) is also segmented into five ADUs to configure the bitstream. The number of segmented data units (five in this example) is provided for illustrative purposes only. The number of segmented data units can be fixed or can vary depending on factors such as the number of points. Figure 19 As illustrated in (d), the segmented geometric data units and attribute data units are packaged into corresponding FGS and sent to the receiving device. That is, each segmented data unit is mapped to the corresponding FGS in a one-to-one correspondence.

[0345] In other words, when a slice includes slice geometry and one or more slice attributes, the slice geometry and one or more slice attributes can be divided into multiple geometric data units and multiple attribute data units, and each data unit can constitute an FGS.

[0346] exist Figure 19 In (d), a GDU is an independent geometric data unit corresponding to the first geometric data unit from the slice geometry segment. That is, a GDU does not depend on any other geometric data units and can be decoded independently. A DGDU is a related geometric data unit that depends on one or more previous geometric data units within the same slice geometry. That is, they cannot be decoded independently. Similarly, for slice attributes, an ADU corresponds to an independent attribute data unit, while a DADU corresponds to a related attribute data unit.

[0347] In this scenario, the transmitting device can first send the geometric data units to the receiving device, and then send the attribute data units to the receiving device. The receiving device can first decode the geometric data units, and then decode the attribute data units.

[0348] However, as Figure 19 (a) to Figure 19The example in (d) illustrates that transmitting data may introduce latency. For example, when performing partial decoding, decoding a particular DADU requires receiving all previous DGDUs. Furthermore, when two or more slice attributes exist, such as... Figure 19 As shown in (b), the delay becomes larger. This delay can cause problems in transmission environments with limited bandwidth or in environments with limited receiver buffers. In other words, sequentially delivering data units can lead to significant delays, taking into account the FGS decoding time.

[0349] As another example, when delivering a bitstream consisting of FGS (i.e., a G-PCC bitstream), progressive decoding can be performed, allowing sequential decoding and display. However, in cases where attributes are sent after geometry is sent, as described above, progressive decoding / progressive display is only available for geometry, or during attribute decoding after geometry decoding is complete.

[0350] To address this issue, this disclosure may employ a method of configuring FGS on a per-subgroup basis.

[0351] Figure 20 (a) to Figure 20 (f) illustrates another example of the slicing method according to the implementation.

[0352] because Figure 20 (a) to Figure 20 (c) and [[ (a) to ​ (c) is the same, therefore its detailed description will be omitted and refer to ​ (a) to ​ The description in (c) is as follows. That is, the slice geometry in the slice is segmented into multiple geometric data units and encoded, and each slice attribute is segmented into multiple attribute data units and encoded. Here, the multiple geometric data units can be generated by a geometry encoder (e.g., ​ The geometric encoder) encodes, and multiple partitioned attribute data units can be encoded by the attribute encoder (e.g., ​ The attribute encoder encodes the data. In this case, within the geometric data unit, the first geometric data unit can be a GDU, and the remaining geometric data units can be all DGDUs, all GDUs, or a mixture of DGDUs and GDUs. Similarly, within the attribute data unit, the first attribute data unit can be an ADU, and the remaining attribute data units can be all DADUs, all ADUs, or a mixture of DADUs and ADUs.

[0353] In one implementation, segmentation can be performed based on a layer group structure. That is, when geometry and attributes are sliced ​​based on the same layer group structure, each segmented data unit corresponds to a subgroup.

[0354] After that, as ​ As illustrated in (d), a data cell rearrangement process is performed. Specifically, the rearrangement is performed for each subgroup. For example, the data cells are rearranged in the following order: first geometric data cell of the slice geometry, first attribute data cell of the first slice attribute, first attribute data cell of the second slice attribute, second geometric data cell of the slice geometry, second attribute data cell of the first slice attribute, second attribute data cell of the second slice attribute, etc.

[0355] Then, as ​ As illustrated in (e), the rearranged data units are packaged into corresponding FGS (FGS geom, FGS attr0, FGS attr1, FGS geom, FGS attr0, FGS attr1, ...). That is, each rearranged geometric data unit constitutes an FGS geometry, and each attribute data unit constitutes an FGS attribute. In this disclosure, FGS geometry and FGS attributes may be referred to as sub-FGS. In other words, each data unit can constitute a sub-FGS. Then, as... ​ As illustrated in (f), the rearranged sub-FGSs are grouped to configure the FGS. For example, FGS0 consists of sub-FGSs (FGSgeom, FGS attr0, and FGS attr1) corresponding to the first data units (GDU, ADU, and ADU) partitioned from the slice geometry, the first slice attribute, and the second slice attribute, respectively. FGS1 consists of sub-FGSs (FGS geom, FGS attr0, and FGSattr1) corresponding to the second data units (DGDU1, DADU1, and DADU1) partitioned from the slice geometry, the first slice attribute, and the second slice attribute, respectively. As another example of this disclosure, such as ​ The rearranged data units illustrated in (d) can be packaged into an FGS for each subgroup. For example, FGS0 may include first data units (GDU, ADU, and ADU) partitioned by slice geometry, first slice attribute, and second slice attribute, respectively, and FGS1 may include second data units (DGDU1, DADU1, and DADU1) partitioned by slice geometry, first slice attribute, and second slice attribute, respectively. In other words, data units included in an FGS (e.g., GDU, ADU, and ADU) may have the same subgroup ID.

[0356] As mentioned above, ​ (a) to ​Example (f) illustrates the process of rearranging coded data units in geometric / attribute order for each subgroup. That is, when geometry and attributes are sliced ​​based on the same hierarchical structure, the geometric and attribute data corresponding to each subgroup are sent together as a bundle. In this case, the FGS can be viewed as a unit for sending all data within a subgroup, and the geometric and attribute data can be sent together as a bundle (or group or set).

[0357] In this context, an FGS can be defined similarly to a G-PCC slice. Specifically, a G-PCC slice is defined as a set of points located within a predetermined spatial region in a tile, where slice geometry and slice attributes are encoded as separate data units and grouped together by slice_id to form higher-level slice units. In this disclosure, slices are organized in a layer-group structure, and data units are encoded based on nodes corresponding to the predetermined spatial region and depth. FGS geometry and FGS attributes are grouped together to define an FGS. That is, geometric data units and one or more attribute data units sharing the same layer-group index and sub-group index can be grouped together. In this disclosure, this grouped unit is referred to as an FGS or encoded unit.

[0358] In this disclosure, the data unit may be provided by a transmission processor (e.g., ​ The transmission processor was rearranged and packaged into an FGS.

[0359] In short, such as ​ (a) to ​ As illustrated in (f), in this disclosure, geometric data units and attribute data units are rearranged according to each subgroup. Data units belonging to the same subgroup are then grouped together and packaged into an FGS. For example, the first geometric data unit of a slice geometry, the first attribute data unit of a first slice attribute, and the first attribute data unit of a second slice attribute are packaged into FGS0. Thus, an FGS includes one geometric data unit belonging to a subgroup and one or more attribute data units belonging to the same subgroup. In each FGS, the geometric data unit precedes the attribute data unit.

[0360] In other words, one FGS corresponds to one subgroup. That is, a geometric data unit and one or more attribute data units belonging to the same subgroup in the same layer group are packaged into one FGS.

[0361] Therefore, when geometric data cells and attribute data cells are sliced ​​based on the same layer groups and subgroups, they are packaged into the same FGS. In this case, the FGS is based on a layer group structure and can include data from nodes within specific regions and at specific depths within shared subgroups.

[0362] When data units are like ​ (a) to ​ When the geometry illustrated in (f) is rearranged and packaged into an FGS and then transmitted, the receiver's decoder can reconstruct the necessary geometry and properties without delay according to various use cases. In this operation, geometry reconstruction can be performed by a geometry decoder (e.g., ​ The geometry decoder performs the reconstruction, and the attribute reconstruction can be performed by the attribute decoder (e.g., ​ The attribute decoder performs this process. For example, in partial decoding, geometry and attributes are provided as a set, so it can be determined whether geometric data units are necessary, and the determination result can be immediately applied to the corresponding attribute data units. Furthermore, since geometric data units and attribute data units are sent consecutively, such as... ​ As illustrated in (f), the time required to filter out unnecessary data units or the transmission time consumed by unnecessary data units can be reduced.

[0363] Additionally, from the perspective of progressive decoding / progressive display, since geometry and attributes are sent together as a bundle, geometry and attributes corresponding to the same region and depth can be progressively displayed simultaneously.

[0364] Next, a signaling method for distinguishing geometric data and attribute data belonging to the same FGS is described when geometric data and attribute data are packaged into an FGS and sent according to the above method.

[0365] In one embodiment of this disclosure, an FGS index (e.g., fgs_id) can be defined separately. In this case, FGS geometry and FGS attributes belonging to the same FGS can be identified based on fgs_id. Public information can be delivered via GDU or DGDU, while redundant information can be delivered without ADU or DADU.

[0366] In one embodiment of this disclosure, such as ​ and ​ As shown, in SPS, a signal is sent to notify `unified_layer_group_structure_enabled`. ​ As shown, when unified_layer_group_structure_enabled is enabled, fgs_id can be signaled in the geometric data unit (GDU or DGDU).

[0367] `unified_layer_group_structure_enabled` indicates whether the geometry encoding and attribute encoding use the same layer group structure. For example, `unified_layer_group_structure_enabled` equal to 1 indicates that the geometry and attribute encoding use the same layer group structure. `unified_layer_group_structure_enabled` equal to 0 indicates that the layer group structures for geometry encoding and attribute encoding may be different.

[0368] fgs_id specifies an indicator for fine-grained slices. In other words, a specific FGS can be identified based on fgs_id.

[0369] Therefore, the decoder of the receiving device (e.g., ​ The geometry decoder and attribute decoder can selectively decode the FGS geometry (i.e., geometric data unit) and one or more FGS attributes (i.e., attribute data unit) of the desired FGS based on signaling information including the aforementioned unified_layer_group_structure_enabled and fgs_id.

[0370] In another embodiment of this disclosure, it can be based on ​ or ​ The geometric data unit (i.e., GDU or DGDU) shown contains a pair of indices: layer_group_id for identifying layer groups and subgroup_id for identifying subgroups, used to distinguish geometric data units and attribute data units belonging to the same FGS. In this case, the pair is defined as a concept used to group GDU / ADU or DGDU / DADU together, and the indices layer_group_id and subgroup_id used to identify subgroups can be used as FGS indices.

[0371] `layer_group_id` can represent an indicator of the layer group to which the FGS belongs. `layer_group_id` can be in the range of 0 to `num_layer_groups_minus1`. If it does not exist, it is inferred to be equal to 0.

[0372] `subgroup_id` specifies an indicator of a subgroup within the layer group referenced by `layer_group_id`. `subgroup_id` can range from 0 to `num_subgroups_minus1[layer_group_id]`. Here, `subgroup_id` can indicate the order of slices within the same `layer_group_id`. If it does not exist, it is inferred to be equal to 0.

[0373] Therefore, the decoder of the receiving device (e.g., ​ The geometry decoder and attribute decoder can select and decode the FGS geometry (i.e., geometric data unit) and one or more FGS attributes (i.e., attribute data unit) of the desired FGS based on signaling information including layer_group_id and subgroup_id.

[0374] Next, the signaling methods used for FGS identification and the aforementioned scalable transmission are described below.

[0375] ​ A bitstream configuration according to an implementation method is illustrated.

[0376] The method / apparatus according to the implementation method can generate, as follows: ​ The bitstream shown can contain encoded geometric data and attribute information, as well as parameter information.

[0377] The syntax and semantics related to parameter information are as follows.

[0378] Information regarding the segmentation, according to the implementation method, can be defined in the parameter set and SEI message of the bitstream as follows.

[0379] According to the implementation, a bitstream can contain a sequence parameter set, a geometric parameter set, an attribute parameter set, a geometric slice header (also called a geometric data unit header), and an attribute slice header (also called an attribute data unit header). Depending on the application or system, it can be defined at corresponding locations or individual locations, resulting in different scopes and application methods. That is, it can have different meanings depending on where the signal is delivered. When defined in SPS, it can be consistently applied to the entire sequence. When defined in GPS, it can be used for geometric reconstruction. When defined in APS, it can be used for attribute reconstruction. When defined in TPS, signaling can be applied only to points within a tile. When delivered by slice, signaling can be applied only to that slice. Depending on the application or system, it can be defined at corresponding locations or individual locations, resulting in different scopes and application methods. Furthermore, the syntax elements defined below can be applied not only to the current point cloud data stream but also to multiple point cloud data streams and can be sent through higher-level parameter sets.

[0380] Each abbreviation has the following meaning: SPS: Sequence Parameter Set, GPS: Geometric Parameter Set, APS: Attribute Parameter Set, TPS: Tile Parameter Set, Geom: Geometric Bitstream = Geometric Tile Header + Geometric Tile Data, Attr: Attribute Bitstream = Attribute Tile Header + Attribute Tile Data.

[0381] The implementation method defines the following information independently of the encoding technique, but this information can also be defined in conjunction with the encoding method. To support locally different scalability, it can be defined in the Tile Parameter Set (TPS) within the bitstream. Furthermore, when the syntax elements defined below can be applied not only to the current point cloud data stream but also to multiple point cloud data streams, they can be delivered through a higher-level parameter set.

[0382] Alternatively, bitstreams can be selected at the system level by defining Network Abstraction Layer (NAL) units for the bitstream and delivering relevant information for layer selection (such as layer_id).

[0383] According to the implementation, the parameters required for encoding and / or decoding point cloud data can be newly defined in a parameter set (e.g., SPS, GPS, APS, and TPS (or a list of tiles)) and / or the header of the corresponding tile (i.e., the data cell header or the tile header). For example, when performing geometric encoding and / or decoding, parameters can be added to GPS, or when performing tile-based encoding and / or decoding, parameters can be added to the tile and / or data cell headers.

[0384] According to the implementation method, slice-related information based on layer groups can be signaled in SPS and / or GPS and / or APS and / or LGSI.

[0385] According to the implementation method, slice-related information based on layer groups can be signaled in the TPS and / or geometric data cell header and / or attribute data cell header and / or geometric data cell and / or attribute data cell.

[0386] According to the implementation, when the syntax elements defined below can be applied to the current stream and multiple point cloud data streams, slice-related information based on layer groups can be delivered through a higher-level parameter set.

[0387] According to the implementation, slice-related information based on layer groups can be defined at corresponding or individual locations depending on the application or system, thereby allowing for different scopes and application methods. The term "field" used in the syntax described below can have the same meaning as "parameter" or "syntax element".

[0388] In the following text, parameters (which may be referred to as metadata, signaling information, etc.) according to the implementation method can be generated during the transmitter process and delivered to the receiver for use during the reconstruction process.

[0389] According to the implementation, parameters containing slice-related information based on layer groups can be generated by the metadata processor (or metadata generator or signaling processor) of the transmitting device and delivered to the receiving device for decoding / reconstruction. For example, parameters generated and transmitted by the transmitting device can be obtained by the metadata parser (or signaling processor) of the receiving device.

[0390] ​ and ​ An example of the syntax structure of a Sequence Parameter Set (SPS) according to an implementation is illustrated. The SPS may contain sequence information of a point cloud data bitstream. Specifically, in this example, it contains slice-related information based on layer groups.

[0391] ​ and ​ The syntax in can be included ​ The bitstream is generated by the point cloud encoder according to the implementation method and decoded by the point cloud decoder.

[0392] According to the SPS implementation, simple_profile_compatibility_flag: indicates whether the bitstream conforms to a simple profile (when it is 1) or not (when it is 0).

[0393] dense_profile_compatibility_flag: Indicates whether the bitstream conforms to a dense profile (when it is 1) or not (when it is 0).

[0394] predictive_profile_compatibility_flag: Indicates whether the bitstream conforms to the predictive profile (when it is 1) or does not (when it is 0).

[0395] main_profile_compatibility_flag: Indicates whether the bitstream conforms to the main profile (when it is 1) or not (when it is 0).

[0396] `slice_reordering_constraint_flag`: Indicates whether the bitstream is sensitive (when it is 1) or insensitive (when it is 0) to the reordering or removal of slices within the encoded point cloud frame. When `slice_reordering_constraint_flag` is 1, the resulting bitstream may not be fully decoded if slices are reordered or removed.

[0397] `unique_point_positions_constraint_flag`: When equal to 1, it indicates that each point should have a unique position in each coded point cloud frame. `unique_point_positions_constraint_flag` equal to 0 indicates that two or more points can have the same position.

[0398] sps_seq_parameter_set_id: Identifies the SPS for reference by other Data Units (DUs).

[0399] seq_origin_bits: Indicates the length of each seq_origin_xyz syntax element in bits, excluding the sign bit.

[0400] seq_origin_xyz[k] and seq_origin_log2_scale: Together, these indicate the XYZ origin and encoded coordinates of the sequence relative to the origin of the applied coordinate system, in sequence coordinate units. When seq_origin_bits is 0, seq_origin_xyz[k] and seq_origin_log2_scale are inferred to be 0.

[0401] seq_bounding_box_size_bits: Indicates the length in bits of each seq_bbox_size_minus1_xyz syntax element.

[0402] seq_bounding_box_size_minus1_xyz[k]: Add 1 to indicate the k-th XYZ component of the encoded volume dimension in the sequence coordinate system.

[0403] seq_unit_numerator_minus1, seq_unit_denominator_minus1, and seq_unit_is_metres together indicate the length in the sequence coordinate system as a unit vector.

[0404] sps_num_attribute_sets: Indicates the number of attributes listed in the SPS attribute list.

[0405] attribute_instance_id[attrId]: Indicates the instance identifier of the attribute being identified.

[0406] attribute_bitdepth_minus1[attrId]: Incrementing by 1 specifies the bit depth of all components of the identified attribute.

[0407] `layer_group_enabled_flag`: When equal to 1, it indicates that the geometric bitstream of a slice is contained in multiple slices that match the group of the coding layer or its corresponding subgroup. When equal to 0, `layer_group_enabled_flag` specifies that the geometric bitstream is contained in a single slice.

[0408] `num_layer_groups_minus1 + 1` indicates the number of layer groups, where a layer group represents a group of consecutive tree layers that are part of the geometric coding tree structure. `num_layer_groups_minus1` ranges from 0 to the total number of coding tree layers.

[0409] `layer_group_id` is an index used to identify a layer group and indicates the layer group identifier (or indicator). `layer_group_id` is in the range of 0 to `num_layer_groups_minus1`.

[0410] num_layers_minus1 + 1 indicates the number of coding layers included in the i-th layer group. When i is 0, the total number of layer groups can be derived by adding all (num_layers_minus1[i] + 1) to num_layer_groups_minus1.

[0411] `subgroup_enabled` equal to 1 indicates that the i-th layer group is divided into two or more subgroups, where the point set in the subgroups of the layer group is the same as the point set in the layer group. When `subgroup_enabled` of the i-th layer group is equal to 1, `subgroup_enabled` of the j-th layer group should be equal to 1 when j is greater than or equal to i. `subgroup_enabled` equal to 0 indicates that the current layer group is not subdivided into multiple subgroups and is contained in a single slice.

[0412] `subgroup_bbox_origin_bits_minus1 + 1` indicates the bit length of the `subgroup_bbox_origin` field (or syntax element).

[0413] The subgroup_bbox_size_bits_minus1 + 1 indicates the bit length of the subgroup_bbox_size field.

[0414] `unified_layer_group_structure_enabled` indicates whether geometry and attribute encoding use the same layer group structure. For example, `unified_layer_group_structure_enabled` equal to 1 indicates that geometry and attribute encoding use the same layer group structure. `unified_layer_group_structure_enabled` equal to 0 indicates that the layer group structures for geometry and attribute encoding may be different.

[0415] In this disclosure, slice-related information based on layer groups may include one or more of layer_group_enabled_flag, num_layer_groups_minus1, layer_group_id[i], num_layers_minus1[i], subgroup_enabled[i], subgroup_bbox_origin_bits_minus1, subgroup_bbox_size_bits_minus1, and unified_layer_group_structure_enabled.

[0416] ​ An example of the syntax structure of the relevant geometric data unit header according to the implementation method is shown. ​ An example is shown where slice-related information based on layer groups is included in the header of the relevant geometric data cell.

[0417] ​ The syntax can be included ​ The bitstream, generated by the point cloud encoder according to the embodiment, is sent to the decoder of the receiving device and can be decoded by the point cloud decoder.

[0418] According to the implementation method, the header of the relevant geometric data unit may be included in the geometric data unit. The geometric data unit may have the same meaning as the relevant geometric data unit or geometric slice.

[0419] In the header of the relevant geometric data unit according to the implementation, dgsh_geometry_parameter_set_id indicates the active GPS as indicated by gps_geom_parameter_set_id. dgsi_geometry_parameter_set_id is equal to gdu_geometry_parameter_set_id of the corresponding slice.

[0420] dgsi_slice_id specifies the geometric slice to which the currently relevant geometric data cell belongs.

[0421] When the unified_layer_group_structure_enabled included in SPS is enabled, the header of the associated geometric data cell may also contain fgs_id.

[0422] fgs_id specifies the indicator for fine-grained slices. In other words, fgs_id is an identifier used to identify the FGS. fgs_id can be included in the header of individual geometric data cells.

[0423] `layer_group_id` is an index used to identify a layer group and can represent an indicator of the layer group. `layer_group_id` can be in the range of 0 to `num_layer_groups_minus1`. If it does not exist, it is inferred to be equal to 0.

[0424] `subgroup_id` is an index used to identify a subgroup of the layer group referenced by `layer_group_id`, and specifies the subgroup's identifier. `subgroup_id` can range from 0 to `num_subgroups_minus1[layer_group_id]`. Here, `subgroup_id` can indicate the order of slices with the same `layer_group_id`. When it does not exist, it is inferred to be equal to 0.

[0425] subgroup_bbox_origin indicates the origin of the bounding box of the subgroup of the j-th layer group, indicated by layer_group_id, and the i-th subgroup, indicated by subgroup_id.

[0426] subgroup_bbox_size indicates the size of the subgroup bounding box of the j-th layer group, indicated by layer_group_id, and the i-th subgroup, indicated by subgroup_id.

[0427] According to the implementation, the bounding boxes of points in a subgroup are described by `subgroup_bbox_origin` and `subgroup_bbox_size`. In this case, when i is not equal to j, the region in the bounding box of the i-th subgroup does not overlap with the bounding box of the j-th subgroup.

[0428] The `ref_layer_group_id` is an indicator that specifies the layer group identifier used for the context reference of the currently relevant data cell. `ref_layer_group_id` ranges from 0 to the `layer_group_id` of the currently relevant data cell.

[0429] `ref_subgroup_id` indicates the reference subgroup of the layer group indicated by `ref_layer_group_id`. `ref_subgroup_id` ranges from 0 to `num_subgroup_id_minus1` of the layer group indicated by `ref_layer_group_id`.

[0430] A `context_reference_indication_flag` equal to 1 indicates that the context state of the current related data unit is inherited by one or more subsequent related data units. A `context_reference_indication_flag` equal to 0 indicates that the context state of the current related slice (i.e., data unit) is not inherited by subsequent related slices (i.e., data units). When this flag is not present, the `context_reference_indication_flag` of the corresponding data unit is inferred to be equal to 1.

[0431] The decoder of the receiving device can manage the context buffer based on `context_reference_indication_flag`. When `context_reference_indication_flag` equals 1, the context state of the currently relevant slice (or data unit) is stored in the context buffer at the end of decoding the current data unit. When `context_reference_indication_flag` equals 0, the context state of the currently relevant slice (or data unit) is not stored in the context buffer. Here, the decoder can correspond to... ​ , 2 One of 7, 9, or 18.

[0432] When context_reference_indication_flag is true (i.e., 1), the header of the relevant geometric data cell can also contain num_subsequent_subgroups.

[0433] num_subsequent_subgroups specifies the number of subsequent related data units that reference the current data unit or related data unit.

[0434] In this disclosure, FGS geometries and FGS attributes belonging to the same FGS can be identified by fgs_id, or by a pair of layer_group_id and subgroup_id.

[0435] The bitstream according to the implementation may contain at least one slice or sub-slice, and may contain geometric data units and attribute data units. Each data unit includes a header. Additionally, data units may include independent data units and related data units. Related data units may indicate inclusion relationships based on dependencies between parent and child nodes. ​ The syntax for the relevant geometric data cell header is illustrated. The syntax for the geometric data cell header can be compared with... ​ The syntax is similar or the same.

[0436] The encoding process based on the above description will be described below.

[0437] According to one implementation, the attribute slicing encoder is based on generating partial LoDs for each subgroup. This method requires less memory because it does not require generating LoDs for the entire point cloud. However, before encoding begins, a process is needed to assign corresponding attribute values ​​to nodes in each intermediate layer group. In this disclosure, the attribute slicing encoder may be included within the attribute encoder.

[0438] In other words, during the geometric encoding process, octree geometric encoding is performed for each subgroup (data unit). In this case, partial octree encoding can be performed. For intermediate layer groups, a sub-index can be stored for each node. Then, for nodes in the intermediate layer group, a small LoD can be generated based on the stored sub-index, and a sampling index indicating the points within the LoD can be calculated based on the generated LoD.

[0439] In this case, during the recoloring operation in the attribute encoding process, the encoded points can be recolored.

[0440] Then, attribute encoding can be performed for each subgroup. In this case, QP / weight adjustment can be performed. For intermediate layer groups, attribute values ​​derived from the recolored points specified by the sampling index can be assigned to each intermediate node. Additionally, a partial LoD can be generated based on the intermediate nodes and the encoded IDCM points in the current subgroup. In this case, similar to scalable encoding, the sampling direction of the octree subsamples at odd and even levels can be reversed. Attribute values ​​can then be encoded based on the generated partial LoD. In this case, at the highest level of the LoD, the attribute value for each point is predicted based on the corresponding parent node. At other levels, the attribute value is predicted based on the points within the LoD.

[0441] ​ An example of storing sampling indices in a geometric octree encoding according to an implementation is illustrated. That is, in a geometric octree encoding, for each intermediate node, the index of the corresponding point can be stored as a sampling index based on the little LoD of the child point.

[0442] Next, the decoding process based on the above description will be described.

[0443] Specifically, during the geometric decoding process, octree geometric decoding is performed for each subgroup (i.e., data unit). In this case, partial octree decoding can be performed.

[0444] During attribute decoding, attribute decoding can be performed for each subgroup. According to the implementation, a partial LoD can be generated based on intermediate nodes and the encoded IDCM points in the current subgroup. In this case, similar to scalable coding, the sampling direction of the octree subsamples at odd and even levels can be reversed. The attribute values ​​can then be decoded based on the generated partial LoD. In this case, at the highest level of the LoD, the attribute value of each point can be predicted based on the corresponding parent node. At other levels, the attribute values ​​can be predicted based on the points within the LoD.

[0445] ​ An example of the syntax structure of the Attribute Parameter Set (APS) according to the implementation method is shown.

[0446] When the value of aps_extension_present is true (i.e., 1), ​ The APS can contain a layer_group_aligned_flag. A layer_group_aligned_flag of 1 indicates that the attribute bitstream contains at least one independent attribute data unit and a correlated data unit, and each unit contains an attribute value corresponding to each individual decoding point of the independent / correlated geometric data unit. On the other hand, when layer_group_aligned_flag is equal to 0, the attribute bitstream contains only independent attribute data units and contains attribute values ​​corresponding to all joint decoding points of the independent / correlated data units.

[0447] ​ Another example of the syntax structure of the Attribute Parameter Set (APS) according to the implementation method is shown.

[0448] When the value of layer_group_enabled_flag is true (i.e., 1), ​ An APS can contain the aps_attr_ref_id_present_flag.

[0449] An aps_attr_ref_id_present_flag value of 1 indicates that the context reference of the attribute slice will be indicated by attr_ref_layer_group_id and attr_ref_subgroup_id, and the context state used in subsequent attribute slices will be indicated by attr_context_reference_indication_flag in the header of the associated attribute data unit. Conversely, when aps_attr_ref_id_present_flag is equal to 0, the context reference and context state used will be inherited from the geometry slice corresponding to the layer_group_id and subgroup_id of the currently associated attribute slice.

[0450] `layer_group_enabled_flag` is a flag included in SPS. `layer_group_enabled_flag` equal to 1 indicates that the geometric bitstream of a slice is included in multiple slices corresponding to a group or subgroup of the coding layer. `layer_group_enabled_flag` equal to 0 specifies that the geometric bitstream is included in a single slice.

[0451] ​ An example of the syntax structure of the dependent_attribute_data_unit_header() according to the implementation method is shown.

[0452] exist ​ In the header of the relevant attribute data unit, dadu_attribute_parameter_set_id indicates the active APS indicated by aps_attr_parameter_set_id. dadu_attribute_parameter_set_id is equal to adu_attribute_parameter_set_id of the corresponding slice.

[0453] dadu_sps_attr_idx identifies the encoded attribute by its index into the active SPS attribute list.

[0454] dadu_slice_id specifies the attribute slice to which the current related attribute data unit belongs.

[0455] When the unified_layer_group_structure_enabled included in SPS is enabled, the header of the relevant attribute data cell can also include fgs_id.

[0456] fgs_id specifies an indicator for fine-grained slices. fgs_id can also be included in the header of an independent attribute data cell.

[0457] When unified_layer_group_structure_enabled is disabled, the header of the relevant attribute data cell can also include dadu_layer_group_id.

[0458] `dauu_layer_group_id` is an index used to identify a layer group and can represent an indicator of the layer group. `dauu_layer_group_id` can be in the range of 0 to `num_layer_groups_minus1`. When it does not exist, it is inferred to be equal to 0.

[0459] `dadou_subgroup_id` specifies an indicator of the subgroup of the layer group referenced by `dadu_layer_group_id`. `dadou_subgroup_id` can range from 0 to `num_subgroups_minus1[dadu_layer_group_id]`. `dadou_subgroup_id` can also indicate the order of slices with the same `dadu_layer_group_id`. When it does not exist, it is inferred to be equal to 0.

[0460] When `aps_attr_ref_id_present_flag` included in the APS is equal to 1, the header of the associated attribute data unit can also contain `attr_ref_layer_group_id`. That is, when `aps_attr_ref_id_present_flag` is equal to 1, the context reference of the attribute slice is indicated by `attr_ref_layer_group_id` and `attr_ref_subgroup_id`, and whether the context state is used in subsequent attribute slices is indicated by `attr_context_reference_indication_flag` in the header of the associated attribute data unit. When `aps_attr_ref_id_present_flag` is equal to 0, the context reference and context state are inherited from the geometry slice corresponding to the `layer_group_id` and `subgroup_id` of the current associated attribute slice.

[0461] The `attr_ref_layer_group_id` is an indicator that specifies the layer group identifier used for the context reference of the current related attribute data cell. `attr_ref_layer_group_id` should be in the range of 0 to the `attr_layer_group_id` of the current related attribute data cell.

[0462] The attr_ref_subgroup_id specifies the indicator that points to the reference subgroup of the layer group indicated by the attr_ref_layer_group_id. The attr_ref_subgroup_id should be in the range of 0 to num_subgroup_id_minus1 of the layer group indicated by the attr_ref_layer_group_id.

[0463] `attr_context_reference_indication_flag` equal to 1 indicates that the context state of the current related property slice is inherited by one or more subsequent related property slices. `attr_context_reference_indication_flag` equal to 0 indicates that the context state of the current related slice is not inherited by subsequent related slices.

[0464] The decoder can manage the context buffer based on `attr_context_reference_indication_flag`. For example, when `attr_context_reference_indication_flag` equals 1, the context state of the currently relevant slice is stored in the context buffer at the end of decoding. As another example, when `attr_context_reference_indication_flag` equals 0, the context state of the currently relevant slice is not stored in the context buffer.

[0465] According to the implementation, when unified_layer_group_structure_enabled is disabled, the header of the relevant attribute data unit may contain one or more of dadu_layer_group_id, dadu_subgroup_id, attr_ref_layer_group_id, attr_ref_subgroup_id, and attr_context_reference_indication_flag.

[0466] In addition, ​In the formula, `dadu_last_comp_pred_coeff_diff[i]` specifies the scaling factor applied to the second coefficient component to predict the last coefficient component of the multi-component properties. Each scaling factor is applied sequentially from the first coefficient to the last coefficient at a single transform level `i`.

[0467] `dadou_inter_comp_pred_coeff_diff[i][c]` specifies the scaling factor applied to the prediction of the c-th coefficient component based on the first coefficient component of the multi-component attribute. Each scaling factor is applied sequentially from the first coefficient to the last coefficient at a single transform level `i`.

[0468] dadu_attr_qp_offset specifies whether the QP offset dadu_attr_qp_offset[c] based on the data unit exists in the DADU header.

[0469] dadu_attr_qp_layers_present specifies whether the QP offset of each transform layer exists in the DADU (it exists when the value is 1, and does not exist when the value is 0).

[0470] dadu_attr_qp_layer_cnt_minus1 plus 1 (if present) specifies the number of LoD layers or levels of the RAHT tree for which the QP offset is signaled.

[0471] `dadu_attr_qp_layer_offset[dpth][qc]` specifies the QP offset for the attributes used for the fundamental components (qc = 0) and auxiliary components (qc = 1). Each offset is applied to the transform coefficients at the depth of the RAHT tree or LoD layer. If the LoD layer or RAHT tree has more levels than `dadu_attr_qp_layer_cnt_minus1 + 1`, the value of `dadu_attr_qp_layer_offset[dadu_attr_qp_layer_cnt_minus1][qc]` is also used as the QP offset for the transform coefficients beyond the depth.

[0472] In this disclosure, the header of the associated attribute data cell does not contain an ID indicating the parent data cell. This information can be inherited from the header of the corresponding associated geometric data cell.

[0473] Another embodiment of the fine-grained slicing (FGS) described above will be described below. In this case, some descriptions may overlap with the descriptions related to FGS described above.

[0474] First, some terminology will be redefined.

[0475] Fine-grained slices (FGS) are subsets of slices that carry geometric or attribute information related to subgroups within a layer group.

[0476] A layer group is a group of consecutive tree levels that occupy a tree or attribute assignment occupy a tree.

[0477] A subgroup is a spatial subset of a layer group, wherein the bounding box of a subgroup should not overlap with other subgroups in the same layer group.

[0478] The root group is a group that includes the root node of the occupancy tree or the attribute assignment occupancy tree.

[0479] A parent-child group is a subgroup within a layer group that is adjacent to the lowest depth of the top layer of the current subgroup, where the bounding box of the parent-child group is a superset of the bounding box of the current subgroup.

[0480] A subgroup is a subgroup within a layer group that is adjacent to the lowest layer of the current subgroup at the maximum depth, where the bounding box of the subgroup is a subset of the bounding box of the current subgroup.

[0481] An attribute assignment occupancy tree is an occupancy tree in which the attributes of nodes at each tree level are assigned by the attributes of their child nodes.

[0482] In this case, a slice can be composed of FGS, and each FGS can be mapped one-to-one to the FGS geometry or FGS attribute of a subgroup in the layer group.

[0483] An FGS within a slice, or a slice of an FGS, is identified by a public slice identifier (slice_id).

[0484] In this disclosure, the FGS can be indicated by a pair of layer group indexes (e.g., layer_group_id) and subgroup indexes (e.g., subgroup_id), or it can be indicated by fgs_id. When the FGS has the same index pair, the FGS can be either the geometry or attribute of the subgroup indicated by the layer group index and the subgroup index.

[0485] Each FGS includes a geometric data unit (GDU) or associated geometric data unit (DGDU) for encoding partial slice geometry, or an attribute data unit (ADU) or associated attribute data unit (DADU) for encoding partial slice attributes.

[0486] The first FGS in the slice is the FGS of the GDU. The FGS that follows this FGS can be the FGS of the DGDU, which depends on the previously decoded GDU and DGDU.

[0487] The first FGS in the slice attributes is the FGS of the ADU. This FGS can be followed by the FGS of the DADU, which depends on the corresponding ADU or a previously decoded ADU or DADU. The FGS of the ADU and DADU appear after the FGS of the GDU and DGDU.

[0488] In this disclosure, if no attributes exist, that is, if only geometry is sent or only geometry FGS is decoded, predefined attributes can be used for geometry nodes. This scheme can also be applied to nodes generated in cases of partial region or depth decoding.

[0489] In the hierarchical structure disclosed herein, nodes in the occupancy tree or attribute assignment occupancy tree are grouped into hierarchical groups and subgroups.

[0490] A layer group is a group of consecutive tree levels, and each tree level belongs to only one layer group. The minimum depth of a subgroup is equal to the maximum depth of the parent-child group plus 1, or 0 in the case of the root layer group. The maximum depth of a subgroup is equal to the minimum depth of the sub-child group minus 1, or the maximum depth of the occupied tree in the case of the last layer group. In this case, the layer group is identified by the layer group index (layer_group_idx).

[0491] A subgroup is a spatial subset of a layer group, and nodes within a tree level belong to one of the subgroups of the layer group.

[0492] The location range of nodes in a subgroup is described by a bounding box, and the bounding box of a subgroup does not overlap with the bounding boxes of other subgroups within the same level group.

[0493] The set of nodes in all subgroups of a layer group is the same as the set of nodes in the layer group.

[0494] The root layer group has only one subgroup. Subgroups within a layer group are identified by their subgroup index (subgroup_idx).

[0495] ​ (a) and ​ (b) illustrates the relationship between layers, subgroups, and FGS according to the implementation method. Specifically, ​ (a) illustrates the layer group structure of an occupied tree with a maximum depth of 8, and ​ (b) illustrates the parent-child relationship between a subgroup and its corresponding FGS.

[0496] exist ​In (a), three layers are defined: layer 0 consists of depths 0 to 3, layer 1 consists of depths 4 to 6, and layer 2 consists of depths 7 and 8. That is, layer 0 consists of one subgroup, layer 1 consists of two subgroups, and layer 2 consists of four subgroups. In other words, each layer, except for the root layer, can include subgroups, and each subgroup can be indicated by a pair of layer group indices and subgroup indices. For example, the root layer is indicated by (0, 0).

[0497] exist ​ In (b), ​ The spatial regions of subgroups in (a) are represented by rectangular bounding boxes on the xy-plane. Subgroups in adjacent layers are considered parent-child when the bounding box of a subgroup in a layer group is a superset of the bounding boxes of one or more subgroups in the next layer group. For example, subgroup (0,0) is the parent of subgroups (1,0) and (1,1). Similarly, subgroups (2,0) and (2,1) are children of subgroup (1,0). Subgroups indicated by a pair of layer group and subgroup indices belong to different FGS.

[0498] exist ​ (a) and ​ In (b), Sl,n represents the subgroup n associated with layer group l. Additionally, di represents the octree depth i.

[0499] ​ and ​ Another example of the syntax structure of a Sequence Parameter Set (SPS) according to an implementation is illustrated. The SPS may contain sequence information related to the point cloud data bitstream. Specifically, the accompanying figures illustrate an example where the SPS contains slice-related information based on layer groups.

[0500] ​ and Figure 29b The syntax can be included Figure 21 The bitstream can be generated by a point cloud encoder according to the implementation method and decoded by a point cloud decoder.

[0501] Next, we will describe slice-related information based on layer groups. For information other than slice-related information based on layer groups, please refer to [link / reference]. Figure 22a and Figure 22b The description.

[0502] For example, in Figure 29a and Figure 29b In the context, simple_profile_compliant corresponds to Figure 22a and Figure 22bIn the `simple_profile_compatibility_flag`, `dense_profile_compliant` corresponds to `dense_profile_compatibility_flag`, and `predictive_profile_compliant` corresponds to `predictive_profile_compatibility_flag`. `main_profile_compliant` corresponds to `main_profile_compatibility_flag`, `slice_reordering_constraint` corresponds to `slice_reordering_constraint_flag`, and `unique_point_positions_constraint` corresponds to `unique_point_positions_constraint_flag`. `sps_seq_parameter_set_id` corresponds to `sps_seq_parameter_set_id`, `seq_origin_bits` corresponds to `seq_origin_bits`, and `seq_origin_xyz[k]` and `seq_origin_log2_scale` correspond to `seq_origin_xyz[k]` and `seq_origin_log2_scale`. seq_bbox_size_bits corresponds to seq_bounding_box_size_bits, and seq_bbox_size_minus1_xyz[k] corresponds to seq_bounding_box_size_minus1_xyz[k].

[0503] exist Figure 29a and Figure 29b In this context, `layer_group_enabled` equal to 1 indicates that a slice consists of multiple FGSs (Frame Groups) representing a portion of the slice geometry or a portion of the slice attributes. `layer_group_enabled` equal to 0 indicates that a slice does not consist of FGSs.

[0504] `num_layer_groups_minus1 + 1` indicates the number of layer groups, where a layer group represents a group of consecutive tree levels (or layers) that are part of an occupied tree. `num_layer_groups_minus1` should be in the range of 0 to the number of occupied tree levels (or coded tree layers).

[0505] `layer_group_id[i]` specifies the layer group index, i.e., the indicator, used to identify the i-th layer group of the slice. `layer_group_id` should be in the range of 0 to `num_layer_groups_minus1`.

[0506] num_layers_minus1[i] + 1 indicates the number of tree levels included in the i-th layer group. When i is 0, the total number of layer groups can be derived by adding all (num_layers_minus1[i] + 1) values ​​to num_layer_groups_minus1.

[0507] A subgroup_enabled[i] equal to 1 indicates that the i-th level group includes two or more subgroups. A subgroup_enabled[i] equal to 0 indicates that the i-th level group includes one subgroup. When the subgroup_enabled of the i-th level group is equal to 1, the aggregation (i.e., the set of nodes) of all nodes in each subgroup of the i-th level group is the same as the set of nodes in the level group. Additionally, when the subgroup_enabled of the i-th level group is equal to 1, if j is greater than i, then the subgroup_enabled of the j-th level group is equal to 1.

[0508] subgroup_bbox_origin_bits_minus1 + 1 is the bit length of the subgroup_bbox_origin field (or syntax element).

[0509] subgroup_bbox_size_bits_minus1 + 1 is the bit length of the subgroup_bbox_size field.

[0510] Figure 30 An example of the syntax structure of a geometry data unit (geometry_data_unit()) according to an implementation is shown.

[0511] According to the implementation, the geometric data unit includes a geometric data unit header (geometry_data_unit_header()). Additionally, the geometric data unit can include occupancy_tree(startDepth, endDepth) or predictive_tree() depending on the geom_tree_type. Furthermore, the geometric data unit may also include trisoup() when the value of trisoup_enable_flag is true.

[0512] `geom_tree_type` is included in the geometry parameter set and indicates the encoding type of the geometry data. A `geom_tree_type` of 0 indicates that the geometry data is encoded based on an occupancy tree. A `geom_tree_type` of 1 indicates that the geometry data is encoded based on a prediction tree.

[0513] Figure 31 An example of the syntax structure of the geometric data unit header according to the implementation method is shown. Figure 31 The syntax can be included Figure 21 The bitstream, generated by the point cloud encoder according to the embodiment, is sent to the decoder of the receiving device and can be decoded by the point cloud decoder.

[0514] exist Figure 31 In the configuration, gdu_geometry_parameter_set_id specifies the active GPS via gps_geom_parameter_set_id.

[0515] gdu_temporal_id specifies the time ID of the frame associated with the geometric data unit (GDU).

[0516] The slice_id identifies the slice for reference by another DU.

[0517] The slice_tag identifies a slice as a member of a slice group with the same slice_tag value. When a tile list DU exists, the slice group is a tile identified by its tile ID. On the other hand, when a tile list DU does not exist, the interpretation of the slice_tag depends on the application.

[0518] frame_ctr_lsb specifies the frame_ctr_lsb_bits LSBs of FrameCtr, where FrameCtr is a conceptual frame counter.

[0519] `entropy_continuation_enabled` equal to 1 specifies that the entropy resolution recovery process should be applied at the beginning of the GDU and all ADUs within the slice. When `slice_inter_entropy_continuation` equals 0, `entropy_continuation_enabled` equal to 0 specifies that the resolution of the GDU and all ADUs within the slice is independent of other slices within the frame. When `slice_entropy_continuation` does not exist, it is inferred to be equal to 0.

[0520] prev_slice_id should be equal to the slice_id of the GDU of the previous slice in the bitstream sequence.

[0521] Next, we will describe the slicing information based on layer groups.

[0522] When the value of layer_group_enabled included in the sequence parameter set is true (i.e., 1), the geometry data cell header can contain num_subsequent_subgroups. A layer_group_enabled value of 1 indicates that the slice consists of multiple FGS geometries or FGS attributes.

[0523] num_subsequent_subgroups specifies the number of subsequent related data units that reference the current data unit or related data unit.

[0524] If occtree_planar_enabled contained in the geometry parameter set is true (i.e., 1) and geom_angular_enabled is false (i.e., 0), then the geometry data cell header can contain subgroup_planar_eligibility_by_density[i] by the value of num_layers_minus1[0].

[0525] `occtree_planar_enabled` specifies whether the encoding of the node occupancy bitmap is (when 1) partially enforced through signaling of occupancy and non-occupancy planes (when 0, it is not enforced). If `occtree_planar_enabled` is not present, it should be inferred to be 0.

[0526] geom_angular_enabled specifies whether (when 1) the slice geometry is encoded using information about a set of beams that are positioned along the V-axis at the origin of the angle and rotated around that V-axis (when 0, it is not used).

[0527] A subgroup_planar_eligibility_by_density[i] equal to 1 indicates that the planar eligibility flag is enabled for the i-th depth of the current subgroup. In other words, a planar_eligibility_by_density[i] equal to 1 indicates that planar eligibility is enabled for the i-th depth of the current subgroup. A planar_eligibility_by_density[i] equal to 0 indicates that planar eligibility is disabled for the i-th depth of the current subgroup. In other words, a planar_eligibility_by_density[i] equal to 0 indicates that planar eligibility is disabled for the i-th depth of the current subgroup.

[0528] According to the implementation method, Figure 31The header of the geometric data unit may also contain a layer group index (layer_group_id) and a subgroup index (subgroup_id), or it may also contain an fgs_id. In this case, the decoder of the receiving device (e.g., Figure 18 The geometry decoder and attribute decoder can perform partial decoding based on signaling information including layer group index (layer_group_id), subgroup index (subgroup_id) and / or fgs_id, by selecting the FGS geometry (i.e., geometric data unit) of the desired FGS and one or more FGS attributes (i.e., attribute data units).

[0529] Figure 32 An example of the syntax structure of the dependent geometry data unit (dependent_geometry_data_unit()) according to the implementation method is shown.

[0530] According to the implementation, the dependent geometry data unit includes a dependent geometry data unit header (dependent_geometry_data_unit_header()). Additionally, the dependent geometry data unit may also include occupancy_tree(startDepth, endDepth) and geometry_data_unit_footer(occtreeMaxDepthMinus1) carrying the actual compressed geometry data.

[0531] Figure 33 Another example of the syntax structure of the header of the relevant geometric data unit according to the implementation method is shown. Figure 33 An example is shown where slice-related information based on layer groups is included in the header of the relevant geometric data cell.

[0532] Figure 33 The syntax is included in Figure 21 The bitstream, generated by the point cloud encoder according to the embodiment, is sent to the decoder of the receiving device and can be decoded by the point cloud decoder.

[0533] In this disclosure, the DGDU carries FGS information related to the partial slice geometry (e.g., a pair of layer group indexes and subgroup indexes, subgroup bounding boxes, context state reference indexes, etc.). The DGDU includes a DGDU header, geometry encoded using occupancy tree, and a GDU trailer.

[0534] exist Figure 37In the header of the relevant geometric data unit, dgdu_geometry_parameter_set_id specifies the active GPS indicated by gps_geom_parameter_set_id. dgdu_geometry_parameter_set_id should be equal to gdu_geometry_parameter_set_id within the same slice.

[0535] slice_id specifies the slice to which the current relevant geometric data cell belongs.

[0536] `layer_group_id` specifies an indicator of the layer group in the layer group structure associated with the slice. `layer_group_id` is in the range of 0 to `num_layer_groups_minus1`. When it does not exist, `layer_group_id` is inferred to be equal to 0.

[0537] `subgroup_id` specifies the indicator of the subgroup within the layer group indicated by `layer_group_id`. `subgroup_id` ranges from 0 to `num_subgroups_minus1[layer_group_id]` and indicates a portion of the layer group specified by `layer_group_id`. If it does not exist, `subgroup_id` is inferred to be equal to 0.

[0538] subgroup_bbox_origin specifies the minimum position of the subgroup bounding box of the subgroup specified by the pair of layer_group_id and subgroup_id.

[0539] subgroup_bbox_size specifies the size of the subgroup bounding box of the subgroup specified by the pair of layer_group_id and subgroup_id.

[0540] The subgroup bounding box is represented by subgroup_bbox_origin and subgroup_bbox_size. The minimum position of the bounding box is defined by subgroup_bbox_origin, and the maximum position is inferred by adding subgroup_bbox_size to subgroup_bbox_origin.

[0541] Within a layer group, if i and j are not equal, the bounding box of the i-th subgroup should not overlap with the bounding box of the j-th subgroup.

[0542] `ref_layer_group_id` specifies an indicator of the layer group identifier used for context reference of the currently relevant data unit. `ref_layer_group_id` ranges from 0 to `num_layer_groups_minus1`.

[0543] `ref_subgroup_id` specifies an indicator of the subgroup identifier used for context reference of the currently related data unit. When it does not exist, `ref_subgroup_id` is inferred to be equal to 0.

[0544] The reference context state is identified by a pair of identifiers: ref_layer_group_id and ref_subgroup_id.

[0545] A subgroup_context_reference_indication_enabled value of 1 indicates that the context state of the current data unit is used to initialize one or more subsequent data units. A subgroup_context_reference_indication_enabled value of 0 indicates that the context state of the current data unit is not used to initialize subsequent data units. When it does not exist, subgroup_context_reference_indication_enabled is inferred to be equal to 1.

[0546] num_subsequent_subgroups indicates the number of subsequent related data units that reference the current data unit or related data unit.

[0547] A subgroup_planar_eligibility_by_density[i] equal to 1 indicates that planar eligibility is enabled for the i-th depth of the current subgroup. A subgroup_planar_eligibility_by_density[i] equal to 0 indicates that planar eligibility is disabled for the i-th depth of the current subgroup. When it does not exist, subgroup_planar_eligibility_by_density[i] is inferred to be equal to 0.

[0548] The decoder of the receiving device can manage the context buffer using `subgroup_context_reference_indication_enabled`. When `subgroup_context_reference_indication_enabled` equals 1, the context state of the current relevant data unit is stored in the context buffer at the end of decoding the current data unit. When `subgroup_context_reference_indication_enabled` equals 0, the context state of the current relevant data unit is not stored in the context buffer. Here, the decoder can correspond to... Figure 1 , Figure 2 , Figure 7 , Figure 9 , Figure 16 , Figure 18 or Figure 40 one of them.

[0549] The following will describe Figure 1 , Figure 2 , Figure 7 , Figure 9 , Figure 16 , Figure 18 or Figure 40 The decoding process performed in one of them.

[0550] The slice decoding process will be described below.

[0551] According to the implementation method, the slices in the point cloud frame encoded by the encoder of the transmitting device are decoded as follows: 1) Decode point location from one GDU in a slice. When layer_group_enabled is 1, decode point location from one GDU and zero or more DGDUs.

[0552] 2) Set a default value for each attribute.

[0553] 3) Decode point attributes from each ADU in the slice. When layer_group_enabled equals 1, decode point attributes from one ADU and zero or more DADUs. ADUs and DADUs are decoded after GDUs and DGDUs indicated by the same subgroup index pair (layer_group_id and subgroup_id) are decoded.

[0554] 4) The decoded point position is offset, and the number of output points increases.

[0555] Additionally, for all slice sets in an encoded point cloud frame with the same slice_id value, only one slice is decoded.

[0556] Decode the GDU and store the decoded location in the output point cloud.

[0557] Next, the fine-grained slice decoding process will be described. GDUs are decoded before all DGDUs in the slice. DGDUs of child-child groups are decoded after the DGDUs of parent-child groups are decoded.

[0558] When decoding GDU, startDepth and endDepth are set to 0 and num_layers_minus1[0] +1, respectively.

[0559] When decoding DGDU, startDepth is set to the cumulative value of num_layers_minus1[k] + 1 for each k in the range from 0 to (layer_group_id - 1), and endDepth is set to the cumulative value of num_layers_minus1[k] + 1 for each k in the range from 0 to layer_group_id.

[0560] Additionally, the occupancy tree represents the slice geometry as a tree of occupancy tree nodes. Parsing or traversing the encoded occupancy tree implicitly generates the representation of the slice geometry.

[0561] When layer_group_enabled equals 1, the partial occupancy tree represents the FGS geometry as a partial tree of occupancy tree nodes in terms of spatial region and depth. Parsing or traversing the encoded partial occupancy tree implicitly generates the FGS geometric representation.

[0562] The location of each point is represented by the position of the leaf node in the occupancy tree or by the direct node that encodes the relative position of the node.

[0563] Occupancy tree nodes identify the presence of at least one point contained within the volume of an axis-aligned cuboid. When layer_group_enabled equals 1, occupancy tree nodes identify the presence of at least one point contained within the volume of a cube. This volume is defined in slice coordinates by the inclusion lower angle p_min and the exclusion upper angle p_max.

[0564] Additionally, when `layer_group_enabled` equals 1, the encoded partially occupied tree can be traversed in breadth-first order within a subgroup, with the traversal starting from the minimum depth of the subgroup. All nodes within the subgroup's tree level are traversed sequentially before proceeding to the next level. Within the subgroup's tree level, nodes are traversed in ascending Morton order of their node positions.

[0565] Next, we will describe part of the decoding of fine-grained slices.

[0566] The decoder can decode and reconstruct a portion of the FGS. Slices segmented into the FGS support partial decoding in terms of density and / or spatial regions. According to one implementation, when using partial decoding, unnecessary data units can be filtered out before decoding the occupancy tree or attribute coefficients to generate a partial output.

[0567] Next, we will describe some density decoding.

[0568] The decoder generates a low-density slice point cloud. The low-density slice point cloud is specified by the following variables.

[0569] The variable SkippedLayerGroup indicates the specific number of layer groups to apply for partial decoding skipping in the density direction. SkippedLayerGroup should be in the range of 0 to num_layer_groups_minus1.

[0570] The variable MinNodeSizeLog2 indicates the minimum occupied tree node size specified by SkippedLayerGroup.

[0571] The array SubgroupNodePos[layerGroupIdx][subgroupIdx][ptIdx][k] indicates the output node of the subgroup identified by the layer group index (layerGroupIdx or layer_group_id) and the subgroup index (subgroupIdx or subgroup_id).

[0572] The array SubgroupNodeCnt[layerGroupIdx][subgroupIdx] indicates the number of output nodes in the subgroup identified by the layer group index (layerGroupIdx or layer_group_id) and the subgroup index (subgroupIdx or subgroup_id).

[0573] Next, the selection of FGS in partial density decoding will be described below.

[0574] When SkippedLayerGroup is greater than 0, layer groups with indices ranging from 0 to OutLayerGroup are selected for decoding. OutLayerGroup is the maximum value of the layer group indices used for partial decoding, specified as the value obtained by subtracting SkippedLayerGroup from the total number of layer groups (num_layer_groups_minus1), as shown below.

[0575] OutLayerGroup := num_layer_groups_minus1 – SkippedLayerGroup

[0576] The following code indicates whether a data unit or related data unit is decoded or skipped.

[0577] For example, if layer_group_id == 0, then decode the geometry data unit (GDU) or attribute data unit (ADU). If layer_group_id ≤ OutLayerGroup, then decode the associated geometry data unit (DGDU) or associated attribute data unit (DADU). Otherwise, skip the associated geometry data unit (DGDU) or associated attribute data unit (DADU).

[0578] if (layer_group_id == 0)

[0579] decode GDU or ADU

[0580] else if (layer_group_id ≤ OutLayerGroup)

[0581] decode DGDU or DADU

[0582] else

[0583] Skip DGDU or DADU

[0584] Therefore, the depth of the geometric occupancy tree used for partial decoding is inferred as the sum of the number of layers (or levels) of each index in the layer group ranging from 0 to OutLayerGroup (num_layers_minus1[i] + 1), as shown in the code below.

[0585] PartialDepth = 0

[0586] for (i = 0; i < NumOutLayerGroup; i++)

[0587] PartialDepth += num_layers_minus1[i] + 1

[0588] Next, the geometric position correction will be described.

[0589] When all layer groups are decoded, the maximum depth (TotalDepth) of the geometric occupancy tree is inferred as the sum of the number of layers (or levels) (num_layers_minus1[i] + 1) of each layer group in the range from 0 to num_layer_groups_minus1, as shown in the code below.

[0590] TotalDepth = 0

[0591] for (i = 0; i < num_layer_groups_minus1; i++)

[0592] TotalDepth += num_layers_minus1[i] + 1

[0593] Furthermore, MinNodeSizeLog2 is inferred as the difference between TotalDepth and PartialDepth, as shown below.

[0594] MinNodeSizeLog2 = TotalDepth - PartialDepth

[0595] When MinNodeSizeLog2 is greater than 1, the point is placed at the center of the corresponding block, as shown in the code below. That is, when MinNodeSizeLog2 is greater than 1, the following code adjusts the coordinate values ​​so that each point is placed at the center of the corresponding block. In the following code, the first for loop iterates over all points (or nodes) belonging to the corresponding subgroup, and the second for loop iterates over each axis (x, y, and z) of each point.

[0596] for (ptIdx = 0; ptIdx < SubgroupNodeCnt[layerGroupIdx][subgroupIdx];ptIdx++)

[0597] for (k = 0; k < 3; k++)

[0598] SubgroupNode[layerGroupIdx][subgroupIdx][ptIdx][k] |=(MinNodeSizeLog2 > 1) << (MinNodeSizeLog2 – 1)

[0599] Next, we will describe the decoding of some regions.

[0600] The decoder generates a point cloud of a portion of the slice. In this case, the low-density slice point cloud is specified by the following variables.

[0601] The arrays RoiBoundingBoxMin and RoiBoundingBoxMax are application-specific arrays that specify the minimum and maximum positions of the bounding box for the region of interest (ROI).

[0602] The array SubgroupNodePos[layerGroupIdx][subgroupIdx] indicates the output node of the subgroup identified by the layer group index (layerGroupIdx or layer_group_id) and the subgroup index (subgroupIdx or subgroup_id).

[0603] The array SubgroupNodeCnt[layerGroupIdx][subgroupIdx] indicates the number of output nodes in the subgroup identified by the layer group index (layerGroupIdx or layer_group_id) and the subgroup index (subgroupIdx or subgroup_id).

[0604] Next, the selection of FGS in partial region decoding will be described below.

[0605] When RoiBoundingBoxMin and RoiBoundingBoxMax exist, the subgroup whose bounding box overlaps with the bounding box of the ROI is selected for decoding. In other words, the subgroup is only selected for decoding FGS (geometric / attribute data) when the bounding box of the ROI overlaps with the bounding box of the subgroup.

[0606] The following code indicates whether a data unit or related data unit is decoded or skipped.

[0607] For example, if `layer_group_id == 0`, then either a geometric data unit (GDU) or an attribute data unit (ADU) is decoded. That is, layer group 0 is decoded unconditionally. Other layer groups are selected and decoded only if their ROI regions overlap with the bounding boxes of their subgroups. For example, if `RoiBoundingBoxMin < SubgroupBoundingBoxMax[layerGroupIdx][subgroupIdx] && RoiBoundingBoxMax < SubgroupBoundingBoxMin[layerGroupIdx][subgroupIdx]`, then either a relevant geometric data unit (DGDU) or a relevant attribute data unit (DADU) is decoded. Otherwise, either a DGDU or a DADU is skipped. In this way, the decoder can skip decoding subgroup data that is irrelevant to the ROI, thus achieving partial decoding and optimizing decoding efficiency.

[0608] if (layerGroupIdx == 0)

[0609] decode GDU or ADU

[0610] else if ((RoiBoundingBoxMin[0] < SubgroupBoundingBoxMax[layerGroupIdx][subgroupIdx][0] &&

[0611] RoiBoundingBoxMin[1] < SubgroupBoundingBoxMax[layerGroupIdx][subgroupIdx][1] &&

[0612] RoiBoundingBoxMin[2] < SubgroupBoundingBoxMax[layerGroupIdx][subgroupIdx][2]) &&

[0613] (RoiBoundingBoxMax[0] > SubgroupBoundingBoxMin[layerGroupIdx][subgroupIdx][0] &&

[0614] RoiBoundingBoxMax[1] > SubgroupBoundingBoxMin[layerGroupIdx][subgroupIdx][1] && RoiBoundingBoxMax[2] > SubgroupBoundingBoxMin[layerGroupIdx][subgroupIdx][2]))

[0615] decode DGDU or DADU

[0616] else

[0617] Skip DGDU or DADU

[0618] The encoder of the transmitting device according to this disclosure can be as follows: Figure 20 (a) to Figure 20 As shown in (f), the slice geometry and one or more slice attributes within a slice are divided into multiple geometric data units and multiple attribute data units, respectively. The geometric data units and attribute data units are rearranged according to each subgroup, and then the data units of the same subgroup are packaged into an FGS for transmission. For example, the first geometric data unit of the slice geometry, the first attribute data unit of the first slice attribute, and the first attribute data unit of the second slice attribute are packaged into FGS0. Therefore, an FGS includes a geometric data unit of a corresponding subgroup and one or more attribute data units of the corresponding subgroup.

[0619] In other words, an FGS is grouped by subgroup. Specifically, within the same level group, a geometric data unit belonging to the same subgroup and one or more attribute data units belonging to the same subgroup are packaged into one FGS.

[0620] In this scenario, the receiver's decoder can recover the necessary portions of the geometry and attributes without delay, depending on various use cases. In this case, geometry recovery can be performed by a geometry decoder (e.g., Figure 18 The geometry decoder) performs the operation, and attribute recovery can be performed by the attribute decoder (e.g., Figure 18 The attribute decoder is executed.

[0621] In one embodiment of this disclosure, an FGS index (e.g., fgs_id) can be defined separately. In this case, FGS geometries and FGS attributes belonging to the same FGS can be identified by fgs_id. Additionally, public information can be transmitted via GDU or DGDU, while the transmission of redundant information in ADU or DADU can be skipped. For example, as... Figure 22a and Figure 22bAs shown, a signal can be sent in SPS to notify `unified_layer_group_structure_enabled`. For example... Figure 23 As shown, when `unified_layer_group_structure_enabled` is enabled, `fgs_id` can be signaled in the geometric data unit (GDU or DGDU). In this case, the decoder of the receiving device (e.g., Figure 18 The geometry decoder and attribute decoder can select and decode the FGS geometry (i.e., geometric data unit) and one or more FGS attributes (i.e., attribute data unit) of the desired FGS based on signaling information including unified_layer_group_structure_enabled and fgs_id.

[0622] In another embodiment of this disclosure, it can be based on Figure 31 or Figure 33 In a geometric data unit (i.e., GDU or DGDU), a pair of indexes, layer_group_id, used to identify layer groups and subgroup_id, used to identify subgroups, is signaled to distinguish geometric data units and attribute data units belonging to the same FGS. In this case, the pair can be defined as a concept for grouping GDU / ADU or DGDU / DADU, and the layer_group_id and subgroup_id of the subgroup can be used as FGS indices. In this case, the decoder of the receiving device (e.g., Figure 18 The geometry decoder and attribute decoder can select and decode the FGS geometry (i.e., geometric data unit) and one or more FGS attributes (i.e., attribute data unit) of the desired FGS based on signaling information including layer_group_id and subgroup_id.

[0623] Therefore, this disclosure can provide the following effects.

[0624] This disclosure describes a method for partitioning and transmitting compressed data for point cloud data according to predetermined criteria. Specifically, when using layered encoding (or hierarchical encoding) as an application of this disclosure, compressed data can be partitioned and transmitted according to layers. In this case, the storage and transmission efficiency on the transmitting side can be improved.

[0625] Furthermore, since geometric data units and attribute data units are rearranged for each subgroup, and one geometric data unit and one or more attribute data units within the same subgroup are packaged into a single FGS, the decoder can selectively perform partial decoding only on the FGS of a specific subgroup. Therefore, decoding latency can be significantly reduced, and efficient data recovery can be achieved even in real-time decoding environments.

[0626] Furthermore, since an FGS includes both geometric and attribute data at the subgroup level, the necessity of attribute data for a corresponding subgroup can be determined immediately based solely on whether the geometric data within the corresponding FGS has been decoded. Therefore, unnecessary attribute data can be omitted during partial decoding, thereby improving the efficiency of the overall recovery process.

[0627] As described above, in this disclosure, compressed data for point cloud data is divided and transmitted according to predetermined criteria. Specifically, when using hierarchical encoding, compressed data can be divided and transmitted according to layers, thereby improving the storage and transmission efficiency on the transmitting side.

[0628] Figure 34 An example is illustrated by providing services through compressing the geometry and attributes of point cloud data. That is, in point cloud compression (PCC)-based services, the compression rate or data volume can be adjusted for transmission based on receiver performance or transmission environment. However, if... Figure 34 As shown, when point cloud data is grouped by slice, changes in receiver performance or transmission environment necessitate either 1) pre-transforming the bitstream according to its respective environment, storing the transformed bitstream separately, and selecting one for transmission, or 2) performing the transformation process (transcoding) before transmission. In this case, as the number of receiver environments to be supported increases or transmission environments change frequently, storage space issues or transcoding-related delays may occur.

[0629] Figure 35 Another example of providing services by compressing the geometry and attributes of point cloud data according to an implementation method is illustrated.

[0630] Before transmission, point cloud data, including geometry and attributes, can be preserved as is and / or resampled (or downscaled) depending on the performance of various decoders. Each resulting data point can then be encoded by its corresponding encoder and stored in a storage unit. The individual data points can then be transmitted to a receiving device via a network. In this case, full-resolution point cloud data and / or resampled (or downscaled) point cloud data can be decoded by the corresponding decoder and then rendered as either full-resolution or low-resolution (i.e., resampled) point cloud data.

[0631] Figure 36This illustrates yet another example of providing services by compressing the geometry and attributes of point cloud data according to an implementation method.

[0632] As proposed in this disclosure, when compressed data is divided and delivered according to layers, the advantage is that, for pre-compressed data, only the necessary portions can be selectively delivered at the bitstream stage via a bitstream selector without the need for additional conversion processes. This scheme is efficient in terms of storage space, as only one storage space is required per stream. Furthermore, since the bitstream selector selects only the necessary layers before transmission, efficient transmission in terms of bandwidth can be achieved.

[0633] Regarding the effects of the features of this disclosure from the receiver's perspective, when layered coding (or hierarchical coding) is used as one of the applications, compressed data can be divided and transmitted according to layers. In this case, the efficiency of the receiver can be improved. Specifically, although applying scalable attribute coding can impose latency and computational burden on the receiver because the entire encoded geometry must be received and decoded, the scheme of this disclosure allows decoding only the geometry that matches the tree level used in scalable attribute coding, thereby reducing latency and improving decoder efficiency by saving the computational power required for decoding.

[0634] Figure 37 This diagram illustrates the operations of the transmitting and receiving sides when transmitting point cloud data consisting of layers. In this regard, if the information used to reconstruct the entire point cloud data is delivered regardless of the receiver's performance, the receiver requires a process (e.g., data selection or resampling) to select only the point cloud data corresponding to the necessary layers after reconstructing the point cloud data through decoding. Since the bitstream delivered in this case is decoded, it may cause latency in receivers designed for low latency, or decoding may not be performed depending on the receiver's performance.

[0635] Figure 38 Another example of providing services by compressing the geometry and attributes of point cloud data according to an implementation method is illustrated.

[0636] exist Figure 38In this process, the bitstream is divided into slices by a scalable encoder and sent to a receiving device or stored in a storage unit. The receiving device can then select the cloud data required for decoding using a bitstream selector and perform full-resolution decoding or partial decoding using its respective decoder. Specifically, when the bitstream is divided into slices and sent, the receiving device can selectively provide the bitstream to the decoder based on the density of the point cloud data to be represented, based on decoder performance or application domain. In this case, the bitstream selector can select information related to the layer to be selected at the NAL unit level based on the nal_spatial_id and nal_data_type in the NAL unit header. In this case, the selection can be performed before decoding. Therefore, decoder efficiency can be improved, and decoders with various performance levels can be supported with a single bitstream.

[0637] Figure 39 This is a flowchart illustrating a method for sending point cloud data according to an implementation method.

[0638] The method for transmitting point cloud data according to the implementation may include acquiring point cloud data, encoding the point cloud data, and transmitting the encoded point cloud data and signaling information. In this case, the bitstream containing the encoded point cloud data and signaling information can be encapsulated into a file and transmitted.

[0639] In operation 71001, which acquires point cloud data, the following can be executed: Figure 1 Some or all of the operations of point cloud video acquisition unit 10001 or Figure 8 Some or all of the operations of the data input unit 8000.

[0640] In operation 71002, which encodes point cloud data, the following can be executed: Figure 1 Point cloud video encoder 10002 Figure 2 The code 20001 Figure 3 Point cloud video encoder, Figure 8 Point cloud video encoder, Figure 14 encoder, Figure 15 Attribute encoding and Figure 17 Some or all of the operations of the geometry encoder and attribute encoder are used to encode geometric and attribute information.

[0641] The operation 71002 for encoding point cloud data according to the embodiment may include compressing the geometric information and compressing the attribute information of the input point cloud data.

[0642] According to the implementation, when compressing geometric information and attribute information, the geometric information and attribute information can be compressed based on layer groups and / or subgroups. That is, the operation 71002 for encoding point cloud data may include dividing the slice geometry in the slice into multiple geometric data units, dividing one or more slice attributes into multiple attribute data units respectively, and compressing the geometric data units and attribute data units through the operations of compressing geometric information and compressing attribute information.

[0643] Compressed geometric and attribute information can be entropy encoded separately and output as geometric bitstreams and attribute bitstreams, respectively.

[0644] In this context, compressing attribute information may include compressing the attribute information based on the location and / or reconstructed geometry information from which no geometric encoding has been performed. In one implementation, attribute information may be encoded using one or a combination of two or more of RAHT coding, LOD-based predictive transform coding, and lifting transform coding.

[0645] The operation of transmitting encoded point cloud data and signaling information according to the implementation method can use a method based on subgroup configuration and transmitting FGS. For example, as Figure 20 (a) to Figure 20 As shown in (f), after rearranging the encoded data units according to the geometric / attribute order based on the subgroups, a geometric data unit and one or more attribute data units from the same subgroup can be packaged into an FGS and sent. That is, when geometry and attributes are sliced ​​based on the same layer group structure, the operation of sending encoded point cloud data and signaling information can include sending the geometric and attribute data corresponding to each subgroup as a bundle (i.e., FGS). In this case, the FGS can be considered as a unit for sending all data in the subgroup, and the geometric and attribute data can be sent as a bundle.

[0646] In other words, in the case of FGS, slices are configured in a layer group structure, and data units are encoded in units of nodes belonging to a predetermined spatial region and depth. Then, FGS geometry (i.e., geometric data units) and FGS attributes (i.e., attribute data units) can be bundled and defined as a single encoding unit, i.e., an FGS.

[0647] In one implementation, to identify the FGS geometry (i.e., geometric data unit) and FGS attributes (i.e., attribute data unit) included in the FGS, it can be done as follows: Figure 22a and Figure 22b As shown, in SPS, a signal is sent to notify unified_layer_group_structure_enabled, and it can be done as follows: Figure 23The signal is sent to fgs_id in the geometric data unit (GDU or DGDU).

[0648] In another embodiment, in order to identify the FGS geometry (i.e., geometric data unit) and FGS attributes (i.e., attribute data unit) included in the FGS, it is possible to... Figure 31 or Figure 33 The geometric data unit (i.e., GDU or DGDU) signals the layer_group_id and subgroup_id.

[0649] In this disclosure, fgs_id and / or layer_group_id and subgroup_id may be referred to as FGS identification information.

[0650] In this disclosure, the signaling information may include layer-group-based slice-related information for encoding and decoding geometric and attribute information on a layer-group and / or sub-group basis at the receiving end. Layer-group-based slice-related information may include FGS identification information.

[0651] Layer-group-based slice-related information may be included in at least one of SPS, APS, LGSI, geometric data cell header (or associated geometric data cell header), and / or attribute data cell header (or associated attribute data cell header) and sent to the receiving side. For details included in layer-group-based slice-related information not described below, please refer to [reference needed]. Figures 21 to 27 or Figures 28 to 33 .

[0652] Figure 40 This is a flowchart illustrating a method for receiving point cloud data according to an embodiment.

[0653] The method for receiving point cloud data according to the embodiments may include receiving encoded point cloud data and signaling information (81001), decoding the point cloud data based on the signaling information (81002), and rendering the decoded point cloud data (81003).

[0654] The operation 81001 for receiving point cloud data and signaling information according to the implementation method can be performed by... Figure 1 Receiver 10005 Figure 2 Transmission 20002 or decoding 20003, or Figure 9 The receiver 9000 or receiver processor 9001 shall execute this.

[0655] In the point cloud data decoding operation 81002 according to the implementation method, the following can be performed: Figure 1 Point cloud video decoder 10006 Figure 2 Decoding 20003 Figure 8Point cloud video decoder, Figure 9 Point cloud video decoder, Figure 14 decoder Figure 16 Decoding and Figure 18 Some or all of the operations of the geometry decoder and attribute decoder are used to decode geometric and attribute information.

[0656] In one implementation, during the point cloud data decoding operation 81002, the desired FGS geometry (i.e., geometric data unit) and FGS attributes (i.e., attribute data unit) can be selected and decoded based on signaling information including unified_layer_group_structure_enabled and fgs_id.

[0657] In another embodiment, during the point cloud data decoding operation 81002, the FGS geometry (i.e., geometric data unit) and FGS attributes (i.e., attribute data unit) of the desired FGS can be selected and decoded based on signaling information including layer_group_id and subgroup_id.

[0658] In this disclosure, fgs_id and / or layer_group_id and subgroup_id may be referred to as FGS identification information. Additionally, slice-related information based on layer groups may include FGS identification information.

[0659] The operation 81002 for decoding point cloud data according to the embodiment may include decoding geometric information and decoding attribute information.

[0660] Decoding geometric information can include decoding (i.e., reconstructing) the geometric information (e.g., geometric information included in a specific FGS) on a per-layer-group and / or-sub-group basis, based on slice-related information included in the signaling information. Since the details of decoding geometric information have been sufficiently described above, their redundant description will be omitted here.

[0661] Decoding attribute information may include decoding (i.e., decompressing) attribute information (e.g., attribute information included in a specific FGS) on a per-layer-group and / or-sub-group basis, based on the reconstructed geometric information and slice-related information included in the signaling information. In one implementation, attribute information may be decoded using one or a combination of RAHT coding, LOD-based predictive transform coding, and lifting transform coding. Since the details of decoding attribute information have been sufficiently described above, their redundant description will be omitted here.

[0662] The rendering (81003) according to the embodiment may include reconstructing point cloud data based on reconstructed geometric and attribute information, and performing rendering according to various rendering methods. For example, points in the point cloud content may be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at vertex positions, or circles centered at vertex positions. All or part of the rendered point cloud content is provided to the user through a display (e.g., a VR / AR display, a general display, etc.). The operation (81003) of rendering point cloud data according to the embodiment may be performed by… Figure 1 Renderer 10007 Figure 2 Rendering 20004 or Figure 9 The renderer 9011 is used to execute it.

[0663] Each of the above-described parts, modules, or units can be a software, processor, or hardware component that performs a series of processes stored in memory (or storage units). Each of the steps described in the above embodiments can be executed by a processor, software, or hardware component. Each module / block / unit described in the above embodiments can operate as a processor, software, or hardware. Furthermore, the methods presented by the embodiments can be executed as code. This code can be written to a processor-readable storage medium and thus read by a processor provided by the device.

[0664] In this specification, when a part "contains" or "includes" an element, unless otherwise stated, it means that the part also contains or includes another element. Furthermore, the term "module (or unit)" disclosed in this specification refers to a unit for performing at least one function or operation, and can be implemented by hardware, software, or a combination of hardware and software.

[0665] Although embodiments have been explained with reference to each of the accompanying drawings for simplicity, new embodiments can be designed by incorporating the embodiments illustrated in the drawings. If a person skilled in the art designs a computer-readable recording medium containing a program for performing the embodiments described above, it may fall within the scope of the appended claims and their equivalents.

[0666] The apparatus and methods are not limited to the configurations and methods of the above embodiments. The above embodiments can be configured by selectively or wholly combining with each other to achieve various modifications.

[0667] Although preferred embodiments of the embodiments have been shown and described, the embodiments are not limited to the specific embodiments described above, and various modifications can be made by those skilled in the art without departing from the spirit of the embodiments claimed in the claims, and these modifications should not be understood in isolation from the technical concept or view of the embodiments.

[0668] Various elements of the device according to the embodiments can be implemented by hardware, software, firmware, or a combination thereof. Various elements in the embodiments can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiments, the components according to the embodiments can be implemented as separate chips. According to the embodiments, at least one or more components of the device according to the embodiments can include one or more processors capable of executing one or more programs. One or more programs can perform any one or more operations / methods according to the embodiments or include instructions for performing them. Executable instructions for performing the methods / operations of the device according to the embodiments can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transient CRM or other computer program product configured to be executed by one or more processors. Additionally, the memory according to the embodiments can be used to encompass not only volatile memory (e.g., RAM) but also the concepts of non-volatile memory, flash memory, and PROM. It can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, the processor-readable recording medium can be distributed to computer systems connected via a network, such that processor-readable code can be stored and executed in a distributed manner.

[0669] In this disclosure, the terms “ / ” and “,” should be interpreted as indicating “and / or”. For example, the expression “A / B” can mean “A and / or B”. Furthermore, “A, B” can mean “A and / or B”. Additionally, “A / B / C” can mean “at least one of A, B, and / or C”. Also, “A / B / C” can mean “at least one of A, B, and / or C”. Furthermore, in this document, the term “or” should be interpreted as indicating “and / or”. For example, the expression “A or B” can include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term “or” in this disclosure should be interpreted as indicating “additionally or alternatively”.

[0670] The various elements of the embodiments can be implemented by hardware, software, firmware, or a combination thereof. The various elements of the embodiments can be executed by a single chip, such as a single hardware circuit. According to the embodiments, the elements can be selectively executed by separate chips. According to the embodiments, at least one of the elements of the embodiments can be executed in one or more processors including instructions for performing the operations according to the embodiments.

[0671] Operations according to the embodiments described in this specification can be performed by a transmitting / receiving device including one or more memories and / or one or more processors according to the embodiments. One or more memories may store programs for processing / controlling the operations according to the embodiments, and one or more processors may control the various operations described in this specification. One or more processors may be referred to as controllers, etc. In the embodiments, operations can be performed by firmware, software, and / or combinations thereof. Firmware, software, and / or combinations thereof may be stored in a processor or memory.

[0672] Terms such as "first" and "second" can be used to describe various elements of an embodiment. However, the various components according to an embodiment should not be limited by the terms described above. These terms are used only to distinguish one element from another. For example, a first user input signal can be referred to as a second user input signal. Similarly, a second user input signal can be referred to as a first user input signal. The use of these terms should not be interpreted beyond the scope of the various embodiments. Both the first user input signal and the second user input signal are user input signals, but they do not mean the same user input signal unless the context clearly specifies otherwise. The terms used to describe embodiments are used only for the purpose of describing particular embodiments and are not intended to limit the embodiments. As used in the description of embodiments and claims, the singular forms "a," "an," and "the" include the plural of the referred objects unless the context clearly specifies otherwise. The expression "and / or" is used to include all possible combinations of terms. Terms such as "comprising" or "having" are intended to indicate the presence of figures, quantities, steps, elements, and / or components and should be understood not to exclude the possibility of additional figures, quantities, steps, elements, and / or components.

[0673] As used herein, conditional expressions such as “if” and “when” are not limited to optional cases and are intended to be interpreted as performing a related operation or interpreting a related definition in accordance with a specific condition when that condition is met. Implementations may include variations / modifications within the scope of the claims and their equivalents. It will be apparent to those skilled in the art that various modifications and variations can be made to this disclosure without departing from the spirit and scope thereof. Therefore, this disclosure is intended to cover modifications and variations thereof as long as they fall within the scope of the appended claims and their equivalents.

[0674] Public content model

[0675] As described above, the relevant content is presented in the best mode for implementing the embodiment.

[0676] Industrial applicability

[0677] As described above, the embodiments can be applied in whole or in part to point cloud data transmitting / receiving apparatuses and systems. It will be apparent to those skilled in the art that various changes or modifications can be made to the embodiments within their scope. Therefore, the embodiments are intended to cover modifications and variations as long as they fall within the scope of the appended claims and their equivalents.

Claims

1. A decoding method, the decoding method comprising the following steps: Receive point cloud data and signaling information; as well as Decode the point cloud data. The point cloud data includes geometric data and attribute data.

2. The decoding method according to claim 1, wherein, The point cloud data is decoded based on the layer groups and subgroups of the occupancy tree. Each of the layer groups is a group of consecutive tree levels of the occupied tree, and Each of the subgroups is a spatial subset of the layer group.

3. The decoding method according to claim 2, wherein, The geometric data is included in multiple geometric data units. The attribute data is included in multiple attribute data units. The plurality of geometric data units and the plurality of attribute data units are packaged into subgroup-based coding units, and Each of the encoding units includes geometric data units and attribute data units in the same subgroup.

4. The decoding method according to claim 3, wherein, The signaling information includes identification information for identifying each of the coding units, and The step of decoding the point cloud data includes: The geometric data units and attribute data units included in the encoding unit identified by the identification information are decoded.

5. The decoding method according to claim 4, wherein, The identification information includes a pair of layer group identification information and subgroup identification information. The layer group identification information is used to identify the layer group, and the subgroup identification information is used to identify the subgroup.

6. The decoding method according to claim 4, wherein, The identification information includes the encoding unit ID information.

7. An encoding method, the encoding method comprising the following steps: Encode the point cloud data; as well as Send the encoded point cloud data and signaling information. The point cloud data includes geometric data and attribute data.

8. The encoding method according to claim 7, wherein, The geometric data and attribute data of the point cloud data are encoded based on the layer groups and subgroups of the occupancy tree. Each of the layer groups is a group of consecutive tree levels of the occupied tree, and Each of the subgroups is a spatial subset of the layer group.

9. The encoding method according to claim 8, wherein, The encoded geometric data is included in multiple geometric data units. The encoded attribute data is contained within multiple attribute data units. The plurality of geometric data units and the plurality of attribute data units are packaged into a subgroup-based coding unit and sent. Each of the encoding units includes geometric data units and attribute data units in the same subgroup.

10. The encoding method according to claim 9, wherein, The plurality of geometric data units and the plurality of attribute data units are rearranged based on the subgroups and then packaged into the encoding unit.

11. The encoding method according to claim 9, wherein, The signaling information includes identification information used to identify each of the coding units.

12. The encoding method according to claim 11, wherein, The identification information includes a pair of layer group identification information and subgroup identification information. The layer group identification information is used to identify the layer group, and the subgroup identification information is used to identify the subgroup.

13. The encoding method according to claim 11, wherein, The identification information includes the encoding unit ID information.

14. A method for transmitting data, the method comprising the following steps: Encode the point cloud data; as well as Send a bitstream containing encoded point cloud data and signaling information. The point cloud data includes geometric data and attribute data.

15. The sending method according to claim 14, wherein, The geometric data and attribute data of the point cloud data are encoded based on the layer groups and subgroups of the occupancy tree. Each of the layers in the group is a group of consecutive tree levels of the occupied tree. Each of the subgroups is a spatial subset of the layer group. The encoded geometric data is contained within multiple geometric data units. The encoded attribute data is contained within multiple attribute data units. The plurality of geometric data units and the plurality of attribute data units are packaged into a subgroup-based coding unit and sent. Each of the encoding units includes geometric data units and attribute data units in the same subgroup.

Images