Method and apparatus for processing point cloud data

Abstract

In the method of processing point cloud data according to the embodiments, the point cloud data can be encoded and sent to a bitstream. In the method of processing point cloud data according to the embodiments, the bitstream including the point cloud data can be received, and the point cloud data can be decoded.

Description

TECHNICAL FIELD

[0001] The disclosure provides a method for providing point cloud content to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and self-driving services to a user. BACKGROUND

[0002] Point cloud content is content represented by a point cloud, which is a set of points belonging to a coordinate system representing a three-dimensional space. The point cloud content can be expressed in a media configured in three dimensions, and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and self-driving services. However, tens of thousands to hundreds of thousands of point data are required to represent the point cloud content. Therefore, a method for efficiently processing a large amount of point data is required. SUMMARY

[0003] TECHNICAL PROBLEM

[0004] Embodiments provide an apparatus and a method for efficiently processing point cloud data. Embodiments provide a point cloud data processing method and apparatus for solving latency and encoding / decoding complexity.

[0005] The technical scope of the embodiments is not limited to the above technical objects, and can be extended to other technical objects which can be inferred from the entire contents disclosed herein by those skilled in the art.

[0006] TECHNICAL SOLUTION

[0007] To achieve these objects and other advantages and according to the purpose of the disclosure, in some embodiments, a method of transmitting point cloud data can include the steps of encoding point cloud data including geometry information and attribute information and transmitting a bitstream including the encoded point cloud data. In some embodiments, the geometry information represents positions of points of the point cloud data, and the attribute information represents attributes of the points of the point cloud data.

[0008] In some embodiments, a method of processing point cloud data can include the steps of receiving a bitstream including point cloud data. In some embodiments, the point cloud data includes geometry information and attribute information, wherein the geometry information represents positions of points of the point cloud data, and the attribute information indicates one or more attributes of the points of the point cloud data. The point cloud data processing method can include the step of decoding the point cloud data.

[0009] In some embodiments, a method for processing point cloud data can include the steps of receiving a bitstream including point cloud data and decoding the point cloud data. In some embodiments, the point cloud data includes geometry information and attribute information, wherein the geometry information represents positions of points of the point cloud data, and the attribute information indicates one or more attributes of the points of the point cloud data.

[0010] In some embodiments, an apparatus for processing point cloud data can include a receiver configured to receive a bitstream including point cloud data, and a decoder configured to decode the point cloud data. In some embodiments, the point cloud data includes geometry information and attribute information, wherein the geometry information represents positions of points of the point cloud data, and the attribute information indicates one or more attributes of the points of the point cloud data.

[0011] Advantages

[0012] The apparatus and method according to embodiments can efficiently process point cloud data.

[0013] The apparatus and method according to embodiments can provide high-quality point cloud services.

[0014] The apparatus and method according to embodiments can provide point cloud content for providing general services such as VR services and self-driving services. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure.

[0016] In order to understand the various embodiments described below, it is to be understood that terms such as "including", "comprising", and "having" are intended to be open and permeable and thus do not exclude other elements from being added. Other apparently similar terms have meanings that are similar to the meanings of the previously mentioned terms.

[0017] FIG. 1 An exemplary point cloud content providing system according to an embodiment is illustrated.

[0018] FIG. 2 is a block diagram illustrating a point cloud content providing operation according to an embodiment.

[0019] FIG. 3 An exemplary process of capturing a point cloud video according to an embodiment is illustrated.

[0020] FIG. 4 An exemplary point cloud encoder according to an embodiment is illustrated.

[0021] FIG. 5 An example of a voxel according to an embodiment is illustrated.

[0022] FIG. 6 An example of an octree and an occupancy code according to an embodiment is illustrated.

[0023] FIG. 7 An example of a neighbor node pattern according to an embodiment is illustrated.

[0024] FIG. 8An example of point configuration in various LODs according to an embodiment is shown.

[0025] FIG. 9 An example of point configuration in various LODs according to an embodiment is shown.

[0026] FIG. 10 An example of a point cloud decoder according to an embodiment is shown.

[0027] FIG. 11 An example of a point cloud decoder according to an embodiment is shown.

[0028] FIG. 12 An example of a transmitting apparatus according to an embodiment is shown.

[0029] FIG. 13 An example of a receiving apparatus according to an embodiment is shown.

[0030] FIG. 14 An architecture for streaming G-PCC based point cloud data according to an embodiment is shown.

[0031] FIG. 15 An example of a point cloud transmitting apparatus according to an embodiment is shown.

[0032] FIG. 16 An example of a point cloud receiving apparatus according to an embodiment is shown.

[0033] FIG. 17 An example of an architecture operatively connectable with methods / apparatuses for transmitting and receiving point cloud data according to an embodiment is shown.

[0034] FIG. 18 is a block diagram showing an example point cloud encoder.

[0035] FIG. 19 is a block diagram showing an example geometry information encoder.

[0036] FIG. 20 An example of an attribute information encoder according to an embodiment is shown.

[0037] FIG. 21 An example of an attribute information encoder according to an embodiment is shown.

[0038] FIG. 22 An example of an attribute information predictor according to an embodiment is shown.

[0039] FIG. 23 An example LOD generation process is shown.

[0040] FIG. 24 An example Morton order based sampling process is shown.

[0041] FIG. 25 An LOD generation process according to an embodiment is shown.

[0042] FIG. 26 An exemplary structure diagram of a point cloud compression (PCC) bitstream is shown.

[0043] FIG. 27 An exemplary syntax of an APS according to an embodiment is shown.

[0044] FIG. 28 An exemplary syntax of an APS according to an embodiment is shown.

[0045] FIG. 29 An exemplary syntax of a TPS according to an embodiment is shown.

[0046] FIG. 30 An exemplary syntax of a TPS according to an embodiment is shown.

[0047] FIG. 31 An exemplary syntax of an attribute header according to an embodiment is shown.

[0048] FIG. 32 An exemplary syntax of an attribute header according to an embodiment is shown.

[0049] FIG. 33 is a block diagram showing an exemplary point cloud decoder.

[0050] FIG. 34 is a block diagram showing an example of a geometry information decoder.

[0051] FIG. 35 is a block diagram showing an exemplary attribute information decoder.

[0052] FIG. 36 is an exemplary flowchart of a point cloud data processing method according to an embodiment.

[0053] FIG. 37 is an exemplary flowchart of a point cloud data processing method according to an embodiment. DETAILED DESCRIPTION

[0054] Reference will now be made in detail to the preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The detailed description, which will be given below with reference to the accompanying drawings, is intended to explain exemplary embodiments of the present disclosure, rather than to show the only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without such specific details.

[0055] Although most of the terms used in the present disclosure are selected from general terms widely used in the art, some terms are arbitrarily selected by the applicant and the meaning thereof is explained in detail in the following description as needed. Therefore, the present disclosure should be understood based on the intended meaning of the terms rather than their simple names or meanings.

[0056] FIG. 1 An exemplary point cloud content providing system according to an embodiment is illustrated.

[0057] FIG. 1 The illustrated point cloud content providing system can include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 can communicate in wired or wireless communication to transmit and receive point cloud data.

[0058] The point cloud data transmitting device 10000 according to an embodiment can acquire and process point cloud videos (or point cloud contents) and transmit the same. According to an embodiment, the transmitting device 10000 can include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device, and / or a server. According to an embodiment, the transmitting device 10000 can include a device configured to perform communication with a base station and / or other wireless devices using a radio access technology (e.g., 5G new RAT (NR), long term evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an internet of things (IoT) device, and an AI device / server.

[0059] The transmitting device 10000 according to an embodiment includes a point cloud video acquirer 10001, a point cloud video encoder 10002, and / or a transmitter (or a communication module) 10003.

[0060] The point cloud video acquirer 10001 according to an embodiment acquires a point cloud video through a processing procedure such as capturing, synthesizing, or generating. The point cloud video is point cloud content represented by a point cloud, which is a set of points located in a 3D space, and can be referred to as point cloud video data. The point cloud video according to an embodiment can include one or more frames. One frame represents a still image / picture. Accordingly, the point cloud video can include point cloud images / frames / pictures, and can be referred to as a point cloud image, frame, or picture.

[0061] The point cloud video encoder 10002 according to the embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 can encode the point cloud video data based on point cloud compression encoding. The point cloud compression encoding according to the embodiment can include geometry-based point cloud compression (G-PCC) encoding and / or video-based point cloud compression (V-PCC) encoding or next generation encoding. The point cloud compression encoding according to the embodiment is not limited to the above-described embodiment. The point cloud video encoder 10002 can output a bitstream containing the encoded point cloud video data. The bitstream can contain not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0062] The transmitter 10003 according to the embodiment transmits the bitstream containing the encoded point cloud video data. The bitstream according to the embodiment is encapsulated in a file or a segment (e.g., a streaming segment) and transmitted via various networks such as a broadcast network and / or a broadband network. Although not shown in the drawing, the transmission device 10000 can include an encapsulator (or an encapsulation module) configured to perform an encapsulation operation. According to the embodiment, the encapsulator can be included in the transmitter 10003. According to the embodiment, the file or the segment can be transmitted to the reception device 10004 via a network or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitter 10003 according to the embodiment is capable of wired / wireless communication with the reception device 10004 (or the receiver 10005) via a 4G, 5G, 6G, etc. network. In addition, the transmitter can perform necessary data processing operations according to a network system (e.g., a 4G, 5G, or 6G communication network system). The transmission device 10000 can transmit the encapsulated data in an on-demand manner.

[0063] The reception device 10004 according to the embodiment includes the receiver 10005, the point cloud video decoder 10006, and / or the Tenderer 10007. According to the embodiment, the reception device 10004 can include a device, a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server configured to perform communication with a base station and / or other wireless devices using a radio access technology (e.g., 5G New RAT (NR), Long Term Evolution (LTE)).

[0064] The receiver 10005 according to the embodiments receives a bitstream containing point cloud video data or a file / segment encapsulating the bitstream from a network or a storage medium. The receiver 10005 can perform necessary data processing according to a network system (e.g., a communication network system of 4G, 5G, 6G, or the like). The receiver 10005 according to the embodiments can decapsulate the received file / segment and output the bitstream. According to the embodiments, the receiver 10005 can include a decapsulator (or a decapsulation module) configured to perform a decapsulation operation. The decapsulator can be implemented as a separate element (or component) from the receiver 10005.

[0065] The point cloud video decoder 10006 decodes the bitstream containing the point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to a method by which the point cloud video data is encoded (e.g., in the reverse process of the operation of the point cloud video encoder 10002). Accordingly, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding (the inverse process of point cloud compression). The point cloud decompression encoding includes G-PCC encoding.

[0066] The renderer 10007 renders the decoded point cloud video data. The renderer 10007 can output the point cloud content by rendering not only the point cloud video data but also audio data. According to the embodiments, the renderer 10007 can include a display configured to display the point cloud content. According to the embodiments, the display can be implemented as a separate device or component rather than being included in the renderer 10007.

[0067] The arrow indicated by the dotted line in the drawing represents a transmission path of feedback information acquired by the reception device 10004. The feedback information is information reflecting the interactivity with a user who consumes the point cloud content, and includes information about the user (e.g., head orientation information, viewport information, or the like). Specifically, when the point cloud content is content for a service (e.g., a self-driving service or the like) that requires interaction with the user, the feedback information can be provided to a content sender (e.g., the transmission device 10000) and / or a service provider. According to the embodiments, the feedback information can be used in the reception device 10004 as well as the transmission device 10000, or can not be provided.

[0068] The head orientation information according to the embodiments is information about a head position, orientation, angle, motion, etc. of the user. The reception device 10004 according to the embodiments can calculate viewport information based on the head orientation information. The viewport information can be information about a region of the point cloud video that the user is watching. The viewpoint is a point through which the user watches the point cloud video, and can refer to a center point of the viewport region. That is, the viewport is a region centered on the viewpoint, and the size and shape of the region can be determined by a field of view (FOV). Thus, the reception device 10004 can extract the viewport information based on a vertical or horizontal FOV supported by the device in addition to the head orientation information. In addition, the reception device 10004 performs gaze analysis, etc. to check the way in which the user consumes the point cloud, a region in which the user gazes in the point cloud video, a gaze time, etc. According to the embodiments, the reception device 10004 can transmit feedback information including the gaze analysis result to the transmission device 10000. The feedback information according to the embodiments can be acquired in the rendering and / or display process. The feedback information according to the embodiments can be taken by one or more sensors included in the reception device 10004. According to the embodiments, the feedback information can be taken by the Tenderer 10007 or a separate external element (or device, component, etc.). FIG. 1 The dotted line in the above equation indicates a process of transmitting feedback information taken by the Tenderer 10007. The point cloud content providing system can process (encode / decode) point cloud data based on the feedback information. Thus, the point cloud video data decoder 10006 can perform a decoding operation based on the feedback information. The reception device 10004 can transmit the feedback information to the transmission device 10000. The transmission device 10000 (or the point cloud video data encoder 10002) can perform an encoding operation based on the feedback information. Thus, the point cloud content providing system can efficiently process necessary data (e.g., point cloud data corresponding to the head position of the user) based on the feedback information rather than processing (encoding / decoding) the entire point cloud data, and provide point cloud content to the user.

[0069] According to the embodiments, the transmission device 10000 can be referred to as an encoder, a transmission device, a transmitter, etc., and the reception device 10004 can be referred to as a decoder, a reception device, a receiver, etc.

[0070] The point cloud content data processed in the point cloud content providing system according to the embodiments (through a series of processes of acquisition / encoding / transmission / decoding / rendering) can be referred to as point cloud content data or point cloud video data. According to the embodiments, the point cloud content data can be used as a concept that encompasses metadata or signaling information related to the point cloud data. FIG. 1 The point cloud content data processed in the point cloud content providing system according to the embodiments (through a series of processes of acquisition / encoding / transmission / decoding / rendering) can be referred to as point cloud content data or point cloud video data. According to the embodiments, the point cloud content data can be used as a concept that encompasses metadata or signaling information related to the point cloud data.

[0071] FIG. 1 The elements of the point cloud content providing system shown in the above equation can be implemented by hardware, software, a processor, and / or a combination thereof.

[0072] FIG. 2 is a block diagram illustrating a point cloud content providing operation according to an embodiment.

[0073] FIG. 2 the block diagram of FIG. 1 described in the point cloud content providing system. As described above, the point cloud content providing system can process point cloud data based on point cloud compression encoding (e.g., G-PCC).

[0074] The point cloud content providing system (e.g., point cloud transmitting apparatus 10000 or point cloud video acquirer 10001) according to an embodiment can acquire a point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system for expressing a 3D space. The point cloud video according to an embodiment can include a Ply (Polygon file format or Stanford Triangle format) file. When the point cloud video has one or more frames, the acquired point cloud video can include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. The geometry includes a position of a point. The position of each point can be represented by a parameter (e.g., values of X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). The attributes include attributes (e.g., information about a texture, a color (YCbCr or RGB), reflectance r, transparency, etc. of each point) of a point. A point has one or more attributes. For example, a point can have a color attribute or both color and reflectance attributes. According to an embodiment, the geometry can be referred to as a position, geometry information, geometry data, etc., and the attributes can be referred to as attributes, attribute information, attribute data, etc. The point cloud content providing system (e.g., point cloud transmitting apparatus 10000 or point cloud video acquirer 10001) can acquire point cloud data from information (e.g., depth information, color information, etc.) related to a point cloud video acquisition process.

[0075] The point cloud content providing system (e.g., transmitting apparatus 10000 or point cloud video encoder 10002) according to an embodiment can encode point cloud data (20001). The point cloud content providing system can encode the point cloud data based on point cloud compression encoding. As described above, the point cloud data can include geometry and attributes of points. Accordingly, the point cloud content providing system can perform geometry encoding that encodes the geometry and output a geometry bitstream. The point cloud content providing system can perform attribute encoding that encodes the attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system can perform the attribute encoding based on the geometry encoding. The geometry bitstream and the attribute bitstream according to an embodiment can be multiplexed and output as one bitstream. The bitstream according to an embodiment can further contain signaling information related to the geometry encoding and the attribute encoding.

[0076] A point cloud content providing system (e.g., a transmitting apparatus 10000 or a transmitter 10003) according to an embodiment can transmit encoded point cloud data (20002). As shown in FIG. 1 The encoded point cloud data can be represented by a geometry bitstream and an attribute bitstream. In addition, the encoded point cloud data can be transmitted in the form of a bitstream together with signaling information related to encoding of the point cloud data (e.g., signaling information related to geometry encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and transmit it in the form of a file or a segment.

[0077] A point cloud content providing system (e.g., a receiving apparatus 10004 or a receiver 10005) according to an embodiment can receive a bitstream containing encoded point cloud data. In addition, the point cloud content providing system (e.g., a receiving apparatus 10004 or a receiver 10005) can demultiplex the bitstream.

[0078] The point cloud content providing system (e.g., a receiving apparatus 10004 or a point cloud video decoder 10005) can decode the encoded point cloud data (e.g., a geometry bitstream, an attribute bitstream) transmitted in the bitstream. The point cloud content providing system (e.g., a receiving apparatus 10004 or a point cloud video decoder 10005) can decode the point cloud video data based on the signaling information related to encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., a receiving apparatus 10004 or a point cloud video decoder 10005) can decode the geometry bitstream to reconstruct the positions (geometry) of points. The point cloud content providing system can reconstruct the attributes of points by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., a receiving apparatus 10004 or a point cloud video decoder 10005) can reconstruct the point cloud video based on the positions from the reconstructed geometry and the decoded attributes.

[0079] A point cloud content providing system (e.g., a receiving apparatus 10004 or a renderer 10007) according to an embodiment can render the decoded point cloud data (20004). The point cloud content providing system (e.g., a receiving apparatus 10004 or a renderer 10007) can render the geometry and the attributes decoded through the decoding process using various rendering methods. The points in the point cloud content can be rendered as a vertex having a certain thickness, a cube having a certain minimum size centered on a corresponding vertex position, or a circle centered on a corresponding vertex position. All or part of the rendered point cloud content is provided to a user through a display (e.g., a VR / AR display, a general display, etc.).

[0080] The point cloud content providing system (e.g., reception device 10004) according to embodiments can acquire feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. The feedback information and operations of the point cloud content providing system according to embodiments are the same as those described with reference to FIG. 1 The feedback information and operations described are the same, and thus a detailed description thereof is omitted.

[0081] FIG. 3 An exemplary process of capturing a point cloud video according to embodiments is illustrated.

[0082] FIG. 3 An exemplary point cloud video capturing process of the point cloud content providing system described with reference to FIG. 1-2 An exemplary point cloud video capturing process of the point cloud content providing system described with reference to

[0083] Point cloud content includes point cloud videos (images and / or videos) representing objects and / or environments located in various 3D spaces (e.g., 3D spaces representing real environments, 3D spaces representing virtual environments, etc.). Thus, the point cloud content providing system according to embodiments can capture point cloud videos using one or more cameras (e.g., infrared cameras capable of acquiring depth information, RGB cameras capable of extracting color information corresponding to depth information, etc.), projectors (e.g., infrared pattern projectors acquiring depth information), LiDARs, etc. The point cloud content providing system according to embodiments can extract a geometry consisting of points in a 3D space from depth information and extract attributes of each point from color information to acquire point cloud data. The images and / or videos according to embodiments can be captured based on at least one of an inside-out technique and an outside-in technique.

[0084] FIG. 3 The left portion of FIG. 1 illustrates an inside-out technique. The inside-out technique refers to a technique of capturing an image of a central object using one or more cameras (or camera sensors) positioned around the central object. The inside-out technique can be used to generate point cloud content providing a 360-degree image of a key object to a user (e.g., VR / AR content providing a 360-degree image of an object (e.g., a key object such as a character, a player, an object, or an actor) to a user).

[0085] FIG. 3 The right portion of FIG. 1 illustrates an outside-in technique. The outside-in technique refers to a technique of capturing an image of an environment of a central object, rather than the central object, using one or more cameras (or camera sensors) positioned around the central object. The outside-in technique can be used to generate point cloud content providing a surrounding environment appearing from a user's perspective (e.g., content representing an external environment that can be provided to a user of a self-driving vehicle).

[0086] As illustrated, the point cloud content can be generated based on the capturing operation of one or more cameras. In this case, the coordinate system can be different between the cameras, and thus the point cloud content providing system can calibrate the one or more cameras to set a global coordinate system before the capturing operation. In addition, the point cloud content providing system can generate the point cloud content by synthesizing an arbitrary image and / or video with an image and / or video captured through the above-described capturing technique. The point cloud content providing system can not perform the capturing operation described in FIG. 3 FIG. 1. The point cloud content providing system according to the embodiments can perform post-processing on the captured image and / or video. In other words, the point cloud content providing system can remove an unwanted area (e.g., a background), identify a space to which the captured image and / or video is connected, and perform an operation of filling a space hole when there is a space hole.

[0087] The point cloud content providing system can generate one piece of point cloud content by performing coordinate transformation on points of a point cloud video taken from each camera. The point cloud content providing system can perform coordinate transformation on the points based on position coordinates of each camera. Thus, the point cloud content providing system can generate content representing a wide range, or can generate point cloud content having a high density of points.

[0088] FIG. 4 An exemplary point cloud encoder according to an embodiment is illustrated.

[0089] FIG. 4 An example of a point cloud video encoder 10002 of FIG. 1 is illustrated. The point cloud encoder reconstructs and encodes point cloud data (e.g., positions and / or attributes of points) to adjust the quality of the point cloud content (e.g., lossless, lossy, or close to lossless) according to network conditions or applications. When the total size of the point cloud content is large (e.g., 60 Gbps of point cloud content is given for 30 fps), the point cloud content providing system can not be able to stream the content in real time. Thus, the point cloud content providing system can reconstruct the point cloud content based on a maximum target bitrate to provide the point cloud content according to network environments, etc.

[0090] As described with reference to FIG. 1 and FIG. 2 , the point cloud encoder can perform geometry encoding and attribute encoding. The geometry encoding is performed before the attribute encoding.

[0091] The point cloud encoder according to the embodiments includes a coordinate transformer (transform coordinates) 40000, a quantizer (quantize and remove points (voxelize)) 40001, an octree analyzer (analyze octree) 40002 and a surface approximation analyzer (analyze surface approximation) 40003, an arithmetic encoder (arithmetic encode) 40004, a geometry reconstructor (reconstruct geometry) 40005, a color transformer (transform color) 40006, an attribute transformer (transform attribute) 40007, a RAHT transformer (RAHT) 40008, a LOD generator (generate LOD) 40009, a lifting transformer (lift) 40010, a coefficient quantizer (quantize coefficients) 40011, and / or an arithmetic encoder (arithmetic encode) 40012.

[0092] The coordinate transformer 40000, the quantizer 40001, the octree analyzer 40002, the surface approximation analyzer 40003, the arithmetic encoder 40004, and the geometry reconstructor 40005 can perform geometry encoding. The geometry encoding according to the embodiments can include octree geometry encoding, direct encoding, triplet geometry encoding, and entropy encoding. The direct encoding and the triplet geometry encoding are selectively or in combination applied. The geometry encoding is not limited to the above-described examples.

[0093] As illustrated, the coordinate transformer 40000 according to the embodiments receives a position and transforms it into coordinates. For example, the position can be transformed into position information in a three-dimensional space (for example, a three-dimensional space represented by an XYZ coordinate system). The position information in the three-dimensional space according to the embodiments can be referred to as geometry information.

[0094] The quantizer 40001 according to the embodiment performs geometry quantization. For example, the quantizer 40001 can quantize points based on minimum position values (e.g., minimum values on each of X, Y, and Z axes) of all points. The quantizer 40001 performs a quantization operation of multiplying a difference between the minimum position values and position values of respective points by a preset quantization scale value and then finding a nearest integer value by rounding the value obtained through the multiplication. Accordingly, one or more points can have the same quantized position (or position value). The quantizer 40001 according to the embodiment performs voxelization based on the quantized position to reconstruct quantized points. As in the case of a pixel (a minimum unit of 2D image / video information), points of a point cloud content (or 3D point cloud video) according to the embodiment can be included in one or more voxels. As a compound of a volume and a pixel, the term voxel refers to a 3D cubic space generated when a 3D space is divided into units (unit = 1.0) based on axes (e.g., X, Y, and Z axes) representing the 3D space. The quantizer 40001 can match a group of points in the 3D space to voxels. According to the embodiment, one voxel can include only one point. According to the embodiment, one voxel can include one or more points. In order to represent one voxel as one point, a position of a center of the voxel can be set based on positions of one or more points included in the voxel. In this case, attributes of all positions included in one voxel can be combined and assigned to the voxel.

[0095] The octree analyzer 40002 according to the embodiment performs octree geometry encoding (or octree encoding) to present voxels in an octree structure. The octree structure represents points matched to voxels based on an octal tree structure.

[0096] The surface approximation analyzer 40003 according to the embodiment can analyze and approximate an octree. The octree analysis and approximation according to the embodiment is a process of analyzing a region including a plurality of points to efficiently provide an octree and voxelization.

[0097] The arithmetic encoder 40004 according to the embodiment performs entropy encoding on an octree and / or an approximated octree. For example, an encoding scheme includes arithmetic encoding. As a result of encoding, a geometry bitstream is generated.

[0098] The color transformer 40006, the attribute transformer 40007, the RAHT transformer 40008, the LOD generator 40009, the lifting transformer 40010, the coefficient quantizer 40011, and / or the arithmetic encoder 40012 perform attribute encoding. As described above, one point can have one or more attributes. Attribute encoding according to embodiments is equally applied to attributes that one point has. However, when an attribute (e.g., color) includes one or more elements, attribute encoding is independently applied to each element. Attribute encoding according to embodiments includes color transform encoding, attribute transform encoding, region-adaptive hierarchical transform (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) encoding, and interpolation-based hierarchical nearest neighbor prediction with update / lifting step (lifting transform) encoding. According to the content of a point cloud, the above-described RAHT encoding, prediction transform encoding, and lifting transform encoding can be selectively used, or a combination of one or more encoding schemes can be used. Attribute encoding according to embodiments is not limited to the above-described examples.

[0099] The color transformer 40006 according to embodiments performs color transform encoding that transforms a color value (or texture) included in an attribute. For example, the color transformer 40006 can transform the format of color information (e.g., from RGB to YCbCr). Alternatively, the operation of the color transformer 40006 according to embodiments can be applied according to a color value included in an attribute.

[0100] The geometry reconstructor 40005 according to embodiments reconstructs (decompresses) an octree and / or an approximate octree. The geometry reconstructor 40005 reconstructs an octree / voxel based on the result of analyzing the distribution of points. The reconstructed octree / voxel can be referred to as reconstructed geometry (restored geometry).

[0101] The attribute transformer 40007 according to embodiments performs attribute transform to transform an attribute based on reconstructed geometry and / or a position for which geometry encoding is not performed. As described above, since an attribute depends on geometry, the attribute transformer 40007 can transform an attribute based on reconstructed geometry information. For example, based on a position value of a point included in a voxel, the attribute transformer 40007 can transform the attribute of the point at the position. As described above, when a central position of a voxel is set based on positions of one or more points included in the voxel, the attribute transformer 40007 transforms the attributes of the one or more points. When triplet geometry encoding is performed, the attribute transformer 40007 can transform an attribute based on triplet geometry encoding.

[0102] The attribute transformer 40007 can perform attribute transformation by calculating an average of attributes or attribute values (e.g., colors or reflectances of respective points) of neighbor points within a certain position / radius from a center position (or position value) of each voxel. The attribute transformer 40007 can apply a weight according to a distance from the center to each point when calculating the average. Accordingly, each voxel has a position and a calculated attribute (or attribute value).

[0103] The attribute transformer 40007 can search for neighbor points existing within a certain position / radius from a center position of each voxel based on a K-D tree or a Morton code. The K-D tree is a binary search tree and supports a data structure capable of managing points based on positions, so that a nearest neighbor search (NNS) can be quickly performed. The Morton code is generated by presenting coordinates (e.g., (x, y, z)) representing 3D positions of all points as bit values and mixing the bits. For example, when the coordinates representing the positions of points are (5, 9, 1), the bit values of the coordinates are (0101, 1001, 0001). Mixing the bit values according to bit indexes in the order of z, y, and x results in 010001000111. This value is represented as a decimal number 1095. That is, the Morton code value of the point having the coordinates (5, 9, 1) is 1095. The attribute transformer 40007 can sort points based on the Morton code values and perform NNS through a depth-first traversal process. The K-D tree or the Morton code is used when NNS is required in another transformation process for attribute encoding after the attribute transformation operation.

[0104] As illustrated, the transformed attribute is input to the RAHT transformer 40008 and / or the LOD generator 40009.

[0105] The RAHT transformer 40008 according to an embodiment performs RAHT encoding for predicting attribute information based on reconstructed geometry information. For example, the RAHT transformer 40008 can predict attribute information of a node at a higher level in an octree based on attribute information associated with a node at a lower level in the octree.

[0106] The LOD generator 40009 according to an embodiment generates a level of detail (LOD) to perform predictive transform encoding. The LOD according to an embodiment is a degree of detail of point cloud content. As the LOD value decreases, the detail of the point cloud content is indicated to deteriorate. As the LOD value increases, the detail of the point cloud content is indicated to enhance. Points can be classified by LOD.

[0107] The lifting transformer 40010 according to an embodiment performs lifting transform encoding that transforms point cloud attributes based on weights. As described above, the lifting transform encoding can be optionally applied.

[0108] The coefficient quantizer 40011 according to an embodiment quantizes attributes of attribute encoding based on coefficients.

[0109] The arithmetic encoder 40012 according to the embodiment encodes the quantized attribute based on arithmetic encoding.

[0110] Although not shown in the drawings, FIG. 4 The elements of the point cloud encoder can be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing apparatus. The one or more processors can perform at least one of the operations and / or functions of the elements of the point cloud encoder described above. FIG. 4 In addition, the one or more processors can operate or execute a software program and / or a set of instructions for performing the operations and / or functions of the elements of the point cloud encoder described above. The one or more memories according to the embodiment can include a high-speed random access memory, or include a non-volatile memory (e.g., one or more disk storage devices, flash memory devices, or other non-volatile solid state memory devices). FIG. 4

[0111] FIG. 5 An example of a voxel according to the embodiment is illustrated.

[0112] FIG. 5 A voxel located in a 3D space represented by a coordinate system composed of three axes (X-axis, Y-axis, and Z-axis) is illustrated. As described with reference to FIG. 4 The point cloud encoder (e.g., quantizer 40001) can perform voxelization as described with reference to. A voxel refers to a 3D cubic space generated when a 3D space is divided into units (unit = 1.0) based on axes (e.g., X-axis, Y-axis, and Z-axis) representing the 3D space. FIG. 5 An example of a voxel generated by an octree structure in which a cubic axis-aligned bounding box defined by two poles (0, 0, 0) and (2d, 2d, 2d) is recursively subdivided is illustrated. One voxel includes at least one point. The spatial coordinates of the voxel can be estimated from the positional relationship with a group of voxels. As described above, a voxel has attributes (e.g., color or reflectance) similar to pixels of a 2D image / video. Details of the voxel are the same as those described with reference to, and thus a description thereof is omitted. FIG. 4 An example of an octree and an occupancy code according to the embodiment is illustrated.

[0113] FIG. 6 An example of an octree and an occupancy code according to the embodiment is illustrated.

[0114] As described with reference to, the point cloud content providing system (point cloud video encoder 10002) or the point cloud encoder (e.g., octree analyzer 40002) performs octree geometry encoding (or octree encoding) based on an octree structure to efficiently manage the area and / or position of a voxel. FIG. 1-4

[0115] As described with reference to, the point cloud content providing system (point cloud video encoder 10002) or the point cloud encoder (e.g., octree analyzer 40002) performs octree geometry encoding (or octree encoding) based on an octree structure to efficiently manage the area and / or position of a voxel.​FIG. 6 The upper part shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes of a coordinate system (e.g., the X, Y, and Z axes). This is achieved by using two poles (0,0,0) and (2... d ,2 d ,2 d An octree structure is created by recursively subdividing the bounding box with the cubic axis aligned to the bounding box. Here, 2 d This can be set to the value of the minimum bounding box that constitutes all points surrounding the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following formula. In the following formula, (x int n ,y int n ,z int n ) indicates the position (or position value) of the quantized point.

[0116]

[0117] like FIG. 6 As shown in the upper center, the entire 3D space can be divided into eight spaces according to partitions. Each partitioned space is represented by a cube with six faces. For example... FIG. 6 As shown in the upper right, each of the eight spaces is further subdivided based on a coordinate system axis (e.g., the X, Y, and Z axes). Thus, each space is divided into eight smaller spaces. These smaller spaces are also represented by cubes with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0118] FIG. 6 The lower part shows the octree occupancy code. The occupancy code generates the octree to indicate whether each of the eight partitions generated by dividing a space contains at least one node. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of a partitioned space, and each child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to a child node contains at least one node, the node is assigned a value of 1. When the space corresponding to a child node does not contain a node (the space is empty), the node is assigned a value of 0. Since... FIG. 4The shown occupancy code is 00100001, so it indicates that each of the spaces corresponding to the third and eighth child nodes among the eight child nodes contains at least one point. As shown, each of the third and eighth child nodes has eight child nodes, and the child nodes are represented by 8-bit occupancy codes. The accompanying figure shows that the occupancy code of the third child node is 10000111, and the occupancy code of the eighth child node is 01001111. The point cloud encoder (e.g., the arithmetic encoder 40004) according to the embodiments can perform entropy encoding on the occupancy codes. To increase compression efficiency, the point cloud encoder can perform intra / inter-frame encoding on the occupancy codes. The receiving device (e.g., the receiving device 10004 or the point cloud video decoder 10006) according to the embodiments reconstructs the octree based on the occupancy codes.

[0119] The point cloud encoder (e.g., the point cloud encoder or the octree analyzer 40002) according to the embodiments can perform voxelization and octree encoding to store the point positions. However, points are not always uniformly distributed in 3D space, so there can be a certain region in which fewer points exist. Therefore, it is inefficient to perform voxelization on the entire 3D space. For example, when a certain region contains very few points, voxelization does not need to be performed in the certain region. FIG. 7

[0120] Therefore, for the above certain region (or a node other than a leaf node of the octree), the point cloud encoder according to the embodiments can skip voxelization and perform direct encoding to directly encode the point positions included in the certain region. The coordinates of the points directly encoded according to the embodiments are referred to as direct coding mode (DCM). The point cloud encoder according to the embodiments can also perform triplet geometry encoding based on a surface model, which is to reconstruct the point positions in the certain region (or node) based on voxels. Triplet geometry encoding is a geometry encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. The direct encoding and the triplet geometry encoding according to the embodiments can be selectively performed. In addition, the direct encoding and the triplet geometry encoding according to the embodiments can be performed in combination with the octree geometry encoding (or octree encoding).

[0121] To perform direct encoding, an option to use a direct mode to apply direct encoding should be enabled. The node to which direct encoding is to be applied is not a leaf node, and there should be fewer points than a threshold value within the certain node. In addition, the total number of points to which direct encoding is to be applied should not exceed a preset threshold value. When the above conditions are satisfied, the point cloud encoder (or the arithmetic encoder 40004) according to the embodiments can perform entropy encoding on the point positions (or position values).

[0122] ​The point cloud encoder (e.g., the surface approximation analyzer 40003) according to the embodiments can determine a certain level of the octree (a level less than the depth d of the octree), and can start using the surface model from the level to perform the triplet geometry encoding to reconstruct the point positions in the node region based on the voxels (triplet mode). The point cloud encoder according to the embodiments can designate the level to which the triplet geometry encoding is to be applied. For example, when the certain level is equal to the depth of the octree, the point cloud encoder does not operate in the triplet mode. In other words, only when the designated level is less than the depth value of the octree, the point cloud encoder according to the embodiments can operate in the triplet mode. The 3D cubical region of the node designated according to the embodiments is referred to as a block. One block can include one or more voxels. The block or the voxel can correspond to a tile. The geometry is represented as a surface within each block. The surface according to the embodiments can intersect with each edge of the block at most once.

[0123] One block has 12 edges, and thus there are at least 12 intersection points in one block. Each intersection point is referred to as a vertex. When there is at least one occupied voxel adjacent to an edge among all the blocks sharing the edge, the vertex existing along the edge is detected. The occupied voxel according to the embodiments refers to a voxel including a point. The vertex position detected along the edge is an average position of the edges of all the voxels adjacent to the edge among all the blocks sharing the edge.

[0124] Once the vertex is detected, the point cloud encoder according to the embodiments can perform entropy encoding on the origin of the edge (x, y, z), the direction vector of the edge (Δx, Δy, Δz), and the vertex position value (a relative position value within the edge). When the triplet geometry encoding is applied, the point cloud encoder (e.g., the geometry reconstructor 40005) according to the embodiments can generate the restored geometry (reconstructed geometry) by performing a triangle reconstruction, an upsampling, and a voxelization process.

[0125] The vertex located at the edge of the block determines the surface passing through the block. The surface according to the embodiments is a non-planar polygon. In the triangle reconstruction process, the surface represented by a triangle is reconstructed based on the origin of the edge, the direction vector of the edge, and the position value of the vertex. The triangle reconstruction process is performed as follows: 1) a centroid value of each vertex is calculated, 2) the centroid value is subtracted from each vertex value, and 3) a square sum of the values obtained by the subtraction is estimated.

[0126]

[0127] The minimum of the sum is estimated, and a projection process is performed according to the axis having the minimum. For example, when the element x is the smallest, the respective vertices are projected on the x-axis with respect to the center of the block, and on the (y, z) plane. When the value obtained by the projection on the (y, z) plane is (ai, bi), the value of θ is estimated by atan2(bi, ai), and the vertices are ordered based on the value of θ. The following expression shows vertex combinations creating triangles according to the number of vertices. The vertices are ordered from 1 to n. The following expression shows that, for four vertices, two triangles can be constructed according to the vertex combinations. The first triangle can consist of vertices 1, 2, and 3 among the ordered vertices, and the second triangle can consist of vertices 3, 4, and 1 among the ordered vertices.

[0128] Table. Triangles formed from vertices ordered as 1

[0129] [Table 1]

[0130]

[0131]

[0132] An upsampling process is performed to add points in the middle along the edges of the triangle, and voxelization is performed. The added points are generated based on an upsampling factor and the width of the block. The added points are referred to as refined vertices. The point cloud encoder according to the embodiment can voxelize the refined vertices. In addition, the point cloud encoder can perform attribute encoding based on the positions (or position values) of the voxelization.

[0133] FIG. 1-6 An example of a neighbor node pattern according to the embodiment is shown.

[0134] To increase the compression efficiency of the point cloud video, the point cloud encoder according to the embodiment can perform entropy encoding based on context adaptive arithmetic encoding.

[0135] As described with reference to FIG. 4 , the point cloud content providing system or the point cloud encoder (e.g., the point cloud video encoder 10002, FIG. 7 , or the arithmetic encoder 40004 of the point cloud encoder) can immediately perform entropy encoding on the occupancy code. In addition, the point cloud content providing system or the point cloud encoder can perform entropy encoding (intra encoding) based on the occupancy code of the current node and the occupancy of the neighboring nodes, or perform entropy encoding (inter encoding) based on the occupancy code of the previous frame. The frame according to the embodiment represents a set of simultaneously generated point cloud videos. The compression efficiency of the intra / inter encoding according to the embodiment can depend on the number of neighboring nodes referred to. When the bits increase, the operation becomes complex, but the encoding can be biased to one side, which can increase the compression efficiency. For example, when 3 bits of context are given, it is necessary to use 2 3= 8 methods to perform encoding. The partitioning for encoding affects the implementation complexity. Therefore, it is necessary to meet an appropriate level of compression efficiency and complexity.

[0136] FIG. 7 The process of obtaining an occupancy pattern based on the occupancy of neighbor nodes is shown. The point cloud encoder according to the embodiments determines the occupancy of neighbor nodes of each node of an octree and obtains a value of a neighbor pattern. The neighbor node pattern is used to infer the occupancy pattern of a node. FIG. 7 The left part of FIG. 8 shows a cube corresponding to a node (the cube in the middle) and six cubes (neighbor nodes) that share at least one face with the cube. The node shown in the figure is a node of the same depth. The numbers shown in the figure respectively represent the weights (1, 2, 4, 8, 16, and 32) associated with the six nodes. The weights are assigned in turn according to the positions of the neighboring nodes.

[0137] FIG. 8 The right part of FIG. 8 shows the neighbor node pattern value. The neighbor node pattern value is the sum of the values multiplied by the weights of the occupied neighbor nodes (the neighbor nodes with points). Therefore, the neighbor node pattern value is 0 to 63. When the neighbor node pattern value is 0, it indicates that there is no node with a point among the neighbor nodes of the node (unoccupied node). When the neighbor node pattern value is 63, it indicates that all the neighbor nodes are occupied nodes. As shown in the figure, since the neighbor nodes assigned with the weights 1, 2, 4, and 8 are occupied nodes, the neighbor node pattern value is 15 (the sum of 1, 2, 4, and 8). The point cloud encoder can perform encoding according to the neighbor node pattern value (for example, when the neighbor node pattern value is 63, 64 types of encoding can be performed). According to the embodiments, the point cloud encoder can reduce the encoding complexity by changing the neighbor node pattern value (for example, based on a table that changes 64 to 10 or 6).

[0138] FIG. 1-7 An example of the point configuration in each LOD according to the embodiments is shown.

[0139] As described with reference to FIG. 9 the encoded geometry is reconstructed (decompressed) before performing attribute encoding. When direct encoding is applied, the geometry reconstruction operation can include changing the placement of the directly encoded points (for example, placing the directly encoded points in front of the point cloud data). When triplet geometry encoding is applied, the geometry reconstruction process is performed by triangle reconstruction, upsampling, and voxelization. Since the attributes depend on the geometry, the attribute encoding is performed based on the reconstructed geometry.

[0140] A point cloud encoder (e.g., the LOD generator 40009) can classify (reorganize) points by LOD. The figure shows point cloud contents corresponding to LODs. The leftmost picture in the figure represents original point cloud contents. The second picture from the left in the figure represents point distribution in the lowest LOD, and the rightmost picture in the figure represents point distribution in the highest LOD. That is, points are sparsely distributed in the lowest LOD, and points are densely distributed in the highest LOD. That is, as LODs increase in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between points becomes narrower.

[0141] FIG. 1-8 An example of point configuration for each LOD according to an embodiment is shown.

[0142] As described with reference to FIG. 4 , a point cloud content providing system or a point cloud encoder (e.g., the point cloud video encoder 10002, FIG. 9 or the point cloud encoder or the LOD generator 40009 of the point cloud video encoder) can generate LODs. LODs are generated by reorganizing points into a set of refinement levels according to a set of LOD distance values (or a set of Euclidean distances). The LOD generation process is not only performed by a point cloud encoder but also by a point cloud decoder.

[0143] FIG. 9 The upper part of the figure shows examples of points (P0 to P9) of point cloud contents distributed in 3D space. In FIG. 9 , the original order represents the order of points P0 to P9 before LOD generation. In FIG. 9 , the LOD-based order represents the order of points generated according to LODs. Points are reorganized by LODs. In addition, a high LOD contains points belonging to a lower LOD. As FIG. 4 indicated, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, the points of LOD1, P9, P8, and P7.

[0144] As described with reference to FIG. 10 , a point cloud encoder according to an embodiment can selectively or in combination perform prediction transform encoding, lifting transform encoding, and RAHT transform encoding.

[0145] A point cloud encoder according to an embodiment can generate predictors for points to perform prediction transform encoding for setting prediction properties (or prediction property values) of respective points. That is, N predictors can be generated for N points. Predictors according to an embodiment can calculate weights (= 1 / distance) based on LOD values of respective points, index information about neighbor points existing within a set distance of respective LODs, and distances to the neighbor points.

[0146] The predicted attribute (or attribute value) according to the embodiment is set to an average of values obtained by multiplying attributes (or attribute values) (e.g., color, reflectance, etc.) of neighbor points set in the predictor of each point by weights (or weight values) calculated based on distances to each neighbor point. The point cloud encoder (e.g., the coefficient quantizer 40011) according to the embodiment can quantize and inverse quantize a residual (which can be referred to as a residual attribute, a residual attribute value, or an attribute prediction residual) obtained by subtracting the predicted attribute (attribute value) from the attribute (attribute value) of each point. The quantization process is configured as shown in the following table.

[0147] Attribute prediction residual quantization pseudo code

[0148] [Table 2]

[0149]

[0150] Attribute prediction residual inverse quantization pseudo code

[0151] [Table 3]

[0152]

[0153] When the predictor of each point has neighbor points, the point cloud encoder (e.g., the arithmetic encoder 40012) according to the embodiment can perform entropy encoding on the quantized and inverse quantized residual values as described above. When the predictor of each point does not have neighbor points, the point cloud encoder (e.g., the arithmetic encoder 40012) according to the embodiment can perform entropy encoding on the attribute of the corresponding point without performing the above-described operation.

[0154] The point cloud encoder (e.g., the lifting transformer 40010) according to the embodiment can generate a predictor of each point, set a calculated LOD and register neighbor points in the predictor, and set weights according to distances to the neighbor points to perform lifting transform encoding. The lifting transform encoding according to the embodiment is similar to the prediction transform encoding described above, but differs in that weights are applied to attribute values cumulatively. The process of applying weights to attribute values cumulatively according to the embodiment is configured as follows.

[0155] 1) Create an array quantized weight (QW) for storing weight values of each point. The initial value of all elements of QW is 1.0. Multiply the QW value of the predictor index of the neighbor node registered in the predictor by the weight of the predictor of the current point, and add the value obtained by the multiplication.

[0156] 2) Lifting prediction process: subtract a value obtained by multiplying the attribute value of a point by a weight from the existing attribute value to calculate a predicted attribute value.

[0157] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0158] 4) The weight calculated by multiplying the weight calculated for all predictors by the weight stored in QW corresponding to the predictor index is accumulated with the updateweight array as the index of the neighbor node. The value obtained by multiplying the attribute value of the neighbor node index by the calculated weight is accumulated with the update array.

[0159] 5) Promotion update process: The attribute value of the update array of all predictors is divided by the weight value of the updateweight array of the predictor index, and the existing attribute value is added to the value obtained by the division.

[0160] 6) The predicted attribute is calculated for all predictors by multiplying the attribute value updated by the promotion update process by the weight (stored in QW) updated by the promotion prediction process. The point cloud encoder (e.g., the coefficient quantizer 40011) according to the embodiment quantizes the predicted attribute value. In addition, the point cloud encoder (e.g., the arithmetic encoder 40012) performs entropy encoding on the quantized attribute value.

[0161] The point cloud encoder (e.g., the RAHT transformer 40008) according to the embodiment can perform RAHT transform encoding in which the attribute of a node of a higher level is predicted using the attribute associated with a node of a lower level in the octree. The RAHT transform encoding is an example of attribute intra-frame encoding by octree backward scanning. The point cloud encoder according to the embodiment scans the entire region from a voxel and repeats a merging process of merging voxels into larger blocks at each step until the root node is reached. The merging process according to the embodiment is performed only on an occupied node. The merging process is not performed on an empty node. The merging process is performed on the upper node directly above the empty node.

[0162] The following equation represents the RAHT transform matrix. In the equation, represents the average attribute value of the voxel of level l. can be calculated based on and and The weight of and

[0163]

[0164]

[0165] Here, is a low-pass value and is used in the merging process of the next highest level. ​This represents the high-pass coefficient. The high-pass coefficient at each step is quantized and subjected to entropy encoding (e.g., encoded by an arithmetic encoder 400012). Weights are calculated as follows: pass and Create the root node as follows.

[0166]

[0167] Similar to high-pass coefficients, the value of gDC is also quantized and subjected to entropy encoding.

[0168] FIG. 10 A point cloud decoder according to an embodiment is shown.

[0169] FIG. 1 The point cloud decoder shown is FIG. 1 The example of the point cloud video decoder 10006 described in [the document], and it can be executed with [the following]. FIG. 11 The operation of the point cloud video decoder 10006 shown is the same or similar. As shown, the point cloud decoder can receive a geometry bitstream and an attribute bitstream contained in one or more bitstreams. The point cloud decoder includes a geometry decoder and an attribute decoder. The geometry decoder performs geometry decoding on the geometry bitstream and outputs the decoded geometry. The attribute decoder performs attribute decoding based on the decoded geometry and the attribute bitstream and outputs the decoded attributes. The decoded geometry and decoded attributes are used to reconstruct the point cloud content (the decoded point cloud).

[0170] FIG. 11 A point cloud decoder according to an embodiment is shown.

[0171] FIG. 10 The point cloud decoder shown is FIG. 1-9 The example shown is a point cloud decoder that can perform decoding operations. FIG. 1 The reverse process of the encoding operation of the point cloud encoder is shown.

[0172] For reference FIG. 10 and FIG. 1-9 As described, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0173] The point cloud decoder according to the embodiments includes an arithmetic decoder (arithmetic decoding) 11000, an octree synthesizer (synthesize octree) 11001, a surface approximation synthesizer (synthesize surface approximation) 11002 and a geometry reconstructor (reconstruct geometry) 11003, a coordinate inverse transformer (inverse transform coordinate) 11004, an arithmetic decoder (arithmetic decoding) 11005, an inverse quantizer (inverse quantization) 11006, an RAHT transformer 11007, an LOD generator (generate LOD) 11008, an inverse elevator (inverse elevation) 11009, and / or a color inverse transformer (inverse transform color) 11010.

[0174] The arithmetic decoder 11000, the octree synthesizer 11001, the surface approximation synthesizer 11002, the geometry reconstructor 11003, and the coordinate inverse transformer 11004 can perform geometry decoding. The geometry decoding according to the embodiments can include direct encoding and triplet geometry decoding. The direct encoding and the triplet geometry decoding are selectively applied. The geometry decoding is not limited to the above-described examples, and is described in detail with reference to FIG. 1-9 The inverse process of the geometry encoding described is performed.

[0175] The arithmetic decoder 11000 according to the embodiments decodes the received geometry bitstream based on arithmetic encoding. The operation of the arithmetic decoder 11000 corresponds to the inverse process of the arithmetic encoder 40004.

[0176] The octree synthesizer 11001 according to the embodiments can generate an octree by acquiring an occupancy code (or information about the geometry obtained as a result of decoding) from the decoded geometry bitstream. The occupancy code is described in detail with reference to FIG. 1-9 as described in detail.

[0177] When the triplet geometry encoding is applied, the surface approximation synthesizer 11002 according to the embodiments can synthesize a surface based on the decoded geometry and / or the generated octree.

[0178] The geometry reconstructor 11003 according to the embodiments can regenerate a geometry based on the surface and / or the decoded geometry. As described with reference to FIG. 6 The direct encoding and the triplet geometry encoding are selectively applied as described. Accordingly, the geometry reconstructor 11003 directly imports position information about the point to which the direct encoding is applied and adds the same. When the triplet geometry encoding is applied, the geometry reconstructor 11003 can reconstruct the geometry by performing the reconstruction operation (e.g., triangle reconstruction, upsampling, and voxelization) of the geometry reconstructor 40005. Details are the same as those described with reference to FIG. 10 are described, and thus a description thereof is omitted. The reconstructed geometry can include a point cloud picture or frame that does not include attributes.

[0179] The coordinate inverse transformer 11004 according to the embodiment can acquire point positions based on the reconstructed geometry-transformed coordinates.

[0180] The arithmetic decoder 11005, the inverse quantizer 11006, the RAHT transformer 11007, the LOD generator 11008, the inverse upscaler 11009, and / or the color inverse transformer 11010 can perform attribute decoding described above with reference to FIGS. 1 to 3. FIG. 11 The attribute decoding described above. The attribute decoding according to the embodiment includes a region-adaptive hierarchical transform (RAHT) decoding, an interpolation-based hierarchical nearest neighbor prediction (prediction transform) decoding, and an interpolation-based hierarchical nearest neighbor prediction with an update / upscale step (upscale transform) decoding. The above three decoding schemes can be selectively used, or a combination of one or more decoding schemes can be used. The attribute decoding according to the embodiment is not limited to the above-described examples.

[0181] The arithmetic decoder 11005 according to the embodiment decodes an attribute bitstream by arithmetic coding.

[0182] The inverse quantizer 11006 according to the embodiment inverse quantizes information on a decoded attribute bitstream or an attribute acquired as a result of decoding, and outputs inverse quantized attributes (or attribute values). The inverse quantization can be selectively applied based on attribute encoding of the point cloud encoder.

[0183] According to the embodiment, the RAHT transformer 11007, the LOD generator 11008, and / or the inverse upscaler 11009 can process reconstructed geometry and inverse quantized attributes. As described above, the RAHT transformer 11007, the LOD generator 11008, and / or the inverse upscaler 11009 can selectively perform decoding operations corresponding to encoding of the point cloud encoder.

[0184] The color inverse transformer 11010 according to the embodiment performs inverse transform coding to inverse transform color values (or textures) included in a decoded attribute. Operations of the color inverse transformer 11010 can be selectively performed based on operations of the color transformer 40006 of the point cloud encoder.

[0185] Although not shown in the drawings, FIG. 11 Elements of the point cloud decoder of FIG. 11 can be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing apparatus. The one or more processors can perform at least one or more of operations and / or functions of elements of the point cloud decoder of FIG. 11 described above. FIG. 11 In addition, the one or more processors can operate or execute a software program and / or a set of instructions for performing operations and / or functions of elements of the point cloud decoder of FIG. 11 described above. FIG. 12

[0186] ​FIG. 12 An exemplary transmitting apparatus according to an embodiment is shown.

[0187] FIG. 1 The transmitting device shown is FIG. 4 The transmitting device 10000 (or FIG. 12 Example of a point cloud encoder. FIG. 1-9 The transmitting device shown can perform the same operation as the reference. FIG. 2 The described point cloud encoder includes one or more of the same or similar operations and methods. The transmitting apparatus according to the embodiment may include a data input unit 12000, a quantization processor 12001, a voxelization processor 12002, an octree occupancy code generator 12003, a surface model processor 12004, an intra / inter-frame coding processor 12005, an arithmetic encoder 12006, a metadata processor 12007, a color transformation processor 12008, an attribute transformation processor 12009, a prediction / boosting / RAHT transformation processor 12010, an arithmetic encoder 12011, and / or a transmission processor 12012.

[0188] According to the embodiment, the data input unit 12000 receives or acquires point cloud data. The data input unit 12000 can perform operations and / or acquisition methods similar to those of the point cloud video acquirer 10001 (or refer to...). FIG. 1-9 The described acquisition process (20000) is the same as or similar to the operation and / or acquisition method.

[0189] The data input unit 12000, quantization processor 12001, voxelization processor 12002, octree occupancy code generator 12003, surface model processor 12004, intra / inter-frame coding processor 12005, and arithmetic encoder 12006 perform geometric coding. Geometric coding according to the implementation method and reference... FIG. 4 The geometric codes described are the same or similar, so their detailed descriptions are omitted.

[0190] The quantization processor 12001 according to the embodiment quantizes geometry (e.g., point position values). The operation of the quantization processor 12001 and / or quantization with reference... FIG. 1-9 The operation and / or quantization of the described quantizer 40001 are the same or similar. Details and references FIG. 4 The descriptions are the same.

[0191] According to the embodiment, the voxelization processor 12002 voxels the quantized position values ​​of points. The voxelization processor 120002 can execute and reference... FIG. 1-9 The operation and / or voxelization process of the quantizer 40001 described are the same as or similar to the operation and / or process. Details and references FIG. 4 The descriptions are the same.

[0192] According to the implementation method, the octree occupancy code generator 12003 performs octree encoding based on the voxelized positions of points in the octree structure. The octree occupancy code generator 12003 can generate occupancy codes. The octree occupancy code generator 12003 can execute and reference... FIG. 6 and FIG. 1-9 The operations and / or methods described are the same as or similar to those of the point cloud encoder (or octree analyzer 40002). Details and references FIG. 4 The descriptions are the same.

[0193] According to the implementation, the surface model processor 12004 can perform triadic geometry encoding based on a surface model to reconstruct point positions in a specific region (or node) based on voxels. The surface model processor 12004 can perform operations related to reference... FIG. 1-9 The operations and / or methods described are the same as or similar to those of the point cloud encoder (e.g., surface approximation analyzer 40003). Details and references are available. FIG. 7 The descriptions are the same.

[0194] The intra / inter-frame coding processor 12005 according to the embodiment can perform intra / inter-frame coding on point cloud data. The intra / inter-frame coding processor 12005 can perform operations similar to those described above. FIG. 7 The described intra / inter-frame coding is the same or similar. Details and references FIG. 1-9 The descriptions are the same. According to an implementation, the intra / inter-frame coding processor 12005 may be included in the arithmetic encoder 12006.

[0195] The arithmetic encoder 12006 according to the embodiment performs entropy encoding on octrees and / or approximate octrees of point cloud data. For example, the encoding scheme includes arithmetic encoding. The arithmetic encoder 12006 performs the same or similar operations and / or methods as the arithmetic encoder 40004.

[0196] The metadata processor 12007 according to an embodiment processes metadata (e.g., set values) about point cloud data and provides it to necessary processing procedures such as geometric encoding and / or attribute encoding. Additionally, the metadata processor 12007 according to an embodiment can generate and / or process signaling information related to geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be encoded separately from the geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be interleaved.

[0197] Color transformation processor 12008, attribute transformation processor 12009, prediction / boosting / RAHT transformation processor 12010, and arithmetic encoder 12011 perform attribute encoding. Attribute encoding according to the implementation method and reference... FIG. 1-9The described attribute encodes the same or similar, and thus a detailed description thereof is omitted.

[0198] The color transform processor 12008 according to the embodiment performs color transform encoding to transform color values included in the attribute. The color transform processor 12008 can perform the color transform encoding based on the reconstructed geometry. The reconstructed geometry is the same as described with reference to FIG. 4 The described. In addition, it performs the same or similar operations and / or methods as the color transformer 40006 described with reference to FIG. 4 The described operations and / or methods of the color transformer 40006 are the same or similar, and thus a detailed description thereof is omitted.

[0199] The attribute transform processor 12009 according to the embodiment performs attribute transform to transform the attribute based on the reconstructed geometry and / or positions for which geometry encoding is not performed. The attribute transform processor 12009 performs the same or similar operations and / or methods as the attribute transformer 40007 described with reference to FIG. 4 The described operations and / or methods of the attribute transformer 40007 are the same or similar, and thus a detailed description thereof is omitted. The prediction / lifting / RAHT transform processor 12010 according to the embodiment can encode the transformed attribute by any one or a combination of RAHT encoding, prediction transform encoding, and lifting transform encoding. The prediction / lifting / RAHT transform processor 12010 performs the same or similar operations and / or methods as the RAHT transformer 40008, the LOD generator 40009, and the lifting transformer 40010 described with reference to FIG. 1-9 The described operations of the RAHT transformer 40008, the LOD generator 40009, and the lifting transformer 40010 are the same or similar, and thus a detailed description thereof is omitted. In addition, the prediction transform encoding, the lifting transform encoding, and the RAHT transform encoding are the same as described with reference to FIG. 1 The described are the same, and thus a detailed description thereof is omitted.

[0200] The arithmetic encoder 12011 according to the embodiment can encode the encoded attribute based on arithmetic encoding. The arithmetic encoder 12011 performs the same or similar operations and / or methods as the operations and / or methods of the arithmetic encoder 400012.

[0201] The transmission processor 12012 according to the embodiment can transmit individual bitstreams containing encoded geometry and / or encoded attributes and metadata information, or transmit one bitstream configured with encoded geometry and / or encoded attributes and metadata information. When the encoded geometry and / or encoded attributes and metadata information according to the embodiment are configured as one bitstream, the bitstream can include one or more sub-bitstreams. The bitstream according to the embodiment can contain signaling information including a sequence parameter set (SPS) for sequence level signaling, a geometry parameter set (GPS) for geometry information encoding signaling, an attribute parameter set (APS) for attribute information encoding signaling, and a tile parameter set (TPS) for tile level signaling, and slice data. The slice data can include information about one or more slices. One slice according to the embodiment can include one geometry bitstream Geom0 0 and one or more attribute bitstreams Attr0 0 and Attr1 0 The TPS according to the embodiment can include information about individual tiles in one or more tiles (e.g., coordinate information about a bounding box and height / size information). The geometry bitstream can contain a header and a payload. The header of the geometry bitstream according to the embodiment can contain a parameter set identifier (geom_parameter_set_id) included in the GPS, a tile identifier (geom_tile_id), and a slice identifier (geom_slice_id), and information about data contained in the payload. As described above, the metadata processor 12007 according to the embodiment can generate and / or process signaling information and transmit it to the transmission processor 12012. According to the embodiment, an element that performs geometry encoding and an element that performs attribute encoding can share data / information with each other as indicated by dotted lines. The transmission processor 12012 according to the embodiment can perform operations and / or transmission methods the same as or similar to those of the transmitter 10003. Details are the same as those described with reference to FIG. 2 and FIG. 13 and thus a description thereof is omitted.

[0202] FIG. 13 An exemplary reception apparatus according to the embodiment is illustrated.

[0203] FIG. 1 The illustrated reception apparatus is an example of the reception apparatus 10004 (or FIG. 10 a point cloud decoder) of FIG. 11 and FIG. 13 FIG. 1-11 The illustrated reception apparatus can perform one or more operations and methods the same as or similar to those of the point cloud decoder described with reference to FIG. 1

[0204] ​​The receiving apparatus according to the embodiment includes a receiver 13000, a reception processor 13001, an arithmetic decoder 13002, an occupancy code-based octree reconstruction processor 13003, a surface model processor (triangle reconstruction, upsampling, voxelization) 13004, an inverse quantization processor 13005, a metadata parser 13006, an arithmetic decoder 13007, an inverse quantization processor 13008, a prediction / lifting / RAHT inverse transform processor 13009, a color inverse transform processor 13010, and / or a Tenderer 13011. Each decoding element according to the embodiment can perform an inverse process of the operation of the corresponding encoding element according to the embodiment.

[0205] The receiver 13000 according to the embodiment receives point cloud data. The receiver 13000 can perform the same or similar operation and / or reception method as those of the receiver 10005 of the reception method 1000. The detailed description thereof is omitted. FIG. 1-10

[0206] The reception processor 13001 according to the embodiment can acquire a geometry bitstream and / or an attribute bitstream from the received data. The reception processor 13001 can be included in the receiver 13000.

[0207] The arithmetic decoder 13002, the occupancy code-based octree reconstruction processor 13003, the surface model processor 13004, and the inverse quantization processor 13005 can perform geometry decoding. The geometry decoding according to the embodiment is the same as or similar to the geometry decoding described with reference to the geometry decoding 1100, and thus the detailed description thereof is omitted. FIG. 12

[0208] The arithmetic decoder 13002 according to the embodiment can decode a geometry bitstream based on arithmetic encoding. The arithmetic decoder 13002 performs the same or similar operation and / or encoding as those of the arithmetic decoder 11000.

[0209] The occupancy code-based octree reconstruction processor 13003 according to the embodiment can reconstruct an octree by acquiring an occupancy code from the decoded geometry bitstream (or information about the geometry acquired as a decoding result). The occupancy code-based octree reconstruction processor 13003 performs the same or similar operation and / or method as those of the synthesizer 11001 of the octree and / or the octree generation method. When the triplet geometry encoding is applied, the surface model processor 13004 according to the embodiment can perform triplet geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on a surface model method. The surface model processor 13004 performs the same or similar operation as that of the surface approximation synthesizer 11002 and / or the geometry reconstructor 11003.

[0210] ​​The inverse quantization processor 13005 according to the embodiments can inverse quantize decoded geometry.

[0211] The metadata parser 13006 according to the embodiments can parse metadata (e.g., setting values) included in the received point cloud data. The metadata parser 13006 can pass the metadata to the geometry decoding and / or attribute decoding. The metadata is the same as described in the metadata, and thus a detailed description thereof is omitted. FIG. 1-10 The attribute decoding described is the same as or similar to that described in the attribute decoding, and thus a detailed description thereof is omitted.

[0212] The arithmetic decoder 13007, the inverse quantization processor 13008, the prediction / lifting / RAHT inverse transformer 13009, and the color inverse transformer 13010 perform attribute decoding. The attribute decoding is the same as or similar to that described in the attribute decoding, and thus a detailed description thereof is omitted. FIG. 14 The attribute decoding described is the same as or similar to that described in the attribute decoding, and thus a detailed description thereof is omitted.

[0213] The arithmetic decoder 13007 according to the embodiments can decode an attribute bitstream by arithmetic encoding. The arithmetic decoder 13007 can decode the attribute bitstream based on reconstructed geometry. The arithmetic decoder 13007 performs the same or similar operations and / or encoding as those of the arithmetic decoder 11005.

[0214] The inverse quantization processor 13008 according to the embodiments can inverse quantize a decoded attribute bitstream. The inverse quantization processor 13008 performs the same or similar operations and / or method as those of the inverse quantizer 11006.

[0215] The prediction / lifting / RAHT inverse transformer 13009 according to the embodiments can process reconstructed geometry and inverse quantized attributes. The prediction / lifting / RAHT inverse transformer 13009 performs the same or similar operation and / or decoding as those of the RAHT transformer 11007, the LOD generator 11008, and / or the inverse lifter 11009. The color inverse transformer 13010 according to the embodiments performs inverse transform encoding to inverse transform color values (or textures) included in decoded attributes. The color inverse transformer 13010 performs the same or similar operations and / or inverse transform encoding as those of the color inverse transformer 11010. The Tenderer 13011 according to the embodiments can render point cloud data.

[0216] FIG. 14 An architecture for streaming G-PCC based point cloud data according to the embodiments is shown.

[0217] FIG. 1-13 The upper part of FIG. 1 shows a transmitting apparatus (e.g., the transmitting apparatus 10000, FIG. 12 described in the transmitting apparatus (e.g., the transmitting apparatus 10000, FIG. 1-13the process of processing and transmitting the point cloud content by the transmitting apparatus, etc.

[0218] As described with reference to FIG. 1-13 , the transmitting apparatus can acquire the audio Ba of the point cloud content (audio acquisition), encode the acquired audio (audio encoding), and output the audio bitstream Ea. In addition, the transmitting apparatus can acquire the point cloud (or point cloud video) Bv of the point cloud content (point acquisition), and perform point cloud encoding on the acquired point cloud to output the point cloud video bitstream Eb. The point cloud encoding of the transmitting apparatus is the same as or similar to the point cloud encoding (e.g., encoding of the point cloud encoder) described with reference to FIG. 4 , and thus a detailed description thereof will be omitted. FIG. 1-13

[0219] The transmitting apparatus can encapsulate the generated audio bitstream and video bitstream into a file and / or a segment (file / segment encapsulation). The encapsulated file and / or segment Fs,File can include a file format such as ISOBMFF or DASH segment. The point cloud related metadata according to the embodiments can be contained in the encapsulated file format and / or segment. The metadata can be contained in various levels of boxes on the ISOBMFF file format, or can be contained in a separate track within the file. According to the embodiments, the transmitting apparatus encapsulates the metadata into a separate file. The transmitting apparatus according to the embodiments can transmit the encapsulated file format and / or segment via a network. The encapsulation and transmission processing method of the transmitting apparatus is the same as described with reference to FIG. 2 , (e.g., the transmission step 20002 of the transmitter 10003, FIG. 14 , and thus a detailed description thereof will be omitted.

[0220] FIG. 1-13 The lower part of FIG. 1 shows the process of processing and outputting the point cloud content by the receiving apparatus (e.g., the receiving apparatus 10004, FIG. 13 , and thus a detailed description thereof will be omitted. FIG. 1-13

[0221] According to the embodiments, the receiving apparatus can include an apparatus (e.g., a speaker, a headphone, a display) configured to output final audio data and final video data, and a point cloud player (point cloud player) configured to process the point cloud content. The final data output apparatus and the point cloud player can be configured as separate physical apparatuses. The point cloud player according to the embodiments can perform geometry-based point cloud compression (G-PCC) encoding, video-based point cloud compression (V-PCC) encoding, and / or next generation encoding.

[0222] ​​The reception apparatus according to the embodiments can acquire and de-encapsulate (file / segment de-encapsulation) files and / or segments F’, Fs’ contained in the received data (e.g., broadcast signal, signal transmitted via a network, etc.). The reception and de-encapsulation methods of the reception apparatus are the same as those described with reference to FIG. 1-13 The details thereof will be omitted.

[0223] The reception apparatus according to the embodiments acquires an audio bitstream E’a and a video bitstream E’v contained in the files and / or segments. As illustrated, the reception apparatus outputs decoded audio data B’a by performing audio decoding on the audio bitstream, and renders the decoded audio data (audio rendering) to output final audio data A’a through speakers or earphones.

[0224] In addition, the reception apparatus performs point cloud decoding on the video bitstream E’v and outputs decoded video data B’v. The point cloud decoding according to the embodiments is the same as or similar to the point cloud decoding described with reference to FIG. 11 the decoder of the point cloud decoder described with reference to FIG. 12-13 The details thereof will be omitted. The reception apparatus can render the decoded video data and output final video data through a display.

[0225] The reception apparatus according to the embodiments can perform at least one of de-encapsulation, audio decoding, audio rendering, point cloud decoding, and point cloud video rendering based on the transmitted metadata. The details of the metadata are the same as those described with reference to FIG. 1-13 The details thereof will be omitted.

[0226] As indicated by dotted lines illustrated in the drawing, the reception apparatus (e.g., point cloud player or sensing / tracking unit in the point cloud player) according to the embodiments can generate feedback information (orientation, viewport). According to the embodiments, the feedback information can be used in the de-encapsulation process, the point cloud decoding process, and / or the rendering process of the reception apparatus, or can be transmitted to the transmission apparatus. The details of the feedback information are the same as those described with reference to FIG. 15 The details thereof will be omitted.

[0227] FIG. 15 An exemplary transmission apparatus according to the embodiments is illustrated.

[0228] FIG. 1-14 The transmission apparatus according to the embodiments is an apparatus configured to transmit point cloud content, and corresponds to the transmission apparatus described with reference to FIG. 1 the transmission apparatus 10000 described with reference to FIG. 4 the transmission apparatus 10000 described with reference to FIG. 12 the point cloud encoder described with reference to FIG. 14 the transmission apparatus described with reference to FIG. 15of the transmitting apparatus) is shown. Thus, FIG. 1-14 The transmitting apparatus performs the same or similar operations as those of the transmitting apparatus described with reference to FIG. 1-14 The transmitting apparatus performs the same or similar operations as those of the transmitting apparatus described with reference to

[0229] The transmitting apparatus according to the embodiments can perform one or more of point cloud acquisition, point cloud encoding, file / segment packaging, and delivery.

[0230] Since the operations of point cloud acquisition and delivery shown in the drawing are the same as those described with reference to FIG. 1-14 a detailed description thereof will be omitted.

[0231] As described above with reference to FIG. 1-14 the transmitting apparatus according to the embodiments can perform geometry encoding and attribute encoding. The geometry encoding can be referred to as geometry compression, and the attribute encoding can be referred to as attribute compression. As described above, one point can have one geometry and one or more attributes. Thus, the transmitting apparatus performs attribute encoding on each attribute. The transmitting apparatus performs one or more attribute compressions (attribute #1 compression,..., attribute #N compression) is shown in the drawing. In addition, the transmitting apparatus according to the embodiments can perform auxiliary compression. The auxiliary compression is performed on metadata. Details of the metadata are the same as those described with reference to FIG. 1-14 a detailed description thereof will be omitted. The transmitting apparatus can also perform mesh data compression. The mesh data compression according to the embodiments can include triplet geometry encoding described with reference to FIG. 1-14 .

[0232] The transmitting apparatus according to the embodiments can package a bitstream (e.g., point cloud stream) output according to point cloud encoding into a file and / or a segment. According to the embodiments, the transmitting apparatus can perform media track packaging for carrying data other than metadata (e.g., media data), and perform metadata track packaging for carrying metadata. According to the embodiments, the metadata can be packaged into a media track.

[0233] As described with reference to FIG. 1-14 the transmitting apparatus can receive feedback information (orientation / viewport metadata) from the receiving apparatus, and perform at least one of point cloud encoding, file / segment packaging, and delivery operations based on the received feedback information. Details are the same as those described with reference to FIG. 16 a detailed description thereof will be omitted.

[0234] FIG. 16 An exemplary receiving apparatus according to the embodiments is shown.

[0235] FIG. 1-14 The receiving apparatus is an apparatus for receiving point cloud content, and corresponds to an example of the receiving apparatus described with reference to FIG. 1 , for example, the receiving apparatus 10004 of FIG. 11 a detailed description thereof will be omitted.FIG. 13 Point cloud decoder and FIG. 14 The receiving device FIG. 16 (The receiving device). Therefore, FIG. 1-14 The receiving device performs and references FIG. 16 The described receiving device operates in the same or similar manner. FIG. 15 The receiving device can receive from FIG. 15 The signal sent by the transmitting device, and the execution FIG. 1-14 The reverse process of operating the transmitting device.

[0236] The receiving device according to the embodiment can perform at least one of transmission, file / fragment decapsulation, point cloud decoding, and point cloud rendering.

[0237] Because the point cloud receiving and point cloud rendering operations shown in the figure are different from the reference... FIG. 1-14 The descriptions are the same, so their detailed descriptions will be omitted.

[0238] For reference FIG. 1-14 The receiving device, according to an embodiment, decapsulates files and / or fragments acquired from a network or storage device. According to an embodiment, the receiving device may perform media track decapsulation to carry data other than metadata (e.g., media data), and perform metadata track decapsulation to carry metadata. According to an embodiment, metadata track decapsulation is omitted when metadata is encapsulated into a media track.

[0239] For reference FIG. 1-14 The receiving device described herein can perform geometric decoding and attribute decoding on a bitstream (e.g., a point cloud stream) obtained through decapsulation. Geometric decoding may be referred to as geometric decompression, and attribute decoding may be referred to as attribute decompression. As mentioned above, a point may have one geometry and one or more attributes, each encoded by the transmitting device. Therefore, the receiving device performs attribute decoding on each attribute. The figure shows the receiving device performing one or more attribute decompressions (attribute #1 decompression, ..., attribute #N decompression). The receiving device according to the embodiment may also perform auxiliary decompression. Auxiliary decompression is performed on metadata. Details and references to metadata are provided. FIG. 1-14 The descriptions are identical, therefore their confusion will be omitted. The receiving device can also perform grid data decompression. Grid data decompression according to the embodiment may include reference to... FIG. 1-14 The described triplet geometry decoding. The receiving device according to the embodiment can render point cloud data output from the point cloud decoding.

[0240] For reference FIG. 15 As described, the receiving device can use a separate sensing / tracking element to acquire orientation / viewport metadata and send feedback information including it to the transmitting device (e.g., FIG. 1-14the transmitting apparatus). In addition, the receiving apparatus can perform at least one of a reception operation, file / segment decapsulation, and point cloud decoding based on the feedback information. Details are described with reference to FIG. 17 Those described above are the same, and thus the description thereof will be omitted.

[0241] FIG. 17 An exemplary structure operatively connectable to a method / apparatus of transmitting and receiving point cloud data according to an embodiment is illustrated.

[0242] FIG. 17 A structure representing a configuration in which at least one of a server 1760, a robot 1710, a self-driving vehicle 1720, an XR apparatus 1730, a smart phone 1740, a home appliance 1750, and / or an HMD 1770 is connected to a cloud network 1700. The robot 1710, the self-driving vehicle 1720, the XR apparatus 1730, the smart phone 1740, or the home appliance 1750 is referred to as an apparatus. Furthermore, the XR apparatus 1730 can correspond to a point cloud data (PCC) apparatus according to an embodiment or can be operatively connected to the PCC apparatus.

[0243] The cloud network 1700 can represent a network that constitutes a part of or exists in a cloud computing infrastructure. Here, the cloud network 1700 can be configured using a 3G network, a 4G or long term evolution (LTE) network, or a 5G network.

[0244] The server 1760 can be connected to at least one of the robot 1710, the self-driving vehicle 1720, the XR apparatus 1730, the smart phone 1740, the home appliance 1750, and / or the HMD 1770 via the cloud network 1700, and can assist in processing of at least a part of the connected apparatuses 1710 to 1770.

[0245] The HMD 1770 represents one of implementation types of the XR apparatus and / or the PCC apparatus according to an embodiment. According to an embodiment, the HMD-type apparatus includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0246] Hereinafter, various embodiments of the apparatuses 1710 to 1750 to which the above-described technology is applied will be described. <PCC+XR> The apparatuses 1710 to 1750 illustrated can be operatively connected / coupled to a point cloud data transmitting / receiving apparatus according to the above-described embodiments.

[0247] <PCC+Self-driving+XR>

[0248] The XR / PCC device 1730 can employ PCC technology and / or XR (AR+VR) technology, and can be implemented as an HMD, a head-up display (HUD) disposed in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a stationary robot, or a mobile robot.

[0249] The XR / PCC device 1730 can analyze 3D point cloud data or image data acquired through various sensors or from an external device and generate position data and attribute data regarding 3D points. Thereby, the XR / PCC device 1730 can acquire information regarding a surrounding space or a real object, and render and output an XR object. For example, the XR / PCC device 1730 can match an XR object including auxiliary information regarding an identified object with the identified object and output the matched XR object.

[0250] FIG. 18

[0251] The self-driving vehicle 1720 can be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0252] The self-driving vehicle 1720 to which XR / PCC technology is applied can denote an autonomous vehicle provided with a means for providing an XR image, or an autonomous vehicle that is a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, the self-driving vehicle 1720 can be distinguished from and operatively connected with the XR device 1730.

[0253] The self-driving vehicle 1720 having a means for providing an XR / PCC image can acquire sensor information from sensors including a camera, and output a generated XR / PCC image based on the acquired sensor information. For example, the self-driving vehicle 1720 can have a HUD and output an XR / PCC image thereto to provide a passenger with an XR / PCC object corresponding to a real object or an object presented on a screen.

[0254] In this case, when the XR / PCC object is output to the HUD, at least a part of the XR / PCC object can be output to overlap with a real object pointed by the passenger's eyes. On the other hand, when the XR / PCC object is output on a display disposed inside the self-driving vehicle, at least a part of the XR / PCC object can be output to overlap with an object on a screen. For example, the self-driving vehicle 1220 can output an XR / PCC object corresponding to an object such as a road, another vehicle, a traffic light, a traffic sign, a two-wheeled vehicle, a pedestrian, and a building.

[0255] Virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or point cloud compression (PCC) technologies according to the implementation methods are applicable to various devices.

[0256] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. On the other hand, AR technology refers to the technology of displaying virtually created CG images on top of images of real objects. MR technology is similar to AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology differs from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with the same characteristics as real objects. More specifically, an example of MR technology application is holographic services.

[0257] Recently, VR, AR, and MR technologies have often been referred to as Extended Display (XR) technologies rather than being clearly distinguished from each other. Therefore, embodiments of this disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are applicable to such technologies.

[0258] The PCC method / apparatus according to the embodiments can be applied to vehicles that provide self-driving services.

[0259] Vehicles providing autonomous driving services connect to the PCC device for wired / wireless communication.

[0260] When the point cloud data (PCC) transmitting / receiving device according to the embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services (which may be provided together with autonomous driving services) and transmit it to the vehicle. When the PCC transmitting / receiving device is installed in the vehicle, it can receive / process content data related to AR / VR / PCC services based on user input signals input through a user interface device and provide it to the user. The vehicle or user interface device according to the embodiment can receive user input signals. User input signals according to the embodiment may include signals indicating autonomous driving services.

[0261] FIG. 1 This is a block diagram illustrating an exemplary point cloud encoder.

[0262] The point cloud encoder 1800 according to the embodiment (e.g., FIG. 4 Point cloud video encoder 10002 FIG. 12 Point cloud encoder, reference FIG. 14 , FIG. 15 and FIG. 1-17The described point cloud encoder) can perform the operations described with reference to FIG. 18 The described encoding operations. The point cloud encoder 1800 according to the embodiments can include a spatial partitioner 1810, a geometry information encoder 1820, and an attribute information encoder 1830. Although FIG. 1-17 not shown in the description, the point cloud encoder 1800 according to the embodiments can further include one or more elements to perform the operations described with reference to FIG. 4 The described encoding operations.

[0263] Point cloud compression (PCC) data (or PCC data or point cloud data) is input data of the point cloud encoder 1800 and can include geometry and / or attribute. The geometry according to the embodiments refers to information indicating a position of a point and can be expressed as a parameter of a coordinate system such as a Cartesian coordinate system, a cylindrical coordinate system, or a spherical coordinate system. According to the embodiments, the geometry can be referred to as geometry information, and the attribute can be referred to as attribute information.

[0264] The spatial partitioner 1810 according to the embodiments can generate geometry and attribute of the point cloud data. The spatial partitioner 1810 according to the embodiments can partition the point cloud data into one or more 3D blocks in a 3D space in order to store point information of the point cloud data. The block according to the embodiments can represent at least one of a tile group, a tile, a slice, a coding unit (CU), a prediction unit (PU), or a transform unit (TU). The spatial partitioner 1810 according to the embodiments can perform a partitioning operation based on at least one of an octree, a quadtree, a binary tree, a ternary tree, or a k-d tree. The block can contain one or more points. The block according to the embodiments can be a block having a hexahedral shape with a preset width, depth, and height. The size of the block according to the embodiments is changeable and is not limited to the above-described examples. The spatial partitioner 1810 according to the embodiments can generate geometry information about one or more points contained in the block.

[0265] The geometry information encoder (or geometry encoder) 1820 according to the embodiments can perform geometry encoding and generate geometry bitstream and reconstructed geometry information. In the geometry encoding according to the embodiments, the reconstructed geometry information is input to the attribute information encoder (or attribute encoder) 1830. The geometry information encoder 1820 according to the embodiments can perform at least one operation of the coordinate transformer 40000, the quantizer 40001, the octree analyzer 40002, the surface approximation analyzer 40003, the arithmetic encoder 40003, and the geometry reconstructor (reconstructed geometry) 40005 described with reference to FIG. 12 The described encoding operations. The point cloud encoder 1800 according to the embodiments can include a spatial partitioner 1810, a geometry information encoder 1820, and an attribute information encoder 1830. Although FIG. 19At least one operation of the described data input unit 12000, quantization processor 12001, voxelization processor 12002, octree occupancy code generator 12003, surface model processor 12004, intra / inter-frame encoding processor 12005, arithmetic encoder 12006, and metadata processor 12007 is described.

[0266] The attribute information encoder 1830 according to the embodiments can generate an attribute information bitstream (or attribute bitstream) based on the reconstructed geometry information and attributes.

[0267] The point cloud encoder according to the embodiments can transmit a bitstream in which the geometry information bitstream and the attribute information bitstream or the geometry information bitstream and the attribute information bitstream are multiplexed. As described above, the bitstream can further include signaling information related to the geometry information and the attribute information and signaling information related to the coordinate transformation. In addition, the point cloud encoder according to the embodiments can encapsulate the bitstream and transmit it in the form of a segment and / or a file.

[0268] FIG. 18 is a block diagram illustrating an exemplary geometry information encoder.

[0269] The geometry information encoder 1900 (or geometry encoder) according to the embodiments is FIG. 1-18 an example of the geometry information encoder 1820 of FIG. 18, and can perform geometry encoding. Since the geometry encoding according to the embodiments is the same as or similar to the geometry encoding described with reference to FIG. 19 , a detailed description thereof will be omitted. As illustrated, the geometry information encoder 1900 can include a coordinate transformer 1910, a geometry information transform quantizer 1920, a geometry information predictor 1930, a residual geometry information quantizer 1940, a geometry information entropy encoder 1950, a residual geometry information inverse quantizer 1960, a filter 1970, and a memory 1980. Although FIG. 1-18 not shown in FIG. 19, the geometry information encoder 1900 according to the embodiments can further include one or more elements to perform the geometry encoding described with reference to FIG. 4 .

[0270] The coordinate transformer 1910 according to the embodiments can transform the received geometry information into information in a coordinate system in order to express positions of respective points indicated by the input geometry information as positions in a 3D space. The coordinate transformer 1910 performs the same or similar operations as those of the coordinate transformer 40000 described with reference to FIG. 4 . As described above, the coordinate system according to the embodiments can include, but is not limited to, a three-dimensional Cartesian coordinate system, a cylindrical coordinate system, and a spherical coordinate system. The coordinate transformer 1910 according to the embodiments can transform a set coordinate system into another coordinate system.

[0271] The coordinate transformer 1910 according to the embodiments can perform a coordinate transformation on a unit such as a sequence, a frame, a tile, a slice, or a block. According to the embodiments, whether to transform a coordinate and information related to a coordinate system and / or a transformation can be signaled on a per sequence, frame, tile, slice, or block basis. Accordingly, the point cloud data receiving apparatus according to the embodiments can obtain information related to a coordinate system and / or a transformation based on a coordinate transformation state of a neighbor block, a size of a block, a number of points, a quantization value, a block partitioning depth, a location of a unit, a distance between a unit and an origin, or the like.

[0272] The geometry information transformation quantizer 1920 according to the embodiments can quantize geometry information presented in a coordinate system and generate transformation-quantized geometry information. The geometry information transformation quantizer 1920 according to the embodiments can apply one or more transformations such as a position transformation and / or a rotation transformation to a point position indicated by geometry information output from the coordinate transformer 1910, and perform quantization by dividing the transformed geometry information by a quantization value. The geometry information transformation quantizer 1920 can perform the same or similar operations as those of the quantizer 40001 of FIG. 4 and / or the quantization processor 12001 of FIG. 12. FIG. 12 FIG. 18 The quantization value according to the embodiments can vary based on a distance between an encoding unit (e.g., a tile, a slice, or the like) and an origin of a coordinate system or an angle with a reference direction. According to the embodiments, the quantization value can be a pre-set value.

[0273] The geometry information predictor 1930 according to the embodiments can calculate a prediction value (or predicted geometry information) based on quantization values of neighboring encoding units.

[0274] The residual geometry information quantizer 1940 can receive transformation-quantized geometry information and residual geometry information obtained by subtracting a prediction value, and quantize the residual geometry information using a quantization value to generate quantized residual geometry information.

[0275] The geometry information entropy encoder 1950 can entropy-encode quantized residual geometry information. The entropy encoding according to the embodiments can include exponential Golomb, context adaptive variable length coding (CAVLC), and context adaptive binary arithmetic coding (CABAC).

[0276] The residual geometry information inverse quantizer 1960 can reconstruct residual geometry information by scaling quantized geometry information using a quantization value. The reconstructed residual geometry information and predicted geometry information can be summed to generate reconstructed geometry information.

[0277] ​The filter 1970 can filter the reconstructed geometry information. According to embodiments, the filter 1970 can include a deblocking filter and an offset corrector. The filter 1970 according to embodiments can perform additional filtering on a boundary between two different coding units for geometry information obtained by differently coordinate transforming two coding units.

[0278] The memory 1980 can store the reconstructed geometry information (or restored geometry information). The stored geometry information can be provided to the geometry information predictor 1930. In addition, the reconstructed geometry information stored in the memory can be provided to the reference FIG. 20 The attribute information encoder 1830 described above.

[0279] FIG. 20 An exemplary attribute information encoder according to embodiments is illustrated.

[0280] FIG. 18 The attribute information encoder 2000 illustrated can perform attribute encoding as an example of the attribute information encoder 1830 described above. FIG. 1-17 The attribute information encoder 2000 illustrated can perform attribute encoding as an example of the attribute information encoder 1830 described above. FIG. 21 Since the attribute encoding according to embodiments is the same or similar to the attribute encoding described above, detailed descriptions thereof will be omitted. As illustrated, the attribute information encoder 2000 according to embodiments can include an attribute characteristic transformer 2010, a geometry information mapper 2020, an attribute information transformer 2030, an attribute information quantizer 2040, and an attribute information entropy encoder 2050.

[0281] The attribute characteristic transformer 2010 according to embodiments can receive attribute information and transform a characteristic (e.g., color, etc.) of the received attribute information. For example, when the attribute information includes color information, the attribute characteristic transformer 2010 can transform a color space of the attribute information (e.g., from RGB to YCbCr). In addition, the attribute characteristic transformer 2010 can selectively skip the transformation of the characteristic of the attribute information. The attribute characteristic transformer 2010 can perform operations the same as or similar to operations of the attribute transformer 40007 and / or the color transform processor 12008.

[0282] The geometry information mapper 2020 according to the embodiment can generate reconstructed attribute information by mapping attribute information output from the attribute characteristic transformer 2010 and the received reconstructed geometry information. The geometry information mapper 2020 can generate attribute information by reconstructing attribute information about one or more points based on the reconstructed geometry information. As described above, geometry information about one or more points included in a voxel can be reconstructed based on the median of the voxel. Since attribute information depends on geometry information, the geometry information mapper 2020 reconstructs attribute information based on the reconstructed geometry information. The geometry information mapper 2020 can perform operations that are the same as or similar to those of the attribute transformation processor 12009.

[0283] The attribute information transformer 2030 according to the embodiment can receive and transform reconstructed attribute information. The attribute information transformer 2030 according to the embodiment can predict attribute information and use one or more transformation types (e.g., DCT, DST, SADCT, RAHT) to transform residual attribute information corresponding to the residual between the received reconstructed attribute information and the predicted attribute information.

[0284] According to the implementation, the attribute information quantizer 2040 can receive transformed residual attribute information and generate transformed quantized residual attribute information based on the quantized value.

[0285] The attribute information entropy encoder 2050 according to the embodiment can receive transform-quantized residual attribute information, perform entropy encoding on it, and output an attribute information bitstream. The entropy encoding according to the embodiment may include (but is not limited to) one or more of the exponential Golomb, CAVLC, and CABAC. The attribute information entropy encoder 2050 can perform the same or similar operations as the arithmetic encoder 12011.

[0286] FIG. 21 An exemplary attribute information encoder according to an implementation is shown.

[0287] FIG. 18 The attribute information encoder 2100 shown corresponds to the reference. FIG. 20 The described attribute information encoder 1830 and reference FIG. 21 An example of the attribute information encoder 2000 described. According to an embodiment, the attribute information encoder 2100 may include an attribute characteristic transformer 2110, a geometric information mapper 2120, an attribute information predictor 2130, a residual attribute information transformer 2140, a residual attribute information inverse transformer 2145, a residual attribute information quantizer 2150, a residual attribute information inverse quantizer 2155, a filter 2160, a memory 2170, and an attribute information entropy encoder 2180. FIG. 20 The attribute information encoder 2100 shown is... FIG. 20The illustrated attribute information encoder 2000 is different in that a residual attribute information transformer 2140, a residual attribute information inverse transformer 2145, a residual attribute information quantizer 2150, a residual attribute information inverse quantizer 2155, a filter 2160, and a memory 2170 are further included in the encoder.

[0288] The attribute characteristic transformer 2110 and the geometry information mapper 2120 according to the embodiments can perform the same or similar operations as those of the attribute characteristic transformer 1010 and the geometry information mapper 1020 with reference to FIG. 10. FIG. 19 The attribute characteristic transformer 2010 and the geometry information mapper 2020 described above perform the same or similar operations as those of the attribute characteristic transformer 1010 and the geometry information mapper 1020 with reference to FIG. 10. The attribute information predictor 2130 according to the embodiments can generate predicted attribute information. The residual attribute information transformer 2140 can receive residual attribute information generated by obtaining a difference between reconstructed attribute information output from the geometry information mapper 2120 and the predicted attribute information. The residual attribute information transformer 2140 can transform a residual 3D block including the received residual attribute information using one or more transform types (e.g., DCT, DST, SA-DCT, RAHT, etc.).

[0289] The residual attribute information quantizer 2150 according to the embodiments can transform input transform residual attribute information based on a quantization value. The transformed residual attribute information can be input to the residual attribute information inverse quantizer 2155. The residual attribute information inverse quantizer 2155 according to the embodiments can transform the transform-quantized residual attribute information based on a quantization value and generate transformed residual attribute information. The transformed residual attribute information generated by the residual attribute information inverse quantizer 2155 is input to the residual attribute inverse transformer 2145. The residual attribute inverse transformer 2145 according to the embodiments can inverse-transform a residual 3D block including the transformed residual attribute information using one or more transform types (e.g., DCT, DST, SA-DCT, RAHT, etc.). According to the embodiments, reconstructed attribute information can be generated by combining the inverse-transformed residual attribute information and predicted attribute information output from the attribute information predictor 2130. According to the embodiments, reconstructed attribute information can be generated by combining non-inverse-transformed residual attribute information and predicted attribute information. The reconstructed attribute information can be input to the filter 2160. The attribute information predictor 2130, the residual attribute information transformer 2140, and / or the residual attribute information quantizer 2150 according to the embodiments can perform the same or similar operations as those of the prediction / lifting / RAHT transform processor 12010.

[0290] The filter 2160 according to the embodiments can filter the reconstructed attribute information. The filter 2160 according to the embodiments can include a deblocking filter, an offset corrector, and an adaptive loop filter (ALF). The filter 2160 can perform the same or similar operations as those of the filter 1970 of FIG. 19. FIG. 20 The filter 2160 according to the embodiments can filter the reconstructed attribute information. The filter 2160 according to the embodiments can include a deblocking filter, an offset corrector, and an adaptive loop filter (ALF). The filter 2160 can perform the same or similar operations as those of the filter 1970 of FIG. 19.

[0291] The memory 2170 according to the embodiments can store the reconstructed attribute information output from the filter 2160. The stored reconstructed attribute information can be provided as input data for a prediction operation of the attribute information predictor 2130. The attribute information predictor 2130 can generate predicted attribute information based on the reconstructed attribute information about the points. Although the memory 2170 is illustrated as one block in the diagram, it can include one or more physical memories. The attribute information entropy encoder 2180 according to the embodiments can perform the same or similar operations as those of the attribute information entropy encoder 2050 described with reference to FIG. 22 FIG. 21.

[0292] FIG. 22 An exemplary attribute information predictor according to the embodiments is illustrated.

[0293] FIG. 21 The illustrated attribute information predictor 2200 corresponds to an example of the attribute information predictor 2130 described with reference to FIG. 4 FIG. 21. The attribute information predictor 2200 can perform the same or similar operations as those of the attribute information predictor 2130. The attribute information predictor 2200 according to the embodiments can include an LOD configurator 2210 and a neighbor point set configurator 2220. The LOD configurator 2210 can perform the same or similar operations as those of the LOD generator 40009. That is, as illustrated, the LOD configurator 2210 can receive attributes and reconstructed geometry, and configure one or more LODs based on the received attributes and reconstructed geometry. As described above with reference to FIG. 8 and FIG. 4 The LODs can be generated by reorganizing points distributed in a 3D space into a set of refinement levels. According to the embodiments, the LODs can include one or more points distributed at regular intervals. As described above, the LODs according to the embodiments indicate levels of detail of point cloud content. Thus, as the level indicated by the LODs (or LOD values) decreases, the details of the point cloud content can deteriorate. As the level indicated by the LODs increases, the details of the point cloud content are enhanced. That is, the LODs indicating higher levels can include points distributed at shorter intervals. The point cloud encoder (e.g., the point cloud encoder of FIG. 11 ) and the point cloud decoder (e.g., the point cloud decoder of FIG. 10-11 ) according to the embodiments can generate the LODs to increase attribute compression rates. Since points having similar attributes are likely to be within the neighborhood of a target point, the residual values between the predicted attributes obtained based on the neighbor points having similar attributes and the attributes of the target point are likely to be close to 0. Thus, the point cloud encoder and the point cloud decoder according to the embodiments can generate the LODs so as to select appropriate neighbor points that can be used for attribute prediction.

[0294] The LOD configurator 2210 according to the implementation can use one or more methods to configure LOD. As described above, the point cloud decoder (e.g., referring to...) FIG. 1 The described point cloud decoder can also generate LODs. Therefore, information related to the LOD configuration method (or LOD generation method) or LOD configuration method information according to the embodiment is sent to the receiving device (e.g., via a bitstream containing encoded point cloud video data). FIG. 10 The receiving device 10004 or FIG. 11 and FIG. 9 (Point cloud decoder). Therefore, the receiving device can generate LOD based on LOD configuration method information.

[0295] When generating individual LODs (or sets of LODs), the neighbor point set configurator 2220, according to the implementation method, can search or retrieve LODs. l A set of points has one or more neighboring points. The number of one or more neighboring points can be X, where X is a positive integer. According to the implementation, a neighboring point is the one closest to the LOD in 3D space. l The set of points is the nearest neighbor (NN) points, and includes points in the LOD of the target (e.g., LOD). l A set of LODs of the same level or at a lower level than the target LOD (e.g., LODs). l-1 LOD l-2 In the neighbor point set configurator 2220 according to the embodiment, one or more searched neighbor points can be registered as a neighbor point set in the predictor. The number of neighbor points according to the embodiment can be set to a maximum number of neighbor points according to a user input signal, or can be preset to a specific value according to the neighbor point search method.

[0296] According to the implementation method, the neighbor set configurator 2220 can search for belonging points within LOD0 and LOD1. FIG. 9 The neighboring points of point P3 in LOD1 are shown. FIG. 9 As shown, LOD0 includes P0, P5, P4, and P2. LOD2 includes points from LOD0, points from LOD1, P9, P8, and P7. When the number of neighboring points X is 3, the neighboring point set configurator 2220... FIG. 1searches for three neighbor points closest to P3 among points belonging to LOD0 or LOD1 in the 3D space shown at the top of FIG. 23. That is, the neighbor point set configurator 2220 can search for P6 belonging to the same LOD level of LOD1 and P2 and P4 belonging to a lower LOD level of LOD0 as neighbor points of P3. In the 3D space, P7 is a point close to P3, but is not searched as a neighbor point because it is at a higher LOD level. The neighbor point set configurator 2220 can register the searched neighbor points P2, P4, and P6 as a neighbor point set in the predictor of P3. The method of generating a neighbor point set according to the embodiment is not limited to this example. Neighbor point set generation information (hereinafter referred to as neighbor point set generation information) about the method of generating a neighbor point set according to the embodiment is transmitted to a receiving apparatus (e.g., a point cloud decoder, etc.) through a bitstream including the above-described encoded point cloud video data. FIG. 10 the receiving apparatus 10004, FIG. 11 and FIG. 9 the point cloud decoder, etc.) of the above-described point cloud encoder.

[0297] As described above, each point can have one predictor. The point cloud encoder according to the embodiment can encode attribute values of a corresponding point by applying a predictor, and can generate a predicted attribute (or a predicted attribute value). According to the embodiment, a predictor is generated based on searched neighbor points after LOD generation. The predictor is used to predict an attribute of a target point. Accordingly, the predictor can generate a predicted attribute by applying weights to attributes of neighbor points.

[0298] For example, the predictor can calculate and register weights based on distance values (e.g., 1 / 2 distances) between a target point (e.g., P3) and each neighbor point in a neighbor point set. As described above, since the neighbor point set of P3 includes P2, P4, and P6, the point cloud encoder (or predictor) according to the embodiment calculates weights based on distance values between P3 and each neighbor point. Accordingly, the weight of each neighbor point is expressed as When configuring a neighbor point set of a predictor, the point cloud encoder according to the embodiment can normalize weights of neighbor points using a sum of the weights of the neighbor points. For example, the sum of the weights of all neighbor points in the neighbor point set of node P3 is expressed as follows. By dividing the sum of the weights (total_weight) by the weight of each neighbor point, a normalized weight is expressed as

[0299] The point cloud encoder (or attribute information predictor) according to the embodiment can predict an attribute through a predictor. The predicted attribute (or predicted attribute information) according to the embodiment can be a value obtained by applying weights to attributes of neighbor points. For example, the predicted attribute of P3 can be calculated by applying weights to attributes of neighbor points P2, P4, and P6. The predicted attribute of P3 is expressed as follows. FIG. 1The average of the values obtained by multiplying the attributes of the registered neighbor points by the calculated weights, or can be a value obtained by multiplying the attributes of a certain point by the weights. According to embodiments, after the compression result values are calculated in advance, the point cloud encoder can selectively use the predicted attribute values to generate the minimum stream (predicted attribute value with the highest compression efficiency) among the above attribute values. The method of predicting attributes is not limited to the above example.

[0300] The point cloud encoder (e.g., the coefficient quantizer 40011, etc.) according to embodiments can encode and transmit information on attribute values of points, residuals of predicted attribute values, and selected predicted attributes (or information on a method of selecting predicted attributes) to a receiving device (e.g., the receiving device 10004 of FIG. 1, or a point cloud decoder of FIG. 2) according to embodiments. FIG. 10 FIG. 11 and FIG. 1 of FIG. 2). The receiving device according to embodiments performs the same processes of LOD generation, neighbor point set generation, weighted normalization of neighbor points, and attribute prediction as those performed by the transmitting device. The receiving device can predict attributes based on information on selected predicted attributes using the same method as that performed by the transmitting device. The receiving device can decode the received residual values and reconstruct attribute values by adding the decoded residual values to predicted attribute values.

[0301] As described above, the LOD configurator 2210 according to embodiments can generate LODs based on one or more LOD generation methods.

[0302] The LOD configurator 2210 according to embodiments can generate LODs based on distances (distance-based LOD generation method). The LDO configurator 2210 according to embodiments can set distances (e.g., Euclidean distances) between at least two points for each LOD, and calculate distances between all points distributed in a 3D space to generate LODs based on the calculation results (LOD generation method based on calculation of distances between neighbor points). As described above, each LOD can include points distributed at regular intervals according to the level indicated by the LOD. Accordingly, the LOD configurator 2210 should calculate distances between all points. That is, since the point cloud encoder and the point cloud decoder need to calculate distances between all points each time an LOD is generated, unnecessary burdens can be caused in the process of processing point cloud data. However, when the density of point cloud content is high, dense points have a high geometric-based proximity relationship, and thus are highly likely to have similar attributes. Accordingly, the LOD configurator 2210 according to embodiments can generate LODs of point cloud content having a high density by calculating distances between points.

[0303] ​According to embodiments, the LOD configurator 2210 can configure the LOD based on the Morton code of the points. As described above, the Morton code is generated by presenting coordinate values (e.g., (x, y, z)) representing 3D positions of all points as bit values and mixing the bits.

[0304] The LOD configurator 2210 according to embodiments can generate the Morton code of each point based on the reconstructed geometry and sort the points in ascending order based on the Morton code. The order of the points sorted in ascending order of the Morton code can be referred to as a Morton order. The LOD configurator 2210 can configure the LOD by performing sampling on the points sorted in the Morton order. For example, the LOD configurator 2210 according to embodiments can sequentially select points based on a sampling rate gap according to the Morton order of the points included in each region corresponding to a node. The sampling rate (which can be expressed as, for example, k l ) according to embodiments can be automatically changed according to the distribution of the point cloud and the content of the point cloud, or can be changed according to a user input. In addition, the sampling rate according to embodiments can have a fixed value. The LOD configurator 2210 can select points sorted at a position separated from a point (0th point) sorted first according to the Morton order by a sampling rate (e.g., 5) (e.g., when the sampling rate is 5, the configurator can select the fifth point after the first point). For the remaining points, one can be selected from every 5 points (e.g., a point at the fifth position after the fifth point), and the selected points can be classified as a level (LOD l-1 ) lower than the current LOD (e.g., LOD1). Accordingly, the LOD configurator 2210 can configure the LOD without calculating the distance between all points. That is, the point cloud encoder and the point cloud decoder do not need to calculate the distance between all points each time the LOD is generated, so they can process the point cloud data more quickly. Furthermore, the LOD configurator 2210 according to embodiments can perform sampling differently for each LOD.

[0305] However, when the density of the content of the point cloud is low, the geometric-based proximity between points is low, and thus the distributed points are less likely to have similar attributes. Accordingly, in order to increase the accuracy of encoding and decoding of attribute information, the LOD configurator 2210 according to embodiments can generate the LOD by performing sampling based on any one of a fixed sampling range, an octree-based fixed sampling range, and an octree-based dynamic sampling range. Information about the sampling range and the sampling according to embodiments is included in the above-described LOD configuration method (or LOD generation method) information and transmitted to a receiving device (e.g., the receiving device 10004 of FIG. 10 , or the point cloud decoder of FIG. 11 and FIG. 23 ) containing the encoded point cloud video data. Accordingly, the receiving device can generate the LOD based on the LOD configuration method information.

[0306] The LOD configurator 2210 according to embodiments can perform sampling based on a fixed sampling range. The sampling rate according to embodiments can be set differently for various regions of the 3D space according to the density of the point cloud. In addition, the sampling rate according to embodiments can be set differently for various LODs. The sampling range according to embodiments is fixed.

[0307] FIG. 23 An example LOD generation process is shown.

[0308] FIG. 23 An example 2300 of the LOD generation process is shown. FIG. 22 An example of generating LODs based on a fixed sampling range when the sampling rate (which can be denoted as, for example, k l ) is 4 according to embodiments is shown. The fixed sampling range according to embodiments is the same as described with reference to FIG. 23 , and thus the description thereof will be omitted. According to embodiments, the sampling rate can vary according to the distribution of the point cloud, user input, etc. In addition, the sampling rate according to embodiments can have a fixed value.

[0309] FIG. 23 The upper portion of shows the points included in each of the two spaces created by partitioning the 3D space. As shown, the first space 2301 can contain five points P0, P1, P5, P6, and P9, and the second space 2302 can contain five points P2, P3, P4, P7, and P8.

[0310] In FIG. 22 , the first index 2310 indicates the original order of the 10 points distributed in the 3D space. As described above, the LOD configurator (e.g., the LOD configurator 2210 of FIG. 23 ) can calculate the Morton code of each point and sort the points in ascending order I l based on the calculated Morton codes. In FIG. 23 , the second index 2320 indicates the Morton order of the 10 points. Since the sampling rate according to embodiments is 4, the example 2330 of the selected points for each of every four points indicates points P5, P1, and P7. In the LOD l , only the unselected points are included. In FIG. 23 , the third index 2340 indicates the indices of the points in LOD1. The LOD configurator according to embodiments can re-perform sampling (or sub-sampling) on the selected points P5, P1, and P7 to generate LOD l-1 . FIG. 24 An example 2350 of the selected point according to sampling (i.e., P5) is shown. In the LOD l-1The graph only includes unselected points (e.g., P1, P7). The fourth index 2350 shown in the graph indicates the AN LOD order to which the points belong. As a result of sampling, LOD0 can have P5, LOD1 can have points P5, P1, and P7, and LOD2 can have all points 2360.

[0311] FIG. 24 An exemplary Morton order-based sampling process is shown.

[0312] FIG. 24 The left part of shows the Morton order 2400 of points in one of the spaces formed by partitioning the three-dimensional space. As described above, the LOD configurator (e.g., the LOD configurator 2210) according to the embodiments can calculate the Morton code of each point and sort the points in ascending order based on the calculated Morton code. The numbers shown in the graph indicate the order of the points sorted in ascending order based on the Morton code. FIG. 24 The right part of shows an example 2410 of an LOD generated by performing sampling on the Morton order of points in the plurality of spaces partitioned in the three-dimensional space. FIG. 24 The example 2410 of shows an example in which the sampling rate is 4. The circles 2415 presented in the coordinate system represent the points selected in the sampling process. The lines 2418 indicated in the coordinate system represent the process of selecting the points. As described above, the LOD configurator (e.g., the LOD configurator 2210) can configure the LOD by performing sampling on the points sorted in the Morton order. However, as described above, the distance between the points sorted in the Morton order can not be constant. In addition, the points are not uniformly distributed in the 3D region (e.g., the 3D region represented by the X-axis, the Y-axis, and the Z-axis). Therefore, even if sampling is performed on the points sorted in the Morton order at a constant sampling rate, the sampling result can not guarantee the minimum and maximum values of the distance between the points. FIG. 5 As shown in the example of, the Morton order can be represented by a zigzag line. Therefore, the distance between the points sorted in the Morton order can not be constant. In addition, the points are not uniformly distributed in the 3D region (e.g., the 3D region represented by the X-axis, the Y-axis, and the Z-axis). Therefore, even if sampling is performed on the points sorted in the Morton order at a constant sampling rate, the sampling result can not guarantee the minimum and maximum values of the distance between the points.

[0313] The LOD configurator according to the embodiments can perform sampling according to a fixed sampling range based on an octree. The sampling rate according to the embodiments can be set differently for each depth of the octree. The sampling range according to the embodiments is fixed. As described with reference to FIG. 6 and FIG. 25The octree according to the embodiment can be generated by recursively dividing a 3D space of point cloud content into eight equal parts. The region of the recursive division according to the embodiment is a cube or cuboid having the same volume. The octree according to the embodiment has an occupancy code indicating whether each of the eight divided spaces generated by dividing one space includes at least one point. The occupancy code can include a plurality of nodes, and each node can indicate whether there is any point in each divided space. For example, when at least one point is included in the divided space, the node corresponding to the space indicates that there is a point (e.g., assigns "1"). The depth of each node corresponds to the level indicated by at least one LOD.

[0314] The node region corresponding to each level can guarantee a maximum distance between points in the LOD. The maximum distance between selected points among points within the node according to the embodiment is the node size, and the maximum distance between selected nodes among the nodes is limited by the node size. Accordingly, the sampling according to the embodiment can maintain a constant maximum distance between points, and thus the LOD configurator can reduce the complexity of the LOD generation and generate an LOD having high compression efficiency.

[0315] In addition, the LOD configurator according to the embodiment can perform sampling according to a dynamic sampling range based on the octree. The sampling rate according to the embodiment can be set based on the level of the LOD and the depth of the octree. In addition, the sampling range according to the embodiment is not fixed, and can be calculated based on the LOD level and the Morton code value. The LOD configurator according to the embodiment can group points based on the octree for sampling, rather than grouping (selecting) a number of points corresponding to each sampling rate for sampling. In the octree structure, it is clear that points belonging to the same parent node are adjacent to each other. Accordingly, the LOD configurator according to the embodiment does not need to additionally calculate the distance between selected points in order to check whether the selected points are actually neighbors. In addition, the LOD configurator according to the embodiment can check the density of the point cloud content based on whether the nodes of the occupancy code of the octree are allocated (occupied), and select points for sampling based on the density. For example, if all nodes are allocated, this indicates that the density of points in the region represented by each node is high. If the node is not used, this indicates that the density of points is low. Since high-density points have high similarity in attributes, the LOD configurator can select fewer points for sampling than low-density points.

[0316] The sampling based on the fixed sampling range according to the embodiment includes an operation of repeatedly selecting the same number of points according to the sampling rate regardless of the density of points. The repeated point selection process can cause unnecessary burden in the LOD generation operation.

[0317] FIG. 25 An LOD generation process according to the embodiment is shown.

[0318] FIG. 24 An example 2500 of a LOD generation procedure according to the octree-based dynamic sampling range is shown. Since the octree-based dynamic sampling range according to the embodiment is the same as that described with reference to FIG. 25 FIG. 2, a detailed description thereof will be omitted. The octree-based dynamic sampling range according to the embodiment can be set based on child points from the same parent node in the octree.

[0319] The maximum level of the LOD according to the embodiment is denoted as lmax. Since the density of the point cloud contents is different as described above, the maximum level of the LOD can be differently set for each point cloud content. Thus, according to the embodiment, each point cloud content can have the same or different octree depth for the maximum level of the LOD. The depth of the octree according to the embodiment can be denoted as d. The maximum level of the LOD according to the embodiment is lower than the depth of the octree (l max < d).

[0320] The maximum sampling rate of the LOD (e.g., LOD1) according to the embodiment can be expressed as follows.

[0321] [Equation 1]

[0322] k = 8 l max-l+1

[0323] Here, k denotes the maximum sampling rate. The LOD configurator according to the embodiment can change the sampling rate according to the point density of the point cloud content. In the above equation, 1 denotes sampling performed at the depth level (e.g., level d-1) of the octree. When points are dense, it is highly likely that all nodes of the level d-1 are allocated. However, since the density is different between the point cloud contents, the LOD configurator according to the embodiment can set a value for configuring the LOD at a certain depth of the occupancy code of the octree. The maximum sampling rate of the LOD (e.g., LOD1) according to the embodiment can be expressed as follows.

[0324] [Equation 2]

[0325] k = 8 lmax-l+X

[0326] Here, X has a value for configuring the LOD at a certain depth of the octree and is greater than 0. According to the embodiment, X can be differently set for each point cloud content.

[0327] FIG. 1The illustrated example 2500 shows nodes from which sampling is performed in the case where X is 3 and in the case where X is 1. When X is 3, the LOD configurator according to the embodiments can perform sampling from node 2510 corresponding to depth 3 (e.g., LOD value of 0). When X is 1, the LOD configurator according to the embodiments can perform sampling from node 2520 corresponding to depth 1. According to the embodiments, the number of child nodes is represented as pow(8, x), which is indicated by 2530.

[0328] According to the embodiments, the sampling range of each LOD can be expressed as follows.

[0329] [Equation 3]

[0330] P i. mc~P i. mc+(k-(P i. mc%k))

[0331] Here, Pi.mc denotes the Morton code value of the i-th point. The sampling range according to the embodiments can be limited to the range of the Morton code having the same upper parent node among the occupancy codes of the octree. For example, the LOD level is the maximum level, and the sampling range of the point having the Morton code of 2 among the points ordered in the Morton order is set to the range including the points having the Morton codes of 0 to 7 (because the occupancy code has 8 nodes at each depth). When the Morton code value of the point is greater than the maximum value of the sampling range (e.g., P i. mc+(k-(P i. mc%k)) according to the embodiments can recalculate the next sampling range.

[0332] Accordingly, when the points are configured in the octree structure, the maximum sampling rate of LOD0 becomes 8 (8 0+1 = 8). The sampling range of LOD0 can be the Morton codes of 0 to 7, 8 to 15, 16 to 23, …. The maximum sampling rate of LOD1 according to the embodiments can be 64 (8 1+1 = 64), and the sampling range of LOD1 can be 0 to 63, 64 to 127, 128 to 192, ….

[0333] According to the embodiments, the sampling range can include only one point. The one point included in the sampling range can be referred to as an isolated point. In order to increase compression efficiency, the LOD configurator can use one or more methods to process the isolated point.

[0334] For example, the LOD configurator can perform sampling on the isolated point. The LOD can select one point from among k consecutive isolated points, where k is the fixed sampling rate described above.

[0335] The LOD configurator according to the embodiment can separate the isolated point as a candidate group of LODs of a level lower than the current LOD. For a sampling range having the isolated point, the LOD configurator according to the embodiment selects the isolated point as a candidate for the LOD l during generation of the LOD (0至l-1) . When generating the LOD l-1-α , the LOD configurator according to the embodiment can select the isolated point by applying a wider sampling range and a sampling rate.

[0336] The LOD configurator according to the embodiment can register the isolated point in the current LOD. For example, when the isolated point is generated during generation of the LOD l , the LOD configurator registers the isolated point in the LOD l . This is because the isolated point is less likely to have attributes that can represent neighbor points, and thus does not need to be selected as a neighbor point set of other points belonging to the LOD l .

[0337] According to the embodiment, information about the isolated point processing method is transmitted to a receiving device (e.g., the receiving device 10004 or the point cloud decoder of FIG. 10 and FIG. 11 and FIG. 22 by including encoded point cloud video data in a bitstream of the above-described LOD configuration method (or LOD generation method) information. Accordingly, the receiving device can generate the LOD based on the LOD configuration method information.

[0338] The LOD configurator according to the embodiment (e.g., the LOD configurator 2210 of FIG. 1 ) can select one point from among points within a sampling range, classify the selected point as an LOD (e.g., LOD (0至-l-1) ) of a level lower than a level of a current LOD (e.g., LOD1), and register the remaining unselected points in the current LOD.

[0339] The LOD constructor according to the embodiment can select points based on one or more methods in order to generate a good neighbor node set for prediction encoding of attributes and increase compression efficiency.

[0340] For example, the LOD configurator according to the embodiment can select an Nth point within a sampling range. According to the embodiment, N can be set according to a user input signal. In addition, according to the embodiment, N can be set based on rate-distortion optimization (RDO).

[0341] The LOD configurator according to the embodiments can select a point whose Morton code is most similar to the mean value or mean value Morton code value of points belonging to the sampling range. As the similarity of the Morton code value increases, the probability that the distance between the points is close increases. This is because a point whose Morton code is most similar to the mean value Morton code value of points within the same sampling range is highly likely to have an attribute (e.g., color, reflectance value, etc.) that can represent the points in the sampling range or an attribute that can have a relatively greater influence on the points belonging to the sampling range. Specifically, when a point that can represent a neighboring value is selected as a neighbor node set, compression efficiency can increase. In addition, the LOD configurator according to the embodiments can select an Nth point from among the points sorted based on the mean value Morton code value. According to the embodiments, N is a preset value. When there is no set value of N, the LOD configurator according to the embodiments can select a 0th point from among the sorted points.

[0342] However, when the points within the sampling range are not uniformly distributed (e.g., they are unevenly distributed), the mean value Morton code value of the points can not correctly represent the sampling range. Accordingly, the LOD configurator according to the embodiments can select a point whose Morton code is most similar to the ideal mean value Morton code value of the sampling range. Assuming that all of the child nodes of a parent node are allocated in an octree structure, the LOD configurator according to the embodiments calculates the ideal mean value Morton code value and selects a point whose Morton code is most similar to the ideal mean value Morton code value. The LOD configurator according to the embodiments can select an Nth point from among the points sorted based on the ideal mean value Morton code value. According to the embodiments, N is a preset value. When there is no set value of N, the LOD configurator according to the embodiments can select a 0th point from among the sorted points.

[0343] The LOD configurator according to the embodiments can select one or more points within the sampling range. The number of points to be selected can be preset. According to a method of selecting an Nth point within the sampling range, the LOD configurator can iteratively select an Nth point from among the points within the sampling range. For example, when N is 2, the LOD configurator repeatedly selects a second point, a fourth point, etc., as many as a preset number X within the sampling range. According to a method of selecting a point whose Morton code is most similar to the mean value Morton code value of points belonging to the sampling range, the LOD configurator can repeatedly select as many points corresponding to an Nth position as a preset number X in the order sorted according to the mean value Morton code value. According to a method of selecting a point whose Morton code is most similar to the ideal mean value Morton code value of points belonging to the sampling range, the LOD configurator can iteratively select as many points corresponding to an Nth position as a preset number X in the order sorted according to the ideal mean value Morton code value.

[0344] The point selection method according to the embodiments can also be applied when selecting the above-described isolated points. In addition, information about the point selection method according to the embodiments is transmitted to a receiving device (e.g., a decoder) through a bitstream including encoded point cloud video data by being included in the above-described LOD configuration method (or LOD generation method) information.FIG. 10 receiving apparatus 10004 or FIG. 11 and FIG. 26 point cloud decoder). Thus, the receiving apparatus can generate an LOD based on the LOD configuration method information.

[0345] FIG. 1 An exemplary structure diagram of a point cloud compression (PCC) bitstream is illustrated.

[0346] As described above, the point cloud data transmitting apparatus (for example, the point cloud data transmitting apparatus described with reference to FIG. 12 , FIG. 14 , FIG. 15 and FIG. 1 may transmit the encoded point cloud data in the form of a bitstream 2600. The bitstream 2600 according to the embodiment can include one or more sub-bitstreams.

[0347] The point cloud data transmitting apparatus (for example, the point cloud data transmitting apparatus described with reference to FIG. 12 , FIG. 14 , FIG. 15 and FIG. 25 may divide an image of the point cloud data into one or more packets in consideration of errors of a transmission channel, and transmit the same via a network. According to the embodiment, the bitstream 2600 can include one or more packets (for example, network abstraction layer (NAL) units). Thus, even when some packets are lost in a poor network environment, the point cloud data receiving apparatus can reconstruct an image using the remaining packets. The point cloud data can be segmented into one or more slices or one or more tiles to be processed. The tile and the slice according to the embodiment are regions in which point cloud compression encoding is performed by segmenting a picture of the point cloud data. The point cloud data transmitting apparatus can provide high-quality point cloud content by processing data corresponding to each region according to the importance of each segmented region of the point cloud data. That is, the point cloud data transmitting apparatus according to the embodiment can perform point cloud compression encoding with better compression efficiency and appropriate delay on data corresponding to a region important to a user.

[0348] According to the embodiment, an image (or a picture) of the point cloud content can be segmented into a basic processing unit for point cloud compression encoding. The basic processing unit for point cloud compression encoding according to the embodiment can include, but is not limited to, a coding tree unit (CTU) and a block. The slice according to the embodiment is a region including an integer number of one or more basic processing units for point cloud compression encoding and does not have a rectangular shape. The slice according to the embodiment includes data transmitted by grouping. The tile according to the embodiment is a region segmented in a rectangular shape in an image and includes one or more basic processing units for point cloud compression encoding. According to the embodiment, one slice can be included in one or more tiles. In addition, according to the embodiment, one tile can be included in one or more slices.

[0349] The bitstream 2600 according to the embodiment can include signaling information including a sequence parameter set (SPS) for sequence level signaling, a geometry parameter set (GPS) for geometry information encoding signaling, an attribute parameter set (APS) for attribute information encoding signaling, and a tile parameter set (TPS) for tile level signaling, and one or more slices.

[0350] The SPS according to the embodiment is encoding information about an entire sequence including a profile and a level, and can include comprehensive information about an entire file, such as a picture resolution and a video format.

[0351] According to the embodiment, one slice (for example, slice 0 of FIG. 22-25 ) includes a slice header and slice data. The slice data can include one geometry bitstream (Geom0 0 ) and one or more attribute bitstreams (Attr0 0 , Attr1 0 ).

[0352] The geometry bitstream can include a header (for example, a geometry slice header) and a payload (for example, geometry slice data). The header of the geometry bitstream according to the embodiment can include identification information of a parameter set included in the GPS (geom_geom_parameter_set_id), a tile identifier (geom_tile_id), a slice identifier (geom_slice_id), and information about data included in the payload. The attribute bitstream can include a header (for example, an attribute slice header or an attribute tile header) and a payload (for example, attribute slice data or attribute tile data).

[0353] As described above with reference to FIG. 26 , the point cloud encoder and the point cloud decoder according to the embodiment can generate an LOD for attribute prediction. Accordingly, FIG. 18-25 , the bitstream can contain LOD configuration method information described with reference to FIG. 12 . The point cloud decoder according to the embodiment can generate an LOD based on the LOD configuration method information.

[0354] The signaling information included in the bitstream according to the embodiment can be generated by a metadata processor or a transmission processor (for example, the transmission processor 12012 of FIG. 27 ) included in the point cloud encoder or an element of the metadata processor or the transmission processor. According to the embodiment, the signaling information can be generated based on the results of geometry encoding and attribute encoding.

[0355] FIG. 27 An exemplary syntax of the APS according to the embodiment is shown.

[0356] FIG. 27 An example 3100 of syntax of an APS according to an embodiment is shown, and can include the following information (or fields, parameters, etc.).

[0357] The aps_attr_parameter_set_id specifies an identifier of the APS for other syntax elements to refer to. The value of aps_attr_parameter_set_id is in the range of 0 to 15. As shown, since one or more attribute bitstreams are included in the bitstream, a field (e.g., ash_attr_parameter_set_id) having the same value as aps_attr_parameter_set_id can be included in the header of each attribute bitstream. FIG. 22

[0358] A point cloud decoder according to an embodiment can acquire an APS corresponding to each attribute bitstream based on aps_attr_parameter_set_id and process the attribute bitstream.

[0359] The aps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id of the active SPS. The value of aps_seq_parameter_set_id is in the range of 0 to 15.

[0360] For a given value of attr_coding_type, attr_coding_type indicates the attribute coding type. In the bitstream according to an embodiment, the value of attr_coding_type is equal to 0, 1, or 2. Other values of attr_coding_type can be reserved for future use by ISO / IEC. Accordingly, a decoder according to an embodiment can ignore the reserved values of attr_coding_type described above. Among the values, 0 can indicate a prediction weight lifting transform coding as the attribute coding type, 1 can indicate a RAHT transform coding as the attribute coding type, and 2 can indicate a fixed weight lifting transform coding as the attribute coding type.

[0361] When attr_coding_type is equal to 0 or 2, the following parameters are given.

[0362] The num_pred_nearest_neighbours indicates the maximum number of nearest neighbors to be used for prediction. The value of numberOfNearestNeighboursInPrediction can be included in the range of 1 to xx.

[0363] ​max_num_direct_predictors indicates the number of predictors to be used for direct prediction. The value of max_num_direct_predictors is in the range of 0 to num_pred_nearest_neighbours. The value of the variable MaxNumPredictors used in the decoding operation is as follows.

[0364] MaxNumPredictors = max_num_direct_predicots + 1

[0365] lifting_search_range indicates a search range for lifting.

[0366] lifting_quant_step_size indicates a quantization step size of a first component of the attribute. The value of lifting_quant_step_size can be in the range of 1 to xx.

[0367] lifting_quant_step_size_chroma can indicate a quantization step size of a chroma component of the attribute when the attribute is a color. The value of lifting_quant_step_size_chroma can be in the range of 1 to xx.

[0368] lod_binary_tree_enabled_flag indicates whether a binary tree is applied to LOD generation.

[0369] num_detail_levels_minus1 indicates the number of LODs for attribute coding. The value of num_detail_levels_minus1 can be in the range of 0 to xx. The following "for" statement is information about each LOD.

[0370] sampling_distance_squared[idx] can indicate the square of the sampling distance of each LOD indicated by idx. According to an embodiment, the value of idx is in the range of 0 to the number of LODs applied to attribute coding indicated by num_detail_levels_minus1. The value of sample_distance_squared is in the range of 0 to xx.

[0371] The LOD configuration information 2700 according to an embodiment includes the following information.

[0372] `lod_generation_type` is information indicating the type of method used to configure (generate) the LOD collection. `lod_generation_type` can be set to either 1 or 2, but its value is not limited to this example. When `lod_generation_type` equals 1, it indicates that the LOD collection generation method is distance-based LOD generation (e.g., see reference). FIG. 22 The description refers to distance-based LOD generation. When lod_generation_type equals 2, lod_generation_type indicates that the LOD set generation method is based on Merton order sampling LOD generation (e.g., see reference). FIG. 22 The described method is a sampling LOD generation method based on the Morton order. Because the LOD set generation method differs from the reference method... FIG. 22 Since the descriptions are the same, their detailed descriptions will be omitted.

[0373] The following is information related to sampling when lod_generation_type equals 2.

[0374] `sampling_range_type` indicates the sampling range and sampling rate, or the type of sampling range. `sampling_range_type` can be set to any of 1, 2, and 3, but the value of `sampling_range_type` is not limited to this example. When `sampling_range_type` equals 1, `sampling_range_type` indicates that the sampling range is a fixed sampling range (e.g., see reference...). FIG. 23 and FIG. 22 The sampling range is described as a fixed sampling range. When sampling_range_type equals 2, sampling_range_type indicates that the sampling range is a fixed sampling range based on an octree (e.g., see reference). FIG. 24 and FIG. 22 The description refers to a fixed sampling range based on an octree. When sampling_range_type equals 3, sampling_range_type indicates a dynamic sampling range based on an octree (e.g., see reference). ​ and FIG. 25 The description refers to the dynamic sampling range based on an octree. Since the sampling range and sampling rate according to the implementation differ from the reference... FIGS. 22-25 The descriptions are the same, so their detailed descriptions will be omitted.

[0375] The sampling_rate indicates the sampling rate. The value of sampling_rate is an integer greater than 0.

[0376] The sampling_select_type indicates a point selection method of selecting a point within a sampling range. The sampling_select_type can be set to any one of 1, 2, and 3, but the value of the sampling_select_type is not limited to this example. When the sampling_select_type is equal to 1, the sampling_select_type indicates that the point selection method is a method of selecting an Nth point within a sampling range (for example, refer to the description of the method of selecting an Nth point within a sampling range described above). FIG. 25 When the sampling_select_type is equal to 2, the sampling_select_type indicates that the point selection method is a method of selecting a point whose mean Morton code value is most similar to that of a point belonging to a sampling range (for example, refer to the description of the method of selecting a point whose mean Morton code value is most similar to that of a point belonging to a sampling range described above). FIG. 25 When the sampling_select_type is equal to 3, the sampling_select_type indicates that the point selection method is a method of selecting a point whose ideal mean Morton code value is most similar to that of a sampling range (for example, refer to the description of the method of selecting a point whose ideal mean Morton code value is most similar to that of a sampling range described above). Since the point selection method according to the embodiment is the same as the point selection method described with reference to FIG. 25 When the sampling_select_type is equal to 3, the sampling_select_type indicates that the point selection method is a method of selecting a point whose ideal mean Morton code value is most similar to that of a sampling range (for example, refer to the description of the method of selecting a point whose ideal mean Morton code value is most similar to that of a sampling range described above). Since the point selection method according to the embodiment is the same as the point selection method described with reference to FIG. 25

[0377] The sampling_select_idx indicates a fixed index of a point to be selected. For example, when the sampling_select_type is equal to 1, the sampling_select_idx indicates an index of an Nth point to be selected among points ordered within a sampling range. Also, when the sampling_select_type is equal to 2 and 3, the sampling_select_idx indicates an index of an Nth point to be selected among points ordered based on a mean Morton code value or an ideal mean Morton code value.

[0378] The sampling_select_max_num_of_points indicates the maximum number of points that can be selected within a sampling range.

[0379] The following indicates sampling information when the sampling_select_type is equal to 2. The sampling_begin_depth indicates an octree depth at which an LOD0 set is to be generated. The sampling_begin_depth has a positive integer value.

[0380] ​sampling_isolated_point_threshold indicates a limit of the number of points that can be defined as isolated points.

[0381] sampling_isolated_point_sampling_type indicates an isolated point processing method. The sampling_isolated_point_sampling_type can indicate any one of sampling isolated points, separating and processing isolated points into a candidate group of LODs of a lower level than the current LOD, and registering isolated points in the current LOD. A method of sampling isolated points is to select a point from among k consecutive isolated points, where k is a fixed sampling rate. In a method of separating and processing isolated points into a candidate group of LODs of a lower level than the current LOD, isolated points in a sampling range are separated into LODs (0至1-1) other than being selected, and the selected are processed to generate LOD l-1-α . A method of registering isolated points in the current LOD is to register isolated points in the LOD l-1-α during the generation of isolated points to generate LOD l . Since the isolated point processing method is the same as the isolated point processing method described with reference FIG. 25 to FIG. 6, a detailed description thereof will be omitted.

[0382] When attribute_coding_type is equal to 0, the related parameters are given as follows.

[0383] adaptive_prediction_threshold indicates a threshold of prediction.

[0384] When attribute_coding_type is equal to 1, the related parameters are given as follows.

[0385] raht_depth indicates the number of LODs used for RAHT. The value of depthRAHT is in the range of 1 to xx (arbitrary value).

[0386] raht_quant_step_size indicates a quantization step size of a first component of an attribute. The value of rate_quant_step_size is in the range of 1 to xx (arbitrary value).

[0387] raht_quant_step_size_chroma can indicate a quantization step size of a chroma component of an attribute when RAHT is applied.

[0388] aps_extension_present_flag is a flag having a value of 0 or 1.

[0389] aps_extension_present_flag equal to 1 indicates that the aps_extension_data syntax structure is present in the APS RBSP syntax structure. aps_extension_present_flag equal to 0 indicates that the syntax structure is not present. When the syntax structure is not present, the value of aps_extension_present_flag is inferred to be equal to 0.

[0390] aps_extension_data_flag can have any value. According to embodiments, the presence and value of this field can not affect the decoder performance.

[0391] FIG. 28 An exemplary syntax of the APS according to embodiments is shown.

[0392] FIG. 28 An exemplary syntax of the APS according to embodiments is shown with reference to FIG. 27 An exemplary syntax of the APS according to embodiments is shown with reference to FIG. 28 An exemplary syntax of the APS according to embodiments is shown with reference to FIG. 27 For simplicity, the description of the information and / or parameters that are the same as those described with reference to

[0393] The LOD configurator (e.g., the LOD configurator 2210) according to embodiments can perform sampling differently for each LOD. In this case, when the lod_generation_type is equal to 2, the syntax of the APS according to embodiments can further include the following sampling-related information 2800.

[0394] sampling_attrs_per_lod_flag is a flag indicating whether the sampling method is different between LODs. As described with reference to FIG. 22 The LOD configurator (e.g., the LOD configurator 2210) according to embodiments can perform sampling differently for each LOD. Thus, the LOD generation information according to embodiments can include sampling information for each LOD. Thus, when the sampling_attrs_per_lod_flag is equal to 1, the shown "for" statement is information 2810 related to sampling for each LOD. In the figure, idx denotes each LOD. Since the information 2810 related to sampling according to embodiments is the same as that described with reference to FIG. 27

[0395] ​The LOD configurator (e.g., the LOD configurator 2210) according to the embodiments can apply different LOD generation methods for each tile or slice. As described above, the point cloud data receiving apparatus also needs to generate LODs. Therefore, the bitstream according to the embodiments can also contain signaling information related to the LOD generation method for each region.

[0396] FIG. 29 An exemplary syntax of the TPS according to the embodiments is shown.

[0397] When the LOD configurator according to the embodiments applies different LOD generation methods for each tile, the TPS according to the embodiments can also include signaling information related to the LOD generation method (e.g., the signaling information related to the LOD generation method described above). FIGS. 27-28 FIG. 29 An example of the syntax of the TPS according to the embodiments is shown, and can include the following information (or fields, parameters, etc.).

[0398] num_tiles indicates the number of tiles signaled for the bitstream. When there is no tile signaled for the bitstream, the value of this information is inferred to be 0. The following are the signaling parameters for each tile.

[0399] tile_bounding_box_offset_x[i] indicates the x offset of the i-th tile in the Cartesian coordinate system. When this parameter is not present, the value of tile_bounding_box_offset_x[0] is inferred to be the value of sps_bounding_box_offset_x included in the SPS.

[0400] tile_bounding_box_offset_y[i] indicates the y offset of the i-th tile in the Cartesian coordinate system. When this parameter is not present, the value of tile_bounding_box_offset_y[0] is inferred to be the value of sps_bounding_box_offset_y included in the SPS.

[0401] tile_bounding_box_offset_z[i] indicates the z offset of the i-th tile in the Cartesian coordinate system. When this parameter is not present, the value of tile_bounding_box_offset_z[0] is inferred to be the value of sps_bounding_box_offset_z included in the SPS.

[0402] ​tile_bounding_box_scale_factor[i] indicates a scale factor related to the i-th tile in the Cartesian coordinate system. When this parameter is not present, the value of tile_bounding_box_scale_factor[0] is inferred to be the value of sps_bounding_box_scale_factor.

[0403] tile_bounding_box_size_width[i] indicates the width of the i-th tile in the Cartesian coordinate system. When this parameter is not present, the value of tile_bounding_box_size_width[0] is inferred to be the value of sps_bounding_box_size_width.

[0404] tile_bounding_box_size_height[i] indicates the height of the i-th tile in the Cartesian coordinate system. When this parameter is not present, the value of tile_bounding_box_size_height[0] is inferred to be the value of sps_bounding_box_size_height.

[0405] tile_bounding_box_size_depth[i] indicates the depth of the i-th tile in the Cartesian coordinate system. When this parameter is not present, the value of tile_bounding_box_size_depth[0] is inferred to be the value of sps_bounding_box_size_depth.

[0406] As illustrated, the TPS according to the embodiment can include LOD configuration information 2900. The LOD configuration information 2900 according to the embodiment is applied to each tile. Since the LOD configuration information 2900 is the same as the LOD configuration information 2700 described with reference to FIG. 27 , a detailed description thereof will be omitted.

[0407] FIG. 30 An exemplary syntax of the TPS according to the embodiment is illustrated.

[0408] FIG. 30 An exemplary syntax of the TPS described with reference to FIG. 29 is illustrated. When the sampling information is differently applied for each LOD, an exemplary syntax of the TPS in FIG. 30 is illustrated. For simplicity, a description of information and / or parameters that are the same as those described with reference to FIG. 29 will be omitted.

[0409] The LOD configurator (e.g., the LOD configurator 2210) according to the embodiments can perform sampling differently for each LOD. In this case, when the lod_generation_type is equal to 2, the syntax of the TPS according to the embodiments can further include a sampling_attrs_per_lod_flag 3000 related to sampling. Since the sampling_attrs_per_lod_flag according to the embodiments is the same as the sampling_attrs_per_lod_flag 2800 described with reference to FIG. 28 to be omitted. When the sampling_attrs_per_lod_flag is equal to 1, the syntax of the TPS according to the embodiments includes a "for" statement indicating sampling-related information 3010 for each LOD. Since the sampling-related information 3010 for each LOD according to the embodiments is the same as the sampling-related information 2810 described with reference to FIG. 28 to be omitted.

[0410] FIG. 31 An exemplary syntax of the attribute header according to the embodiments is shown.

[0411] FIG. 31 The syntax of the attribute header is an example of the syntax of information transmitted by referring to FIG. 26 the header included in the attribute bitstream described with reference to

[0412] When the LOD configurator (e.g., the LOD configurator 2110) according to the embodiments applies a neighbor point set generation method differently for each slice, the attribute header according to the embodiments can further include LOD configuration information 3100 (e.g., refer to the LOD configuration information 2700 described with reference to FIG. 27 and the LOD configuration information 2900 described with reference to FIG. 29 ). FIG. 31 The exemplary syntax of the attribute header shown can include the following information (or fields, parameters, etc.).

[0413] The ash_attr_parameter_set_id has the same value as the aps_attr_parameter_set_id of the active APS.

[0414] The ash_attr_sps_attr_idx can specify the value of the sps_seq_parameter_set_id of the active SPS. The value of the ash_attr_sps_attr_idx is in the range of 0 to the value of the sps_num_attribute_sets included in the active SPS.

[0415] ash_attr_geom_slice_id specifies a value of a geometry slice ID (e.g., geom_slice_id).

[0416] As illustrated, the attribute header according to the embodiment can include LOD configuration information 3100. The LOD configuration information 3100 according to the embodiment is applied to each attribute bitstream (or attribute slice data) belonging to each slice. Since the LOD configuration information 3100 is the same as the LOD configuration information 2700 and 2900 described with reference to FIG. 27 and FIG. 29 The detailed description thereof will be omitted since the LOD configuration information 2700 and 2900 described is the same as the LOD configuration information 3100 described.

[0417] FIG. 32 An exemplary syntax of the attribute header according to the embodiment is illustrated.

[0418] FIG. 32 An exemplary syntax of the attribute header described with reference to FIG. 31 is illustrated. When the sampling information is differently applied for each LOD, the exemplary syntax of the attribute header in FIG. 32 is illustrated.

[0419] The LOD configurator (e.g., the LOD configurator 2210) according to the embodiment can perform sampling differently for each LOD. In this case, when the lod_generation_type is equal to 2, the syntax of the attribute header according to the embodiment can further include sampling_attrs_per_lod_flag 3200 related to sampling. Since the sampling_attrs_per_lod_flag according to the embodiment is the same as the sampling_attrs_per_lod_flag 2800 and 3000 described with reference to FIG. 28 and FIG. 30 The detailed description thereof will be omitted since the sampling_attrs_per_lod_flag 2800 and 3010 described with reference to FIG. 28 and FIG. 30 is the same as the sampling_attrs_per_lod_flag 3200 described. When the sampling_attrs_per_lod_flag is equal to 1, the syntax of the attribute header according to the embodiment includes a "for" statement indicating sampling-related information 3210 of each LOD. Since the sampling-related information 3210 of each LOD according to the embodiment is the same as the sampling-related information 2810 and 3010 described with reference to

[0420] FIG. 33 is a block diagram illustrating an exemplary point cloud decoder.

[0421] The point cloud decoder 3300 according to the embodiment performs the same functions as the decoder described with reference to FIGS. 1-17 FIG. 1 ,​FIGS. 10-11 、 FIG. 13 、 FIG. 14 and FIG. 16 the same or similar decoding operations as the decoding operations of the point cloud decoder described with reference to FIGS. 11 to 13. In addition, the point cloud decoder 3300 can perform decoding operations corresponding to inverse processes of the encoding operations of the point cloud encoder 1800 described with reference to FIGS. 14 to 16. The point cloud decoder 3300 according to an embodiment can include a spatial partitioner 3310, a geometry information decoder (or geometry decoder) 3320, and an attribute information decoder (or attribute decoder) 3330. Although not shown in FIG. 17, the point cloud decoder 3300 according to an embodiment can further include one or more elements to perform the decoding operations described with reference to FIGS. 11 to 13. FIG. 18 FIG. 33 FIGS. 1-17

[0422] The spatial partitioner 3310 according to an embodiment can partition a space based on signaling information (e.g., information about a partitioning operation performed by the spatial partitioner 1810 described with reference to FIGS. 14 to 16) received from a point cloud data transmitting apparatus (e.g., the point cloud data transmitting apparatus described with reference to FIGS. 1 to 3) according to an embodiment or partitioning information derived (generated) by the point cloud decoder 3300. As described above, the partitioning operation of the spatial partitioner 1810 of the point cloud encoder 1800 can be based on at least one of an octree, a quadtree, a binary tree, a ternary tree, or a k-d tree. FIG. 1 FIG. 12 FIG. 14 FIG. 15 The geometry information decoder 3320 according to an embodiment can decode an input geometry bitstream and reconstruct geometry information. The reconstructed geometry information can be input to the attribute information decoder. The geometry information decoder 3320 according to an embodiment can perform operations of the arithmetic decoder (arithmetic decoding) 11000, the octree synthesizer (synthesis octree) 11001, and the surface approximation synthesizer (synthesis surface approximation) 11002, the geometry reconstructor (reconstruct geometry) 11003, and the coordinate inverse transformer (inverse transform coordinate) 11004 described with reference to FIGS. 11 to 13. In addition, the geometry information decoder 3320 according to an embodiment can perform operations of the arithmetic decoder 13002, the occupancy code-based octree reconstruction processor 13003, and the surface model processor (triangle reconstruction, upsampling, voxelization) 13004 and the inverse quantization processor 13005 described with reference to FIGS. 14 to 16. Alternatively, the geometry information decoder 3320 according to an embodiment can perform operations of the point cloud decoder described with reference to FIGS. 17 to 19. FIG. 18

[0423] FIG. 11 FIG. 13 FIG. 16

[0424] ​​​​​​​​​​​The attribute information decoder 3330 according to the embodiments can decode attribute information based on the attribute information bitstream and the reconstructed geometry information. The attribute information decoder 3330 according to the embodiments can perform the same or similar operations as those of the arithmetic decoder 11005, the inverse quantizer 11006, the RAHT transformer 11007, the LOD generator 11008, the inverse elevator 11009, and / or the color inverse transformer 11010 included in the point cloud decoder of FIG. 11. FIG. 11 The attribute information decoder 3330 according to the embodiments can perform the same or similar operations as those of the arithmetic decoder 13007, the inverse quantization processor 13008, the prediction / elevation / RAHT inverse transform processor 13009, and the color inverse transform processor 13010 included in the reception device of FIG. 13. FIG. 13 The attribute information decoder 3330 according to the embodiments can perform the same or similar operations as those of the arithmetic decoder 13007, the inverse quantization processor 13008, the prediction / elevation / RAHT inverse transform processor 13009, and the color inverse transform processor 13010 included in the reception device of FIG. 13.

[0425] The point cloud decoder 3300 can output final PCC data based on the reconstructed geometry information and the reconstructed attribute information.

[0426] FIG. 34 is a block diagram illustrating an example of a geometry information decoder.

[0427] The geometry information decoder 3400 according to the embodiments is an example of the geometry information decoder 3320 of FIG. 13, and can perform the same or similar operations as those of the geometry information decoder 3320. The geometry information decoder 3400 according to the embodiments can perform decoding operations opposite to the encoding operations of the geometry information encoder 1900 described with reference to FIG. 19. The geometry information decoder 3400 according to the embodiments can include a geometry information entropy decoder 3410, a geometry information inverse quantizer 3420, a geometry information predictor 3430, a filter 3440, a memory 3450, a geometry information inverse transform / inverse quantizer 3460, and a coordinate inverse transformer 3470. Although not shown in FIG. 14, the geometry information decoder 3400 according to the embodiments can further include one or more elements to perform the geometry decoding operations described with reference to FIG. 19. FIG. 33 The geometry information decoder 3400 according to the embodiments is an example of the geometry information decoder 3320 of FIG. 13, and can perform the same or similar operations as those of the geometry information decoder 3320. The geometry information decoder 3400 according to the embodiments can perform decoding operations opposite to the encoding operations of the geometry information encoder 1900 described with reference to FIG. 19. The geometry information decoder 3400 according to the embodiments can include a geometry information entropy decoder 3410, a geometry information inverse quantizer 3420, a geometry information predictor 3430, a filter 3440, a memory 3450, a geometry information inverse transform / inverse quantizer 3460, and a coordinate inverse transformer 3470. Although not shown in FIG. 14, the geometry information decoder 3400 according to the embodiments can further include one or more elements to perform the geometry decoding operations described with reference to FIG. 19. FIG. 19 The geometry information decoder 3400 according to the embodiments is an example of the geometry information decoder 3320 of FIG. 13, and can perform the same or similar operations as those of the geometry information decoder 3320. The geometry information decoder 3400 according to the embodiments can perform decoding operations opposite to the encoding operations of the geometry information encoder 1900 described with reference to FIG. 19. The geometry information decoder 3400 according to the embodiments can include a geometry information entropy decoder 3410, a geometry information inverse quantizer 3420, a geometry information predictor 3430, a filter 3440, a memory 3450, a geometry information inverse transform / inverse quantizer 3460, and a coordinate inverse transformer 3470. Although not shown in FIG. 14, the geometry information decoder 3400 according to the embodiments can further include one or more elements to perform the geometry decoding operations described with reference to FIG. 19. FIG. 34 The geometry information decoder 3400 according to the embodiments is an example of the geometry information decoder 3320 of FIG. 13, and can perform the same or similar operations as those of the geometry information decoder 3320. The geometry information decoder 3400 according to the embodiments can perform decoding operations opposite to the encoding operations of the geometry information encoder 1900 described with reference to FIG. 19. The geometry information decoder 3400 according to the embodiments can include a geometry information entropy decoder 3410, a geometry information inverse quantizer 3420, a geometry information predictor 3430, a filter 3440, a memory 3450, a geometry information inverse transform / inverse quantizer 3460, and a coordinate inverse transformer 3470. Although not shown in FIG. 14, the geometry information decoder 3400 according to the embodiments can further include one or more elements to perform the geometry decoding operations described with reference to FIG. 19. FIGS. 1-31 The geometry information decoder 3400 according to the embodiments is an example of the geometry information decoder 3320 of FIG. 13, and can perform the same or similar operations as those of the geometry information decoder 3320. The geometry information decoder 3400 according to the embodiments can perform decoding operations opposite to the encoding operations of the geometry information encoder 1900 described with reference to FIG. 19. The geometry information decoder 3400 according to the embodiments can include a geometry information entropy decoder 3410, a geometry information inverse quantizer 3420, a geometry information predictor 3430, a filter 3440, a memory 3450, a geometry information inverse transform / inverse quantizer 3460, and a coordinate inverse transformer 3470. Although not shown in FIG. 14, the geometry information decoder 3400 according to the embodiments can further include one or more elements to perform the geometry decoding operations described with reference to FIG. 19.

[0428] The geometry information entropy decoder 3410 according to the embodiments can generate quantized residual geometry information by entropy-decoding a geometry information bitstream. The geometry information entropy decoder 3410 can perform the same or similar operations as those of the entropy decoder 19005 of FIG. 19. FIG. 19The described geometry information entropy decoder 1905 performs an entropy decoding operation that is the inverse of the entropy encoding performed by the geometry information entropy encoder 1905. As described above, the entropy encoding operation according to embodiments can include exponential Golomb, CAVLC, and CABAC, and the entropy decoding operation corresponding to the entropy encoding operation exponential Golomb, CAVLC, and CABAC. The geometry information entropy decoder 3410 according to embodiments can decode information related to geometry encoding included in the geometry information bitstream, for example, information related to generation of prediction geometry information, information related to quantization (e.g., quantization values, etc.), signaling information related to coordinate transformation, etc.

[0429] The residual geometry information inverse quantizer 3420 according to embodiments can generate residual geometry information or geometry information by performing an inverse quantization operation on quantized residual geometry information based on quantization-related information.

[0430] The geometry information predictor 3430 according to embodiments can generate prediction geometry information based on information related to generation of prediction geometry information output from the geometry entropy decoder 3410 and previously decoded geometry information stored in the memory 3450. The geometry information predictor 3430 according to embodiments can include an inter-predictioner and an intra-predictioner. The inter-predictioner according to embodiments can perform inter-prediction on a current prediction unit (e.g., a node, etc.) based on information required for inter-prediction of the current prediction unit provided by the geometry information encoder (e.g., the geometry information encoder 1900) in a current space including the current prediction unit based on information included in at least one of a space before or after the current space (e.g., a frame, a picture, etc.). The intra-predictioner according to embodiments can generate prediction geometry information based on geometry information about a point in a current space based on information related to intra-prediction in a prediction unit provided by the geometry information encoder 1900.

[0431] The filter 3440 according to embodiments can filter reconstructed geometry information generated by combining prediction geometry information generated based on filtering-related information and reconstructed residual geometry information. The filtering-related information according to embodiments can be signaled from the geometry information encoder 1900. Alternatively, the geometry information decoder 3400 according to embodiments can derive and calculate the filtering-related information in a decoding process.

[0432] The memory 3450 according to embodiments can store reconstructed geometry information. The geometry inverse transform quantizer 3460 according to embodiments can inverse transform quantize reconstructed geometry information stored in the memory 3450 based on quantization-related information.

[0433] The coordinate system inverse transformer 3470 according to the embodiments can inverse-transform the coordinates of inverse-transformed and quantized geometry information based on the coordinate transformation-related information provided by the geometry information entropy decoder 3410 and the reconstructed geometry information stored in the memory 3450, and output the geometry information.

[0434] FIG. 35 is a block diagram illustrating an exemplary attribute information decoder.

[0435] The attribute information decoder 3500 according to the embodiments is an example of the attribute information decoder 3330 of FIG. 33 and can perform the same or similar operations as those of the attribute information decoder 3330. The attribute information decoder 3500 according to the embodiments can perform a decoding operation opposite to the encoding operation of the attribute information encoder 2000 and the attribute information encoder 2100 described with reference to FIG. 20 and FIG. 21 The attribute information decoder 3500 according to the embodiments can include an attribute information entropy decoder 3510, a geometry information mapper 3520, a residual attribute information inverse quantizer 3530, a residual attribute information inverse transformer 3540, a memory 3550, an attribute information predictor 3560, and an attribute information inverse transformer 3570. Although not shown in FIG. 35 , the attribute information decoder 3500 according to the embodiments can further include one or more elements to perform the attribute decoding operation described with reference to FIGS. 1-31 .

[0436] The attribute information entropy decoder 3510 receives an attribute information bitstream and performs entropy decoding to generate transform-quantized attribute information.

[0437] The geometry information mapper 3520 generates residual attribute information by mapping the transform-quantized attribute information and the reconstructed geometry information.

[0438] The residual attribute information inverse quantizer 3530 can inverse-quantize the residual attribute information based on a quantization value.

[0439] The residual attribute information inverse transformer 3540 can inverse-transform the residual 3D block including the attribute information obtained by inverse quantization by performing transform coding such as DCT, DST, SADCT, or RAHT.

[0440] The memory 3550 can add the predicted attribute information output from the attribute information predictor 3560 and store the inverse-transformed attribute information. Alternatively, the memory 3550 can add and store the predicted attribute information and the attribute information without inverse-transforming the attribute information.

[0441] The attribute information predictor 3560 can generate predicted attribute information based on the attribute information stored in the memory 3550. The attribute information predictor 3560 can generate the predicted attribute information by performing entropy decoding.

[0442] The attribute information inverse transformer 3570 can receive information on a type of attribute information and transform information from the attribute information entropy decoder 3510 to perform various types of color inverse transform coding.

[0443] FIG. 36 is an exemplary flowchart of a point cloud data processing method according to an embodiment.

[0444] FIG. 36 The flowchart 3600 of FIG. 36 illustrates a point cloud data processing method of a point cloud data processing apparatus (e.g., the point cloud data transmitting apparatus or the point cloud data encoder described with reference to FIG. 1 , FIG. 11 , FIG. 14 , FIG. 15 and FIGS. 18-22 The point cloud data processing apparatus according to an embodiment can perform the same or similar operations as the encoding operations described with reference to FIGS. 1-35 .

[0445] The point cloud data processing apparatus according to an embodiment can encode point cloud data including geometry information and attribute information (3310). The geometry information according to an embodiment is information indicating positions of points of the point cloud data. The attribute information according to an embodiment is information indicating attributes of points of the point cloud data.

[0446] The point cloud data processing apparatus according to an embodiment can encode the geometry information and encode the attribute information. The point cloud data processing apparatus according to an embodiment performs the same or similar operations as the geometry information encoding operations described with reference to FIGS. 1-35 . In addition, the point cloud data processing apparatus performs the same or similar operations as the attribute information encoding operations described with reference to FIGS. 1-35 . The point cloud data processing apparatus according to an embodiment can generate at least one LOD by partitioning points. Since the LOD generation scheme or method according to an embodiment is the same or similar to the LOD generation scheme or method described with reference to FIGS. 22-25 , a detailed description thereof will be omitted.

[0447] The point cloud data processing apparatus according to an embodiment can transmit a bitstream containing the encoded point cloud data (3620).

[0448] Since the structure of the bitstream according to an embodiment is the same as described with reference to FIG. 26 , a detailed description thereof will be omitted. The bitstream according to an embodiment can include LOD configuration information (e.g., the LOD configuration information described with reference to FIGS. 27-32The LOD configuration information described is as follows. FIGS. 27-32 The data is sent to the receiving device via APS, TPS, attribute headers, etc.

[0449] The LOD configuration information according to the implementation method may include type information indicating the type of LOD generation method (e.g., referring to...). FIGS. 27-32 The described lod_generation_type). The type information, depending on the implementation, can indicate a first type of LOD generation based on the distance between points (e.g., see reference). FIG. 22 The described distance-based LOD generation method and a second type of LOD generation method that generates LODs through point-based Morton code sampling (e.g., see reference). FIG. 22 At least one of the described sampling LOD generation methods based on the Morton order. Because the LOD set generation method and the reference... FIG. 22 Since the descriptions are the same, their detailed descriptions will be omitted.

[0450] When the type information of the implementation method indicates a second type, the LOD configuration information may include sampling range type information (e.g., referring to...). FIGS. 27-32 The description of sampling_range_type), sampling rate information (e.g., refer to...) FIGS. 27-32 The description of the sampling rate), and fixed index information about points that can be selected based on sampling (e.g., refer to...). FIGS. 27-32 The description of sampling_select_idx) and the point selection method information for selecting points within the sampling range (e.g., refer to...) FIGS. 27-32 The description of sampling_select_type and information about the maximum number of points that can be selected based on sampling (e.g., see reference). FIGS. 27-32 The description of sampling_select_max_num_of_points). Due to the LOD configuration information according to the implementation method and reference... FIGS. 27-32 Since the descriptions are the same, their detailed descriptions will be omitted.

[0451] FIG. 37 This is an exemplary flowchart of a point cloud data processing method according to an implementation method.

[0452] FIG. 37 Flowchart 3700 shows a point cloud data processing apparatus (e.g., refer to...). FIG. 1 , FIG. 13 , FIG. 14 , FIG. 16 , FIG. 27 and FIG. 28A point cloud data processing method of the described point cloud data receiving apparatus or point cloud data decoder. The point cloud data processing apparatus according to the embodiments can perform the same or similar operations as the decoding operations described with reference to FIGS. 1-35 the same or similar operations as the decoding operations described with reference to

[0453] The point cloud data processing apparatus according to the embodiments receives a bitstream (3710) including point cloud data.

[0454] The point cloud data processing apparatus according to the embodiments decodes the point cloud data (3320). The decoded point cloud data according to the embodiments includes geometry information and attribute information. The geometry information refers to information indicating positions of points of the point cloud data. The attribute information according to the embodiments refers to information indicating attributes of points of the point cloud data. Since the structure of the bitstream according to the embodiments is the same as that described with reference to FIG. 26 , a detailed description thereof will be omitted.

[0455] The point cloud data processing apparatus according to the embodiments can decode the geometry information and decode the attribute information. The point cloud data processing apparatus according to the embodiments performs the same or similar operations as the geometry information decoding operations described with reference to FIGS. 1-35 . In addition, the point cloud data processing apparatus performs the same or similar operations as the attribute information decoding operations described with reference to FIGS. 1-35 . The point cloud data processing apparatus according to the embodiments can generate at least one LOD by splitting points. Since the LOD generation scheme or method according to the embodiments is the same as that described with reference to FIGS. 22-32 , a detailed description thereof will be omitted.

[0456] Since the structure of the bitstream according to the embodiments is the same as that described with reference to FIG. 26 , a detailed description thereof will be omitted. The bitstream according to the embodiments can include LOD configuration information (e.g., the LOD configuration information described with reference to FIGS. 27-32 ). The neighbor point set generation information according to the embodiments can be transmitted to the receiving apparatus through an APS, a TPS, an attribute header, etc. as described with reference to FIGS. 27-32 .

[0457] The LOD configuration information according to the embodiments can include type information (e.g., the lod_generation_type described with reference to FIGS. 27-32 ) indicating a type of an LOD generation method. The type information according to the embodiments can indicate at least one of a first type (e.g., the distance-based LOD generation method described with reference to FIG. 22 ) of generating an LOD based on distances between points and a second type (e.g., the Morton code-based sampling of points for LOD generation described with reference to FIG. 22The LOD set generation method based on the Morton order described above). Since the LOD set generation method is the same as that described with reference to FIG. 22 above, a detailed description thereof will be omitted.

[0458] When the type information according to the embodiment indicates the second type, the LOD configuration information can include sampling range type information (e.g., with reference to FIGS. 27-32 above), sampling rate information (e.g., with reference to FIGS. 27-32 above), fixed index information about points that can be selected according to sampling (e.g., with reference to FIGS. 27-32 above), point selection method information for selecting points in a sampling range (e.g., with reference to FIGS. 27-32 above), and information about the maximum number of points that can be selected according to sampling (e.g., with reference to FIGS. 27-32 above). Since the LOD configuration information according to the embodiment is the same as that described with reference to FIGS. 1-37 above, a detailed description thereof will be omitted.

[0459] The components of the point cloud data processing apparatus according to the embodiment described with reference to FIGS. 1-37 above can be implemented as hardware, software, firmware, or a combination thereof, including one or more processors coupled with a memory. The components of the apparatus according to the embodiment can be implemented as a single chip, such as a single hardware circuit. Alternatively, the components of the point cloud data processing apparatus according to the embodiment can be implemented as separate chips. In addition, at least one component of the point cloud data processing apparatus according to the embodiment can include one or more processors capable of executing one or more programs, wherein the one or more programs can include instructions to perform or be configured to perform one or more operations / methods of the point cloud data processing apparatus described with reference to ​ above.

[0460] Although the drawings are described separately for simplicity, new embodiments can be designed by combining the embodiments shown in the respective drawings. Designing a recording medium on which a program for executing the above-described embodiments is recorded, which can be read by a computer, also falls within the scope of the appended claims and equivalents thereof, as needed by those skilled in the art. The apparatus and method according to the embodiments can not be limited to the configurations and methods of the above-described embodiments. Various modifications can be made to the embodiments by selectively combining all or some of the embodiments. Although the preferred embodiments are described with reference to the drawings, it will be understood by those skilled in the art that various modifications and changes can be made to the embodiments without departing from the spirit or scope of the disclosure described in the appended claims. Such modifications should not be construed as being separate from the technical idea or concept of the embodiments.

[0461] The descriptions of the methods and apparatuses can be applied to supplement each other. For example, the point cloud data transmission method according to the embodiments can be performed by the point cloud data transmission apparatus according to the embodiments or components included in the point cloud data transmission apparatus. In addition, the point cloud data reception method according to the embodiments can be performed by the point cloud data reception apparatus according to the embodiments or components included in the point cloud data reception apparatus.

[0462] The various elements of the apparatus of the embodiments can be implemented by hardware, software, firmware, or a combination thereof. The various elements in the embodiments can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiments, the components according to the embodiments can be implemented as separate chips, respectively. According to the embodiments, at least one or more components of the apparatus according to the embodiments can include one or more processors capable of executing one or more programs. The one or more programs can perform any one or more operations / methods according to the embodiments or include instructions for performing the same. Executable instructions for performing the methods / operations of the apparatus according to the embodiments can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transitory CRM or other computer program product configured to be executed by one or more processors. In addition, the memory according to the embodiments can be used as a concept encompassing not only a volatile memory (e.g., a RAM), but also a non-volatile memory, a flash memory, and a PROM. In addition, it can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). In addition, the processor-readable recording medium can be distributed to computer systems connected via a network, so that the processor-readable code can be stored and executed in a distributed manner.

[0463] In the present specification, the terms “ / ” and “,” should be interpreted as “and / or”. For example, the expression “A / B” can mean “A and / or B”. Also, “A, B” can mean “A and / or B”. Also, “A / B / C” can mean “at least one of A, B, and / or C”. Also, “A / B / C” can mean “at least one of A, B, and / or C”. Also, in the present specification, the term “or” should be interpreted as “and / or”. For example, the expression “A or B” can mean 1) only A, 2) only B, or 3) both A and B. In other words, the term “or” used in the present document should be interpreted as “additionally or alternatively”.

[0464] Terms such as first and second can be used to describe various elements of the embodiments. However, the various components according to the embodiments should not be limited by the above terms. The terms are used only to distinguish one element from another element. For example, a first user input signal can be referred to as a second user input signal. Similarly, a second user input signal can be referred to as a first user input signal. The use of these terms should not be interpreted as departing from the scope of the various embodiments. The first user input signal and the second user input signal are both user input signals, but do not mean the same user input signal unless the context clearly specifies otherwise.

[0465] The terms used to describe the embodiments are used to describe the specific embodiments, and are not intended to limit the embodiments. As used in the description of the embodiments and the claims, the singular form includes the plural unless the context clearly specifies otherwise. The expression “and / or” is used to include all possible combinations of the terms. The terms such as “include” or “have” are intended to indicate the presence of the graphics, numbers, steps, elements, and / or components, and it should be understood that the possibility of the presence of other graphics, numbers, steps, elements, and / or components is not excluded. As used herein, conditional expressions such as “if” and “when” are not limited to optional cases, and are intended to be interpreted as performing the related operation when a certain condition is met, or interpreting the related definition according to the certain condition.

[0466] Disclosed modes

[0467] As described above, the relevant content is described in the best mode of implementing the embodiments.

[0468] Industrial applicability

[0469] It will be apparent to those skilled in the art that various modifications and changes can be made to the embodiments within the scope of the embodiments. Therefore, the embodiments are intended to cover the modifications and changes of the present disclosure, as long as they fall within the scope of the appended claims and their equivalents.

Claims

1. A method for encoding point cloud data, the method comprising the following steps: Encoding the point cloud data, which includes geometric information and attribute information, wherein the geometric information represents the position of points in the point cloud data, and the attribute information represents the attributes of the points in the point cloud data, wherein the step of encoding the point cloud data includes the following steps: The geometric information is encoded using an octree; and The attribute information is encoded based on the Level of Detail (LOD) generated through the sampling process. The sampling process is applied based on the nodes of the octree, and the point closest to the centroid of the node is selected based on the Morton order associated with the points in the node; and Sending a bitstream comprising encoded point cloud data, wherein the bitstream includes attribute parameter set information, the attribute parameter set information including: First information indicating the type associated with the LOD, second information indicating the sampling type of the LOD, and third information related to the number of points in the sampling range of the octree. Specifically, based on a first value of the sampling type, points are selected from the points within the sampling range of the octree, and the points within the sampling range of the octree are sorted based on Morton codes. The selected point has a specific index within the sampling range.

2. The method according to claim 1, wherein, The LOD is generated based on the Morton order of the points or the distance between the points.

3. A method for decoding point cloud data, the method comprising the following steps: Receive a bitstream comprising point cloud data, wherein the point cloud data includes geometric information and attribute information, wherein the geometric information represents the position of points in the point cloud data, and the attribute information represents the attributes of the points in the point cloud data; and Decoding the point cloud data includes the following steps: Decoding the geometric information based on an octree; and The attribute information is decoded based on the Level of Detail (LOD) generated through the sampling process. The sampling process is applied based on the nodes of the octree, and the point closest to the centroid of the node is selected based on the Morton order associated with the points in the node. The bitstream includes attribute parameter set information, which includes: First information indicating the type associated with the LOD, second information indicating the sampling type of the LOD, and third information related to the number of points in the sampling range of the octree. Specifically, based on a first value of the sampling type, points are selected from the points within the sampling range of the octree, and the points within the sampling range of the octree are sorted based on Morton codes. The selected point has a specific index within the sampling range.

4. The method according to claim 3, wherein, The LOD is generated based on the Morton order of the points or the distance between the points.

5. An apparatus for decoding point cloud data, the apparatus comprising: A receiver configured to receive a bitstream comprising point cloud data, wherein the point cloud data includes geometric information and attribute information, wherein the geometric information represents the positions of points in the point cloud data, and the attribute information represents the attributes of the points in the point cloud data; and The decoder is configured as follows: Decoding the geometric information based on an octree; and The attribute information is decoded based on the Level of Detail (LOD) generated through the sampling process. The sampling process is applied based on the nodes of the octree, and the point closest to the centroid of the node is selected based on the Morton order associated with the points in the node. The bitstream includes attribute parameter set information, which includes: First information indicating the type associated with the LOD, second information indicating the sampling type of the LOD, and third information related to the number of points in the sampling range of the octree. Specifically, based on a first value of the sampling type, points are selected from the points within the sampling range of the octree, and the points within the sampling range of the octree are sorted based on Morton codes. The selected point has a specific index within the sampling range.

6. The apparatus according to claim 5, wherein, The LOD is generated based on the Morton order of the points or the distance between the points.

Images