Method and device for coding and decoding point cloud data

CN118945395APending Publication Date: 2024-11-12LG ELECTRONICS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410943970.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2020-01-15
Filing Date
2020-11-02
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The prior art is difficult to efficiently process and transmit large amounts of point cloud data, resulting in high latency and encoding/decoding complexity, affecting the performance of services such as virtual reality, augmented reality and self-driving.

Method used

Using the encoding and decoding method of point cloud data, the combination of geometric encoding and attribute encoding can generate a compressible bit stream, and effectively process and reconstruct the data during the transmission and reception process, using the octree structure and attribute detail level (LOD) performs spatial scalability encoding and decoding.

Benefits of technology

It realizes efficient point cloud data processing and transmission, provides high-quality point cloud services, and is suitable for applications such as virtual reality and self-driving, reducing computing volume and bandwidth requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945395A_ABST
    Figure CN118945395A_ABST
Patent Text Reader

Abstract

The invention discloses a method and a device for encoding and decoding point cloud data. A point cloud data transmission method according to an embodiment may comprise the steps of: encoding point cloud data; and transmitting the bit stream containing the point cloud data. A point cloud data receiving method according to an embodiment may comprise the steps of: receiving a bitstream containing point cloud data; and decoding the point cloud data.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the invention patent application with the original application number 202080091844.5 (International application number: PCT / KR2020 / 015138, application date: November 2, 2020, invention name: Point cloud data sending device, point cloud data sending method, point cloud data receiving device and point cloud data receiving method). Technical Field

[0002] Embodiments relate to methods and apparatus for processing point cloud content. Background Art

[0003] Point cloud content is content represented by a point cloud, which is a set of points belonging to a coordinate system representing a three-dimensional space. Point cloud content can represent media configured in three dimensions and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and self-driving services. However, tens of thousands to hundreds of thousands of point data are required to represent point cloud content. Therefore, methods for efficiently processing large amounts of point data are needed. Summary of the invention

[0004] Technical issues

[0005] The embodiment provides an apparatus and method for efficiently processing point cloud data. The embodiment provides a point cloud data processing method and apparatus for solving delay and encoding / decoding complexity.

[0006] The technical scope of the embodiments is not limited to the above-mentioned technical purposes, and can be extended to other technical purposes that can be inferred by those skilled in the art based on the entire contents disclosed herein.

[0007] Technical Solution

[0008] To achieve these objects and other advantages, and in accordance with the purposes of the present disclosure, as embodied and broadly described herein, a method for transmitting point cloud data may include: encoding the point cloud data; and transmitting a bitstream containing the point cloud data. In another aspect of the present disclosure, a method for receiving point cloud data may include: receiving a bitstream containing the point cloud data; and decoding the point cloud data.

[0009] Beneficial effects

[0010] The apparatus and method according to the embodiments can efficiently process point cloud data.

[0011] The apparatus and method according to the embodiments can provide high-quality point cloud services.

[0012] The device and method according to the embodiment may provide point cloud content for providing general services such as VR service and self-driving service. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of this application. The accompanying drawings illustrate embodiments of the present disclosure and together with the description are used to illustrate the principles of the present disclosure. In order to better understand the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the accompanying drawings. The same reference numerals will be used throughout the drawings to represent the same or similar parts. In the drawings:

[0014] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown.

[0015] Figure 2 is a block diagram illustrating point cloud content providing operations according to an embodiment.

[0016] Figure 3 An exemplary process for capturing point cloud video according to an implementation is shown.

[0017] Figure 4 An exemplary point cloud encoder according to an implementation is shown.

[0018] Figure 5 An example of a voxel according to an embodiment is shown.

[0019] Figure 6 An example of an octree and occupancy code according to an embodiment is shown.

[0020] Figure 7 An example of a neighbor node pattern according to an embodiment is shown.

[0021] Figure 8 An example of point configuration in each LOD according to an embodiment is shown.

[0022] Figure 9 An example of point configuration in each LOD according to an embodiment is shown.

[0023] Figure 10 A point cloud decoder according to an embodiment is shown.

[0024] Figure 11 A point cloud decoder according to an embodiment is shown.

[0025] Figure 12 A transmitting device according to an embodiment is shown.

[0026] Figure 13 A receiving device according to an embodiment is shown.

[0027] Figure 14 An exemplary structure that can be operated in conjunction with a point cloud data transmitting / receiving method / apparatus according to an embodiment is shown.

[0028] Figure 15 shows a geometry octree and attribute level of detail (LOD) generated based on the octree according to an embodiment;

[0029] Figure 16 A method for processing a bitstream based on LOD according to an embodiment is illustrated;

[0030] Figure 17 shows spatial scalability structures of geometry and attributes according to embodiments;

[0031] Figure 18 The operation of geometric encoding according to an embodiment is illustrated;

[0032] Figure 19 illustrates floor quantization and round quantization according to an embodiment;

[0033] Figure 20 An operation of reconstructing the geometry (position) of an intermediate node of an octree according to an embodiment is illustrated;

[0034] Figure 21 illustrates the operation of partial geometry decoding according to an embodiment;

[0035] Figure 22 The structure of a point cloud data encoder according to an embodiment is shown;

[0036] Figure 23 The structure of a point cloud data decoder according to an embodiment is shown;

[0037] Figure 24 shows the structure of a bitstream associated with point cloud data according to an embodiment;

[0038] Figure 25 shows a sequence parameter set (SPS) according to an embodiment;

[0039] Figure 26 shows a set of geometric parameters (GPS) according to an embodiment;

[0040] Figure 27 shows a tile parameter set (TPS) according to an embodiment;

[0041] Figure 28 shows a geometry slice header (GSH) according to an embodiment;

[0042] Figure 29 shows a geometry node according to an embodiment;

[0043] Figure 30 shows an SPS according to an embodiment;

[0044] Figure 31 shows a GPS according to an embodiment;

[0045] Figure 32 shows a TPS according to an embodiment;

[0046] Figure 33 shows GSH according to an embodiment;

[0047] Figure 34 A method for transmitting point cloud data according to an embodiment is illustrated; and

[0048] Figure 35 A method for receiving point cloud data according to an embodiment is illustrated. DETAILED DESCRIPTION

[0049] Reference will now be made in detail to preferred embodiments of the present disclosure, examples of which are shown in the accompanying drawings. The detailed description given below with reference to the accompanying drawings is intended to illustrate exemplary embodiments of the present disclosure, rather than to illustrate the only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0050] Although most of the terms used in the present disclosure are selected from common terms widely used in the art, some terms are arbitrarily selected by the applicant and their meanings are explained in detail in the following description as needed. Therefore, the present disclosure should be understood based on the intended meanings of the terms rather than their simple names or meanings.

[0051] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown.

[0052] Figure 1 The point cloud content providing system shown may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 can transmit and receive point cloud data via wired or wireless communication.

[0053] According to an embodiment, the point cloud data sending device 10000 can obtain and process point cloud video (or point cloud content) and send it. According to an embodiment, the sending device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device and / or a server. According to an embodiment, the sending device 10000 may include a device configured to communicate with a base station and / or other wireless devices using a radio access technology (e.g., 5G new RAT (NR), long term evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server.

[0054] According to the implementation manner, the sending device 10000 includes a point cloud video acquirer 10001, a point cloud video encoder 10002 and / or a transmitter (or communication module) 10003.

[0055] The point cloud video acquirer 10001 according to the embodiment acquires the point cloud video through a processing process such as capture, synthesis or generation. The point cloud video is a point cloud content represented by a point cloud, which is a set of points located in a 3D space and can be referred to as point cloud video data. The point cloud video according to the embodiment can include one or more frames. A frame represents a still image / picture. Therefore, the point cloud video can include a point cloud image / frame / picture and can be referred to as a point cloud image, frame or picture.

[0056] The point cloud video encoder 10002 according to the embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 may encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to the embodiment may include geometry-based point cloud compression (G-PCC) coding and / or video-based point cloud compression (V-PCC) coding or next generation coding. The point cloud compression coding according to the embodiment is not limited to the above-mentioned embodiment. The point cloud video encoder 10002 may output a bit stream containing the encoded point cloud video data. The bit stream may include not only the encoded point cloud video data, but also signaling information related to the encoding of the point cloud video data.

[0057] The transmitter 10003 according to the embodiment transmits a bitstream containing encoded point cloud video data. The bitstream according to the embodiment is encapsulated in a file or a fragment (e.g., a stream fragment) and is transmitted via various networks such as a broadcast network and / or a broadband network. Although not shown in the figure, the transmitting device 10000 may include a packager (or a package module) configured to perform a packaging operation. According to an embodiment, the packager may be included in the transmitter 10003. According to an embodiment, the file or fragment may be sent to the receiving device 10004 via a network, or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitter 10003 according to the embodiment can communicate with the receiving device 10004 (or the receiver 10005) via a 4G, 5G, 6G or other network by wire / wireless communication. In addition, the transmitter may perform necessary data processing operations according to the network system (e.g., a 4G, 5G or 6G communication network system). The transmitting device 10000 may transmit the encapsulated data in an on-demand manner.

[0058] The receiving device 10004 according to an embodiment includes a receiver 10005, a point cloud video decoder 10006 and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include a device configured to communicate with a base station and / or other wireless devices using a radio access technology (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server.

[0059] The receiver 10005 according to the embodiment receives a bitstream containing point cloud video data or a file / segment encapsulated with a bitstream from a network or a storage medium. The receiver 10005 may perform necessary data processing according to a network system (e.g., a communication network system of 4G, 5G, 6G, etc.). The receiver 10005 according to the embodiment may decapsulate the received file / segment and output a bitstream. According to the embodiment, the receiver 10005 may include a decapsulator (or decapsulation module) configured to perform a decapsulation operation. The decapsulator may be implemented as an element (or component) separated from the receiver 10005.

[0060] The point cloud video decoder 10006 decodes the bitstream containing the point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to the method in which the point cloud video data is encoded (for example, inverse processing of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding (inverse processing of point cloud compression). Point cloud decompression encoding includes G-PCC encoding.

[0061] The renderer 10007 renders the decoded point cloud video data. The renderer 10007 may output the point cloud content by rendering not only the point cloud video data but also the audio data. According to an embodiment, the renderer 10007 may include a display configured to display the point cloud content. According to an embodiment, the display may be implemented as a separate device or component rather than being included in the renderer 10007.

[0062] The arrow indicated by the dotted line in the figure represents the transmission path of the feedback information obtained by the receiving device 10004. Feedback information is information that reflects the interactivity with the user who consumes the point cloud content, and includes information about the user (e.g., head orientation information, viewport information, etc.). Specifically, when the point cloud content is content for a service that requires interaction with the user (e.g., self-driving service, etc.), the feedback information may be provided to the content sender (e.g., sending device 10000) and / or the service provider. Depending on the embodiment, the feedback information may be used in the receiving device 10004 and the sending device 10000, or may not be provided.

[0063] The head orientation information according to the embodiment is information about the user's head position, orientation, angle, movement, etc. The receiving device 10004 according to the embodiment can calculate the viewport information based on the head orientation information. The viewport information can be information about the area of ​​the point cloud video that the user is watching. The viewpoint is the point through which the user watches the point cloud video, and can refer to the center point of the viewport area. That is, the viewport is an area centered on the viewpoint, and the size and shape of the area can be determined by the field of view (FOV). Therefore, in addition to the head orientation information, the receiving device 10004 can also extract the viewport information based on the vertical or horizontal FOV supported by the device. In addition, the receiving device 10004 performs gaze analysis, etc. to check the way the user consumes the point cloud, the area where the user gazes in the point cloud video, the gaze time, etc. According to the embodiment, the receiving device 10004 can send feedback information including the gaze analysis result to the sending device 10000. The feedback information according to the embodiment can be obtained during the rendering and / or display process. The feedback information according to the embodiment can be obtained by one or more sensors included in the receiving device 10004. According to an embodiment, the feedback information may be obtained by the renderer 10007 or a separate external element (or device, component, etc.). Figure 1The dotted line in represents the process of sending the feedback information obtained by the renderer 10007. The point cloud content providing system can process (encode / decode) the point cloud data based on the feedback information. Therefore, the point cloud video data decoder 10006 can perform a decoding operation based on the feedback information. The receiving device 10004 can send the feedback information to the sending device 10000. The sending device 10000 (or the point cloud video data encoder 10002) can perform an encoding operation based on the feedback information. Therefore, the point cloud content providing system can effectively process necessary data (for example, point cloud data corresponding to the user's head position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide the point cloud content to the user.

[0064] According to an embodiment, the transmitting device 10000 may be referred to as an encoder, a transmitting device, a transmitter, etc., and the receiving device 10004 may be referred to as a decoder, a receiving device, a receiver, etc.

[0065] According to the implementation Figure 1 The point cloud data processed in the point cloud content providing system (through a series of processes of acquisition / encoding / transmission / decoding / rendering) may be referred to as point cloud content data or point cloud video data. According to an embodiment, point cloud content data may be used as a concept covering metadata or signaling information related to point cloud data.

[0066] Figure 1 The elements of the point cloud content providing system shown may be implemented by hardware, software, a processor and / or a combination thereof.

[0067] Figure 2 is a block diagram illustrating point cloud content providing operations according to an embodiment.

[0068] Figure 2 The block diagram shows Figure 1 The operation of the point cloud content providing system described in . As described above, the point cloud content providing system can process point cloud data based on point cloud compression coding (e.g., G-PCC).

[0069] According to the embodiment, the point cloud content providing system (e.g., the point cloud sending device 10000 or the point cloud video acquirer 10001) can acquire a point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system for expressing a 3D space. According to the embodiment, the point cloud video may include a Ply (Polygon file format or Stanford Triangle format) file. When the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of the point. The position of each point can be represented by a parameter (e.g., the value of the X, Y and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system composed of X, Y and Z axes). Attributes include attributes of the point (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc. of each point). A point has one or more attributes. For example, a point may have a color attribute or two attributes of color and reflectivity. According to an embodiment, geometry may be referred to as position, geometry information, geometry data, etc., and attributes may be referred to as attributes, attribute information, attribute data, etc. The point cloud content providing system (e.g., the point cloud transmitting device 10000 or the point cloud video acquirer 10001) may obtain point cloud data from information related to a point cloud video acquisition process (e.g., depth information, color information, etc.).

[0070] According to an embodiment, a point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) may encode point cloud data (20001). The point cloud content providing system may encode point cloud data based on point cloud compression coding. As described above, point cloud data may include geometry and attributes of points. Therefore, the point cloud content providing system may perform geometry coding for encoding geometry and output a geometry bitstream. The point cloud content providing system may perform attribute coding for encoding attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system may perform attribute coding based on geometry coding. The geometry bitstream and the attribute bitstream according to the embodiment may be multiplexed and output as one bitstream. The bitstream according to the embodiment may also include signaling information related to geometry coding and attribute coding.

[0071] According to the embodiment, the point cloud content providing system (eg, the transmitting device 10000 or the transmitter 10003) may transmit the encoded point cloud data (20002). Figure 1 As shown, the encoded point cloud data can be represented by a geometry bitstream and an attribute bitstream. In addition, the encoded point cloud data can be sent in the form of a bitstream together with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometry encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and send it in the form of a file or a fragment.

[0072] The point cloud content providing system (eg, receiving device 10004 or receiver 10005) according to the embodiment may receive a bit stream containing encoded point cloud data. In addition, the point cloud content providing system (eg, receiving device 10004 or receiver 10005) may demultiplex the bit stream.

[0073] The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) can decode the encoded point cloud data (e.g., geometry bitstream, attribute bitstream) sent in the bitstream. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) can decode the point cloud video data based on the signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) can decode the geometry bitstream to reconstruct the position (geometry) of the point. The point cloud content providing system can reconstruct the attributes of the point by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) can reconstruct the point cloud video based on the position according to the reconstructed geometry and the decoded attributes.

[0074] According to an embodiment, a point cloud content providing system (e.g., receiving device 10004 or renderer 10007) may render decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) may use various rendering methods to render the geometry and attributes decoded by the decoding process. Points in the point cloud content may be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to the user through a display (e.g., a VR / AR display, a general display, etc.).

[0075] The point cloud content providing system (e.g., receiving device 10004) according to the embodiment may obtain feedback information (20005). The point cloud content providing system may encode and / or decode the point cloud data based on the feedback information. Figure 1 The feedback information and operations described are the same, so their detailed description is omitted.

[0076] Figure 3 An exemplary process of capturing point cloud video according to an embodiment is shown.

[0077] Figure 3 Show reference Figures 1 to 2 An exemplary point cloud video capture process of a point cloud content providing system is described.

[0078] Point cloud content includes point cloud videos (images and / or videos) representing objects and / or environments located in various 3D spaces (e.g., 3D spaces representing real environments, 3D spaces representing virtual environments, etc.). Therefore, a point cloud content providing system according to an embodiment may use one or more cameras (e.g., an infrared camera capable of obtaining depth information, an RGB camera capable of extracting color information corresponding to the depth information, etc.), a projector (e.g., an infrared pattern projector that obtains depth information), LiDAR, etc. to capture point cloud videos. A point cloud content providing system according to an embodiment may extract a geometric shape composed of points in a 3D space from the depth information and extract the attributes of each point from the color information to obtain point cloud data. Images and / or videos according to an embodiment may be captured based on at least one of an inward-facing technique and an outward-facing technique.

[0079] Figure 3 The left portion of shows inward-facing techniques. Inward-facing techniques refer to techniques that utilize one or more cameras (or camera sensors) positioned around a central object to capture an image of the central object. Inward-facing techniques may be used to generate point cloud content that provides a 360-degree image of a key object to a user (e.g., VR / AR content that provides a 360-degree image of an object (e.g., a key object such as a character, player, object, or actor) to a user).

[0080] Figure 3 The right portion of shows outward-facing techniques. Outward-facing techniques refer to techniques that utilize one or more cameras (or camera sensors) positioned around a central object to capture the environment of the central object rather than an image of the central object. Outward-facing techniques can be used to generate point cloud content that provides the surrounding environment as it appears from the user's perspective (e.g., content representing the external environment that can be provided to a user of a self-driving vehicle).

[0081] As shown in the figure, point cloud content can be generated based on the capture operation of one or more cameras. In this case, the coordinate system may be different between the cameras, so the point cloud content providing system may calibrate one or more cameras to set the global coordinate system before the capture operation. In addition, the point cloud content providing system can generate point cloud content by synthesizing any image and / or video with the image and / or video captured by the above-mentioned capture technology. The point cloud content providing system may not perform when generating point cloud content representing a virtual space. Figure 3 The point cloud content providing system according to the embodiment may perform post-processing on the captured image and / or video. In other words, the point cloud content providing system may remove unwanted areas (e.g., background), identify the space to which the captured image and / or video is connected, and when there is a space hole, perform an operation to fill the space hole.

[0082] The point cloud content providing system can generate a point cloud content by performing coordinate transformation on the points of the point cloud video obtained from each camera. The point cloud content providing system can perform coordinate transformation on the points based on the position coordinates of each camera. Therefore, the point cloud content providing system can generate content representing a wide range, or can generate point cloud content with high density points.

[0083] Figure 4 An exemplary point cloud encoder according to an implementation is shown.

[0084] Figure 4 Show Figure 1 An example of a point cloud video encoder 10002. The point cloud encoder reconstructs and encodes point cloud data (e.g., the location and / or attributes of a point) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) according to network conditions or applications. When the total size of the point cloud content is large (e.g., 60 Gbps of point cloud content given for 30 fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system can reconstruct the point cloud content based on the maximum target bit rate to provide the point cloud content according to the network environment, etc.

[0085] As reference Figures 1 to 2 As described above, the point cloud encoder can perform both geometry encoding and attribute encoding. Geometry encoding is performed before attribute encoding.

[0086] According to an embodiment, the point cloud encoder includes a coordinate transformer (transforming coordinates) 40000, a quantizer (quantizing and removing points (voxelization)) 40001, an octree analyzer (analyzing octree) 40002 and a surface approximation analyzer (analyzing surface approximation) 40003, an arithmetic encoder (arithmetic coding) 40004, a geometry reconstructor (reconstructing geometry) 40005, a color transformer (transforming color) 40006, an attribute transformer (transforming attributes) 40007, a RAHT transformer (RAHT) 40008, an LOD generator (generating LOD) 40009, a lifting transformer (lifting) 40010, a coefficient quantizer (quantizing coefficients) 40011 and / or an arithmetic encoder (arithmetic coding) 40012.

[0087] The coordinate transformer 40000, the quantizer 40001, the octree analyzer 40002, the surface approximation analyzer 40003, the arithmetic encoder 40004 and the geometry reconstructor 40005 may perform geometry coding. The geometry coding according to the embodiment may include octree geometry coding, direct coding, triplet geometry coding and entropy coding. Direct coding and triplet geometry coding are applied selectively or in combination. The geometry coding is not limited to the above examples.

[0088] As shown in the figure, the coordinate transformer 40000 according to the embodiment receives the position and transforms it into coordinates. For example, the position can be transformed into position information in a three-dimensional space (e.g., a three-dimensional space represented by an XYZ coordinate system). The position information in the three-dimensional space according to the embodiment can be referred to as geometric information.

[0089] The quantizer 40001 according to the embodiment quantizes the geometry. For example, the quantizer 40001 can quantize the points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). The quantizer 40001 performs a quantization operation: the difference between the minimum position value and the position value of each point is multiplied by a preset quantization scale value, and then the nearest integer value is found by rounding the value obtained by the multiplication. Therefore, one or more points may have the same quantized position (or position value). The quantizer 40001 according to the embodiment performs voxelization based on the quantized position to reconstruct the quantized point. As in the case of a pixel (the smallest unit containing 2D image / video information), the point of the point cloud content (or 3D point cloud video) according to the embodiment may be included in one or more voxels. As a composite of volume and pixel, the term voxel refers to a 3D cubic space generated when the 3D space is divided into units (unit = 1.0) based on the axes representing the 3D space (e.g., X-axis, Y-axis, and Z-axis). Quantizer 40001 can match a point group in a 3D space with a voxel. According to an embodiment, a voxel may include only one point. According to an embodiment, a voxel may include one or more points. In order to represent a voxel as a point, the position of the center of the voxel can be set based on the position of one or more points included in the voxel. In this case, the properties of all positions included in a voxel can be combined and assigned to the voxel.

[0090] The octree analyzer 40002 according to the embodiment performs octree geometry encoding (or octree encoding) to present voxels in an octree structure. The octree structure represents points matched to voxels based on an octal tree structure.

[0091] The surface approximation analyzer 40003 according to an embodiment may analyze and approximate an octree. The octree analysis and approximation according to an embodiment is a process of analyzing a region including a plurality of points to efficiently provide an octree and voxelization.

[0092] The arithmetic encoder 40004 according to the embodiment performs entropy coding on the octree and / or the approximate octree. For example, the coding scheme includes arithmetic coding. As a result of the coding, a geometry bitstream is generated.

[0093] The color converter 40006, the attribute converter 40007, the RAHT converter 40008, the LOD generator 40009, the lifting converter 40010, the coefficient quantizer 40011 and / or the arithmetic encoder 40012 perform attribute encoding. As described above, a point may have one or more attributes. The attribute encoding according to the embodiment is also applied to the attributes of a point. However, when the attribute (e.g., color) includes one or more elements, the attribute encoding is applied independently to each element. The attribute encoding according to the embodiment includes color transform encoding, attribute transform encoding, regional adaptive hierarchical transform (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) encoding, and interpolation-based hierarchical nearest neighbor prediction (lifting transform) encoding with an update / lifting step. Depending on the point cloud content, the above-mentioned RAHT encoding, prediction transform encoding, and lifting transform encoding may be selectively used, or a combination of one or more encoding schemes may be used. The attribute encoding according to the embodiment is not limited to the above examples.

[0094] The color converter 40006 according to the embodiment performs color conversion encoding to convert the color value (or texture) included in the attribute. For example, the color converter 40006 can convert the format of the color information (e.g., from RGB to YCbCr). Optionally, the operation of the color converter 40006 according to the embodiment can be applied according to the color value included in the attribute.

[0095] The geometry reconstructor 40005 according to an embodiment reconstructs (decompresses) an octree and / or an approximate octree. The geometry reconstructor 40005 reconstructs an octree / voxel based on the result of analyzing the point distribution. The reconstructed octree / voxel may be referred to as a reconstructed geometry (restored geometry).

[0096] The attribute transformer 40007 according to the embodiment performs attribute transformation to transform the attribute based on the reconstructed geometry and / or the position where the geometry encoding is not performed. As described above, since the attribute depends on the geometry, the attribute transformer 40007 can transform the attribute based on the reconstructed geometry information. For example, based on the position value of the point included in the voxel, the attribute transformer 40007 can transform the attribute of the point at the position. As described above, when the center position of the voxel is set based on the position of one or more points included in the voxel, the attribute transformer 40007 transforms the attributes of one or more points. When triplet geometry encoding is performed, the attribute transformer 40007 can transform the attribute based on the triplet geometry encoding.

[0097] The attribute converter 40007 can perform attribute conversion by calculating the average of the attributes or attribute values ​​(e.g., the color or reflectivity of each point) of the neighboring points within a specific position / radius from the center position (or position value) of each voxel. The attribute converter 40007 can apply weights according to the distance from the center to each point when calculating the average. Therefore, each voxel has a position and a calculated attribute (or attribute value).

[0098] Attribute converter 40007 can search for adjacent points within a specific position / radius from the center position of each voxel based on a KD tree or a Morton code. KD tree is a binary search tree and supports a data structure capable of managing points based on position, so that a nearest neighbor search (NNS) can be performed quickly. Morton code is generated by presenting the coordinates (e.g., (x, y, z)) representing the 3D positions of all points as bit values ​​and mixing the bits. For example, when the coordinates representing the point position are (5, 9, 1), the bit values ​​of the coordinates are (0101, 1001, 0001). Mixing the bit values ​​in the order of z, y, and x according to the bit index produces 010001000111. This value is represented as a decimal number 1095. That is, the Morton code value of the point with coordinates (5, 9, 1) is 1095. Attribute converter 40007 can sort the points based on the Morton code value and perform NNS through a depth-first traversal process. After the attribute transform operation, KD tree or Morton code is used when NNS is needed in another transform process for attribute encoding.

[0099] As shown, the transformed attributes are input to the RAHT transformer 40008 and / or the LOD generator 40009.

[0100] The RAHT transformer 40008 according to the embodiment performs RAHT encoding for predicting attribute information based on the reconstructed geometric information. For example, the RAHT transformer 40008 may predict attribute information of a higher-level node in the octree based on attribute information associated with a lower-level node in the octree.

[0101] The LOD generator 40009 according to an embodiment generates a level of detail (LOD) to perform predictive transform coding. The LOD according to an embodiment is the degree of detail of the point cloud content. As the LOD value decreases, the details of the point cloud content are degraded. As the LOD value increases, the details of the point cloud content are enhanced. Points can be classified by LOD.

[0102] The lifting transformer 40010 according to an embodiment performs lifting transform coding that transforms point cloud attributes based on weights. As described above, lifting transform coding can be optionally applied.

[0103] The coefficient quantizer 40011 according to the embodiment quantizes the attribute of the attribute encoding based on the coefficient.

[0104] The arithmetic encoder 40012 according to the embodiment encodes quantized properties based on arithmetic coding.

[0105] Although not shown in the figure, Figure 4 The elements of the point cloud encoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. The one or more processors may execute the above Figure 4 At least one of the operations and / or functions of the elements of the point cloud encoder. In addition, one or more processors may be operable or executable to perform Figure 4 A software program and / or set of instructions for the operation and / or functionality of the elements of the point cloud encoder. One or more memories according to an embodiment may include high-speed random access memory, or include non-volatile memory (e.g., one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0106] Figure 5 An example of a voxel according to an embodiment is shown.

[0107] Figure 5 The voxels are shown in a 3D space represented by a coordinate system consisting of three axes (X axis, Y axis and Z axis). Figure 4 As described above, the point cloud encoder (eg, quantizer 40001) may perform voxelization. A voxel refers to a 3D cubic space generated when a 3D space is divided into units (unit=1.0) based on axes representing the 3D space (eg, X-axis, Y-axis, and Z-axis). Figure 5 An example of voxels generated by an octree structure is shown, where a cubic axis-aligned bounding box defined by two poles (0,0,0) and (2d,2d,2d) is recursively subdivided. A voxel includes at least one point. The spatial coordinates of the voxel can be estimated from the positional relationship with the voxel group. As described above, the voxel has properties similar to the pixels of a 2D image / video (e.g., color or reflectivity). The details of the voxel are similar to those of the reference Figure 4 Those described are the same, so their description is omitted.

[0108] Figure 6 An example of an octree and occupancy code according to an embodiment is shown.

[0109] As reference Figures 1 to 4As described, a point cloud content providing system (point cloud video encoder 10002) or a point cloud encoder (e.g., octree analyzer 40002) performs octree geometry encoding (or octree encoding) based on an octree structure to efficiently manage the area and / or position of voxels.

[0110] Figure 6 The upper part of FIG. 1 shows the octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes (eg, X-axis, Y-axis, and Z-axis) of the coordinate system. d ,2 d ,2 d ) to create an octree structure. Here, 2 d It can be set to the value of the minimum bounding box that surrounds all points of the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following formula. In the following formula, (x int n,y int n,z int n) represents the position (or position value) of the quantized point.

[0111] d=Ceil(Log2(Max(x_n^int,y_n^int,z_n^int,n=1,...,N)+1))

[0112] like Figure 6 As shown in the middle of the upper part of , the entire 3D space can be divided into eight spaces according to the segmentation. Each divided space is represented by a cube with six faces. Figure 6 As shown in the upper right part of , each of the eight spaces is divided again based on the axis of the coordinate system (e.g., X-axis, Y-axis, and Z-axis). Therefore, each space is divided into eight smaller spaces. The divided smaller spaces are also represented by cubes with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0113] Figure 6 The lower part shows the octree occupancy code. The occupancy code of the octree is generated to indicate whether each of the eight divided spaces generated by dividing one space contains at least one point. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of the divided space, and the child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to the child node contains at least one point, the node is assigned a value of 1. When the space corresponding to the child node does not contain a point (the space is empty), the node is assigned a value of 0. Since Figure 6The occupancy code shown is 00100001, so it indicates that the spaces corresponding to the third child node and the eighth child node among the eight child nodes each contain at least one point. As shown in the figure, each of the third child node and the eighth child node has eight child nodes, and the child nodes are represented by 8-bit occupancy codes. The accompanying drawings show that the occupancy code of the third child node is 10000111, and the occupancy code of the eighth child node is 01001111. The point cloud encoder (e.g., arithmetic encoder 40004) according to the embodiment may perform entropy encoding on the occupancy code. In order to increase the compression efficiency, the point cloud encoder may perform intra-frame / inter-frame encoding on the occupancy code. The receiving device (e.g., receiving device 10004 or point cloud video decoder 10006) according to the embodiment reconstructs the occupancy code based on the occupancy code.

[0114] According to an embodiment, a point cloud encoder (e.g., Figure 4 The point cloud encoder or octree analyzer 40002) may perform voxelization and octree encoding to store point locations. However, points are not always evenly distributed in the 3D space, so there may be specific areas where fewer points exist. Therefore, performing voxelization on the entire 3D space is inefficient. For example, when a specific area contains very few points, voxelization does not need to be performed in the specific area.

[0115] Therefore, for the above-mentioned specific area (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment may skip voxelization and perform direct encoding to directly encode the point positions included in the specific area. The coordinates of the directly encoded points according to the embodiment are called direct coding mode (DCM). The point cloud encoder according to the embodiment may also perform triplet geometry encoding based on the surface model, which is to reconstruct the point positions in the specific area (or node) based on voxels. Triplet geometry encoding is a geometry encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct encoding and triplet geometry encoding according to the embodiment may be performed selectively. In addition, direct encoding and triplet geometry encoding according to the embodiment may be performed in combination with octree geometry encoding (or octree encoding).

[0116] In order to perform direct encoding, the option of using direct mode to apply direct encoding should be enabled. The node to which direct encoding is to be applied is not a leaf node, and there should be points less than a threshold value within a specific node. In addition, the total number of points to which direct encoding is to be applied should not exceed a preset threshold. When the above conditions are met, the point cloud encoder (or arithmetic encoder 40004) according to the embodiment can perform entropy encoding on the point position (or position value).

[0117] A point cloud encoder (e.g., surface approximation analyzer 40003) according to an embodiment may determine a specific level of the octree (a level less than the depth d of the octree), and may use a surface model starting from this level to perform triplet geometry encoding to reconstruct point positions in the node area based on voxels (triplet mode). A point cloud encoder according to an embodiment may specify a level to which triplet geometry encoding is to be applied. For example, when a specific level is equal to the depth of the octree, the point cloud encoder does not operate in triplet mode. In other words, a point cloud encoder according to an embodiment may operate in triplet mode only when the specified level is less than the depth value of the octree. A 3D cubic area of ​​a node at a specified level according to an embodiment is referred to as a block. A block may include one or more voxels. A block or voxel may correspond to a block. Geometry is represented as a surface within each block. A surface according to an embodiment may intersect each edge of a block at most once.

[0118] A block has 12 edges, so there are at least 12 intersections in a block. Each intersection is called a vertex. When there is at least one occupied voxel adjacent to the edge among all blocks sharing the edge, the vertex existing along the edge is detected. An occupied voxel according to an embodiment refers to a voxel containing a point. The vertex position detected along the edge is the average position of the edge of all voxels adjacent to the edge among all blocks along the shared edge.

[0119] Once the vertex is detected, the point cloud encoder according to the embodiment can perform entropy encoding on the start point (x, y, z) of the edge, the direction vector (Δx, Δy, Δz) of the edge, and the vertex position value (relative position value within the edge). When triplet geometry encoding is applied, the point cloud encoder according to the embodiment (e.g., geometry reconstructor 40005) can generate restored geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization processes.

[0120] The vertices located at the edges of the block determine the surface passing through the block. The surface according to an embodiment is a non-planar polygon. In the triangle reconstruction process, the surface represented by the triangle is reconstructed based on the starting point of the edge, the direction vector of the edge and the position value of the vertex. The triangle reconstruction process is performed by i) calculating the centroid value of each vertex, ii) subtracting the center value from each vertex value, and iii) estimating the sum of the squares of the values ​​obtained by subtraction.

[0121]

[0122] The minimum value of the sum is estimated, and the projection process is performed according to the axis with the minimum value. For example, when the element x is the smallest, each vertex is projected on the x-axis relative to the center of the block and projected on the (y,z) plane. When the value obtained by the projection on the (y,z) plane is (ai,bi), the value of θ is estimated by atan2(bi,ai), and the vertices are sorted based on the value of θ. The following table shows that for 4 vertices, two triangles can be constructed according to the vertex combination. The first triangle can be composed of vertices 1, 2, and 3 among the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 among the sorted vertices.

[0123] Table 2-1. Triangles formed from vertices ordered 1,…,n

[0124] n triangle

[0125] 3(1,2,3)

[0126] 4(1,2,3),(3,4,1)

[0127] 5(1,2,3),(3,4,5),(5,1,3)

[0128] 6(1,2,3),(3,4,5),(5,6,1),(1,3,5)

[0129] 7(1,2,3),(3,4,5),(5,6,7),(7,1,3),(3,5,7)

[0130] 8(1,2,3),(3,4,5),(5,6,7),(7,8,1),(1,3,5),(5,7,1)

[0131] 9(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,1,3),(3,5,7),(7,9,3)

[0132] 10(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,1),(1,3,5),(5,7,9),(9,1,5)

[0133] 11(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,11),(11,1,3),(3,5,7),(7,9,11),(11,3,7)

[0134] 12(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,11),(11,12,1),(1,3,5),(5,7,9),(9,11,1),(1,5,9)

[0135] An upsampling process is performed to add points in the middle along the edges of the triangle, and voxelization is performed. The added points are generated based on the upsampling factor and the width of the block. The added points are called refined vertices. The point cloud encoder according to the embodiment may voxelize the refined vertices. In addition, the point cloud encoder may perform attribute encoding based on the voxelized position (or position value).

[0136] Figure 7 An example of a neighbor node pattern according to an embodiment is shown.

[0137] In order to increase the compression efficiency of the point cloud video, the point cloud encoder according to the embodiment may perform entropy coding based on context adaptive arithmetic coding.

[0138] As reference Figures 1 to 6 As described, the point cloud content providing system or point cloud encoder (eg, point cloud video encoder 10002, Figure 4 The point cloud encoder or arithmetic encoder 40004) may immediately perform entropy encoding on the occupancy code. In addition, the point cloud content providing system or the point cloud encoder may perform entropy encoding (intra-frame encoding) based on the occupancy code of the current node and the occupancy of the neighboring nodes, or perform entropy encoding (inter-frame encoding) based on the occupancy code of the previous frame. The frame representation according to the embodiment is a collection of point cloud videos generated simultaneously. The compression efficiency of the intra-frame coding / inter-frame coding according to the embodiment may depend on the number of neighboring nodes referenced. When the number of bits increases, the operation becomes complicated, but the encoding can be biased to one side, which can increase the compression efficiency. For example, when a 3-bit context is given, 2 bits need to be used. 3 = 8 ways to perform encoding. The division of parts for encoding affects the implementation complexity. Therefore, it is necessary to meet appropriate levels of compression efficiency and complexity.

[0139] Figure 7 The process of obtaining an occupancy pattern based on the occupancy of neighbor nodes is shown. The point cloud encoder according to the embodiment determines the occupancy of neighbor nodes of each node of the octree and obtains the value of the neighbor pattern. The neighbor node pattern is used to infer the occupancy pattern of the node. Figure 7 The left part of the diagram shows a cube corresponding to a node (the cube in the middle) and six cubes (neighboring nodes) that share at least one face with the cube. The nodes shown in the diagram are nodes at the same depth. The numbers shown in the diagram represent the weights (1, 2, 4, 8, 16, and 32) associated with the six nodes, respectively. The weights are assigned in sequence according to the positions of the neighboring nodes.

[0140] Figure 7 The right part of shows the neighbor node pattern value. The neighbor node pattern value is the sum of the values ​​multiplied by the weights of the occupied neighbor nodes (neighbor nodes with points). Therefore, the neighbor node pattern value is 0 to 63. When the neighbor node pattern value is 0, it indicates that there is no node with a point among the neighbor nodes of the node (no occupied node). When the neighbor node pattern value is 63, it indicates that all neighbor nodes are occupied nodes. As shown in the figure, since the neighbor nodes assigned weights 1, 2, 4 and 8 are occupied nodes, the neighbor node pattern value is 15 (the sum of 1, 2, 4 and 8). The point cloud encoder can perform encoding according to the neighbor node pattern value (for example, when the neighbor node pattern value is 63, 64 types of encoding can be performed). According to an embodiment, the point cloud encoder can reduce the encoding complexity by changing the neighbor node pattern value (for example, based on a table that changes 64 to 10 or 6).

[0141] Figure 8 An example of point configuration in each LOD according to an embodiment is shown.

[0142] As reference Figures 1 to 7 As described, the encoded geometry is reconstructed (decompressed) before performing attribute encoding. When direct encoding is applied, the geometry reconstruction operation may include changing the placement of the directly encoded points (e.g., placing the directly encoded points in front of the point cloud data). When triplet geometry encoding is applied, the geometry reconstruction process is performed by triangle reconstruction, upsampling, and voxelization. Since the attributes depend on the geometry, attribute encoding is performed based on the reconstructed geometry.

[0143] The point cloud encoder (e.g., LOD generator 40009) can classify (reorganize) the points by LOD. The figure shows the point cloud content corresponding to the LOD. The leftmost screen in the figure shows the original point cloud content. The second screen from the left in the figure shows the distribution of points in the lowest LOD, and the rightmost screen in the figure shows the distribution of points in the highest LOD. That is, the points in the lowest LOD are sparsely distributed, and the points in the highest LOD are densely distributed. That is, as the LOD increases in the direction indicated by the arrow indicated at the bottom of the figure, the space (or distance) between the points becomes narrower.

[0144] Figure 9 An example of point configuration for each LOD according to an embodiment is shown.

[0145] As reference Figures 1 to 8 As described, the point cloud content providing system or point cloud encoder (eg, point cloud video encoder 10002, Figure 4The point cloud encoder or LOD generator 40009) can generate LOD. LOD is generated by reorganizing points into a set of refinement levels according to the set LOD distance value (or Euclidean distance set). The LOD generation process is performed not only by the point cloud encoder, but also by the point cloud decoder.

[0146] Figure 9 The upper part of shows examples of points (P0 to P9) of point cloud content distributed in 3D space. Figure 9 In , the original order represents the order of points P0 to P9 before LOD generation. Figure 9 In the LOD-based order, the order of points generated according to the LOD is shown. Points are reorganized by LOD. In addition, a high LOD contains points belonging to a lower LOD. Figure 9 As shown, LOD 0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD 0, P1, P6, and P3. LOD2 contains the points of LOD 0, the points of LOD1, P9, P8, and P7.

[0147] As reference Figure 4 As described, the point cloud encoder according to the embodiment may selectively or in combination perform prediction transform coding, lifting transform coding, and RAHT transform coding.

[0148] The point cloud encoder according to the embodiment may generate a predictor for a point to perform predictive transform coding for setting a prediction attribute (or a prediction attribute value) of each point. That is, N predictors may be generated for N points. The predictor according to the embodiment may calculate a weight (=1 / distance) based on the LOD value of each point, index information about neighboring points that exist within a set distance of each LOD, and the distance to the neighboring point.

[0149] The predicted attribute (or attribute value) according to the embodiment is set to the average of the values ​​obtained by multiplying the attribute (or attribute value) (e.g., color, reflectivity, etc.) of the neighboring points set in the predictor of each point by the weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 40011) according to the embodiment may quantize and inverse quantize the residual (which may be referred to as residual attribute, residual attribute value, or attribute prediction residual) obtained by subtracting the predicted attribute (attribute value) from the attribute (attribute value) of each point. The quantization process is configured as shown in the following table.

[0150] Table. Attribute prediction residual quantization pseudo code

[0151] int PCCQuantization(intvalue, intquantStep){

[0152] if(value>=0){

[0153] return floor(value / quantStep+1.0 / 3.0);

[0154] }else{

[0155] return-floor(-value / quantStep+1.0 / 3.0);

[0156] }

[0157] }

[0158] Table. Pseudocode for inverse quantization of attribute prediction residuals

[0159] int PCCInverseQuantization(int value, int quantStep){

[0160] if(quantStep==0){

[0161] return value;

[0162] }else{

[0163] return value*quantStep;

[0164] }

[0165] }

[0166] When the predictor of each point has neighboring points, the point cloud encoder (e.g., arithmetic encoder 40012) according to the embodiment may perform entropy encoding on the quantized and inverse quantized residual values ​​as described above. When the predictor of each point has no neighboring points, the point cloud encoder (e.g., arithmetic encoder 40012) according to the embodiment may perform entropy encoding on the attributes of the corresponding points without performing the above operation.

[0167] The point cloud encoder (e.g., lifting transformer 40010) according to an embodiment may generate a predictor for each point, set the calculated LOD and register the neighboring points in the predictor, and set the weight according to the distance to the neighboring points to perform lifting transform encoding. The lifting transform encoding according to an embodiment is similar to the above-mentioned prediction transform encoding, but is different in that the weight is cumulatively applied to the attribute value. The process of cumulatively applying the weight to the attribute value according to an embodiment is configured as follows.

[0168] 1) Create an array quantized weight (QW) for storing weight values ​​of each point. The initial value of all elements of QW is 1.0. Multiply the QW value of the predictor index of the neighbor node registered in the predictor by the weight of the predictor of the current point, and add the values ​​obtained by multiplication.

[0169] 2) Boosting prediction process: The value obtained by multiplying the attribute value of the point by the weight is subtracted from the existing attribute value to calculate the predicted attribute value.

[0170] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0171] 4) Accumulate the weight calculated by multiplying the weight calculated for all predictors by the weight stored in the QW corresponding to the predictor index with the updateweight array as the index of the neighbor node. Accumulate the value obtained by multiplying the attribute value of the neighbor node index by the calculated weight with the update array.

[0172] 5) Boosting update process: Divide the attribute values ​​of the update array of all predictors by the weight value of the updateweight array of the predictor index, and add the existing attribute value to the value obtained by the division.

[0173] 6) For all predictors, the predicted attribute is calculated by multiplying the attribute value updated by the lifting update process by the weight updated by the lifting prediction process (stored in QW). The point cloud encoder (e.g., coefficient quantizer 40011) according to the embodiment quantizes the predicted attribute value. In addition, the point cloud encoder (e.g., arithmetic encoder 40012) performs entropy encoding on the quantized attribute value.

[0174] A point cloud encoder according to an embodiment (e.g., RAHT transformer 40008) may perform RAHT transform coding, in which attributes associated with nodes at a lower level in an octree are used to predict attributes of nodes at a higher level. RAHT transform coding is an example of attribute intra-frame coding by scanning backward through an octree. A point cloud encoder according to an embodiment scans the entire area starting from voxels and repeats a merging process of merging voxels into larger blocks at each step until a root node is reached. The merging process according to an embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on the upper node directly above the empty node.

[0175] The following formula represents the RAHT transformation matrix. In this formula, Represents the average attribute value of voxels at level l. Can be based on and to calculate. and The weight of and

[0176]

[0177] here, Is the low-pass value and is used during the next highest level of merging. Denotes the high-pass coefficient. The high-pass coefficient of each step is quantized and subjected to entropy coding (eg, encoded by arithmetic encoder 400012). The weight is calculated as pass and Create the root node as follows.

[0178]

[0179] The values ​​of gDC are also quantized like the high-pass coefficients and are subjected to entropy coding.

[0180] Figure 10 A point cloud decoder according to an embodiment is shown.

[0181] Figure 10 The point cloud decoder shown is Figure 1 An example of a point cloud video decoder 10006 described in Figure 1 The operations of the point cloud video decoder 10006 shown are the same or similar operations. As shown, the point cloud decoder can receive a geometry bitstream and an attribute bitstream contained in one or more bitstreams. The point cloud decoder includes a geometry decoder and an attribute decoder. The geometry decoder performs geometry decoding on the geometry bitstream and outputs the decoded geometry. The attribute decoder performs attribute decoding based on the decoded geometry and attribute bitstreams, and outputs the decoded attributes. The decoded geometry and the decoded attributes are used to reconstruct the point cloud content (decoded point cloud).

[0182] Figure 11 A point cloud decoder according to an embodiment is shown.

[0183] Figure 11 The point cloud decoder shown is Figure 10 An example of a point cloud decoder is shown, and a decoding operation can be performed, which is Figures 1 to 9 The inverse process of the encoding operation of the point cloud encoder shown.

[0184] As reference Figure 1 and Figure 10 As described, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0185] According to an embodiment, a point cloud decoder includes an arithmetic decoder (arithmetic decoding) 11000, an octree synthesizer (synthesized octree) 11001, a surface approximation synthesizer (synthesized surface approximation) 11002 and a geometry reconstructor (reconstructed geometry) 11003, an inverse coordinate transformer (inverse transformed coordinates) 11004, an arithmetic decoder (arithmetic decoding) 11005, an inverse quantizer (inverse quantization) 11006, a RAHT transformer 11007, an LOD generator (generate LOD) 11008, an inverse lifting (inverse lifting) 11009 and / or an inverse color transformer (inverse transformed color) 11010.

[0186] The arithmetic decoder 11000, the octree synthesizer 11001, the surface approximation synthesizer 11002, the geometry reconstructor 11003, and the coordinate inverse transformer 11004 may perform geometry decoding. The geometry decoding according to the embodiment may include direct encoding and triplet geometry decoding. Direct encoding and triplet geometry decoding are selectively applied. The geometry decoding is not limited to the above examples, and as a reference Figures 1 to 9 The inverse process of the geometric encoding described is performed.

[0187] The arithmetic decoder 11000 according to the embodiment decodes the received geometry bitstream based on arithmetic coding. The operation of the arithmetic decoder 11000 corresponds to the inverse process of the arithmetic encoder 40004.

[0188] The octree synthesizer 11001 according to the embodiment may generate an octree by acquiring an occupancy code (or information about the geometry obtained as a result of decoding) from the decoded geometry bitstream. Figures 1 to 9 Describe the configuration in detail.

[0189] When triplet geometry encoding is applied, a surface approximation synthesizer 11002 according to an embodiment may synthesize a surface based on the decoded geometry and / or the generated octree.

[0190] According to an embodiment, the geometry reconstructor 11003 can regenerate the geometry based on the surface and / or the decoded geometry. Figures 1 to 9 As described, direct coding and triplet geometry coding are selectively applied. Therefore, the geometry reconstructor 11003 directly imports the position information about the points to which direct coding is applied and adds them. When triplet geometry coding is applied, the geometry reconstructor 11003 can reconstruct the geometry by performing the reconstruction operations (e.g., triangle reconstruction, upsampling, and voxelization) of the geometry reconstructor 40005. Details and References Figure 6 The reconstructed geometry may include a point cloud screen or frame that does not contain attributes.

[0191] The coordinate inverse transformer 11004 according to an embodiment can obtain the point position by transforming the coordinates based on the reconstructed geometry.

[0192] The arithmetic decoder 11005, the inverse quantizer 11006, the RAHT transformer 11007, the LOD generator 11008, the inverse lifter 11009 and / or the color inverse transformer 11010 may perform reference Figure 10 Attribute decoding described. Attribute decoding according to an embodiment includes regional adaptive hierarchical transform (RAHT) decoding, hierarchical nearest neighbor prediction (prediction transform) decoding based on interpolation, and hierarchical nearest neighbor prediction (lifting transform) decoding based on interpolation with an update / lifting step. The above three decoding schemes can be used selectively, or a combination of one or more decoding schemes can be used. Attribute decoding according to an embodiment is not limited to the above examples.

[0193] The arithmetic decoder 11005 according to the embodiment decodes the attribute bit stream through arithmetic coding.

[0194] The inverse quantizer 11006 according to the embodiment inversely quantizes information about the decoded attribute bitstream or the attribute obtained as a result of decoding, and outputs the inversely quantized attribute (or attribute value). The inverse quantization may be selectively applied based on the attribute encoding of the point cloud encoder.

[0195] According to an embodiment, the RAHT transformer 11007, the LOD generator 11008 and / or the inverse lifter 11009 may process the reconstructed geometry and inverse quantized attributes. As described above, the RAHT transformer 11007, the LOD generator 11008 and / or the inverse lifter 11009 may selectively perform a decoding operation corresponding to the encoding of the point cloud encoder.

[0196] The color inverse transformer 11010 according to an embodiment performs inverse transform encoding to inversely transform a color value (or texture) included in the decoded attribute. The operation of the color inverse transformer 11010 may be selectively performed based on the operation of the color transformer 40006 of the point cloud encoder.

[0197] Although not shown in the figure, Figure 11 The elements of the point cloud decoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. The one or more processors may execute the above Figure 11 At least one or more of the operations and / or functions of the elements of the point cloud decoder. In addition, one or more processors may be operable or executed to perform Figure 11 A software program and / or instruction set for the operation and / or functionality of the elements of a point cloud decoder.

[0198] Figure 12 A transmitting device according to an embodiment is shown.

[0199] Figure 12 The sending device shown is Figure 1 The sending device 10000 (or Figure 4 Example of a point cloud encoder). Figure 12 The sending device shown can be executed with reference to Figures 1 to 9 The transmitting device according to the embodiment may include a data input unit 12000, a quantization processor 12001, a voxelization processor 12002, an octree occupancy code generator 12003, a surface model processor 12004, an intra / inter coding processor 12005, an arithmetic encoder 12006, a metadata processor 12007, a color transformation processor 12008, an attribute transformation processor 12009, a prediction / lifting / RAHT transformation processor 12010, an arithmetic encoder 12011 and / or a transmission processor 12012.

[0200] The data input unit 12000 according to the embodiment receives or acquires point cloud data. The data input unit 12000 may perform the same operation and / or acquisition method as the point cloud video acquirer 10001 (or refer to Figure 2 The acquisition process 20000) described above has the same or similar operations and / or acquisition methods.

[0201] The data input unit 12000, the quantization processor 12001, the voxelization processor 12002, the octree occupancy code generator 12003, the surface model processor 12004, the intra / inter coding processor 12005 and the arithmetic encoder 12006 perform geometric coding. Figures 1 to 9 The geometrical encodings described are the same or similar, so their detailed description is omitted.

[0202] According to an embodiment, the quantization processor 12001 quantizes geometry (e.g., position values ​​of points). The operation and / or quantization of the quantization processor 12001 are related to the reference Figure 4 The operation and / or quantization of the quantizer 40001 described above are the same or similar. Figures 1 to 9 Same as those described.

[0203] According to the embodiment, the voxelization processor 12002 voxelizes the quantized position value of the point. The voxelization processor 120002 may perform the same as the reference Figure 4 The operation and / or voxelization process of the quantizer 40001 described above is the same or similar to the operation and / or process described above. Figures 1 to 9 Same as those described.

[0204] The octree occupancy code generator 12003 according to the embodiment performs octree encoding on the voxelized position of the point based on the octree structure. The octree occupancy code generator 12003 may generate an occupancy code. The octree occupancy code generator 12003 may perform the same as the reference code. Figure 4 and Figure 6 The operations and / or methods of the point cloud encoder (or octree analyzer 40002) described above are the same or similar to the operations and / or methods described above. Figures 1 to 9 Same as those described.

[0205] According to an embodiment, the surface model processor 12004 may perform triplet geometry encoding based on the surface model to reconstruct the point position in a specific area (or node) based on voxels. Figure 4 The operations and / or methods described herein are the same or similar to the operations and / or methods of the point cloud encoder (e.g., surface approximation analyzer 40003). Figures 1 to 9 Same as those described.

[0206] According to the embodiment, the intra-frame / inter-frame encoding processor 12005 may perform intra-frame / inter-frame encoding on the point cloud data. The intra-frame / inter-frame encoding processor 12005 may perform the same as the reference. Figure 7 The same or similar encoding as described for intra / inter encoding. Details and references Figure 7 The intra / inter encoding processor 12005 may be included in the arithmetic encoder 12006 according to an embodiment.

[0207] According to an embodiment, the arithmetic encoder 12006 performs entropy coding on the octree and / or approximate octree of the point cloud data. For example, the coding scheme includes arithmetic coding. The arithmetic encoder 12006 performs the same or similar operations and / or methods as the operations and / or methods of the arithmetic encoder 40004.

[0208] The metadata processor 12007 according to an embodiment processes metadata (e.g., set values) about the point cloud data and provides it to necessary processing processes such as geometry coding and / or attribute coding. In addition, the metadata processor 12007 according to an embodiment may generate and / or process signaling information related to geometry coding and / or attribute coding. The signaling information according to an embodiment may be encoded separately from the geometry coding and / or attribute coding. The signaling information according to an embodiment may be interleaved.

[0209] The color conversion processor 12008, the attribute conversion processor 12009, the prediction / lifting / RAHT conversion processor 12010, and the arithmetic encoder 12011 perform attribute encoding.Figures 1 to 9 The attribute codes described are the same or similar, so their detailed description is omitted.

[0210] The color transform processor 12008 according to an embodiment performs color transform encoding to transform the color value included in the attribute. The color transform processor 12008 may perform color transform encoding based on the reconstructed geometry. The reconstructed geometry is compared with the reference Figures 1 to 9 In addition, its execution is the same as that of reference Figure 4 The operations and / or methods of the color converter 40006 described above are the same as or similar to the operations and / or methods described above, and their detailed description is omitted.

[0211] The attribute transformation processor 12009 according to an embodiment performs attribute transformation to transform attributes based on reconstructed geometry and / or locations where geometry encoding is not performed. The attribute transformation processor 12009 performs the attribute transformation with reference to Figure 4 The operations and / or methods of the attribute converter 40007 described in the foregoing description are the same as or similar to those of the attribute converter 40007. Detailed description thereof is omitted. The prediction / lifting / RAHT transform processor 12010 according to the embodiment may encode the transformed attribute by any one or a combination of RAHT coding, prediction transform coding, and lifting transform coding. The prediction / lifting / RAHT transform processor 12010 performs the same as or similar to the reference. Figure 4 The operations of the RAHT transformer 40008, the LOD generator 40009 and the lifting transformer 40010 described above are the same or similar to at least one operation. In addition, the prediction transform coding, the lifting transform coding and the RAHT transform coding are the same as those of the reference Figures 1 to 9 Those described are the same, so their detailed description is omitted.

[0212] The arithmetic encoder 12011 according to the embodiment may encode the properties of the encoding based on arithmetic coding. The arithmetic encoder 12011 performs the same or similar operations and / or methods as those of the arithmetic encoder 400012.

[0213] The transmission processor 12012 according to the embodiment may send individual bitstreams containing encoded geometry and / or encoded attribute and metadata information, or send a bitstream configured with encoded geometry and / or encoded attribute and metadata information. When the encoded geometry and / or encoded attribute and metadata information according to the embodiment is configured into a bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to the embodiment may include signaling information and slice data, and the signaling information includes a sequence parameter set (SPS) for sequence level signaling, a geometry parameter set (GPS) for geometry information coding signaling, an attribute parameter set (APS) for attribute information coding signaling, and a patch parameter set (TPS) for patch level signaling. Slice data may include information about one or more slices. A slice according to the embodiment may include a geometry bitstream Geom00 and one or more attribute bitstreams Attr00 and Attr10.

[0214] A slice is a sequence of syntactic elements that represents all or part of a coded point cloud frame.

[0215] The TPS according to an embodiment may include information about each of the one or more tiles (e.g., coordinate information and height / size information about a bounding box). The geometry bitstream may include a header and a payload. The header of the geometry bitstream according to an embodiment may include a parameter set identifier (geom_parameter_set_id), a tile identifier (geom_tile_id), and a slice identifier (geom_slice_id) included in the GPS, as well as information about the data contained in the payload. As described above, the metadata processor 12007 according to an embodiment may generate and / or process signaling information and send it to the transmission processor 12012. According to an embodiment, an element that performs geometry encoding and an element that performs attribute encoding may share data / information with each other as indicated by the dotted lines. The transmission processor 12012 according to an embodiment may perform operations and / or transmission methods that are the same or similar to those of the transmitter 10003. Details and References Figure 1 and Figure 2 Those described are the same, so their description is omitted.

[0216] Figure 13 A receiving device according to an embodiment is shown.

[0217] Figure 13 The receiving device shown is Figure 1 The receiving device 10004 (or Figure 10 and Figure 11 Example of a point cloud decoder. Figure 13 The receiving device shown can execute and refer to Figures 1 to 11One or more operations and methods that are the same or similar to those of the described point cloud decoder.

[0218] The receiving device according to the embodiment includes a receiver 13000, a receiving processor 13001, an arithmetic decoder 13002, an octtree reconstruction processor based on an occupancy code 13003, a surface model processor (triangle reconstruction, upsampling, voxelization) 13004, an inverse quantization processor 13005, a metadata parser 13006, an arithmetic decoder 13007, an inverse quantization processor 13008, a prediction / lifting / RAHT inverse transform processor 13009, a color inverse transform processor 13010 and / or a renderer 13011. Each decoding element according to the embodiment may perform an inverse process of the operation of the corresponding encoding element according to the embodiment.

[0219] The receiver 13000 according to the embodiment receives point cloud data. The receiver 13000 may perform the same Figure 1 The operation and / or receiving method of the receiver 10005 is the same as or similar to the operation and / or receiving method of the receiver 10005. The detailed description thereof is omitted.

[0220] The reception processor 13001 according to an embodiment may obtain a geometry bitstream and / or an attribute bitstream from the received data. The reception processor 13001 may be included in the receiver 13000.

[0221] The arithmetic decoder 13002, the octtree reconstruction processor 13003 based on the occupancy code, the surface model processor 13004, and the inverse quantization processor 13005 may perform geometric decoding. Figures 1 to 10 The geometric decoding described is the same or similar, so its detailed description is omitted.

[0222] The arithmetic decoder 13002 according to the embodiment may decode the geometry bitstream based on arithmetic coding. The arithmetic decoder 13002 performs the same or similar operation and / or encoding as that of the arithmetic decoder 11000.

[0223] The octree reconstruction processor 13003 based on the occupancy code according to the embodiment can reconstruct the octree by obtaining the occupancy code from the decoded geometry bitstream (or information about the geometry obtained as a result of decoding). The octree reconstruction processor 13003 based on the occupancy code performs the same or similar operations and / or methods as the operations of the octree synthesizer 11001 and / or the octree generation method. When triplet geometry coding is applied, the surface model processor 13004 according to the embodiment can perform triplet geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on the surface model method. The surface model processor 13004 performs the same or similar operations as the operations of the surface approximation synthesizer 11002 and / or the geometry reconstructor 11003.

[0224] The inverse quantization processor 13005 according to an embodiment may inversely quantize the decoded geometry.

[0225] According to an embodiment, the metadata parser 13006 may parse metadata (e.g., setting values) contained in the received point cloud data. The metadata parser 13006 may pass the metadata to the geometry decoding and / or attribute decoding. Figure 12 The metadata described is the same, so a detailed description thereof is omitted.

[0226] The arithmetic decoder 13007, the inverse quantization processor 13008, the prediction / lifting / RAHT inverse transform processor 13009 and the color inverse transform processor 13010 perform attribute decoding. Figures 1 to 10 The attribute decoding described is the same or similar, so the detailed description thereof is omitted.

[0227] The arithmetic decoder 13007 according to the embodiment can decode the attribute bitstream by arithmetic coding. The arithmetic decoder 13007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 13007 performs the same or similar operation and / or encoding as the operation and / or encoding of the arithmetic decoder 11005.

[0228] The inverse quantization processor 13008 according to an embodiment may inversely quantize the decoded attribute bitstream. The inverse quantization processor 13008 performs the same or similar operation and / or method as the inverse quantizer 11006 and / or the inverse quantization method.

[0229] The prediction / lifting / RAHT inverse transformer 13009 according to an embodiment may process the reconstructed geometry and inverse quantized attributes. The prediction / lifting / RAHT inverse transformer 13009 performs one or more operations and / or decoding that are the same or similar to the operations and / or decoding of the RAHT transformer 11007, the LOD generator 11008, and / or the inverse transformer 11009. The color inverse transformer 13010 according to an embodiment performs inverse transform encoding to inversely transform the color value (or texture) included in the decoded attribute. The color inverse transformer 13010 performs the same or similar operations and / or inverse transform encoding as the operations and / or inverse transform encoding of the color inverse transformer 11010. The renderer 13011 according to an embodiment may render point cloud data.

[0230] Figure 14 An exemplary structure that can be operated in conjunction with a point cloud data transmitting / receiving method / apparatus according to an embodiment is shown.

[0231] Figure 14 The structure of represents a configuration in which at least one of the server 1460, the robot 1410, the self-driving vehicle 1420, the XR device 1430, the smart phone 1440, the home appliance 1450, and / or the head mounted display (HMD) 1470 is connected to the cloud network 1400. The robot 1410, the self-driving vehicle 1420, the XR device 1430, the smart phone 1440, or the home appliance 1450 is referred to as a device. In addition, the XR device 1430 may correspond to a point cloud data (PCC) device according to an embodiment or may be operably connected to a PCC device.

[0232] The cloud network 1400 may represent a network that constitutes a part of a cloud computing infrastructure or exists in a cloud computing infrastructure. Here, the cloud network 1400 may be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

[0233] The server 1460 may be connected to at least one of the robot 1410 , the self-driving vehicle 1420 , the XR device 1430 , the smart phone 1440 , the home appliance 1450 , and / or the HMD 1470 via the cloud network 1400 , and may assist in at least a portion of the processing of the connected devices 1410 to 1470 .

[0234] The HMD 1470 represents one of implementation types of an XR device and / or a PCC device according to an embodiment. The HMD type device according to an embodiment includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0235] Hereinafter, various embodiments of devices 1410 to 1450 to which the above-described technology is applied will be described. Figure 14The illustrated devices 1410 to 1450 may be operably connected / coupled to the point cloud data transmitting and receiving devices according to the above-described embodiments.

[0236] <PCC + XR>

[0237] The XR / PCC device 1430 may employ PCC technology and / or XR (AR+VR) technology, and may be implemented as an HMD, a head-up display (HUD) disposed in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a fixed robot, or a mobile robot.

[0238] The XR / PCC device 1430 may analyze 3D point cloud data or image data acquired through various sensors or from an external device and generate position data and attribute data about 3D points. Thus, the XR / PCC device 1430 may acquire information about the surrounding space or real objects, and render and output an XR object. For example, the XR / PCC device 1430 may match an XR object including auxiliary information about the recognized object with the recognized object and output the matched XR object.

[0239] <PCC + XR + Mobile Phone>

[0240] The XR / PCC device 1430 may be implemented as a mobile phone 1440 by applying the PCC technology.

[0241] The mobile phone 1440 may decode and display point cloud content based on the PCC technology.

[0242] <PCC + Self-driving + XR>

[0243] The self-driving vehicle 1420 can be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0244] The self-driving vehicle 1420 to which the XR / PCC technology is applied may represent a self-driving vehicle provided with a means for providing an XR image, or a self-driving vehicle as a control / interaction target in the XR image. Specifically, as a control / interaction target in the XR image, the self-driving vehicle 1420 may be distinguished from the XR device 1430 and may be operably connected thereto.

[0245] The self-driving vehicle 1420 having means for providing XR / PCC images may acquire sensor information from sensors including cameras, and output generated XR / PCC images based on the acquired sensor information. For example, the self-driving vehicle 1420 may have a HUD and output XR / PCC images thereto, thereby providing passengers with XR / PCC objects corresponding to real objects or objects presented on the screen.

[0246] When the XR / PCC object is output to the HUD, at least a portion of the XR / PCC object may be output to overlap with the real object to which the passenger's eyes are directed. On the other hand, when the XR / PCC object is output on a display provided in the self-driving vehicle, at least a portion of the XR / PCC object may be output to overlap with the object on the screen. For example, the self-driving vehicle 1220 may output XR / PCC objects corresponding to objects such as a road, another vehicle, a traffic light, a traffic sign, a two-wheeled vehicle, a pedestrian, and a building.

[0247] Virtual reality (VR) technology, augmented reality (AR) technology, mixed reality (MR) technology, and / or point cloud compression (PCC) technology according to an embodiment are applicable to various devices.

[0248] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. On the other hand, AR technology refers to a technology that displays a virtually created CG image on an image of a real object. MR technology is similar to the above-mentioned AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology is different from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with properties equivalent to real objects. More specifically, an example of the application of MR technology is a holographic service.

[0249] Recently, VR, AR, and MR technologies are often referred to as extended display (XR) technologies rather than being clearly distinguished from each other. Therefore, embodiments of the present disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are applicable to such technologies.

[0250] The PCC method / apparatus according to the embodiment may be applied to a vehicle that provides a self-driving service.

[0251] Vehicles providing self-driving services are connected to the PCC device for wired / wireless communication.

[0252] When a point cloud data (PCC) transmitting / receiving device according to an embodiment is connected to a vehicle for wired / wireless communication, the device may receive / process content data related to AR / VR / PCC services (which may be provided together with self-driving services) and send it to the vehicle. In the case where the PCC transmitting / receiving device is installed on the vehicle, the PCC transmitting / receiving device may receive / process content data related to AR / VR / PCC services and provide it to the user based on a user input signal input through a user interface device. The vehicle or user interface device according to the embodiment may receive a user input signal. The user input signal according to the embodiment may include a signal indicating a self-driving service.

[0253] In the present disclosure, the method / apparatus for sending and receiving point cloud data according to an embodiment may be referred to as the method / apparatus according to an embodiment.

[0254] In this disclosure, geometry may be referred to as geometric data, geometric information, position value of a point, etc.

[0255] In the present disclosure, an attribute may be referred to as attribute data, attribute information, an attribute value of a point (color, reflectivity, etc.), or the like.

[0256] Figure 15 A geometry octree and attribute level of detail (LOD) generated based on the octree according to an embodiment are illustrated.

[0257] Figure 15 Example of Figure 1 Point cloud video encoder 10002, Figure 2 Code 20001, Figure 4 The encoder and Figure 12 The sending device generates a geometry octree 15000 and an attribute LOD 15010 during point encoding.

[0258] Figure 15 Example of Figure 1 Point cloud video decoder 10006, Figure 2 Decoding 20003, Figure 4 The decoder, Figure 11 The decoder and Figure 13 The geometric octree 15000 and attribute LOD 15010 are generated during the decoding of the bit stream containing the point cloud data by the receiving device.

[0259] Embodiments are directed to a method for improving geometry compression efficiency by providing a spatially scalable function of geometry-based point cloud compression (G-PCC) for compressing 3D point cloud data based on the locations of octree reconstruction points.

[0260] The geometric octree 15000 is generated by the octree analyzer 40002 (see Figure 5 and Figure 6 ) generates an octree of point position values.

[0261] The attribute LOD 15010 is generated by the LOD generator 40009 (see Figure 8 and Figure 9 ) for an octree-based attribute.

[0262] The geometry octree 15000 and attribute LOD 15010 may include specific levels 15030. Within these levels, low LOD indices may represent full resolution point cloud raw, while high LOD indices may represent full resolution point cloud bitstreams (see Figure 8 ).

[0263] The method / apparatus according to an embodiment may perform scalable encoding and / or decoding based on a specific level 15030 .

[0264] That is, an embodiment may generate an octree 15000 of points generated by scalable geometry encoding / decoding at a specific depth 15030 of the octree. In addition, an attribute LOD 15010 may be generated by scalable attribute encoding / decoding (lifting scheme).

[0265] Here, since the leaf node at position 15040 on the octree 15000 of the point maximally matches the position of the attribute LOD 15010 at position 15050 on the octree (as indicated by 15060), the point cloud data compression efficiency can be improved.

[0266] In this regard, an embodiment proposes the operation of an encoder / decoder that processes point cloud data so that the position of a point on a geometric octree 15040 is similar to the position 15050 of a point on an attribute octree, and the syntax of signaling information therefor.

[0267] The method / apparatus according to the embodiment may perform geometric position reconstruction for spatial scalability, and may generate and transmit / receive signaling information to support the method.

[0268] Figure 16 A method for processing a bitstream based on LOD according to an embodiment is shown.

[0269] Encoder 1602 may correspond to Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 2 Encoder 20001, Figure 4 The encoder,Figure 12 The sending device and Figure 14 XR device 1430.

[0270] Decoder 1630 may correspond to Figure 1 The receiving device 10004, Figure 1 Point cloud video decoder 10006, Figure 2 Decoder 20003, Figure 10 and Figure 11 The decoder, Figure 13 The receiving device Figure 14 XR device 1430, and Figure 17 Decoders 17000 and 17010.

[0271] Embodiments may perform geometry position encoding / decoding to support spatial scalability of Geometry-Based Point Cloud Compression (G-PCC) for compressing 3D point cloud data.

[0272] When point cloud data is dense and bandwidth is low, the method / apparatus according to the embodiment can provide the function of sending and processing point cloud with low resolution. The method / apparatus according to the embodiment can process the point cloud according to the function of the receiving device to allow the receiving device to present the point cloud data.

[0273] In addition, when the point cloud data is dense and the performance of the decoder is low, the method / apparatus according to the embodiment may provide a function of providing a low-resolution point cloud by partially decoding the data to reduce the amount of calculation on the decoder side.

[0274] As in operation 16000, encoder 16020 may receive full-resolution point cloud raw data. Encoder 16020 may construct the geometry of point cloud data as an octree. Encoder 16020 may encode only the geometry 16040 of the level up to a specific level based on LOD. Depending on the LOD level, encoder 16020 may generate a low-resolution point cloud i-th LOD or a high-resolution point cloud bitstream. Decoder 16030 may receive a low-resolution point cloud i-th LOD or a high-resolution point cloud bitstream sent from the encoder. Decoder 16030 may decode, reconstruct, and render point cloud data corresponding to a specific level 16050. Decoder 16030 may generate a low-resolution decoded point cloud or a high-resolution decoded point cloud.

[0275] As in operation 16010, encoder 16020 may receive full-resolution point cloud raw data. Encoder 1602 may encode all levels of the geometric octree, rather than a specific level. Encoder 16020 may generate a full-resolution point cloud bitstream. Decoder 16030 may receive a full-resolution point cloud bitstream. Decoder 16030 may decode and reconstruct point cloud data up to a specific level 16060. Decoder 16030 may generate low-resolution point cloud data.

[0276] like Figure 16 The data processing shown provides effects related to the spatial scalability of point cloud data.

[0277] The transmitting device or encoder according to the embodiment may partially encode and transmit the point cloud data as in operation 16000. The transmitting device or encoder according to the embodiment may completely encode and transmit the point cloud data as in operation 16010. The receiving device or decoder according to the embodiment may reconstruct the received point cloud data by partially decoding the point cloud data in a scalable manner based on a specific level of the octree.

[0278] By utilizing the spatial scalability function according to the embodiment, point cloud data can be efficiently processed according to the transmission environment, the characteristics or performance of the receiving device, etc.

[0279] Figure 17 The spatial scalability structure of the geometry and attributes according to the implementation is shown.

[0280] Decoders 17000 and 17010 may correspond to Figure 1 The point cloud video decoder 10006 of the receiving device 10004 is configured to perform a decoding operation Figure 2 The decoder 20003 is configured to decode the geometry bitstream and the attribute bitstream. Figure 11 Decoder 20003, Figure 13 The receiving device Figure 14 XR device 1430 or Figure 16 decoder 16030, or a combination thereof.

[0281] The decoder 17000 according to the embodiment can reconstruct low-resolution point cloud data with attributes. The decoder 17000 can receive partial geometry data and partial attribute data corresponding to a specific level 17020 of the octree from a geometry bitstream and an attribute bitstream represented by a level of detail (LOD), and reconstruct low-resolution point cloud data with attributes.

[0282] The decoder 17010 according to the embodiment can reconstruct full-resolution point cloud data. The decoder 17010 can receive geometry data and attribute data including levels up to the end of the octree from the geometry bitstream and attribute bitstream represented by the level of detail (LOD) level, and reconstruct full-resolution point cloud data.

[0283] Geometry encoding and geometry decoding according to an embodiment may provide a spatial scalability function by encoding or decoding only occupied bits of a depth level (eg, 15030, 16040, 16050, 16060) selected by adjusting the depth level of an octree.

[0284] In order to provide spatial scalability functionality according to embodiments, the encoder and / or decoder may need to re-arrange the geometry values.

[0285] The encoder and / or decoder of the transmitting / receiving device according to the embodiment can increase the geometric compression efficiency by reconstructing the position (eg, geometry) of the point based on the octree for the spatial scalability function.

[0286] Furthermore, the spatial scalable function according to the embodiment may be combined with other embodiments. That is, the terms used in the present specification may be understood based on the intended meaning of the terms within a range widely adopted in the relevant art.

[0287] The method / apparatus according to the embodiment may apply the same octree structure to geometry to provide spatial scalability in encoding and / or decoding (see Figure 17 ). A scheme is proposed, wherein a method / apparatus according to an embodiment constructs partial geometry up to a selected depth level 17020 of an octree from the entire geometry bitstream. The method of reconstructing geometry for each LOD may be performed by a PCC decoder / encoder according to an embodiment.

[0288] Figure 18 The operation of geometric encoding according to an embodiment is shown.

[0289] Figure 18 Show that when the geometry is Figure 1 The transmitting device 10000, the point cloud video encoder 10002, Figure 2 Encoder 20001, Figure 4 The encoder, Figure 12 The sending device Figure 14 The XR device 1430 is used to support the encoding of Figure 15 and Figure 16 Operation of spatial scalability is shown.

[0290] S18000: Quantization for Lossy Geometry Coding

[0291] The encoder according to the embodiment may perform lossy geometry coding. For lossy geometry coding, the encoder according to the embodiment may quantize the geometry. Figure 19 A detailed operation of quantization according to an embodiment is described.

[0292] S18010: Partial geometry coding

[0293] According to an embodiment, the encoder may partially encode the geometry. Figure 20 Partial geometry encoding according to an embodiment is described.

[0294] Figure 19 Floor quantization and round quantization according to an embodiment are shown.

[0295] Figure 19 Shows Figure 18 Detailed operation of S18000 in;

[0296] Figure 19 The operation of represents the operation of quantizing geometry according to the encoder of the embodiment. For example, Figure 4 Quantizer or voxelizer 40001 or Figure 12 The voxelization processor 12002 can perform Figure 19 Quantification shown.

[0297] The encoder according to the embodiment may perform quantization by multiplying the geometry position value by a scaling value for lossy geometry coding. When geometry is quantized, the position value represented by the geometry data is changed.

[0298] Quantization according to an embodiment may include ceiling, floor, and rounding operations.

[0299] An encoder according to an embodiment may quantize geometry based on various quantization types for spatial scalability. An encoder may send quantization type information applied to geometry in a point cloud bitstream. A decoder according to an embodiment may reconstruct the position of geometry based on the quantization type information contained in the bitstream.

[0300] According to the embodiment, the rounding down quantization 19000 can change the geometric point position value by the rounding down operation. The point 19020 in the voxel 19030 can have a position value (1.2, 1.1, 1.4) in the coordinate system. When the encoder quantizes the point 19020 by the rounding down operation, the decimal part of each position value can be removed. For example, the position value (1.2, 1.1, 1.4) is quantized to the position value (1, 1, 1). The point 19050 in the voxel 19060 can have a position value (0.6, 0.5, 0.6) in the coordinate system. When the encoder quantizes the point 19050 by the rounding down operation, the position value (0.6, 0.5, 0.6) is quantized to (0, 0, 0).

[0301] According to the rounding quantization 19010 of the embodiment, the point position value of the geometry can be transformed into an approximate integer. The point 190804 in the voxel 19090 can have a position value (1.2, 1.1, 1.4). 1.2, 1.1 and 1.4 are between 1 and 2. In the rounding quantization, each coordinate value can be set to the nearest value between 1 and 2. That is, (1.2, 1.1, 1.4) is quantized to (1, 1, 1). The point 19100 in the voxel 19110 can have a position value (0.6, 0.5, 0.6). The position value (0.6, 0.5, 0.6) can be quantized to a position value (1, 1, 1). According to the embodiment, 0.5 can be quantized to 1 by rounding. According to the setting of the embodiment, the point at the center between two integers (for example, 0.5) can be quantized to 0 or 1.

[0302] The method / apparatus according to the embodiment may set the representative position of the node of the octree based on a quantization technique (eg, rounding up, rounding down, etc.).

[0303] like Figure 19 As shown, the encoder according to the embodiment can perform lossy encoding by quantizing the geometric position value.

[0304] According to an embodiment, the encoder may quantize the points along directions 19140, 19150 according to the encoding order.

[0305] Quantization according to an embodiment may be used for lossy geometry coding. When lossless coding is applied instead of lossy coding, the quantization operation may be optional.

[0306] Figure 20 An operation of reconstructing the geometry (position) on an intermediate node of an octree according to an embodiment is shown.

[0307] Figure 20 Shows Figure 18 Detailed operation of S18010 in.

[0308] An encoder according to an embodiment may generate and send a partial geometry bitstream by encoding only the octree up to a certain depth level of the octree for spatial scalability, rather than encoding the entire octree (up to the leaf nodes of the octree).

[0309] like Figure 15 and Figure 16 As shown, occupied bits up to a certain depth level of the octree may be generated, and the encoder according to the embodiment may reconstruct geometric position information based on the encoded geometric information to encode the attribute.

[0310] like Figures 18 to 20 As shown, when reconfiguring the geometric position, the encoder according to the embodiment can set the octree intermediate node ( Figure 20 The LOD N in octree is used instead of the representative position of the leaf node of the octree, and the position is reconstructed as a point.

[0311] The representative position can be set to the left, bottom, front point of the node area (or bounding box, voxel, etc.) 20020, 20070, 20090, 20100 (20000), or to the middle / center position. It can be set to the center of mass according to the distribution (20010). The representative position can be set to various other corner positions.

[0312] The middle position according to the embodiment may refer to the middle position of each node area in LOD N (the middle value of the bounding box, regardless of the point position within the node).

[0313] The centroid according to an embodiment is an actual center value of a point in each node area included in LOD N. After checking whether the centroid is a left / lower / front point of a secondary node including the point or the center of the secondary node, the type of its position and the index (0 to 7) of the secondary node are signaled to the decoder through signaling information.

[0314] The method / device according to the implementation mode reconstructs the geometric value as a representative position when reconstructing the geometric position. This reconstruction can ensure a better match between the geometric position and the result from the attribute encoding. This is because in the attribute encoding, the weight (distance) is calculated by shifting to the position representing the middle position. In addition, in partial decoding, when the position at the middle level is reconstructed as bottom / left / front, the value may not be the actual value. That is, it is a virtual value. This is because the position of the actual point in the area is known only after the data is decoded to the leaf level. Therefore, using the center point as the representative position can improve the encoding / decoding efficiency.

[0315] In partial decoding according to an embodiment, for a specific reason, for example, because the decoder does not want, cannot, or need to decode the entire level, only data up to level L, which is a specific level among the multiple levels, may be reconstructed. Therefore, not all representative positions of nodes from level 0 to level N are reconstructed, but the representative positions are reconstructed according to the occupied bits of each octree node at level L expected by the decoder. When the occupied bit of the octree node is 0, the corresponding representative position is not reconstructed.

[0316] The encoder according to the embodiment can represent point cloud data, such as geometry in an octree structure. The octree is composed of a node area 20020 and points included in the node area 20020 according to the octree level.

[0317] The octree level may be represented by LOD 20030. LOD is from LOD 0 to LOD N, and the size of the node area decreases as the LOD index increases to N. In a level corresponding to a leaf node, the node area may correspond to a voxel.

[0318] As in operation 20000 , the encoder according to an embodiment may represent a geometric point located in a space in an octree structure based on a point distance such as LOD.

[0319] For example, in level LOD N-1, node area 20020 may include one or more points. The encoder according to the embodiment may set the representative position of the point of node area 20020 to the left, lower, and front positions 20040. The point of node area 20020 may be considered as the point of representative position 20040. In the octree structure, the node of level LOD N-1 may include up to 8 nodes as secondary nodes (or child nodes). One or more nodes included in level LOD N may also set the representative position of each node area to the left, lower, and front positions 20050 of each node area.

[0320] In operation 20010 , compared to operation 20000 , the encoder according to the embodiment may set the representative position of each node region to a middle position 20060 of the node region.

[0321] For example, at level LOD N-1, the representative position of the node area 20070 may be the center. In the octree structure, the node area 20070 may include up to eight sub-nodes (or child nodes). At level LOD N, one or more nodes may use the middle position of each node as the representative position 20080 of one or more points included in the node.

[0322] The representative position type of the octree node according to the embodiment may be notified by a signal. The encoder according to the embodiment may send information indicating the representative position type of the octree node according to the embodiment (representing operations 20000, 20010, etc.) as metadata, signaling information or parameters through the point cloud bitstream. The receiving device according to the embodiment may reconstruct the position of the geometry included in the point cloud bitstream based on the representative position type related information included in the bitstream.

[0323] When the encoder according to an embodiment encodes an attribute based on the LOD, the encoder may search for neighboring points while considering the center as a representative position of the attribute.

[0324] For example, when a weight calculation method is used in attribute encoding, the reference distance may be calculated based on an offset to the center.

[0325] Figure 20 A spatially scalable geometry coding approach compared to attribute coding is shown.

[0326] When the encoder according to an embodiment sets the centroid of a point according to the distribution of the point in space, the encoder may calculate the position to which the centroid at the sub-level position belongs at the current octree depth level (0 to N). The centroid may be set to the left, bottom, front, middle, or corners of the sub-level.

[0327] The encoder or metadata processor according to the embodiment may add the lower-level position index and the lower-level representative position type as signaling information or parameters to the point cloud bitstream. Based on the signaling information, the decoder according to the embodiment may reconstruct the partial geometric position.

[0328] A representative value of a representative bit type applied in partial geometry encoding may be set according to a quantization type applied in quantization of lossy geometry.

[0329] For example, when the quantization type is rounding, the representative position type can be set to the center. When the quantization type is rounding up, the representative position type can be set to the right-upper-back position. The quantization type and / or representative position type according to the implementation method can be transmitted in the bitstream as a parameter or signaling information. The decoder can reconstruct the partial geometric point position based on the parameter or signaling information.

[0330] The method according to the embodiment can be applied to an encoder and / or a decoder in the same manner. In attribute encoding and / or attribute decoding, attribute encoding / decoding can be performed based on the same point.

[0331] When partial geometry is encoded, not only the depth of the full octree but also the depth level of the octree used for encoding can be transmitted in the bitstream as signaling information.

[0332] Will refer to Figure 24 The signaling information and the structure of the point cloud bitstream according to the implementation manner are described in detail.

[0333] Figure 20 The geometric position reconstruction can be operated through the following syntax.

[0334] for(child=0;child <GeometryNodeChildrenCnt;child++){

[0335] childIdx=GeometryNodeChildren[child];

[0336] x=2×xN+(childIdx&4)==1;

[0337] y=2×yN+(childIdx&2)==1;

[0338] z=2×zN+(childIdx&1)==1;

[0339] for(i=0;i <GeometryNodeDupPoints[child]+1;i++,PointCount++){

[0340] PointPos[PointCount][0]=geomScale(x,0);

[0341] PointPos[PointCount][1]=geomScale(y,1);

[0342] PointPos[PointCount][2]=geomScale(z,2);

[0343] }

[0344] if (direct_mode_flag) {

[0345] if(!duplicatedPointFlag){

[0346] for(i=0;i<=num_direct_points_minus1;i++,PointCount++){

[0347] PointPos[PointCount][0]=geomScale((x<<EffectiveChildNodeSizeLog2)+PointOffsetX[i],PointPos[PointCount][1]=geomScale((y<<EffectiveChildNodeSizeLog2)+PointOffsetY[i],PointPos[PointCount][2]=geomScale((z<<EffectiveChildNodeSizeLog2)+PointOffsetZ[i],}

[0348] }

[0349] else{

[0350] for(i=0;i<=num_direct_points_minus1;i++,PointCount++){

[0351] PointPos[PointCount][0]=geomScale((x<<EffectiveChildNodeSizeLog2)+PointOffsetX[0],PointPos[PointCount][1]=geomScale((y<<EffectiveChildNodeSizeLog2)+PointOffsetY[0],PointPos[PointCount][2]=geomScale((z<<EffectiveChildNodeSizeLog2)+PointOffsetZ[0],}

[0352] }

[0353] }

[0354] }

[0355] if(minGeomNodeSizeLog2>1){

[0356] mask=(-1)<<minGeomNodeSizeLog2;

[0357] for(i=0;i<PointCount;i++){

[0358] PointPos[i][0]=(PointPos[i][0]&mask)+(1<<(minGeomNodeSizeLog2-1))

[0359] PointPos[i][1] = (PointPos[i][1] & mask) + (1 << (minGeomNodeSizeLog2 - 1))

[0360] PointPos[i][2] = (PointPos[i][2] & mask) + (1 << (minGeomNodeSizeLog2 - 1))

[0361] }

[0362] }

[0363] The method / apparatus according to the embodiment can set the positions of the nodes at a specific octree level (e.g., LOD 0 to N) to central values (e.g., 20060, 20080) through bit operations. That is, the & and << operations leave or delete 0 / 1 bits to move the position to the center.

[0364] The level of the octree can be set to minGeomNodeSizeLog2.

[0365] To apply a specific octree level, the Mask = (-1) << minGeomNodeSizeLog2 operation can generate bits based on the 0 and & operations to delete the rightmost digits.

[0366] The PointPos[i][0] & mask operation can delete the right digits from the position value. Then, the position values of the left, lower, and front of the octree at this level can be generated.

[0367] The +(1 << (minGeomNodeSizeLog2 - 1)) operation can obtain the size value of a sub - octree node. By adding the corresponding values, the middle position of the octree level can be obtained.

[0368] Figure 20 The operation of geometric position reconstruction can be performed by a geometric encoder or a point cloud encoder. In addition, geometric position reconstruction is performed in association with attribute encoding and / or attribute decoding. That is, when reconstructing the geometric position, the point cloud data can also be reconstructed.

[0369] When reconstructing point cloud data, when not all levels are reconstructed or the data is only reconstructed until the middle level (partial decoding, 16030), the reconstructed value can be set to the left / bottom / front value (i.e., a virtual value) by default. It is not known which secondary octree node is occupied at the middle node. Left / bottom / front is also a virtual value. When encoding and / or decoding attributes, the weights of the nearest neighbors (NN) of each point need to be calculated. Therefore, the weight (distance) is calculated after moving the value of the node to the middle position value.

[0370] Therefore, when reconstructing geometry at an intermediate level rather than at the entire level (17020), reconstructing the position of the geometry as a center value according to the weight calculation method for the attribute can enhance the match between the reconstructed point and the reconstructed color, thereby achieving a very high PSNR.

[0371] Reconstructing the geometry to the left / bottom / front point can reduce the error caused by mismatch when the weights of the NN are calculated based on the assumption that the attributes are located in the center. In partial decoding, since both geometry / attributes are reconstructed to the center (20060, 20080) to match, the PSNR can be improved.

[0372] The representative position can be reconstructed to decode only data up to a certain level. That is, partial decoding is possible. According to an embodiment, encoding can be performed at the full level (16010). In addition to shifting the attribute encoding NN weights to the center in the attribute encoding, the geometric position reconstruction can provide the above effect.

[0373] When the weight is estimated by calculating the distance to NN in attribute encoding, the positions of both the target point and the point of NN can be changed to the center, and the distance thereof can be calculated. That is, scalability encoding is possible. In partial decoding or full-level decoding, the same weight can be applied.

[0374] Figure 21 The operation of partial geometry decoding according to an embodiment is shown.

[0375] Decoders 17000 and 17010 correspond to the operation of partial geometry decoding, which consists of Figure 1 The point cloud video decoder 10006 of the receiving device 10004 is configured to perform decoding Figure 2 The decoder 20003 is configured to decode the geometry bitstream and the attribute bitstream. Figure 13 The decoder, Figure 14 XR device 1430, Figure 16 The decoder 16030 executes.

[0376] Operation 21000 represents a partial geometry decoding operation according to an embodiment.

[0377] To provide spatial scalability, not all of the octree (up to the leaf nodes of the octree) may be encoded, but only a portion of the octree up to a certain depth level may be encoded to generate a partial geometry bitstream and send the generated bitstream to a decoder. For example, partial geometry encoding may be used to support thumbnails.

[0378] In addition, although the full octree bitstream is encoded, there may be a situation where the decoder needs thumbnails first and quickly due to the capacity or lack of resources of the decoder or according to the scene. In this case, the decoder may not decode all of the received octree-based bitstream, but may only decode a portion of the bitstream up to a certain depth level of the octree.

[0379] The geometric decoding may include the following three cases. The receiving device and / or the decoder according to the embodiment may respectively perform the following three cases according to the embodiment.

[0380] Case 1 (S21000): receiving a full geometry bitstream (full geometry encoding) and reconstructing geometry position information based on a full octree (full geometry decoding).

[0381] The receiving device or decoder according to the embodiment receives a bitstream containing the entire geometry at full resolution from the transmitting device or encoder. The receiving device can reconstruct the entire position information about the geometry from the entire geometry based on the octree.

[0382] Case 2 (S21010): receiving the full geometry bitstream (full geometry encoding) and reconstructing the geometry position information based on the partial octree (partial geometry decoding).

[0383] The receiving device receives a bitstream containing the entire geometry at full resolution from the transmitting device. The receiving device can reconstruct partial position information of the geometry up to a certain level from the octree-based geometry (spatial scalability).

[0384] Case 3 (S21020): Receive a partial geometry bitstream (partial geometry encoding) and reconstruct the entire received octree (partial geometry decoding).

[0385] The receiving device receives the partial geometry bitstream from the transmitting device. The receiving device can reconstruct the entire octree corresponding to the partial geometry.

[0386] Case 2 (S21010) and / or Case 3 (S21020) may correspond to partial geometry decoding. Case 2 (S21010) may correspond to Figure 16 Operation 16010, while case 3 (S21020) may correspond to Figure 16 Operation 16000.

[0387] For partial geometry decoding, the receiving device according to the embodiment may perform the following operations S21030 and S21040 using the information described below.

[0388] S21030: Based on the quantization type information, decode the geometry and reconstruct the position of the geometry.

[0389] Based on the quantization type information contained in the bitstream received from the encoder, the decoder can decode the geometry and reconstruct the position value of the geometry.

[0390] S21040: Based on the representative position information, decode the geometry and reconstruct the position of the geometry.

[0391] Based on the representative position information contained in the bitstream received from the encoder, the decoder can decode the geometry and reconstruct the position value of the geometry.

[0392] The receiving device may decode the position of the partial geometry according to the implementation in the following operations S21050 and S21060.

[0393] S21050: Parse the occupied bits up to a certain level of the octree and construct the octree (partial decoding).

[0394] The decoder may parse the occupied bits included in the received geometry. Then, the occupied bits corresponding to the octree up to a certain level are parsed for partial decoding. The octree may be constructed based on the parsed occupied bits.

[0395] In other words, occupied bits of an octree up to a certain level may be parsed from the geometry bitstream for partial decoding, and the octree may be constructed. Nodes may be constructed according to occupied bits at a certain level.

[0396] S21060: Reconstruct the position of the node.

[0397] The decoder can reconstruct the position value of the constructed node. In the process of constructing the position value, the position within the node can be calculated as the same position as that obtained by the quantization type. For example, Figure 19 As shown, the position of point 19080 quantized as point 19120 can be reconstructed and calculated.

[0398] Figure 22 The structure of a point cloud data encoder according to an embodiment is shown.

[0399] Figure 22 It shows that according to Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 2 Encoder 20001,Figure 4 The encoder, Figure 12 The sending device Figure 14 XR device 1430 and Figure 16 Detailed block diagram of an encoder of an embodiment of encoder 16020. Each element corresponds to hardware, software, processor and / or a combination thereof.

[0400] Figure 22 Each element of may be always included in the encoder according to the embodiment, or may be selectively included in the encoder. Signaling information about the operation of the encoding elements of the encoder may be transmitted in the bitstream.

[0401] The transmitting device according to the embodiment or the encoder included in the transmitting device encodes the point cloud data and generates a geometry information bit stream and / or an attribute information bit stream.

[0402] The data input unit 22000 may receive geometric data, attribute data and / or parameters related thereto.

[0403] The coordinate transformer 22010 may transform the coordinates related to the position (coordinate) information of the geometric data. The coordinate transformer 17010 may correspond to Figure 4 The coordinate transformer 40000 in.

[0404] The geometric information transform quantization processor 22020 may be referred to as a geometric information transform quantizer. The geometric information transform quantization processor 22020 may correspond to Figure 4 Quantizer 40001 in. The geometric information transform quantization processor 22020 can receive a completed scale value (quantization value) and adjust the scale according to the distribution of the content.

[0405] The spatial segmenter 22030 may segment the point cloud data into spatial segmentation units according to the implementation. The spatial segmenter 22030 may segment the data space spatially adaptively using tiles, blocks, density-based methods, axis-based methods, attribute-based methods, geometry-based methods, etc. according to the data characteristics of the point cloud data. The spatial segmenter 22030 may perform data segmentation. A frame may be divided into tiles and slices. The source point cloud data may be segmented into multiple slices and encoded in a bitstream. A slice is a set of points that can be independently encoded or decoded. A slice may include a geometric data unit and zero or one or more attribute data units. The attribute data unit may depend on the corresponding geometric data in the same slice. In a slice, a geometric data unit may appear before a related attribute unit. The data units in a slice are continuous. A group of slices may be identified by a common tile identifier. Tile information describing the bounding box of each tile may be present in the bitstream. Tiles may overlap with other tiles in the bounding box. Each slice may include an index for identifying the tile to which it belongs.

[0406] The geometry information encoder 22040 may encode the geometry data of the point cloud data. The detailed process of geometry encoding may be performed by the following elements. The geometry information encoder 22040 may allow the octree generator to generate an octree based on the quantized point cloud, and allow the color re-adjuster to use the generated octree.

[0407] In order to provide spatial scalability, the geometry information encoder 22040 (eg, the octree generator 22060) may set a depth level of the octree to set a point to be encoded from the set octree depth level.

[0408] The octree depth level applied to the partial geometry may be included in the bitstream as a parameter and signaled to the decoder. The octree depth level setting environment may be changed according to the use purpose or computational performance of the decoder.

[0409] The voxelization processor 22050 may voxelize the geometric data. The voxelization processor 22050 may correspond to Figure 4 Voxelizer 40001 in .

[0410] The voxelization type can be varied according to the implementation (see Figure 18 and Figure 19 ) quantizes the geometry according to the quantization type setting.

[0411] The quantization type applied to the geometry by the encoder according to the embodiment may be included in a bitstream as a parameter (signaling information) and transmitted to the receiving device according to the embodiment.

[0412] As described above, the voxelization processor 22050 can signal the quantization type used in lossy geometry encoding to the decoder so that the quantization type can be used in the encoder and / or decoder geometry reconstructors 22090 and 23030. The quantization type can include ceiling, floor, or rounding.

[0413] The octree generator 22060 may correspond to Figure 4 The octree analyzer 40002 in. The octree generator 22060 can generate geometric data in an octree structure.

[0414] When using a geometric octree, geometric encoding can be performed as follows: 1) A bounding box defined by two points can be defined. The octree structure is constructed by recursively subdividing the bounding box. Each cube is divided into 8 sub-cubes. The 8-bit code can be called an occupancy code. When the point is occupied, one bit of each sub-cube can be represented as 1, or when the point is empty, it is represented as 0. Sub-cubes with sides greater than 1 are further divided. Sub-cubes of size 1 can be voxels. Multiple points can overlap and map to the same sub-cube (voxel). The points of each sub-cube can be arithmetically encoded.

[0415] The decoding process can be started by reading the dimensions of the bounding box from the bitstream. The same octree structure can be generated by dividing the bounding box according to the occupancy code. When a subcube of size 1 is reached, the points of the subcube can be arithmetically decoded.

[0416] The octree generator 22060 can generate an octree for the geometry. It can generate an octree for the entire geometry or an octree for a portion of the geometry.

[0417] Depending on the index of the depth level according to the embodiment, the octree generator 22060 may generate an octree including leaf nodes of all levels (16000), or may generate a partial octree including nodes up to a specific level (16010). The transmitting device may include the depth level of the octree as a parameter (signaling information) in the bitstream and transmit it to the receiving device.

[0418] The geometry information predictor 22070 may perform prediction on the geometry data. The geometry information predictor 22070 may generate a prediction value for the geometry data between the parent node and the child node in the octree structure. A residual bit value may be generated by the prediction value to perform entropy encoding on the residual to construct a geometry bitstream. Efficient bitstream transmission may be achieved by encoding only the residual of the prediction value. The geometry information predictor 22070 may reconstruct (reconfigure) the geometry data from the geometry octree for attribute encoding. This is because attribute encoding is performed based on the reconstructed geometry data.

[0419] The geometry information predictor 22070 may not always be executed. There is an operation option. The geometry information prediction is performed according to the option value. For example, whether to perform the operation can be notified by the AdjacentChildNeibhgour flag.

[0420] The value generated by the geometry information predictor 22070 may indicate whether the current octree node is occupied. The value is sent and input to the geometry position reconstructor 22090.

[0421] It is not important for the geometry reconstructor 22090 whether the input value of the geometry reconstructor 22090 is an occupied bit predicted by the geometry information predictor 22070. That is, it is only important for the geometry reconstructor 22090 whether the node is occupied.

[0422] The geometry information entropy encoder 22080 may perform entropy encoding on the geometry data to generate a geometry bitstream.

[0423] The geometry position reconstructor 22090 may reconstruct the geometry. That is, the geometry encoded by the geometry encoder 22040 may be reconstructed. The geometry position value is reconstructed. The reconstructed geometry position value is provided to the attribute encoder 22100.

[0424] The geometric position reconstructor 22090 may receive the representative position type of each octree node and reconstruct the geometric position value according to the representative position type (see Figure 20 ). The representative position type used in the encoder can be included in the parameters in the bitstream and provided to the decoder. The representative position type can include, for example, the quantization type (see Figure 19 ) types of use, such as left, bottom, front / right, bottom, front corner (see Figure 20 20000) of any angular position, the choice of the intermediate position (see Figure 20 20010), and the choice of the middle position.

[0425] The middle position is the center of the bounding box that is the node area.

[0426] When selecting the position of the centroid, the geometric position reconstructor 22090 can analyze the distribution of the points and send the index of the node at the lower level of the octree depth level to which the calculated centroid value belongs as signaling information (parameters) in the bitstream to the decoder. For example, the index 0 to 7 of the lower level node can be sent.

[0427] When selecting the position of the centroid, the geometric position reconstructor 22090 can reconstruct the geometric position value according to the representative position type of the position to be reconstructed based on the node index of the lower level where the centroid is located. The representative position type can be included in the bitstream as parameter information and sent to the decoder. The representative position type can include the use of types such as quantization type, the selection of any angular position such as left, bottom, front / right, bottom, front corner, the selection of the middle position, and the selection of the middle position.

[0428] According to an embodiment, the encoder can send data completely and the decoder can partially decode the data. According to an embodiment, the encoder can partially encode and send data. When the encoder partially encodes and sends data, the reconstruction operation performed by selecting the centroid position may additionally require the index of the lower-level node because there is no additional information on the child node. Therefore, the index of the lower-level node can be sent to the decoder. Since there are up to 8 secondary nodes, the encoder can send the index value of the secondary node corresponding to the centroid position value among the secondary nodes to the decoder.

[0429] The node index received in decoding becomes the centroid. When the encoder fully encodes and sends data at full resolution, the decoder does not need to receive the node index of the lower level. This is because the decoder can find the lower level and get the centroid. However, when the encoder performs partial encoding instead of full resolution encoding, the decoder can receive the centroid point index and can set it to be the same as the representative position value.

[0430] The attribute information encoder 22100 can encode attribute data.

[0431] The attribute information encoder 22100 may encode attributes based on the reconstructed geometry according to the representative location type.

[0432] In case of partial encoding, attributes may be encoded based on representative positions of nodes of an octree based on an octree including up to a certain level.

[0433] The attribute information encoder 22100 may include the following elements.

[0434] The color conversion processor 21100 can convert the color of the attribute data according to the data characteristics.

[0435] When overlapping points are merged into the reconstructed geometry octree structure, a color rescaler (not shown) may perform reshading, or perform RAHT attribute encoding, or configure LOD according to prediction / lifting transform attribute encoding.

[0436] The LOD configurator (not shown) may configure the LOD according to the point cloud data. The LOD configurator may generate the LOD according to the attribute data recolored based on the reconstructed geometry information (geometry data). The LOD may be generated when a prediction / lifting transform is applied during attribute encoding instead of RAHT.

[0437] A neighbor point set configurer (not shown) configures neighbor point sets in the LOD and higher LODs for points belonging to the LOD based on the LOD.

[0438] After the LOD1 set is generated, X (>0) nearest neighbors can be found in the group of the same or lower LOD (the group with long distances between nodes) based on the LOD1 set, and registered in the predictor as a neighboring point set.

[0439] Each point can have a single predictor. Attributes are predicted from neighboring points registered in the predictor.

[0440] The predictor can be used to predict the attribute. The average value of the value obtained by multiplying the attribute of the registered adjacent points by the weight can be used as the prediction result, or a specific point can be used as the prediction result. As for which method to use, the compression result value can be calculated in advance, and then the method that can generate the minimum flow can be selected.

[0441] The residual between the attribute value of the point and the attribute value predicted by the predictor of the point can be encoded together with the method selected by the predictor and sent to the receiver.

[0442] The decoder can decode the transmitted prediction method and predict the property value according to the method. The property value can be reconstructed by decoding the transmitted residual and adding the predicted value to it.

[0443] The attribute information predictor (not shown) predicts the attribute value of a point based on the neighboring point set. For example, when encoding a point Px, when the neighboring point set of Px is configured, the attribute value of Px can be predicted based on the attribute values ​​of the neighboring points in the set, the residual between the predicted attribute value and the attribute value of Px can be estimated, and the residual value can be encoded and sent to the decoder. The decoder configures the neighboring point set of Px through LOD generation and the received additional information, predicts the attribute value of Px based on the attribute values ​​of the neighboring points in the neighboring set, and reconstructs the attribute value of Px by adding the received residual to the predicted value.

[0444] The residual attribute information quantization processor (not shown) may generate a predicted value of the attribute data, produce a residual between the predicted value and the attribute value, and quantize the value of the residual.

[0445] The attribute information entropy encoder 22120 may perform entropy encoding on the attribute data and generate an attribute bitstream. For example, the residual may be encoded and an attribute bitstream may be generated.

[0446] The transmitting device according to the embodiment may generate signaling information (metadata) or parameters indicating operations related to geometry coding and / or attribute coding. The transmitting device according to the embodiment may generate and transmit a bitstream containing coded geometry data, coded attribute data, related metadata, etc. (see Figures 24 to 31 ).

[0447] The point cloud data transmitting method according to the embodiment may include encoding the point cloud data and transmitting a bit stream including the point cloud data.

[0448] According to an embodiment, encoding of point cloud data may include encoding geometric data of the point cloud data and encoding attribute data of the point cloud data.

[0449] According to an implementation, encoding of attribute data may include: generating a level of detail (LOD) based on point cloud data, searching for nearest neighboring points of a current point based on the LOD, generating a predicted value of an attribute value of the current point based on the nearest neighboring points, and encoding a residual between the attribute value of the current point and the predicted value.

[0450] Encoding the geometry according to an embodiment may include voxelizing the geometry or representing the geometry based on an octree.

[0451] The encoding according to an embodiment may also include reconstructing the position of the geometry. The reconstruction of the position of the geometry may reconstruct the position of the geometry based on an octree of the geometry up to a certain level of the octree.

[0452] The reconstruction of the position of the geometry according to an embodiment may reconstruct the position of the geometry based on the center position of each octree node at a specific level.

[0453] Figure 23 The structure of a point cloud data decoder according to an embodiment is shown.

[0454] Figure 23 The decoder can correspond to Figure 1 The point cloud video decoder 10006 of the receiving device 10004, Figure 2 Decoding 20003, Figure 10 and Figure 11 A decoder configured to decode the geometry bitstream and the attribute bitstream Figure 13 A receiving device, and a device configured to process point cloud data Figure 14 The XR device 1430 of the embodiment of the present invention may be a combination thereof. Each element may correspond to hardware, software, a processor and / or a combination thereof.

[0455] Figure 23 The respective elements of the decoder may be always included in the encoder according to the embodiment, or may be selectively included in the encoder. The operations of the respective elements of the decoder may be performed based on signaling information about the operations of the encoding elements of the encoder.

[0456] The receiving device according to the embodiment or the decoder included in the receiving device can receive a bitstream. The bitstream contains encoded geometry data, encoded attribute data and related metadata. The receiving device according to the embodiment decodes and reconstructs the point cloud data based on the bitstream and / or the metadata (parameters) related to geometry encoding and the metadata (parameters) related to attribute encoding contained in the bitstream.

[0457] The geometry information decoder 23000 may receive the geometry information bitstream and decode the geometry information. The geometry information decoder 23000 may be a decoder on the receiving side corresponding to the geometry information encoder 22040 on the transmitting side. The geometry information decoder 23000 may perform a process opposite to the operation of the geometry information encoder 22040.

[0458] The geometry information entropy decoder 23010 may receive the geometry information bitstream and decode the geometry information based on an entropy scheme. The geometry information entropy decoder 23010 may perform a process opposite to the operation of the geometry information entropy encoder 22080.

[0459] The octree reconstructor 23020 may reconstruct the octree according to the geometric information. The octree reconstructor 23020 may perform the inverse process of the operation of the octree generator 22060.

[0460] The octree reconstructor 23020 can construct and decode points to be attribute decoded from the reconstructed octree depth levels for partial geometry decoding.

[0461] The octree reconstructor 23020 can generate a complete or partial octree based on information related to the partial octree levels contained in the received bitstream.

[0462] The geometric position reconstructor 23030 may reconstruct the position value of the node reconstructed according to the occupied bits in the octree partially decoded by the restored quantization type or the restored representative position type.

[0463] The geometric position reconstructor 23030 can reconstruct the geometric position based on information related to the quantization type and the representative position type included in the received bitstream.

[0464] The geometry information predictor 23040 may generate a prediction value of the geometry information. For example, an inter-node prediction value may be generated in an octree structure, and the residual of the prediction value may be used to efficiently decode the geometry information. The geometry information predictor 23040 may perform a process opposite to the operation of the geometry information predictor 22070.

[0465] The geometry information transform inverse quantization processor 23050 may inverse quantize the geometry data. For example, the geometry information transform inverse quantization processor 23050 may obtain a scaling value (quantization value) from signaling information (parameters) included in the bitstream, and may apply inverse quantization to the reconstructed geometry information based on the scaling value.

[0466] The coordinate inverse transformer 23060 may inversely transform the coordinate information about the geometric information. The coordinate inverse transformer 23060 may perform an inverse process of the operation of the coordinate transformer 22010.

[0467] The attribute information decoder 23070 may receive the attribute information bitstream and decode the attribute information. The attribute information decoder 23070 may perform a process opposite to the operation of the attribute information encoder 22100.

[0468] The attribute residual information entropy decoder 23080 may decode the attribute residual information (eg, the residual of the attribute information) contained in the attribute information bitstream based on an entropy scheme. The attribute residual information entropy decoder 23080 may perform a process opposite to the operation of the attribute information entropy encoder 22120.

[0469] The attribute information decoder 23070 may also include the following elements.

[0470] An LOD configurator (not shown) may configure LODs based on point cloud data. LOD configurator 22080 may generate LODs from recolored attribute data based on reconstructed geometry information (geometry data). LODs may be generated when a prediction / lifting transform is applied during attribute encoding instead of RAHT. The LOD configurator may correspond to Figure 22 The attribute information encoder includes the LOD configurator.

[0471] A neighboring point set configurator (not shown) may search for neighboring points of a point included in the LOD based on the LOD generated from the attribute information bitstream and register them as a neighboring point set. The neighboring point set configurator may correspond to Figure 22 The adjacent point set configurator included in the attribute information encoder.

[0472] The attribute information predictor (not shown) may generate a predicted value of the attribute information. For example, the attribute information predictor may generate a predicted value of a point from the LOD and a set of neighboring points, and generate a residual of the predicted value. The attribute information predictor may perform the same operation as that included in Figure 22 The operation of the attribute information predictor in the attribute information encoder is the opposite of the operation of the attribute information predictor in the attribute information encoder.

[0473] The residual attribute information inverse quantization processor 23090 may inverse quantize the residual attribute information, for example, inverse quantize the residual of the attribute information. The residual attribute information inverse quantization processor 23090 may execute the steps included in Figure 22 The inverse process of the operation of the residual attribute information quantization processor in the attribute information encoder.

[0474] The color inverse conversion processor 23100 may inversely convert the color information as the attribute information. The color inverse conversion processor 23100 may perform an inverse process of the operation of the color conversion processor 22100.

[0475] A point cloud data receiving apparatus according to an embodiment may include: a receiver configured to receive a bit stream containing point cloud data; and a decoder configured to decode the bit stream.

[0476] According to an embodiment, a decoder configured to decode point cloud data includes: a geometry decoder configured to decode geometry data of the point cloud data; and an attribute decoder configured to decode attribute data of the point cloud data.

[0477] According to an embodiment, an attribute decoder configured to decode attribute data generates a level of detail (LOD) based on point cloud data, searches for nearest neighboring points of a current point from the LOD, generates a predicted value of an attribute value of the current point based on the nearest neighboring points, and reconstructs the point cloud data based on the predicted value and a residual of the point cloud data, wherein the predicted value is generated based on a prediction mode.

[0478] A geometry decoder configured to decode geometry according to an embodiment may represent geometry based on an octree.

[0479] A decoder according to an embodiment may reconstruct the position of the geometry based on the octree of the geometry up to a certain level of the octree.

[0480] A decoder according to an embodiment may reconstruct the position of geometry based on the intermediate position of each octree node at a particular level.

[0481] Figure 24 The structure of a bit stream associated with point cloud data according to an embodiment is shown.

[0482] Figure 24 Shown by Figure 1 The point cloud video encoder 10002 of the sending device 10000, Figure 2 Code 20001, Figure 4 The encoder, Figure 12 The sending device Figure 14 XR devices, Figure 22 The structure of the bit stream generated by the encoder, etc.

[0483] Figure 24 The bit stream can be obtained by Figure 1 The point cloud video decoder 10006 of the receiving device 10004, Figure 2 Decoding 20003, Figure 10 and Figure 11 The decoder, Figure 13 The receiving device Figure 14 XR devices, Figure 23 The decoder etc. receives and decodes.

[0484] The method / device according to the embodiment may encode point cloud data, generate relevant information, and send / receive relevant information in a bitstream. For example, a point cloud data encoder configured to perform geometry encoding and / or attribute encoding may generate an encoded point cloud (or a bitstream containing a point cloud) as described below. In addition, as described below, signaling information about the point cloud data may be generated and processed by a metadata processor of a point cloud data sending device and included in the point cloud.

[0485] The signaling information according to the embodiment may be generated and transmitted by the transmitting / receiving device according to the embodiment (e.g., the metadata processor 12007 of the transmitting device (which may be referred to as a metadata generator, etc.)), and may be received and acquired by the metadata parser 13006 of the receiving device. Each operation of the receiving device according to the embodiment may be performed based on the signaling information.

[0486] The following abbreviations are used in the present disclosure: SPS (sequence parameter set), GPS (geometry parameter set), APS (attribute parameter set), TPS (tile parameter set), Geom (geometry bitstream (geometry bitstream = geometry slice header + geometry slice data)), Attr (attribute bitstream (= attribute block (slice) header + attribute block (slice) data)). Here, a block may be referred to as a block, a slice, etc.

[0487] According to the implementation scheme, the point cloud data can be used as follows Figure 24In the form of a bit stream shown. According to an embodiment, the point cloud data may include a sequence parameter set (SPS) containing signaling information, a geometry parameter set (GPS), an attribute parameter set (APS) and a tile parameter set (TPS). The point cloud data may include one or more geometries and / or attributes. The geometry and / or attributes in the point cloud data may be divided into one or more slices (or blocks / squares). The geometry may have a structure of a geometry fragment header and geometry fragment data. For example, a TPS containing signaling information may include Tile(0)_tile_bounding_box_xyz0 and Tile(0)_tile_bounding_box_whd. The geometry may include geom_geom_parameter_set_id, geom_tile_id, geom_slice_id, geomBoxOrigin, geom_box_log2_scale, geom_max_node_size_log2 and geom_num_points.

[0488] Signaling information according to an embodiment may be added to SPS, GPS, APS, TPS, etc. and signaled.

[0489] According to an embodiment, signaling information may be added to TPS, geometry (Geom) for each slice, or attributes (Attr) for each slice, and signaled.

[0490] The structure of point cloud data according to an embodiment can effectively provide parameter sets including signaling information about encoding / decoding / data, geometry and attributes.

[0491] The point cloud data associated with the point cloud data transmitting / receiving apparatus according to the embodiment may include at least one of a sequence parameter, a geometry parameter, an attribute parameter, a patch parameter, a geometry bit stream, or an attribute bit stream.

[0492] Hereinafter, the syntax of specific signaling information will be described with reference to the accompanying drawings. For reference, the name of the signaling information according to the implementation mode may be changed / expanded within the scope of the predetermined meaning / function of the signaling information. The fields of the signaling information may be distinguishably referred to as the first signaling information, the second signaling information, etc.

[0493] As described above, the point cloud data transmitting device may transmit the encoded point cloud data in the form of a bitstream. According to an embodiment, the bitstream may include one or more sub-bitstreams.

[0494] Taking into account the errors of the transmission channel, the point cloud data sending device divides the image of the point cloud data into one or more packets and transmits it through the network. The bitstream according to the embodiment may contain one or more packets (e.g., network abstraction layer (NAL) units). Therefore, even when some packets are lost in a bad network environment, the point cloud data receiving device can use the remaining packets to reconstruct the image. The point cloud data may be divided into one or more slices or one or more tiles and processed. The tiles and slices according to the embodiment are areas for segmenting the screen of the point cloud data and processing it through point cloud compression coding. The point cloud data sending device can provide high-quality point cloud content by processing the data corresponding to each area according to the importance of each divided area of ​​the point cloud data. That is, the point cloud data sending device according to the embodiment can process the data corresponding to the area important to the user with better compression efficiency and appropriate delay through point cloud compression coding.

[0495] The image (or picture) of the point cloud content according to the embodiment is divided into basic processing units for point cloud compression encoding. The basic processing unit for point cloud compression encoding according to the embodiment may include a coding tree unit (CTU) and a block, but is not limited to this example.

[0496] A slice according to an embodiment is an area including a basic processing unit for one or more point cloud compression encodings, and does not have a rectangular shape. A slice according to an embodiment contains data transmitted by grouping. A patch according to an embodiment is an area divided in a rectangular shape in an image, and includes a basic processing unit for one or more point cloud compression encodings. A slice according to an embodiment may be included in one or more slices. In addition, a patch according to an embodiment may be included in one or more slices.

[0497] According to an embodiment, a bitstream may include signaling information including a sequence parameter set (SPS) for signaling a sequence level, a geometry parameter set (GPS) for signaling geometry information encoding, an attribute parameter set (APS) for signaling attribute information encoding, and a patch parameter set (TPS) for signaling a patch level, as well as one or more slices.

[0498] The SPS according to an embodiment may encode information of the entire sequence such as a profile and a level, and may include comprehensive information about the entire file such as a picture resolution and a video format.

[0499] A slice (e.g., slice 0) according to an embodiment includes a slice header and slice data. The slice data may include a geometry bitstream Geom00 and one or more attribute bitstreams Attr00 and Attr10. The geometry bitstream may contain a header (e.g., a geometry slice header) and a payload (e.g., geometry slice data). The header of the geometry bitstream according to an embodiment may contain identification information about a parameter set included in GPS (geom_geom_parameter_set_id), a tile identifier (geom_tileid), a slice identifier (geom_slice_id), information about the origin of the geometry box (geomBoxOrigin), a geometry box scale (geom_box_log2_scale), a maximum geometry node size (geom_max_node_size_log2), the number of geometry points (geom_num_poins), and information about the data included in the payload. The attribute bitstream may contain a header (e.g., an attribute slice header or an attribute block header) and a payload (e.g., attribute slice data or attribute block data).

[0500] The relevant information may be notified by signaling to implement the implementation. The signaling information according to the implementation may be used at the transmitting side or the receiving side.

[0501] Geometry data (Geom): Geometry bitstream = geometry slice header + geometry slice data

[0502] Attribute data (Attr): Attribute bitstream = attribute block header + attribute block data. According to an embodiment, a block may be referred to as a slice (block = slice).

[0503] The method / apparatus according to the embodiment may add option information related to the predictor selection according to the embodiment to the SPS or APS and transmit the information.

[0504] The method / apparatus according to an embodiment may add option information related to predictor selection to a TPS or attribute header for each slice and transmit the option information.

[0505] The method / apparatus according to the embodiment may divide the data into tiles or slices, so that the point cloud may be divided and processed by regions, and signaling information may be generated and sent for each tile / slice.

[0506] When dividing the point cloud into multiple regions, you can set the option to generate different sets of neighboring points for each region, so that the reliability of the result is slightly lower but the complexity is low, or the complexity is high but the reliability is high. It can be set differently according to the processing power of the receiver.

[0507] The method / apparatus according to the embodiment may signal, in the SPS and / or GPS, geometry point reconstruction related information for a spatial scalability function used in a process of encoding and / or decoding geometry.

[0508] The method / apparatus according to an embodiment may signal, in the TPS and / or Geom_slice_header for each slice, geometry point reconstruction related information for a spatial scalability function used in a process of encoding and / or decoding geometry.

[0509] Since different geometric encodings may be performed according to specific regions in the 3D space, when the method / apparatus according to the embodiment divides point cloud data into regions, different geometric position reconstruction methods may be used.

[0510] Therefore, when the method / apparatus according to the embodiment divides the point cloud into patches, different geometric position reconstruction methods may be applied to each patch.

[0511] When the method / apparatus according to the embodiment divides the point cloud into slices, different geometric position reconstruction methods may be applied to each slice.

[0512] The method / apparatus according to the embodiment may add centroid position information obtained through distribution analysis of partially encoded nodes of geometry (Geom) to the geometry node and notify it with a signal.

[0513] Hereinafter, the syntax or definition of specific information (elements) included in a bitstream will be described.

[0514] Figure 25 A sequence parameter set (SPS) according to an embodiment is shown.

[0515] Figure 25 The SPS shown is included in Figure 23 middle.

[0516] The SPS according to an embodiment may include information for reconstructing geometric points to encode and / or decode spatial scalability of the geometry.

[0517] profile_idc indicates the profile to which the bitstream conforms according to an embodiment. The bitstream may not contain values ​​of profile_idc other than those specified in the embodiment. The values ​​of profile_idc may be reserved by ISO / IEC for future use.

[0518] profile_compatibility_flag equal to 1 indicates that the bitstream conforms to the profile indicated by profile_idc equal to j.

[0519] sps_num_attribute_sets indicates the number of coded attributes in the bitstream. The value of sps_num_attribute_sets can be in the range of 0 to 63.

[0520] Include the following elements of the attribute in the SPS according to the sps_num_attribute_set sets.

[0521] attribute_dimension[i] specifies the number of components of the i-th attribute.

[0522] attribute_instance_id[i] specifies the instanceID for the i-th attribute.

[0523] spatial_scalability_LOD may indicate the encoded octree depth level used to support spatial scalability.

[0524] spatial_scalability_geom_quantization_type can indicate the geometry quantization type.

[0525] For example, spatial_scalability_geom_quantization_type equal to 0 may indicate a rounding method. spatial_scalability_geom_quantization_type equal to 1 may indicate a ceiling method. spatial_scalability_geom_quantization_type equal to 2 may indicate a floor method. According to an embodiment, the integer value of the indication information may be changed in various ways.

[0526] spatial_scalability_geom_position_type may indicate a method for reconstructing the geometric position belonging to a node.

[0527] For example, when spatial_scalability_geom_position_type is equal to 0, the geometry reconstruction method is the same as the quantization type. In the case of rounding, the geometry can be reconstructed as the center. In the case of the upper rounding method, the geometry can be reconstructed as the right-upper-back position. When spatial_scalability_geom_position_type is equal to 1, the geometry can be reconstructed as the left-lower-front position. When spatial_scalability_geom_position_type is equal to 2, the geometry can be reconstructed as the left-lower-back position. When spatial_scalability_geom_position_type is equal to 3, the geometry can be reconstructed as the left-upper-front position. When spatial_scalability_geom_position_type is equal to 4, the geometry can be reconstructed as the left-upper-back position. When spatial_scalability_geom_position_type is equal to 5, the geometry can be reconstructed as the right-lower-front position. When spatial_scalability_geom_position_type is equal to 6, the geometry can be reconstructed as the right-lower-back position. When spatial_scalability_geom_position_type is equal to 7, the geometric position may be reconstructed as a right-top-front position. When spatial_scalability_geom_position_type is equal to 8, the geometric position may be reconstructed as a right-top-back position. When spatial_scalability_geom_position_type is equal to 9, the geometric position may be reconstructed as a middle / center position. When spatial_scalability_geom_position_type is equal to 10, the geometric position may be reconstructed as a child node position with a centroid computed by a distribution.

[0528] When spatial_scalability_geom_position_type is equal to 10, spatial_scalability_geom_centroid_position_type is included in the SPS.

[0529] When the geometric position is reconstructed as the centroid calculated by distribution, spatial_scalability_geom_centroid_position_type may indicate a method of reconstructing the position of the node to which the centroid belongs.

[0530] For example, when spatial_scalability_geom_centroid_position_type is equal to 0, the position of the node to which the centroid belongs can be reconstructed in the same way as the quantization type. When the quantization type is rounding, the position of the node can be reconstructed by constructing the center. When the quantization type is the upper rounding method, the position of the node can be reconstructed by constructing the right-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 1, the position of the node to which the centroid belongs can be constructed as a left-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 2, the position of the node to which the centroid belongs can be constructed as a left-lower-back position. When spatial_scalability_geom_centroid_position_type is equal to 3, the position of the node to which the centroid belongs can be constructed as a left-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 4, the position of the node to which the centroid belongs can be constructed as a left-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 5, the position of the node to which the centroid belongs can be constructed as a right-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 6, the position of the node to which the centroid belongs can be constructed as a right-lower-rear position. When spatial_scalability_geom_centroid_position_type is equal to 7, the position of the node to which the centroid belongs can be constructed as a right-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 8, the position of the node to which the centroid belongs can be constructed as a right-upper-rear position. When spatial_scalability_geom_centroid_position_type is equal to 9, the position of the node to which the centroid belongs can be constructed as a middle / center position.

[0531] Figure A geometry parameter set (GPS) according to an embodiment is shown.

[0532] ​ The GPS corresponds to ​ GPS.

[0533] A GPS according to an embodiment may include information for reconstructing geometric points to encode and / or decode spatial scalability of the geometry.

[0534] spatial_scalability_LOD may indicate the encoded octree depth level used to support spatial scalability.

[0535] spatial_scalability_geom_quantization_type can indicate the geometry quantization type.

[0536] For example, spatial_scalability_geom_quantization_type equal to 0 may indicate a rounding method. spatial_scalability_geom_quantization_type equal to 1 may indicate a ceiling method. spatial_scalability_geom_quantization_type equal to 2 may indicate a floor method. According to an embodiment, the integer value of the indication information may be changed in various ways.

[0537] spatial_scalability_geom_position_type may indicate a method for reconstructing the geometric position belonging to a node.

[0538] For example, when spatial_scalability_geom_position_type is equal to 0, the geometry reconstruction method is the same as the quantization type. In the case of rounding, the geometry can be reconstructed as the center. In the case of the upper rounding method, the geometry can be reconstructed as the right-upper-back position. When spatial_scalability_geom_position_type is equal to 1, the geometry can be reconstructed as the left-lower-front position. When spatial_scalability_geom_position_type is equal to 2, the geometry can be reconstructed as the left-lower-back position. When spatial_scalability_geom_position_type is equal to 3, the geometry can be reconstructed as the left-upper-front position. When spatial_scalability_geom_position_type is equal to 4, the geometry can be reconstructed as the left-upper-back position. When spatial_scalability_geom_position_type is equal to 5, the geometry can be reconstructed as the right-lower-front position. When spatial_scalability_geom_position_type is equal to 6, the geometry can be reconstructed as the right-lower-back position. When spatial_scalability_geom_position_type is equal to 7, the geometric position may be reconstructed as a right-top-front position. When spatial_scalability_geom_position_type is equal to 8, the geometric position may be reconstructed as a right-top-back position. When spatial_scalability_geom_position_type is equal to 9, the geometric position may be reconstructed as a middle / center position. When spatial_scalability_geom_position_type is equal to 10, the geometric position may be reconstructed as a child node position with a centroid computed by a distribution.

[0539] When spatial_scalability_geom_position_type is equal to 10, spatial_scalability_geom_centroid_position_type is included in GPS.

[0540] When the geometric position is reconstructed as the centroid calculated by distribution, spatial_scalability_geom_centroid_position_type may indicate a method of reconstructing the position of the node to which the centroid belongs.

[0541] For example, when spatial_scalability_geom_centroid_position_type is equal to 0, the position of the node to which the centroid belongs can be reconstructed in the same way as the quantization type. When the quantization type is rounding, the position of the node can be reconstructed by constructing the center. When the quantization type is the upper rounding method, the position of the node can be reconstructed by constructing the right-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 1, the position of the node to which the centroid belongs can be constructed as a left-front-lower position. When spatial_scalability_geom_centroid_position_type is equal to 2, the position of the node to which the centroid belongs can be constructed as a left-lower-back position. When spatial_scalability_geom_centroid_position_type is equal to 3, the position of the node to which the centroid belongs can be constructed as a left-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 4, the position of the node to which the centroid belongs can be constructed as a left-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 5, the position of the node to which the centroid belongs can be constructed as a right-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 6, the position of the node to which the centroid belongs can be constructed as a right-lower-rear position. When spatial_scalability_geom_centroid_position_type is equal to 7, the position of the node to which the centroid belongs can be constructed as a right-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 8, the position of the node to which the centroid belongs can be constructed as a right-upper-rear position. When spatial_scalability_geom_centroid_position_type is equal to 9, the position of the node to which the centroid belongs can be constructed as a middle / center position.

[0542] spatial_scalability_tile_based_geom_reconstruction_flag is a flag indicating whether to apply different geometry reconstruction schemes on a per-geometry tile basis.

[0543] gps_geom_parameter_set_id provides an identifier for GPS for reference by other syntax elements. The value of gps_seq_parameter_set_id can be in the range of 0 to 15, inclusive.

[0544] gps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id for the active SPS. The value of gps_seq_parameter_set_id can be in the range of 0 to 15, inclusive.

[0545] ​ A tile parameter set (TPS) according to an embodiment is shown.

[0546] ​ Corresponds to ​ TPS.

[0547] The TPS according to an embodiment may include information for reconstructing geometry points to encode and / or decode spatial scalability of the geometry.

[0548] When spatial_scalability_tile_based_geom_reconstruction_flag is equal to 'true', the TPS includes the following elements.

[0549] spatial_scalability_LOD may indicate the encoding octree depth level applied to the tile to support spatial scalability.

[0550] spatial_scalability_geom_quantization_type may indicate the type of geometry quantization applied to the tile.

[0551] For example, spatial_scalability_geom_quantization_type equal to 0 may indicate a rounding method. spatial_scalability_geom_quantization_type equal to 1 may indicate a ceiling method. spatial_scalability_geom_quantization_type equal to 2 may indicate a floor method. According to an embodiment, the integer value of the indication information may be changed in various ways.

[0552] spatial_scalability_geom_position_type may indicate a method of reconstructing the geometric positions of nodes belonging to a tile applied to it.

[0553] For example, when spatial_scalability_geom_position_type is equal to 0, the geometry reconstruction method is the same as the quantization type. In the case of rounding, the geometry can be reconstructed as the center. In the case of the upper rounding method, the geometry can be reconstructed as the right-upper-back position. When spatial_scalability_geom_position_type is equal to 1, the geometry can be reconstructed as the left-lower-front position. When spatial_scalability_geom_position_type is equal to 2, the geometry can be reconstructed as the left-lower-back position. When spatial_scalability_geom_position_type is equal to 3, the geometry can be reconstructed as the left-upper-front position. When spatial_scalability_geom_position_type is equal to 4, the geometry can be reconstructed as the left-upper-back position. When spatial_scalability_geom_position_type is equal to 5, the geometry can be reconstructed as the right-lower-front position. When spatial_scalability_geom_position_type is equal to 6, the geometry can be reconstructed as the right-lower-back position. When spatial_scalability_geom_position_type is equal to 7, the geometric position may be reconstructed as a right-top-front position. When spatial_scalability_geom_position_type is equal to 8, the geometric position may be reconstructed as a right-top-back position. When spatial_scalability_geom_position_type is equal to 9, the geometric position may be reconstructed as a middle / center position. When spatial_scalability_geom_position_type is equal to 10, the geometric position may be reconstructed as a child node position with a centroid computed by a distribution.

[0554] When spatial_scalability_geom_position_type is equal to 10, spatial_scalability_geom_centroid_position_type is included in the TPS.

[0555] When the geometric position is reconstructed as the centroid calculated by distribution, spatial_scalability_geom_centroid_position_type may indicate a method of reconstructing the position of the node to which the centroid belongs.

[0556] For example, when spatial_scalability_geom_centroid_position_type is equal to 0, the position of the node to which the centroid belongs can be reconstructed in the same way as the quantization type. When the quantization type is rounding, the position of the node can be reconstructed by constructing the center. When the quantization type is the upper rounding method, the position of the node can be reconstructed by constructing the right-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 1, the position of the node to which the centroid belongs can be constructed as a left-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 2, the position of the node to which the centroid belongs can be constructed as a left-lower-back position. When spatial_scalability_geom_centroid_position_type is equal to 3, the position of the node to which the centroid belongs can be constructed as a left-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 4, the position of the node to which the centroid belongs can be constructed as a left-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 5, the position of the node to which the centroid belongs can be constructed as a right-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 6, the position of the node to which the centroid belongs can be constructed as a right-lower-rear position. When spatial_scalability_geom_centroid_position_type is equal to 7, the position of the node to which the centroid belongs can be constructed as a right-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 8, the position of the node to which the centroid belongs can be constructed as a right-upper-rear position. When spatial_scalability_geom_centroid_position_type is equal to 9, the position of the node to which the centroid belongs can be constructed as a middle / center position.

[0557] spatial_scalability_slice_based_geom_reconstitution_flag is a flag indicating whether different geometry reconstruction schemes are applied on a per-slice basis.

[0558] num_tiles specifies the number of tiles signaled for the bitstream. When not present, num_tiles may be inferred to be 0.

[0559] The tile bounding box offset element described below may be transmitted in the TPS according to num_tiles.

[0560] tile_bounding_box_offset_x[i] may represent the x offset of the i-th tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_x[0] may be inferred to be sps_bounding_box_offset_x.

[0561] tile_bounding_box_offset_y[i] represents the y offset of the i-th tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_y[0] may be inferred to be sps_bounding_box_offset_y.

[0562] tile_bounding_box_offset_z[i] represents the z offset of the ith tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_z[0] may be inferred to be sps_bounding_box_offset_z.

[0563] ​ A geometry slice head (GSH) according to an embodiment is shown.

[0564] ​ Corresponds to ​ of GSH.

[0565] A GSH according to an embodiment may include information for reconstructing geometry points to encode and / or decode spatial scalability of the geometry.

[0566] When spatial_scalability_slice_based_geom_reconstruction_flag is equal to 'true', the following elements may be included in the GSH.

[0567] spatial_scalability_LOD may indicate the encoded octree depth level applied to slices that support spatial scalability.

[0568] spatial_scalability_geom_quantization_type may indicate the type of geometry quantization applied to the slice.

[0569] For example, spatial_scalability_geom_quantization_type equal to 0 may indicate a rounding method. spatial_scalability_geom_quantization_type equal to 1 may indicate a ceiling method. spatial_scalability_geom_quantization_type equal to 2 may indicate a floor method. According to an embodiment, the integer value of the indication information may be changed in various ways.

[0570] spatial_scalability_geom_position_type may indicate a method of reconstructing the geometric positions of nodes belonging to the slice applied.

[0571] For example, when spatial_scalability_geom_position_type is equal to 0, the geometry reconstruction method is the same as the quantization type. In the case of rounding, the geometry can be reconstructed as the center. In the case of the upper rounding method, the geometry can be reconstructed as the right-upper-back position. When spatial_scalability_geom_position_type is equal to 1, the geometry can be reconstructed as the left-lower-front position. When spatial_scalability_geom_position_type is equal to 2, the geometry can be reconstructed as the left-lower-back position. When spatial_scalability_geom_position_type is equal to 3, the geometry can be reconstructed as the left-upper-front position. When spatial_scalability_geom_position_type is equal to 4, the geometry can be reconstructed as the left-upper-back position. When spatial_scalability_geom_position_type is equal to 5, the geometry can be reconstructed as the right-front-lower position. When spatial_scalability_geom_position_type is equal to 6, the geometry can be reconstructed as the right-lower-back position. When spatial_scalability_geom_position_type is equal to 7, the geometric position may be reconstructed as a right-top-front position. When spatial_scalability_geom_position_type is equal to 8, the geometric position may be reconstructed as a right-top-back position. When spatial_scalability_geom_position_type is equal to 9, the geometric position may be reconstructed as a middle / center position. When spatial_scalability_geom_position_type is equal to 10, the geometric position may be reconstructed as a child node position with a centroid computed by a distribution.

[0572] When spatial_scalability_geom_position_type is equal to 10, spatial_scalability_geom_centroid_position_type is included in the GSH.

[0573] When the geometric position is reconstructed as the centroid calculated by distribution, spatial_scalability_geom_centroid_position_type may indicate a method of reconstructing the position of the node to which the centroid belongs.

[0574] For example, when spatial_scalability_geom_centroid_position_type is equal to 0, the position of the node to which the centroid belongs can be reconstructed in the same way as the quantization type. When the quantization type is rounding, the position of the node can be reconstructed by constructing the center. When the quantization type is the upper rounding method, the position of the node can be reconstructed by constructing the right-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 1, the position of the node to which the centroid belongs can be constructed as a left-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 2, the position of the node to which the centroid belongs can be constructed as a left-lower-back position. When spatial_scalability_geom_centroid_position_type is equal to 3, the position of the node to which the centroid belongs can be constructed as a left-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 4, the position of the node to which the centroid belongs can be constructed as a left-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 5, the position of the node to which the centroid belongs can be constructed as a right-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 6, the position of the node to which the centroid belongs can be constructed as a right-lower-rear position. When spatial_scalability_geom_centroid_position_type is equal to 7, the position of the node to which the centroid belongs can be constructed as a right-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 8, the position of the node to which the centroid belongs can be constructed as a right-upper-rear position. When spatial_scalability_geom_centroid_position_type is equal to 9, the position of the node to which the centroid belongs can be constructed as a middle / center position.

[0575] gsh_geometry_parameter_set_id specifies the value of gps_geom_parameter_set_id for the active GPS.

[0576] gsh_tile_id specifies the value of tile_id that can be referenced by GSH. The value of gsh_tile_id should be in the range of 0 to XX, inclusive.

[0577] gsh_slice_id identifies a slice header referenced by other syntax elements. The value of gsh_slice_id can be in the range 0 to XX, inclusive.

[0578] ​ A geometry node according to an embodiment is shown.

[0579] ​ Shows the information contained in ​ The node information of the geometry in the bitstream.

[0580] In the process of reconstructing geometry points for spatial scalability during geometry encoding and / or decoding, when the representative position type is the centroid, the method / apparatus according to the embodiment may add centroid related information to the geometry node.

[0581] child_occupancy_idx: When reconstructing a geometric position based on the centroid calculated by the distribution, child_occupancy_idx indicates the position of the node to which the centroid belongs. The node position may correspond to the index of the child node. For example, it may have a value from 0 to 7.

[0582] single_occupancy_flag: When equal to 1, single_occupancy_flag indicates that the current node includes a single child node. When equal to 0, single_occupancy_flag indicates that the current node includes multiple child nodes.

[0583] When spatial_scalability_geom_position_type is equal to 10 (centroid), the geometry node may include a sub-occupancy index.

[0584] The method / apparatus according to the embodiment may provide effects such as spatial scalability based on the operation and signaling information according to the embodiment.

[0585] Spatial scalability is the ability to send and process low-resolution point clouds when the point cloud data is dense and the bandwidth is low. It is also the ability to provide low-resolution point clouds by decoding only the bitstream up to a certain depth level of the octree instead of decoding the entire received octree-based bitstream when the point cloud data is dense and the entire octree stream is received but the decoder lacks the capacity or resources of the decoder, or thumbnails are required first and quickly based on the scene.

[0586] In order to provide a spatial scalability effect in the G-PCC geometry encoding / decoding process of an embodiment, an operation of constructing a partial octree and reconstructing positions according to the constructed last level octree nodes is provided.

[0587] The proposed method of reconstructing the points of octree nodes can increase the compression efficiency of partially encoded / decoded geometry for spatial scalability, depending on the location where the geometry is reconstructed.

[0588] Therefore, in the present disclosure, a geometry compression method for providing a spatially scalable function of a geometry-based point cloud compression (G-PCC) encoder / decoder for compressing 3D point cloud data may be provided, and the geometry compression efficiency may be improved.

[0589] The method / apparatus (transmitting method / apparatus and / or receiving method / apparatus) according to the embodiment can provide various geometric position reconstruction methods for spatial scalability, thereby effectively encoding / decoding various types of point cloud data.

[0590] Reference ​ , when reconstructing geometry, the method / device according to the embodiment may set the centroid differently for each LOD level. ​ In the case of, based on a centroid setting method, the geometry is reconstructed based on the representative positions of nodes from LOD 0 to a specific level (LODN). In other embodiments, different types of centroid setting methods may be applied to corresponding LOD levels in various ways. That is, operation S21040 may be performed differently for each LOD level. A sending device and / or an encoder according to an embodiment may analyze the centroid for each LOD and send information indicating the centroid analysis method for each LOD to a receiving device and / or a decoder according to an embodiment.

[0591] All schemes for representing location types according to embodiments may be applied to each LOD and may be signaled to the decoder.

[0592] Operation 21020 (geometric position reconstruction) according to an embodiment may be performed on each LOD of the octree.

[0593] ​ An SPS according to an embodiment is shown.

[0594] ​ Can correspond to ​ SPS.

[0595] for ​ The definition of the elements included in can be referred to ​ Description of the corresponding element.

[0596] The SPS according to an embodiment may include information for reconstructing geometric points to encode and / or decode spatial scalability of the geometry.

[0597] profile_idc indicates the profile to which the bitstream conforms according to an embodiment. The bitstream may not contain values ​​of profile_idc other than those specified in the embodiment. The values ​​of profile_idc may be reserved by ISO / IEC for future use.

[0598] profile_compatibility_flag equal to 1 indicates that the bitstream conforms to the profile indicated by profile_idc equal to j.

[0599] sps_num_attribute_sets indicates the number of coded attributes in the bitstream. The value of sps_num_attribute_sets can be in the range of 0 to 63.

[0600] Depending on the sps_num_attribute_set sets, the following elements of an attribute may be included in the SPS.

[0601] attribute_dimension[i] specifies the number of components of the i-th attribute.

[0602] attribute_instance_id[i] specifies the instanceID for the i-th attribute.

[0603] spatial_scalability_LOD may indicate the encoded octree depth level used to support spatial scalability.

[0604] spatial_scalability_geom_quantization_type can indicate the geometry quantization type.

[0605] For example, spatial_scalability_geom_quantization_type equal to 0 may indicate a rounding method. spatial_scalability_geom_quantization_type equal to 1 may indicate a ceiling method. spatial_scalability_geom_quantization_type equal to 2 may indicate a floor method. According to an embodiment, the integer value of the indication information may be changed in various ways.

[0606] spatial_scalability_geom_position_type may indicate a method for reconstructing the geometric position belonging to a node.

[0607] For example, when spatial_scalability_geom_position_type is equal to 0, the geometry reconstruction method is the same as the quantization type. In the case of rounding, the geometry can be reconstructed as the center. In the case of the upper rounding method, the geometry can be reconstructed as the right-upper-back position. When spatial_scalability_geom_position_type is equal to 1, the geometry can be reconstructed as the left-lower-front position. When spatial_scalability_geom_position_type is equal to 2, the geometry can be reconstructed as the left-lower-back position. When spatial_scalability_geom_position_type is equal to 3, the geometry can be reconstructed as the left-upper-front position. When spatial_scalability_geom_position_type is equal to 4, the geometry can be reconstructed as the left-upper-back position. When spatial_scalability_geom_position_type is equal to 5, the geometry can be reconstructed as the right-front-lower position. When spatial_scalability_geom_position_type is equal to 6, the geometry can be reconstructed as the right-lower-back position. When spatial_scalability_geom_position_type is equal to 7, the geometric position may be reconstructed as a right-top-front position. When spatial_scalability_geom_position_type is equal to 8, the geometric position may be reconstructed as a right-top-back position. When spatial_scalability_geom_position_type is equal to 9, the geometric position may be reconstructed as a middle / center position. When spatial_scalability_geom_position_type is equal to 10, the geometric position may be reconstructed as a child node position with a centroid computed by a distribution.

[0608] When spatial_scalability_geom_position_type is equal to 10, spatial_scalability_geom_centroid_position_type is included in the SPS.

[0609] When the geometric position is reconstructed as the centroid calculated by distribution, spatial_scalability_geom_centroid_position_type may indicate a method of reconstructing the position of the node to which the centroid belongs.

[0610] For example, when spatial_scalability_geom_centroid_position_type is equal to 0, the position of the node to which the centroid belongs can be reconstructed in the same way as the quantization type. When the quantization type is rounding, the position of the node can be reconstructed by constructing the center. When the quantization type is the upper rounding method, the position of the node can be reconstructed by constructing the right-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 1, the position of the node to which the centroid belongs can be constructed as a left-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 2, the position of the node to which the centroid belongs can be constructed as a left-lower-back position. When spatial_scalability_geom_centroid_position_type is equal to 3, the position of the node to which the centroid belongs can be constructed as a left-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 4, the position of the node to which the centroid belongs can be constructed as a left-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 5, the position of the node to which the centroid belongs can be constructed as a right-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 6, the position of the node to which the centroid belongs can be constructed as a right-lower-rear position. When spatial_scalability_geom_centroid_position_type is equal to 7, the position of the node to which the centroid belongs can be constructed as a right-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 8, the position of the node to which the centroid belongs can be constructed as a right-upper-rear position. When spatial_scalability_geom_centroid_position_type is equal to 9, the position of the node to which the centroid belongs can be constructed as a middle / center position.

[0611] octree_height_depth indicates the octree height.

[0612] analyzed_distribution_at_depth indicates a result value of a centroid position obtained by analyzing the distribution of points at each octree depth. Specifically, analyzed_distribution_at_depth may indicate an index based on an octree child node.

[0613] For example, when analyzed_distribution_at_depth is equal to 1, the centroid position is the left-lower-front position. When analyzed_distribution_at_depth is equal to 2, the centroid position is the left-lower-rear position. When analyzed_distribution_at_depth is equal to 3, the centroid position is the left-upper-front position. When analyzed_distribution_at_depth is equal to 4, the centroid position is the left-upper-rear position. When analyzed_distribution_at_depth is equal to 5, the centroid position is the right-lower-front position. When analyzed_distribution_at_depth is equal to 6, the centroid position is the right-lower-rear position. When analyzed_distribution_at_depth is equal to 7, the centroid position is the right-upper-front position. When analyzed_distribution_at_depth is equal to 8, the centroid position is the right-upper-rear position.

[0614] ​ A GPS according to an embodiment is shown.

[0615] ​ Can correspond to ​ GPS.

[0616] for ​ The definition of the elements included in can be referred to ​ Description of the corresponding element.

[0617] A GPS according to an embodiment may include information for reconstructing geometric points to encode and / or decode spatial scalability of the geometry.

[0618] spatial_scalability_LOD may indicate the encoded octree depth level used to support spatial scalability.

[0619] spatial_scalability_geom_quantization_type can indicate the geometry quantization type.

[0620] For example, spatial_scalability_geom_quantization_type equal to 0 may indicate a rounding method. spatial_scalability_geom_quantization_type equal to 1 may indicate a ceiling method. spatial_scalability_geom_quantization_type equal to 2 may indicate a floor method. According to an embodiment, the integer value of the indication information may be changed in various ways.

[0621] spatial_scalability_geom_position_type may indicate a method for reconstructing the geometric position belonging to a node.

[0622] For example, when spatial_scalability_geom_position_type is equal to 0, the geometry reconstruction method is the same as the quantization type. In the case of rounding, the geometry can be reconstructed as the center. In the case of the upper rounding method, the geometry can be reconstructed as the right-upper-back position. When spatial_scalability_geom_position_type is equal to 1, the geometry can be reconstructed as the left-lower-front position. When spatial_scalability_geom_position_type is equal to 2, the geometry can be reconstructed as the left-lower-back position. When spatial_scalability_geom_position_type is equal to 3, the geometry can be reconstructed as the left-upper-front position. When spatial_scalability_geom_position_type is equal to 4, the geometry can be reconstructed as the left-upper-back position. When spatial_scalability_geom_position_type is equal to 5, the geometry can be reconstructed as the right-lower-front position. When spatial_scalability_geom_position_type is equal to 6, the geometry can be reconstructed as the right-lower-back position. When spatial_scalability_geom_position_type is equal to 7, the geometric position may be reconstructed as a right-top-front position. When spatial_scalability_geom_position_type is equal to 8, the geometric position may be reconstructed as a right-top-back position. When spatial_scalability_geom_position_type is equal to 9, the geometric position may be reconstructed as a middle / center position. When spatial_scalability_geom_position_type is equal to 10, the geometric position may be reconstructed as a child node position with a centroid computed by a distribution.

[0623] When spatial_scalability_geom_position_type is equal to 10, spatial_scalability_geom_centroid_position_type is included in GPS.

[0624] When the geometric position is reconstructed as the centroid calculated by distribution, spatial_scalability_geom_centroid_position_type may indicate a method of reconstructing the position of the node to which the centroid belongs.

[0625] For example, when spatial_scalability_geom_centroid_position_type is equal to 0, the position of the node to which the centroid belongs can be reconstructed in the same way as the quantization type. When the quantization type is rounding, the position of the node can be reconstructed by constructing the center. When the quantization type is the upper rounding method, the position of the node can be reconstructed by constructing the right-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 1, the position of the node to which the centroid belongs can be constructed as a left-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 2, the position of the node to which the centroid belongs can be constructed as a left-lower-back position. When spatial_scalability_geom_centroid_position_type is equal to 3, the position of the node to which the centroid belongs can be constructed as a left-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 4, the position of the node to which the centroid belongs can be constructed as a left-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 5, the position of the node to which the centroid belongs can be constructed as a right-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 6, the position of the node to which the centroid belongs can be constructed as a right-lower-rear position. When spatial_scalability_geom_centroid_position_type is equal to 7, the position of the node to which the centroid belongs can be constructed as a right-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 8, the position of the node to which the centroid belongs can be constructed as a right-upper-rear position. When spatial_scalability_geom_centroid_position_type is equal to 9, the position of the node to which the centroid belongs can be constructed as a middle / center position.

[0626] spatial_scalability_tile_based_geom_reconstruction_flag is a flag indicating whether to apply different geometry reconstruction schemes on a per-geometry tile basis.

[0627] spatial_scalability_per_LOD_flag is a flag indicating whether to apply a different geometry reconstruction scheme for each octree depth.

[0628] octree_height_depth indicates the octree height.

[0629] gps_geom_parameter_set_id provides an identifier for the GPS for reference by other syntax elements. The value of gps_seq_parameter_set_id can be in the range of 0 to 15, inclusive.

[0630] gps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id for the active SPS. The value of gps_seq_parameter_set_id can be in the range of 0 to 15, inclusive.

[0631] ​ A TPS according to an embodiment is shown.

[0632] ​ Corresponds to ​ TPS.

[0633] for ​ The definition of the elements included in can be referred to ​ Description of the corresponding element.

[0634] The TPS according to an embodiment may include information for reconstructing geometry points to encode and / or decode spatial scalability of the geometry.

[0635] When spatial_scalability_tile_based_geom_reconstruction_flag is equal to 'true', the TPS includes the following elements.

[0636] spatial_scalability_LOD may indicate the encoding octree depth level applied to the tile to support spatial scalability.

[0637] spatial_scalability_geom_quantization_type may indicate the type of geometry quantization applied to the tile.

[0638] For example, spatial_scalability_geom_quantization_type equal to 0 may indicate a rounding method. spatial_scalability_geom_quantization_type equal to 1 may indicate a ceiling method. spatial_scalability_geom_quantization_type equal to 2 may indicate a floor method. According to an embodiment, the integer value of the indication information may be changed in various ways.

[0639] spatial_scalability_geom_position_type may indicate a method of reconstructing the geometric positions of nodes belonging to a tile applied to it.

[0640] For example, when spatial_scalability_geom_position_type is equal to 0, the geometry reconstruction method is the same as the quantization type. In the case of rounding, the geometry can be reconstructed as the center. In the case of the upper rounding method, the geometry can be reconstructed as the right-upper-back position. When spatial_scalability_geom_position_type is equal to 1, the geometry can be reconstructed as the left-lower-front position. When spatial_scalability_geom_position_type is equal to 2, the geometry can be reconstructed as the left-lower-back position. When spatial_scalability_geom_position_type is equal to 3, the geometry can be reconstructed as the left-upper-front position. When spatial_scalability_geom_position_type is equal to 4, the geometry can be reconstructed as the left-upper-back position. When spatial_scalability_geom_position_type is equal to 5, the geometry can be reconstructed as the right-lower-front position. When spatial_scalability_geom_position_type is equal to 6, the geometry can be reconstructed as the right-lower-back position. When spatial_scalability_geom_position_type is equal to 7, the geometric position may be reconstructed as a right-top-front position. When spatial_scalability_geom_position_type is equal to 8, the geometric position may be reconstructed as a right-top-back position. When spatial_scalability_geom_position_type is equal to 9, the geometric position may be reconstructed as a middle / center position. When spatial_scalability_geom_position_type is equal to 10, the geometric position may be reconstructed as a child node position with a centroid computed by a distribution.

[0641] When spatial_scalability_geom_position_type is equal to 10, spatial_scalability_geom_centroid_position_type is included in the TPS.

[0642] When the geometric position is reconstructed as the centroid calculated by distribution, spatial_scalability_geom_centroid_position_type may indicate a method of reconstructing the position of the node to which the centroid belongs.

[0643] For example, when spatial_scalability_geom_centroid_position_type is equal to 0, the position of the node to which the centroid belongs can be reconstructed in the same way as the quantization type. When the quantization type is rounding, the position of the node can be reconstructed by constructing the center. When the quantization type is the upper rounding method, the position of the node can be reconstructed by constructing the right-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 1, the position of the node to which the centroid belongs can be constructed as a left-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 2, the position of the node to which the centroid belongs can be constructed as a left-lower-back position. When spatial_scalability_geom_centroid_position_type is equal to 3, the position of the node to which the centroid belongs can be constructed as a left-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 4, the position of the node to which the centroid belongs can be constructed as a left-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 5, the position of the node to which the centroid belongs can be constructed as a right-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 6, the position of the node to which the centroid belongs can be constructed as a right-lower-rear position. When spatial_scalability_geom_centroid_position_type is equal to 7, the position of the node to which the centroid belongs can be constructed as a right-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 8, the position of the node to which the centroid belongs can be constructed as a right-upper-rear position. When spatial_scalability_geom_centroid_position_type is equal to 9, the position of the node to which the centroid belongs can be constructed as a middle / center position.

[0644] spatial_scalability_slice_based_geom_reconstitution_flag is a flag indicating whether different geometry reconstruction schemes are applied on a slice-by-slice basis.

[0645] spatial_scalability_per_LOD_flag is a flag indicating whether to apply a different geometry reconstruction scheme for each octree depth.

[0646] octree_height_depth indicates the octree height.

[0647] num_tiles specifies the number of tiles signaled for the bitstream. When not present, num_tiles may be inferred to be 0.

[0648] The tile bounding box offset element described below may be transmitted in the TPS according to num_tiles.

[0649] tile_bounding_box_offset_x[i] may represent the x offset of the i-th tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_x[0] may be inferred to be sps_bounding_box_offset_x.

[0650] tile_bounding_box_offset_y[i] represents the y offset of the i-th tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_y[0] may be inferred to be sps_bounding_box_offset_y.

[0651] tile_bounding_box_offset_z[i] represents the z offset of the ith tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_z[0] may be inferred to be sps_bounding_box_offset_z.

[0652] ​ GSH according to an embodiment is shown.

[0653] ​ Corresponds to ​ of GSH.

[0654] for ​ The definition of the elements included in can be referred to ​ Description of the corresponding element.

[0655] A GSH according to an embodiment may include information for reconstructing geometry points to encode and / or decode spatial scalability of the geometry.

[0656] When spatial_scalability_slice_based_geom_reconstruction_flag is equal to 'true', the following elements may be included in the GSH.

[0657] spatial_scalability_LOD may indicate the encoded octree depth level applied to slices that support spatial scalability.

[0658] spatial_scalability_geom_quantization_type may indicate the type of geometry quantization applied to the slice.

[0659] For example, spatial_scalability_geom_quantization_type equal to 0 may indicate a rounding method. spatial_scalability_geom_quantization_type equal to 1 may indicate a ceiling method. spatial_scalability_geom_quantization_type equal to 2 may indicate a floor method. According to an embodiment, the integer value of the indication information may be changed in various ways.

[0660] spatial_scalability_geom_position_type may indicate a method of reconstructing the geometric positions of nodes belonging to the slice applied.

[0661] For example, when spatial_scalability_geom_position_type is equal to 0, the geometry reconstruction method is the same as the quantization type. In the case of rounding, the geometry can be reconstructed as the center. In the case of the upper rounding method, the geometry can be reconstructed as the right-upper-back position. When spatial_scalability_geom_position_type is equal to 1, the geometry can be reconstructed as the left-lower-front position. When spatial_scalability_geom_position_type is equal to 2, the geometry can be reconstructed as the left-lower-back position. When spatial_scalability_geom_position_type is equal to 3, the geometry can be reconstructed as the left-upper-front position. When spatial_scalability_geom_position_type is equal to 4, the geometry can be reconstructed as the left-upper-back position. When spatial_scalability_geom_position_type is equal to 5, the geometry can be reconstructed as the right-lower-front position. When spatial_scalability_geom_position_type is equal to 6, the geometry can be reconstructed as the right-lower-back position. When spatial_scalability_geom_position_type is equal to 7, the geometric position may be reconstructed as a right-top-front position. When spatial_scalability_geom_position_type is equal to 8, the geometric position may be reconstructed as a right-top-back position. When spatial_scalability_geom_position_type is equal to 9, the geometric position may be reconstructed as a middle / center position. When spatial_scalability_geom_position_type is equal to 10, the geometric position may be reconstructed as a child node position with a centroid computed by a distribution.

[0662] When spatial_scalability_geom_position_type is equal to 10, spatial_scalability_geom_centroid_position_type is included in the GSH.

[0663] When the geometric position is reconstructed as the centroid calculated by distribution, spatial_scalability_geom_centroid_position_type may indicate a method of reconstructing the position of the node to which the centroid belongs.

[0664] For example, when spatial_scalability_geom_centroid_position_type is equal to 0, the position of the node to which the centroid belongs can be reconstructed in the same way as the quantization type. When the quantization type is rounding, the position of the node can be reconstructed by constructing the center. When the quantization type is the upper rounding method, the position of the node can be reconstructed by constructing the right-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 1, the position of the node to which the centroid belongs can be constructed as a left-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 2, the position of the node to which the centroid belongs can be constructed as a left-lower-back position. When spatial_scalability_geom_centroid_position_type is equal to 3, the position of the node to which the centroid belongs can be constructed as a left-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 4, the position of the node to which the centroid belongs can be constructed as a left-upper-back position. When spatial_scalability_geom_centroid_position_type is equal to 5, the position of the node to which the centroid belongs can be constructed as a right-lower-front position. When spatial_scalability_geom_centroid_position_type is equal to 6, the position of the node to which the centroid belongs can be constructed as a right-lower-rear position. When spatial_scalability_geom_centroid_position_type is equal to 7, the position of the node to which the centroid belongs can be constructed as a right-upper-front position. When spatial_scalability_geom_centroid_position_type is equal to 8, the position of the node to which the centroid belongs can be constructed as a right-upper-rear position. When spatial_scalability_geom_centroid_position_type is equal to 9, the position of the node to which the centroid belongs can be constructed as a middle / center position.

[0665] spatial_scalability_per_LOD_flag is a flag indicating whether to apply a different geometry reconstruction scheme for each octree depth.

[0666] octree_height_depth indicates the octree height.

[0667] gsh_geometry_parameter_set_id specifies the value of gps_geom_parameter_set_id for the active GPS.

[0668] GSH_tile_id specifies the value of tile_id that can be referenced by GSH. The value of gsh_tile_id should be in the range of 0 to XX, inclusive.

[0669] gsh_slice_id identifies a slice header referenced by other syntax elements. The value of gsh_slice_id can be in the range 0 to XX, inclusive.

[0670] ​ A method for transmitting point cloud data according to an embodiment is shown.

[0671] S34000: The point cloud data transmission method according to an embodiment may include encoding the point cloud data. The encoding operation according to an embodiment may include ​ The sending device 10000, ​ Point cloud video encoder 10002, ​ Code 20001, ​ The encoder, ​ Point cloud data processing of the sending device, ​ XR device 1430, ​ and ​ Point cloud data encoding, ​ The geometric coding, ​ The encoder, and ​ and 33 Bitstream generation and encoding.

[0672] S34010: The point cloud data transmission method according to the embodiment may further include transmitting a bit stream containing the point cloud data. The transmission operation according to the embodiment may include ​ The operation of transmitter 10003 and ​ Send 20002.

[0673] ​ A method for receiving point cloud data according to an embodiment is shown.

[0674] S35000: The point cloud data receiving method according to an embodiment includes receiving a bit stream containing point cloud data. The receiving operation according to an embodiment may include ​ The receiving device 10004, ​ The receiver 10005, according to the transmission ​ Receiving 20002 and ​The bit stream receiving operation.

[0675] 35010: According to an embodiment, the method for receiving point cloud data may further include decoding the point cloud data. The decoding operation according to an embodiment may include ​ Point cloud video decoder 10006, ​ Decoding 20003, ​ Rendering 20004, ​ and ​ The decoder, ​ Point cloud data processing of the receiving device, ​ XR device 1430, ​ and 17 Point cloud data decoding, ​ The geometric decoding of ​ The operation of the decoder, and Figures 24 to 33 Decode the bitstream.

[0676] Spatial scalability is the ability to send and process low-resolution point clouds when the point cloud data is dense and the bandwidth is low. It is also the ability to provide low-resolution point clouds by decoding only the bitstream up to a certain depth level of the octree instead of decoding the entire received octree-based bitstream when the point cloud data is dense and the entire octree stream is received but the decoder lacks the capacity or resources of the decoder, or thumbnails are required first and quickly based on the scene.

[0677] A method for constructing a partial octree and reconstructing positions according to the constructed last-level octree nodes to provide spatial scalability in the G-PCC geometry encoding / decoding process of an embodiment has been described above.

[0678] The method for reconstructing the points of an octree node according to an embodiment can increase the compression efficiency of partially encoded / decoded geometry for spatial scalability according to the reconstructed position of the geometry.

[0679] Therefore, according to the present disclosure, a geometry compression method for providing a spatially scalable function of a geometry-based point cloud compression (G-PCC) encoder / decoder for compressing 3D point cloud data can be provided, and the geometry compression efficiency can be improved.

[0680] The method / apparatus (transmitting method / apparatus and / or receiving method / apparatus) according to the embodiment can provide various geometric position reconstruction methods for spatial scalability, thereby effectively encoding / decoding various types of point cloud data.

[0681] Since a geometric position reconstruction method and a signaling method for spatial scalability are provided according to the above-mentioned embodiments, the point cloud data sending / receiving method / device according to the embodiments can reconstruct the position of the point based on the octree to provide spatial scalability, thereby improving the geometric compression efficiency.

[0682] The embodiments have been described from the perspective of methods and / or devices, and the descriptions of the methods and devices may be applied to complement each other.

[0683] Although the drawings have been described separately for simplicity, new embodiments can be designed by combining the embodiments shown in the various drawings. Designing a computer-readable recording medium on which a program for executing the above-mentioned embodiments is recorded, as required by those skilled in the art, also falls within the scope of the attached claims and their equivalents. The devices and methods according to the embodiments may not be limited by the configurations and methods of the above-mentioned embodiments. Various modifications may be made to the embodiments by selectively combining all or some of the embodiments. Although preferred embodiments have been described with reference to the drawings, those skilled in the art will understand that various modifications and changes may be made in the embodiments without departing from the spirit or scope of the present disclosure described in the attached claims. These modifications should not be understood separately from the technical ideas or viewpoints of the embodiments.

[0684] Various elements of the device of the embodiment can be implemented by hardware, software, firmware or a combination thereof. Various elements in the embodiment can be implemented by a single chip, such as a single hardware circuit. According to the embodiment, the components according to the embodiment can be implemented as separate chips respectively. According to the embodiment, at least one or more of the components of the device according to the embodiment can include one or more processors capable of executing one or more programs. One or more programs can execute any one or more of the operations / methods according to the embodiment, or include instructions for executing the operations / methods. The executable instructions for executing the method / operation of the device according to the embodiment can be stored in a non-temporary CRM or other computer program product configured to be executed by one or more processors, or can be stored in a temporary CRM or other computer program product configured to be executed by one or more processors. In addition, the memory according to the embodiment can be used as a concept that not only covers volatile memory (such as RAM) but also covers non-volatile memory, flash memory and PROM. In addition, it can also be implemented in the form of a carrier wave, such as transmission through the Internet. In addition, the processor-readable recording medium can be distributed to a computer system connected by a network so that the processor-readable code can be executed with the processor-readable code.

[0685] In this specification, the terms " / " and "," should be interpreted as meaning "and / or". For example, the expression "A / B" may mean "A and / or B". Furthermore, "A, B" may mean "A and / or B". Furthermore, "A / B / C" may mean "at least one of A, B and / or C". Furthermore, "A / B / C" may mean "at least one of A, B and / or C". Furthermore, in this specification, the term "or" should be interpreted as meaning "and / or". For example, the expression "A or B" may mean 1) only A, 2) only B, or 3) both A and B. In other words, the term "or" used in this document should be interpreted as meaning "additionally or alternatively".

[0686] Wordings such as first and second can be used to describe various elements of the embodiments. However, various components according to the embodiments should not be limited by the above terms. These terms are only used to distinguish one element from another element. For example, a first user input signal can be referred to as a second user input signal. Similarly, a second user input signal can be referred to as a first user input signal. The use of these terms should be interpreted as not departing from the scope of various embodiments. Both the first user input signal and the second user input signal are user input signals, but do not mean the same user input signal unless the context clearly specifies otherwise.

[0687] The terms used to describe the embodiments are for the purpose of describing specific embodiments and are not intended to limit the embodiments. As used in the description of the embodiments and in the claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly indicates otherwise. The expression "and / or" is used to include all possible combinations of terms. Terms such as "include" or "have" are intended to indicate the presence of drawings, numbers, steps, elements and / or components, and should be understood as not excluding the possibility of additional presence of drawings, numbers, steps, elements and / or components. As used herein, conditional expressions such as "if" and "when..." are not limited to optional cases, and are intended to be interpreted as performing related operations or interpreting related definitions according to specific conditions when specific conditions are met.

[0688] The operations according to the embodiments described in this specification may be performed by a transmitting / receiving device including a memory and / or a processor according to the embodiments. The memory may store a program for processing / controlling the operations according to the embodiments, and the processor may control the various operations described in this specification. The processor may be referred to as a controller, etc. In an embodiment, the operations may be performed by firmware, software, and / or a combination thereof. The firmware, software, and / or a combination thereof may be stored in a processor or a memory.

[0689] The operations according to the above-described embodiments may be performed by a transmitting device and / or a receiving device according to the embodiments. The transmitting / receiving device includes: a transmitter / receiver configured to transmit and receive media data; a memory configured to store instructions (program code, algorithm, flow chart and / or data) for processing according to the embodiments; and a processor configured to control the operation of the transmitting / receiving device.

[0690] The processor may be referred to as a controller, etc., and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above-described embodiments may be performed by the processor. In addition, the processor may be implemented as an encoder / decoder for the operations of the above-described embodiments.

[0691] Invention Mode

[0692] As described above, the relevant contents have been described in the best mode for carrying out the embodiment.

[0693] Industrial Applicability

[0694] As described above, the embodiments may be applied in whole or in part to a point cloud data transmitting / receiving device and system.

[0695] It is obvious to those skilled in the art that various changes or modifications can be made to the embodiments within the scope of the embodiments.

[0696] Thus, it is intended that the embodiments cover the modifications and variations of this disclosure provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method for decoding point cloud data by a decoding device, the method comprising the following steps: receiving a bit stream comprising point cloud data; partially decoding geometric data of the point cloud data, wherein the decoded geometry data is represented based on an octree, and Decoding attribute data of the point cloud data.

2. The method according to claim 1, in, reconstructing geometric positions of nodes in a level of the octree based on the center positions of the nodes for spatial scalability at the level in a level of detail LoD of the octree skipped from a bottom of the octree; and The first node of the octree is reconstructed based on the center position of the first node. wherein the second node of the octree is reconstructed based on the center position of the second node, and The first node and the second node are not leaf nodes of the octree.

3. A device for decoding point cloud data, the device comprising: a receiver configured to receive a bitstream comprising point cloud data; A decoder, the decoder being configured to: partially decoding geometric data of the point cloud data, wherein the decoded geometry data is represented based on an octree, and Decoding attribute data of the point cloud data.

4. The device according to claim 3, in, reconstructing geometric positions of nodes in a level of the octree based on the center positions of the nodes for spatial scalability at the level in a level of detail LoD of the octree skipped from a bottom of the octree; and The first node of the octree is reconstructed based on the center position of the first node. wherein the second node of the octree is reconstructed based on the center position of the second node, and The first node and the second node are not leaf nodes of the octree.

5. A method for encoding point cloud data, the method comprising the following steps: Encode the geometric data of the point cloud data, wherein the encoded geometric data is represented based on an octree, and wherein the geometric positions of the nodes in a level of the octree are reconstructed based on the center positions of the nodes for spatial scalability at the level in the level of detail LoD of the octree skipped from the bottom of the octree; encoding attribute data of the point cloud data based on the reconstructed geometric position; and A bit stream including the geometric data and the attribute data of the point cloud data is sent.

6. The method according to claim 5, in, The first node of the octree is reconstructed based on the center position of the first node, wherein the second node of the octree is reconstructed based on the center position of the second node, and The first node and the second node are not leaf nodes of the octree.

7. A device for encoding point cloud data, the device comprising: an encoder configured to encode geometric data of the point cloud data, wherein the encoded geometric data is represented based on an octree, and encoding attribute data of the point cloud data based on the reconstructed geometric position; and A transmitter is configured to transmit a bit stream including the geometric data and the attribute data of the point cloud data.

8. The device according to claim 7, in, reconstructing geometric positions of nodes in a level of the octree based on the center positions of the nodes for spatial scalability at the level in a level of detail LoD of the octree skipped from a bottom of the octree; and The first node of the octree is reconstructed based on the center position of the first node. wherein the second node of the octree is reconstructed based on the center position of the second node, and The first node and the second node are not leaf nodes of the octree.