Point cloud data transmitting / receiving device and computer readable storage medium

By employing point cloud compression coding technology and utilizing octree structure and Morton code sorting, the problem of high complexity in point cloud data processing is solved, enabling efficient point cloud services that support virtual reality and autonomous driving applications.

CN118972623APending Publication Date: 2024-11-15LG ELECTRONICS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411394309.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2019-10-07
Filing Date
2020-08-04
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively process large amounts of point cloud data, resulting in high latency and encoding/decoding complexity, which fails to meet the needs of services such as virtual reality, augmented reality, and autonomous driving.

Method used

Point cloud compression coding techniques, including geometry-based point cloud compression (G-PCC) and video-based point cloud compression (V-PCC), are employed. By using an octree structure and Morton code sorting, combined with attribute encoding and decoding, efficient encoding and decoding of point cloud data are achieved.

Benefits of technology

It achieves efficient processing of point cloud data, provides high-quality point cloud services, supports virtual reality, augmented reality and autonomous driving services, and reduces encoding and decoding complexity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118972623A_ABST
    Figure CN118972623A_ABST
Patent Text Reader

Abstract

The invention relates to a point cloud data transmitting / receiving device and a computer readable storage medium. A method for receiving point cloud data according to an embodiment may comprise the steps of: receiving a bitstream including point cloud data; and decoding the point cloud data. A method for transmitting point cloud data according to an embodiment may comprise the steps of: encoding point cloud data; and transmitting the bit stream including the point cloud data.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the original invention patent application with application number 202080069413.9 (International application number: PCT / KR2020 / 010283, application date: August 4, 2020, invention name: Point cloud data sending device, point cloud data sending method, point cloud data receiving device and point cloud data receiving method). Technical Field

[0002] Embodiments relate to a method and apparatus for processing point cloud content. Background Art

[0003] Point cloud content is represented by a point cloud, which is a collection of points belonging to a coordinate system representing three-dimensional space. Point cloud content can express media configured in three dimensions and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and self-driving services. However, tens to hundreds of thousands of points of data are required to represent point cloud content. Therefore, a method for efficiently processing large amounts of point data is needed. Summary of the Invention

[0004] Technical issues

[0005] Embodiments provide a device and method for efficiently processing point cloud data. Embodiments provide a method and device for processing point cloud data to address latency and encoding / decoding complexity.

[0006] The technical scope of the embodiments is not limited to the above technical objectives, and can be extended to other technical objectives that can be inferred by those skilled in the art based on the entire content disclosed herein.

[0007] Technical Solution

[0008] To achieve these objectives and other advantages and in accordance with the purposes of the present disclosure, as embodied and broadly described herein, a method for receiving point cloud data may include: receiving a bitstream containing point cloud data; and decoding the point cloud data.

[0009] In another aspect of the present disclosure, a method for transmitting point cloud data may include: encoding the point cloud data; and transmitting a bitstream including the point cloud data.

[0010] Beneficial effects

[0011] The apparatus and method according to the embodiment can efficiently process point cloud data.

[0012] The apparatus and method according to the embodiment can provide high-quality point cloud services.

[0013] The apparatus and method according to the embodiment may provide point cloud content for providing general services such as VR services and self-driving services. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of this application. The drawings illustrate embodiments of the present disclosure and, together with the description, serve to illustrate the principles of the present disclosure. For a better understanding of the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the accompanying drawings. The same reference numerals will be used throughout the drawings to represent the same or similar parts. In the drawings:

[0015] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown.

[0016] Figure 2 is a block diagram illustrating a point cloud content providing operation according to an embodiment.

[0017] Figure 3 An exemplary process for capturing point cloud video is shown, according to an embodiment.

[0018] Figure 4 An exemplary point cloud encoder according to an embodiment is shown.

[0019] Figure 5 An example of voxels according to an embodiment is shown.

[0020] Figure 6 Shown are examples of octrees and occupancy codes according to an embodiment.

[0021] Figure 7 An example of a neighbor node pattern according to an embodiment is shown.

[0022] Figure 8 An example of point arrangement in each LOD according to an embodiment is shown.

[0023] Figure 9 An example of point arrangement in each LOD according to an embodiment is shown.

[0024] Figure 10 A point cloud decoder according to an embodiment is shown.

[0025] Figure 11 A point cloud decoder according to an embodiment is shown.

[0026] Figure 12 A transmitting device according to an embodiment is shown.

[0027] Figure 13 A receiving device according to an embodiment is shown.

[0028] Figure 14 An exemplary structure that can be operated in conjunction with a method / apparatus for transmitting / receiving point cloud data according to an embodiment is shown.

[0029] Figure 15 Shown are structures of geometry data and attribute data for spatial scalability according to an embodiment.

[0030] Figure 16 The attribute encoding / decoding process according to an embodiment is shown.

[0031] Figure 17 FIG. 4 shows an octree structure of point cloud data according to an embodiment.

[0032] Figure 18 A method of reorganizing points belonging to a Morton code-based node according to an embodiment is shown.

[0033] Figure 19 Illustrates merging of isolated nodes according to an embodiment.

[0034] Figure 20 A PCC data encoder according to an embodiment is shown.

[0035] Figure 21 A PCC data decoder according to an embodiment is shown.

[0036] Figure 22 FIG. 1 shows a bitstream structure of point cloud data according to an embodiment.

[0037] Figure 23 A sequence parameter set according to an embodiment is shown.

[0038] Figure 24 Attribute parameter sets according to implementations are shown.

[0039] Figure 25 A patch parameter set according to an embodiment is shown.

[0040] Figure 26 An attribute slice header according to an embodiment is shown.

[0041] Figure 27 Attribute encoding / decoding according to an embodiment is shown.

[0042] Figure 28 Shown are bounding boxes and nodes according to an embodiment.

[0043] Figure 29 Shown are the matches between various nodes and locations according to an embodiment.

[0044] Figure 30 1 shows the selection of a reference node in an octree structure according to an embodiment.

[0045] Figure 31 A PCC data encoder according to an embodiment is shown.

[0046] Figure 32 A PCC data decoder according to an embodiment is shown.

[0047] Figure 33 A sequence parameter set (SPS) according to an embodiment is shown.

[0048] Figure 34 An attribute parameter set (APS) according to an embodiment is shown.

[0049] Figure 35 A patch parameter set (TPS) according to an embodiment is shown.

[0050] Figure 36 An attribute slice header (ASH) according to an embodiment is shown.

[0051] Figure 37 A method of transmitting point cloud data according to an embodiment is shown.

[0052] Figure 38 A method of receiving point cloud data according to an embodiment is shown. DETAILED DESCRIPTION

[0053] Reference will now be made in detail to preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The detailed description given below with reference to the accompanying drawings is intended to illustrate exemplary embodiments of the present disclosure and is not intended to illustrate the only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0054] Although most of the terms used in this disclosure are selected from common terms widely used in the art, some terms are arbitrarily selected by the applicant and their meanings are explained in detail in the following description as needed. Therefore, this disclosure should be understood based on the intended meaning of the terms rather than their simple names or meanings.

[0055] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown.

[0056] Figure 1 The illustrated point cloud content providing system may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 may be capable of wired or wireless communication to transmit and receive point cloud data.

[0057] According to an embodiment, the point cloud data transmitting device 10000 can obtain and process point cloud video (or point cloud content) and transmit it. According to an embodiment, the transmitting device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device and / or a server. According to an embodiment, the transmitting device 10000 may include a device configured to communicate with a base station and / or other wireless devices using a radio access technology (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server.

[0058] According to an embodiment, the sending device 10000 includes a point cloud video acquirer 10001, a point cloud video encoder 10002 and / or a transmitter (or communication module) 10003.

[0059] The point cloud video acquirer 10001 according to an embodiment acquires a point cloud video through a process such as capture, synthesis, or generation. A point cloud video is point cloud content represented by a point cloud. A point cloud is a collection of points located in 3D space and can be referred to as point cloud video data. A point cloud video according to an embodiment may include one or more frames. A frame represents a still image / screen. Therefore, a point cloud video may include point cloud images / frames / screens and may be referred to as a point cloud image, frame, or screen.

[0060] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 may encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to an embodiment may include geometry-based point cloud compression (G-PCC) coding and / or video-based point cloud compression (V-PCC) coding or next-generation coding. The point cloud compression coding according to an embodiment is not limited to the above-mentioned embodiment. The point cloud video encoder 10002 may output a bit stream containing the encoded point cloud video data. The bit stream may include not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0061] According to an embodiment, the transmitter 10003 transmits a bitstream containing encoded point cloud video data. According to an embodiment, the bitstream is encapsulated in a file or segment (e.g., a stream segment) and transmitted via various networks such as a broadcast network and / or a broadband network. Although not shown in the figure, the transmitting device 10000 may include an encapsulator (or encapsulation module) configured to perform an encapsulation operation. According to an embodiment, the encapsulator may be included in the transmitter 10003. According to an embodiment, the file or segment may be transmitted to the receiving device 10004 via a network or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). According to an embodiment, the transmitter 10003 is capable of wired / wireless communication with the receiving device 10004 (or receiver 10005) via a 4G, 5G, 6G, etc. network. In addition, the transmitter may perform necessary data processing operations according to the network system (e.g., a 4G, 5G, or 6G communication network system). The transmitting device 10000 may transmit the encapsulated data on demand.

[0062] The receiving device 10004 according to an embodiment includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include a device configured to communicate with a base station and / or other wireless devices using a radio access technology (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server.

[0063] The receiver 10005 according to the embodiment receives a bitstream containing point cloud video data or a file / segment encapsulated with a bitstream from a network or a storage medium. The receiver 10005 may perform necessary data processing according to a network system (e.g., a communication network system of 4G, 5G, 6G, etc.). The receiver 10005 according to the embodiment may decapsulate the received file / segment and output a bitstream. According to the embodiment, the receiver 10005 may include a decapsulator (or decapsulation module) configured to perform a decapsulation operation. The decapsulator may be implemented as an element (or component) separate from the receiver 10005.

[0064] The point cloud video decoder 10006 decodes the bitstream containing the point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to the method by which the point cloud video data was encoded (e.g., by performing the inverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding (the inverse process of point cloud compression). Point cloud decompression encoding includes G-PCC encoding.

[0065] The renderer 10007 renders the decoded point cloud video data. The renderer 10007 may output point cloud content by rendering not only the point cloud video data but also the audio data. Depending on the embodiment, the renderer 10007 may include a display configured to display the point cloud content. Depending on the embodiment, the display may be implemented as a separate device or component rather than being included in the renderer 10007.

[0066] The arrow indicated by the dotted line in the figure represents the transmission path of the feedback information obtained by the receiving device 10004. Feedback information is information reflecting the interactivity with the user consuming the point cloud content, and includes information about the user (e.g., head orientation information, viewport information, etc.). Specifically, when the point cloud content is content for a service that requires interaction with the user (e.g., a self-driving service, etc.), the feedback information can be provided to the content sender (e.g., the sending device 10000) and / or the service provider. Depending on the embodiment, the feedback information may be used in the receiving device 10004 and the sending device 10000, or may not be provided.

[0067] According to an embodiment, head orientation information refers to information regarding the user's head position, orientation, angle, movement, and the like. According to an embodiment, the receiving device 10004 may calculate viewport information based on the head orientation information. Viewport information may be information regarding the area of ​​the point cloud video being viewed by the user. The viewpoint is the point through which the user views the point cloud video and may refer to the center point of the viewport area. That is, the viewport is the area centered on the viewpoint, and the size and shape of the area may be determined by the field of view (FOV). Therefore, in addition to head orientation information, the receiving device 10004 may also extract viewport information based on the vertical or horizontal FOV supported by the device. Furthermore, the receiving device 10004 may perform gaze analysis, etc., to examine how the user consumes the point cloud, the area within the point cloud video the user is gazing at, the duration of the gaze, and other factors. According to an embodiment, the receiving device 10004 may transmit feedback information including the gaze analysis results to the sending device 10000. According to an embodiment, the feedback information may be obtained during the rendering and / or display process. According to an embodiment, the feedback information may be acquired by one or more sensors included in the receiving device 10004. Depending on the implementation, the feedback information may be obtained by the renderer 10007 or a separate external element (or device, component, etc.). Figure 1The dotted line in represents the process of sending the feedback information obtained by the renderer 10007. The point cloud content providing system can process (encode / decode) the point cloud data based on the feedback information. Therefore, the point cloud video decoder 10006 can perform a decoding operation based on the feedback information. The receiving device 10004 can send the feedback information to the sending device 10000. The sending device 10000 (or the point cloud video data encoder 10002) can perform an encoding operation based on the feedback information. Therefore, the point cloud content providing system can effectively process necessary data (for example, point cloud data corresponding to the user's head position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide the point cloud content to the user.

[0068] Depending on the embodiment, the transmitting device 10000 may be referred to as an encoder, a transmitting device, a transmitter, etc., and the receiving device 10004 may be referred to as a decoder, a receiving device, a receiver, etc.

[0069] According to the embodiment Figure 1 Point cloud data processed in a point cloud content providing system (through a series of processes such as acquisition / encoding / transmission / decoding / rendering) may be referred to as point cloud content data or point cloud video data. Depending on the embodiment, point cloud content data may be used as a concept that encompasses metadata or signaling information related to point cloud data.

[0070] Figure 1 The elements of the illustrated point cloud content providing system may be implemented by hardware, software, a processor, and / or a combination thereof.

[0071] Figure 2 is a block diagram illustrating a point cloud content providing operation according to an embodiment.

[0072] Figure 2 The block diagram shows Figure 1 The point cloud content providing system described in

[0045] As described above, the point cloud content providing system may process point cloud data based on point cloud compression coding (eg, G-PCC).

[0073] According to an embodiment, a point cloud content providing system (e.g., a point cloud sending device 10000 or a point cloud video acquirer 10001) can acquire a point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system for expressing a 3D space. According to an embodiment, a point cloud video may include a Ply (Polygon file format or Stanford Triangle format) file. When a point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of a point. The position of each point can be represented by parameters (e.g., X, Y, and Z axis values) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include attributes of the point (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc. of each point). A point has one or more attributes. For example, a point may have a color attribute or two attributes, color and reflectivity. Depending on the embodiment, geometry may be referred to as position, geometry information, geometry data, etc., and attribute may be referred to as attribute, attribute information, attribute data, etc. The point cloud content providing system (e.g., the point cloud transmitting device 10000 or the point cloud video acquirer 10001) may obtain point cloud data from information related to the point cloud video acquisition process (e.g., depth information, color information, etc.).

[0074] According to an embodiment, a point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) may encode point cloud data (20001). The point cloud content providing system may encode point cloud data based on point cloud compression coding. As described above, point cloud data may include geometry and attributes of points. Therefore, the point cloud content providing system may perform geometry coding for encoding the geometry and output a geometry bitstream. The point cloud content providing system may perform attribute coding for encoding the attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system may perform attribute coding based on geometry coding. The geometry bitstream and the attribute bitstream according to the embodiment may be multiplexed and output as one bitstream. The bitstream according to the embodiment may also include signaling information related to geometry coding and attribute coding.

[0075] The point cloud content providing system according to the embodiment (eg, the transmitting device 10000 or the transmitter 10003) may transmit the encoded point cloud data (20002). Figure 1 As shown, the encoded point cloud data can be represented by a geometry bitstream and an attribute bitstream. In addition, the encoded point cloud data can be sent in the form of a bitstream together with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometry encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and send it in the form of a file or fragment.

[0076] According to an embodiment, a point cloud content providing system (e.g., receiving device 10004 or receiver 10005) may receive a bitstream containing encoded point cloud data. In addition, the point cloud content providing system (e.g., receiving device 10004 or receiver 10005) may demultiplex the bitstream.

[0077] The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) can decode the encoded point cloud data (e.g., geometry bitstream, attribute bitstream) sent in the bitstream. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) can decode the point cloud video data based on the signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) can decode the geometry bitstream to reconstruct the position (geometry) of the point. The point cloud content providing system can reconstruct the attributes of the point by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) can reconstruct the point cloud video based on the position according to the reconstructed geometry and the decoded attributes.

[0078] According to an embodiment, a point cloud content providing system (e.g., receiving device 10004 or renderer 10007) can render the decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) can use various rendering methods to render the geometry and attributes decoded by the decoding process. Points in the point cloud content can be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to the user via a display (e.g., a VR / AR display, a general display, etc.).

[0079] According to an embodiment, the point cloud content providing system (e.g., receiving device 10004) may obtain feedback information (20005). The point cloud content providing system may encode and / or decode the point cloud data based on the feedback information. Figure 1 The feedback information and operations described are the same, so their detailed description is omitted.

[0080] Figure 3 An exemplary process of capturing point cloud video according to an embodiment is shown.

[0081] Figure 3 Show reference Figures 1 to 2 An exemplary point cloud video capture process of a point cloud content providing system is described.

[0082] Point cloud content includes point cloud videos (images and / or videos) representing objects and / or environments located in various 3D spaces (e.g., 3D spaces representing real environments, 3D spaces representing virtual environments, etc.). Therefore, a point cloud content providing system according to an embodiment may use one or more cameras (e.g., an infrared camera capable of obtaining depth information, an RGB camera capable of extracting color information corresponding to the depth information, etc.), a projector (e.g., an infrared pattern projector that obtains depth information), LiDAR, etc. to capture point cloud videos. A point cloud content providing system according to an embodiment may extract a geometric shape composed of points in a 3D space from the depth information and extract attributes of each point from the color information to obtain point cloud data. Images and / or videos according to an embodiment may be captured based on at least one of an inward-facing technique and an outward-facing technique.

[0083] Figure 3 The left portion of FIG shows an inward-facing technique. Inward-facing techniques refer to techniques for capturing images of a central object using one or more cameras (or camera sensors) positioned around the central object. Inward-facing techniques can be used to generate point cloud content that provides a 360-degree image of a key object to the user (e.g., VR / AR content that provides a 360-degree image of an object (e.g., a key object such as a character, player, object, or actor) to the user).

[0084] Figure 3 The right side of the figure shows outward-facing techniques. Outward-facing techniques utilize one or more cameras (or camera sensors) positioned around a central object to capture the surroundings of the central object rather than an image of the central object. Outward-facing techniques can be used to generate point cloud content that provides the surrounding environment as it appears from the user's perspective (e.g., content representing the external environment that can be provided to a user of a self-driving vehicle).

[0085] As shown in the figure, point cloud content may be generated based on a capture operation of one or more cameras. In this case, the coordinate system may be different between the cameras, so the point cloud content providing system may calibrate one or more cameras to set a global coordinate system before the capture operation. In addition, the point cloud content providing system may generate point cloud content by synthesizing arbitrary images and / or videos with the images and / or videos captured by the above-mentioned capture technology. The point cloud content providing system may not perform a point cloud operation when generating point cloud content representing a virtual space. Figure 3 The point cloud content providing system according to an embodiment may perform post-processing on the captured image and / or video. In other words, the point cloud content providing system may remove unwanted areas (e.g., background), identify the space to which the captured image and / or video is connected, and, when there is a space hole, perform an operation to fill the space hole.

[0086] The point cloud content providing system generates a piece of point cloud content by performing coordinate transformation on points in point cloud videos acquired from various cameras. The point cloud content providing system performs coordinate transformation on the points based on the position coordinates of each camera. This allows the point cloud content providing system to generate content representing a wide range or point cloud content with a high point density.

[0087] Figure 4 An exemplary point cloud encoder according to an embodiment is shown.

[0088] Figure 4 Show Figure 1 An example of a point cloud video encoder 10002 is provided. The point cloud encoder reconstructs and encodes point cloud data (e.g., the position and / or attributes of a point) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) according to network conditions or applications. When the total size of the point cloud content is large (e.g., 60 Gbps of point cloud content for 30 fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system may reconstruct the point cloud content based on a maximum target bit rate to provide the point cloud content according to the network environment, etc.

[0089] As reference Figures 1 to 2 As described above, the point cloud encoder can perform both geometry encoding and attribute encoding. Geometry encoding is performed before attribute encoding.

[0090] The point cloud encoder according to an embodiment includes a coordinate transformer (transforming coordinates) 40000, a quantizer (quantizing and removing points (voxelization)) 40001, an octree analyzer (analyzing octree) 40002 and a surface approximation analyzer (analyzing surface approximation) 40003, an arithmetic encoder (arithmetic coding) 40004, a geometry reconstructor (reconstructing geometry) 40005, a color transformer (transforming color) 40006, an attribute transformer (transforming attributes) 40007, a RAHT transformer (RAHT) 40008, an LOD generator (generating LOD) 40009, a lifting transformer (lifting) 40010, a coefficient quantizer (quantizing coefficients) 40011 and / or an arithmetic encoder (arithmetic coding) 40012.

[0091] The coordinate transformer 40000, quantizer 40001, octree analyzer 40002, surface approximation analyzer 40003, arithmetic encoder 40004, and geometry reconstructor 40005 may perform geometry coding. Geometric coding according to embodiments may include octree geometry coding, direct coding, triplet geometry coding, and entropy coding. Direct coding and triplet geometry coding may be applied selectively or in combination. Geometric coding is not limited to the above examples.

[0092] As shown in the figure, the coordinate converter 40000 according to an embodiment receives a position and converts it into coordinates. For example, the position can be converted into position information in a three-dimensional space (e.g., a three-dimensional space represented by an XYZ coordinate system). The position information in the three-dimensional space according to an embodiment can be referred to as geometric information.

[0093] The quantizer 40001 according to an embodiment quantizes the geometry. For example, the quantizer 40001 may quantize the points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). The quantizer 40001 performs a quantization operation by multiplying the difference between the minimum position value and the position value of each point by a preset quantization scale value, and then finding the nearest integer value by rounding the value obtained by the multiplication. Therefore, one or more points may have the same quantized position (or position value). The quantizer 40001 according to an embodiment performs voxelization based on the quantized position to reconstruct the quantized points. As in the case of a pixel (the smallest unit containing 2D image / video information), the points of the point cloud content (or 3D point cloud video) according to an embodiment may be included in one or more voxels. As a composite of volume and pixel, the term voxel refers to the 3D cubic space generated when the 3D space is divided into units (unit = 1.0) based on the axes representing the 3D space (e.g., the X, Y, and Z axes). Quantizer 40001 can match point groups in 3D space to voxels. Depending on the embodiment, a voxel may include only one point. Depending on the embodiment, a voxel may include one or more points. To represent a voxel as a point, the position of the voxel's center can be set based on the positions of one or more points included in the voxel. In this case, the attributes of all positions included in a voxel can be combined and assigned to the voxel.

[0094] The octree analyzer 40002 according to an embodiment performs octree geometry encoding (or octree encoding) to represent voxels in an octree structure. The octree structure represents points that are matched to voxels based on an octal tree structure.

[0095] The surface approximation analyzer 40003 according to an embodiment may analyze and approximate an octree. The octree analysis and approximation according to an embodiment is a process of analyzing a region including a plurality of points to efficiently provide an octree and voxelization.

[0096] According to an embodiment, the arithmetic encoder 40004 performs entropy coding on the octree and / or approximate octree. For example, the coding scheme includes arithmetic coding. As a result of the coding, a geometry bitstream is generated.

[0097] The color converter 40006, attribute converter 40007, RAHT converter 40008, LOD generator 40009, lifting converter 40010, coefficient quantizer 40011, and / or arithmetic encoder 40012 perform attribute coding. As described above, a point may have one or more attributes. Attribute coding according to an embodiment is also applied to the attributes of a point. However, when an attribute (e.g., color) includes one or more elements, attribute coding is applied independently to each element. Attribute coding according to an embodiment includes color transform coding, attribute transform coding, region adaptive hierarchical transform (RAHT) coding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) coding, and interpolation-based hierarchical nearest neighbor prediction (lifting transform) coding with an update / lifting step. Depending on the point cloud content, the above-mentioned RAHT coding, prediction transform coding, and lifting transform coding can be selectively used, or a combination of one or more coding schemes can be used. Attribute coding according to an embodiment is not limited to the above examples.

[0098] Color converter 40006 according to an embodiment performs color conversion encoding to convert the color value (or texture) included in the attribute. For example, color converter 40006 may convert the format of color information (e.g., from RGB to YCbCr). Alternatively, the operation of color converter 40006 according to an embodiment may be applied based on the color value included in the attribute.

[0099] The geometry reconstructor 40005 according to an embodiment reconstructs (decompresses) an octree and / or an approximate octree. The geometry reconstructor 40005 reconstructs the octree / voxel based on the result of analyzing the point distribution. The reconstructed octree / voxel may be referred to as reconstructed geometry (restored geometry).

[0100] The attribute converter 40007 according to an embodiment performs attribute conversion to convert attributes based on reconstructed geometry and / or locations where geometry encoding is not performed. As described above, since attributes depend on geometry, the attribute converter 40007 can convert attributes based on reconstructed geometry information. For example, based on the position value of a point included in a voxel, the attribute converter 40007 can convert the attributes of the point at that position. As described above, when the center position of a voxel is set based on the positions of one or more points included in the voxel, the attribute converter 40007 converts the attributes of one or more points. When triplet geometry encoding is performed, the attribute converter 40007 can convert attributes based on the triplet geometry encoding.

[0101] The attribute converter 40007 can perform attribute conversion by calculating the average of the attributes or attribute values ​​(e.g., the color or reflectivity of each point) of neighboring points within a specific position / radius from the center position (or position value) of each voxel. The attribute converter 40007 can apply weights based on the distance from the center to each point when calculating the average. Thus, each voxel has a position and a calculated attribute (or attribute value).

[0102] The attribute converter 40007 can search for neighboring points within a specific position / radius from the center position of each voxel based on a KD tree or a Morton code. The KD tree is a binary search tree and supports a data structure that can manage points based on position, so that a nearest neighbor search (NNS) can be performed quickly. The Morton code is generated by presenting the coordinates (e.g., (x, y, z)) representing the 3D positions of all points as bit values ​​and mixing the bits. For example, when the coordinates representing the point position are (5, 9, 1), the bit values ​​of the coordinates are (0101, 1001, 0001). Mixing the bit values ​​in the order of z, y, and x according to the bit index produces 010001000111. This value is represented as a decimal number 1095. That is, the Morton code value of the point with coordinates (5, 9, 1) is 1095. The attribute converter 40007 can sort the points based on the Morton code value and perform NNS through a depth-first traversal process. After the attribute transformation operation, KD tree or Morton code is used when NNS is needed in another transformation process for attribute encoding.

[0103] As shown, the transformed attributes are input to the RAHT transformer 40008 and / or the LOD generator 40009.

[0104] The RAHT transformer 40008 according to an embodiment performs RAHT encoding for predicting attribute information based on the reconstructed geometric information. For example, the RAHT transformer 40008 may predict attribute information of a higher-level node in the octree based on attribute information associated with a lower-level node in the octree.

[0105] According to an embodiment, LOD generator 40009 generates a level of detail (LOD) to perform predictive transform coding. According to an embodiment, LOD is the degree of detail of the point cloud content. As the LOD value decreases, the detail of the point cloud content degrades. As the LOD value increases, the detail of the point cloud content increases. Points can be categorized by LOD.

[0106] The lifting transformer 40010 according to an embodiment performs lifting transform coding that transforms point cloud attributes based on weights. As described above, lifting transform coding can be optionally applied.

[0107] The coefficient quantizer 40011 according to the embodiment quantizes the attribute of the attribute encoding based on the coefficient.

[0108] The arithmetic encoder 40012 according to the embodiment encodes quantized properties based on arithmetic coding.

[0109] Although not shown in the figure, Figure 4 The elements of the point cloud encoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. The one or more processors may execute the above Figure 4 In addition, one or more processors may be operable or executable to perform at least one of the operations and / or functions of the elements of the point cloud encoder. Figure 4 The one or more memories of the embodiment may include high-speed random access memory, or include non-volatile memory (e.g., one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0110] Figure 5 An example of voxels according to an embodiment is shown.

[0111] Figure 5 The voxels are shown in a 3D space represented by a coordinate system consisting of three axes (X axis, Y axis and Z axis). Figure 4 As described above, the point cloud encoder (e.g., quantizer 40001) may perform voxelization. A voxel refers to a 3D cubic space generated by dividing a 3D space into units (unit=1.0) based on axes representing the 3D space (e.g., X-axis, Y-axis, and Z-axis). Figure 5 An example of voxels generated by an octree structure is shown, where a cubic axis-aligned bounding box defined by two poles (0,0,0) and (2d,2d,2d) is recursively subdivided. A voxel includes at least one point. The spatial coordinates of the voxel can be estimated from the positional relationship with the voxel group. As described above, the voxel has properties similar to the pixels of a 2D image / video (e.g., color or reflectivity). The details of the voxel are similar to those of the reference Figure 4 Those described are the same, so their description is omitted.

[0112] Figure 6 Shown are examples of octrees and occupancy codes according to an embodiment.

[0113] As reference Figures 1 to 4 As described, the point cloud content providing system (point cloud video encoder 10002) or the point cloud encoder (e.g., octree analyzer 40002) performs octree geometry encoding (or octree encoding) based on the octree structure to efficiently manage the area and / or position of voxels.

[0114] Figure 6 The upper part of FIG shows the octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes (eg, X-axis, Y-axis, and Z-axis) of the coordinate system. The octree structure is represented by the two poles (0,0,0) and (2 d ,2 d ,2 d ) to create an octree structure. Here, 2 d It can be set to the value of the minimum bounding box that surrounds all points of the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following formula. In the following formula, (x int n ,y int n ,z int n ) represents the position (or position value) of the quantized point.

[0115]

[0116] like Figure 6 As shown in the middle of the upper part of , the entire 3D space can be divided into eight spaces according to the partition. Each divided space is represented by a cube with six faces. Figure 6 As shown in the upper right portion of the octree, each of the eight spaces is further divided based on the axes of the coordinate system (e.g., X-axis, Y-axis, and Z-axis). Thus, each space is divided into eight smaller spaces. Each of the divided smaller spaces is also represented by a cube with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0117] Figure 6 The lower part shows the octree occupancy code. The occupancy code of the octree is generated to indicate whether each of the eight divided spaces generated by dividing one space contains at least one point. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of the divided space, and the child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to the child node contains at least one point, the node is assigned a value of 1. When the space corresponding to the child node does not contain a point (the space is empty), the node is assigned a value of 0. Since Figure 6The occupancy code shown is 00100001, indicating that the spaces corresponding to the third and eighth child nodes among the eight child nodes each contain at least one point. As shown in the figure, each of the third and eighth child nodes has eight child nodes, and the child nodes are represented by 8-bit occupancy codes. The accompanying figure shows that the occupancy code of the third child node is 10000111, and the occupancy code of the eighth child node is 01001111. The point cloud encoder according to the embodiment (e.g., the arithmetic encoder 40004) may perform entropy coding on the occupancy code. In order to increase compression efficiency, the point cloud encoder may perform intra-frame / inter-frame coding on the occupancy code. The receiving device according to the embodiment (e.g., the receiving device 10004 or the point cloud video decoder 10006) reconstructs the octree based on the occupancy code.

[0118] According to an embodiment of the point cloud encoder (e.g., Figure 4 The point cloud encoder or octree analyzer 40002 may perform voxelization and octree encoding to store point locations. However, points are not always evenly distributed in 3D space, so there may be specific areas with fewer points. Therefore, performing voxelization on the entire 3D space is inefficient. For example, when a specific area contains very few points, voxelization does not need to be performed in that specific area.

[0119] Therefore, for the above-mentioned specific area (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment may skip voxelization and perform direct encoding to directly encode the point positions included in the specific area. The coordinates of the directly encoded points according to the embodiment are called direct coding mode (DCM). The point cloud encoder according to the embodiment may also perform triplet geometry encoding based on the surface model, which is to reconstruct the point positions in the specific area (or node) based on voxels. Triplet geometry encoding is a geometry encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct encoding and triplet geometry encoding according to the embodiment can be performed selectively. In addition, direct encoding and triplet geometry encoding according to the embodiment can be performed in combination with octree geometry encoding (or octree encoding).

[0120] To perform direct encoding, the option to use direct mode to apply direct encoding should be enabled. The node to which direct encoding is to be applied is not a leaf node, and there should be less than a threshold number of points within the specific node. In addition, the total number of points to which direct encoding is to be applied should not exceed a preset threshold. When the above conditions are met, the point cloud encoder (or arithmetic encoder 40004) according to the embodiment can perform entropy encoding on the point position (or position value).

[0121] A point cloud encoder according to an embodiment (e.g., a surface approximation analyzer 40003) may determine a specific level of the octree (a level less than the depth d of the octree), and may use a surface model starting from this level to perform triplet geometry encoding to reconstruct the point positions in the node area based on voxels (triplet mode). A point cloud encoder according to an embodiment may specify the level to which triplet geometry encoding is to be applied. For example, when the specific level is equal to the depth of the octree, the point cloud encoder does not operate in triplet mode. In other words, the point cloud encoder according to an embodiment may operate in triplet mode only when the specified level is less than the depth value of the octree. A 3D cubic area of ​​a node at a specified level according to an embodiment is called a block. A block may include one or more voxels. A block or voxel may correspond to a block. The geometry is represented as a surface within each block. A surface according to an embodiment may intersect each edge of the block at most once.

[0122] A block has 12 edges, so there are at least 12 intersections within a block. Each intersection is called a vertex. Vertices along an edge are detected when there is at least one occupied voxel adjacent to the edge across all blocks that share the edge. An occupied voxel, according to embodiments, refers to a voxel containing a point. The vertex position detected along an edge is the average position of all voxels adjacent to the edge across all blocks along the edge.

[0123] Once the vertex is detected, the point cloud encoder according to an embodiment may perform entropy encoding on the edge start point (x, y, z), the edge direction vector (Δx, Δy, Δz), and the vertex position value (relative position value within the edge). When triplet geometry encoding is applied, the point cloud encoder according to an embodiment (e.g., geometry reconstructor 40005) may generate recovered geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization processes.

[0124] Vertices located at the edges of a block define a surface that passes through the block. The surface according to an embodiment is a non-planar polygon. During triangle reconstruction, the surface represented by the triangles is reconstructed based on the starting points of the edges, the direction vectors of the edges, and the position values ​​of the vertices. The triangle reconstruction process is performed by i) calculating the centroid value of each vertex, ii) subtracting the centroid value from each vertex value, and iii) estimating the sum of the squares of the values ​​obtained by the subtraction.

[0125]

[0126] The minimum value of the sum is estimated, and the projection process is performed along the axis with the minimum value. For example, when the element x is minimum, each vertex is projected onto the x-axis relative to the center of the block and onto the (y,z) plane. When the value obtained by projection onto the (y,z) plane is (ai,bi), the value of θ is estimated by atan2(bi,ai), and the vertices are sorted based on the value of θ. The following table shows the vertex combinations that create triangles based on the number of vertices. The vertices are sorted from 1 to n. The following table shows that for four vertices, two triangles can be constructed based on the vertex combinations. The first triangle can be composed of vertices 1, 2, and 3 among the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 among the sorted vertices.

[0127] Table 2-1. Triangles formed from vertices sorted 1,…,n

[0128] n triangle

[0129] 3(1,2,3)

[0130] 4(1,2,3),(3,4,1)

[0131] 5(1,2,3),(3,4,5),(5,1,3)

[0132] 6(1,2,3),(3,4,5),(5,6,1),(1,3,5)

[0133] 7(1,2,3),(3,4,5),(5,6,7),(7,1,3),(3,5,7)

[0134] 8(1,2,3),(3,4,5),(5,6,7),(7,8,1),(1,3,5),(5,7,1)

[0135] 9(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,1,3),(3,5,7),(7,9,3)

[0136] 10(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,1),(1,3,5),(5,7,9),(9,1,5)

[0137] 11(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,11),(11,1,3),(3,5,7),(7,9,11),(11,3,7)

[0138] 12(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,11),(11,12,1),(1,3,5),(5,7,9),(9,11,1),(1,5,9)

[0139] An upsampling process is performed to add points in the middle along the edges of the triangle, and voxelization is performed. The added points are generated based on the upsampling factor and the width of the block. The added points are called refinement vertices. The point cloud encoder according to an embodiment may voxelize the refinement vertices. In addition, the point cloud encoder may perform attribute encoding based on the voxelized positions (or position values).

[0140] Figure 7 An example of a neighbor node pattern according to an embodiment is shown.

[0141] In order to increase the compression efficiency of the point cloud video, the point cloud encoder according to an embodiment may perform entropy encoding based on context-adaptive arithmetic coding.

[0142] As reference Figures 1 to 6 As described, the point cloud content providing system or point cloud encoder (eg, point cloud video encoder 10002, Figure 4 The point cloud encoder or arithmetic encoder 40004) may immediately perform entropy coding on the occupancy code. In addition, the point cloud content providing system or the point cloud encoder may perform entropy coding (intra-frame coding) based on the occupancy code of the current node and the occupancy of the neighboring nodes, or perform entropy coding (inter-frame coding) based on the occupancy code of the previous frame. The frame representation according to the embodiment is a collection of point cloud videos generated simultaneously. The compression efficiency of the intra-frame coding / inter-frame coding according to the embodiment may depend on the number of neighboring nodes referenced. When the number of bits increases, the operation becomes complicated, but the coding can be biased to one side, which can increase the compression efficiency. For example, when a 3-bit context is given, 2 bits need to be used. 3 = 8 ways to perform encoding. The division of parts for encoding affects the implementation complexity. Therefore, it is necessary to meet the appropriate level of compression efficiency and complexity.

[0143] Figure 7 The process of obtaining an occupancy pattern based on the occupancy of neighbor nodes is shown. According to an embodiment, a point cloud encoder determines the occupancy of neighbor nodes of each node of an octree and obtains the value of a neighbor pattern. The neighbor node pattern is used to infer the occupancy pattern of the node. Figure 7 The upper portion of the diagram shows the cube corresponding to the node (the middle cube) and the six cubes (neighboring nodes) that share at least one face with it. The nodes shown in the diagram are at the same depth. The numbers in the diagram represent the weights associated with the six nodes (1, 2, 4, 8, 16, and 32), respectively. Weights are assigned sequentially based on the positions of neighboring nodes.

[0144] Figure 7 The lower part shows the neighbor node pattern value. The neighbor node pattern value is the sum of the values ​​multiplied by the weights of the occupied neighbor nodes (neighbor nodes with points). Therefore, the neighbor node pattern value is 0 to 63. When the neighbor node pattern value is 0, it indicates that there is no node with a point among the neighbor nodes of the node (no occupied node). When the neighbor node pattern value is 63, it indicates that all neighbor nodes are occupied nodes. As shown in the figure, since the neighbor nodes assigned with weights 1, 2, 4 and 8 are occupied nodes, the neighbor node pattern value is 15 (the sum of 1, 2, 4 and 8). The point cloud encoder can perform encoding according to the neighbor node pattern value (for example, when the neighbor node pattern value is 63, 64 types of encoding can be performed). According to an embodiment, the point cloud encoder can reduce the encoding complexity by changing the neighbor node pattern value (for example, based on a table that changes 64 to 10 or 6).

[0145] Figure 8 An example of point arrangement in each LOD according to an embodiment is shown.

[0146] As reference Figures 1 to 7 As described, the encoded geometry is reconstructed (decompressed) before attribute encoding is performed. When direct encoding is applied, the geometry reconstruction operation may include changing the placement of directly encoded points (e.g., placing directly encoded points in front of the point cloud data). When triplet geometry encoding is applied, the geometry reconstruction process is performed through triangle reconstruction, upsampling, and voxelization. Since attributes depend on the geometry, attribute encoding is performed based on the reconstructed geometry.

[0147] The point cloud encoder (e.g., LOD generator 40009) can classify (reorganize) points by LOD. The figure shows the point cloud content corresponding to the LOD. The leftmost frame in the figure shows the original point cloud content. The second frame from the left in the figure shows the point distribution in the lowest LOD, and the rightmost frame in the figure shows the point distribution in the highest LOD. In other words, the points in the lowest LOD are sparsely distributed, while the points in the highest LOD are densely distributed. In other words, as the LOD increases in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between points becomes narrower.

[0148] Figure 9 An example of point configuration for each LOD according to an embodiment is shown.

[0149] As reference Figures 1 to 8 As described, the point cloud content providing system or point cloud encoder (eg, point cloud video encoder 10002, Figure 4The point cloud encoder or LOD generator 40009 can generate LODs. LODs are generated by reorganizing points into a set of refinement levels based on a set LOD distance value (or Euclidean distance set). The LOD generation process is performed not only by the point cloud encoder but also by the point cloud decoder.

[0150] Figure 9 The upper part of shows examples of points (P0 to P9) of point cloud contents distributed in 3D space. Figure 9 In , the original order represents the order of points P0 to P9 before LOD generation. Figure 9 In the LOD-based order, the order of points generated according to the LOD is shown. Points are reorganized by LOD. In addition, higher LODs contain points belonging to lower LODs. Figure 9 As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, LOD1, P9, P8, and P7.

[0151] As reference Figure 4 As described, the point cloud encoder according to the embodiment may selectively or in combination perform predictive transform coding, lifting transform coding, and RAHT transform coding.

[0152] The point cloud encoder according to an embodiment may generate a predictor for each point to perform predictive transform coding for setting a predicted attribute (or predicted attribute value) for each point. That is, N predictors may be generated for N points. The predictor according to an embodiment may calculate a weight (=1 / distance) based on the LOD value of each point, index information about neighboring points within a set distance of each LOD, and the distance to the neighboring point.

[0153] The predicted attribute (or attribute value) according to the embodiment is set to the average of the values ​​obtained by multiplying the attribute (or attribute value) (e.g., color, reflectivity, etc.) of the neighboring points set in the predictor of each point by the weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 40011) according to the embodiment can quantize and inverse quantize the residual (which may be referred to as residual attribute, residual attribute value, or attribute prediction residual) obtained by subtracting the predicted attribute (attribute value) from the attribute (attribute value) of each point. The quantization process is configured as shown in the following table.

[0154] Table. Pseudocode for attribute prediction residual quantization

[0155] int PCCQuantization(int value,int quantStep){

[0156] if(value>=0){

[0157] return floor(value / quantStep+1.0 / 3.0);

[0158] }else{

[0159] return-floor(-value / quantStep+1.0 / 3.0);

[0160] }

[0161] }

[0162] Table. Pseudocode for inverse quantization of attribute prediction residuals

[0163] int PCCInverseQuantization(int value,int quantStep){

[0164] if(quantStep==0){

[0165] return value;

[0166] }else{

[0167] return value*quantStep;

[0168] }

[0169] }

[0170] When the predictor of each point has neighboring points, the point cloud encoder (e.g., the arithmetic encoder 40012) according to an embodiment may perform entropy encoding on the quantized and inverse quantized residual values ​​as described above. When the predictor of each point has no neighboring points, the point cloud encoder (e.g., the arithmetic encoder 40012) according to an embodiment may perform entropy encoding on the attributes of the corresponding point without performing the above operation.

[0171] The point cloud encoder (e.g., lifting transformer 40010) according to an embodiment may generate a predictor for each point, set the calculated LOD, register neighboring points in the predictor, and set weights based on the distances to the neighboring points to perform lifting transform encoding. Lifting transform encoding according to an embodiment is similar to the above-described predictive transform encoding, but differs in that weights are cumulatively applied to attribute values. The process of cumulatively applying weights to attribute values ​​according to an embodiment is configured as follows.

[0172] 1) Create an array called quantized weights (QW) to store the weight values ​​for each point. All elements of QW are initially set to 1.0. Multiply the QW value of the predictor index of the neighboring node registered in the predictor by the weight of the predictor for the current point, and add the resulting values.

[0173] 2) Boosting prediction process: The value obtained by multiplying the attribute value of the point by the weight is subtracted from the existing attribute value to calculate the predicted attribute value.

[0174] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0175] 4) The weight calculated by multiplying the weight calculated for all predictors by the weight stored in the QW corresponding to the predictor index is added to the updateweight array as the index of the neighbor node. The value obtained by multiplying the attribute value of the neighbor node index by the calculated weight is added to the update array.

[0176] 5) Boosting update process: Divide the attribute values ​​of the update array of all predictors by the weight value of the updateweight array of the predictor index, and add the existing attribute value to the value obtained by the division.

[0177] 6) For all predictors, the predicted attribute is calculated by multiplying the attribute value updated by the lifting update process by the weight (stored in QW) updated by the lifting prediction process. The point cloud encoder (e.g., coefficient quantizer 40011) according to the embodiment quantizes the predicted attribute value. In addition, the point cloud encoder (e.g., arithmetic encoder 40012) performs entropy encoding on the quantized attribute value.

[0178] A point cloud encoder according to an embodiment (e.g., RAHT transformer 40008) may perform RAHT transform coding, in which attributes associated with nodes at a lower level in an octree are used to predict attributes of nodes at a higher level. RAHT transform coding is an example of intra-coding of attributes by scanning backward through an octree. A point cloud encoder according to an embodiment scans the entire area starting from voxels and repeats a merging process of merging voxels into larger blocks at each step until a root node is reached. The merging process according to an embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on the upper node directly above the empty node.

[0179] The following formula represents the RAHT transformation matrix. In this formula, Represents the average attribute value of voxels at level l. Can be based on and To calculate. and The weight is and

[0180] here, is the low-pass value and is used during the next highest level of merging. Denotes the high-pass coefficient. The high-pass coefficient of each step is quantized and subjected to entropy coding (eg, by arithmetic coder 400012). The weight is calculated as pass and Create the root node as follows.

[0181]

[0182] Figure 10 A point cloud decoder according to an embodiment is shown.

[0183] Figure 10 The point cloud decoder shown is Figure 1 An example of a point cloud video decoder 10006 described in Figure 1 The operations of the point cloud video decoder 10006 shown in FIG. 10006 are the same as or similar to those of the point cloud video decoder 10006 shown in FIG. As shown in the figure, the point cloud decoder can receive a geometry bitstream and an attribute bitstream contained in one or more bitstreams. The point cloud decoder includes a geometry decoder and an attribute decoder. The geometry decoder performs geometry decoding on the geometry bitstream and outputs decoded geometry. The attribute decoder performs attribute decoding based on the decoded geometry and attribute bitstreams and outputs decoded attributes. The decoded geometry and decoded attributes are used to reconstruct the point cloud content (decoded point cloud).

[0184] Figure 11 A point cloud decoder according to an embodiment is shown.

[0185] Figure 11 The point cloud decoder shown is Figure 10 An example of a point cloud decoder is shown, and a decoding operation can be performed, which is Figures 1 to 9 The inverse process of the encoding operation of the point cloud encoder is shown.

[0186] As reference Figure 1 and Figure 10 As described, the point cloud decoder can perform both geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0187] According to an embodiment, the point cloud decoder includes an arithmetic decoder (arithmetic decoding) 11000, an octree synthesizer (synthesize octree) 11001, a surface approximation synthesizer (synthesize surface approximation) 11002 and a geometry reconstructor (reconstruct geometry) 11003, an inverse coordinate transformer (inverse transform coordinates) 11004, an arithmetic decoder (arithmetic decoding) 11005, an inverse quantizer (inverse quantization) 11006, a RAHT transformer 11007, an LOD generator (generate LOD) 11008, an inverse lifting (inverse lifting) 11009 and / or an inverse color transformer (inverse transform color) 11010.

[0188] The arithmetic decoder 11000, the octree synthesizer 11001, the surface approximation synthesizer 11002, the geometry reconstructor 11003, and the coordinate inverse transformer 11004 may perform geometry decoding. The geometry decoding according to the embodiment may include direct encoding and triplet geometry decoding. Direct encoding and triplet geometry decoding are selectively applied. The geometry decoding is not limited to the above example, and as a reference Figures 1 to 9 The inverse process of the geometric encoding described is performed.

[0189] The arithmetic decoder 11000 according to the embodiment decodes the received geometry bitstream based on arithmetic coding. The operation of the arithmetic decoder 11000 corresponds to the inverse process of the arithmetic encoder 40004.

[0190] The octree synthesizer 11001 according to the embodiment can generate an octree by obtaining an occupancy code (or information about the geometry obtained as a result of decoding) from the decoded geometry bitstream. Figures 1 to 9 Describe the configuration in detail.

[0191] When triplet geometry encoding is applied, the surface approximation synthesizer 11002 according to an embodiment may synthesize a surface based on the decoded geometry and / or the generated octree.

[0192] The geometry reconstructor 11003 according to an embodiment may regenerate the geometry based on the surface and / or decoded geometry. Figures 1 to 9 As described, direct coding and triplet geometry coding are selectively applied. Therefore, the geometry reconstructor 11003 directly imports the position information about the points to which direct coding is applied and adds them. When triplet geometry coding is applied, the geometry reconstructor 11003 can reconstruct the geometry by performing the reconstruction operations (e.g., triangle reconstruction, upsampling, and voxelization) of the geometry reconstructor 40005. Details and References Figure 6 The reconstructed geometry may include a point cloud frame or screen that does not contain attributes.

[0193] The coordinate inverse transformer 11004 according to an embodiment may acquire a point position by transforming the coordinates based on the reconstructed geometry.

[0194] The arithmetic decoder 11005, the inverse quantizer 11006, the RAHT transformer 11007, the LOD generator 11008, the inverse lifter 11009 and / or the inverse color transformer 11010 may perform a reference Figure 10 Attribute decoding described. Attribute decoding according to an embodiment includes region adaptive hierarchical transform (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) decoding, and interpolation-based hierarchical nearest neighbor prediction (lifting transform) decoding with an update / lifting step. The above three decoding schemes can be used selectively, or a combination of one or more decoding schemes can be used. Attribute decoding according to an embodiment is not limited to the above examples.

[0195] The arithmetic decoder 11005 according to the embodiment decodes the attribute bit stream through arithmetic coding.

[0196] The inverse quantizer 11006 according to an embodiment inversely quantizes information about a decoded attribute bitstream or an attribute obtained as a result of decoding, and outputs the inversely quantized attribute (or attribute value). Inverse quantization may be selectively applied based on attribute encoding of the point cloud encoder.

[0197] Depending on the embodiment, the RAHT transformer 11007, the LOD generator 11008, and / or the inverse lifter 11009 may process the reconstructed geometry and inverse quantized attributes. As described above, the RAHT transformer 11007, the LOD generator 11008, and / or the inverse lifter 11009 may selectively perform a decoding operation corresponding to the encoding of the point cloud encoder.

[0198] The color inverse converter 11010 according to an embodiment performs inverse transform encoding to inversely transform the color value (or texture) included in the decoded attribute. The operation of the color inverse converter 11010 may be selectively performed based on the operation of the color converter 40006 of the point cloud encoder.

[0199] Although not shown in the figure, Figure 11 The elements of the point cloud decoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. The one or more processors may execute the above Figure 11 In addition, one or more processors may be operable or executed to perform at least one or more of the operations and / or functions of the elements of the point cloud decoder. Figure 11 A software program and / or instruction set for the operation and / or functionality of the elements of the point cloud decoder.

[0200] Figure 12 A transmitting device according to an embodiment is shown.

[0201] Figure 12 The sending device shown is Figure 1 The sending device 10000 (or Figure 4 An example of a point cloud encoder. Figure 12 The sending device shown can be executed with reference to Figures 1 to 9 The transmitting apparatus according to the embodiment may include a data input unit 12000, a quantization processor 12001, a voxelization processor 12002, an octree occupancy code generator 12003, a surface model processor 12004, an intra / inter coding processor 12005, an arithmetic encoder 12006, a metadata processor 12007, a color transform processor 12008, an attribute transform processor 12009, a prediction / lifting / RAHT transform processor 12010, an arithmetic encoder 12011 and / or a transmission processor 12012.

[0202] The data input unit 12000 according to the embodiment receives or acquires point cloud data. The data input unit 12000 may perform the same operation and / or acquisition method as the point cloud video acquirer 10001 (or refer to Figure 2 The acquisition process 20000) described above has the same or similar operations and / or acquisition methods.

[0203] The data input unit 12000, the quantization processor 12001, the voxelization processor 12002, the octree occupancy code generator 12003, the surface model processor 12004, the intra / inter coding processor 12005 and the arithmetic encoder 12006 perform geometric coding. Figures 1 to 9 The geometric encoding described is the same or similar, so its detailed description is omitted.

[0204] The quantization processor 12001 according to an embodiment quantizes geometry (e.g., position values ​​of points). The operation and / or quantization of the quantization processor 12001 are related to the reference Figure 4 The operation and / or quantization of the quantizer 40001 described above are the same or similar. Figures 1 to 9 Same as those described.

[0205] The voxelization processor 12002 according to the embodiment voxelizes the quantized position value of the point. The voxelization processor 120002 may perform the same operation as the reference. Figure 4 The operation and / or voxelization process of the quantizer 40001 described above is the same or similar to the operation and / or process described above. Figures 1 to 9 Same as those described.

[0206] The octree occupancy code generator 12003 according to the embodiment performs octree encoding on the voxelized position of the point based on the octree structure. The octree occupancy code generator 12003 can generate an occupancy code. The octree occupancy code generator 12003 can perform the octree encoding with reference to Figure 4 and Figure 6 The operations and / or methods of the point cloud encoder (or octree analyzer 40002) described herein are the same or similar to the operations and / or methods described herein. Figures 1 to 9 Same as those described.

[0207] According to an embodiment, the surface model processor 12004 may perform triplet geometry encoding based on the surface model to reconstruct the point position in a specific area (or node) based on voxels. Figure 4 The operations and / or methods of the point cloud encoder (e.g., surface approximation analyzer 40003) described herein are the same or similar to the operations and / or methods described herein. Figures 1 to 9 Same as those described.

[0208] The intra-frame / inter-frame encoding processor 12005 according to the embodiment may perform intra-frame / inter-frame encoding on the point cloud data. Figure 7 The same or similar encoding as described for intra / inter encoding. Details and references Figure 7 According to an embodiment, the intra / inter encoding processor 12005 may be included in the arithmetic encoder 12006.

[0209] According to an embodiment, arithmetic encoder 12006 performs entropy encoding on an octree and / or approximate octree of point cloud data. For example, the encoding scheme includes arithmetic coding. Arithmetic encoder 12006 performs operations and / or methods that are the same as or similar to those of arithmetic encoder 40004.

[0210] The metadata processor 12007 according to an embodiment processes metadata (e.g., set values) about the point cloud data and provides it to necessary processing processes such as geometry coding and / or attribute coding. In addition, the metadata processor 12007 according to an embodiment may generate and / or process signaling information related to geometry coding and / or attribute coding. The signaling information according to an embodiment may be encoded separately from the geometry coding and / or attribute coding. The signaling information according to an embodiment may be interleaved.

[0211] The color conversion processor 12008, the attribute conversion processor 12009, the prediction / lifting / RAHT conversion processor 12010, and the arithmetic encoder 12011 perform attribute coding. Figures 1 to 9The attribute codes described are the same or similar, so their detailed description is omitted.

[0212] The color transform processor 12008 according to an embodiment performs color transform encoding to transform the color value included in the attribute. The color transform processor 12008 may perform color transform encoding based on the reconstructed geometry. The reconstructed geometry is compared with the reference Figures 1 to 9 In addition, its execution is the same as that of reference Figure 4 The operations and / or methods of the color converter 40006 are the same as or similar to those described above, and detailed description thereof is omitted.

[0213] The attribute transformation processor 12009 according to an embodiment performs attribute transformation to transform attributes based on the reconstructed geometry and / or the location where geometry encoding is not performed. Figure 4 The operations and / or methods of the attribute converter 40007 described above are the same as or similar to those of the attribute converter 40007. Detailed description thereof is omitted. The prediction / lifting / RAHT transform processor 12010 according to the embodiment may encode the transformed attributes by any one or a combination of RAHT coding, prediction transform coding, and lifting transform coding. The prediction / lifting / RAHT transform processor 12010 performs the same as the reference. Figure 4 The operations of the RAHT transformer 40008, the LOD generator 40009 and the lifting transformer 40010 described above are identical or similar to at least one operation. In addition, the prediction transform coding, the lifting transform coding and the RAHT transform coding are the same as those of the reference Figures 1 to 9 Those described are the same, so detailed descriptions thereof are omitted.

[0214] The arithmetic encoder 12011 according to the embodiment may encode the properties of the encoding based on arithmetic coding. The arithmetic encoder 12011 performs the same or similar operations and / or methods as those of the arithmetic encoder 400012.

[0215] The transmission processor 12012 according to the embodiment may send individual bitstreams containing encoded geometry and / or encoded attribute and metadata information, or send a bitstream configured with encoded geometry and / or encoded attribute and metadata information. When the encoded geometry and / or encoded attribute and metadata information according to the embodiment is configured as a bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to the embodiment may include signaling information and slice data, and the signaling information includes a sequence parameter set (SPS) for sequence level signaling, a geometry parameter set (GPS) for geometry information coding signaling, an attribute parameter set (APS) for attribute information coding signaling, and a patch parameter set (TPS) for patch level signaling. The slice data may include information about one or more slices. A slice according to the embodiment may include a geometry bitstream Geom00 and one or more attribute bitstreams Attr00 and Attr10.

[0216] A slice is a sequence of syntax elements that represents all or part of a coded point cloud frame.

[0217] The TPS according to an embodiment may include information about each of the one or more tiles (e.g., coordinate information and height / size information about a bounding box). The geometry bitstream may include a header and a payload. The header of the geometry bitstream according to an embodiment may include a parameter set identifier (geom_parameter_set_id), a tile identifier (geom_tile_id), and a slice identifier (geom_slice_id) included in the GPS, as well as information about the data contained in the payload. As described above, the metadata processor 12007 according to an embodiment may generate and / or process signaling information and send it to the transmission processor 12012. According to an embodiment, an element that performs geometry coding and an element that performs attribute coding may share data / information with each other as indicated by the dotted lines. The transmission processor 12012 according to an embodiment may perform operations and / or transmission methods that are the same as or similar to those of the transmitter 10003. Details and References Figure 1 and Figure 2 Those described are the same, so their description is omitted.

[0218] Figure 13 A receiving device according to an embodiment is shown.

[0219] Figure 13 The receiving device shown is Figure 1 The receiving device 10004 (or Figure 10 and Figure 11 An example of a point cloud decoder. Figure 13 The receiving device shown can execute and refer to Figures 1 to 11One or more operations and methods that are the same or similar to those described for the point cloud decoder.

[0220] The receiving apparatus according to the embodiment includes a receiver 13000, a receiving processor 13001, an arithmetic decoder 13002, an octtree reconstruction processor based on an occupancy code 13003, a surface model processor (triangle reconstruction, upsampling, voxelization) 13004, an inverse quantization processor 13005, a metadata parser 13006, an arithmetic decoder 13007, an inverse quantization processor 13008, a prediction / lifting / RAHT inverse transform processor 13009, an inverse color transform processor 13010, and / or a renderer 13011. Each decoding element according to the embodiment may perform an inverse process of the operation of the corresponding encoding element according to the embodiment.

[0221] The receiver 13000 according to the embodiment receives point cloud data. The receiver 13000 may perform the same Figure 1 The operation and / or receiving method of the receiver 10005 are the same as or similar to the operation and / or receiving method of the receiver 10005. A detailed description thereof is omitted.

[0222] According to an embodiment, the reception processor 13001 may obtain a geometry bitstream and / or an attribute bitstream from the received data. The reception processor 13001 may be included in the receiver 13000.

[0223] The arithmetic decoder 13002, the octtree reconstruction processor 13003 based on the occupancy code, the surface model processor 13004 and the inverse quantization processor 13005 may perform geometric decoding. Figures 1 to 10 The geometric decoding described is the same or similar, so its detailed description is omitted.

[0224] The arithmetic decoder 13002 according to an embodiment may decode a geometry bitstream based on arithmetic coding. The arithmetic decoder 13002 performs operations and / or encoding that are the same as or similar to those of the arithmetic decoder 11000.

[0225] The octtree reconstruction processor 13003 based on the occupancy code according to the embodiment can reconstruct the octtree by obtaining the occupancy code from the decoded geometry bitstream (or information about the geometry obtained as a result of decoding). The octtree reconstruction processor 13003 based on the occupancy code performs the same or similar operations and / or methods as the operations of the octree synthesizer 11001 and / or the octree generation method. When triplet geometry coding is applied, the surface model processor 13004 according to the embodiment can perform triplet geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on the surface model method. The surface model processor 13004 performs the same or similar operations as the operations of the surface approximation synthesizer 11002 and / or the geometry reconstructor 11003.

[0226] The inverse quantization processor 13005 according to an embodiment may inversely quantize the decoded geometry.

[0227] The metadata parser 13006 according to an embodiment may parse metadata (e.g., setting values) contained in the received point cloud data. The metadata parser 13006 may pass the metadata to the geometry decoding and / or attribute decoding. Figure 12 The metadata described are the same, so a detailed description thereof is omitted.

[0228] The arithmetic decoder 13007, the inverse quantization processor 13008, the prediction / lifting / RAHT inverse transform processor 13009 and the color inverse transform processor 13010 perform attribute decoding. Figures 1 to 10 The attribute decoding described is the same or similar, so the detailed description thereof is omitted.

[0229] According to an embodiment, the arithmetic decoder 13007 can decode the attribute bitstream through arithmetic coding. The arithmetic decoder 13007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 13007 performs the same or similar operations and / or encoding as those of the arithmetic decoder 11005.

[0230] The inverse quantization processor 13008 according to an embodiment may inversely quantize the decoded attribute bitstream. The inverse quantization processor 13008 performs the same or similar operation and / or inverse quantization method as the inverse quantizer 11006.

[0231] The prediction / lifting / RAHT inverse transformer 13009 according to an embodiment may process the reconstructed geometry and inverse quantized attributes. The prediction / lifting / RAHT inverse transformer 13009 performs one or more operations and / or decoding that are the same or similar to the operations and / or decoding of the RAHT transformer 11007, the LOD generator 11008, and / or the inverse lifter 11009. The color inverse transformer 13010 according to an embodiment performs inverse transform encoding to inversely transform the color values ​​(or textures) included in the decoded attributes. The color inverse transformer 13010 performs operations and / or inverse transform encoding that are the same or similar to the operations and / or inverse transform encoding of the color inverse transformer 11010. The renderer 13011 according to an embodiment may render point cloud data.

[0232] Figure 14 An exemplary structure that can be operated in conjunction with a method / apparatus for transmitting / receiving point cloud data according to an embodiment is shown.

[0233] Figure 14 The structure of 1460 represents a configuration in which at least one of the server 1460, the robot 1410, the self-driving vehicle 1420, the XR device 1430, the smartphone 1440, the home appliance 1450, and / or the head-mounted display (HMD) 1470 is connected to the cloud network 1400. The robot 1410, the self-driving vehicle 1420, the XR device 1430, the smartphone 1440, or the home appliance 1450 is referred to as a device. In addition, the XR device 1430 may correspond to a point cloud data (PCC) device according to an embodiment or may be operatively connected to a PCC device.

[0234] The cloud network 1400 may represent a network that constitutes a part of a cloud computing infrastructure or exists in a cloud computing infrastructure. Here, the cloud network 1400 may be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

[0235] The server 1460 may be connected to at least one of the robot 1410 , the self-driving vehicle 1420 , the XR device 1430 , the smart phone 1440 , the home appliance 1450 , and / or the HMD 1470 via the cloud network 1400 , and may assist in at least a portion of processing of the connected devices 1410 to 1470 .

[0236] The HMD 1470 represents one of implementation types of an XR device and / or a PCC device according to an embodiment. The HMD type device according to an embodiment includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0237] Hereinafter, various embodiments of devices 1410 to 1450 to which the above-described technology is applied will be described. Figure 14The illustrated devices 1410 to 1450 may be operatively connected / coupled to the point cloud data transmitting and receiving devices according to the above-described embodiments.

[0238] <PCC+XR>

[0239] The XR / PCC device 1430 may employ PCC technology and / or XR (AR+VR) technology, and may be implemented as an HMD, a head-up display (HUD) provided in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a fixed robot, or a mobile robot.

[0240] The XR / PCC device 1430 can analyze 3D point cloud data or image data acquired through various sensors or from external devices and generate positional data and attribute data about the 3D points. Thus, the XR / PCC device 1430 can obtain information about the surrounding space or real objects and render and output XR objects. For example, the XR / PCC device 1430 can match an XR object that includes auxiliary information about the recognized object with the recognized object and output the matched XR object.

[0241] <PCC+Autopilot+XR>

[0242] The self-driving vehicle 1420 may be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0243] Self-driving vehicle 1420, which utilizes XR / PCC technology, may represent a self-driving vehicle equipped with a means for providing XR images, or a self-driving vehicle that serves as a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, self-driving vehicle 1420 may be distinguished from and operatively connected to XR device 1430.

[0244] The self-driving vehicle 1420, which has a means for providing XR / PCC images, can obtain sensor information from sensors including cameras and output generated XR / PCC images based on the obtained sensor information. For example, the self-driving vehicle 1420 may have a HUD and output XR / PCC images thereto, thereby providing passengers with XR / PCC objects corresponding to real objects or objects displayed on the screen.

[0245] When an XR / PCC object is output to the HUD, at least a portion of the XR / PCC object may be output to overlap with the real object that the passenger's eyes are pointing at. On the other hand, when an XR / PCC object is output on a display provided within a self-driving vehicle, at least a portion of the XR / PCC object may be output to overlap with the object on the screen. For example, the self-driving vehicle 1220 may output XR / PCC objects corresponding to objects such as a road, another vehicle, a traffic light, a traffic sign, a two-wheeled vehicle, a pedestrian, and a building.

[0246] Virtual reality (VR) technology, augmented reality (AR) technology, mixed reality (MR) technology, and / or point cloud compression (PCC) technology according to an embodiment are applicable to various devices.

[0247] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. On the other hand, AR technology refers to a technology that displays a virtually created CG image on an image of a real object. MR technology is similar to the above-mentioned AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology is different from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with properties equivalent to real objects. More specifically, an example of the application of MR technology is holographic services.

[0248] Recently, VR, AR, and MR technologies are often referred to as extended reality (XR) technologies, rather than being clearly distinguished from each other. Therefore, embodiments of the present disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are applicable to such technologies.

[0249] The PCC method / apparatus according to the embodiment may be applied to a vehicle providing a self-driving service.

[0250] Vehicles providing self-driving services are connected to the PCC device for wired / wireless communication.

[0251] When the point cloud data (PCC) transmitting / receiving device according to the embodiment is connected to the vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services (which can be provided together with self-driving services) and send it to the vehicle. When the PCC transmitting / receiving device is installed on the vehicle, the PCC transmitting / receiving device can receive / process content data related to AR / VR / PCC services and provide it to the user based on a user input signal input through a user interface device. The vehicle or the user interface device according to the embodiment can receive the user input signal. The user input signal according to the embodiment may include a signal indicating a self-driving service.

[0252] Figure 15 Shown are structures of geometry data and attribute data for spatial scalability according to an embodiment.

[0253] Figure 15 The G-PCC decoder 15000 and / or the G-PCC decoder 15010 may correspond to Figure 1 The receiving device 10004 in Figure 1 Point cloud video decoder 10006, Figure 2 Decoding 20003 and Figure 10 and Figure 11 The decoder in Figure 13 The decoder and Figure 14 The XR device 1430 in FIG. Figure 15 The G-PCC decoder 15000 and / or the G-PCC decoder 15010 in the embodiment may be executed Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 2 The code in 20001, Figure 4 The encoder in Figure 12 The encoder and Figure 14 The reverse process of the operation on the transmitting side of the XR device 1430 in FIG.

[0254] According to an embodiment, the PCC data encoder may correspond to an encoding device, an encoder, a point cloud transmitting device, a point cloud data encoder, etc.

[0255] According to an embodiment, geometric information may be referred to as geometric data, etc., and attribute information may be referred to as attribute data, etc.

[0256] The method / apparatus according to an embodiment refers to a method / apparatus for transmitting / receiving point cloud data according to an embodiment.

[0257] Embodiments relate to an attribute compression method that supports spatial scalability of a geometry-based point cloud compression (G-PCC) encoder / decoder for 3D point cloud data compression.

[0258] The embodiment proposes a method for changing the point sampling technique of each LOD based on an octree during attribute encoding / decoding of G-PCC to provide spatial scalability.

[0259] Specifically, we propose 1) a method for reorganizing points through octree nodes using Morton codes, 2) a method for handling isolated points, 3) a point sampling technique for each node, and 4) a signaling method that supports the above three methods.

[0260] When point cloud data is dense, it may be necessary to use a narrower bandwidth (e.g., thumbnails) or send and process lower resolution point clouds to reduce the computational complexity on the decoder side. This feature is called spatial scalability.

[0261] In geometry encoding / decoding, a spatial scalability function may be provided by a process of encoding or decoding occupied bits only to a depth level selected by adjusting the depth level of an octree.

[0262] Attribute encoding / decoding according to an embodiment may perform an octree structure-based operation according to an embodiment to provide spatial scalability. A method for generating LODs from a selected depth level of an octree, a method for calculating neighbor nodes, and a method for calculating weights according to an embodiment may be required to provide spatial scalability in attribute encoding / decoding.

[0263] According to an embodiment, attribute encoding / decoding can change the sampling points of each LOD based on an octree and configure the most appropriate points of each LOD as nodes to provide spatial scalability. As a result, attribute compression efficiency can be enhanced.

[0264] The embodiment proposes a method for changing the point sampling technique of each LOD based on an octree to provide spatial scalability in the attribute encoding / decoding process of G-PCC.

[0265] The LOD sampling method according to an embodiment may refer to a method of sorting points using a Morton code and classifying the points selected according to the embodiment into lower LODs that can be candidates for a neighbor set. The LOD sampling method according to an embodiment may increase the accuracy and compression efficiency of generating a neighbor point set while reducing the complexity of LOD generation.

[0266] like Figure 15As shown, PCC data (or point cloud data) may consist of a geometry bitstream and an attribute bitstream. The point cloud data may be encoded and transmitted by a point cloud data transmitting device or a point cloud data encoder according to an embodiment. The point cloud data receiving device or a point cloud data decoder according to an embodiment may decode the point cloud data.

[0267] In this case, for geometry data (or geometry bitstream) and / or attribute data (or attribute bitstream), the cloud data receiving device may partially decode the geometry bitstream of the octree structure and / or the attribute bitstream based on lifting coding in the order of detail level (LOD).

[0268] The G-PCC decoder 15000 and / or the G-PCC decoder 15010 represent a decoder of the point cloud data receiving apparatus according to an embodiment. Specifically, the G-PCC decoder 15000 and / or the G-PCC decoder 15010 represent a geometry decoder and / or an attribute decoder of a PCC decoder.

[0269] The G-PCC decoder 15000 may not decode all geometry bitstreams and / or attribute bitstreams in the received bitstream, but may partially decode the geometry bitstreams and / or attribute bitstreams to provide low-resolution point cloud data to the user.

[0270] The G-PCC decoder 15010 may decode all geometry bitstreams and / or attribute bitstreams in the received bitstream and provide the decoded bitstream to a user.

[0271] The PCC encoder on the transmitting side corresponding to the G-PCC decoder 15000 and the G-PCC decoder 15010 may encode and transmit the entire point cloud data, or may encode and transmit part of the point cloud data.

[0272] [[ID=1 The attribute encoding / decoding process according to an embodiment is shown.

[0273] ​ The process may be applied to encoding and / or decoding according to the embodiment. For example, the process may be ​ The decoders 15000 and 15010 and / or the corresponding encoders ( ​ The sending device 10000, ​ Point cloud video encoder 10002, ​ The code in 20001, ​ The encoder in ​ The encoder and ​ The XR device 1430 in the embodiment is executed.

[0274] Changes and combinations between the embodiments of the present disclosure are possible.In addition, the terms used in the present disclosure may be understood based on the intended meanings of the terms within the scope widely adopted in the relevant art.

[0275] Attribute encoders / decoders according to embodiments may employ an octree structure, as in geometry encoding / decoding, to provide spatial scalability when encoding / decoding attributes. Embodiments include methods for reorganizing points belonging to nodes starting from a selected depth level of the octree and methods for sampling points at each level of detail (LOD).

[0276] The method of recombining points and the method of sampling points of each LOD are performed in PCC attribute encoding / decoding of a PCC decoder / encoder and can be applied when a predictive transform technique or a lifting transform technique is used in the attribute encoding / decoding process.

[0277] S16000 sorts point cloud data based on the Morton order

[0278] Points in the point cloud may be converted to Morton codes based on x, y, and z position values ​​and may be sorted based on the Morton code values.

[0279] S16010 Set the partial encoding / decoding area of ​​attribute data

[0280] In attribute encoding / decoding, as in geometry encoding / decoding (see ​ ), the depth level of the octree can be set, and the points to which attribute encoding (decoding) is to be applied can be constructed and encoded / decoded starting from the set octree depth level.

[0281] S16020 uses Morton codes to reorganize points belonging to octree nodes

[0282] The method for reorganizing points sorted by Morton code for each octree node according to the embodiment may adopt the pseudo code (formula) given below.

[0283] nodeSizeLog2: indicates the octree depth level ( ​ 17000 in ).

[0284] PX_mortoncode: represents the Morton code of node PX.

[0285] end_range_mc: indicates the range of the node's Morton code.

[0286] maxNumOfPoints: Indicates the maximum number of points for a particular octree depth level. This is the maximum number of points a node can ideally have.

[0287] Int maxNumOfPoints=pow(8,nodeSizeLog2+1);

[0288] For(int I=1;I <indexCount;++i){

[0289] if(Pi_mortoncode <end_range_mc){

[0290] …

[0291] }else{

[0292] end_range_mc=Pi_mortoncode+

[0293] (maxNumOfPoints-(Pi_mortoncode%maxNumOfPoints));

[0294] }

[0295] }

[0296] According to the above syntax, the range (end_range_mc) with the same Morton code value within a node in the octree structure can be known, and the node can be selected and uploaded to a higher node.

[0297] The same meaning as the above syntax can be expressed as follows.

[0298] When using the boosting method, the following node selection methods are available.

[0299] When lifting_scalability_enabled_flag is equal to 1 (lifting is used), the following applies:

[0300] endIndex=0

[0301] assignedPointCount=0

[0302] for(lod=minGeomNodeSizeLog2; unprocessedPointCount>0; lod++){

[0303] startIndex=assignedPointCount

[0304] nonAssignedPointCount=0

[0305] samplingFromLastFlag=lod&1

[0306] for(i=0;i<unprocessedPointCount;i++){

[0307] currVoxelIndex=McodeUnsorted[unprocessedPointIndexes[i]]>>(3*×(lod+1))

[0308] if(samplingFromLastFlag==0){

[0309] if(i==0)

[0310] nonAssignedPointIndexes[nonAssignedPointCount++]=unprocessedPointIndexes[i]

[0311] else{

[0312] prevVoxelIndex=McodeUnsorted[unprocessedPointIndexes[i-1]]>>(3*×(lod+1))

[0313] if(currVoxelIndex>prevVoxelIndex)

[0314] nonAssignedPointIndexes[nonAssignedPointCount++]=unprocessedPointIndexes[i]

[0315] else

[0316] assignedPointIndexes[assignedPointCount++]=unprocessedPointIndexes[i]

[0317] }

[0318] }else{

[0319] if(i==(unprocessedPointCount–1))

[0320] nonAssignedPointIndexes[nonAssignedPointCount++]=unprocessedPointIndexes[i]

[0321] else{

[0322] nextVoxelIndex=McodeUnsorted[unprocessedPointIndexes[i+1]]>>(3*×(lod+1))

[0323] if(currVoxelIndex<nextVoxelIndex)

[0324] nonAssignedPointIndexes[nonAssignedPointCount++]=unprocessedPointIndexes[i]

[0325] else

[0326] assignedPointIndexes[assignedPointCount++]=unprocessedPointIndexes[i]

[0327] }

[0328] }

[0329] }

[0330] endIndex=assignedPointCount

[0331] if(startIndex!=endIndex){

[0332] numOfPointInSkipped=PointNumInSlice–PointCount

[0333] if((endIndex–startIndex)>(startIndex+numOfPointInSkipped)){

[0334] for((loop=0;loop<lod-minGeomNodeSizeLog2;loop++)){

[0335] computeNearestNeighbours(

[0336] PointCount-unprocessedPointCountPerLevelOfDetail[loop],

[0337] PointCount-unprocessedPointCountPerLevelOfDetail[loop+1],

[0338] loop+minGeomNodeSizeLog2,assignedPointIndexes,

[0339] McodeUnsorted,nonAssignedPointCount,

[0340] nonAssignedPointIndexes))

[0341] }

[0342] }

[0343] }

[0344] computeNearestNeighbours(

[0345] startIndex,endIndex,

[0346] lod,assignedPointIndexes,

[0347] McodeUnsorted,nonAssignedPointCount,

[0348] nonAssignedPointIndexes)

[0349] unprocessedPointCountPerLevelOfDetail[lod+1]=nonAssignedPointCount

[0350] unprocessedPointCount=nonAssignedPointCount

[0351] unprocessedPointIndexes=nonAssignedPointIndexes

[0352] }

[0353] Specifically, when the next voxel index is set to nextVoxelIndex=McodeUnsorted[unprocessedPointIndexes[i+1]]>>(3*×(lod+1)), the Morton code may be shifted to determine whether the Morton code values ​​are within the same range.

[0354] The encoder according to the embodiment may estimate end_range_mc from the Morton code of the point. When the Morton code value of the next point is within the range end_range_mc, these points belong to the same node. When the Morton code has a larger value outside the range, the encoder may estimate end_range_mc based on the Morton code of the point when the node changes to a new node and apply it to the subsequent points. In this way, points belonging to the octree node can be reorganized between points sorted by Morton code (see ​ ).

[0355] S16030 Merge isolated nodes

[0356] Nodes can be merged when the number of points belonging to a node is less than a certain threshold (see ​ ).

[0357] For example, when the minimum number of points for a node is "a" and the number of points is less than the minimum number of points, the node may be merged with the next node. Furthermore, the value of a may be signaled. When the maximum merge range is r, the node may be merged with up to the next r nodes, and the value of r may be signaled. When the number of points is less than the minimum number of points, the node may be merged with the next node. When the number of merged points is less than the minimum number of points, the node may be merged with the next node. This repetition may be performed r times.

[0358] S16040 configures LOD of octree nodes

[0359] Depending on the implementation, LOD configuration may proceed from the root node to the leaf nodes, or may proceed from the leaf nodes to the root node.

[0360] For example, when configuring LOD from root to leaf, LOD configuration can be performed from root node to leaf node. This maintains consistency. When configuring LOD from leaf to root, LOD configuration can be performed from leaf node to root node.

[0361] S16050 performs LOD sampling of octree nodes

[0362] Sampling can be performed by selecting points from among the points belonging to a node (including merged nodes according to embodiments). Attribute compression may be affected by the selected points. The best reference points are selected and reassembled into a reference region (retained list), and a set of neighbor nodes is constructed from the reference region. Therefore, compression efficiency may be affected.

[0363] The LOD configuration unit 20120 according to an embodiment may use the following three sampling methods.

[0364] <Select point N>

[0365] When selecting a point among points belonging to a node (including a merge node), the Nth point may be selected. When compression is performed by an encoder, N may be input by a user, or an optimal value may be found in an RDO (Rate Distortion Optimization) manner.

[0366] The value of N may be sent to a decoder (receiving device) as a parameter (metadata or signaling data). When the value of N is not specified, a value of 0 (ie, the first value) may be selected.

[0367] The compression efficiency may depend on which point is chosen.

[0368] <Select a point close to the center of the octree parent node area>

[0369] When selecting a point from among the points belonging to a node (including a merged node), the point closest to the center of the bounding box of the parent node may be selected. When the selected point is closer to the center position of the parent node, the error can be further reduced when selecting neighbor points and calculating the weights of the neighbor points.

[0370] When selecting neighbor points, the distance between the points is calculated, and the point with a closer distance can be selected as the neighbor point. In order to provide spatial scalability, the position value of the point can be replaced by the value of the center position of the node to which the point belongs (the node can change according to the level of LOD). In this state, the distance can be calculated. Therefore, in order to find the neighbor point of point Pi, the position value of Pi can be changed to the center position of the parent node according to LOD1, and the position of the point Ri in the neighbor point candidate list (retention list) R also changes according to LOD1. Then, the distance of the point can be calculated. In other words, the distance between the center positions of nodes at the same depth level can be calculated, so that the neighbor node is set as the neighbor point. From the distance calculated in this way, the weight of the neighbor point can be obtained, 1 / distance=Pi. Since the distance is calculated by changing the position of the point to the center position of the parent node, the calculation contains errors. As the depth level of the node decreases, that is, the bounding box of the node is larger, the error can increase.

[0371] To reduce errors, the transmitting / receiving apparatus and method according to the embodiment may select a point close to the center of the parent node region when selecting a reference point during encoding or decoding. The error can be reduced based on the position value of the reference point, and the reduced error can have a good impact on compression efficiency.

[0372] <Select the point with the middle index>

[0373] When selecting a point from among the points belonging to a node (including a merge node), a point with an intermediate index among the points may be selected. Since the node with the intermediate index is likely to be close to the center position of the bounding box of the parent node (in many cases), and selecting a point by the intermediate index does not require much additional calculation, the probability of finding a point close to the center position with a small amount of calculation may be high.

[0374] ​ FIG. 4 shows an octree structure of point cloud data according to an embodiment.

[0375] ​ Show ​ Operation S16010 in .

[0376] When the point cloud transmitting / receiving device or encoder / decoder according to the embodiment encodes or decodes the attribute, it can be as follows ​ As shown in , an octree structure is used. The octree structure is interpreted in order from the root node to the leaf node, that is, from the upper node (parent node) to the lower node (child node), and the nodes can be arranged from left to right in Morton order. In addition, LODs can be configured (generated) in order from the leaf node to the root node.

[0377] For example, an octree depth level such as a starting depth of 17000 is set, and the G-PCC decoder 15000 may be as follows ​ As shown, the point cloud data at the set octree depth level 17000 is decoded.

[0378] A parent node may have 8 or fewer nodes as child nodes of the parent node. In the process of sampling a specific node among the child nodes and shifting it to the parent node, the first point 17050 among the points 17040 belonging to the parent node in the order of the Morton code may be selected as a reference node or a retained node (see ​ ).

[0379] Specifically, to support spatial scalability, a transmitting device or encoder according to an embodiment may encode and transmit all points, or encode and transmit points at a specific depth. Similarly, a receiving device or decoder according to an embodiment may decode all points or points at a specific depth.

[0380] like ​ As shown, when there is an octree representing points of received point cloud data, a process of spatial scalability decoding based on a starting depth of 17000 will be described.

[0381] When the octree level is 0 (17010), level 1 (17020), and level 2 (17030), a start depth 1730 may be set, points may be decoded starting from the start depth 1730, and points below the start depth may be ignored.

[0382] Starting from the point at the starting depth 1730, the points may be reorganized and merged to configure the LOD, and sampled. Then, the transmitting device may encode the attribute, or the receiving device may decode the attribute.

[0383] ​A method of reorganizing points belonging to a Morton code-based node according to an embodiment is shown.

[0384] ​ Show ​ Operation S16020 in .

[0385] The encoder according to the embodiment may be based on ​ Operation S16020 in reorganizes the points of the child node 18010 belonging to the parent node 18000.

[0386] Based on ​ After setting the attribute encoding / decoding area at the starting depth 17000 in the octree, the points of the octree at the level at or after the starting depth 17000 can be as follows ​ The order of the levels in ​ The formula in S16020 is reorganized.

[0387] For example, ​ The points of the level (depth) of the octree set in can be P0, P1, P2, ..., P13, .... In order to ensure efficient encoding / decoding, the method / apparatus (including encoder / decoder) according to the embodiment can regroup the points into groups belonging to the same category, such as ​ shown.

[0388] according to ​ The value of end range mc can be calculated using the octree level value of P0 and the Morton code value (e.g., 1) using the formula "end_range_mc=Pi_mortoncode+(maxNumOfPoints-(Pi_mortoncode%maxNumOfPoints)). When the first range value is 8, the Morton code value MC of each of P0 to P5 is less than or equal to 8, so the points can be reorganized into points belonging to the same node (18010).

[0389] The second range value can use the octree level value of P6, the Morton code value (e.g., 9) and ​ When the point with a Morton code value smaller than the second range value is only P6, P6 is reorganized into a single point (18020).

[0390] The third range value can use the octree level value or Morton code value of P7 (for example, 16) and ​ P7 to P10 having Morton code values ​​included in the third range of values ​​may be reorganized as points belonging to the same node (18030).

[0391] The fourth range value can use the octree level value or Morton code value of P11 (for example, 26) and ​Points having Morton code values ​​included in the fourth range of values ​​may be reorganized into points belonging to the same node (18040).

[0392] ​ Illustrates merging of isolated nodes according to an embodiment.

[0393] ​ Show ​ Example of S16030 in .

[0394] Isolated nodes are those whose number upon reorganization is less than or equal to a certain number.

[0395] As an example, the following will be described ​ The minimum number of node points shown in S16030 is 4.

[0396] For example, there may be points 19000 sorted in the order of the Morton code. ​ As a result of the described operation of reordering the points, the number of points belonging to the same node, such as in 19010 , may be greater than 4. In addition, there may be points with 4 or fewer points belonging to the same node, such as in 19020 and 19030 .

[0397] The transmitting / receiving method / apparatus according to the embodiment may merge points whose number is less than or equal to a specific number, rather than leaving the points as separate nodes. According to the embodiment, the merging process may be optional.

[0398] As isolated nodes are merged, the number of recombined points can be greater than or equal to a specific number. Recombined points belonging to the same node have a sampling range. In addition, the merged node has a merged sampling range.

[0399] ​ An example in which a node having one point and a node having four points are merged and an example 19040 in which a node having three points and a node having two points are merged are shown.

[0400] ​ A PCC data encoder according to an embodiment is shown.

[0401] ​ is an encoder according to an embodiment ( ​ The sending device 10000, ​ Point cloud video encoder 10002, ​ The code in 20001, ​ The encoder in ​ The encoder and ​ FIG. 14 shows a configuration diagram of an XR device 1430 in FIG. 1430 , which performs ​ Operations in .

[0402] The PCC data encoder according to the embodiment may be referred to as an encoder, and the PCC data decoder according to the embodiment may be referred to as a decoder.

[0403] PCC data (point cloud data) may be input to and encoded by an encoder, and may output a geometric information bitstream and an attribute information bitstream (a bitstream for point cloud data).

[0404] ​ The various elements in the illustrative embodiment may correspond to hardware, software, a processor and / or a combination thereof.

[0405] The data input unit 20000 may receive geometric data, attribute data and / or parameters related thereto.

[0406] The coordinate converter 20010 can convert the coordinates related to the position (coordinate) information of the geometric data. ​ The coordinate transformer 40000 in .

[0407] The geometric information transform quantization processor 20020 may be referred to as a geometric information transform quantizer. The geometric information transform quantization processor 20020 may correspond to ​ The quantizer 40001 in the geometric information transform quantization processor 20020 can receive a full-scale value (quantization value) and can adjust the scale of each axis according to the distribution of the content.

[0408] The spatial segmenter 20030 may segment the point cloud data into spatially segmented units, depending on the implementation. The spatial segmenter 20030 may perform spatially adaptive segmentation of the data space using tiles, blocks, density-based methods, axis-based methods, attribute-based methods, geometry-based methods, and the like, depending on the data characteristics of the point cloud data. The spatial segmenter 20030 may perform data segmentation. A frame may be segmented into tiles and slices. The source point cloud data may be segmented into multiple slices and encoded within the bitstream. A slice is a collection of points that can be independently encoded or decoded. A slice may include one geometry data unit and zero or one or more attribute data units. Attribute data units may depend on corresponding geometry data within the same slice. Within a slice, a geometry data unit may appear before a related attribute unit. Data units within a slice are contiguous. A group of slices may be identified by a common tile identifier. Tile information describing the bounding box of each tile may be present in the bitstream. Tiles may overlap with other tiles within the bounding box. Each slice may include an index identifying the tile to which it belongs.

[0409] The geometry information encoder 20040 can encode the geometry data of the point cloud data. The detailed process of geometry encoding is described below.

[0410] The voxelization processor 20050 may voxelize the geometric data. The voxelization processor 20050 may correspond to ​ Voxelizer 40001 in .

[0411] The octree generator 20060 may be referred to as an octree generation unit. The octree generator 20060 may correspond to ​ The octree analyzer 40002 in the octree analyzer. The octree generator 20060 can generate geometric data according to the octree structure.

[0412] When using a geometric octree, geometric encoding can be performed as follows: 1) A bounding box defined by two points can be defined. The octree structure is constructed by recursively subdividing the bounding box. Each cube is divided into 8 sub-cubes. The 8-bit code can be called an occupancy code. One bit per sub-cube can be represented as 1 when the point is filled, or as 0 when the point is empty. Sub-cubes with sides greater than 1 are further divided. Sub-cubes of size 1 can be voxels. Multiple points can overlap and map to the same sub-cube (voxel). The points of each sub-cube can be arithmetically coded.

[0413] The decoding process can be started by reading the bounding box dimensions from the bitstream. The same octree structure can be generated by partitioning the bounding box according to the occupancy code. When a sub-cube of size 1 is reached, the points of the sub-cube can be arithmetically decoded.

[0414] The octree generator 20060 constructs an octree from the received geometry information bitstream (geometry data). For spatial scalability, the octree generator 20060 may construct geometry information only at a specific level of the octree. In this case, the position value may be changed to the value of the center of the bounding box.

[0415] The octree generator 20060 may construct and decode points to which attribute decoding is to be applied for partial geometry decoding starting from the restored octree depth level.

[0416] The geometry information predictor 20070 can perform prediction on the geometry data. The geometry information predictor 20070 can generate prediction values ​​for the geometry data between parent nodes and child nodes in the octree structure. Residual bit values ​​can be generated from the prediction values ​​and entropy encoded for the residual values ​​to construct a geometry bitstream. Efficient bitstream transmission can be achieved by encoding only the residual values ​​for the prediction values. The geometry information predictor 20070 can recover (reconstruct) the geometry data from the geometry octree for attribute encoding. This is because attribute encoding is performed based on the reconstructed geometry data.

[0417] The geometry information entropy encoder 20080 may perform entropy encoding on the geometry data to generate a geometry bitstream.

[0418] The attribute information encoder 20090 may encode the attribute data as follows.

[0419] The color conversion processor 20100 may convert the color of attribute data according to data characteristics.

[0420] The color rescaler 20110 may be referred to as a recoloring unit, color recolorer, or recolorer. Depending on the embodiment, when overlapping points are merged in the reconstructed geometry octree structure, the attribute information encoder 20090 may perform recoloring, or perform RAHT attribute encoding, or configure LODs based on prediction / lifting transform attribute encoding. Depending on the embodiment, recoloring and / or LOD configuration may be performed even during RAHT encoding when overlapping points are not merged. Depending on the embodiment, when attribute information encoding is performed based on reconstructed geometry data, recoloring may be performed with lossy geometry, or LOD configuration may be performed with prediction / lifting transform attribute encoding without lossy geometry. The color rescaler operation may be applied when multiple points exist in a voxel and overlapping points are merged to form the voxel using lossy compression. Furthermore, when the geometry quantization coefficient is less than 1, multiple points may be included in a voxel. In this case, recoloring, LOD configuration, and neighbor point set configuration may be performed. The color rescaler 20110 may search for neighboring points near the center of a voxel in the original point cloud to set a representative color for the voxel, and calculate the representative color based on the neighboring points. During the search for neighboring points, geometric information may be inverse quantized to restore the positional difference between the original value and the quantized value. The color rescaler 20110 may perform inverse quantization for each axis during inverse quantization.

[0421] The LOD configuration unit 20120 can configure LODs from point cloud data. The LOD configuration unit 20120 can generate LODs from re-colored attribute data based on restored geometry information (geometry data). LODs can be generated when a prediction / lifting transform is applied during attribute encoding instead of RAHT. The LOD configuration unit 20120 can use Morton codes to reorganize points belonging to nodes, merge isolated nodes, perform LOD sampling in octree nodes according to the embodiment, and generate and transmit relevant information as signaling information.

[0422] The LOD configuration unit 20120 may set the depth level of the octree to support spatial scalability in attribute encoding / decoding, and configure and encode / decode points to which attribute encoding is to be applied starting from the set depth level of the octree (see ​ ). The LOD configuration unit 20120 can be called an LOD generator, an LOD configuration device, etc.

[0423] The octree depth level may be sent as signaling information by the encoder according to an embodiment to the decoder, and may be changed according to its usage or computational performance of the decoder.

[0424] The LOD configuration unit 20120 may perform point reorganization for each octree node by calculating the Morton code range of each octree node at points sorted by the Morton code (see ​ ).

[0425] The LOD configuration unit 20120 may set a minimum number of node points (a value) to distinguish a case where the number of node points is less than the minimum number of node points. The set value may be signaled to the decoder as signaling information.

[0426] The process of merging a node with the next consecutive node when the number of points of the node is less than the minimum number of points of the node can be called isolated node merging processing (see ​ ), whether to perform isolated node merging processing can be notified to the decoder as signaling information.

[0427] When the number of points of a node is less than the minimum number of points of a node, the node can be merged with the next consecutive node. It is allowed to perform the merge up to r times (see ​ ). The maximum number of merging times r can be signaled to the decoder as signaling information.

[0428] There are methods for selecting (sampling) a point from the points belonging to a node (including merging nodes) (see ​ ). This method may be signaled to the decoder as signaling information. As this method, the Nth point may be selected, a point near the center of the octree parent node area may be selected, or a point with an intermediate index may be selected.

[0429] The neighbor point set configuration unit 20130 configures a neighbor point set for points belonging to the LOD in the LOD and higher LODs based on the LOD.

[0430] After the LOD1 set is generated, X (>0) nearest neighbors can be found in the group of the same or lower LOD (the group with longer distances between nodes) based on the LOD1 set and registered in the predictor as a neighbor point set.

[0431] Each point can have a single predictor. Attributes are predicted from neighboring points registered with the predictor.

[0432] The predictor can be used to predict attributes. The prediction result can be the average of the values ​​obtained by multiplying the attributes of registered neighboring points by a weight, or a specific point can be used as the prediction result. The compression result value can be precalculated to determine which method to use, and the method that generates the minimum flow can be selected.

[0433] The residual between the attribute value of a point and the attribute value predicted by the predictor of the point may be encoded and signaled to the receiver along with the method selected by the predictor.

[0434] The decoder can decode the transmitted prediction method and predict the property value based on the method. The property value can be restored by decoding the transmitted residual value and adding the predicted value to it.

[0435] The attribute information predictor 20140 predicts the attribute value of a point from a set of neighboring points.

[0436] The residual attribute information quantization processor 20150 may generate a predicted value of attribute data, generate a residual value between the predicted value and the attribute value, and quantize the residual value.

[0437] For example, when encoding a point Px, when configuring a set of neighboring points of Px, the attribute value of Px can be predicted based on the attribute values ​​of the neighboring points in the set, the residual between the predicted attribute value and the attribute value of Px can be estimated, and the value of the residual can be encoded and sent to the decoder. The decoder configures a set of neighboring points of Px through LOD generation and the received additional information, predicts the attribute value of Px based on the attribute values ​​of the neighboring points in the neighboring set, and restores the attribute value of Px by adding the received residual value to the predicted value.

[0438] When the prediction mode and residual information separation flag is true, the residual attribute information quantizer 20150 may calculate the residual between the predicted value according to the prediction mode and the value of the point, and encode its quantized value together with the prediction mode.

[0439] When the prediction mode and residual information separation flag is false, the residual attribute information quantizer 20150 may calculate the residual between the prediction value and the point value according to the prediction mode, and encode only the quantized value thereof.

[0440] The attribute information entropy encoder 20160 may perform entropy encoding on the attribute data and generate an attribute bitstream. For example, the residual value may be encoded and the attribute bitstream may be generated.

[0441] ​ A PCC data decoder according to an embodiment is shown.

[0442] ​ The decoder in can correspond to ​ G-PCC decoder 15000, ​ G-PCC decoder 15010, ​ The receiving device 10004 in ​ Point cloud video decoder 10006, ​ Decoding 20003,​ and ​ The decoder in ​ The decoder and ​ The XR device 1430 in FIG.

[0443] The geometry information bitstream and the attribute information bitstream encoded as inputs of the decoder may be input and decoded, and reconstructed PCC data may be output.

[0444] ​ The various elements in the illustrative embodiment may correspond to hardware, software, a processor and / or a combination thereof.

[0445] The geometry information decoder 210000 may receive the geometry information bitstream and decode the geometry information. The geometry information decoder 210000 may be a decoder on the receiving side corresponding to the geometry information encoder 20040 on the transmitting side. The geometry information decoder 210000 may perform the inverse process of the operation of the geometry information encoder 20040.

[0446] The geometry information entropy decoder 210010 may receive the geometry information bitstream and decode the geometry information based on an entropy scheme. The geometry information entropy decoder 210010 may perform the inverse process of the operation of the geometry information entropy encoder 20080.

[0447] The octree reconstructor 210020 may reconstruct the octree from the geometric information and may perform the reverse process of the operation of the octree generator 20060.

[0448] The geometry information predictor 210030 may generate a prediction value for the geometry information. For example, it may generate an inter-node prediction value in an octree structure and efficiently decode the geometry information using a residual value of the prediction value. The geometry information predictor 210030 may perform the inverse process of the operation of the geometry information predictor 20070.

[0449] The geometric information transform inverse quantization processor 210040 may inversely quantize the geometric data. For example, the geometric information transform inverse quantization processor 210040 may obtain a scale value (quantization value) of each axis from signaling information (parameters) included in the bitstream, and may apply inverse quantization to the geometric information reconstructed based on the scale value of each axis.

[0450] The coordinate inverse transformer 210050 may inversely transform the coordinate information about the geometric information. The coordinate inverse transformer 210050 may perform an inverse process of the operation of the coordinate transformer 20010.

[0451] The attribute information decoder 210060 may receive the attribute information bitstream and decode the attribute information. The attribute information decoder 210060 may perform the reverse process of the operation of the attribute information encoder 20090.

[0452] The attribute residual information entropy decoder 210070 can decode the attribute residual information (eg, the residual value of the attribute information) contained in the attribute information bitstream based on an entropy scheme. The attribute residual information entropy decoder 210070 can perform the inverse process of the operation of the attribute information entropy encoder 20160.

[0453] The LOD configuration unit 210080 may configure LOD from point cloud data. The LOD configuration unit 210080 may generate LOD from attribute data that has been re-adjusted in color based on restored geometric information (geometry data). When a prediction / lifting transform is applied during attribute encoding instead of RAHT, LOD may be generated. The LOD configuration unit 210080 may correspond to ​ LOD configuration unit 20120.

[0454] The LOD configuration unit 210080 may use Morton code to reorganize points belonging to a node, merge isolated nodes, and perform LOD sampling according to an embodiment in the octree node.

[0455] LOD configuration unit 210080 can construct and decode the point to which attribute decoding is to be applied starting from the octree depth level recovered for partial attribute decoding (see ​ ).

[0456] The LOD configuration unit 210080 may perform point reorganization for each octree node by calculating the Morton code range of each octree node from the points sorted by the Morton code (see ​ ).

[0457] When the number of points of a node is less than the minimum number of restored node points (ie, a), the LOD configuration unit 210080 may merge the continuous nodes with the node (see ​ ).

[0458] The process of merging a node with the next consecutive node when the number of points of the node is less than the minimum number of node points may be referred to as an isolated node merging process, and the merging may be performed according to the decoded indication of whether to perform the isolated node merging process.

[0459] When the number of points of a node is less than the minimum number of points of a node, the node may be merged with the next consecutive node. Merging may be performed r times (ie, the maximum number of merges to be restored).

[0460] A point may be selected from among the points belonging to a node (including a merged node) based on a scheme for selecting a point to be restored. The scheme may include a scheme for selecting the Nth point, a scheme for selecting a point close to the center of the octree parent node area, and a scheme for selecting a point with an intermediate index (see ​ ).

[0461] The neighbor point set configuration unit 210090 may search for neighbor points of a point included in the LOD based on the LOD generated from the attribute information bitstream and register them as a neighbor point set. ​ The neighbor point set configuration unit 20130 in.

[0462] The attribute information predictor 210100 may generate a predicted value for the attribute information. For example, the attribute information predictor may generate a predicted value for a point from a set of LODs and neighboring points, and generate a residual value for the predicted value. The attribute information predictor 210100 may perform the inverse of the operation of the attribute information predictor 20140.

[0463] The residual attribute information inverse quantization processor 210110 may inversely quantize the residual attribute information (eg, the residual value of the attribute information). The residual attribute information inverse quantization processor 210110 may perform an inverse process of the operation of the residual attribute information quantization processor 20150.

[0464] The color inverse conversion processor 210120 may inversely convert color information as attribute information. The color inverse conversion processor 210120 may perform an inverse process of the operation of the color conversion processor 20100.

[0465] ​ FIG. 1 shows a bitstream structure of point cloud data according to an embodiment.

[0466] The method / apparatus according to the embodiment may generate and obtain ​ For example, a point cloud bitstream containing geometric information, attribute information and / or parameters including metadata can be obtained by ​ The sending device 10000, the point cloud video encoder 10002, the receiving device 10004, the point cloud video decoder 10006, ​ Encoding 20001 and decoding 20003, ​ The encoding process in ​ The decoding process in ​ The sending device in ​ The receiving device in ​ XR device 1430, ​ The encoder in ​ The decoder in generates (encodes) and receives (decodes).

[0467] Information of the implementation may be signaled.

[0468] The following abbreviations are used in this disclosure: SPS (Sequence Parameter Set); GPS (Geometry Parameter Set); APS (Attribute Parameter Set); TPS (Patch Parameter Set); Geom (Geometry Bitstream (Geometry Bitstream = Geometry Slice Header + Geometry Slice Data)); Attr (Attribute Bitstream ( = Attribute Block Header + Attribute Block Data)). Here, a block may be referred to as a block, etc.

[0469] According to the embodiment, the point cloud data can be taken as follows ​ The point cloud data may include a sequence parameter set (SPS), a geometry parameter set (GPS), an attribute parameter set (APS), and a tile parameter set (TPS), which may include signaling information according to an embodiment. The point cloud data may include one or more geometries and / or attributes. The geometry and / or attributes in the point cloud data may be divided into one or more slices (or tiles / blocks). The geometry may have a structure of a geometry slice header and geometry slice data. For example, a TPS containing signaling information may include Tile(0)_tile_bounding_box_xyz0 and Tile(0)_tile_bounding_box_whd. The geometry may include geom_geom_parameter_set_id, geom_tile_id, geom_slice_id, geomBoxOrigin, geom_box_log2_scale, geom_max_node_size_log2, and geom_num_points.

[0470] Signaling information according to an embodiment may be added to SPS, GPS, APS, TPS, etc. and signaled.

[0471] According to an embodiment, the signaling information may be added to the TPS, the geometry (Geom) of each slice, or the attributes (Attr) of each slice and signaled.

[0472] The structure of point cloud data according to an embodiment can efficiently provide parameter sets, geometry, and attributes including signaling information in terms of encoding / decoding / data.

[0473] The point cloud data related to the point cloud data transmitting / receiving apparatus according to an embodiment may include at least one of a sequence parameter, a geometry parameter, an attribute parameter, a patch parameter, a geometry bitstream, or an attribute bitstream.

[0474] The syntax of specific signaling information will be described below with reference to the accompanying drawings. For reference, the names of signaling information according to the embodiments may be changed / expanded within the intended meaning / functional scope of the signaling information. The fields of the signaling information may be distinguishably referred to as first signaling information, second signaling information, etc.

[0475] As described above, the point cloud data transmitting device (for example, referring to ​ 、 ​ 、 ​ and ​ The point cloud data transmitting device described herein may transmit the encoded point cloud data in the form of a bitstream. According to an embodiment, the bitstream may include one or more sub-bitstreams.

[0476] Point cloud data transmitting device (for example, see ​ 、 ​ 、 ​ and ​ The point cloud data sending device described herein) may divide the image of the point cloud data into one or more packets and send it via the network in consideration of errors in the transmission channel. The bit stream according to the embodiment may contain one or more packets (e.g., network abstraction layer (NAL) units). Therefore, even when some packets are lost under a poor network environment, the point cloud data receiving device may use the remaining packets to restore the image. The point cloud data may be divided into one or more slices or one or more tiles and processed. The tiles and slices according to the embodiment are areas for segmenting a picture of point cloud data and processed by point cloud compression coding. The point cloud data sending device may provide high-quality point cloud content by processing data corresponding to each area according to the importance of each divided area of ​​the point cloud data. That is, the point cloud data sending device according to the embodiment may process data corresponding to areas important to the user with better compression efficiency and appropriate delay by point cloud compression coding.

[0477] The image (or picture) of the point cloud content according to the embodiment is divided into basic processing units for point cloud compression encoding. The basic processing units of the point cloud compression encoding according to the embodiment may include coding tree units (CTUs) and blocks, but are not limited to this example.

[0478] A slice according to an embodiment is an area including one or more basic processing units for point cloud compression encoding and does not have a rectangular shape. A slice according to an embodiment includes data transmitted in packets. A patch according to an embodiment is an area of ​​an image divided into rectangular shapes and includes one or more basic processing units for point cloud compression encoding. A slice according to an embodiment may be included in one or more patches. In addition, a patch according to an embodiment may be included in one or more slices.

[0479] The bitstream 3000 according to an embodiment may include signaling information and one or more slices, the signaling information including a sequence parameter set (SPS) for sequence level signaling, a geometry parameter set (GPS) for geometry information coding signaling, an attribute parameter set (APS) for attribute information coding signaling, and a patch parameter set (TPS) for patch level signaling.

[0480] The SPS according to an embodiment may be encoding information of the entire sequence such as a profile and a level, and may include comprehensive information about the entire file such as picture resolution and video format.

[0481] According to an embodiment, a slice (e.g., ​ Slice 0) includes a slice header and slice data. The slice data may include a geometry bitstream (Geom00) and one or more attribute bitstreams (Attr00 and Attr10). The geometry bitstream may include a header (e.g., a geometry slice header) and a payload (e.g., geometry slice data). The header of the geometry bitstream according to an embodiment may include identification information about a parameter set included in GPS (geom_geom_parameter_set_id), a tile identifier (geom_tile id), a slice identifier (geom_slice_id), information about the origin of the geometry box (geomBoxOrigin), the geometry box scale (geom_box_log2_scale), the maximum geometry node size (geom_max_node_size_log2), the number of geometry points (geom_num_poins), and information about the data included in the payload. The attribute bitstream may include a header (e.g., an attribute slice header or an attribute block header) and a payload (e.g., attribute slice data or attribute block data).

[0482] The method / apparatus according to an embodiment may add and signal geometric quantization option information for each axis according to an embodiment to the point cloud bitstream to the SPS. The geometric quantization option information for each axis according to an embodiment may be added and signaled to the GPS. The SPS may maintain the scale value (quantization value) applied as a whole, and the GPS may set a scale value (quantization value) fine-tuned for each axis.

[0483] When quantization is performed differently for each tile, geometric quantization option information for each axis may be added to the TPS and signaled according to an embodiment. The geometric distribution may differ between tiles divided by region. Therefore, by setting the geometric quantization value (scaling value) for each axis based on this distribution or rescaling the quantization value for each axis based on the percentage of each axis to a quantization value, geometric quantization for each axis may be performed differently for each tile.

[0484] According to an embodiment, geometric quantization option information for each axis may be added to the geometry header of each slice and signaled. The geometry distribution may differ between slices. Therefore, by setting the geometric quantization value (scaling value) for each axis according to the distribution or rescaling the quantization value for each axis according to the percentage of each axis to a quantization value, geometric quantization for each axis may be performed differently for each slice.

[0485] Therefore, according to the quantization characteristics in the bitstream, the sending method / apparatus according to the embodiment may include relevant parameter information (signaling information) in the bitstream, and may allow the receiving method / apparatus according to the embodiment to statically / dynamically access the point cloud data.

[0486] Related information may be signaled to implement the embodiment. Signaling information according to the embodiment may be used at the transmitting side or the receiving side.

[0487] Geometry bitstream (Geom): Geometry bitstream = geometry slice header + geometry slice data

[0488] Attribute bitstream (Attr): Attribute bitstream = attribute block header + attribute block data.

[0489] The transmission method / apparatus according to the embodiment may add attribute point configuration related information of the spatial scalability function of the attribute encoding / decoding process to the SPS or APS and signal the same.

[0490] The transmission method / apparatus according to the embodiment may add attribute point configuration related information of the spatial scalability function of the attribute encoding / decoding process to the TPS or Attr of each slice and signal it.

[0491] According to an embodiment, tiles or slices are provided and point clouds are processed by segmenting them into regions. That is, point cloud data can be segmented into regions, wherein the segmentation unit can be tiles and / or slices.

[0492] When dividing the point cloud data into regions, you can set the option to generate different sets of neighbor points for each region, resulting in either low complexity but slightly lower reliability, or high reliability but higher complexity. This can be set differently depending on the processing capacity of the receiver.

[0493] When a point cloud is divided into tiles, different handling options for overlapping points can be applied to each tile.

[0494] When a point cloud is divided into slices, different handling options for overlapping points can be applied to each slice.

[0495] The attribute slice data may be changed and signaled according to options related to prediction mode processing.

[0496] ​ Shown included in ​ Detailed syntax of signaling information in the bit stream of point cloud data in

[0045] To support operations according to the embodiments, the transmitting apparatus and the receiving apparatus according to the embodiments may provide a signaling scheme as described below.

[0497] ​ A sequence parameter set according to an embodiment is shown.

[0498] ​ Shown as ​ The sequence parameter set included in the bitstream according to the embodiment is shown. ​ The metadata, parameters or signaling information in ​ The sending device 10000, ​ Point cloud video encoder 10002, ​ The receiving device 10004 in ​ Point cloud video decoder 10006, ​ The code in 20001, ​ Decoding 20003, ​ The encoding process in ​ The decoding process in ​ The sending device in ​ The receiving device in ​ XR device 1430, ​ The encoder in ​ Generate, send and receive by decoder etc.

[0499] Information related to the attribute point configuration of the spatial scalability function of the attribute encoding / decoding process can be added to the sequence parameter set and signaled. ​ The syntax in

[0065] can efficiently signal the attributes of supporting spatial scalability functions. The name of the signaling information can be understood within the scope of the meaning and function of the signaling information.

[0500] LOD_generation_direction indicates the LOD generation direction.

[0501] For example, when LOD_generation_direction is equal to 0, this indicates that LOD is generated from LOD0 to LODN (i.e., from the root node to the leaf node in the case of an octree). When LOD_generation_direction is equal to 1, this indicates that LOD is generated from LODN to LOD0 (i.e., from the leaf node to the root node in the case of an octree). The integer values ​​of this information may vary depending on the implementation.

[0502] sampling_select_type indicates a point selection type of selecting one point from among points belonging to a node (including a merged node).

[0503] For example, when sampling_select_type is equal to 1, this indicates that the Nth point is selected. When sampling_select_type is equal to 2, this indicates that a point close to the center position of the octree parent node area is selected. When sampling_select_type is equal to 3, this indicates that a point with a middle index is selected. Each integer value of this information may vary depending on the implementation.

[0504] sampling_isolated_node_min_number_of_points indicates the minimum number of points belonging to a node that can be defined as an isolated node.

[0505] sampling_isolated_node_merge_flag indicates whether to merge isolated nodes.

[0506] When sampling_isolated_node_merge_flag is 1, sampling_isolated_node_max_merge_range is signaled.

[0507] sampling_isolated_node_max_merge_range indicates the maximum merge range (number of consecutive nodes) for merging isolated nodes.

[0508] sampling_select_idx indicates the index of the point to be selected by Nth point selection according to sampling_select_type. It may be set to 0 as a default value, which means selecting the first point.

[0509] When sampling_select_type is 1, sampling_select_idx is signaled.

[0510] profile_idc indicates the profile that the bitstream complies with according to the embodiment. The bitstream shall not contain values ​​of profile_idc other than those specified in the embodiment. Those values ​​of profile_idc are reserved for future use by ISO / IEC.

[0511] When profile_compatibility_flags is equal to 1, it indicates that the bitstream follows the profile indicated by profile_idc equal to j.

[0512] sps_num_attribute_sets indicates the number of coding attributes in the bitstream. The value of sps_num_attribute_sets can range from 0 to 63.

[0513] attribute_dimension[i] indicates the number of components of the i-th attribute.

[0514] attribute_instance_id[i] indicates the instance identifier of the i-th attribute.

[0515] ​ Attribute parameter sets according to implementations are shown.

[0516] ​ Shown as ​ The sequence parameter set included in the bitstream according to the embodiment is shown. ​ The metadata, parameters or signaling information in ​ The sending device 10000, ​ Point cloud video encoder 10002, ​ The receiving device 10004 in ​ Point cloud video decoder 10006, ​ The code in 20001, ​ Decoding 20003, ​ The encoding process in ​ The decoding process in ​ The sending device in ​ The receiving device in ​ XR device 1430, ​ The encoder in ​ Generate, send and receive by decoder etc.

[0517] aps_attr_parameter_set_id provides an identifier of the APS for reference by other syntax elements. The value of aps_attr_parameter_set_id can be in the range of 0 to 15 (inclusive).

[0518] aps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id for the active SPS. The value of aps_seq_parameter_set_id can be in the range of 0 to 15 (inclusive).

[0519] If attr_coding_type is equal to 0 or 2, the lifting indication information (isLifting) is 1. Otherwise, isLifting is 0.

[0520] The promotion related signaling information is signaled as follows according to the promotion indication information.

[0521] lifting_num_pred_nearest_neighbours specifies the maximum number of nearest neighbors to be used for prediction. The value of lifting_num_pred_nearest_neighbours can be in the range of 1 to xx.

[0522] lifting_max_num_direct_predictors specifies the maximum number of predictors to be used for direct prediction. The value of lifting_max_num_direct_predictors can be in the range of 0 to lifting_num_pred_nearest_neighbours. The value of the variable MaxNumPredictors can be used in the decoding process as follows: MaxNumPredictors=lifting_max_num_direct_predicots+1.

[0523] For spatial_scalability_attribute_octree_level, LOD_generation_direction, sampling_select_type, sampling_isolated_node_min_number_of_points, sampling_isolated_node_merge_flag, sampling_isolated_node_max_merge_range, and sampling_select_idx according to an embodiment, refer to ​ Description.

[0524] ​ A patch parameter set according to an embodiment is shown.

[0525] ​ Shown as ​ The sequence parameter set included in the bitstream according to the embodiment is shown. ​ The metadata, parameters or signaling information in ​ The sending device 10000, ​ Point cloud video encoder 10002, ​ The receiving device 10004 in ​ Point cloud video decoder 10006, ​ The code in 20001, ​Decoding 20003, ​ The encoding process in ​ The decoding process in ​ The sending device in ​ The receiving device in ​ XR device 1430, ​ The encoder in ​ Generate, send and receive by decoder etc.

[0526] The transmitting apparatus or encoder according to the embodiment may signal the receiving apparatus or decoder according to the embodiment by adding property point configuration related information for spatial scalability of property encoding / decoding to a patch parameter set.

[0527] num_tiles specifies the number of tiles signaled to the bitstream. When not present, num_tiles is inferred to be 0.

[0528] tile_bounding_box_offset_x[i] indicates the x offset of the i-th tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_x[0] is inferred to be sps_bounding_box_offset_x.

[0529] tile_bounding_box_offset_y[i] indicates the y offset of the i-th tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_y[0] is inferred to be sps_bounding_box_offset_y.

[0530] For spatial_scalability_attribute_octree_level, LOD_generation_direction, sampling_select_type, sampling_isolated_node_min_number_of_points, sampling_isolated_node_merge_flag, sampling_isolated_node_max_merge_range, and sampling_select_idx according to an embodiment, refer to ​ Description.

[0531] ​ An attribute slice header according to an embodiment is shown.

[0532] ​ Shown as​ The sequence parameter set included in the bitstream according to the embodiment is shown. ​ The metadata, parameters or signaling information in ​ The sending device 10000, ​ Point cloud video encoder 10002, ​ The receiving device 10004 in ​ Point cloud video decoder 10006, ​ The code in 20001, ​ Decoding 20003, ​ The encoding process in ​ The decoding process in ​ The sending device in ​ The receiving device in ​ XR device 1430, ​ The encoder in ​ Generate, send and receive by decoder etc.

[0533] The transmitting apparatus or encoder according to the embodiment may add attribute point configuration related information for a spatial scalability function used in an attribute encoding / decoding process to an attribute slice header and signal it.

[0534] ash_attr_parameter_set_id specifies the value of aps_attr_parameter_set_id of the active APS.

[0535] ash_attr_sps_attr_idx specifies the attribute set in the active SPS. The value of abh_attr_sps_attr_idx shall be in the range of 0 to sps_num_attribute_sets in the active SPS.

[0536] ash_attr_geom_slice_id specifies the value of gsh_slice_id of the active geometry slice header.

[0537] The PCC encoding method, PCC decoding method, and signaling method according to the above-described embodiments can provide the following effects.

[0538] Spatial scalability is the ability to use narrower bandwidth (e.g., thumbnails) when point cloud data is dense, or to send and process lower resolution point clouds to reduce the amount of computation on the decoder side.

[0539] An embodiment provides a method for improving attribute compression efficiency by changing a scheme of sampling points for each LOD based on an octree, so that an encoder / decoder of geometry-based point cloud compression (G-PCC) for 3D point cloud data compression provides a spatial scalability function so that the most appropriate points of each LOD are configured as nodes.

[0540] The proposed method of reorganizing points for each octree node through Morton code can reduce the encoding / decoding execution time by reducing the computational amount of encoding / decoding that constitutes the octree.

[0541] Considering the distance weight based on the center position value of the bounding box of the parent node applied for spatial scalability through the proposed sampling scheme of each node, as proposed in this paper, points close to the center position value or points with intermediate indexes can be selected to reduce the error value of the distance weight to enhance the attribute compression efficiency.

[0542] Therefore, embodiments may provide an attribute compression method for providing a spatial scalability function of an encoder / decoder of Geometry-based Point Cloud Compression (G-PCC) for compressing 3D point cloud data and may increase attribute compression efficiency.

[0543] By providing a method of changing a point sampling technique for each LOD based on an octree, a PCC encoder and / or a PCC decoder according to an embodiment may have a spatial scalability effect.

[0544] Next, a geometry / attribute compression method for compressing 3D point cloud data using spatial scalability of an encoder / decoder supporting G-PCC according to an embodiment will be described.

[0545] The embodiment proposes a method of changing an octree-based method of setting geometric point position values ​​and a point sampling method for each LOD to provide a spatial scalability function in the attribute encoding / decoding process of G-PCC.

[0546] The method according to the embodiment includes the following.

[0547] The present invention proposes 1) a method for calculating geometric positions in partial recovery to support spatial scalability, 2) a method for sampling reference points for each node, and 3) a signaling method for supporting the above methods.

[0548] Reference ​ Embodiments relate to a method for compressing 3D point cloud data using compression attributes to support spatial scalability of G-PCC. When point cloud data is dense, it may be desirable to use a narrower bandwidth (e.g., thumbnails) or to transmit and process lower resolution point clouds to reduce computational complexity at the decoder. This capability is referred to as spatial scalability.

[0549] The transmission / reception method / apparatus according to the embodiment may provide a spatial scalability function through a process of encoding or decoding only occupied bits up to a depth level selected by adjusting the depth level of the octree.

[0550] ​ Attribute encoding / decoding according to an embodiment is shown.

[0551] ​ Shown is the set geometry for providing ​ The method of spatial scalability function is shown.

[0552] ​ The method in ​ The encoder in ​ The decoder and / or the corresponding sending / receiving device according to the embodiment are executed.

[0553] In order to provide spatial scalability even for attribute encoding / decoding, it may be necessary to modify and apply the octree structure. The method of generating LOD starting from the selected depth level of the octree, the method of calculating neighbor nodes, and the method of calculating weights may need to be changed to provide spatial scalability in attribute encoding / decoding.

[0554] The embodiment proposes setting the geometric position value in the geometric part decoding. The embodiment also provides a technology for changing the sampling points for each LOD based on the octree and configuring the most appropriate point as a node for each LOD to provide spatial scalability in attribute encoding / decoding, thereby improving attribute compression efficiency.

[0555] The embodiment provides a method for changing the point sampling type of each LOD based on an octree to provide a spatial scalability function in the attribute encoding / decoding process of G-PCC.

[0556] Embodiments may set representative position values ​​for an octree to provide spatial scalability in geometry decoding. To provide spatial scalability for attribute encoding / decoding, an octree structure may be applied in geometry encoding / decoding. Embodiments provide a method for reassembling points belonging to a node starting from a selected depth level of the octree, as well as a method for sampling points at each level of detail (LOD).

[0557] The method of recombining points and the method for each LOD sampling point are performed in PCC attribute encoding / decoding of a PCC decoder / encoder, and can be applied when a predictive transform technique or a lifting transform technique is used in the attribute encoding / decoding process.

[0558] ​

[0559] When decoding a point in a point cloud, it is necessary to perform decoding only up to a certain octree depth and recover the position value of the point. A function supporting this operation may be spatial scalability. When partially performing decoding at an intermediate depth of the octree, the node closest to the center position of the bounding box of each octree node can be sampled (selected) as the position of the point representing each octree node. In this case, when an attribute is partially decoded, Morton order, LOD configuration, and neighbor node search can be implemented based on the position set to the intermediate position of the node.

[0560] S270010 Sorting based on Morton order

[0561] Points in the point cloud may be converted to Morton codes based on x, y, and z position values ​​and may be sorted based on the Morton code values.

[0562] S270020 Set the attribute part encoding / decoding area

[0563] In attribute encoding / decoding, the depth level of the octree can be set, and points to which attribute encoding is applied can be constructed and encoded / decoded starting from the set octree depth level, just like in geometry encoding / decoding (see Figure 17 ).

[0564] S270030 uses Morton codes to reorganize points belonging to nodes

[0565] The method of reorganizing the points sorted by Morton code for each octree node can be referred to Figure 16 The following pseudo code describes it.

[0566] nodeSizeLog2: octree depth level

[0567] PX_mortoncode: PX Morton code

[0568] end_range_mc: the Morton code range of the node

[0569] Int maxNumOfPoints=pow(8,nodeSizeLog2+1);

[0570] For(int I=1;I <indexCount;++i){

[0571] if(Pi_mortoncode <end_range_mc){

[0572] …

[0573] }else{

[0574] end_range_mc=Pi_mortoncode+

[0575] (maxNumOfPoints-(Pi_mortoncode%maxNumOfPoints));

[0576] }

[0577] }

[0578] The Morton code range of a node can be calculated based on the Morton code of the point, the octree depth level and the maximum number of points.

[0579] like Figure 18 As shown, end_range_mc can be estimated from the Morton code of a point. When the Morton code value of the next point is within the range of end_range_mc, the points may belong to the same node. If the Morton code value of the next point is outside this range, this may mean that the node has changed to a new node. In this case, end_range_mc can be estimated based on the Morton code of the point and applied to subsequent points. In this way, points belonging to an octree node can be reorganized from points sorted by Morton code.

[0580] S270030 LOD sampling at octree nodes

[0581] Sampling can be performed by selecting points from among the points belonging to a node (including merging nodes). Attribute compression can be affected by the selected points. The best reference points are selected and reorganized into a reference region (retention list), and a set of neighbor nodes is constructed from the reference region. Thus, compression efficiency can be increased.

[0582] When the position of the geometric octree node changes to the center point, the error generated at the parent node can be reduced by selecting a position close to the center position value as the point selected in the attribute encoding. Therefore, the LOD sampling at the octree node can adopt the method of selecting a position close to the center of the parent node.

[0583] There are many ways to find a location close to the center. The location can be calculated to find the location correctly. To reduce the time required to find the location, the location on the octree node can be referenced. Various options are given below. Figures 28 to 30 Describes sampling: 1) Select the first node at a LOD, then select the last node at the next consecutive upper LOD (see Figures 28 to 30 ); 2) select a point close to the center of the octree parent node area; 3) select a point with an intermediate index; and / or 4) a combination thereof.

[0584] A method of receiving point cloud data according to an embodiment may include: receiving a bit stream including the point cloud data; and decoding the point cloud data.

[0585] Point cloud data according to an embodiment may include geometric data and attribute data.

[0586] According to an embodiment, decoding of point cloud data may include decoding geometric data and decoding attribute data.

[0587] The geometric data decoding according to an embodiment may include: generating an octree structure for the geometric data; and sampling (selecting) positions of points representing respective nodes included in the octree structure as nodes closest to center positions of bounding boxes of the respective nodes.

[0588] According to an embodiment, attribute data decoding may include: changing the position value of a point in the point cloud data into a Morton code based on the geometric data and the attribute data; sorting the points based on the Morton code; and decoding the points starting from a depth level of an octree structure including the points.

[0589] The attribute data decoding according to an embodiment may further include reorganizing points belonging to each node based on the Morton code and the range value of the Morton code.

[0590] According to an embodiment, attribute data decoding may include: generating an octree structure for geometric data; generating a level of detail (LOD) based on the octree structure; and sampling (selecting) a node belonging to the LOD from the LOD. That is, sampling a point included in the node.

[0591] Figure 28 Shown are bounding boxes and nodes according to an embodiment.

[0592] Figure 28 express Figure 27 S270010 in.

[0593] Eight nodes may be included in the bounding box. When sorted in the order of the Morton code, points having values ​​0 to 7 as Morton code values ​​may be arranged in sequence.

[0594] Figure 29 Shown are the matches between various nodes and locations according to an embodiment.

[0595] Figure 29 Show when point Figure 28 The operation that matches each node to its position when sorted in the order of the Morton code. Figure 29 express Figure 27 S270030 in.

[0596] The reference node (or reserved node) represents the node (point) selected during sampling. For example, Figure 29 Nodes (or points) 29000, 29040, and 29050 in Figure 30 A node (or point) in can be a reference node.

[0597] As shown in the figure, when the first node 29000 (LOD N) is selected, the node closest to the center position 29030 of the bounding box 29020 of the first node's parent node 29010 (LOD N-1) may be the last node 29040. When the last node 29040 is selected at LOD N-1, the node closest to the center position of its parent node at LOD N-2 may be the first node 29050.

[0598] Therefore, in LOD sampling, when the LOD is even, the first node may be selected, and when the LOD is odd, the last node may be selected. Alternatively, when the LOD is even, the last node may be selected, and when the LOD is odd, the first node may be selected. The encoder may signal the configuration information about the above selection scheme to the decoder.

[0599] That is, the encoder and / or decoder according to the embodiment provides various schemes for selecting the node closest to the center position on the parent node among the child nodes when performing LOD sampling at the octree node. The embodiment includes a scheme using even indexes and odd indexes or a scheme for selecting the node closest to the center position of the bounding box. For example, the node closest to the center position can be selected based on the distance, density, and attributes of the point. In addition, when selecting a point close to the center position, if the selected point belongs to a 2×2×2 node, the node close to the center can be sampled. Otherwise, odd / even nodes (i.e., the first / last node) can be sampled.

[0600] Figure 30 FIG. 1 illustrates selecting a reference node in an octree structure according to an embodiment.

[0601] Figure 30 Shows the octree structure Figure 28 and Figure 29 LOD sampling ( Figure 27 S270030 in the ).

[0602] In the octree structure according to the embodiment, nodes may be sequentially distributed from the root node to the leaf node, and the next node belonging to the node may be sorted in the order of the Morton code. The octree structure may include N LODs, and the transmission / reception method / apparatus according to the embodiment may perform partial encoding and partial decoding starting from a specific depth level in the octree structure.

[0603] In the octree structure, the points belonging to LOD N are sorted in the order of the Morton code, and the reference node is the node sampled at each LOD. When LOD N (where N can be called the index of the LOD) is an even LOD, the first point can be selected from the points. When LOD N-1 is an odd LOD, the last point can be selected from the points. As the operation proceeds from a specific node to the parent node in the octree structure, the nodes can be selected in the order of first node, last node, first node, last node, etc. This scheme can increase the sampling gain.

[0604] For example, for point 30000 belonging to LOD N, when LOD N is assumed to be an even LOD, the first node 30010 among the points belonging to the node sorted in the order of the Morton code can be selected (sampled) as the reference node (or retained node) 30010 of each parent node.

[0605] For the point 30020 belonging to LOD N-1, N-1 can be regarded as an odd number, and the last node 30030 among the points belonging to this node sorted in the order of the Morton code can be selected (sampled) as the reference node of each parent node.

[0606] According to an embodiment, sampling may be performed by sampling the first node at the level where LOD sampling starts, sampling the last node at the next level, and sampling the first node at the next level. Alternatively, sampling may be performed by sampling the last node at the level where LOD sampling starts, sampling the first node at the next level, and sampling the last node at the next level.

[0607] This operation can be expressed as pseudo code as follows.

[0608] endIndex=0

[0609] assignedPointCount=0

[0610] for(lod=minGeomNodeSizeLog2; unprocessedPointCount>0; lod++){

[0611] startIndex=assignedPointCount

[0612] nonAssignedPointCount=0

[0613] samplingFromLastFlag = lod & 1 / / If LOD is even, the last bit is assigned 0, if LOD is odd, it is assigned 1. / /

[0614] for(i=0;i <unprocessedPointCount;i++){

[0615] currVoxelIndex=McodeUnsorted[unprocessedPointIndexes[i]]>>(3×(lod+1))

[0616] if(samplingFromLastFlag==0){

[0617] if (i == 0) / / This means that the LOD is even, and the first point is sampled as the reference node.

[0618] nonAssignedPointIndexes[nonAssignedPointCount++]= unprocessedPointIndexes[i]

[0619] else{

[0620] prevVoxelIndex=McodeUnsorted[unprocessedPointIndexes[i-1]]>>(3×(lod +1))

[0621] if(currVoxelIndex>prevVoxelIndex)

[0622] nonAssignedPointIndexes[nonAssignedPointCount++]= unprocessedPointIndexes[i]

[0623] (Here, sampled points are assigned to NonassignedPointsIndexes. Non-sampled points are assigned to assignedPointIndexes. By setting NonassignedPointsIndexes and assignedPointIndexes separately, the method / apparatus according to the embodiment can reorganize the LOD.)

[0624] else

[0625] assignedPointIndexes[assignedPointCount++]=unprocessedPointIndexes[i]

[0626] }

[0627] }else{ / / This means that the LOD is even and the last point is sampled as the reference node.

[0628] if(i==(unprocessedPointCount–1))

[0629] nonAssignedPointIndexes[nonAssignedPointCount++]= unprocessedPointIndexes[i]

[0630] else{

[0631] nextVoxelIndex = McodeUnsorted[unprocessedPointIndexes[i + 1]]>>(3×(lod + 1))

[0632] if (currVoxelIndex < nextVoxelIndex)

[0633] nonAssignedPointIndexes[nonAssignedPointCount++]= unprocessedPointIndexes[i]

[0634] (Here, NonassignedPointsIndexes represents the sampled points.)

[0635] else

[0636] assignedPointIndexes[assignedPointCount++] = unprocessedPointIndexes[i]

[0637] }

[0638] }

[0639] }

[0640] endIndex = assignedPointCount

[0641] if (startIndex!= endIndex) {

[0642] numOfPointInSkipped = PointNumInSlice – PointCount

[0643] if ((endIndex – startIndex) > (startIndex + numOfPointInSkipped)) {

[0644] for (loop = 0; loop < lod - minGeomNodeSizeLog2; loop++) {

[0645] computeNearestNeighbours(

[0646] PointCount - unprocessedPointCountPerLevelOfDetail[loop],

[0647] PointCount - unprocessedPointCountPerLevelOfDetail[loop + 1],

[0648] loop + minGeomNodeSizeLog2, assignedPointIndexes,

[0649] McodeUnsorted, nonAssignedPointCount,

[0650] nonAssignedPointIndexes)

[0651] }

[0652] }

[0653] }

[0654] computeNearestNeighbours(

[0655] startIndex, endIndex,

[0656] lod, assignedPointIndexes,

[0657] McodeUnsorted, nonAssignedPointCount,

[0658] nonAssignedPointIndexes)

[0659] unprocessedPointCountPerLevelOfDetail[lod + 1] = nonAssignedPointCount

[0660] unprocessedPointCount = nonAssignedPointCount

[0661] unprocessedPointIndexes = nonAssignedPointIndexes

[0662] }

[0663] Then, apply the following process:

[0664] for(i = 0; i < PointCount; i++)

[0665] indexes[PointCount - 1 – i] = assignedPointIndexes[i]

[0666] for(lod=0;lod <LevelDetailCount;lod++)

[0667] pointCountPerLevelOfDetail[lod]=

[0668] unprocessedPointCountPerLevelOfDetail[LevelDetailCount-1-lod]

[0669] / / The value of samplingFromLastFlag indicates whether sampling starts from the first node (or left) or the last node (or right) at the starting LOD. Signal notification is available.

[0670] According to an embodiment, if signaling information such as a lifting LOD sampling related flag (lifting_lod_regular_sampling_enabled_flag) included in the bitstream indicates an LOD sampling scheme, the LOD may be reorganized from the octree nodes based on the geometry data (if lifting_scalability_enabled_flag is equal to 1, the level of detail generation process reorganizes the points into a set of refinement levels (R_l)_(l=0…L-1) from the octree nodes based on the geometry. Depending on the value of the sampling related flag (samplingFromLastFlag), the first point in the node (i.e., samplingFromLastFlag is equal to 0) or the last point in the node (i.e., samplingFromLastFlag is equal to 1) is sampled).

[0671] Next, a scheme of selecting a point close to the center of an octree parent node area and a scheme of selecting a point with an intermediate index according to an embodiment will be described.

[0672] Select a point close to the center of the octree's parent node region:

[0673] The transmitting / receiving method / device according to the embodiment can select a point from among the points belonging to a node (including a merged node) and select the point closest to the center of the bounding box of the parent node. Selecting a point closer to the center of the parent node can reduce errors in selecting neighbor points and calculating neighbor point weights.

[0674] When selecting neighbor points, the distance between the points is calculated, and the point with a closer distance can be selected as the neighbor point. In order to provide a spatial scalability function, the position value of the point can be replaced by the value of the center position of the node to which the point belongs (the node can change according to the level of LOD), and then the distance can be calculated. Therefore, in order to find the neighbor point of point Pi, the position value of Pi can be changed to the center position of the parent node according to LOD1, and the position of the point Ri in the neighbor point candidate list (retention list) R also changes according to LOD1. Then, the distance of the point can be calculated. In other words, the distance between the center positions of the nodes at the same depth level can be calculated, so that the neighbor node is set as the neighbor point. From the distance calculated in this way, the weight of the neighbor point can be obtained, 1 / distance=Pi. Since the distance is calculated by changing the position of the point to the center position of the parent node, the calculation contains errors. As the depth level of the node decreases, that is, the bounding box of the node becomes larger, the error can increase.

[0675] In order to reduce the error, a point close to the center of the parent node area can be selected as the reference point. The error can be reduced according to the position value of the reference point, and the reduced error can have a good effect on the compression efficiency.

[0676] Select the point with the middle index:

[0677] When selecting a point from among the points belonging to a node (including a merged node), the transmission / reception method / apparatus according to the embodiment may select a point with an intermediate index from among the points belonging to the node (including a merged node). Since the probability of the node with the intermediate index being close to the center position of the bounding box of the parent node is high (in many cases), and selecting a point using the intermediate index does not require much additional calculation, the probability of finding a point close to the center position with a small amount of calculation can be high.

[0678] The method of receiving point cloud data according to an embodiment may include sampling (selecting) a first point or a last point in a node belonging to the LOD based on the LOD.

[0679] Figure 31 A PCC data encoder according to an embodiment is shown.

[0680] Figure 31 The encoder in corresponds to Figure 20 The point cloud data encoder and / or the corresponding device according to the embodiment. Figure 31 For the operation of each component in Figure 20 Description.

[0681] The LOD configuration unit 31000 may perform LOD configuration and sampling according to an embodiment. The LOD configuration unit 31000 may use Morton code to reorganize points belonging to a node, merge isolated nodes, perform LOD sampling in an octree node according to an embodiment, and signal related information.

[0682] The LOD configuration unit 31000 may set the depth level of the octree to support spatial scalability in attribute encoding / decoding, and configure and encode / decode points to which attribute encoding is to be applied starting from the set depth level of the octree (see Figure 17 and Figure 30 ).

[0683] The transmitting device or encoder according to the embodiment may signal the octree depth level as signaling information to the receiving device or decoder according to the embodiment, and the octree depth level may be changed according to its use or the computational performance of the decoder.

[0684] The LOD construction unit 31000 may perform point reorganization for each octree node by calculating the Morton code range of each octree node at points sorted by the Morton code (see Figure 18 and Figure 27 ).

[0685] When the LOD component 31000 selects a point from among the points belonging to a node, a selection scheme of the operation may be signaled to the decoder. The selection scheme may include a scheme of selecting the first node at an even LOD and the last node at an odd LOD, a scheme of selecting a point close to the center of the octree parent node region, and a scheme of selecting a point with an intermediate index (see Figures 28 to 30 ).

[0686] In a scheme where the first node is selected at an LOD and the last node is selected at the next consecutive LOD, it may be determined whether the first or last node is selected at an even LOD. This determination may be signaled to the decoder (see Figures 28 to 30 ).

[0687] The residual attribute information quantization processor 31010 can generate a predicted value of the attribute data, generate a residual value between the predicted value and the attribute value, and quantize the residual value.

[0688] When the prediction mode and residual information separation flag is true, the residual attribute information quantization processor 31010 can calculate the residual between the predicted value according to the prediction mode and the value of the point, and encode its quantized value together with the prediction mode.

[0689] When the prediction mode and residual information separation flag is false, the residual attribute information quantization processor 31010 may calculate the residual between the prediction value and the point value according to the prediction mode, and encode only the quantized value thereof.

[0690] Figure 32 A PCC data decoder according to an embodiment is shown.

[0691] Figure 32 The decoder in corresponds to Figure 21 The point cloud data decoder and / or the corresponding device according to the embodiment. Figure 32 For the operation of each component in Figure 21 Description.

[0692] The octree reconstructor 32000 reconstructs an octree from the received geometry information bitstream (geometry data). For spatial scalability, the octree reconstructor 32000 may construct geometry information only at a specific level of the octree. In this case, the position value may be changed to the value of the center of the bounding box.

[0693] The octree reconstructor 32000 may construct and decode points to which attribute decoding is to be applied for partial geometry decoding starting from the restored octree depth level.

[0694] The LOD configuration unit 32010 may use Morton code to reorganize points belonging to a node, merge isolated nodes, and perform LOD sampling in the octree node according to an embodiment.

[0695] The LOD configuration unit 32010 may construct and decode a point to which attribute decoding is to be applied starting from the octree depth level restored for partial attribute decoding.

[0696] The LOD configuration unit 32010 may perform point reorganization on each octree node by calculating a Morton code range of each octree node from points sorted by Morton code.

[0697] The LOD configuration unit 32010 may select a point from among the points belonging to a node (including a merged node) based on a scheme for selecting a restored point. The scheme may include selecting the first node at a LOD and then selecting the last node at the next consecutive upper LOD, selecting a point near the center of the octree parent node area, and selecting a point with an intermediate index.

[0698] In a scheme of selecting a first node at an LOD and then selecting a last node at a next consecutive upper LOD, the LOD configuration unit 32010 may configure the LOD by selecting the first node and the last node according to the restored selection type.

[0699] Figures 33 to 36 Show Figure 22 Detailed syntax of the signaling information contained in the bit stream of the point cloud data in

[0054] . In order to support the operation according to the embodiment, the transmitting device and the receiving device according to the embodiment may provide a signaling scheme as described below.

[0700] The transmitting device and the receiving device according to the embodiment can be configured by adding it to the SPS ( Figure 33 ) or APS( Figure 34) to signal information related to the configuration of attribute points of the spatial scalability function.

[0701] The transmitting device and the receiving device according to the embodiment can be configured by adding them to the TPS ( Figure 35 ) or the Attr( Figure 36 ) is used to signal information related to the configuration of attribute points of the spatial scalability function. Due to the signaling information according to the embodiment, the geometry / attributes for supporting the spatial scalability function can be efficiently signaled.

[0702] Figure 33 A sequence parameter set (SPS) according to an embodiment is shown.

[0703] Figure 33 Shown as Figure 22 The sequence parameter set included in the bitstream according to the embodiment is shown. Figure 33 The metadata, parameters or signaling information in Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 1 The receiving device 10004 in Figure 1 Point cloud video decoder 10006, Figure 2 The code in 20001, Figure 2 Decoding 20003, Figure 4 The encoding process in Figure 11 The decoding process in Figure 12 The sending device in Figure 13 The receiving device in Figure 14 XR device 1430, Figure 20 The encoder in Figure 21 Generate, send and receive by decoder etc.

[0704] Information on the configuration of the attribute points of the spatial scalability function may be added to a sequence parameter set and signaled.

[0705] spatial_scalability_octree_level indicates the partially restored octree level.

[0706] spatial_scalability_attribute_octree_level indicates the partially restored octree level. That is, it indicates the partial octree level applied to the attribute. Attributes can have a different restored octree level than the geometry.

[0707] LOD_generation_direction indicates the LOD generation direction.

[0708] For example, when LOD_generation_direction is equal to 0, this indicates that LOD is generated from LOD0 to LODN (i.e., from the root node to the leaf node in the case of an octree). When LOD_generation_direction is equal to 1, this indicates that LOD is generated from LODN to LOD0 (i.e., from the leaf node to the root node in the case of an octree). The integer value indicating the direction may vary depending on the implementation.

[0709] sampling_select_type indicates the point selection type when selecting a point from among the points belonging to a node (including a merge node). For example, when sampling_select_type is equal to 1, this indicates that the first node is selected at even LOD and the last node is selected at odd LOD. When sampling_select_type is equal to 2, this indicates that a point close to the center position of the octree parent node area is selected. When sampling_select_type is equal to 3, this indicates that a point with an intermediate index is selected. The integer value indicating the type may vary depending on the implementation.

[0710] sampling_first_node_flag indicates whether to select the first node or the last node at the starting LOD.

[0711] When sampling_select_type == 1, sampling_first_node_flag may be provided as signaling information.

[0712] For other fields, refer to Figure 23 Description.

[0713] Figure 34 An attribute parameter set (APS) according to an embodiment is shown.

[0714] Figure 34 Shown as Figure 22 The sequence parameter set included in the bitstream according to the embodiment is shown. Figure 34 The metadata, parameters or signaling information in Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 1 The receiving device 10004 in Figure 1 Point cloud video decoder 10006, Figure 2 The code in 20001, Figure 2 Decoding 20003, Figure 4 The encoding process in Figure 11 The decoding process in Figure 12 The sending device in Figure 13 The receiving device in Figure 14 XR device 1430, Figure 20 The encoder in Figure 21 Generate, send and receive by decoder etc.

[0715] Information on the configuration of the attribute points of the spatial scalability function may be added to the attribute parameter set and signaled.

[0716] For spatial_scalability_attribute_octree_level, LOD_generation_direction and sampling_select_type, sampling_first_node_flag, refer to Figure 33 Description.

[0717] For other fields, refer to Figure 24 Description.

[0718] Figure 35 A patch parameter set (TPS) according to an embodiment is shown.

[0719] Figure 35 Shown as Figure 22 The sequence parameter set included in the bitstream according to the embodiment is shown. Figure 35 The metadata, parameters or signaling information in Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 1 The receiving device 10004 in Figure 1 Point cloud video decoder 10006, Figure 2 The code in 20001, Figure 2 Decoding 20003, Figure 4 The encoding process in Figure 11 The decoding process in Figure 12 The sending device in Figure 13 The receiving device in Figure 14 XR device 1430, Figure 20 The encoder in Figure 21 Generate, send and receive by decoder etc.

[0720] Information on the configuration of the attribute points of the spatial scalability function may be added to the tile parameter set and signaled.

[0721] For spatial_scalability_attribute_octree_level, LOD_generation_direction and sampling_select_type, sampling_first_node_flag, refer to Figure 33 Description.

[0722] For other fields, refer to Figure 25 Description.

[0723] Figure 36 An attribute slice header (ASH) according to an embodiment is shown.

[0724] Figure 36 Shown as Figure 22 The sequence parameter set included in the bitstream according to the embodiment is shown. Figure 36 The metadata, parameters or signaling information in Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 1 The receiving device 10004 in Figure 1 Point cloud video decoder 10006, Figure 2 The code in 20001, Figure 2 Decoding 20003, Figure 4 The encoding process in Figure 11 The decoding process in Figure 12 The sending device in Figure 13 The receiving device in Figure 14 XR device 1430, Figure 20 The encoder in Figure 21 Generate, send and receive by decoder etc.

[0725] Information on the configuration of the attribute points of the spatial scalability function may be added to the attribute slice header and signaled.

[0726] For spatial_scalability_attribute_octree_level, LOD_generation_direction and sampling_select_type, sampling_first_node_flag, refer to Figure 33 Description.

[0727] For other fields, refer to Figure 26 Description.

[0728] Figure 37 A method of transmitting point cloud data according to an embodiment is shown.

[0729] S37000: The method for transmitting point cloud data according to an embodiment may include encoding the point cloud data. The encoding operation according to an embodiment may be performed with Figure 1 The sending device 1000, Figure 1 Point cloud video encoder 10002, Figure 2 The code in 20001, Figure 4 The encoder in Figure 12 The encoder in Figure 14 The XR device 1430 in Figure 15 The partial decoding corresponds to the partial encoding, Figures 16 to 19 The encoding process in Figure 20 The encoder in Figures 22 to 26 Point cloud data bit stream processing in Figures 27 to 30 The encoding process in Figure 31 The encoder in Figures 33 to 36 The operations corresponding to or combined with the point cloud data bit stream processing operations.

[0730] S37010: The method for transmitting point cloud data according to an embodiment may further include transmitting a bit stream containing point cloud data. The transmitting operation according to an embodiment may be performed with Figure 1 Transmitter 10003 in Figure 2 Transmission 20002 and Figure 12 The operations of the transmission processor 12012 correspond to or are combined.

[0731] Figure 38 A method of receiving point cloud data according to an embodiment is shown.

[0732] S38000: The method for receiving point cloud data according to an embodiment may include receiving a bit stream containing point cloud data. The receiving operation according to an embodiment may be performed with Figure 1 Receiver 10005 in Figure 2 Transmission 20002 and Figure 13 The operations of the receiver 13000 correspond to or are combined.

[0733] S38010: The method for receiving point cloud data according to an embodiment may further include decoding the point cloud data. The decoding operation according to an embodiment may be performed with Figure 1 Point cloud video decoder 10006, Figure 2 Decoding 20003, Figure 10 The decoder in Figure 11 The decoder in Figure 13 The decoder in Figure 14 XR device 1430, Figure 15 Partial decoding in Figures 16 to 19 The decoding process in Figure 21 The decoder in Figures 22 to 26 Point cloud data bit stream processing and solution Figures 27 to 30 The decoding process in Figure 32 The decoder in Figures 33 to 36 The operations corresponding to or combined with the point cloud data bit stream processing and the like.

[0734] The PCC encoding method, PCC decoding method, and signaling method according to the above-described embodiments can provide the following effects.

[0735] Spatial scalability is the ability to use narrower bandwidth (e.g., thumbnails) when point cloud data is dense, or to send and process lower resolution point clouds to reduce the amount of computation on the decoder side.

[0736] An embodiment provides a method for improving attribute compression efficiency by changing a scheme for restoring the center position value of the bounding box of each node based on an octree as the position value of the point cloud and sampling points for each LOD, so that an encoder / decoder of geometry-based point cloud compression (G-PCC) for 3D point cloud data compression provides a spatial scalability function so that the most appropriate point of each LOD is configured as a node.

[0737] The proposed method of reorganizing points for each octree node through Morton code can reduce the encoding / decoding execution time by reducing the computational amount of encoding / decoding that constitutes the octree.

[0738] Considering the distance weight based on the center position value of the bounding box of the parent node applied for spatial scalability through the proposed sampling scheme of each node, as proposed in this paper, points close to the center position value or points with intermediate indexes can be selected to reduce the error value of the distance weight to enhance the attribute compression efficiency.

[0739] Therefore, embodiments may provide an attribute compression method for providing a spatial scalability function of an encoder / decoder of Geometry-based Point Cloud Compression (G-PCC) for compressing 3D point cloud data and may increase attribute compression efficiency.

[0740] A PCC encoder and / or PCC decoder according to an embodiment may provide a spatial scalability function by providing a scheme for restoring geometric point positions to center position values ​​of nodes based on an octree and changing a point sampling scheme for each LOD.

[0741] By proposing a method for calculating geometric positions in partial restoration for supporting spatial scalability, a method for sampling reference points for each node, and a signaling method supporting the above method, a spatial scalability effect on attribute encoding / decoding of G-PCC can be provided.

[0742] In this document, “A / B” is interpreted as A and / or B.

[0743] The embodiments have been described from the perspective of methods and / or devices, and the descriptions of the methods and devices may be applied to supplement each other.

[0744] Although the drawings are described separately for simplicity, new embodiments can be designed by combining the embodiments shown in the various figures. Designing a recording medium that can be read by a computer and records a program for executing the above-mentioned embodiments according to the needs of those skilled in the art also falls within the scope of the attached claims and their equivalents. The apparatus and method according to the embodiment may not be limited to the configuration and method of the above-mentioned embodiment. Various modifications can be made to the embodiment by selectively combining all or some of the embodiments. Although preferred embodiments have been described with reference to the drawings, it will be understood by those skilled in the art that various modifications and changes can be made to the embodiment without departing from the spirit or scope of the present disclosure described in the attached claims. These modifications should not be understood separately from the technical ideas or viewpoints of the embodiment.

[0745] The various elements of the device of the embodiment can be implemented by hardware, software, firmware or a combination thereof. The various elements in the embodiment can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiment, the components according to the embodiment can be implemented as separate chips respectively. According to the embodiment, at least one or more components of the device according to the embodiment may include one or more processors capable of executing one or more programs. One or more programs can execute any one or more operations / methods according to the embodiment or include instructions for executing them. The executable instructions for executing the method / operation of the device according to the embodiment can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a temporary CRM or other computer program product configured to be executed by one or more processors. In addition, the memory according to the embodiment can be used as not only covering volatile memory (e.g., RAM), but also covering the concept of non-volatile memory, flash memory and PROM. In addition, it can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). In addition, the processor-readable recording medium can be distributed to computer systems connected via a network so that the processor-readable code can be stored and executed in a distributed manner.

[0746] In this specification, the terms " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" may mean "A and / or B". In addition, "A, B" may mean "A and / or B". In addition, "A / B / C" may mean "at least one of A, B and / or C". In addition, "A / B / C" may mean "at least one of A, B and / or C". In addition, in this specification, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" may mean 1) only A, 2) only B, or 3) both A and B. In other words, the term "or" used in this document should be interpreted as indicating "in addition or alternatively".

[0747] Terms such as first and second may be used to describe various elements of an embodiment. However, the various components according to the embodiment should not be limited by the above terms. These terms are only used to distinguish one element from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should not be interpreted without departing from the scope of the various embodiments. Both the first user input signal and the second user input signal are user input signals, but unless the context clearly dictates otherwise, they do not refer to the same user input signal.

[0748] The terms used to describe the embodiments are used to describe specific embodiments and are not intended to limit the embodiments. As used in the description of the embodiments and in the claims, unless the context clearly dictates otherwise, the singular includes the plural referents. The expression "and / or" is used to include all possible combinations of terms. Terms such as "including" or "having" are intended to indicate the presence of graphics, quantities, steps, elements and / or components, and should be understood as not excluding the possibility of additional graphics, quantities, steps, elements and / or components. As used herein, conditional expressions such as "if" and "when" are not limited to optional cases and are intended to be interpreted as performing the relevant operation when a specific condition is met, or interpreting the relevant definition based on the specific condition.

[0749] The operations described in this specification according to the embodiments may be performed by a transmitting / receiving device including a memory and / or a processor according to the embodiments. The memory may store programs for processing / controlling the operations according to the embodiments, and the processor may control the various operations described in this specification. The processor may be referred to as a controller, etc. In the embodiments, the operations may be performed by firmware, software, and / or a combination thereof. The firmware, software, and / or a combination thereof may be stored in the processor or the memory.

[0750] Invention Mode

[0751] As described above, the relevant contents are described in the best mode for carrying out the embodiment.

[0752] Industrial Applicability

[0753] As described above, the embodiments may be applied in whole or in part to a point cloud data transmitting / receiving device and system.

[0754] It will be apparent to those skilled in the art that various changes or modifications can be made to the embodiments within the scope of the embodiments.

[0755] Thus, it is intended that the embodiments cover the modifications and variations of this disclosure provided they come within the scope of the appended claims and their equivalents.

Claims

1. A device, comprising: Memory; as well as at least one processor connected to the memory, the at least one processor being configured to: receiving a bitstream comprising point cloud data, The point cloud data includes geometric data and attribute data; decoding the geometric data; and decoding the attribute data, The geometric data includes an octree, Generate the level of detail LOD based on the octree, Sampling the points in the nodes of the LOD, where the point with the lowest point index in an even level is assigned to the output of that level, and Among them, the point with the maximum point index in an odd-numbered level is assigned to the output of that level.

2. The device according to claim 1, in, The at least one processor is further configured to: generating an octree for the geometric data, The position of the point representing each node of the octree is set to the node closest to the center position of the bounding box for each node.

3. The device according to claim 1, in, The at least one processor is further configured to: changing the position value of the point of the point cloud data into a Morton code based on the geometric data and the attribute data, arranging the points based on the Morton code, The point is decoded from a depth level of the octree that includes the point.

4. The device according to claim 3, in, The processor is further configured to: Points included in respective nodes of an octree are arranged based on the Morton code and a range value of the Morton code.

5. A device, comprising: Memory; as well as at least one processor connected to the memory, the at least one processor being configured to: Encode the geometric data of the point cloud data; encoding the attribute data of the point cloud data, The geometric data includes an octree, Generate the level of detail LOD based on the octree, Sampling points in the nodes of the LOD; generating a bitstream including the point cloud data, where the point with the lowest point index in an even level is assigned to the output of that level, and Among them, the point with the maximum point index in an odd-numbered level is assigned to the output of that level.

6. The device according to claim 5, in, The processor is further configured to: generating an octree for the geometric data, The position of the point representing each node of the octree is set to the node closest to the center position of the bounding box for each node.

7. The device according to claim 5, in, The processor is further configured to: changing the position value of the point of the point cloud data into a Morton code based on the geometric data and the attribute data, arranging the points based on the Morton code, The point is encoded from a depth level of the octree that includes the point.

8. The device according to claim 7, in, The processor is further configured to: Points included in respective nodes of an octree are arranged based on the Morton code and a range value of the Morton code.

9. A device comprising: Memory; as well as at least one processor connected to the memory, the at least one processor being configured to: Encode the geometric data of the point cloud data; encoding the attribute data of the point cloud data, The geometric data includes an octree, Generate the level of detail LOD based on the octree, Sampling points in the nodes of the LOD; as well as sending a bit stream comprising said point cloud data, where the point with the lowest point index in an even level is assigned to the output of that level, and Among them, the point with the maximum point index in an odd-numbered level is assigned to the output of that level.

10. A computer-readable storage medium storing a bitstream generated by the apparatus according to claim 5.