Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Abstract

The point cloud data transmission method according to the embodiments can include the steps of encoding point cloud data, and transmitting a bitstream including the point cloud data. The point cloud data reception method according to the embodiments can include the steps of receiving a bitstream including point cloud data, and decoding the point cloud data.

Description

TECHNICAL FIELD

[0001] Embodiments relate to a method and apparatus for processing point cloud content. BACKGROUND

[0002] Point cloud content is content represented by a point cloud, which is a set of points belonging to a coordinate system representing a three-dimensional space. The point cloud content can be expressed in a media configured in three dimensions, and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and self-driving services. However, tens of thousands to hundreds of thousands of point data are required to represent the point cloud content. Therefore, a method of efficiently processing a large amount of point data is required. SUMMARY

[0003] TECHNICAL PROBLEM

[0004] Embodiments provide an apparatus and method for efficiently processing point cloud data. Embodiments provide a point cloud data processing method and apparatus for solving latency and encoding / decoding complexity.

[0005] The technical scope of the embodiments is not limited to the above technical objects, and can be extended to other technical objects which can be inferred from the entire contents disclosed herein by those skilled in the art.

[0006] TECHNICAL SOLUTION

[0007] To achieve these objects and other advantages and according to the purpose of the present disclosure, as embodied and broadly described herein, a method for transmitting point cloud data can include encoding point cloud data, and transmitting a bitstream including the point cloud data.

[0008] In another aspect of the present disclosure, a method for receiving point cloud data can include receiving a bitstream including point cloud data, and decoding the point cloud data.

[0009] ADVANTAGEOUS EFFECTS

[0010] The apparatus and method according to the embodiments can efficiently process point cloud data.

[0011] The apparatus and method according to the embodiments can provide a high-quality point cloud service.

[0012] The apparatus and method according to the embodiments can provide point cloud content for providing general services such as a VR service and a self-driving service. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings, which are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this application, illustrate embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. In order to facilitate a fuller understanding of the various embodiments described herein, reference is made to the accompanying drawings. Like reference numerals are used to refer to like elements throughout the several views of the drawings. Among the drawings:

[0014] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown.

[0015] Figure 2 A block diagram showing point cloud content providing operations according to an embodiment is shown.

[0016] Figure 3 An exemplary process of capturing a point cloud video according to an embodiment is shown.

[0017] Figure 4 An exemplary point cloud encoder according to an embodiment is shown.

[0018] Figure 5 An example of a voxel according to an embodiment is shown.

[0019] Figure 6 An example of an octree and occupancy code according to an embodiment is shown.

[0020] Figure 7 An example of a neighbor node pattern according to an embodiment is shown.

[0021] Figure 8 An example of point configurations in various LODs according to an embodiment is shown.

[0022] Figure 9 An example of point configurations in various LODs according to an embodiment is shown.

[0023] Figure 10 A point cloud decoder according to an embodiment is shown.

[0024] Figure 11 A point cloud decoder according to an embodiment is shown.

[0025] Figure 12 A transmitting apparatus according to an embodiment is shown.

[0026] Figure 13 A receiving apparatus according to an embodiment is shown.

[0027] Figure 14 An exemplary structure operable in conjunction with the point cloud data transmitting / receiving method / apparatus according to an embodiment is shown.

[0028] Figure 15 A method for generating a Morton code according to an embodiment is shown.

[0029] Figure 16 A method for generating a Morton code according to an embodiment is shown.

[0030] Figure 17 A method for generating a Morton code according to an embodiment is shown.

[0031] Figure 18 A threshold for a Morton code generation order according to an embodiment is shown.

[0032] Figure 19 A Morton code generator according to an embodiment is shown.

[0033] Figure 20 A configuration of a point cloud bitstream according to an embodiment is shown.

[0034] Figure 21 A tile parameter set (TPS) according to an embodiment is shown.

[0035] Figure 22 A geometry parameter set (GPS) according to an embodiment is shown.

[0036] Figure 23 An attribute parameter set (APS) according to an embodiment is shown.

[0037] Figure 24 A geometry slice header (GSH) according to an embodiment is shown.

[0038] Figure 25 A method for transmitting point cloud data according to an embodiment is shown. And

[0039] Figure 26 A method for receiving point cloud data according to an embodiment is shown. DETAILED DESCRIPTION

[0040] Reference will now be made in detail embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The detailed description set forth below, in connection with the appended drawings, describes exemplary embodiments of the present disclosure and is not intended to represent the only embodiments contemplated by the present disclosure. The detailed description includes specific details for the purpose of providing a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0041] Although most of the terms used in the present disclosure are selected from general terms widely used in the art, some terms are arbitrarily selected by the applicant and the meaning thereof is described in detail according to necessity in the following description. Therefore, the present disclosure should be understood based on the intended meaning of the terms rather than their simple names or meanings.

[0042] Figure 1 An exemplary point cloud content providing system according to an embodiment is illustrated.

[0043] Figure 1 The illustrated point cloud content providing system can include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 can communicate in wired or wireless communication to transmit and receive point cloud data.

[0044] The point cloud data transmitting device 10000 according to an embodiment can acquire and process point cloud video (or point cloud content) and transmit the same. According to an embodiment, the transmitting device 10000 can include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device, and / or a server. According to an embodiment, the transmitting device 10000 can include a device configured to perform communication with a base station and / or other wireless devices using a radio access technology (e.g., 5G new RAT (NR), long term evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server.

[0045] The transmitting device 10000 according to an embodiment includes a point cloud video acquirer 10001, a point cloud video encoder 10002, and / or a transmitter (or communication module) 10003.

[0046] The point cloud video acquirer 10001 according to an embodiment acquires a point cloud video through a processing procedure such as capturing, synthesizing, or generating. The point cloud video is point cloud content represented by a point cloud, which is a set of points located in a 3D space, and can be referred to as point cloud video data. The point cloud video according to an embodiment can include one or more frames. One frame represents a still image / picture. Accordingly, the point cloud video can include point cloud images / frames / pictures, and can be referred to as a point cloud image, frame, or picture.

[0047] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 can encode the point cloud video data based on point cloud compression encoding. The point cloud compression encoding according to an embodiment can include geometry-based point cloud compression (G-PCC) encoding and / or video-based point cloud compression (V-PCC) encoding or next generation encoding. The point cloud compression encoding according to an embodiment is not limited to the above-described embodiments. The point cloud video encoder 10002 can output a bitstream containing the encoded point cloud video data. The bitstream can contain not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0048] The transmitter 10003 according to the embodiments transmits a bitstream containing the encoded point cloud video data. The bitstream according to the embodiments is encapsulated in a file or a segment (e.g., a streaming segment) and transmitted via various networks such as a broadcasting network and / or a broadband network. Although not shown in the drawing, the transmission device 10000 can include an encapsulator (or an encapsulation module) configured to perform an encapsulation operation. According to the embodiments, the encapsulator can be included in the transmitter 10003. According to the embodiments, the file or the segment can be transmitted to the reception device 10004 via a network or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitter 10003 according to the embodiments is capable of wired / wireless communication with the reception device 10004 (or the receiver 10005) via a 4G, 5G, 6G, etc. network. In addition, the transmitter can perform necessary data processing operations according to a network system (e.g., a 4G, 5G, or 6G communication network system). The transmission device 10000 can transmit the encapsulated data in an on-demand manner.

[0049] The reception device 10004 according to the embodiments includes the receiver 10005, the point cloud video decoder 10006, and / or the Tenderer 10007. According to the embodiments, the reception device 10004 can include a device, a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server configured to perform communication with a base station and / or other wireless devices using a radio access technology (e.g., 5G New RAT (NR), Long Term Evolution (LTE)).

[0050] The receiver 10005 according to the embodiments receives a bitstream containing point cloud video data or a file / segment in which the bitstream is encapsulated from a network or a storage medium. The receiver 10005 can perform necessary data processing according to a network system (e.g., a communication network system of 4G, 5G, 6G, etc.). The receiver 10005 according to the embodiments can decapsulate the received file / segment and output the bitstream. According to the embodiments, the receiver 10005 can include a decapsulator (or a decapsulation module) configured to perform a decapsulation operation. The decapsulator can be implemented as an element (or a component) separate from the receiver 10005.

[0051] The point cloud video decoder 10006 decodes the bitstream containing the point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to a method in which the point cloud video data is encoded (e.g., in reverse to the operation of the point cloud video encoder 10002). Accordingly, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding (reverse to point cloud compression). The point cloud decompression encoding includes G-PCC encoding.

[0052] The renderer 10007 renders the decoded point cloud video data. The renderer 10007 can output the point cloud content by rendering not only the point cloud video data but also audio data. According to an embodiment, the renderer 10007 can include a display configured to display the point cloud content. According to an embodiment, the display can be implemented as a separate device or component rather than being included in the renderer 10007.

[0053] The arrow indicated by the dotted line in the drawing represents a transmission path of feedback information acquired by the reception device 10004. The feedback information is information reflecting the interaction of a user consuming the point cloud content, and includes information about the user (e.g., head orientation information, viewport information, etc.). Specifically, when the point cloud content is content for a service requiring interaction with the user (e.g., a self-driving service, etc.), the feedback information can be provided to a content sender (e.g., the transmission device 10000) and / or a service provider. According to an embodiment, the feedback information can be used in the reception device 10004 as well as the transmission device 10000, or can not be provided.

[0054] The head orientation information according to an embodiment is information about the position, orientation, angle, motion, etc. of the head of the user. The reception device 10004 according to an embodiment can calculate the viewport information based on the head orientation information. The viewport information can be information about the region of the point cloud video that the user is watching. The viewpoint is a point through which the user watches the point cloud video, and can refer to the center point of the viewport region. That is, the viewport is a region centered on the viewpoint, and the size and shape of the region can be determined by the field of view (FOV). Thus, in addition to the head orientation information, the reception device 10004 can extract the viewport information based on the vertical or horizontal FOV supported by the device. In addition, the reception device 10004 performs gaze analysis, etc. to check the way in which the user consumes the point cloud, the region in which the user gazes in the point cloud video, the gaze time, etc. According to an embodiment, the reception device 10004 can transmit feedback information including the gaze analysis result to the transmission device 10000. The feedback information according to an embodiment can be acquired in the process of rendering and / or display. The feedback information according to an embodiment can be taken by one or more sensors included in the reception device 10004. According to an embodiment, the feedback information can be taken by the renderer 10007 or a separate external element (or device, component, etc.). Figure 1The dotted line in the point cloud content providing system indicates a process of transmitting feedback information acquired by the rendering device 10007. The point cloud content providing system can process (encode / decode) point cloud data based on the feedback information. Accordingly, the point cloud video decoder 10006 can perform a decoding operation based on the feedback information. The reception device 10004 can transmit the feedback information to the transmission device 10000. The transmission device 10000 (or the point cloud video data encoder 10002) can perform an encoding operation based on the feedback information. Accordingly, the point cloud content providing system can efficiently process necessary data (e.g., point cloud data corresponding to a head position of a user) based on the feedback information, rather than processing (encoding / decoding) the entire point cloud data, and provide point cloud content to the user.

[0055] According to embodiments, the transmission device 10000 can be referred to as an encoder, a transmission device, a transmitter, or the like, and the reception device 10004 can be referred to as a decoder, a reception device, a receiver, or the like.

[0056] According to embodiments, the point cloud content providing system Figure 1 The point cloud data processed in the point cloud content providing system (through a series of processes of acquisition / encoding / transmission / decoding / rendering) according to embodiments can be referred to as point cloud content data or point cloud video data. According to embodiments, the point cloud content data can be used as a concept that encompasses metadata or signaling information related to point cloud data.

[0057] Figure 1 The elements of the point cloud content providing system shown can be implemented by hardware, software, a processor, and / or a combination thereof.

[0058] Figure 2 is a block diagram illustrating a point cloud content providing operation according to embodiments.

[0059] Figure 2 The block diagram of FIG. 1 illustrates Figure 1 The operation of the point cloud content providing system described in FIG. 1. As described above, the point cloud content providing system can process point cloud data based on point cloud compression encoding (e.g., G-PCC).

[0060] A point cloud content providing system according to an embodiment (e.g., point cloud sending device 10000 or point cloud video acquirer 10001) can acquire point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system used to represent 3D space. The point cloud video according to an embodiment may include Ply (Polygon file format or Stanford Triangle format) files. When the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of the points. The position of each point may be represented by parameters (e.g., values ​​of the X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include the attributes of the points (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc., of each point). A point has one or more attributes. For example, a point may have a color attribute or two attributes: color and reflectivity. According to the implementation, geometry can be referred to as location, geometric information, geometric data, etc., and attributes can be referred to as attributes, attribute information, attribute data, etc. A point cloud content providing system (e.g., point cloud sending device 10000 or point cloud video acquirer 10001) can acquire point cloud data from information related to the point cloud video acquisition process (e.g., depth information, color information, etc.).

[0061] A point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) according to an embodiment can encode point cloud data (20001). The point cloud content providing system can encode point cloud data based on point cloud compression encoding. As described above, point cloud data can include the geometry and attributes of points. Therefore, the point cloud content providing system can perform geometry encoding to encode the geometry and output a geometry bitstream. The point cloud content providing system can perform attribute encoding to encode the attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system can perform attribute encoding based on geometry encoding. The geometry bitstream and attribute bitstream according to an embodiment can be multiplexed and output as a single bitstream. The bitstream according to an embodiment may also contain signaling information related to geometry encoding and attribute encoding.

[0062] A point cloud content providing system according to an embodiment (e.g., transmitting device 10000 or transmitter 10003) can transmit encoded point cloud data (20002). Figure 1 As shown, encoded point cloud data can be represented by geometric bitstreams and attribute bitstreams. Additionally, the encoded point cloud data can be transmitted as a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometric encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and transmit it as a file or fragment.

[0063] A point cloud content providing system (e.g., reception device 10004 or receiver 10005) according to an embodiment can receive a bitstream including encoded point cloud data. Also, the point cloud content providing system (e.g., reception device 10004 or receiver 10005) can demultiplex the bitstream.

[0064] A point cloud content providing system (e.g., reception device 10004 or point cloud video decoder 10005) can decode encoded point cloud data (e.g., geometry bitstream, attribute bitstream) transmitted in the bitstream. The point cloud content providing system (e.g., reception device 10004 or point cloud video decoder 10005) can decode point cloud video data based on signaling information included in the bitstream regarding encoding of the point cloud video data. The point cloud content providing system (e.g., reception device 10004 or point cloud video decoder 10005) can decode the geometry bitstream to reconstruct positions (geometry) of points. The point cloud content providing system can reconstruct attributes of points by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., reception device 10004 or point cloud video decoder 10005) can reconstruct a point cloud video based on positions from the reconstructed geometry and the decoded attributes.

[0065] A point cloud content providing system (e.g., reception device 10004 or renderer 10007) according to an embodiment can render decoded point cloud data (20004). The point cloud content providing system (e.g., reception device 10004 or renderer 10007) can render geometry and attributes decoded through a decoding process using various rendering methods. Points in the point cloud content can be rendered as a vertex having a certain thickness, a cube having a certain minimum size centered on a corresponding vertex position, or a circle centered on a corresponding vertex position. All or part of the rendered point cloud content is provided to a user through a display (e.g., VR / AR display, general display, etc.).

[0066] A point cloud content providing system (e.g., reception device 10004) according to an embodiment can obtain feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. The feedback information and operations of the point cloud content providing system according to an embodiment are the same as those described with reference to FIGS. 1 to 6, and thus a detailed description thereof is omitted. Figure 1 The feedback information and operations described are the same as those described with reference to FIGS. 1 to 6, and thus a detailed description thereof is omitted.

[0067] Figure 3 An exemplary process of capturing a point cloud video according to an embodiment is shown.

[0068] Figure 3 An exemplary point cloud video capturing process of the point cloud content providing system described with reference to FIGS. 1 to 6 is shown. Figure 1-2 An exemplary point cloud video capturing process of the point cloud content providing system described with reference to FIGS. 1 to 6 is shown.

[0069] Point cloud content includes point cloud videos (images and / or videos) representing objects and / or environments located in various 3D spaces (e.g., a 3D space representing a real environment, a 3D space representing a virtual environment, etc.). Accordingly, the point cloud content providing system according to embodiments can capture point cloud videos using one or more cameras (e.g., an infrared camera capable of acquiring depth information, an RGB camera capable of extracting color information corresponding to depth information, etc.), a projector (e.g., an infrared pattern projector acquiring depth information), a LiDAR, etc. The point cloud content providing system according to embodiments can extract a geometry consisting of points in a 3D space from depth information and extract attributes of respective points from color information to acquire point cloud data. The images and / or videos according to embodiments can be captured based on at least one of an inside-out technique and an outside-in technique.

[0070] Figure 3 The left portion of FIG. 1 illustrates an inside-out technique. The inside-out technique refers to a technique of capturing an image of a central object using one or more cameras (or camera sensors) positioned around the central object. The inside-out technique can be used to generate point cloud content providing a 360-degree image of a key object to a user (e.g., VR / AR content providing a 360-degree image of an object (e.g., a key object such as a character, a player, an object, or an actor) to a user).

[0071] Figure 3 The right portion of FIG. 1 illustrates an outside-in technique. The outside-in technique refers to a technique of capturing an image of an environment of a central object, rather than the central object, using one or more cameras (or camera sensors) positioned around the central object. The outside-in technique can be used to generate point cloud content providing a surrounding environment appearing from a user's perspective (e.g., content representing an external environment that can be provided to a user of a self-driving vehicle).

[0072] As illustrated, point cloud content can be generated based on a capturing operation of one or more cameras. In this case, coordinate systems can be different between cameras, and thus the point cloud content providing system can calibrate the one or more cameras to set a global coordinate system before the capturing operation. In addition, the point cloud content providing system can generate point cloud content by synthesizing an arbitrary image and / or video with images and / or videos captured through the above-described capturing techniques. The point cloud content providing system can not perform the capturing operation described in FIG. 1 when generating point cloud content representing a virtual space. The point cloud content providing system according to embodiments can perform post-processing on captured images and / or videos. In other words, the point cloud content providing system can remove unwanted areas (e.g., a background), identify a space to which a captured image and / or video is connected, and perform an operation of filling a space hole when there is a space hole. Figure 3

[0073] ​The point cloud content providing system can generate a piece of point cloud content by performing coordinate transformation on points of a point cloud video taken from each camera. The point cloud content providing system can perform coordinate transformation on points based on position coordinates of each camera. Accordingly, the point cloud content providing system can generate a piece of point cloud content representing a wide range of content, or can generate a piece of point cloud content having a high density of points.

[0074] Figure 4 An exemplary point cloud encoder according to an embodiment is illustrated.

[0075] Figure 4 An example of a point cloud video encoder 10002 is illustrated. Figure 1 The point cloud encoder reconstructs and encodes point cloud data (e.g., positions and / or attributes of points) to adjust a quality of point cloud content (e.g., lossless, lossy, or near lossless) according to network conditions or applications. When a total size of point cloud content is large (e.g., 60 Gbps of point cloud content is given for 30 fps), the point cloud content providing system can not be able to stream the content in real time. Accordingly, the point cloud content providing system can reconstruct point cloud content based on a maximum target bitrate to provide point cloud content according to network environments, etc.

[0076] As described with reference to Figure 1-2 The point cloud encoder can perform geometry encoding and attribute encoding. Geometry encoding is performed before attribute encoding.

[0077] A point cloud encoder according to an embodiment includes a coordinate transformer (transform coordinates) 40000, a quantizer (quantize and remove points (voxelize)) 40001, an octree analyzer (analyze octree) 40002 and a surface approximation analyzer (analyze surface approximation) 40003, an arithmetic encoder (arithmetic encode) 40004, a geometry reconstructor (reconstruct geometry) 40005, a color transformer (transform color) 40006, an attribute transformer (transform attribute) 40007, a RAHT transformer (RAHT) 40008, a LOD generator (generate LOD) 40009, a lifting transformer (lift) 40010, a coefficient quantizer (quantize coefficients) 40011, and / or an arithmetic encoder (arithmetic encode) 40012.

[0078] The coordinate transformer 40000, the quantizer 40001, the octree analyzer 40002, the surface approximation analyzer 40003, the arithmetic encoder 40004, and the geometry reconstructor 40005 can perform geometry encoding. Geometry encoding according to an embodiment can include octree geometry encoding, direct encoding, triplet geometry encoding, and entropy encoding. Direct encoding and triplet geometry encoding are selectively or in combination applied. Geometry encoding is not limited to the above-described examples.

[0079] As illustrated, the coordinate converter 40000 according to the embodiment receives a position and converts it into a coordinate. For example, the position can be converted into position information in a three-dimensional space (for example, a three-dimensional space represented by an XYZ coordinate system). The position information in the three-dimensional space according to the embodiment can be referred to as geometry information.

[0080] The quantizer 40001 according to the embodiment quantizes geometry. For example, the quantizer 40001 can quantize points based on minimum position values (for example, minimum values on each of X, Y, and Z axes) of all points. The quantizer 40001 performs a quantization operation of multiplying a difference between the minimum position value and a position value of each point by a preset quantization scaling value, and then finding a nearest integer value by rounding the value obtained through the multiplication. Accordingly, one or more points can have the same quantized position (or position value). The quantizer 40001 according to the embodiment performs voxelization based on the quantized position to reconstruct quantized points. As in the case of a pixel (a smallest unit containing 2D image / video information), a point of a point cloud content (or a 3D point cloud video) according to the embodiment can be included in one or more voxels. As a compound of a volume and a pixel, the term voxel refers to a 3D cubic space generated when a 3D space is divided into units (unit = 1.0) based on axes (for example, X, Y, and Z axes) representing the 3D space. The quantizer 40001 can match a group of points in the 3D space with voxels. According to the embodiment, one voxel can include only one point. According to the embodiment, one voxel can include one or more points. In order to represent one voxel as one point, a position of a center of the voxel can be set based on positions of one or more points included in the voxel. In this case, attributes of all positions included in one voxel can be combined and assigned to the voxel.

[0081] The octree analyzer 40002 according to the embodiment performs octree geometry encoding (or octree encoding) to present voxels in an octree structure. The octree structure represents points matched with voxels based on an octal tree structure.

[0082] The surface approximation analyzer 40003 according to the embodiment can analyze and approximate an octree. The octree analysis and approximation according to the embodiment is a process of analyzing a region containing a plurality of points to efficiently provide an octree and voxelization.

[0083] The arithmetic encoder 40004 according to the embodiment performs entropy encoding on the octree and / or the approximated octree. For example, an encoding scheme includes arithmetic encoding. As a result of encoding, a geometry bitstream is generated.

[0084] The color transformer 40006, the attribute transformer 40007, the RAHT transformer 40008, the LOD generator 40009, the lifting transformer 40010, the coefficient quantizer 40011, and / or the arithmetic encoder 40012 perform attribute encoding. As described above, one point can have one or more attributes. Attribute encoding according to embodiments is equally applied to attributes that one point has. However, when an attribute (e.g., color) includes one or more elements, attribute encoding is independently applied to each element. Attribute encoding according to embodiments includes color transform encoding, attribute transform encoding, region-adaptive hierarchical transform (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) encoding, and interpolation-based hierarchical nearest neighbor prediction with update / lifting step (lifting transform) encoding. According to the content of a point cloud, the above-described RAHT encoding, prediction transform encoding, and lifting transform encoding can be selectively used, or a combination of one or more encoding schemes can be used. Attribute encoding according to embodiments is not limited to the above-described examples.

[0085] The color transformer 40006 according to embodiments performs color transform encoding that transforms a color value (or texture) included in an attribute. For example, the color transformer 40006 can transform the format of color information (e.g., from RGB to YCbCr). Alternatively, the operation of the color transformer 40006 according to embodiments can be applied according to a color value included in an attribute.

[0086] The geometry reconstructor 40005 according to embodiments reconstructs (decompresses) an octree and / or an approximate octree. The geometry reconstructor 40005 reconstructs an octree / voxel based on the result of analyzing the distribution of points. The reconstructed octree / voxel can be referred to as reconstructed geometry (restored geometry).

[0087] The attribute transformer 40007 according to embodiments performs attribute transform to transform an attribute based on reconstructed geometry and / or a position for which geometry encoding is not performed. As described above, since an attribute depends on geometry, the attribute transformer 40007 can transform an attribute based on reconstructed geometry information. For example, based on a position value of a point included in a voxel, the attribute transformer 40007 can transform the attribute of the point at the position. As described above, when a central position of a voxel is set based on positions of one or more points included in the voxel, the attribute transformer 40007 transforms the attributes of the one or more points. When triplet geometry encoding is performed, the attribute transformer 40007 can transform an attribute based on triplet geometry encoding.

[0088] The attribute transformer 40007 can perform attribute transformation by calculating an average of attributes or attribute values (e.g., colors or reflectances of respective points) of neighboring points within a certain position / radius from a center position (or position value) of each voxel. The attribute transformer 40007 can apply a weight according to a distance from the center to each point when calculating the average. Accordingly, each voxel has a position and a calculated attribute (or attribute value).

[0089] The attribute transformer 40007 can search for neighboring points existing within a certain position / radius from a center position of each voxel based on a K-D tree or a Morton code. The K-D tree is a binary search tree and supports a data structure capable of managing points based on positions, so that a nearest neighbor search (NNS) can be quickly performed. The Morton code is generated by presenting coordinates (e.g., (x, y, z)) representing 3D positions of all points as bit values and mixing the bits. For example, when the coordinates representing the positions of points are (5, 9, 1), the bit values of the coordinates are (0101, 1001, 0001). Mixing the bit values according to bit indexes in the order of z, y, and x results in 010001000111. This value is represented as a decimal number 1095. That is, the Morton code value of the point having the coordinates (5, 9, 1) is 1095. The attribute transformer 40007 can sort points based on the Morton code values and perform NNS through a depth-first traversal process. The K-D tree or the Morton code is used when NNS is required in another transformation process for attribute encoding after the attribute transformation operation.

[0090] As illustrated, the transformed attribute is input to the RAHT transformer 40008 and / or the LOD generator 40009.

[0091] The RAHT transformer 40008 according to an embodiment performs RAHT encoding for predicting attribute information based on reconstructed geometry information. For example, the RAHT transformer 40008 can predict attribute information of a node at a higher level in an octree based on attribute information associated with a node at a lower level in the octree.

[0092] The LOD generator 40009 according to an embodiment generates a level of detail (LOD) to perform predictive transform encoding. The LOD according to an embodiment is a degree of detail of point cloud content. As the LOD value decreases, the detail of the point cloud content is indicated to deteriorate. As the LOD value increases, the detail of the point cloud content is indicated to enhance. Points can be classified by LOD.

[0093] The lifting transformer 40010 according to an embodiment performs lifting transform encoding that transforms point cloud attributes based on weights. As described above, the lifting transform encoding can be optionally applied.

[0094] The coefficient quantizer 40011 according to an embodiment quantizes attributes of attribute encoding based on coefficients.

[0095] The arithmetic encoder 40012 according to the embodiment encodes the quantized attribute based on arithmetic encoding.

[0096] Although not shown in the drawings, Figure 4 The elements of the point cloud encoder can be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in a point cloud providing apparatus. The one or more processors can perform at least one of the operations and / or functions of the elements of the point cloud encoder described above. Figure 4 In addition, the one or more processors can operate or execute a software program and / or a set of instructions for performing the operations and / or functions of the elements of the point cloud encoder described above. The one or more memories according to the embodiment can include a high-speed random access memory, or include a non-volatile memory (e.g., one or more disk storage devices, flash memory devices, or other non-volatile solid state memory devices). Figure 4

[0097] Figure 5 An example of a voxel according to the embodiment is illustrated.

[0098] Figure 5 A voxel located in a 3D space represented by a coordinate system composed of three axes (X-axis, Y-axis, and Z-axis) is illustrated. As described with reference to Figure 4 The point cloud encoder (e.g., quantizer 40001) can perform voxelization as described with reference to. A voxel refers to a 3D cubic space generated when a 3D space is divided into units (unit = 1.0) based on axes (e.g., X-axis, Y-axis, and Z-axis) representing the 3D space. Figure 5 An example of a voxel generated through an octree structure in which a cubic axis-aligned bounding box defined by two poles (0, 0, 0) and (2d, 2d, 2d) is recursively subdivided is illustrated. One voxel includes at least one point. The spatial coordinates of the voxel can be estimated from the positional relationship with a group of voxels. As described above, a voxel has attributes (e.g., color or reflectance) similar to those of a pixel of a 2D image / video. Details of the voxel are the same as those described with reference to Figure 4 , and thus a description thereof is omitted.

[0099] Figure 6 An example of an octree and an occupancy code according to the embodiment is illustrated.

[0100] As described with reference to Figure 1-4 , the point cloud content providing system (point cloud video encoder 10002) or the point cloud encoder (e.g., octree analyzer 40002) performs octree geometry encoding (or octree encoding) based on an octree structure to efficiently manage the area and / or position of a voxel.

[0101] ​Figure 6 The upper part shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes of a coordinate system (e.g., the X, Y, and Z axes). This is achieved by using two poles (0,0,0) and (2... d ,2 d ,2 d An octree structure is created by recursively subdividing the bounding box with the cubic axis aligned to the bounding box. Here, 2 d This can be set to the value of the minimum bounding box that constitutes all points surrounding the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following formula. In the following formula, (x int n ,y int n ,z int n ) indicates the position (or position value) of the quantized point.

[0102]

[0103] like Figure 6 As shown in the upper center, the entire 3D space can be divided into eight spaces according to partitions. Each partitioned space is represented by a cube with six faces. For example... Figure 6 As shown in the upper right, each of the eight spaces is further subdivided based on a coordinate system axis (e.g., the X, Y, and Z axes). Thus, each space is divided into eight smaller spaces. These smaller spaces are also represented by cubes with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0104] Figure 6 The lower part shows the octree occupancy code. The occupancy code generates the octree to indicate whether each of the eight partitions generated by dividing a space contains at least one node. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of a partitioned space, and each child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to a child node contains at least one node, the node is assigned a value of 1. When the space corresponding to a child node does not contain a node (the space is empty), the node is assigned a value of 0. Since... Figure 6The shown occupancy code is 00100001, so it indicates that each of the spaces corresponding to the third and eighth child nodes among the eight child nodes contains at least one point. As shown, each of the third and eighth child nodes has eight child nodes, and the child nodes are represented by 8-bit occupancy codes. The accompanying figure shows that the occupancy code of the third child node is 10000111, and the occupancy code of the eighth child node is 01001111. The point cloud encoder (e.g., the arithmetic encoder 40004) according to the embodiments can perform entropy encoding on the occupancy codes. To increase compression efficiency, the point cloud encoder can perform intra / inter-frame encoding on the occupancy codes. The receiving device (e.g., the receiving device 10004 or the point cloud video decoder 10006) according to the embodiments reconstructs the octree based on the occupancy codes.

[0105] The point cloud encoder (e.g., the arithmetic encoder 40004) according to the embodiments can perform voxelization and octree encoding to store the point positions. However, points are not always uniformly distributed in the 3D space, so there can be a certain region where fewer points exist. Therefore, it is inefficient to perform voxelization on the entire 3D space. For example, when a certain region contains very few points, voxelization does not need to be performed in the certain region. Figure 4

[0106] Therefore, for the above certain region (or a node other than a leaf node of the octree), the point cloud encoder according to the embodiments can skip voxelization and perform direct encoding to directly encode the point positions included in the certain region. The coordinates of the directly encoded points according to the embodiments are referred to as direct coding mode (DCM). The point cloud encoder according to the embodiments can also perform triplet geometry encoding based on a surface model, which is to reconstruct the point positions in the certain region (or node) based on voxels. Triplet geometry encoding is a geometry encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. The direct encoding and the triplet geometry encoding according to the embodiments can be selectively performed. In addition, the direct encoding and the triplet geometry encoding according to the embodiments can be performed in combination with the octree geometry encoding (or octree encoding).

[0107] To perform direct encoding, an option to apply direct encoding using a direct mode should be enabled. The node to which direct encoding is to be applied is not a leaf node, and there should be fewer points than a threshold within the certain node. In addition, the total number of points to which direct encoding is to be applied should not exceed a preset threshold. When the above conditions are satisfied, the point cloud encoder (or the arithmetic encoder 40004) according to the embodiments can perform entropy encoding on the point positions (or position values).

[0108] ​The point cloud encoder (e.g., the surface approximation analyzer 40003) according to the embodiments can determine a certain level of the octree (a level less than the depth d of the octree), and can start using the surface model from the level to perform the triplet geometry encoding to reconstruct the point positions in the node region based on the voxels (triplet mode). The point cloud encoder according to the embodiments can designate the level to which the triplet geometry encoding is to be applied. For example, when the certain level is equal to the depth of the octree, the point cloud encoder does not operate in the triplet mode. In other words, only when the designated level is less than the depth value of the octree, the point cloud encoder according to the embodiments can operate in the triplet mode. The 3D cubical region of the node designated according to the embodiments is referred to as a block. One block can include one or more voxels. The block or the voxel can correspond to a tile. The geometry is represented as a surface within each block. The surface according to the embodiments can intersect with each edge of the block at most once.

[0109] One block has 12 edges, and thus there are at least 12 intersection points in one block. Each intersection point is referred to as a vertex. When there is at least one occupied voxel adjacent to an edge among all the blocks sharing the edge, a vertex existing along the edge is detected. The occupied voxel according to the embodiments refers to a voxel including a point. The vertex position detected along the edge is an average position of the edges of all the voxels adjacent to the edge among all the blocks sharing the edge.

[0110] Once the vertex is detected, the point cloud encoder according to the embodiments can perform entropy encoding on the origin of the edge (x, y, z), the direction vector of the edge (Δx, Δy, Δz), and the vertex position value (a relative position value within the edge). When the triplet geometry encoding is applied, the point cloud encoder (e.g., the geometry reconstructor 40005) according to the embodiments can generate the restored geometry (reconstructed geometry) by performing a triangle reconstruction, an upsampling, and a voxelization process.

[0111] The vertex located at the edge of the block determines a surface passing through the block. The surface according to the embodiments is a non-planar polygon. In the triangle reconstruction process, the surface represented by a triangle is reconstructed based on the origin of the edge, the direction vector of the edge, and the position value of the vertex. The triangle reconstruction process is performed as follows: i) a centroid value of each vertex is calculated, ii) the centroid value is subtracted from each vertex value, and iii) a square sum of the values obtained by the subtraction is estimated.

[0112]

[0113] The minimum of the sum is estimated and the projection process is performed according to the axis with the minimum. For example, when the element x is the smallest, the respective vertices are projected on the x-axis with respect to the center of the block and on the (y, z) plane. When the values obtained by the projection on the (y, z) plane are (ai, bi), the value of θ is estimated by atan2(bi, ai) and the vertices are ordered based on the value of θ. The following table shows the vertex combinations creating triangles according to the number of vertices. The vertices are ordered from 1 to n. The following table shows that for four vertices, two triangles can be constructed according to the vertex combinations. The first triangle can consist of vertices 1, 2 and 3 among the ordered vertices, and the second triangle can consist of vertices 3, 4 and 1 among the ordered vertices.

[0114] Table 2-1. Triangles formed from vertices ordered by 1, …, n

[0115] n triangles

[0116] 3 (1, 2, 3)

[0117] 4 (1, 2, 3), (3, 4, 1)

[0118] 5 (1, 2, 3), (3, 4, 5), (5, 1, 3)

[0119] 6 (1, 2, 3), (3, 4, 5), (5, 6, 1), (1, 3, 5)

[0120] 7 (1, 2, 3), (3, 4, 5), (5, 6, 7), (7, 1, 3), (3, 5, 7)

[0121] 8 (1, 2, 3), (3, 4, 5), (5, 6, 7), (7, 8, 1), (1, 3, 5), (5, 7, 1)

[0122] 9 (1, 2, 3), (3, 4, 5), (5, 6, 7), (7, 8, 9), (9, 1, 3), (3, 5, 7), (7, 9, 3)

[0123] 10 (1, 2, 3), (3, 4, 5), (5, 6, 7), (7, 8, 9), (9, 10, 1), (1, 3, 5), (5, 7, 9), (9, 1, 5)

[0124] 11 (1, 2, 3), (3, 4, 5), (5, 6, 7), (7, 8, 9), (9, 10, 11), (11, 1, 3), (3, 5, 7), (7, 9, 11), (11, 3, 7)

[0125] 12 (1, 2, 3), (3, 4, 5), (5, 6, 7), (7, 8, 9), (9, 10, 11), (11, 12, 1), (1, 3, 5), (5, 7, 9), (9, 11, 1), (1, 5, 9)

[0126] An upsampling process is performed to add points in the middle along the edges of the triangle, and voxelization is performed. The added points are generated based on an upsampling factor and the width of the block. The added points are referred to as refinement vertices. The point cloud encoder according to the embodiment can voxelize the refinement vertices. In addition, the point cloud encoder can perform attribute encoding based on the positions (or position values) of the voxelization.

[0127] Figure 7 An example of a neighbor node pattern according to the embodiment is illustrated.

[0128] To increase the compression efficiency of the point cloud video, the point cloud encoder according to the embodiment can perform entropy encoding based on context adaptive arithmetic encoding.

[0129] As described with reference to Figure 1-6 , the point cloud content providing system or the point cloud encoder (e.g., the point cloud video encoder 10002, Figure 4 , or the arithmetic encoder 40004 of the point cloud encoder) can immediately perform entropy encoding on the occupancy code. In addition, the point cloud content providing system or the point cloud encoder can perform entropy encoding (intra encoding) based on the occupancy code of the current node and the occupancy of the neighboring nodes, or perform entropy encoding (inter encoding) based on the occupancy code of the previous frame. The frame according to the embodiment represents a set of simultaneously generated point cloud videos. The compression efficiency of the intra encoding / inter encoding according to the embodiment can depend on the number of the neighboring nodes referred to. When the bits increase, the operation becomes complex, but the encoding can be biased to one side, which can increase the compression efficiency. For example, when 3 bits of context are given, 2 3 = 8 methods need to be used to perform the encoding. The portion divided for encoding affects the implementation complexity. Therefore, it is necessary to satisfy an appropriate level of compression efficiency and complexity.

[0130] Figure 7 A process of obtaining an occupancy pattern based on the occupancy of the neighbor nodes is illustrated. The point cloud encoder according to the embodiment determines the occupancy of the neighbor nodes of each node of the octree and obtains the value of the neighbor pattern. The neighbor node pattern is used to infer the occupancy pattern of the node. Figure 7 The upper part of FIG. 4 illustrates a cube (a cube located in the middle) corresponding to a node and six cubes (neighbor nodes) sharing at least one face with the cube. The node illustrated in the figure is a node of the same depth. The numbers illustrated in the figure respectively represent the weights (1, 2, 4, 8, 16, and 32) associated with the six nodes. The weights are sequentially assigned according to the positions of the neighboring nodes.

[0131] Figure 7 The lower part of the figure shows the neighbor node pattern value. The neighbor node pattern value is the sum of the values multiplied by the weight of the occupied neighbor node (neighbor node with a dot). Thus, the neighbor node pattern value is 0 to 63. When the neighbor node pattern value is 0, it indicates that there is no node with a dot among the neighbor nodes of the node (no occupied node). When the neighbor node pattern value is 63, it indicates that all neighbor nodes are occupied nodes. As shown in the figure, since the neighbor nodes assigned with the weights 1, 2, 4, and 8 are occupied nodes, the neighbor node pattern value is 15 (sum of 1, 2, 4, and 8). The point cloud encoder can perform encoding according to the neighbor node pattern value (for example, when the neighbor node pattern value is 63, 64 types of encoding can be performed). According to the embodiment, the point cloud encoder can reduce the encoding complexity by changing the neighbor node pattern value (for example, based on a table that changes 64 to 10 or 6).

[0132] Figure 8 An example of point configuration in each LOD according to the embodiment is shown.

[0133] As described with reference to Figure 1-7 , the encoded geometry is reconstructed (decompressed) before performing attribute encoding. When direct encoding is applied, the geometry reconstruction operation can include changing the placement of the directly encoded points (for example, placing the directly encoded points in front of the point cloud data). When triad geometry encoding is applied, the geometry reconstruction process is performed by triangle reconstruction, upsampling, and voxelization. Since the attribute depends on the geometry, the attribute encoding is performed based on the reconstructed geometry.

[0134] The point cloud encoder (for example, the LOD generator 40009) can classify (reorganize) the points by LOD. The figure shows the point cloud content corresponding to the LOD. The leftmost picture in the figure indicates the original point cloud content. The second picture from the left in the figure indicates the point distribution in the lowest LOD, and the rightmost picture in the figure indicates the point distribution in the highest LOD. That is, the points are sparsely distributed in the lowest LOD, and the points are densely distributed in the highest LOD. That is, as the LOD increases in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between the points becomes narrower.

[0135] Figure 9 An example of point configuration for each LOD according to the embodiment is shown.

[0136] As described with reference to Figure 1-8 , the point cloud content providing system or the point cloud encoder (for example, the point cloud video encoder 10002, Figure 4A point cloud encoder or LOD generator (40009) can generate LODs. LODs are generated by reorganizing points into a set of refined levels based on a set of LOD distance values ​​(or a set of Euclidean distances). The LOD generation process is performed not only by the point cloud encoder but also by the point cloud decoder.

[0137] Figure 9 The upper part shows examples of points (P0 to P9) of point cloud content distributed in 3D space. Figure 9 In this context, the original order represents the order of points P0 to P9 before LOD generation. Figure 9 In this context, LOD-based order represents the order in which points are generated according to their LOD. Points are reorganized by LOD. Additionally, higher LODs include points belonging to lower LODs. For example... Figure 9 As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, the points of LOD1, P9, P8, and P7.

[0138] For reference Figure 4 As described, the point cloud encoder according to the implementation can selectively or in combination perform predictive transform coding, lifting transform coding, and RAHT transform coding.

[0139] The point cloud encoder according to the embodiment can generate predictors for points to perform predictive transformation encoding for setting the predictive attributes (or predictive attribute values) of each point. That is, N predictors can be generated for N points. The predictors according to the embodiment can calculate weights (=1 / distance) based on the LOD value of each point, index information of neighboring points existing within a set distance of each LOD, and the distance to the neighboring points.

[0140] According to the implementation, the predicted attribute (or attribute value) is set as the average of the values ​​obtained by multiplying the attributes (or attribute values) of neighboring points (e.g., color, reflectivity, etc.) set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 40011) according to the implementation can quantize and inverse quantize the residuals (which may be referred to as residual attributes, residual attribute values, or attribute prediction residuals) obtained by subtracting the predicted attribute (attribute value) from the attributes (attribute values) of each point. The quantization process is configured as shown in the table below.

[0141] Table. Pseudocode for Attribute Prediction Residual Quantization

[0142] int PCCQuantization(int value,int quantStep){

[0143] if(value>=0){

[0144] return floor(value / quantStep+1.0 / 3.0);

[0145] }else{

[0146] return-floor(-value / quantStep+1.0 / 3.0);

[0147] }

[0148] }

[0149] Table. Attribute prediction residual inverse quantization pseudo code

[0150] int PCCInverseQuantization(int value, int quantStep){

[0151] if(quantStep==0){

[0152] return value;

[0153] }else{

[0154] return value*quantStep;

[0155] }

[0156] }

[0157] When the predictor of each point has neighbor points, the point cloud encoder (e.g., the arithmetic encoder 40012) according to the embodiment can perform entropy encoding on the quantized and inverse quantized residual values as described above. When the predictor of each point has no neighbor points, the point cloud encoder (e.g., the arithmetic encoder 40012) according to the embodiment can perform entropy encoding on the attribute of the corresponding point without performing the above-described operation.

[0158] The point cloud encoder (e.g., the lifting transformer 40010) according to the embodiment can generate the predictor of each point, set the calculated LOD and register the neighbor points in the predictor, and set the weight according to the distance to the neighbor points to perform the lifting transform encoding. The lifting transform encoding according to the embodiment is similar to the prediction transform encoding described above, but differs in that the weight is applied to the attribute value cumulatively. The process of applying the weight to the attribute value cumulatively according to the embodiment is configured as follows.

[0159] 1) Create an array quantized weight (QW) for storing the weight value of each point. The initial value of all elements of QW is 1.0. The QW value of the predictor index of the neighbor node registered in the predictor is multiplied by the weight of the predictor of the current point, and the value obtained by multiplication is added.

[0160] 2) Lift prediction process: subtract a value obtained by multiplying an attribute value of a point by a weight from an existing attribute value to calculate a predicted attribute value.

[0161] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0162] 4) Accumulate a weight calculated by multiplying a weight calculated for all predictors by a weight stored in QW corresponding to a predictor index to the updateweight array as an index of a neighbor node. Accumulate a value obtained by multiplying an attribute value of a neighbor node index by the calculated weight to the update array.

[0163] 5) Lift update process: divide attribute values of the update array of all predictors by weight values of the updateweight array of a predictor index, and add existing attribute values to values obtained by the division.

[0164] 6) Calculate a predicted attribute by multiplying an attribute value updated by the lift update process by a weight (stored in QW) updated by the lift prediction process for all predictors. A point cloud encoder (e.g., the coefficient quantizer 40011) according to the embodiment quantizes the predicted attribute value. In addition, a point cloud encoder (e.g., the arithmetic encoder 40012) performs entropy encoding on the quantized attribute value.

[0165] A point cloud encoder (e.g., the RAHT transformer 40008) according to the embodiment can perform RAHT transform encoding in which attributes associated with nodes of a lower level in an octree are used to predict attributes of nodes of a higher level. The RAHT transform encoding is an example of attribute intra-frame encoding by octree backward scanning. A point cloud encoder according to the embodiment scans an entire region from a voxel and repeatedly performs a merging process of merging voxels into larger blocks at each step until reaching a root node. The merging process according to the embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on an upper node directly above an empty node.

[0166] The following equation represents a RAHT transform matrix. In the equation, represents an average attribute value of voxels of level l. may be calculated based on and and The weight of and

[0167]

[0168] Here,​ It is a low-pass value and is used in the next higher level of merging. This represents the high-pass coefficient. The high-pass coefficient at each step is quantized and subjected to entropy encoding (e.g., encoded by an arithmetic encoder 400012). Weights are calculated as follows: pass and Create the root node as follows.

[0169]

[0170] Figure 10 A point cloud decoder according to an embodiment is shown.

[0171] Figure 10 The point cloud decoder shown is Figure 1 The example of the point cloud video decoder 10006 described in [the document], and it can be executed with [the following]. Figure 1 The operation of the point cloud video decoder 10006 shown is the same or similar. As shown, the point cloud decoder can receive a geometry bitstream and an attribute bitstream contained in one or more bitstreams. The point cloud decoder includes a geometry decoder and an attribute decoder. The geometry decoder performs geometry decoding on the geometry bitstream and outputs the decoded geometry. The attribute decoder performs attribute decoding based on the decoded geometry and the attribute bitstream and outputs the decoded attributes. The decoded geometry and decoded attributes are used to reconstruct the point cloud content (the decoded point cloud).

[0172] Figure 11 A point cloud decoder according to an embodiment is shown.

[0173] Figure 11 The point cloud decoder shown is Figure 10 The example shown is a point cloud decoder that can perform decoding operations. Figure 1-9 The reverse process of the encoding operation of the point cloud encoder is shown.

[0174] For reference Figure 1 and Figure 10 As described, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0175] The point cloud decoder according to the embodiments includes an arithmetic decoder (arithmetic decoding) 11000, an octree synthesizer (synthesize octree) 11001, a surface approximation synthesizer (synthesize surface approximation) 11002 and a geometry reconstructor (reconstruct geometry) 11003, a coordinate inverse transformer (inverse transform coordinate) 11004, an arithmetic decoder (arithmetic decoding) 11005, an inverse quantizer (inverse quantization) 11006, an RAHT transformer 11007, an LOD generator (generate LOD) 11008, an inverse elevator (inverse elevation) 11009, and / or a color inverse transformer (inverse transform color) 11010.

[0176] The arithmetic decoder 11000, the octree synthesizer 11001, the surface approximation synthesizer 11002, the geometry reconstructor 11003, and the coordinate inverse transformer 11004 can perform geometry decoding. The geometry decoding according to the embodiments can include direct encoding and triplet geometry decoding. The direct encoding and the triplet geometry decoding are selectively applied. The geometry decoding is not limited to the above-described examples, and as a reference Figure 1-9 The inverse process of the geometry encoding described is performed.

[0177] The arithmetic decoder 11000 according to the embodiments decodes the received geometry bitstream based on arithmetic encoding. The operation of the arithmetic decoder 11000 corresponds to the inverse process of the arithmetic encoder 40004.

[0178] The octree synthesizer 11001 according to the embodiments can generate an octree by acquiring an occupancy code (or information about the geometry obtained as a decoding result) from the decoded geometry bitstream. The occupancy code is as described with reference to Figure 1-9 is configured as described in detail.

[0179] When the triplet geometry encoding is applied, the surface approximation synthesizer 11002 according to the embodiments can synthesize a surface based on the decoded geometry and / or the generated octree.

[0180] The geometry reconstructor 11003 according to the embodiments can regenerate a geometry based on the surface and / or the decoded geometry. As described with reference to Figure 1-9 The direct encoding and the triplet geometry encoding are selectively applied as described. Accordingly, the geometry reconstructor 11003 directly imports position information about the point to which the direct encoding is applied and adds the same. When the triplet geometry encoding is applied, the geometry reconstructor 11003 can reconstruct the geometry by performing the reconstruction operation (e.g., triangle reconstruction, upsampling, and voxelization) of the geometry reconstructor 40005. Details are the same as those described with reference to Figure 6 are described, and thus a description thereof is omitted. The reconstructed geometry can include a point cloud picture or frame that does not include attributes.

[0181] The coordinate inverse transformer 11004 according to the embodiment can acquire point positions based on the reconstructed geometry-transformed coordinates.

[0182] The arithmetic decoder 11005, the inverse quantizer 11006, the RAHT transformer 11007, the LOD generator 11008, the inverse upscaler 11009, and / or the color inverse transformer 11010 can perform the attribute decoding described with reference to FIGS. 1 to 3. Figure 10 The attribute decoding described above. The attribute decoding according to the embodiment includes a region-adaptive hierarchical transform (RAHT) decoding, an interpolation-based hierarchical nearest neighbor prediction (prediction transform) decoding, and an interpolation-based hierarchical nearest neighbor prediction with an update / upscale step (upscale transform) decoding. The above three decoding schemes can be selectively used, or a combination of one or more decoding schemes can be used. The attribute decoding according to the embodiment is not limited to the above-described examples.

[0183] The arithmetic decoder 11005 according to the embodiment decodes an attribute bitstream by arithmetic coding.

[0184] The inverse quantizer 11006 according to the embodiment inverse quantizes information on a decoded attribute bitstream or an attribute acquired as a decoding result, and outputs an inverse quantized attribute (or attribute value). The inverse quantization can be selectively applied based on attribute encoding of the point cloud encoder.

[0185] According to the embodiment, the RAHT transformer 11007, the LOD generator 11008, and / or the inverse upscaler 11009 can process the reconstructed geometry and the inverse quantized attribute. As described above, the RAHT transformer 11007, the LOD generator 11008, and / or the inverse upscaler 11009 can selectively perform a decoding operation corresponding to encoding of the point cloud encoder.

[0186] The color inverse transformer 11010 according to the embodiment performs inverse transform coding to inverse transform a color value (or texture) included in a decoded attribute. The operation of the color inverse transformer 11010 can be selectively performed based on the operation of the color transformer 40006 of the point cloud encoder.

[0187] Although not shown in the drawings, Figure 11 The elements of the point cloud decoder of Figure 11 at least one or more of the operations and / or functions of the elements of the point cloud decoder of Figure 11 The one or more processors can operate or execute a software program and / or a set of instructions for performing the operations and / or functions of the elements of the point cloud decoder of

[0188] Figure 12 A transmitting device according to an embodiment is shown.

[0189] Figure 12 The transmitting device shown is Figure 1 The transmitting device 10000 (or Figure 4 Example of a point cloud encoder. Figure 12 The transmitting device shown can perform the same operation as the reference. Figure 1-9 The described point cloud encoder includes one or more of the same or similar operations and methods. The transmitting apparatus according to the embodiment may include a data input unit 12000, a quantization processor 12001, a voxelization processor 12002, an octree occupancy code generator 12003, a surface model processor 12004, an intra / inter-frame coding processor 12005, an arithmetic encoder 12006, a metadata processor 12007, a color transformation processor 12008, an attribute transformation processor 12009, a prediction / boosting / RAHT transformation processor 12010, an arithmetic encoder 12011, and / or a transmission processor 12012.

[0190] According to the embodiment, the data input unit 12000 receives or acquires point cloud data. The data input unit 12000 can perform operations and / or acquisition methods similar to those of the point cloud video acquirer 10001 (or refer to...). Figure 2 The described acquisition process (20000) is the same as or similar to the operation and / or acquisition method.

[0191] The data input unit 12000, quantization processor 12001, voxelization processor 12002, octree occupancy code generator 12003, surface model processor 12004, intra / inter-frame coding processor 12005, and arithmetic encoder 12006 perform geometric coding. Geometric coding according to the implementation method and reference... Figure 1-9 The geometric codes described are the same or similar, so their detailed descriptions are omitted.

[0192] The quantization processor 12001 according to the embodiment quantizes geometry (e.g., point position values). The operation of the quantization processor 12001 and / or quantization with reference... Figure 4 The operation and / or quantization of the described quantizer 40001 are the same or similar. Details and references Figure 1-9 The descriptions are the same.

[0193] According to the embodiment, the voxelization processor 12002 voxels the quantized position values ​​of points. The voxelization processor 120002 can execute and reference... Figure 4 The operation and / or voxelization process of the quantizer 40001 described are the same as or similar to the operation and / or process. Details and references Figure 1-9 The descriptions are the same.

[0194] According to the implementation method, the octree occupancy code generator 12003 performs octree encoding based on the voxelized positions of points in the octree structure. The octree occupancy code generator 12003 can generate occupancy codes. The octree occupancy code generator 12003 can execute and reference... Figure 4 and Figure 6 The operations and / or methods described are the same as or similar to those of the point cloud encoder (or octree analyzer 40002). Details and references Figure 1-9 The descriptions are the same.

[0195] According to the implementation, the surface model processor 12004 can perform triadic geometry encoding based on a surface model to reconstruct point positions in a specific region (or node) based on voxels. The surface model processor 12004 can perform operations related to reference... Figure 4 The operations and / or methods described are the same as or similar to those of the point cloud encoder (e.g., surface approximation analyzer 40003). Details and references are available. Figure 1-9 The descriptions are the same.

[0196] The intra / inter-frame coding processor 12005 according to the embodiment can perform intra / inter-frame coding on point cloud data. The intra / inter-frame coding processor 12005 can perform operations similar to those described above. Figure 7 The described intra / inter-frame coding is the same or similar. Details and references Figure 7 The descriptions are the same. According to an implementation, the intra / inter-frame coding processor 12005 may be included in the arithmetic encoder 12006.

[0197] The arithmetic encoder 12006 according to the embodiment performs entropy encoding on octrees and / or approximate octrees of point cloud data. For example, the encoding scheme includes arithmetic encoding. The arithmetic encoder 12006 performs the same or similar operations and / or methods as the arithmetic encoder 40004.

[0198] The metadata processor 12007 according to an embodiment processes metadata (e.g., set values) about point cloud data and provides it to necessary processing procedures such as geometric encoding and / or attribute encoding. Additionally, the metadata processor 12007 according to an embodiment can generate and / or process signaling information related to geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be encoded separately from the geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be interleaved.

[0199] The color transformation processor 12008, attribute transformation processor 12009, prediction / boosting / RAHT transformation processor 12010, and arithmetic encoder 12011 perform attribute encoding. Attribute encoding according to the implementation method and reference... Figure 1-9The described attribute encodes the same or similar, and thus a detailed description thereof is omitted.

[0200] The color transform processor 12008 according to the embodiment performs color transform encoding to transform color values included in the attribute. The color transform processor 12008 can perform the color transform encoding based on the reconstructed geometry. The reconstructed geometry is the same as described with reference to Figure 1-9 The described color transformer 40006. In addition, it performs the same or similar operations and / or methods as the color transform encoder 12008 described with reference to Figure 4 The described color transformer 40006. In addition, it performs the same or similar operations and / or methods as the color transform encoder 12008 described with reference to

[0201] The attribute transform processor 12009 according to the embodiment performs attribute transform to transform the attribute based on the reconstructed geometry and / or positions for which geometry encoding is not performed. The attribute transform processor 12009 performs the same or similar operations and / or methods as the attribute transformer 40007 described with reference to Figure 4 The described color transformer 40006. In addition, it performs the same or similar operations and / or methods as the color transform encoder 12008 described with reference to Figure 4 The described color transformer 40006. In addition, it performs the same or similar operations and / or methods as the color transform encoder 12008 described with reference to Figure 1-9 The described color transformer 40006. In addition, it performs the same or similar operations and / or methods as the color transform encoder 12008 described with reference to

[0202] The arithmetic encoder 12011 according to the embodiment can encode the encoded attribute based on arithmetic encoding. The arithmetic encoder 12011 performs the same or similar operations and / or methods as the operations and / or methods of the arithmetic encoder 400012.

[0203] The transmission processor 12012 according to the embodiment can transmit individual bitstreams containing the encoded geometry and / or the encoded attributes and metadata information, or transmit one bitstream configured with the encoded geometry and / or the encoded attributes and metadata information. When the encoded geometry and / or the encoded attributes and metadata information according to the embodiment are configured into one bitstream, the bitstream can include one or more sub-bitstreams. The bitstream according to the embodiment can contain signaling information and slice data, the signaling information including a sequence parameter set (SPS) for sequence level signaling, a geometry parameter set (GPS) for geometry information encoding signaling, an attribute parameter set (APS) for attribute information encoding signaling, and a tile parameter set (TPS) for tile level signaling. The slice data can include information about one or more slices. One slice according to the embodiment can include one geometry bitstream Geom00 and one or more attribute bitstreams Attr00 and Attr10.

[0204] A slice refers to a series of syntax elements representing all or part of an encoded point cloud frame.

[0205] The TPS according to the embodiment can include information about individual tiles in one or more tiles (e.g., coordinate information about a bounding box and height / size information). A geometry bitstream can contain a header and a payload. The header of the geometry bitstream according to the embodiment can contain a parameter set identifier (geom_parameter_set_id) included in the GPS, a tile identifier (geom_tile_id), and a slice identifier (geom_slice_id), and information about data contained in the payload. As described above, the metadata processor 12007 according to the embodiment can generate and / or process signaling information and transmit it to the transmission processor 12012. According to the embodiment, an element performing geometry encoding and an element performing attribute encoding can share data / information with each other as indicated by dotted lines. The transmission processor 12012 according to the embodiment can perform operations and / or transmission methods the same as or similar to those of the transmitter 10003. Details are the same as those described with reference to Figure 1 and Figure 2 Those described are the same, and thus a description thereof is omitted.

[0206] Figure 13 A receiving apparatus according to the embodiment is shown.

[0207] Figure 13 The receiving apparatus shown is an example of a receiving apparatus 10004 (or Figure 1 a point cloud decoder) of Figure 10 and Figure 11 the same as those described with reference to Figure 13 The receiving apparatus shown can perform operations and / or reception methods the same as or similar to those of the receiver 10004. Details are the same as those described with reference to Figure 1-11one or more operations and methods of the point cloud decoder described.

[0208] The reception device according to the embodiments can include a receiver 13000, a reception processor 13001, an arithmetic decoder 13002, an occupancy code based octree reconstruction processor 13003, a surface model processor (triangle reconstruction, upsampling, voxelization) 13004, an inverse quantization processor 13005, a metadata parser 13006, an arithmetic decoder 13007, an inverse quantization processor 13008, a prediction / lifting / RAHT inverse transform processor 13009, a color inverse transform processor 13010, and / or a Tenderer 13011. The respective decoding elements according to the embodiments can perform inverse processes of the operations of the corresponding encoding elements according to the embodiments.

[0209] The receiver 13000 according to the embodiments receives point cloud data. The receiver 13000 can perform the same or similar operations and / or reception methods as those of the receiver 10005 of Figure 1 The detailed description thereof is omitted.

[0210] The reception processor 13001 according to the embodiments can acquire a geometry bitstream and / or an attribute bitstream from the received data. The reception processor 13001 can be included in the receiver 13000.

[0211] The arithmetic decoder 13002, the occupancy code based octree reconstruction processor 13003, the surface model processor 13004, and the inverse quantization processor 13005 can perform geometry decoding. The geometry decoding according to the embodiments is the same or similar to the geometry decoding described with reference to Figure 1-10 The detailed description thereof is omitted.

[0212] The arithmetic decoder 13002 according to the embodiments can decode the geometry bitstream based on arithmetic encoding. The arithmetic decoder 13002 performs the same or similar operations and / or encoding as those of the arithmetic decoder 11000.

[0213] The occupancy code-based octree reconstruction processor 13003 according to the embodiments can reconstruct an octree by acquiring an occupancy code from a decoded geometry bitstream (or information about the geometry acquired as a result of decoding). The occupancy code-based octree reconstruction processor 13003 performs the same or similar operations and / or methods as those of the octree synthesizer 11001 and / or the octree generation method. When the triplet geometry coding is applied, the surface model processor 13004 according to the embodiments can perform triplet geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on a surface model method. The surface model processor 13004 performs the same or similar operations as those of the surface approximation synthesizer 11002 and / or the geometry reconstructor 11003.

[0214] The inverse quantization processor 13005 according to the embodiments can inverse quantize decoded geometry.

[0215] The metadata parser 13006 according to the embodiments can parse metadata (e.g., setting values) contained in received point cloud data. The metadata parser 13006 can pass the metadata to geometry decoding and / or attribute decoding. The metadata is the same as that described with reference to Figure 12 A detailed description of the metadata is omitted as it is the same as that described with reference to

[0216] The arithmetic decoder 13007, the inverse quantization processor 13008, the prediction / lifting / RAHT inverse transform processor 13009, and the color inverse transform processor 13010 perform attribute decoding. The attribute decoding is the same as or similar to that described with reference to Figure 1-10 A detailed description of the attribute decoding is omitted as it is the same as or similar to that described with reference to

[0217] The arithmetic decoder 13007 according to the embodiments can decode an attribute bitstream by arithmetic encoding. The arithmetic decoder 13007 can decode the attribute bitstream based on reconstructed geometry. The arithmetic decoder 13007 performs the same or similar operations and / or encoding as those of the arithmetic decoder 11005 and / or encoding.

[0218] The inverse quantization processor 13008 according to the embodiments can inverse quantize a decoded attribute bitstream. The inverse quantization processor 13008 performs the same or similar operations and / or methods as those of the inverse quantizer 11006 and / or inverse quantization method.

[0219] The prediction / lifting / RAHT inverse transform processor 13009 according to the embodiments can process the reconstructed geometry and inverse quantized attributes. The prediction / lifting / RAHT inverse transform processor 13009 performs one or more operations and / or decodes identical or similar to operations and / or decodes of the RAHT transformer 11007, the LOD generator 11008, and / or the inverse lifter 11009. The color inverse transform processor 13010 according to the embodiments performs inverse transform encoding to inverse transform color values (or textures) included in the decoded attributes. The color inverse transform processor 13010 performs operations and / or inverse transform encoding identical or similar to operations and / or inverse transform encoding of the color inverse transformer 11010. The renderer 13011 according to the embodiments can render point cloud data.

[0220] Figure 14 An exemplary structure operable in connection with the point cloud data transmission / reception method / apparatus according to the embodiments is illustrated.

[0221] Figure 14 The structure represents a configuration in which at least one of a server 1460, a robot 1410, a self-driving vehicle 1420, an XR device 1430, a smart phone 1440, a home appliance 1450, and / or a head-mounted display (HMD) 1470 is connected to a cloud network 1400. The robot 1410, the self-driving vehicle 1420, the XR device 1430, the smart phone 1440, or the home appliance 1450 is referred to as a device. Also, the XR device 1430 can correspond to a point cloud data (PCC) device according to the embodiments or can be operatively connected to the PCC device.

[0222] The cloud network 1400 can represent a network constituting a part of or existing in a cloud computing infrastructure. Here, the cloud network 1400 can be configured using a 3G network, a 4G or long term evolution (LTE) network, or a 5G network.

[0223] The server 1460 can be connected to at least one of the robot 1410, the self-driving vehicle 1420, the XR device 1430, the smart phone 1440, the home appliance 1450, and / or the HMD 1470 via the cloud network 1400, and can assist in processing of at least a part of the connected devices 1410 to 1470.

[0224] The HMD 1470 represents one of implementation types of the XR device and / or the PCC device according to the embodiments. The HMD-type device according to the embodiments includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0225] Hereinafter, various embodiments of the devices 1410 to 1450 to which the above-described technology is applied will be described. Figure 14The illustrated devices 1410 to 1450 can be operatively connected / coupled to the point cloud data transmission / reception device according to the above-described embodiments.

[0226] <PCC+XR>

[0227] The XR / PCC device 1430 can employ PCC technology and / or XR (AR+VR) technology, and can be implemented as an HMD, a head-up display (HUD) provided in a vehicle, a television, a mobile phone, a smartphone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a stationary robot, or a mobile robot.

[0228] The XR / PCC device 1430 can analyze 3D point cloud data or image data acquired through various sensors or from external devices and generate position data and attribute data regarding 3D points. Thereby, the XR / PCC device 1430 can acquire information regarding a surrounding space or a real object, and render and output an XR object. For example, the XR / PCC device 1430 can match an XR object including auxiliary information regarding an identified object with the identified object and output the matched XR object.

[0229] <PCC+Self-driving+XR>

[0230] The self-driving vehicle 1420 can be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0231] The self-driving vehicle 1420 to which XR / PCC technology is applied can mean a self-driving vehicle provided with a means for providing an XR image, or a self-driving vehicle that is a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, the self-driving vehicle 1420 can be distinguished from and operatively connected to the XR device 1430.

[0232] The self-driving vehicle 1420 having a means for providing an XR / PCC image can acquire sensor information from sensors including a camera, and output a generated XR / PCC image based on the acquired sensor information. For example, the self-driving vehicle 1420 can have a HUD and output an XR / PCC image thereto, thereby providing a passenger with an XR / PCC object corresponding to a real object or an object presented on a screen.

[0233] When the XR / PCC object is output to the HUD, at least a part of the XR / PCC object can be output to overlap with a real object pointed by the eyes of the passenger. On the other hand, when the XR / PCC object is output on a display provided inside the self-driving vehicle, at least a part of the XR / PCC object can be output to overlap with an object on the screen. For example, the self-driving vehicle 1220 can output XR / PCC objects corresponding to objects such as a road, another vehicle, a traffic light, a traffic sign, a two-wheeled vehicle, a pedestrian, and a building.

[0234] Virtual reality (VR) technology, augmented reality (AR) technology, mixed reality (MR) technology, and / or point cloud compression (PCC) technology according to embodiments are applicable to various devices.

[0235] In other words, the VR technology is a display technology that provides only a CG image of a real world object, background, etc. On the other hand, the AR technology refers to a technology that displays a CG image created virtually on an image of a real object. The MR technology is similar to the above-described AR technology in that a virtual object to be displayed is mixed and combined with the real world. However, the MR technology is different from the AR technology in that the AR technology clearly distinguishes between a real object and a virtual object created as a CG image and uses the virtual object as a complementary object of the real object, whereas the MR technology treats the virtual object as an object having equivalent characteristics to the real object. More specifically, an example to which the MR technology is applied is a hologram service.

[0236] Recently, the VR, AR, and MR technologies are commonly referred to as extended display (XR) technology rather than being clearly distinguished from each other. Accordingly, embodiments of the disclosure are applicable to any one of the VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies is applicable to such technologies.

[0237] The PCC method / apparatus according to embodiments can be applied to a vehicle that provides a self-driving service.

[0238] The vehicle that provides a self-driving service is connected to a PCC apparatus to perform wired / wireless communication.

[0239] When the point cloud data (PCC) transmitting / receiving device according to the embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services (which may be provided together with autonomous driving services) and transmit it to the vehicle. When the PCC transmitting / receiving device is installed in the vehicle, it can receive / process content data related to AR / VR / PCC services based on user input signals input through a user interface device and provide it to the user. The vehicle or user interface device according to the embodiment can receive user input signals. User input signals according to the embodiment may include signals indicating autonomous driving services.

[0240] The PCC data encoder according to the implementation method can correspond to an encoding device, an encoder, a point cloud transmitting device, a point cloud data encoder, etc.

[0241] The PCC data decoder according to the implementation method can correspond to a decoding device, a decoder, a point cloud receiving device, a point cloud data decoder, etc.

[0242] According to the implementation method, geometry can be referred to as geometric shape information, geometric shape data, etc., and attributes can be referred to as attribute information, attribute data, etc.

[0243] The method / apparatus according to the embodiments may refer to a method / apparatus for sending or receiving point cloud data according to the embodiments.

[0244] Figure 15 A method for generating Morton codes according to an embodiment is shown.

[0245] It can be done Figure 1 Transmitting device 10000, point cloud video encoder 10002, receiving device 10004, point cloud video decoder 10006. Figure 2 Encoding 20001, Decoding 20003, Figure 4 PCC encoder Figure 10 and Figure 11 PCC decoder Figure 12 Point cloud data transmission device Figure 13 Point cloud data receiving device, XR device 1730, Figure 14 Smartphone 1440 Figure 19 The Morton code generator 19000, etc., are used to execute based on Figure 15 The implementation of the Morton code generation method. The apparatus and method for performing operations related to Morton code generation according to the implementation method may be abbreviated as the method / apparatus according to the implementation method. The various components of the apparatus according to the implementation method may correspond to hardware, software, a processor, and / or combinations thereof.

[0246] Embodiments relate to a method of supporting efficient point cloud compression and / or improving decoding performance by using a per-sequence and / or per-slice axis transform. During point cloud compression, similarity between points can be increased by an axis transform so that attribute values are closest to each other. Thereby, a bitstream size can be reduced and quality of a decoded point cloud can be improved compared to an original point cloud.

[0247] The method / apparatus for transmitting or receiving point cloud data according to embodiments can provide a Morton code generation method that improves encoding / decoding performance.

[0248] Embodiments use a Morton code generation, a spatial coordinate search method, to compress three-dimensional point cloud data (data about positions and attributes of points related to an object).

[0249] Further, the method / apparatus according to embodiments can adaptively perform the Morton code generation method according to embodiments based on attributes of axes. Point clouds according to embodiments are subjected to geometric compression in the order of xyz, and address values of a Morton code are generated according to the order of xyz. A prediction order of attribute values is also determined by the xyz order of geometry.

[0250] The method / apparatus according to embodiments is not limited to the order of xyz, and can further supplement a Morton code of attribute values applied in positions in a 3D space of a point cloud. The Morton code according to embodiments is used to connect geometric values and attribute values of point cloud data by Z-order search in a space. In this case, geometric values are ordered by search in a space, but the Morton code cannot reflect distances of actual geometric values. Accordingly, the method / apparatus according to embodiments is proposed to additionally generate an adaptive Morton code by using an axis transform.

[0251] In point cloud compression according to embodiments, points close in distance in a 3D space are found as neighbor nodes, and an index is transmitted to decode an average value of one or three most similar values based on the neighbor nodes. A decoder according to embodiments identifies positions of neighbor nodes closest to a current point based on the received index, and predicts attribute values of the neighbor nodes from attribute values of the current point by weight calculation. Accordingly, by selecting neighbor nodes having similar attributes but having close spatial distances, an encoder and a decoder can improve efficiency of compression, encoding, decoding, etc.

[0252] According to embodiments, an adaptive Morton encoding method can be used as a method for searching neighbor nodes in a space.

[0253] According to embodiments, a Morton code can be generated by a transmitter, an encoder, a Morton code generator, etc. A Morton code generation method according to embodiments can generate a Morton code based on weights. For example, the method can include 1) an axis-based adaptive Morton code generation method, 2) a bit allocation Morton code generation method, and 3) a Morton code generation method according to value distribution.

[0254] Regarding the axis-based adaptive Morton code generation method according to embodiments, a method / apparatus according to embodiments can perform axis-based adaptive Morton code generation. A method / apparatus according to embodiments can calculate a bounding box of a point cloud and adjust a bit interleaving unit based on axis lengths. In Morton code calculation according to embodiments, when binary bit values of coordinates of a point on xyz axes are given, a Morton code can be generated by bit interleaving based on the order of zyxzyxzyx… A method / apparatus according to embodiments allocates lengths of x, y, and z axes to a maximum edge, an intermediate edge, and a minimum edge, respectively, and identifies the longest axis. It divides the lengths of the longest axis and the intermediate axis by the shortest axis, and takes log2 of the value obtained by the division. Here, a value obtained by rounding the calculated value is countY and countZ. Each count value indicates the number of repeated bits. For example, when the length of the x axis is 10, the length of the y axis is 40, and the length of the z axis is 20, countY is 2 and countZ is 1. Also, a bit order of the Morton code is given as zzyyxzyyxzyx. That is, z is repeated once and y is repeated twice. Accordingly, a Morton code is generated based on lengths of axes.

[0255] A method / apparatus according to embodiments can perform the above-described process on all points in the bounding box. Each point is mapped one-to-one to a Morton code. When the mapped Morton codes are sorted in ascending order, the points can be searched in an octree in a depth-first search order.

[0256] Regarding the bit allocation type Morton code generation method according to embodiments, a method / apparatus according to embodiments can bit-interleave a Morton code by applying weights to any one of x, y, and z values, regardless of data of a point cloud. According to embodiments, even without calculating a bounding box related to data characteristics, a method / apparatus according to embodiments can acquire point cloud data that is long or short along a specific axis based on width, height, and depth. That is, even without any information about a bounding box, a Morton code can be generated.

[0257] For example, point cloud data can be spread very wide along the x axis and the z axis in space, but its points can be gathered along the y axis in a distribution chart. Also, in the case of LiDAR (Light Detection and Ranging) data, data is generally spread to the left, right, and front and back of a car, but data at the top and bottom thereof can be less important.

[0258] Thus, for such a data sequence, bits for the Morton code generation can be further allocated to the x and z axes to generate the Morton code.

[0259] Regarding the method of generating a Morton code according to a value distribution according to the embodiment, the method / apparatus according to the embodiment can generate a Morton code according to a value distribution. The method / apparatus according to the embodiment can generate a Morton code by bit interleaving a Morton code based on a distribution chart obtained by scanning all x, y, and z values of a point cloud. Generating a Morton code according to a value distribution is different from generating a Morton code based on axes in that there is no need to calculate the weight of a portion where there is no value. For example, the y-axis of a bounding box has values from 0 to 100, but points can not actually be distributed in a range from 10 to 50 within the bounding box. A Morton code can be generated according to a value distribution even if the portion where there is no point is considered.

[0260] In addition to the adaptive Morton code method according to the embodiment, a method for calculating an adaptive Morton code can be specified and diversified by extending a bounding box of a point cloud. With the adaptive Morton coding method, performance can be improved.

[0261] According to the embodiment, a problem of a difference in a Morton code occurring at the same Euclidean distance in space can be solved. What is presented here is a condition that can change the order of xyz axes in various forms, and this can be performed based on content, tile, slice, geometry value, attribute value, etc. Also, the embodiment defines whether to change the axis transformation according to a threshold. Thus, the embodiment proposes a condition for mapping an adaptive Morton code to perform an adaptive spatial search based on content, tile, slice, range, or attribute value, and search for a nearby neighbor node. Point cloud encoding and decoding according to the embodiment can change a Morton code according to various data characteristics of point cloud data, rather than using only one type of Morton code.

[0262] To solve a problem of failing to reflect a difference between an actual coordinate occurring in a processing of a Morton code of point cloud data and a Morton code distance, the embodiment provides a method for generating an adaptive Morton code. That is, the method according to the embodiment can generate different adaptive Morton codes for each axis according to content, tile, slice, range, attribute value, or geometry value. In addition, by configuring different options for each axis / density through a threshold setting method according to the embodiment, whether to apply an operation to each content, tile, slice, attribute value, or geometry value can be differently set.

[0263] For example, the operation of changing the axis order according to the embodiment can be performed before geometric encoding by the transmitting device or encoder according to the embodiment. Furthermore, the operation of changing the axis order according to the embodiment can be performed as a process of changing the axis order during attribute encoding after geometric encoding. Moreover, during the axis-based change, this operation can be applied to cases where a threshold exists and cases where a threshold does not exist. Operations related to the presence or absence of a threshold can be performed before geometric encoding or during attribute encoding after geometric encoding.

[0264] The implementation method can be combined with the above-described methods for axis-based adaptive Morton code generation, bit allocation type Morton code generation, or value distribution-based Morton code generation.

[0265] The method / apparatus according to the embodiments can generate Morton codes based on the method according to the embodiments (axis-based adaptive Morton code generation, bit allocation type Morton code generation, or value distribution-based Morton code generation), and also based on changing the generation order of Morton codes according to axis / density / threshold conditions. By changing the Morton address based on the conditions according to the embodiments, different Morton codes can be generated according to the range.

[0266] Figure 15 A method for generating Morton codes according to an embodiment is shown.

[0267] The Morton code generation order according to the implementation method includes two types: one is the order in which Morton codes are generated according to the length of the axis, and the other is the order in which Morton codes are generated according to the density.

[0268] A point can have X, Y, and Z coordinate values ​​of 15000 representing the position of geometric data. The X, Y, and Z coordinate values ​​on each axis are represented by bits 15010. When the coordinate values ​​on each axis have N bits, the bits of each axis can be sorted into a single bit stream by scanning each axis (15020), thereby generating Morton codes (15030). For example, the order of the bits can be XYZ, and Morton codes can be generated using bits sorted in the order of the Nth bit for the X-axis, the Nth bit for the Y-axis, the Nth bit for the Z-axis, ..., the second bit for the X-axis, the second bit for the Y-axis, the second bit for the Z-axis, ..., the first bit for the X-axis, the first bit for the Y-axis, and the first bit for the Z-axis.

[0269] It can be by Figure 4 PCC encoder in Figure 4 The relationship between geometric encoding and attribute encoding in a PCC encoder is determined by... Figure 4 The property transformer 4007, by Figure 11 The PCC decoder in Figure 11between geometry decoding and attribute decoding of a PCC decoder of Figure 4 and Figure 11 Morton code generation according to embodiments can be performed by an encoder / decoder, etc. corresponding to

[0270] Coordinates of points representing point cloud data according to embodiments can be X, Y, and Z axes, and can further include another type of coordinates. The Morton code generation method according to embodiments can be applied to position values of points according to various coordinate systems. Hereinafter, points are explained by way of example of XYZ coordinates. XYZ can be replaced with other factors depending on the type of coordinate system. For example, other coordinate systems can further include a cylindrical coordinate system, a spherical coordinate system, etc., and xyz coordinates are transformed into these coordinate systems to represent a point cloud.

[0271] Figure 16 A method for generating a Morton code according to embodiments is illustrated.

[0272] Figure 16 One or more Morton code generation operations according to embodiments are illustrated, which can be combined with the Morton code generation method described with reference to Figure 15 Figure 16 Each operation of can be performed by a method / apparatus according to embodiments described with reference to Figure 15

[0273] A method / apparatus according to embodiments can generate a Morton code according to an axis (S16000). A method / apparatus according to embodiments can generate a Morton code according to a density (S16010). A method / apparatus according to embodiments can generate a Morton code according to a threshold (S16020). A method / apparatus according to embodiments can use at least one of the axis-based method (S16000), the density-based method (S16010), or the threshold-based method (S16020), or can use a combination of at least one method. Each method is described below. It should be noted that the order of description of the methods does not limit the meaning of the embodiments.

[0274] For example, the Morton code generation method according to embodiments can include a Morton code generation according to only an axis, a Morton code generation according to only a density, a Morton code generation according to both an axis and a density, a method using both a Morton code generation according to an axis and a Morton code generation according to a density, and a method using both a Morton code generation according to a density and an application of a threshold, and a method using both a Morton code generation according to an axis and a density and an application of a threshold.

[0275] S16000 Generating a Morton code from an axis

[0276] ​​According to embodiments, the point cloud data can include one or more frames. A frame includes points existing on x, y, and z axes. A unit containing points distributed in space is referred to as a bounding box. The bounding box can be defined as a space including a maximum value of x, y, and z values to a minimum value of x, y, and z values. The length / size of the bounding box can vary according to a sequence. The length of each axis (length of the X axis, length of the Y axis, length of the Z axis) indicates how long or short the distribution of points is. The order of the three axes of xyz can be changed by arranging the axes in ascending order and / or descending order. The order of the axes can be determined for each sequence, tile, or slice or based on a geometry unit, an attribute unit, a geometry and attribute unit, etc.

[0277] According to embodiments, the same M-Codes can be generated for each axis. According to embodiments, different M-Codes can be generated for each axis. The bounding box according to embodiments has a width, a depth, and a height. Based on each axis in which the bounding box is located, the axis-wise length can be examined. For example, the y-axis, the z-axis, and the x-axis can be arranged in descending order of the length of the axes from the longest to the shortest. In generating M-Codes from the three axes, the bits constituting the M-Codes can be generated in the order of the nth bit of the y-axis, the nth bit of the z-axis, and the nth bit of the x-axis, which can be an operation of interleaving the bits of the M-Codes in ascending order and / or descending order of the length of the three axes. That is, the M-Codes can be generated by bit interleaving the bits of the axes based on the method and conditions according to embodiments.

[0278] Accordingly, the method / apparatus according to embodiments can compare the lengths of the axes of the bounding box, and the bit interleaving can be diversified based on the characteristics of the lengths. The bit interleaving of the M-Codes can provide variations in the M-Codes, which reflect the characteristics of the actual data and the data distribution. As a result, the point cloud can be efficiently compressed and reconstructed.

[0279] Operation of M-Codes generation based on axes according to embodiments

[0280] When (x_i, y_i, z_i) is the position of the current point I, the length of the bounding box along all three axes can be calculated as length_x, length_y, and length_z. Considering the direction of the longest axis, more relevant attribute neighbor candidates can be found. The longest axis can be considered for searching for the corresponding neighbors. In particular, in the case of category 3 data, the points can be densely distributed within the X axis or the Y axis. By searching the X and Y axes rather than the Z axis, the nearest neighbors can be found more quickly.

[0281] The obtained lengths length_x, length_y, and length_z can be stored in ascending order. The lengths of the axes can correspond to SortedAxisOrder[0], SortedAxisOrder[1], SortedAxisOrder[2], respectively. The sorted order can be an input of the order of X, Y, Z coordinates of the Morton code.

[0282] The sorted order according to the embodiment is a bit interleaving order. The current order of z, y, and x can be changed by axisSortedAxisOrder[2], SortedAxisOrder[1], SortedAxisOrder[0]. The order can be created to have the same Morton code implementation. The sorted axes can be adaptively changed to the order of MC_i corresponding to the spatial search order. For the first candidate of the nearest neighbor weight prediction, the influence in the direction of the longest axis can be considered. The method / apparatus according to the embodiment can provide a changed spatial order for the bit interleaving order of yxz.

[0283] The embodiment can set the order of the search space through a dynamic Morton code.

[0284] The candidate of the nearest neighbor can be changed due to the changed adaptive Morton code.

[0285] The neighbor candidate can be located closer in the direction of the longest axis. Accordingly, the adaptive Morton code prediction can be more efficient in terms of the existence of space.

[0286] The Morton code generation operation according to the embodiment can efficiently provide the adaptive Morton code through the change of the Morton address generation.

[0287] The embodiment additionally applies a threshold of the ratio of the minimum / maximum length of the slice to change the Morton address order.

[0288] When the Morton code according to the embodiment is used, the time complexity can be reduced and the encoding / decoding time efficiency can be increased.

[0289] S16010 Generating a Morton code from a density

[0290] There is a density with respect to each axis of a space in which points constituting point cloud data are distributed. The method / apparatus according to the embodiment can generate the density of the points with respect to each axis based on an axis-based adaptive Morton code generation method, a bit allocation type Morton code generation method, a Morton code generation method according to a value distribution, etc.

[0291] The densities between the axes can be different. For example, the density can be higher on a shorter axis.

[0292] According to embodiments, the bits of the coordinates on each axis can be ordered in ascending order of density (bit interleaving). According to embodiments, the bits of the coordinates on each axis can be ordered in descending order of density (bit interleaving). When the bits are ordered in ascending order of density, according to embodiments, the points having data represented in the order of x, y, and z axes can be reordered according to the density. For example, when the density on the x axis is 10, the density on the y axis is 5, and the density on the z axis is 8, the bit interleaving can be performed in ascending order of density to generate the Morton code in the order of y, z, and x in the Morton code generation.

[0293] The changed positions of the points can be changed according to a geometry unit, an attribute unit, a geometry and attribute unit, etc. Further, the axis order unit can change in a sequential unit, a tile unit, and a slice unit.

[0294] In addition to the method of using the Morton code in the order of xyz according to embodiments Figure 16 ), the Morton code can be generated by changing the order of xyz based on various conditions. With this Morton code, the point cloud data is compressed and reconstructed. As the positions of the points change, the neighbor nodes with respect to the geometry values and / or the attribute values can be easily found with respect to the longest axis and / or the shortest axis. That is, since the Morton code performs bit interleaving according to the characteristics (e.g., axis or density) of the points of the actual point cloud data, the neighbor nodes can be easily found.

[0295] That is, by changing only the order of the xyz data of the point cloud input to the encoder and / or the decoder according to embodiments, the encoding / decoding efficiency can be improved. The delay of the operation of changing the order of the axes is not a problem, and the encoding / decoding efficiency can be efficiently increased.

[0296] In particular, there can be point cloud data corresponding to Category 3, for example, low density data such as autonomous driving data, which has low correlation between adjacent data, or static data such as Category 1, which has high density and high correlation between adjacent data. For reflectance information about such data, a particular axis can be short. The longest axis can be searched first so that the longest axis is arranged in the first number (the first number arranged in order when configuring the Morton code, or the rightmost bit in the bitstream). When a particular axis is searched first, more neighbor nodes can be ensured within the search range, and thus the attribute value can be efficiently predicted. Further, by searching the long axis first, the compression efficiency can be improved. Accordingly, the method / apparatus according to embodiments uses a Morton code ordering method capable of searching more neighbor nodes within a range based on the axis, density, or other characteristics, to perform fast compression and reconstruction of point cloud data.

[0297] A method for generating a Morton code according to conditions according to embodiments will be further described with reference to Figure 17 FIG. 1 illustrates a method for generating a Morton code according to conditions according to embodiments.

[0298] In the density-based Morton code generation according to the implementation, the ratio can be generated by performing a projection based on xoy / yoz / xoz in a specific plane in space. Here, the ratio can be generated by dividing the number of points by the total number of points. Here, the threshold for the density-based Morton code transformation can be set to, for example, 20%.

[0299] Depending on the implementation conditions, the setting information for the Morton code can be generated by the encoder and sent to the decoder as signaling information of the bit stream.

[0300] Threshold for S16020 Morton code generation order

[0301] For example, the shape of the bounding box can depend on the distribution of points, i.e., data characteristics. When the bounding box has a cube-like shape, Morton code bit interleaving may not be performed. Therefore, to cover various scenarios, a method for determining whether a bounding box is cube-like based on a threshold, according to an implementation, is also provided.

[0302] Regarding the characteristics of the point cloud data according to the implementation method, there may be bounding boxes with a cube shape, which may be long or short along one axis of the bounding box.

[0303] The compression and reconstruction efficiency of points within a cube bounding box can be greater than that of points within a non-cube bounding box.

[0304] The point cloud data according to the implementation method can have the same data characteristics as Category 1. The bounding boxes of these points can be cubic. That is, data obtained through 3D scanning of static objects, high-density and highly correlated data, etc., can have cubic sequences. When segmenting point cloud data, there may be data whose slices are cubic. Due to these data characteristics, it may be difficult to improve the efficiency of data compression and reconstruction.

[0305] Therefore, after determining the ascending / descending method based on axis / density according to the embodiment, the shape of the data can be determined. For example, the Morton code generation order may remain unchanged when the ratio of the shortest axis to the longest axis of the bounding box is less than or greater than a certain threshold. In addition to the axis-based generation S16000 and / or density-based generation S16010 according to the embodiment, a threshold-based determination option may exist. (See also...) Figure 18 Describe a threshold-based determination method.

[0306] An apparatus for transmitting point cloud data according to an embodiment includes: an encoder configured to encode point cloud data; and a transmitter configured to transmit a bit stream including point cloud data.

[0307] According to the implementation method, the encoder generates Morton codes for point cloud data.

[0308] According to the implementation method, the encoder generates Morton codes in the order of the geometric data of the point cloud data.

[0309] The geometric data contains coordinate values ​​on each axis, and the Morton code is generated based on the binary bits of these coordinate values. Furthermore, the binary bits of the Morton code are ordered according to the length of the axis.

[0310] According to the implementation method, the encoder generates Morton codes in the order of the density of geometric data of the point cloud data.

[0311] The geometric data contains coordinate values ​​on each axis, and the Morton code is generated based on the binary bits of these coordinate values. Furthermore, the binary bits of the Morton code are ordered based on the density relative to one or more points on each axis.

[0312] According to the implementation method, the encoder generates Morton codes in the order of the density of geometric data of the point cloud data.

[0313] The geometric data contains coordinate values ​​on each axis, and the Morton code is generated based on the binary bits of these coordinate values. Furthermore, the binary bits of the Morton code are ordered based on the density relative to one or more points on each axis.

[0314] The Morton code generation operation of the encoder according to the implementation method can be performed based on a threshold related to the distribution of one or more points in the point cloud data.

[0315] According to the implementation, the receiver or receiving interface receiver is configured to receive a bit stream including point cloud data, and the decoder is configured to decode the point cloud data.

[0316] According to the implementation method, the decoder generates Morton codes for point cloud data.

[0317] Figure 17 A method for generating Morton codes according to an embodiment is shown.

[0318] Figure 17 The generation of S16000 based on the Morton code of the axis and based on... Figure 16 The implementation of Morton code generation S16010 based on density. The method / apparatus according to the embodiment can generate Morton codes based on axis, density, and / or both axis and density.

[0319] The points in the point cloud data of the encoder and decoder according to the embodiment have position values. These positions have values ​​represented in bits (17000). Each point in space has geometric coordinate values ​​(17000) of x, y, and z. These coordinate values ​​have binary bits (17000).

[0320] The method / apparatus according to the embodiment can determine the sorting type of x, y, and z (17000) represented in binary bits in ascending order of axes, descending order of axes (S16010), ascending order of density, descending order of density (S16000), or a sorting based on a combination of axes and densities. The position values ​​on each axis are rearranged based on the determined order (17010).

[0321] For example, based on the specific conditions (17020) above, xyz bits ordered in the order of z, y and x are generated from the bits in the order of x, y and z.

[0322] According to the implementation method, when input is provided directly to the encoder without specific conditions (the user provides input for encoding in the order of zxy), sorting can be omitted. Morton codes can be generated solely through signaling without sorting.

[0323] After performing bit interleaving (17020) in a changed order, a Morton address (17030) is generated for a coordinate value with one binary bit. The Morton code 17030 is generated as a bit stream for the three axes by sorting the bits one by one in the order of z, x, and y.

[0324] The method / apparatus according to the implementation can encode and decode points based on the modified Morton address.

[0325] Because of the modified Morton address, points on the longest axis can be searched first. To generate Morton addresses for efficient searching, the method / apparatus according to the implementation can use information (axial length, density value) related to the bounding box containing the points.

[0326] The method / apparatus according to the embodiment can calculate the density along each axis. Shorter axes have higher densities. This is because the number of points does not vary according to each axis, but the number of points included in a particular space or bounding box is the same. The density can be taken into account when interleaving the Morton code bits. Furthermore, according to the embodiment, a threshold (S16020) can be applied in the bit interleaving of the Morton code.

[0327] According to the implementation method, finding neighboring nodes can be done without searching the minor axis. Therefore, searching for neighboring nodes around the major axis is more efficient. To achieve this efficiency in neighboring node searching, Morton code is bit-interleaved.

[0328] Figure 18 The threshold for the Morton code generation order according to the implementation method is shown.

[0329] Figure 18 It shows the relationship with Figure 16The threshold-related operation of the Morton code generation order (S16020).

[0330] For example, when an execution process with a threshold is represented as pseudocode as follows.

[0331] inline void

[0332] computeAxisOrder()

[0333] {

[0334] autobbox=pointCloud.computeBoundingBox();

[0335] for(int i = 0; i < 3; i++){

[0336] axisInfo[i].length=bbox.max[i]-bbox.min[i];

[0337] axisInfo[i].order = i;

[0338] }

[0339] sort(axisInfo.begin(),axisInfo.end());

[0340] double minMaxAxisRatio=double(axisInfo[2].length) / double(axisInfo[0].length);

[0341] if (minMaxAxisRatio < 5.0) {

[0342] axisInfo[0].order = 0;

[0343] axisInfo[1].order = 1;

[0344] axisInfo[2].order = 2;

[0345] }

[0346] }

[0347] The length of each axis, axisInfo[i].length, can be obtained based on the difference between the major axis (bbox.max[i]) and the minor axis (bbox.min[i]) of the bounding box.

[0348] For each axis (for(int i = 0; i < 3; i++)), you can check the length of each axis and the order of each axis (axisInfo[i].order).

[0349] minMaxAxisRatio represents a threshold. This threshold can be the ratio of the median axis to the major axis. According to embodiments, the threshold may also include the ratio of the minor axis to the median axis and the ratio of the minor axis to the major axis. According to embodiments, various ratios of the axis lengths can be used as thresholds.

[0350] According to the implementation method, the ratio between the two axes is calculated as (double minMaxAxisRatio = double(axisInfo[2].length) / double(axisInfo[0].length)).

[0351] In the comparison threshold (minMaxAxisRatio) operation, it is checked whether minMaxAxisRatio is less than, for example, 5.0. When minMaxAxisRatio is less than a specific value (a value other than 5.0 can be used), the axis order can be determined as axisInfo[0].order = 0; axisInfo[1].order = 1; axisInfo[2].order = 2. For example, when the bounding box is close to the shape of a cube, the Morton code order can be left unchanged. When the bounding box is close to the shape of a cuboid, the Morton code order can be changed.

[0352] According to the implementation, the Morton code generation order (S16000 and / or S16010) and the threshold of the Morton code generation order (S16020) can be used together. In addition, operations S16000, S16010 and S16020 can be applied individually and differently depending on the individual geometry, individual attributes or individual combinations of geometry and attributes.

[0353] Figure 19 A Morton code generator according to an implementation is shown.

[0354] Figure 19 The implementation of the reference is shown in the figure. Figure 15-18 The operation described is for the Morton code generator 19000.

[0355] Figure 19 The Morton code generator can correspond to or be included in Figure 1 The device includes a transmitting device 10000, a point cloud video encoder 10002, a receiving device 10004, and a point cloud video decoder 10006. Figure 2 Encoding 20001, Decoding 20003, Figure 4 PCC encoder Figure 10and Figure 11 PCC decoder Figure 12 Point cloud data transmission device Figure 13 Point cloud data receiving device, Figure 14 XR devices such as the 1730 and smartphones such as the 1440.

[0356] Figure 19 The implementation according to the embodiment is shown. Figure 16 Components of the PCC encoder and decoder for related operations.

[0357] The Morton code generator 19000 can perform operations such as generating Morton codes based on conditions according to the implementation method, and generating Morton codes for each geometry / attribute after checking a threshold.

[0358] The Morton code generator 19000 can be included in the PCC encoder according to the embodiment. The Morton code generator 19000 can generate Morton codes between the geometric coding unit and the attribute coding unit in the PCC encoder. The method / apparatus according to the embodiment can perform operations such as LOD generation based on the Morton codes generated according to the embodiment.

[0359] The Morton code generator 19000 can generate Morton codes based on conditions for each axis / density. These conditions can be system presets or user-input values. Furthermore, the transmission method / apparatus according to the implementation can transmit the bitstream (see...) Figure 20 The receiving method / device transmits parameter information related to the conditions of each axis / density (see) Figures 21-24 ).

[0360] The Morton code generator 19000 can be referred to as the Morton code generation unit, etc.

[0361] It can be done before geometry encoding (e.g., Figure 4 (the geometric encoding part) or after geometric encoding (e.g., in the geometric encoding part) Figures 20-24 The Morton code generator 19000 operates between the geometric encoding section and the attribute encoding section. For example, the Morton code generator can be connected to the geometric encoder according to an embodiment. According to an embodiment, configurations such as Morton code generator / geometric encoder and / or geometric encoder / Morton code generator can exist. Furthermore, on the receiving side corresponding to the transmitting side, the Morton code processor according to an embodiment can be connected to the geometric decoder. According to an embodiment, configurations such as Morton code processor / geometric decoder and / or geometric decoder / Morton code processor can exist.

[0362] The receiving method / apparatus according to the implementation can decode point cloud data based on the reverse process of the operation of the Morton code generator 19000.

[0363] The transmission method / apparatus according to the implementation can encode point cloud data based on the following parameters (which may be referred to as signaling information or metadata).

[0364] The method / apparatus according to the embodiment can perform the encoding operation according to the embodiment, generate the following parameters related to the encoding operation, and include the parameters in the bitstream (see Figures 20-24 The data is transmitted to the receiving method / apparatus according to the embodiment.

[0365] The transmission method / apparatus according to the implementation can determine conditions (axis_condition_selection) for each axis / density. The Morton code generator 19000 performs bit interleaving of the Morton code under predetermined conditions.

[0366] The transmission method / apparatus according to the implementation can define the bit interleaving order based on ascending / descending order (ascending_ascending_tag). The bit interleaving order can be passed from the encoder to the decoder via the bit stream (see...). Figures 20-24 For example, an ascending flag equal to 0 indicates ascending order. An ascending flag equal to 1 indicates descending order. Integer values ​​can be changed.

[0367] According to the method / apparatus of the implementation, it is possible to determine whether to generate Morton codes based on a threshold (axis_ordering_threshold) determined according to the data characteristics.

[0368] The application of Morton code generation conditions and thresholds can vary between sequences / pieces / slices. In this case, the values ​​used can be defined as signaling information (see...). Figure 20 ).

[0369] The decoder corresponding to the encoder according to the embodiment can perform all the operations of the encoder in the same way.

[0370] The conditional Morton code generator 19000 and the change of the order number based on the threshold can be applied before or after geometric encoding, or both before and after geometric encoding.

[0371] Figure 20 The configuration of the point cloud bitstream according to an implementation method is shown.

[0372] The method / apparatus according to the implementation method can generate and obtain, as shown in the example. Figure 1 The point cloud bitstream shown. For example, Figure 2 Transmitting device 10000, point cloud video encoder 10002, receiving device 10004, point cloud video decoder 10006. Figure 4 Encoding 20001, Decoding 20003, Figure 11 The encoding process Figure 12 Decoding processFigure 13 The transmitting device Figure 14 The receiving device Figure 19 XR device 1430, Figure 25 The Morton code generator 19000, respectively based on Figure 26 and Figure 20 The methods of implementation can generate (encode) and receive (decode) point cloud bitstreams including geometric information, attribute information and / or their metadata.

[0373] Information about the implementation method can be communicated using signals.

[0374] The following abbreviations are used in this disclosure: SPS (Sequence Parameter Set); GPS (Geometric Parameter Set); APS (Attribute Parameter Set); TPS (Patch Parameter Set); Geom (Geometric Bitstream (=Geometric Slice Header + Geometric Slice Data)); Attr (Attribute Bitstream (=Attribute Block Header + Attribute Block Data)). Here, a block may be referred to as a chunk, slice, etc.

[0375] The point cloud data, according to the implementation method, can be used as follows: Figure 1 The bitstream is shown in the form of a bitstream. Point cloud data may contain a Sequence Parameter Set (SPS), a Geometry Parameter Set (GPS), an Attribute Parameter Set (APS), and a Patch Parameter Set (TPS), which contain signaling information according to the implementation. Point cloud data may contain one or more geometries and / or attributes. Geometries and / or attributes in point cloud data may be divided into one or more slices (or blocks / blocks). Geometries may have a structure of a geometry slice header and geometry slice data. For example, a TPS containing signaling information may include Tile(0)_tile_bounding_box_xyz0 and Tile(0)_tile_bounding_box_whd. Geometries may include geom_geom_parameter_set_id, geom_tile_id, geom_slice_id, geomBoxOrigin, geom_box_log2_scale, geom_max_node_size_log2, and geom_num_points.

[0376] According to the implementation method, the signaling information can be added to SPS, GPS, APS, TPS, etc. and notified by signal.

[0377] According to the implementation method, signaling information can be added to the TPS, the Geom of each slice, or the Attr of each slice and notified by signal.

[0378] The structure of point cloud data according to the implementation method can efficiently provide a set of parameters, geometry, and attributes, including signaling information, in terms of encoding / decoding / data.

[0379] The point cloud data associated with the point cloud data transmitting / receiving device according to the embodiments may include at least one of sequence parameters, geometric parameters, attribute parameters, tile parameters, geometric bitstream, or attribute bitstream.

[0380] The syntax of a specific signaling information will now be described with reference to the accompanying drawings. For reference, the name of the signaling information according to the implementation may be changed / expanded within the intended meaning / function of the signaling information. Fields of the signaling information may be distinguishably referred to as first signaling information, second signaling information, etc.

[0381] As described above, the point cloud data transmission device (e.g., reference) Figure 11 , Figure 14 , Figure 1 and Figure 1 The described point cloud data transmission apparatus can transmit encoded point cloud data in the form of a bit stream. According to an embodiment, the bit stream may include one or more sub-bit streams.

[0382] Considering errors in the transmission channel, point cloud data transmission devices (e.g., Figure 4 , Figure 12 , Figure 19 and Figures 21-24 The point cloud data transmission apparatus described herein can divide an image of point cloud data into one or more packets and transmit these packets over a network. According to an embodiment, the bitstream may include one or more packets (e.g., Network Abstraction Layer (NAL) units). Therefore, even if some packets are lost in a poor network environment, the point cloud data receiving apparatus can use the remaining packets to reconstruct the image. Point cloud data can be segmented into one or more slices or one or more tiles to be processed. According to the embodiment, the tiles and slices are regions used to perform point cloud compression encoding by segmenting the image of the point cloud data. The point cloud data transmission apparatus can provide high-quality point cloud content by processing data corresponding to each region according to the importance of each partition region of the point cloud data. That is, the point cloud data transmission apparatus can perform point cloud compression encoding on data corresponding to regions important to the user with better compression efficiency and appropriate latency.

[0383] According to the embodiments, the image (or scene) of the point cloud content is segmented into basic processing units for point cloud compression encoding. The basic processing units for point cloud compression encoding according to the embodiments may include, but are not limited to, coding tree units (CTUs) and blocks (= slices).

[0384] A slice according to an embodiment is a region comprising basic processing units for one or more point cloud compression encodings and does not have a rectangular shape. A slice according to an embodiment contains data transmitted in groups. A tile according to an embodiment is a region in an image divided into rectangular shapes and includes basic processing units for one or more point cloud compression encodings. A slice according to an embodiment may be included in one or more tiles. Alternatively, a tile according to an embodiment may be included in one or more slices.

[0385] According to the implementation, the bitstream may contain signaling information and one or more slices. The signaling information includes a sequence parameter set (SPS) for sequence-level signaling, a geometric parameter set (GPS) for geometric information-encoded signaling, an attribute parameter set (APS) for attribute information-encoded signaling, and a tile parameter set (TPS) for tile-level signaling.

[0386] According to the implementation method, the SPS can be encoded information of the entire sequence, such as profiles and levels, and can include comprehensive information about the entire file, such as screen resolution and video format.

[0387] A slice (e.g., slice 0) according to an embodiment includes a slice header and slice data. The slice data may include a geometry bitstream (Geom00) and one or more attribute bitstreams (Attr00 and Attr10). The geometry bitstream may include a header (e.g., a geometry slice header) and a payload (e.g., geometry slice data). The header of the geometry bitstream according to an embodiment may include identification information about the parameter set included in the GPS (geom_geom_parameter_set_id), tile identifier (geom_tile id), slice identifier (geom_slice_id), origin information about the geometry box (geomBoxOrigin), geometry box scale (geom_box_log2_scale), maximum geometry node size (geom_max_node_size_log2), number of geometry points (geom_num_poins), and information about the data included in the payload. The attribute bitstream may include a header (e.g., an attribute slice header or an attribute block header) and a payload (e.g., attribute slice data or attribute block data).

[0388] Parameter information related to the implementation can be transmitted in various regions (units) such as sequences, geometry, attributes, tiles, and slices, and the receiving method / apparatus according to the implementation can access point cloud data statically / dynamically.

[0389] Relevant information can be communicated using signals to implement the implementation method. The signaling information according to the implementation method can be used on either the transmitting or receiving side.

[0390] Geometry data (Geom): Geometry bitstream = Geometry slice header + Geometry slice data

[0391] Attribute data (Attr): Attribute bitstream = Attribute slice header + Attribute slice data (Attribute block data).

[0392] The method / apparatus according to the embodiments provides tiles or slices, such that point clouds can be divided into regions and processed. When the point cloud is divided into multiple regions, each region can have different importance.

[0393] By providing such regions, different filters and different filter units can be applied according to their importance, allowing for the use of highly complex yet high-quality filtering methods in important regions.

[0394] When dividing point cloud data into regions, there is an option to generate different sets of neighbor points for each region. This results in either slightly lower reliability but lower complexity, or higher complexity but higher reliability. This can be configured according to the processing capabilities of the receiver.

[0395] By applying different filters to individual regions (divided into patches or slices) based on receiver capabilities, rather than applying complex filtering methods to the entire point cloud data, better image quality can be ensured in areas important to the user, and appropriate latency can be guaranteed within the system.

[0396] When dividing point cloud data into tiles according to the method / apparatus of the implementation, different filters or different filter units can be applied to each tile.

[0397] When dividing point cloud data into slices according to the method / apparatus of the implementation, different filters or different filter units can be applied to each slice.

[0398] When a point cloud is divided into tiles, signaling information can be sent to each tile. When a point cloud is divided into slices, signaling information can be sent to each slice.

[0399] The signaling information, depending on the implementation method, may be selectively or redundantly included in various locations.

[0400] The sequence according to the implementation may have multiple pieces, each piece may have multiple slices, and each slice may include an octree. The SPS is at the sequence level, and the APS is also at the sequence level. The SPS contains more general information. The ASP contains more specific information in the attributes. In the TPS, different techniques can be applied to each piece when the sequence is divided into pieces. When information configured in both the APS and TPS exists, the APS can use the default configuration. When no configuration information exists in the TPS, the default value can be used. When configuration information exists in the TPS, the receiving method / apparatus according to the implementation can use the TPS information. In the attribute slice header, each piece can be divided into slices. That is, configuration information can be configured for each slice.

[0401] Figure 20 It shows that it includes Figure 21 The detailed syntax of signaling information in the bitstream of point cloud data. To support operation according to the embodiment, the transmitting / receiving apparatus according to the embodiment can provide the following signaling scheme. The name of the signaling information can be understood within the scope of its meaning and function.

[0402] Signaling information related to the conditions and thresholds for the Morton code generation order according to the implementation can be transmitted in the bitstream. To perform operations related to the conditions and thresholds for the Morton code generation order, parameter information related to the conditions and thresholds used for generation can be signaled.

[0403] In the following text, the signaling information of the encoder / decoder according to the Morton code generation order will be described based on the parameter information according to the implementation method.

[0404] Signaling operations can be added to the encoder / decoder based on the thresholds of the conditions used for Morton code generation.

[0405] Figure 21 The tile parameter set (TPS) according to the implementation method is shown.

[0406] Figure 20 It shows that it includes Figure 22 TPS in the middle.

[0407] The transmitting method / apparatus according to the embodiments can add Morton code generation-related information to the TPS and transmit the information, and the receiving method / apparatus according to the embodiments can obtain Morton code generation-related information based on the TPS.

[0408] `axis_condition_selection` indicates the method for selecting axis conditions used in Morton code generation. It can indicate an axis-based condition, a density-based condition, or other conditions. For example, 0 can indicate sorting by axis (S16000), 1 can indicate sorting by density (S16010), and 2 can indicate using other methods. Integer values ​​can be changed.

[0409] The `ascending_order_flag` indicates whether the order of operations according to the condition indicated by `axis_condition_selection` is ascending or descending (sorting order). For example, 0 indicates that sorting is performed in ascending order, while 1 indicates that sorting is performed in descending order. Integer values ​​can be changed.

[0410] axis_ordering_threshold indicates whether to perform conditional Morton interleaving by applying a threshold based on certain conditions determined in relation to the Morton code.

[0411] `num_tiles` specifies the number of tiles that are signaled for the bitstream. If it does not exist, `num_tiles` can be presumed to be 0.

[0412] Based on num_tiles, the following tile-related parameter information can be included in the tile's TPS for each quantity.

[0413] tile_bounding_box_offset_x[i] represents the x-offset of the i-th tile in Cartesian coordinates. When it does not exist, the value of tile_bounding_box_offset_x[0] can be presumed to be sps_bounding_box_offset_x.

[0414] `tile_bounding_box_offset_y[i]` represents the y-offset of the i-th tile in Cartesian coordinates. When it does not exist, the value of `tile_bounding_box_offset_y[0]` can be presumed to be `sps_bounding_box_offset_y`.

[0415] tile_bounding_box_offset_z[i] represents the z-offset of the i-th tile in Cartesian coordinates. When it does not exist, the value of tile_bounding_box_offset_z[0] can be presumed to be sps_bounding_box_offset_z.

[0416] `tile_bounding_box_scale_factor[i]` indicates the scaling factor of the i-th tile in Cartesian coordinates. When it does not exist, the value of `tile_bounding_box_scale_factor[0]` can be presumed to be `sps_bounding_box_scale_factor`.

[0417] `tile_bounding_box_size_width[i]` represents the width of the i-th tile in Cartesian coordinates. When it does not exist, the value of `tile_bounding_box_size_width[0]` can be presumed to be `sps_bounding_box_size_width`.

[0418] `tile_bounding_box_size_height[i]` represents the height of the i-th tile in Cartesian coordinates. When it does not exist, the value of `tile_bounding_box_size_height[0]` can be presumed to be `sps_bounding_box_size_height`.

[0419] `tile_bounding_box_size_depth[i]` indicates the depth of the i-th tile in Cartesian coordinates. When it does not exist, the value of `tile_bounding_box_size_depth[0]` can be presumed to be `sps_bounding_box_size_depth`.

[0420] Figure 22 The geometry parameter set (GPS) according to the implementation method is shown.

[0421] Figure 20 It shows that it includes Figure 23 GPS in the system.

[0422] The transmitting method / apparatus according to the embodiments can add Morton code generation-related information to GPS and transmit the information, and the receiving method / apparatus according to the embodiments can obtain Morton code generation-related information based on GPS.

[0423] `axis_condition_selection` indicates the method for selecting axis conditions used in Morton code generation. It can indicate an axis-based condition, a density-based condition, or other conditions. For example, 0 can indicate sorting by axis (S16000), 1 can indicate sorting by density (S16010), and 2 can indicate using other methods. Integer values ​​can be changed.

[0424] The `ascending_order_flag` indicates whether the order of operations according to the condition indicated by `axis_condition_selection` is ascending or descending (sorting order). For example, 0 indicates that sorting is performed in ascending order, while 1 indicates that sorting is performed in descending order. Integer values ​​can be changed.

[0425] axis_ordering_threshold indicates whether to perform conditional Morton interleaving by applying a threshold based on certain conditions determined in relation to the Morton code.

[0426] `gps_geom_parameter_set_id` provides an identifier for GPS for reference by other syntax elements. The value of `gps_seq_parameter_set_id` should be in the range of 0 to 15 (inclusive).

[0427] The `gps_seq_parameter_set_id` specifies the value of `sps_seq_parameter_set_id` for the active SPS. The value of `gps_seq_parameter_set_id` should be in the range of 0 to 15 (inclusive).

[0428] The given value of `geometry_coding_type` indicates the encoding type used for the geometry. In a bitstream conforming to this version of the specification, the value of `geometry_coding_type` should be equal to 0 or 1. Other values ​​of `geometry_coding_type` are reserved by ISO / IEC for later use: 0 = octree; 1 = trisoup. Decoders conforming to this version of the specification should ignore the reserved values ​​of `geometry_coding_type`.

[0429] A gps_box_present_flag value of 1 specifies that additional bounding box information is provided in the geometry head referencing the current GPS. A gps_bounding_box_present_flag value of 0 specifies that no additional bounding box information is signaled in the geometry head.

[0430] A unique_geometry_points_flag value of 1 indicates that all output points have unique locations. A unique_geometry_points_flag value of 0 indicates that output points may have the same location.

[0431] A `neighbour_context_restriction_flag` of 0 indicates that the octree encoding uses the context determined by the 6 neighboring parent nodes. A `neighbour_context_restriction_flag` of 1 indicates that the octree encoding uses only the context determined by the sibling nodes.

[0432] A flag of 0 for `incided_direct_coding_mode_enabled_flag` indicates that octree encoding uses `incided_direct_coding_mode`. A flag of 1 for `incided_direct_coding_mode_enabled_flag` indicates that octree encoding uses multiple contexts determined based on sibling nodes.

[0433] A bitwise_occupancy_coding_flag of 1 indicates that geometry node occupancy is encoded using bitwise contextualization of the syntactic element occupancy_map. A bitwise_occupancy_coding_flag of 0 indicates that geometry node occupancy is encoded using dictionary-encoded syntactic element occypancy_byte.

[0434] The `child_neighbors_enabled_flag` indicates whether child neighbor nodes are enabled.

[0435] A `adjacent_child_contextualization_enabled_flag` equal to 1 indicates that the neighboring child nodes of the neighboring octree node are used for bit-by-bit context handling. A `adjacent_child_contextualization_enabled_flag` equal to 0 indicates that the child nodes of the neighboring octree node are not used for this context handling.

[0436] geom_occupancy_ctx_reduction_factor indicates the reduction factor used for occupancy.

[0437] The value of the variable neighborravailboundary used during the decoding process can be specified as follows: neighborravailboundary = 2log2_neighbor_avail_boundary.

[0438] When neighbor_context_restriction_flag equals 1, NeighbAvailabilityMask(NeighbAvailBoundary) can be set to 13. When neighbor_context_restriction_flag equals 0, NeighbAvailabilityMask can be set to (1 < ... ) <log2_neighbour_avail_boundary)。

[0439] log2_intra_pred_max_node_size indicates the size of the octree nodes used for intra-frame prediction.

[0440] The variable `trisoupnodesize` can be specified as the size of the triangle node as follows: `trisoupnodesize = 2log2_trisoup_node_size`. The value of `log2_trisoup_node_size` should be equal to or greater than 0. When `log2_trisoup_node_size` is equal to 0, the geometric bitstream may contain only the octree-coded syntax.

[0441] `trisoup_depth` specifies the number of bits used to represent each component of the point coordinates. The value of `trisoup_depth` should be in the range of 2 to 21. [Ed(df): 21 should be a level limit].

[0442] The `trisoup_triangle_level` specifies the level at which the octree is pruned. The value of `trisoup_triangle_level` should be in the range of 1 to `trisoup_depth-1`.

[0443] A `gps_extension_present_flag` value of 1 indicates that the `GPS_extension_data` syntax structure exists within the `GPSRBSP` syntax structure. A `gps_extension_present_flag` value of 0 indicates that this syntax structure does not exist. When it does not exist, the value of `gps_extension_present_flag` is presumed to be 0.

[0444] The `gps_extension_data_flag` indicates the presence of extension data. `gps_extension_data_flag` can have any value. Its presence and value do not affect the decoder's conformance to the configuration file specified in Appendix A.

[0445] Figure 23 The attribute parameter set (APS) according to the implementation method is shown.

[0446] Figure 20 It shows that it includes Figure 24 APS in the middle.

[0447] The transmitting method / apparatus according to the embodiment can add Morton code generation related information to the APS and transmit the information, and the receiving method / apparatus according to the embodiment can obtain Morton code generation related information based on the APS.

[0448] `axis_condition_selection` indicates the method for selecting axis conditions used in Morton code generation. It can indicate an axis-based condition, a density-based condition, or other conditions. For example, 0 can indicate sorting by axis (S16000), 1 can indicate sorting by density (S16010), and 2 can indicate using other methods. Integer values ​​can be changed.

[0449] The `ascending_order_flag` indicates whether the order of operations according to the condition indicated by `axis_condition_selection` is ascending or descending (sorting order). For example, 0 indicates that sorting is performed in ascending order, while 1 indicates that sorting is performed in descending order. Integer values ​​can be changed.

[0450] axis_ordering_threshold indicates whether to perform conditional Morton interleaving by applying a threshold based on certain conditions determined in relation to the Morton code.

[0451] `aps_attr_parameter_set_id` provides an identifier for the APS for reference by other syntax elements. The value of `aps_attr_parameter_set_id` should be in the range of 0 to 15 (inclusive).

[0452] The aps_seq_parameter_set_id parameter can be used to specify the value of sps_seq_parameter_set_id for an active SPS. The value of aps_seq_parameter_set_id should be in the range of 0 to 15 (inclusive).

[0453] The given value for `attr_coding_type` indicates the encoding type used for the attributes in Table 72. In this version of the bitstream conforming to this specification, the value of `attr_coding_type` should be equal to 0, 1, or 2: 0 = Predicted weight boost; 1 = Region Adaptive Layered Transport (RAHT); 2 = Fixed weight boost. Other values ​​for `attr_coding_type` are reserved by ISO / IEC for future use. Decoders conforming to this specification in this version should ignore the reserved `attr_coding_type` values.

[0454] `num_pred_nearest_neighbours` specifies the maximum number of nearest neighbors to use for prediction. The value of `numberOfNearestNeighboursInPrediction` should be in the range of 1 to xx.

[0455] `max_num_direct_predictors` specifies the maximum number of predictors to be used for direct prediction. The value of `max_num_direct_predictors` should be in the range of 0 to `num_pred_nearest_neighbors`. The value of the variable `maxnumpredictors` that can be used during decoding is as follows: `maxnumpredictors = max_num_direct_predictors + 1`.

[0456] lifting_search_range specifies the search range to be used for lifting.

[0457] `lifting_quant_step_size` specifies the quantization step size for the first component of the attribute. The value of `quant_step_size` should be in the range of 1 to xx.

[0458] `lifting_quant_step_size_chroma` specifies the quantization step size of the chromaticity component of the attribute when the attribute is in color. The value of `quant_step_size_chroma` should be in the range of 1 to xx.

[0459] The `lod_binary_tree_enabled_flag` flag specifies whether a binary tree is enabled for generating logs.

[0460] `num_detail_levels_minus1` specifies the number of Levels of Detail (LODs) used for attribute encoding. The value of `num_detail_levels_minus1` should be in the range of 0 to `xx`.

[0461] `sampling_distance_squared[idx]` specifies the square of the sampling distance used for `idx`. The value of `sampling_distance_squared[]` should be in the range of 0 to `xx`.

[0462] adaptive_prediction_threshold specifies the threshold for prediction.

[0463] `raht_depth` specifies the number of detail layers in the RAHT. The value of `depthRAHT` should be in the range of 1 to xx.

[0464] `raht_binarylevel_threshold` specifies the level of detail for clipping the RAHT coefficients. The value of `binaryLevelThresholdRAHT` should be in the range of 0 to xx.

[0465] `raht_quant_step_size` specifies the quantization step size for the first component of the attribute. The value of `quant_step_size` should be in the range of 1 to `xx`.

[0466] An aps_extension_present_flag value of 1 indicates that the aps_extension_data syntax structure exists in the APS RBSP syntax structure. An aps_extension_present_flag value of 0 indicates that this syntax structure does not exist. When it does not exist, the value of aps_extension_present_flag can be presumed to be 0.

[0467] The `aps_extension_data_flag` indicates the presence of APS extension data. `aps_extension_data_flag` can have any value. Its presence and value do not affect the decoder's conformity to the configuration file specified in Appendix A.

[0468] Figure 25 A geometry slice head (GSH) according to an implementation method is shown.

[0469] The sending method / apparatus according to the embodiment can add Morton code generation related information to GSH and send the information, and the receiving method / apparatus according to the embodiment can obtain Morton code generation related information based on GSH.

[0470] `axis_condition_selection` indicates the method for selecting axis conditions used in Morton code generation. It can indicate an axis-based condition, a density-based condition, or other conditions. For example, 0 can indicate sorting by axis (S16000), 1 can indicate sorting by density (S16010), and 2 can indicate using other methods. Integer values ​​can be changed.

[0471] The `ascending_order_flag` indicates whether the order of operations according to the condition indicated by `axis_condition_selection` is ascending or descending (sorting order). For example, 0 indicates that sorting is performed in ascending order, while 1 indicates that sorting is performed in descending order. Integer values ​​can be changed.

[0472] axis_ordering_threshold indicates whether to perform conditional Morton interleaving by applying a threshold based on certain conditions determined in relation to the Morton code.

[0473] gsh_geometry_parameter_set_id specifies the value of gps_geom_parameter_set_id for the active GPS.

[0474] gsh_tile_id specifies the tile ID.

[0475] gsh_slice_id specifies the slice ID.

[0476] Depending on the value of gps_box_present_flag, the following parameters related to the box in GSH can be included in GSH.

[0477] `gsh_box_log2_scale` specifies the scaling value. It represents the scaling factor of the origin of the piece bounding box. When it does not exist, `gsh_box_log2_scale` can be presumed to be `gps_gs_box_log2_scale`.

[0478] gsh_box_origin_x specifies the x-axis of the source bounding box in Cartesian coordinates.

[0479] gsh_box_origin_y specifies the y-coordinate of the source bounding box in Cartesian coordinates.

[0480] gsh_box_origin_z specifies the z-axis of the source bounding box in Cartesian coordinates.

[0481] gsh_log2_max_nodesize specifies the value of the variable maxnodesize used during decoding. maxnodesize can be equal to 2 (gbh_log2_max_nodesize).

[0482] gbh_points_number specifies the number of coded points in a slice.

[0483] Figure 1 A method for transmitting point cloud data according to an embodiment is shown.

[0484] S25000: The method for transmitting point cloud data according to the embodiment may include encoding the point cloud data. The encoding operation may include the following operations: transmitting device 10000, Figure 2 Point cloud video encoder 10002, obtain 20000, Figure 4 The code 20001 comes from Figure 4 Encoding of the geometric bitstream of the position, from Figure 12 The encoding of the attribute of the bit stream, Figure 14 The transmitting device encodes the map position values ​​and / or the attribute values ​​of points. Figures 15-19 XR device 1430, according to Figures 20-24 The generation of Morton codes and the encoding of point cloud data based on Morton codes, and Figure 1 The generation of bitstreams.

[0485] S25010: The method for transmitting point cloud data may further include transmitting a bit stream including the point cloud data. The transmission operation may include... Figure 2 Transmitter 10003 Figure 12 Transmission 20002, Figure 14 Transmission processor 12012, Figures 20-24 XR device 1430 and Figure 26 The operation of sending the bit stream.

[0486] Figure 1 A method for receiving point cloud data according to an embodiment is shown.

[0487] S26000: The method for receiving point cloud data according to the embodiment may include receiving a bit stream including point cloud data. The receiving operation may include... Figure 2 The receiving device 10004, according to Figure 13 The transmission of 20002 and the reception of Figure 14 Receiver 13000 and receiver processor 13001 Figures 20-24 The XR device 1430 receives, and Figure 1 The operation of receiving bit streams.

[0488] S26010: The method for receiving point cloud data may further include decoding the point cloud data. The decoding operation may include the following operations: Figure 2 Point cloud video decoder 10006 Figure 10 Decoding 20003, from Figure 11 and Figure 13 Decoding of the geometric bitstream and / or attribute bitstream position and / or attributes, by Figure 14 The receiving device decodes the geometric bitstream and attribute bitstream, and presents the point cloud data. Figures 15-19 Decoding of the XR device 1430, based on... Figures 20-24 Morton code generation and decoding of point cloud data, and ​ The bitstream is parsed / decoded.

[0489] The method / apparatus according to the embodiments provides a method for generating adaptive Morton codes, conditions for changing the order of Morton codes, and operations related to their thresholds. In the operation of generating different Morton codes according to the conditions, the conditions include, for example, ascending / descending order based on individual axes / densities. The conditions can be applied differently depending on the characteristics of the content, individual sequences / slices / pieces, geometry, attributes, or both geometry and attributes.

[0490] The method / apparatus according to the implementation can improve image quality by reducing the size of the bitstream when searching for neighboring nodes based on distance / attribute. Furthermore, Morton codes can be generated by relevant signaling based on the same conditions during the encoding / decoding process, using sequence and thresholds. Based on Morton codes, the encoder / transmitter can efficiently compress point cloud data. Based on Morton codes, the decoder / receiver can efficiently reconstruct and reproduce point cloud data.

[0491] The point cloud data transmission / reception method / apparatus and / or PCC encoder / decoder according to the embodiments can solve the problem of Morton code differences that occur when the Euclidean distances between points in the point cloud data are spatially equal.

[0492] Furthermore, by changing the order of the xyz axes according to the axes, encoding and decoding can be performed based on individual content units, tiles, slices, geometric values, or attribute values. Moreover, the axis transformation can be changed depending on a threshold. Therefore, implementations can provide mapping conditions for adaptive Morton codes to search for nearby neighbor nodes, while performing adaptive spatial search on a per-content, per-tile, per-slice, per-range, or per-attribute basis.

[0493] This can reduce the time and complexity of point cloud compression and PCC data encoding processing, such as Morton code generation, depending on the type of point cloud data. Additionally, it can provide point cloud transmission / reception methods / apparatus that improve compression performance.

[0494] The operation according to the implementation method can reduce waiting time and increase the accuracy of neighbor node search. In addition, it can reduce the burden of the bitstream size including point cloud data.

[0495] The method / apparatus according to the implementation can change the bit interleaving order of the Morton code by reflecting the characteristics of various point cloud data.

[0496] In this document, “A / B” is interpreted as A and / or B.

[0497] The implementation has been described from the perspective of methods and / or apparatus, and the descriptions of methods and apparatus may be applied to complement each other.

[0498] Although the accompanying drawings are described separately for simplicity, new embodiments can be designed by combining the embodiments shown in the various figures. Recording media containing programs for executing the above embodiments, designed to be computer-readable by those skilled in the art, also fall within the scope of the appended claims and their equivalents. The apparatus and methods according to the embodiments are not limited to the configurations and methods of the above embodiments. Various modifications can be made to the embodiments by selectively combining all or some of the embodiments. Although preferred embodiments have been described with reference to the accompanying drawings, those skilled in the art will understand that various modifications and variations can be made to the embodiments without departing from the spirit or scope of this disclosure as described in the appended claims. These modifications should not be understood solely from the technical concept or viewpoint of the embodiments.

[0499] Various elements of the apparatus according to the embodiments can be implemented by hardware, software, firmware, or a combination thereof. Various elements in the embodiments can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiments, the components according to the embodiments can be implemented as separate chips. According to the embodiments, at least one or more components of the apparatus according to the embodiments can include one or more processors capable of executing one or more programs. One or more programs can perform any one or more operations / methods according to the embodiments or include instructions for performing them. Executable instructions for performing the methods / operations of the apparatus according to the embodiments can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transient CRM or other computer program product configured to be executed by one or more processors. Additionally, the memory according to the embodiments can be used as a concept encompassing not only volatile memory (e.g., RAM) but also non-volatile memory, flash memory, and PROM. It can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, the processor-readable recording medium can be distributed to computer systems connected via a network, such that processor-readable code can be stored and executed in a distributed manner.

[0500] In this specification, the terms “ / ” and “,” should be interpreted as indicating “and / or”. For example, the expression “A / B” can mean “A and / or B”. Furthermore, “A, B” can mean “A and / or B”. Additionally, “A / B / C” can mean “at least one of A, B, and / or C”. Also, “A / B / C” can mean “at least one of A, B, and / or C”. Furthermore, in this specification, the term “or” should be interpreted as indicating “and / or”. For example, the expression “A or B” can mean 1) only A, 2) only B, or 3) both A and B. In other words, the term “or” as used in this document should be interpreted as indicating “additionally or alternatively”.

[0501] Terms such as "first" and "second" can be used to describe various elements of the embodiments. However, the various components according to the embodiments should not be limited by the terms used above. These terms are used only to distinguish one element from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should not be interpreted outside the scope of the various embodiments. Both the first user input signal and the second user input signal are user input signals, but they do not mean the same user input signal unless the context clearly specifies otherwise.

[0502] The terminology used to describe embodiments is for the purpose of describing particular embodiments and is not intended to limit the embodiments. As used in the description of embodiments and claims, the singular form includes the plural of the referred objects unless the context clearly specifies otherwise. The expression “and / or” is used to include all possible combinations of terms. Terms such as “comprising” or “having” are intended to indicate the presence of figures, quantities, steps, elements, and / or components and should be understood not to exclude the possibility of additional figures, quantities, steps, elements, and / or components. As used herein, conditional expressions such as “if” and “when” are not limited to optional cases and are intended to be interpreted as performing the relevant operation when a specific condition is met, or to interpret the relevant definition based on a specific condition.

[0503] The operations described in this specification according to embodiments can be performed by a transmitting / receiving device including a memory and / or a processor according to embodiments. The memory may store programs for processing / controlling the operations according to embodiments, and the processor may control the various operations described in this specification. The processor may be referred to as a controller, etc. In embodiments, the operations may be performed by firmware, software, and / or combinations thereof. Firmware, software, and / or combinations thereof may be stored in a processor or memory.

[0504] Invention Model

[0505] As described above, the relevant content is presented in the best mode for implementing the embodiment.

[0506] Industrial applicability

[0507] As described above, the implementation methods can be applied in whole or in part to point cloud data transmission / reception devices and systems.

[0508] It will be apparent to those skilled in the art that various changes or modifications can be made to the implementation within the scope of the implementation.

[0509] Therefore, the embodiments are intended to cover modifications and variations of this disclosure, provided they fall within the scope of the appended claims and their equivalents.

Claims

1. A method for transmitting point cloud data, the method comprising: encoding point cloud data; and transmitting a bitstream comprising the point cloud data, wherein the step of encoding the point cloud data comprises: generating a Morton code for the point cloud data based on changing an order of axes for the Morton code, wherein the bitstream comprises information related to the changed order of axes and information for representing a threshold related to an ascending order based on axes for geometry data.

2. The method of claim 1, wherein, generating the Morton code comprises: generating an order of the Morton code based on respective axes of geometry data for the point cloud data and a group size comprising a number of points of the point cloud data.

3. The method of claim 2, wherein, the geometry data contains coordinate values on respective axes, wherein the Morton code is generated based on binary bits of the coordinate values on respective axes, and wherein the binary bits of the Morton code are ordered based on lengths of the axes.

4. The method of claim 1, wherein, generating the Morton code comprises: generating an order of the Morton code based on a density of geometry data for the point cloud data.

5. The method of claim 4, wherein, the geometry data contains coordinate values on respective axes, wherein the Morton code is generated based on binary bits of the coordinate values, and wherein the binary bits of the Morton code are ordered based on a density of one or more points with respect to respective axes.

6. The method of claim 1, wherein, generating the Morton code is performed based on a threshold related to a distribution of one or more points of the point cloud data.

7. The method of claim 1, wherein, the bitstream comprises one or more parameters for signaling the point cloud data.

8. An apparatus for transmitting point cloud data, the apparatus comprising: an encoder configured to encode point cloud data; and a transmitter configured to transmit a bitstream comprising the point cloud data, wherein the encoder performs generating a Morton code for the point cloud data based on changing an order of axes for the Morton code, wherein the bitstream comprises information related to the changed order of axes and information for representing a threshold related to an ascending order based on axes for geometry data.

9. The apparatus of claim 8, wherein, the encoder generates an order of the Morton code based on respective axes of geometry data for the point cloud data and a group size comprising a number of points of the point cloud data.

10. The apparatus of claim 9, wherein, the geometry data contains coordinate values on respective axes, wherein the Morton code is generated based on binary bits of the coordinate values on respective axes, and wherein the binary bits of the Morton code are ordered based on lengths of the axes.

11. The apparatus of claim 8, wherein, the encoder generates an order of the Morton code based on a density of geometry data for the point cloud data.

12. The apparatus of claim 11, wherein, the geometry data contains coordinate values on respective axes, wherein the Morton code is generated based on binary bits of the coordinate values, and wherein the binary bits of the Morton code are ordered based on a density of one or more points with respect to respective axes.

13. The apparatus of claim 8, wherein, the encoder generates the Morton code based on a threshold related to a distribution of one or more points of the point cloud data.

14. The apparatus of claim 8, wherein, the bitstream comprises one or more parameters for signaling the point cloud data.

15. A method for receiving point cloud data, the method comprising: receiving a bitstream comprising point cloud data; and decoding the point cloud data, wherein the step of decoding the point cloud data comprises: generating the Morton code for the point cloud data based on changing an order of axes for the Morton code, wherein the bitstream comprises information related to the changed order of axes and information for representing a threshold related to an ascending order of axes for geometry data.

16. The method of claim 15, wherein, generating an order of the Morton code based on respective axes for geometry data of the point cloud data and a group size comprising a number of points of the point cloud data.

17. An apparatus for receiving point cloud data, the apparatus comprising: a receiver configured to receive a bitstream comprising point cloud data; and a decoder configured to decode the point cloud data, wherein the decoder performs generating the Morton code for the point cloud data based on changing an order of axes for the Morton code, wherein the bitstream comprises information related to the changed order of axes and information for representing a threshold related to an ascending order of axes for geometry data.

18. The apparatus of claim 17, wherein, generating an order of the Morton code based on respective axes for geometry data of the point cloud data and a group size comprising a number of points of the point cloud data.

Images