Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Abstract

The point cloud data transmission method according to the embodiment of the present invention may comprise the following steps: encoding point cloud data; and transmitting the bit stream including the point cloud data. A point cloud data receiving method according to an embodiment may comprise the steps of: receiving a bitstream including point cloud data; and decoding the point cloud data.

Description

Technical Field

[0001] The embodiments relate to a method and an apparatus for processing point cloud content. Background Art

[0002] Point cloud content is content represented by a point cloud, which is a set of points belonging to a coordinate system representing a three-dimensional space. Point cloud content can represent media configured in three dimensions and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and self-driving services. However, tens of thousands to hundreds of thousands of point data are required to represent point cloud content. Therefore, a method for effectively processing a large amount of point data is needed. Summary of the Invention

[0003] [Technical Problem]

[0004] Embodiments provide an apparatus and a method for effectively processing point cloud data. Embodiments provide a method and an apparatus for processing point cloud data to solve latency and encoding / decoding complexity.

[0005] The technical scope of the embodiments is not limited to the above technical objects and may be extended to other technical objects that those skilled in the art can infer based on all the content disclosed herein.

[0006] [Technical Solution]

[0007] In one aspect of the present disclosure, a method of transmitting point cloud data may include encoding the point cloud data and transmitting a bitstream including the point cloud data. In another aspect of the present disclosure, a method of receiving point cloud data may include receiving a bitstream including the point cloud data and decoding the point cloud data.

[0008] [Advantageous Effects]

[0009] The apparatus and method according to the embodiments can efficiently process point cloud data.

[0010] The apparatus and method according to the embodiments can provide high-quality point cloud services.

[0011] The apparatus and method according to the embodiments can provide point cloud content for providing general services such as VR services and self-driving services. Brief Description of the Drawings

[0012] The drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of this application. The drawings illustrate embodiments of the present disclosure and, together with the description, are used to explain the principles of the present disclosure. To better understand the various embodiments described below, reference should be made to the following embodiments described in conjunction with the drawings. In the drawings, the same reference numerals will be used to refer to the same or similar parts.

[0013] Figure 1 Shows an exemplary point cloud content providing system according to an embodiment;

[0014] Figure 2 Is a block diagram illustrating an operation of providing point cloud content according to an embodiment;

[0015] Figure 3 Illustrates an exemplary point cloud encoder according to an embodiment;

[0016] Figure 4 Shows an example of an octree and occupancy code according to an embodiment;

[0017] Figure 5 Illustrates an example of point configuration in each LOD according to an embodiment;

[0018] Figure 6 Illustrates an example of point configuration in each LOD according to an embodiment;

[0019] Figure 7 Illustrates a point cloud decoder according to an embodiment;

[0020] Figure 8 Illustrates an exemplary transmitting device according to an embodiment;

[0021] Figure 9 Illustrates an exemplary receiving device according to an embodiment;

[0022] Figure 10 Illustrates an exemplary structure operable in combination with a point cloud data transmitting / receiving method / device according to an embodiment;

[0023] Figure 11 Illustrates I-frames and P-frames in each group of pictures (GoP) according to an embodiment;

[0024] Figure 12 Illustrates a context-based successive coding and decoding method according to an embodiment;

[0025] Figure 13 Illustrates a context-based successive coding and decoding method according to an embodiment;

[0026] Figure 14 Illustrates a context-based successive coding and decoding method according to an embodiment;

[0027] Figure 15 Illustrates the relationship between context dependency flags according to an embodiment;

[0028] FIG. 16A and FIG. 16B show a point cloud data encoding device according to an embodiment;

[0029] FIG. 17A and FIG. 17B show a point cloud data decoding device according to an embodiment;

[0030] Figure 18 shows a bitstream including point cloud data and parameters according to an embodiment;

[0031] Figure 19 shows a Sequence Parameter Set (SPS) in the bitstream according to an embodiment;

[0032] Figure 20 illustrates a Tile Parameter Set (TPS) in the bitstream according to an embodiment;

[0033] Figure 21 illustrates a Geometry Parameter Set (GPS) in the bitstream according to an embodiment;

[0034] Figure 22 illustrates an Attribute Parameter Set (APS) in the bitstream according to an embodiment;

[0035] Figure 23 illustrates a Geometry Slice Header (GSH) in the bitstream according to an embodiment;

[0036] Figure 24 illustrates a method for encoding point cloud data according to an embodiment; and

[0037] Figure 25 illustrates a method for decoding point cloud data according to an embodiment. Detailed Description

[0038] Now, reference will be made in detail to the preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The following detailed description made with reference to the accompanying drawings is intended to explain the exemplary embodiments of the present disclosure, and not to show the only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details.

[0039] Although most of the terms used in the present disclosure have been selected from the general terms widely used in the art, some terms have been arbitrarily selected by the applicant, and their meanings will be explained in detail as needed in the following description. Therefore, the present disclosure should be understood based on the intended meaning of the terms rather than their simple names or meanings.

[0040] Figure 1 shows an exemplary point cloud content providing system according to an embodiment.

[0041] Figure 1 The illustrated point cloud content providing system may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 are capable of wired or wireless communication to transmit and receive point cloud data.

[0042] The point cloud data transmission device 10000 according to an embodiment can acquire and process point cloud video (or point cloud content) and transmit it. According to an embodiment, the transmission device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device and / or a server. According to an embodiment, the transmission device 10000 may include a device, a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server configured to perform communication with a base station and / or other wireless devices using a radio access technology (e.g., 5G new radio access technology (NR), long term evolution (LTE)).

[0043] The transmission device 10000 according to an embodiment includes a point cloud video acquirer 10001, a point cloud video encoder 10002, and / or a transmitter (or communication module) 10003.

[0044] The point cloud video acquirer 10001 according to an embodiment acquires a point cloud video through a processing process such as capture, synthesis, or generation. The point cloud video is point cloud content represented by a point cloud, and the point cloud is a set of points located in a 3D space and may be referred to as point cloud video data, point cloud data, etc. The point cloud video according to an embodiment may include one or more frames. One frame represents a still image / picture. Therefore, the point cloud video may include point cloud images / frames / pictures and may be referred to as point cloud images, frames, or pictures.

[0045] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 may encode the point cloud video data based on point cloud compression compilation. The point cloud compression compilation according to an embodiment may include geometry-based point cloud compression (G-PCC) compilation and / or video-based point cloud compression (V-PCC) compilation or next-generation compilation. The point cloud compression compilation according to an embodiment is not limited to the above embodiments. The point cloud video encoder 10002 may output a bitstream containing the encoded point cloud video data. The bitstream may contain not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0046] The transmitter 10003 according to an embodiment transmits a bitstream including encoded point cloud video data. The bitstream according to an embodiment is encapsulated in a file or a segment (e.g., a stream segment) and is transmitted via various networks such as a broadcast network and / or a broadband network. Although not shown in the figure, the transmitting device 10000 may include an encapsulator (or an encapsulation module) configured to perform an encapsulation operation. According to an embodiment, the encapsulator may be included in the transmitter 10003. According to an embodiment, the file or the segment may be transmitted to the receiving device 10004 via a network or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitter 10003 according to an embodiment is capable of wired / wireless communication with the receiving device 10004 (or the receiver 10005) via networks such as 4G, 5G, 6G, etc. Additionally, the transmitter may perform necessary data processing operations according to a network system (e.g., a 4G, 5G, or 6G communication network system). The transmitting device 10000 may transmit the encapsulated data in an on-demand manner.

[0047] The receiving device 10004 according to an embodiment includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include a device, a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server configured to perform communication with a base station and / or other wireless devices using a radio access technology (e.g., 5G new radio access technology (NR), long term evolution (LTE)).

[0048] The receiver 10005 according to an embodiment receives a bitstream including point cloud video data or a file / segment encapsulating the bitstream from a network or a storage medium. The receiver 10005 may perform necessary data processing according to a network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). The receiver 10005 according to an embodiment may decompress the received file / segment and output the bitstream. According to an embodiment, the receiver 10005 may include a decompressor (or a decompression module) configured to perform a decompression operation. The decompressor may be implemented as an element (or a component) separate from the receiver 10005.

[0049] The point cloud video decoder 10006 decodes the bitstream including point cloud video data. The point cloud video decoder 10006 may decode the point cloud video data according to the method by which the point cloud video data is encoded (e.g., the inverse process of the operation of the point cloud video encoder 10002). Thus, the point cloud video decoder 10006 may decode the point cloud video data by performing point cloud decompression compilation (the inverse process of point cloud compression). The point cloud decompression compilation includes G-PCC compilation.

[0050] Renderer 10007 renders the decoded point cloud video data. Renderer 10007 may output point cloud content by rendering not only the point cloud video data but also audio data. According to an embodiment, renderer 10007 may include a display configured to display the point cloud content. According to an embodiment, the display may be implemented as a separate device or component rather than being included in renderer 10007.

[0051] The arrow indicated by the dashed line in the figure represents the transmission path of the feedback information acquired by receiving device 10004. The feedback information is information reflecting the interactivity with the user consuming the point cloud content, and includes information about the user (e.g., head orientation information, viewport information, etc.). Specifically, when the point cloud content is content for a service that requires interaction with the user (e.g., a self-driving service, etc.), the feedback information may be provided to the content sender (e.g., transmitting device 10000) and / or the service provider. According to an embodiment, the feedback information may be used in receiving device 10004 as well as transmitting device 10000, or may not be provided.

[0052] According to an embodiment, the head orientation information may be information about the user's head position, orientation, angle, movement, etc. According to an embodiment, receiving device 10004 may calculate viewport information based on the head orientation information. The viewport information may be information about the area of the point cloud video that the user is viewing. The viewing point is the point through which the user is viewing the point cloud video, and may refer to the center point of the viewport area. That is, the viewport is an area centered on the viewing point, and the size and shape of the area may be determined by the field of view (FOV). Therefore, receiving device 10004 may extract the viewport information based on the vertical or horizontal FOV supported by the device in addition to the head orientation information. Also, receiving device 10004 performs gaze analysis or the like to examine the way the user consumes the point cloud, the area in the point cloud video that the user gazes at, the gaze time, etc. According to an embodiment, receiving device 10004 may send the feedback information including the gaze analysis result to transmitting device 10000. According to an embodiment, the feedback information may be acquired during the rendering and / or display process. According to an embodiment, the feedback information may be obtained by one or more sensors included in receiving device 10004. According to an embodiment, the feedback information may be ensured by renderer 10007 or a separate external element (or device, component, etc.). Figure 1The dashed line in [the figure] represents the process of sending feedback information ensured by the renderer 10007. The point cloud content providing system can process (encode / decode) the point cloud data based on the feedback information. Therefore, the point cloud video decoder 10006 can perform a decoding operation based on the feedback information. The receiving device 10004 can send the feedback information to the sending device 10000. The sending device 10000 (or the point cloud video data encoder 10002) can perform an encoding operation based on the feedback information. Therefore, the point cloud content providing system can effectively process the necessary data (e.g., the point cloud data corresponding to the user's head position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide the point cloud content to the user.

[0053] According to an embodiment, the sending device 10000 may be referred to as an encoder, a sending device, a transmitter, etc., and the receiving device 10004 may be referred to as a decoder, a receiving device, a receiver, etc.

[0054] According to an embodiment of Figure 1 the point cloud data processed in the point cloud content providing system (through a series of processes of acquisition / encoding / transmission / decoding / rendering) may be referred to as point cloud content data or point cloud video data. According to an embodiment, the point cloud content data may be used as a concept encompassing metadata or signaling information related to the point cloud data.

[0055] Figure 1 The elements of the point cloud content providing system shown can be implemented by hardware, software, a processor, and / or a combination thereof.

[0056] Figure 2 is a block diagram showing a point cloud content providing operation according to an embodiment.

[0057] Figure 2 The block diagram of [the figure] shows Figure 1 the operations of the point cloud content providing system described in [the figure]. As described above, the point cloud content providing system can process the point cloud data based on point cloud compression compilation (e.g., G-PCC).

[0058] According to an embodiment, a point cloud content providing system (e.g., the point cloud transmitting device 10000 or the point cloud video acquirer (10001)) may acquire a point cloud video (20000). The point cloud video is represented by point clouds belonging to a coordinate system for expressing a 3D space. The point cloud video according to an embodiment may include a Ply (Polygon file format or Stanford Triangle format) file. When the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. The geometry includes the position of a point. The position of each point may be represented by parameters (e.g., the values of the X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system composed of the X, Y, and Z axes). The attributes include the attributes of a point (e.g., information about the texture, color (YCbCr or RGB), reflectance r, transparency, etc. of each point). A point has one or more attributes. For example, a point may have a color attribute or two attributes of color and reflectance. According to an embodiment, the geometry may be referred to as position, geometry information, geometry data, position information, position data, etc., and the attributes may be referred to as attributes, attribute information, attribute data, etc. The point cloud content providing system (e.g., the point cloud transmitting device 10000 or the point cloud video acquirer 10001) may obtain point cloud data from information related to the point cloud video acquisition process (e.g., depth information, color information, etc.).

[0059] According to an embodiment, a point cloud content providing system (e.g., the transmitting device 10000 or the point cloud video encoder 10002) may encode the point cloud data (20001). The point cloud content providing system may encode the point cloud data based on point cloud compression compilation. As described above, the point cloud data may include geometric information and attribute information about points. Therefore, the point cloud content providing system may perform geometric encoding for encoding the geometry and output a geometric bitstream. The point cloud content providing system may perform attribute encoding for encoding the attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system may perform attribute encoding based on geometric encoding. The geometric bitstream and the attribute bitstream according to an embodiment may be multiplexed and output as one bitstream. The bitstream according to an embodiment may also contain signaling information related to geometric encoding and attribute encoding.

[0060] According to an embodiment, a point cloud content providing system (e.g., the transmitting device 10000 or the transmitter 10003) may transmit the encoded point cloud data (20002). As Figure 1As shown, the encoded point cloud data can be represented by a geometry bitstream and an attribute bitstream. Additionally, the encoded point cloud data can be sent in the form of a bitstream together with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometry encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and send it in the form of a file or a segment.

[0061] A point cloud content providing system according to an embodiment (e.g., receiving device 10004 or receiver 10005) can receive a bitstream containing the encoded point cloud data. Additionally, the point cloud content providing system (e.g., receiving device 10004 or receiver 10005) can demultiplex the bitstream.

[0062] A point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode the encoded point cloud data (e.g., geometry bitstream, attribute bitstream) sent in the bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode the point cloud video data based on the signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode the geometry bitstream to reconstruct the positions (geometry) of the points. The point cloud content providing system can reconstruct the attributes of the points by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can reconstruct the point cloud video based on the reconstructed geometry and decoded attributes based on the positions.

[0063] A point cloud content providing system according to an embodiment (e.g., receiving device 10004 or renderer 10007) can render the decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) can use various rendering methods to render the geometry and attributes decoded through the decoding process. The points in the point cloud content can be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the corresponding vertex positions, or circles centered at the corresponding vertex positions. All or part of the rendered point cloud content is provided to the user through a display (e.g., VR / AR display, general display, etc.).

[0064] A point cloud content providing system according to an embodiment (e.g., receiving device 10004) can obtain feedback information (20005). The point cloud content providing system can encode and / or decode the point cloud data based on the feedback information. The feedback information and operations of the point cloud content providing system according to an embodiment are the same as the feedback information and operations described with reference to Figure 1 and thus the detailed description thereof is omitted.

[0065] Figure 3 Shows an exemplary point cloud encoder according to an embodiment.

[0066] Figure 3 Shows Figure 1 An example of the point cloud video encoder 10002. The point cloud encoder reconstructs and encodes point cloud data (e.g., the position and / or attributes of points) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) according to network conditions or applications. When the total size of the point cloud content is large (e.g., 60 Gbps of point cloud content for 30 fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system can reconstruct the point cloud content based on the maximum target bitrate to provide the point cloud content according to the network environment, etc.

[0067] As described with reference to Figure 1 And Figure 2 The point cloud encoder can perform geometric coding and attribute coding. Geometric coding is performed before attribute coding.

[0068] The point cloud encoder according to an embodiment includes a coordinate transformer (transform coordinates) 30000, a quantizer (quantize and remove points (voxelization)) 30001, an octree analyzer (analyze octree) 30002, a surface approximation analyzer (analyze surface approximation) 30003, an arithmetic encoder (arithmetic coding) 30004, a geometric reconstructor (reconstruct geometry) 30005, a color transformer (transform color) 30006, an attribute transformer (transform attributes) 30007, a RAHT transformer (RAHT) 30008, a LOD generator (generate LOD) 30009, a lifting transformer (lifting) 30010, a coefficient quantizer (quantize coefficients) 30011, and / or an arithmetic encoder (arithmetic coding) 30012.

[0069] The coordinate transformer 30000, the quantizer 30001, the octree analyzer 30002, the surface approximation analyzer 30003, the arithmetic encoder 30004, and the geometric reconstructor 30005 can perform geometric coding. Geometric coding according to an embodiment may include octree geometry compilation, prediction tree geometry compilation, direct compilation, triad geometric coding, and entropy coding. Direct compilation and triad geometric coding are selectively or combinatorially applied. Geometric coding is not limited to the above examples.

[0070] As shown in the figure, the coordinate transformer 30000 according to an embodiment receives a position and transforms it into coordinates. For example, the position can be transformed into position information in a three-dimensional space (e.g., a three-dimensional space represented by an XYZ coordinate system). The position information in the three-dimensional space according to an embodiment may be referred to as geometric information.

[0071] The quantizer 30001 according to an embodiment performs geometric quantization. For example, the quantizer 30001 may quantize points based on the minimum position values of all points (e.g., the minimum values on each of the X, Y, and Z axes). The quantizer 30001 performs a quantization operation: multiplying the difference between the minimum position value and the position values of the respective points by a preset quantization scaling value, and then finding the nearest integer value by rounding the value obtained through the multiplication. Thus, one or more points may have the same quantized position (or position value). The quantizer 30001 according to an embodiment performs voxelization based on the quantized position to reconstruct the quantized points. Similar to the case of pixels (the smallest unit containing 2D image / video information), the points of the point cloud content (or 3D point cloud video) according to an embodiment may be included in one or more voxels. As a complex of volume and pixel, the term voxel refers to a 3D cubic space generated when a 3D space is divided into units (unit = 1.0) based on axes representing the 3D space (e.g., the X axis, the Y axis, and the Z axis). The quantizer 30001 may match a group of points in the 3D space with voxels. According to an embodiment, one voxel may include only one point. According to an embodiment, one voxel may include one or more points. To represent a voxel as one point, the position of the center of the voxel may be set based on the positions of one or more points included in the voxel. In this case, the attributes of all positions included in one voxel may be combined and assigned to the voxel.

[0072] The octree analyzer 30002 according to an embodiment performs octree geometry compilation (or octree compilation) to present voxels in an octree structure. The octree structure represents points that match voxels based on an octal tree structure.

[0073] The surface approximation analyzer 30003 according to an embodiment may analyze and approximate the octree. The octree analysis and approximation according to an embodiment is a process of analyzing a region containing multiple points to effectively provide an octree and voxelization.

[0074] The arithmetic encoder 30004 according to an embodiment performs entropy encoding on the octree and / or the approximated octree. For example, the encoding scheme includes arithmetic coding. As a result of the encoding, a geometric bitstream is generated.

[0075] The color transformer 30006, the attribute transformer 30007, the RAHT transformer 30008, the LOD generator 30009, the lifting transformer 30010, the coefficient quantizer 30011, and / or the arithmetic coder 30012 perform attribute encoding. As described above, a point may have one or more attributes. The attribute encoding according to an embodiment is also applied to the attributes that a point has. However, when an attribute (e.g., color) includes one or more elements, the attribute encoding is applied independently to each element. The attribute encoding according to an embodiment includes color transform compilation, attribute transform compilation, region adaptive hierarchical transform (RAHT) compilation, interpolation-based hierarchical nearest neighbor prediction (prediction transform) compilation, and interpolation-based hierarchical nearest neighbor prediction (lifting transform) compilation with an update / lifting step. Depending on the point cloud content, the above RAHT compilation, prediction transform compilation, and lifting transform compilation may be selectively used, or a combination of one or more compilation schemes may be used. The attribute encoding according to an embodiment is not limited to the above examples.

[0076] The color transformer 30006 according to an embodiment performs color transform compilation of color values (or textures) included in attributes as a transform. For example, the color transformer 30006 may transform the format of color information (e.g., from RGB to YCbCr). Optionally, the operation of the color transformer 30006 according to an embodiment may be applied according to the color values included in the attributes.

[0077] The geometry reconstructor 30005 according to an embodiment reconstructs (decompresses) an octree and / or an approximate octree. The geometry reconstructor 30005 reconstructs the octree / voxel based on the result of analyzing the point distribution. The reconstructed octree / voxel may be referred to as reconstructed geometry (restored geometry).

[0078] The attribute transformer 30007 according to an embodiment performs an attribute transform to transform an attribute based on the reconstructed geometry and / or a position where geometry encoding is not performed. As described above, since an attribute depends on geometry, the attribute transformer 30007 may transform an attribute based on the reconstructed geometry information. For example, based on the position value of a point included in a voxel, the attribute transformer 30007 may transform the attribute of the point at that position. As described above, when the center position of a voxel is set based on the position of one or more points included in the voxel, the attribute transformer 30007 transforms the attributes of one or more points. When performing triplet geometry encoding, the attribute transformer 30007 may transform an attribute based on the triplet geometry encoding.

[0079] The attribute transformer 30007 can perform attribute transformation by calculating the average of the attributes or attribute values (e.g., the color or reflectivity of each point) of neighboring points within a specific position / radius from the center position (or position value) of each voxel. The attribute transformer 30007 can apply weights according to the distance from the center to each point when calculating the average. Thus, each voxel has a position and a calculated attribute (or attribute value).

[0080] The attribute transformer 30007 can search for neighboring points within a specific position / radius from the center position of each voxel based on a K-D tree or a Morton code (Morton code). A K-D tree is a binary search tree and supports a data structure that can manage points based on position, enabling fast nearest neighbor search (NNS). The Morton code is generated by presenting the coordinates representing the 3D positions of all points (e.g., (x, y, z)) as bit values and mixing the bits. For example, when the coordinates representing a point position are (5, 9, 1), the bit values of the coordinates are (0101, 1001, 0001). Mixing the bit values in the order of z, y, and x according to the bit index produces 010001000111. This value is represented as a decimal number 1095. That is, the Morton code value of the point with coordinates (5, 9, 1) is 1095. The attribute transformer 30007 can sort the points based on the Morton code value and perform NNS through a depth-first traversal process. The K-D tree or Morton code is used when NNS is required in another transformation process for attribute compilation after the attribute transformation operation.

[0081] As shown in the figure, the transformed attributes are input to the RAHT transformer 30008 and / or the LOD generator 30009.

[0082] The RAHT transformer 30008 according to an embodiment performs RAHT compilation for predicting attribute information based on the reconstructed geometric information. For example, the RAHT transformer 30008 can predict the attribute information of higher-level nodes in the octree based on the attribute information associated with lower-level nodes in the octree.

[0083] The LOD generator 30009 according to an embodiment generates a level of detail (LOD) to perform predictive transformation compilation. The LOD according to an embodiment is the level of detail of the point cloud content. As the LOD value decreases, it indicates that the detail of the point cloud content deteriorates. As the LOD value increases, it indicates that the detail of the point cloud content is enhanced. Points can be classified according to the LOD.

[0084] The lifting transformer 30010 according to an embodiment performs lifting transformation compilation for transforming the point cloud attributes based on weights. As described above, the lifting transformation compilation can be optionally applied.

[0085] The coefficient quantizer 30011 according to an embodiment quantizes the attributes encoded by attributes based on coefficients.

[0086] The arithmetic encoder 30012 according to an embodiment encodes quantized properties based on arithmetic coding.

[0087] Although not shown in the figure, Figure 3 The elements of the point cloud encoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. The one or more processors may perform the above Figure 3 In addition, one or more processors may be operable or executable to perform at least one of the operations and / or functions of the elements of the point cloud encoder. Figure 3 The software program and / or instruction set for the operation and / or functionality of the elements of the point cloud encoder. The one or more memories according to an embodiment may include high-speed random access memory, or include non-volatile memory (e.g., one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0088] Figure 4 Examples of octrees and occupancy codes are shown according to an embodiment.

[0089] As reference Figures 1 to 3 As described, the point cloud content providing system (point cloud video encoder 10002) or the point cloud encoder (eg, octree analyzer 30002) performs octree geometry coding (or octree coding) based on the octree structure to efficiently manage the regions and / or positions of voxels.

[0090] Figure 4 The upper part of FIG shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes (eg, X-axis, Y-axis, and Z-axis) of the coordinate system. The octree structure is represented by two poles (0, 0, 0) and (2 d , 2 d , 2 d ) to create an octree structure. Here, 2d can be set to the value of the minimum bounding box that surrounds all points of the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following equation. In the following equation, (x int n , y int n , z int n ) represents the position (or position value) of the quantized point.

[0091]

[0092] like Figure 4As shown in the middle of the upper part of, the entire 3D space can be divided into eight spaces according to partitions. Each of the divided spaces is represented by a cube with six faces. As Figure 4 As shown in the upper right part of, each of the eight spaces is further divided based on the axes of the coordinate system (e.g., the X-axis, Y-axis, and Z-axis). Thus, each space is divided into eight smaller spaces. The divided smaller spaces are also represented by cubes with six faces. Applying this division scheme until the leaf nodes of the octree become voxels.

[0093] Figure 4 The lower part of shows the octree occupancy code. The occupancy code of the octree is generated to indicate whether each of the eight divided spaces generated by dividing a space contains at least one point. Thus, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of the divided space, and the child node has a value of 1 bit. Thus, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to the child node contains at least one point, the node is assigned the value 1. When the space corresponding to the child node does not contain a point (the space is empty), the node is assigned the value 0. Since Figure 4 the shown occupancy code is 00100001, it indicates that the spaces corresponding to the third child node and the eighth child node among the eight child nodes each contain at least one point. As shown in the figure, each of the third child node and the eighth child node has eight child nodes, and the child nodes are represented by 8-bit occupancy codes. The accompanying figure shows that the occupancy code of the third child node is 10000111, and the occupancy code of the eighth child node is 01001111. A point cloud encoder according to an embodiment (e.g., the arithmetic encoder 30004) can perform entropy encoding on the occupancy code. To increase the compression efficiency, the point cloud encoder can perform in / inter-frame encoding on the occupancy code. A receiving device according to an embodiment (e.g., the receiving device 10004 or the point cloud video decoder 10006) reconstructs the octree based on the occupancy code.

[0094] A point cloud encoder according to an embodiment (e.g., Figure 4 the point cloud encoder or the octree analyzer 30002 of ) can perform voxelization and octree encoding to store point positions. However, points are not always evenly distributed in the 3D space, so there may be specific regions with fewer points. Therefore, performing voxelization on the entire 3D space is inefficient. For example, when a specific region contains very few points, voxelization does not need to be performed in that specific region.

[0095] Therefore, for the above specific regions (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment can skip voxelization and perform direct compilation to directly encode the point positions included in the specific regions. The coordinates of the points directly compiled according to the embodiment are referred to as the direct compilation mode (DCM). The point cloud encoder according to the embodiment can also perform triad geometry encoding based on the surface model, which is to reconstruct the point positions in the specific region (or node) based on voxels. Triad geometry encoding is a geometry encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. The direct compilation and triad geometry encoding according to the embodiment can be selectively performed. In addition, the direct compilation and triad geometry encoding according to the embodiment can be performed in combination with octree geometry compilation (or octree compilation).

[0096] To perform direct compilation, the option to use the direct mode to apply direct compilation should be enabled. The node to which direct compilation is to be applied is not a leaf node, and there should be fewer points than a threshold within the specific node. In addition, the total number of points to which direct compilation is to be applied should not exceed a preset threshold. When the above conditions are met, the point cloud encoder (or arithmetic encoder 30004) according to the embodiment can perform entropy compilation on the point positions (or position values).

[0097] The point cloud encoder according to the embodiment (e.g., surface approximation analyzer 30003) can determine a specific level of the octree (a level less than the depth d of the octree), and can start from this level to use the surface model to perform triad geometry encoding to reconstruct the point positions in the node region based on voxels (triad mode). The point cloud encoder according to the embodiment can specify the level to which triad geometry encoding is to be applied. For example, when the specific level is equal to the depth of the octree, the point cloud encoder does not operate in the triad mode. In other words, only when the specified level is less than the depth value of the octree, the point cloud encoder according to the embodiment can operate in the triad mode. The 3D cubic region of the node at the specified level according to the embodiment is referred to as a block. A block can include one or more voxels. A block or a voxel can correspond to a brick. The geometry is represented as a surface within each block. The surface according to the embodiment can intersect each edge of the block at most once.

[0098] A block has 12 edges, so there are at least 12 intersection points in a block. Each intersection point is referred to as a vertex. When there is at least one occupied voxel adjacent to an edge among all the blocks sharing the edge, the vertex existing along the edge is detected. The occupied voxel according to the embodiment refers to a voxel containing a point. The vertex position detected along the edge is the average position of the edges of all the voxels adjacent to the edge among all the blocks sharing the edge.

[0099] Once a vertex is detected, the point cloud encoder according to an embodiment can perform entropy encoding on the starting point (x, y, z) of an edge, the direction vector (Δx, Δy, Δz) of the edge, and the vertex position value (the relative position value within the edge). When applying the triplet geometry encoding, the point cloud encoder according to an embodiment (e.g., the geometry reconstructor 30005) can generate the restored geometry (reconstructed geometry) by performing a triangle reconstruction, upsampling, and voxelization process.

[0100] Vertices located at the edges of a block define a surface passing through the block. The surface according to an embodiment is a non-planar polygon. During the triangle reconstruction process, the surface represented by a triangle is reconstructed based on the starting point of the edge, the direction vector of the edge, and the position value of the vertex. The triangle reconstruction process is performed as follows: i) calculate the centroid value of each vertex, ii) subtract the center value from each vertex value, and iii) estimate the sum of squares of the values obtained by the subtraction.

[0101]

[0102] Estimate the minimum value of the sum, and perform a projection process according to the axis with the minimum value. For example, when the element x is the smallest, each vertex is projected onto the x-axis with respect to the center of the block and projected onto the (y, z) plane. When the values obtained by the projection onto the (y, z) plane are (ai, bi), estimate the value of θ by atan2(bi, ai), and sort the vertices based on the value of θ. The following table shows the vertex combinations for creating triangles according to the number of vertices. The vertices are sorted from 1 to n. The following table shows that for four vertices, two triangles can be constructed according to the vertex combinations. The first triangle can be composed of vertices 1, 2, and 3 among the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 among the sorted vertices.

[0103] Table 2-1. Triangles formed from vertices in the order of 1,…,n

[0104] n Triangles

[0105] 3 (1,2,3)

[0106] 4 (1,2,3), (3,4,1)

[0107] 5 (1,2,3), (3,4,5), (5,1,3)

[0108] 6 (1,2,3), (3,4,5), (5,6,1), (1,3,5)

[0109] 7 (1,2,3), (3,4,5), (5,6,7), (7,1,3), (3,5,7)

[0110] 8 (1,2,3), (3,4,5), (5,6,7), (7,8,1), (1,3,5), (5,7,1)

[0111] 9 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,1,3), (3,5,7), (7,9,3)

[0112] 10 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,1), (1,3,5), (5,7,9),(9,1,5)

[0113] 11 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,11), (11,1,3), (3,5,7),(7,9,11), (11,3,7)

[0114] 12 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,11), (11,12,1), (1,3,5),(5,7,9), (9,11,1), (1,5,9)

[0115] The upsampling process is performed to add points in the middle along the sides of the triangle, and voxelization is performed. The added points are generated based on the upsampling factor and the width of the block. The added points are called refinement vertices. The point cloud encoder according to the embodiment can voxelize the refinement vertices. Additionally, the point cloud encoder can perform attribute encoding based on the voxelized position (or position value).

[0116] Figure 5 Examples of point configurations in each LOD according to the embodiment are shown.

[0117] As referred to Figures 1 to 4 As described, the encoded geometry is reconstructed (decompressed) before performing the attribute encoding. When direct compilation is applied, the geometry reconstruction operation can include changing the placement of the directly compiled points (e.g., placing the directly compiled points in front of the point cloud data). When triad geometry encoding is applied, the geometry reconstruction process is performed through triangle reconstruction, upsampling, and voxelization. Since the attributes depend on the geometry, the attribute encoding is performed based on the reconstructed geometry.

[0118] A point cloud encoder (e.g., the LOD generator 30009) can classify (or reorganize) points according to the LOD. The point cloud content corresponding to the LOD is shown in the figure. The leftmost picture in the figure represents the original point cloud content. The second picture from the left in the figure represents the point distribution in the lowest LOD, and the rightmost picture in the figure represents the point distribution in the highest LOD. That is, the points in the lowest LOD are sparsely distributed, and the points in the highest LOD are densely distributed. That is, as the LOD increases in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between points becomes narrower.

[0119] Figure 6 An example of the point configuration for each LOD according to an embodiment is shown.

[0120] As referred to Figures 1 to 5 as described, a point cloud content providing system or a point cloud encoder (e.g., the point cloud video encoder 10002, Figure 3 the point cloud encoder or the LOD generator 30009) can generate the LOD. The LOD is generated by reorganizing points into a set of refinement levels according to a set LOD distance value (or a set of Euclidean distances). The LOD generation process is performed not only by the point cloud encoder but also by the point cloud decoder.

[0121] Figure 6 The upper part of Figure 6 shows examples of points (P0 to P9) of the point cloud content distributed in the 3D space. In Figure 6 the original order represents the order of points P0 to P9 before LOD generation. In Figure 6 the LOD-based order represents the order of points generated according to the LOD. The points are reorganized according to the LOD. In addition, the high LOD contains the points belonging to the lower LOD. As

[0122] As referred to Figure 3 as described, the point cloud encoder according to an embodiment can selectively or combinatorially perform predictive transform compilation, lifting transform compilation, and RAHT transform compilation.

[0123] The point cloud encoder according to an embodiment can generate predictors for points to perform predictive transform compilation for setting the prediction attributes (or prediction attribute values) of each point. That is, N predictors can be generated for N points. The predictor according to an embodiment can calculate weights (= 1 / distance) based on the LOD value of each point, the index information about neighboring points existing within a set distance in each LOD, and the distance to the neighboring points.

[0124] The predicted attribute (or attribute value) according to an embodiment is set to the average of values obtained by multiplying the attributes (or attribute values) of neighboring points set in the predictor of each point (e.g., color, reflectance, etc.) by weights (or weight values) calculated based on the distance to each neighboring point. A point cloud encoder according to an embodiment (e.g., coefficient quantizer 30011) may quantize and inverse-quantize a residual obtained by subtracting the predicted attribute (attribute value) from the attribute (attribute value) of each point (which may be referred to as a residual attribute, residual attribute value, or attribute prediction residual, attribute residual). The quantization process is configured as shown in the following table.

[0125] Table. Attribute Prediction Residual Quantization Pseudocode

[0126] int PCCQuantization(int value, int quantStep) {

[0127] if( value >=0) {

[0128] return floor(value / quantStep + 1.0 / 3.0);

[0129] } else {

[0130] return -floor(-value / quantStep + 1.0 / 3.0);

[0131] }

[0132] }

[0133] Table. Attribute Prediction Residual Inverse Quantization Pseudocode

[0134] int PCCInverseQuantization(int value, int quantStep) {

[0135] if( quantStep ==0) {

[0136] return value;

[0137] } else {

[0138] return value * quantStep;

[0139] }

[0140] }

[0141] When the predictors of the respective points have neighbor points, the point cloud encoder according to an embodiment (e.g., arithmetic encoder 30012) can perform entropy coding on the quantized and inverse quantized residual values as described above. When the predictors of the respective points do not have neighbor points, the point cloud encoder according to an embodiment (e.g., arithmetic encoder 30012) can perform entropy coding on the attributes of the corresponding points without performing the above operations.

[0142] The point cloud encoder according to an embodiment (e.g., lifting transformer 30010) can generate predictors for the respective points, set the calculated LOD, register neighbor points in the predictors, and set weights according to the distances to the neighbor points to perform lifting transform coding. The lifting transform coding according to an embodiment is similar to the above-described prediction transform coding, except that weights are cumulatively applied to the attribute values. The process of cumulatively applying weights to the attribute values according to an embodiment is configured as follows.

[0143] 1) Create an array quantization weight (QW) for storing the weight values of the respective points. The initial value of all elements of QW is 1.0. Multiply the QW value of the predictor index of the neighbor nodes registered in the predictor by the weight of the predictor of the current point, and add the values obtained by the multiplication.

[0144] 2) Lifting prediction process: Subtract the value obtained by multiplying the attribute value of the point by the weight from the existing attribute value to calculate the predicted attribute value.

[0145] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0146] 4) Accumulate the weights calculated by multiplying the weights calculated for all predictors by the weights stored in QW corresponding to the predictor index as the indices of the neighbor nodes. Accumulate the values obtained by multiplying the attribute values of the neighbor node indices by the calculated weights into the update array.

[0147] 5) Lifting update process: Divide the attribute values of the update array of all predictors by the weight values of the updateweight array of the predictor indices, and add the existing attribute values to the values obtained by the division.

[0148] 6) Calculate the predicted attributes for all predictors by multiplying the attribute values updated through the lifting update process by the weights updated through the lifting prediction process (stored in QW). The point cloud encoder according to an embodiment (e.g., coefficient quantizer 30011) quantizes the predicted attribute values. In addition, the point cloud encoder (e.g., arithmetic encoder 30012) performs entropy coding on the quantized attribute values.

[0149] A point cloud encoder according to an embodiment (e.g., the RAHT transformer 30008) may perform RAHT transform compilation, where attributes associated with lower-level nodes in an octree are used to predict the attributes of higher-level nodes. RAHT transform compilation is an example of in-frame attribute compilation by scanning the octree backward. The point cloud encoder according to an embodiment starts scanning the entire area from voxels and repeats the merging process of merging voxels into larger blocks at each step until reaching the root node. The merging process according to the embodiment is only performed on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on the upper node directly above the empty node.

[0150] The following equation represents the RAHT transform matrix. In this equation, represents the average attribute value of the voxel at level . can be calculated based on and . The weights for and are and .

[0151]

[0152] Here, is the low-pass value and is used in the merging process at the second-highest level. represents the high-pass coefficient. The high-pass coefficient at each step is quantized and subjected to entropy compilation (e.g., encoded by the arithmetic encoder 30012). The weights are calculated as . The root node is created by and as follows.

[0153]

[0154] The value of gDC is also quantized and subjected to entropy compilation like the high-pass coefficient.

[0155] Figure 7 FIG. shows a point cloud decoder according to an embodiment.

[0156] Figure 7 The shown point cloud decoder is an example of a point cloud decoder and may perform a decoding operation, which is the inverse process of the encoding operation of the point cloud encoder shown in Figures 1 to 6 .

[0157] As described with reference to Figure 1 and Figure 6 , the point cloud decoder may perform geometric decoding and attribute decoding. Geometric decoding is performed before attribute decoding.

[0158] The point cloud decoder according to an embodiment includes an arithmetic decoder (Arithmetic Decoding) 7000, an octree synthesizer (Synthesize Octree) 7001, a surface approximation synthesizer (Synthesize Surface Approximation) 7002, a geometry reconstructor (Reconstruct Geometry) 7003, a coordinate inverse transformer (Inverse Transform Coordinates) 7004, an arithmetic decoder (Arithmetic Decoding) 7005, an inverse quantizer (Inverse Quantization) 7006, a RAHT transformer 7007, a LOD generator (Generate LOD) 7008, an inverse lifter (Inverse Lifting) 7009, and / or a color inverse transformer (Inverse Transform Color) 7010.

[0159] The arithmetic decoder 7000, the octree synthesizer 7001, the surface approximation synthesizer 7002, the geometry reconstructor 7003, and the coordinate inverse transformer 7004 may perform geometry decoding. The geometry decoding according to an embodiment may include direct decoding and triple geometry decoding. The direct compilation and triple geometry decoding are selectively applied. The geometry decoding is not limited to the above examples and is performed as the inverse process of the geometry encoding described with reference to Figures 1 to 6 the geometry encoding described.

[0160] The arithmetic decoder 7000 according to an embodiment decodes the received geometry bitstream based on arithmetic compilation. The operation of the arithmetic decoder 7000 corresponds to the inverse process of the arithmetic encoder 30004.

[0161] The octree synthesizer 7001 according to an embodiment may generate an octree by obtaining occupancy codes (or information about the geometry obtained as a decoding result) from the decoded geometry bitstream. The occupancy codes are configured as described with reference to Figures 1 to 6 the detailed description.

[0162] When triple geometry encoding is applied, the surface approximation synthesizer 7002 according to an embodiment may synthesize a surface based on the decoded geometry and / or the generated octree.

[0163] The geometry reconstructor 7003 according to an embodiment may regenerate geometry based on the surface and / or the decoded geometry. As described with reference to Figures 1 to 9 the description, the direct compilation and triple geometry encoding are selectively applied. Therefore, the geometry reconstructor 7003 directly imports the position information of the points to which the direct compilation is applied and adds them. When triple geometry encoding is applied, the geometry reconstructor 7003 may reconstruct the geometry by performing the reconstruction operations of the geometry reconstructor 30005 (e.g., triangle reconstruction, upsampling, and voxelization). The details are the same as those described with reference to Figure 6 the description, and thus its description is omitted. The reconstructed geometry may include a point cloud picture or frame that does not contain attributes.

[0164] The coordinate inverse transformer 7004 according to an embodiment may obtain point positions by transforming coordinates based on the reconstructed geometry.

[0165] The arithmetic decoder 7005, inverse quantizer 7006, RAHT transformer 7007, LOD generator 7008, inverse lifter 7009, and / or color inverse transformer 7010 may perform the attribute decoding described in the reference. Figure 6 The attribute decoding according to an embodiment includes region adaptive hierarchical transform (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) decoding, and interpolation-based hierarchical nearest neighbor prediction with an update / lifting step (lifting transform) decoding. The above three decoding schemes may be selectively used, or a combination of one or more decoding schemes may be used. The attribute decoding according to an embodiment is not limited to the above examples.

[0166] The arithmetic decoder 7005 according to an embodiment decodes the attribute bitstream through arithmetic coding.

[0167] The inverse quantizer 7006 according to an embodiment inverse quantizes the information on the decoded attribute bitstream or the attribute obtained as a decoding result, and outputs the inverse quantized attribute (or attribute value). The inverse quantization may be selectively applied based on the attribute coding of the point cloud encoder.

[0168] According to an embodiment, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 may process the reconstructed geometry and the inverse quantized attributes. As described above, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 may selectively perform decoding operations corresponding to the encoding of the point cloud encoder.

[0169] The color inverse transformer 7010 according to an embodiment performs inverse transform coding to inverse transform the color value (or texture) included in the decoded attribute. The operation of the color inverse transformer 7010 may be selectively performed based on the operation of the color transformer 30006 of the point cloud encoder.

[0170] Although not shown in the figure, Figure 7 the elements of the point cloud decoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors may perform at least one or more of the operations and / or functions of the elements of the point cloud decoder described above. Additionally, one or more processors may operate or execute software programs and / or instruction sets for performing the operations and / or functions of the elements of the point cloud decoder described above. Figure 7 the elements of the point cloud decoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors may perform at least one or more of the operations and / or functions of the elements of the point cloud decoder described above. Additionally, one or more processors may operate or execute software programs and / or instruction sets for performing the operations and / or functions of the elements of the point cloud decoder described above. Figure 7 the elements of the point cloud decoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors may perform at least one or more of the operations and / or functions of the elements of the point cloud decoder described above. Additionally, one or more processors may operate or execute software programs and / or instruction sets for performing the operations and / or functions of the elements of the point cloud decoder described above.

[0171] Figure 8 A transmitting device according to an embodiment is shown.

[0172] Figure 8The transmitting device shown is Figure 1 an example of the transmitting device 10000 (or Figure 3 the point cloud encoder). Figure 8 The transmitting device shown can perform one or more operations and methods that are the same as or similar to those of the point cloud encoder described with reference to Figures 1 to 6 . The transmitting device according to an embodiment may include a data input unit 8000, a quantization processor 8001, a voxelization processor 8002, an octree occupancy code generator 8003, a surface model processor 8004, an intra / inter compilation processor 8005, an arithmetic coder 8006, a metadata processor 8007, a color transformation processor 8008, an attribute transformation processor 8009, a prediction / lifting / RAHT transformation processor 8010, an arithmetic compiler 8011, and / or a transmission processor 8012.

[0173] The data input unit 8000 according to an embodiment receives or acquires point cloud data. The data input unit 8000 may perform operations and / or acquisition methods that are the same as or similar to those of the point cloud video acquirer 10001 and / or the acquisition method (or the acquisition process 20000 described with reference to Figure 2 ).

[0174] The data input unit 8000, the quantization processor 8001, the voxelization processor 8002, the octree occupancy code generator 8003, the surface model processor 8004, the intra / inter compilation processor 8005, and the arithmetic compiler 8006 perform geometric coding. The geometric coding according to an embodiment is the same as or similar to the geometric coding described with reference to Figures 1 to 9 , and thus its detailed description is omitted.

[0175] The quantization processor 8001 according to an embodiment quantizes geometry (e.g., the position value of a point). The operations and / or quantization of the quantization processor 8001 are the same as or similar to the operations and / or quantization of the quantizer 30001 described with reference to Figure 3 . The details are the same as those described with reference to Figures 1 to 9 .

[0176] The voxelization processor 8002 according to an embodiment voxelizes the quantized position values of points. The voxelization processor 8002 may perform operations and / or processes that are the same as or similar to the operations and / or voxelization process of the quantizer 30001 described with reference to Figure 3 . The details are the same as those described with reference to Figures 1 to 6

[0177] ​The octree occupancy code generator 8003 according to an embodiment performs octree compilation on the voxelized positions of points based on an octree structure. The octree occupancy code generator 8003 can generate occupancy codes. The octree occupancy code generator 8003 can perform operations and / or methods that are the same as or similar to those of the point cloud encoder (or octree analyzer 30002) described with reference to Figure 3 and Figure 4 The details are the same as those described with reference to Figures 1 to 6 described.

[0178] The surface model processor 8004 according to an embodiment can perform triple geometry coding based on a surface model to reconstruct the point positions in a specific region (or node) based on voxels. The surface model processor 8004 can perform operations and / or methods that are the same as or similar to those of the point cloud encoder (e.g., surface approximation analyzer 30003) described with reference to Figure 3 described. The details are the same as those described with reference to Figures 1 to 6 described.

[0179] The intra / inter compilation processor 8005 according to an embodiment can perform intra / inter compilation on point cloud data. The intra / inter compilation processor 8005 can perform compilation that is the same as or similar to the intra / inter compilation described with reference to Figure 7 described. The details can be the same as those described with reference to Figure 7 described. According to an embodiment, the intra / inter encoding processor 8005 can be included in the arithmetic compiler 8006.

[0180] The arithmetic compiler 8006 according to an embodiment performs entropy coding on the octree and / or approximate octree of point cloud data. For example, the coding scheme includes arithmetic coding. The arithmetic compiler 8006 performs operations and / or methods that are the same as or similar to those of the arithmetic encoder 30004.

[0181] The metadata processor 8007 according to an embodiment processes metadata (e.g., set values) regarding point cloud data and provides it to necessary processing procedures such as geometry coding and / or attribute coding. Additionally, the metadata processor 8007 according to an embodiment can generate and / or process signaling information related to geometry coding and / or attribute coding. The signaling information according to an embodiment can be separately coded from geometry coding and / or attribute coding. The signaling information according to an embodiment can be interleaved.

[0182] The color transformation processor 8008, the attribute transformation processor 8009, the prediction / lifting / RAHT transformation processor 8010, and the arithmetic compiler 8011 perform attribute coding. The attribute coding according to an embodiment is the same as or similar to the attribute coding described with reference to Figures 1 to 6 described, and thus its detailed description is omitted.

[0183] The color transformation processor 8008 according to an embodiment performs color transformation compilation to transform color values included in attributes. The color transformation processor 8008 may perform color transformation compilation based on the reconstructed geometry. The reconstructed geometry is the same as that described with reference to Figures 1 to 9 In addition, it performs operations and / or methods that are the same as or similar to the operations and / or methods of the color transformer 30006 described with reference to Figure 3 The detailed description thereof is omitted.

[0184] The attribute transformation processor 8009 according to an embodiment performs attribute transformation to transform attributes based on the reconstructed geometry and / or positions where geometry coding is not performed. The attribute transformation processor 8009 performs operations and / or methods that are the same as or similar to the operations and / or methods of the attribute transformer 30007 described with reference to Figure 3 The detailed description thereof is omitted. The prediction / lifting / RAHT transformation processor 8010 according to an embodiment may encode the transformed attributes by any one or combination of RAHT compilation, prediction transformation compilation, and lifting transformation compilation. The prediction / lifting / RAHT transformation processor 8010 performs at least one operation that is the same as or similar to the operations of the RAHT transformer 30008, LOD generator 30009, and lifting transformer 30010 described with reference to Figure 3 In addition, the prediction transformation compilation, lifting transformation compilation, and RAHT transformation compilation are the same as those described with reference to Figures 1 to 9 Therefore, the detailed description thereof is omitted.

[0185] The arithmetic compiler 8011 according to an embodiment may encode the compiled attributes based on arithmetic compilation. The arithmetic compiler 8011 performs operations and / or methods that are the same as or similar to the operations and / or methods of the arithmetic encoder 30012.

[0186] The transmission processor 8012 according to an embodiment may transmit each bitstream including the encoded geometry and / or encoded attributes and metadata information, or transmit one bitstream configured with the encoded geometry and / or encoded attributes and metadata information. When the encoded geometry and / or encoded attributes and metadata information according to an embodiment are configured as one bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to an embodiment may include signaling information and slice data. The signaling information includes a sequence parameter set (SPS) for sequence-level signaling, a geometry parameter set (GPS) for signaling for geometry information compilation, an attribute parameter set (APS) for signaling for attribute information compilation, and a tile parameter set (TPS) for tile-level signaling. The slice data may include information about one or more slices. One slice according to an embodiment may include a geometry bitstream Geom00 and one or more attribute bitstreams Attr00 and Attr10.

[0187] A slice refers to a series of syntactic elements that represent all or part of a compiled point cloud frame.

[0188] The TPS according to an embodiment may include information about each of one or more tiles (e.g., coordinate information and height / size information about a bounding box). The geometric bitstream may include a header and a payload. The header of the geometric bitstream according to an embodiment may include a parameter set identifier (geom_parameter_set_id), a tile identifier (geom_tile_id), and a slice identifier (geom_slice_id) included in the GPS, as well as information about the data included in the payload. As described above, the metadata processor 8007 according to an embodiment may generate and / or process signaling information and send it to the transmission processor 8012. According to an embodiment, the element that performs geometric coding and the element that performs attribute coding may share data / information with each other as indicated by the dashed line. The transmission processor 8012 according to an embodiment may perform the same or similar operations and / or sending methods as those of the transmitter 10003. Details are the same as those Figure 1 and Figure 2 those described with reference to, and thus their description is omitted.

[0189] Figure 9 An example of a receiving device according to an embodiment is illustrated.

[0190] Figure 9 The receiving device shown is Figure 1 the receiving device 10004 (or Figure 10 and Figure 11 the point cloud decoder of). Figure 9 The receiving device shown may perform one or more operations and methods that are the same or similar to those of the point cloud decoder described with reference to Figures 1 to 11 those.

[0191] The receiving device according to an embodiment may include a receiver 9000, a receiving processor 9001, an arithmetic decoder 9002, an occupancy code-based octree reconstruction processor 9003, a surface model processor (triangle reconstruction, upsampling, voxelization) 9004, an inverse quantization processor 9005, a metadata parser 9006, an arithmetic decoder 9007, an inverse quantization processor 9008, a prediction / lifting / RAHT inverse transform processor 9009, a color inverse transform processor 9010, and / or a renderer 9011. Each decoding element according to an embodiment may perform the inverse process of the operation of the corresponding coding element according to an embodiment.

[0192] The receiver 9000 according to an embodiment receives point cloud data. The receiver 9000 may perform the same as Figure 1The operations and / or reception method of the receiver 10005 are the same as or similar to the operations and / or reception method. Its detailed description is omitted.

[0193] According to an embodiment, the reception processor 9001 can obtain a geometry bitstream and / or an attribute bitstream from the received data. The reception processor 9001 may be included in the receiver 9000.

[0194] The arithmetic decoder 9002, the occupancy code-based octree reconstruction processor 9003, the surface model processor 9004, and the inverse quantization processor 9005 can perform geometry decoding. The geometry decoding according to an embodiment is the same as or similar to the geometry decoding described with reference to Figures 1 to 10 and thus its detailed description is omitted.

[0195] According to an embodiment, the arithmetic decoder 9002 can decode the geometry bitstream based on arithmetic coding. The arithmetic decoder 9002 performs operations and / or compilations that are the same as or similar to the operations and / or compilations of the arithmetic decoder 7000.

[0196] According to an embodiment, the occupancy code-based octree reconstruction processor 9003 can reconstruct an octree by obtaining occupancy codes from the decoded geometry bitstream (or information about the geometry obtained as a decoding result). The occupancy code-based octree reconstruction processor 9003 performs operations and / or methods that are the same as or similar to the operations and / or methods of the octree synthesizer 7001. When applying triple geometry coding, according to an embodiment, the surface model processor 9004 can perform triple geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on the surface model method. The surface model processor 9004 performs operations that are the same as or similar to the operations of the surface approximation synthesizer 7002 and / or the geometry reconstructor 7003.

[0197] According to an embodiment, the inverse quantization processor 9005 can inverse-quantize the decoded geometry.

[0198] According to an embodiment, the metadata parser 9006 can parse the metadata (e.g., setting values) included in the received point cloud data. The metadata parser 9006 can deliver the metadata to geometry decoding and / or attribute decoding. The metadata is the same as the metadata described with reference to Figure 8 and thus its detailed description is omitted.

[0199] The arithmetic decoder 9007, the inverse quantization processor 9008, the prediction / lifting / RAHT inverse transform processor 9009, and the color inverse transform processor 9010 perform attribute decoding. The attribute decoding is the same as or similar to the attribute decoding described with reference to Figures 1 to 10 and thus its detailed description is omitted.

[0200] According to an embodiment, the arithmetic decoder 9007 can decode an attribute bitstream through arithmetic coding. The arithmetic decoder 9007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 9007 performs operations and / or coding that are the same as or similar to those of the arithmetic decoder 7005.

[0201] According to an embodiment, the inverse quantization processor 9008 can inversely quantize the decoded attribute bitstream. The inverse quantization processor 9008 performs operations and / or methods that are the same as or similar to those of the inverse quantizer 7006.

[0202] According to an embodiment, the prediction / lifting / RAHT inverse transform processor 9009 can process the reconstructed geometry and the inversely quantized attributes. The prediction / lifting / RAHT inverse transform processor 9009 performs one or more operations and / or decoding that are the same as or similar to those of the RAHT transformers 7007, the LOD generators 7008, and / or the inverse lifters 7009. According to an embodiment, the color inverse transform processor 9010 performs an inverse transform operation to inversely transform the color values (or textures) included in the decoded attributes. The color inverse transform processor 9010 performs operations and / or inverse transform operations that are the same as or similar to those of the color inverse transformers 7010. According to an embodiment, the renderer 9011 can render point cloud data.

[0203] Figure 10 An exemplary structure operable in combination with a point cloud data sending / receiving method / device according to an embodiment is shown.

[0204] Figure 10 The structure represents a configuration in which at least one of the server 1060, the robot 1010, the autonomous vehicle 1020, the XR device 1030, the smart phone 1040, the home appliance 1050, and / or the head-mounted display (HMD) 1070 is connected to the cloud network 1000. The robot 1010, the autonomous vehicle 1020, the XR device 1030, the smart phone 1040, or the home appliance 1050 is referred to as a device. In addition, the XR device 1030 may correspond to a point cloud data (PCC) device according to an embodiment or be operably connected to a PCC device.

[0205] The cloud network 1000 may represent a network that forms part of or exists in a cloud computing infrastructure. Here, the cloud network 1000 can be configured using a 3G network, a 4G or Long-Term Evolution (LTE) network, or a 5G network.

[0206] Server 1060 can be connected to at least one of the robot 1010, the self-driving vehicle 1020, the XR device 1030, the smart phone 1040, the household appliance 1050, and / or the HMD 1070 via the cloud network 1000, and can assist in processing at least a part of the connected devices 1010 to 1070.

[0207] The HMD 1070 represents one of the implementation types of the XR device and / or the PCC device according to an embodiment. The HMD-type device according to an embodiment includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0208] Hereinafter, various embodiments of the devices 1010 to 1050 to which the above technologies are applied will be described. Figure 10 The illustrated devices 1010 to 1050 can be operably connected / coupled to the point cloud data transmitting device and the receiving device according to the above embodiments.

[0209] <PCC+XR>

[0210] The XR / PCC device 1030 can adopt PCC technology and / or XR (AR+VR) technology, and can be implemented as an HMD, a head-up display (HUD) provided in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a household appliance, a digital sign, a vehicle, a stationary robot, or a mobile robot.

[0211] The XR / PCC device 1030 can analyze 3D point cloud data or image data obtained through various sensors or from external devices and generate position data and attribute data about 3D points. Accordingly, the XR / PCC device 1030 can obtain information about the surrounding space or real objects, and render and output XR objects. For example, the XR / PCC device 1030 can match an XR object including auxiliary information about the identified object with the identified object and output the matched XR object.

[0212] <PCC+XR+mobile phone>

[0213] The XR / PCC device 1030 can be implemented as the smart phone 1040 by applying PCC technology.

[0214] The smart phone 1040 can decode and display point cloud content based on PCC technology.

[0215] <PCC+self-driving+XR>

[0216] The self-driving vehicle 1020 can be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0217] The self-driving vehicle 1020 that applies XR / PCC technology may represent a self-driving vehicle provided with means for providing XR images, or a self-driving vehicle that is a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, the self-driving vehicle 1020 can be distinguished from and operably connected to the XR device 1030.

[0218] The self-driving vehicle 1020 having means for providing XR / PCC images can obtain sensor information from sensors including cameras and output the generated XR / PCC images based on the obtained sensor information. For example, the self-driving vehicle 1020 can have a HUD and output the XR / PCC images thereto, thereby providing XR / PCC objects corresponding to real objects or objects presented on the screen to passengers.

[0219] When the XR / PCC object is output to the HUD, at least a part of the XR / PCC object can be output to overlap with the real object pointed to by the passenger's eyes. On the other hand, when the XR / PCC object is output on a display provided in the self-driving vehicle, at least a part of the XR / PCC object can be output to overlap with the object on the screen. For example, the self-driving vehicle 1020 can output XR / PCC objects corresponding to objects such as roads, other vehicles, traffic lights, traffic signs, two-wheelers, pedestrians, and buildings.

[0220] According to embodiments, virtual reality (VR) technology, augmented reality (AR) technology, mixed reality (MR) technology, and / or point cloud compression (PCC) technology are applicable to various devices.

[0221] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. On the other hand, AR technology refers to a technology of displaying virtual created CG images on an image of a real object. The MR technology is similar to the above AR technology in that the virtual object to be displayed is mixed and combined with the real world. However, the MR technology is different from the AR technology in that the AR technology clearly distinguishes between the real object and the virtual object created as a CG image and uses the virtual object as a supplementary object for the real object, while the MR technology treats the virtual object as an object having the same characteristics as the real object. More specifically, an example of the application of MR technology is holographic services.

[0222] Recently, VR, AR, and MR technologies are generally referred to as extended reality (XR) technologies rather than being clearly distinguished from each other. Therefore, the embodiments of the present disclosure are applicable to any one of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies is applicable to such technologies.

[0223] The PCC method / device according to an embodiment can be applied to a vehicle providing a self-driving service.

[0224] A vehicle providing a self-driving service is connected to a PCC device for wired / wireless communication.

[0225] When a point cloud data (PCC) transmitting / receiving device according to an embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to an AR / VR / PCC service (which can be provided together with the self-driving service) and transmit it to the vehicle. In the case where the PCC transmitting / receiving device is installed on the vehicle, the PCC transmitting / receiving device can receive / process content data related to an AR / VR / PCC service according to a user input signal input through a user interface device and provide it to the user. A vehicle or a user interface device according to an embodiment can receive a user input signal. A user input signal according to an embodiment may include a signal indicating a self-driving service.

[0226] The point cloud data transmission method / device according to an embodiment can be understood to involve Figure 1 a transmitting device 10000, Figure 1 a point cloud video encoder 10002, Figure 1 a transmitter 10003, Figure 2 an acquisition 20000-encoding 20001-transmission 20002, Figure 3 an encoder, Figure 8 a transmitting device, Figure 10 a device, Figure 11 GoF-based encoding, Figures 12 to 15 context-based encoding, the encoders of FIGS. 16A and 16B, Figures 18 to 23 bitstream and parameter generation, and Figure 24 terms of a transmission method.

[0227] The point cloud data transmission method / device according to an embodiment can be appreciated to involve Figure 1 a transmitting device 10000, Figure 1 a point cloud video encoder 10002, Figure 1 a transmitter 10003, Figure 2 an acquisition 20000 / encoding 20001 / transmission 20002, Figure 3 an encoder, Figure 8 a transmitting device, Figure 10 a device, Figure 11 GoF-based decoding, Figures 12 to 15 context-based decoding, the decoders of FIGS. 17A and 17B, Figures 18 to 23 bitstream and parameter parsing, and Figure 25 terms of a transmission method.

[0228] In addition, the point cloud data transmission / reception method / apparatus according to an embodiment may be abbreviated as the method / apparatus.

[0229] According to an embodiment, the geometric data, geometric information, position information, and geometry constituting the point cloud data shall be understood to have the same meaning. The attribute information and attribute data constituting the point cloud data shall be understood to have the same meaning.

[0230] The embodiment uses the continuity of the reference frames of the point cloud data to extend the encoding and decoding methods.

[0231] The embodiment relates to a method for improving the encoding and decoding performance by using the continuity of the reference frames of the point cloud data. In point cloud compression, the encoding efficiency of the bitstream can be improved by using the information about the prediction between the reference frames. The embodiment provides a compression method using the continuity of the reference frames as a method for improving the performance of point cloud compression.

[0232] This embodiment proposes a method for improving the performance of inter-frame compression of 3D point cloud data based on a Group of Pictures (GoP) that includes an Intra-frame (I-frame), a Predicted frame (P-frame), and a Bi-directional frame (B-frame). Based on the correlation between the frames, the redundant information generated in the inter-frame prediction can be signaled for replacement. In addition, the context continuity flag can be used by indexing the frames with similar geometric / attribute contexts.

[0233] A point cloud consists of a set of points, and each point can have geometric information and attribute information.

[0234] Reference Figure 3 and Figure 7 , the point cloud encoding process may include compressing the geometry and compressing the attribute information based on the geometry reconstructed based on the position information changed by the compression operation (reconstructed geometry = decoded geometry). The point cloud decoding process may include receiving the encoded geometry bitstream and the attribute bitstream, decoding the geometry, and decoding the attribute information based on the geometry reconstructed by the decoding operation.

[0235] Figure 11 Shows the I-frame and P-frame in each Group of Frames (GoF) according to an embodiment.

[0236] The point cloud data transmission method / apparatus according to an embodiment ( Figure 1 the transmitting apparatus 10000 of Figure 1 the point cloud video encoder 10002 of Figure 1 the transmitter 10003 of Figure 2 the acquire 20000 - encode 20001 - transmit 20002 of Figure 3 the encoder of Figure 8 the transmitting apparatus of Figure 10 the apparatus of Figures 12 to 15Context-based encoding, the encoders of FIGS. 16A and 16B, Figures 18 to 23 bitstream and parameter generation, and Figure 24 the transmission method) can perform GoF-based encoding as Figure 11 shown.

[0237] A point cloud data transmission method / apparatus according to an embodiment ( Figure 1 the transmission apparatus 10000, Figure 1 the point cloud video encoder 10002, Figure 1 the transmitter 10003, Figure 2 the acquisition 20000 / encoding 20001 / transmission 20002, Figure 3 the encoder, Figure 8 the transmission apparatus, Figure 10 the apparatus, Figures 12 to 15 context-based decoding, the decoders of FIGS. 17A and 17B, Figures 18 to 23 bitstream and parameter parsing, and Figure 25 the transmission method) can perform GoF-based decoding as Figure 11 shown.

[0238] For example, standards for point cloud compression and reconstruction include two versions: GPCCv1 (intra-frame encoding) and GPCCv2 (inter-frame encoding). In point cloud data, category 3 frame data corresponds to GPCCv2 inter-frame and takes the form of a set of consecutive frames. Similar to 2D video compression schemes, a group of frames (GoF) including I-frames and P / B-frames is defined for the inter-frame codec. Here, depending on how the GoF is defined, the GoF can be encoded / decoded in the form of IPPP, IPPBPP, or IBBPBB. The IPPP form can be defined as Figure 11 shown, depending on the reference frame.

[0239] For example, an I-frame as an intra-frame is an independent frame that is compressed and reconstructed as a single frame. A P-frame as a predictive frame is compressed and reconstructed by prediction or motion compensation based on a previous I-frame or P-frame. A B-frame as a bidirectional predictive frame is compressed and reconstructed by prediction based on previous frames and / or subsequent frames.

[0240] Embodiments can improve the inter-frame compression performance of point cloud data and propose a method for determining whether to use continuity information and signaling based on inter-frame correlation. The encoder can search for inter-frame correlation and perform encoding / decoding based on the continuity of the context of the reference frames in the GoF. In addition to the context-based continuous inter-frame encoding method, embodiments propose the following schemes for signaling the index of the reference frame: 1) the GoF frame structure, 2) searching for the correlation between reference frames in the GoF, 3) encoding / decoding the context continuity of the reference frames in the GoF, and 4) signaling the index of the continuous information.

[0241] The embodiments propose a compression method using a continuous context model of reference frames within the GoF or previous reference frames and a probability model for arithmetic coding. This method can improve the compression efficiency by using a method including encoding / decoding an index based on the correlation between the previous reference frame and the current frame as a way of using continuous probabilities.

[0242] Regarding the GoF frame structure, probability models inside or outside the GoF structure can be used for point clouds. The GoF can form a group of frames in the form of IBBPBBPBBP. The GoF structure can include I, P, and B frames. The group starting with an I frame can be defined in various forms. Context continuity can be defined inside or outside the defined structure.

[0243] The method / device according to an embodiment may include and execute a method of searching for the correlation between reference frames in the GoF. The GoF can be a group starting with an I frame and can be used as a unit for encoding / decoding. During the encoding / decoding process, context adaptive binary arithmetic coding (CABAC) can be used for point cloud data. In this case, the probability mode available for arithmetic decoding can be initialized for each frame. Alternatively, the probability mode may not be initialized, but can be reused in the next frame by searching for inter-frame correlations. When frames have similar characteristics, reusing the probability mode allows for exploiting symbol redundancy. Therefore, by reusing the context between frames with similar characteristics, the compression efficiency can be improved. In addition to context-based coding methods, models of arithmetic coding, Huffman coding, and range coding can be reused to use the probability continuity across frames.

[0244] The following method can be used to determine the inter-frame correlation according to an embodiment.

[0245] 1) I frame / P frame: The inter-frame correlation can be determined by comparing the I frame and the P frame.

[0246] 2) Difference between the global motion vector / matrix between the current frame and the previous frame: The inter-frame correlation can be determined based on the difference between the global motion vector or matrix between the current frame and the previous frame.

[0247] 3) Comparison of the value obtained by applying scaling to the translation and rotation of the global motion vector / matrix between the current frame and the previous frame with a set threshold: The inter-frame correlation can be determined based on whether the value obtained by applying scaling to the translation and / or rotation of the global motion vector or matrix between the current frame and the previous frame is greater than or less than the set threshold.

[0248] 4) Sum of the differences of the local motion vectors / matrices from the previous frame: The inter-frame correlation can be determined based on the sum of the differences of the local motion vectors or matrices between the current frame and the previous frame.

[0249] 5) Comparison of the value obtained by applying scaling to the translation and rotation of the local motion vector / matrix between the current frame and the previous frame with a set threshold: The inter-frame correlation can be determined based on whether the value obtained by applying scaling to the translation and / or rotation of the local motion vector or matrix between the current frame and the previous frame is greater than or less than the set threshold.

[0250] 6) RDO of the points / geometry / attribute values of the previous frame and the current frame: The inter-frame correlation can be determined based on the RDO of the points, the RDO of the geometry data, and / or the RDO of the attribute data from the current frame and the previous frame.

[0251] 7) Change in the motion vector value or motion compensation value of the road or object between the current frame and the previous frame: The inter-frame correlation can be determined based on the change in the motion vector and / or motion compensation of the road and / or object between the current frame and the previous frame.

[0252] Here, the previous frame can be the frame immediately preceding the current frame, or can be indexed as a previously decoded frame with high correlation. The current frame is decoded using the index obtained continuously using a probability model.

[0253] The method / apparatus according to an embodiment may include and use the context continuity of the reference frames in the GoF to perform encoding and decoding methods.

[0254] Encoding / decoding can be performed by sharing a probability model with the frames that determine the correlation between the reference frames. The encoder can encode the geometric information and attribute information about the current frame by continuing the context of the relevant frames. The decoder can perform decoding using the context that has been continuously used for the previous frames.

[0255] The context continuity-based encoding and decoding method according to an embodiment may include: 1) a context continuity-based encoding / decoding method regardless of the correlation within the GoF ( Figure 12 ); 2) a method of encoding and decoding only geometry based on context continuity after searching for correlation within the GoF ( Figure 13 ); 3) a method of encoding and decoding only attributes based on context continuity after searching for attribute correlation within the GoF ( Figure 14 ); and / or 4) a context continuity-based encoding / decoding method using the relationship between context correlation flags ( Figure 15 ).

[0256] Figure 12 A context continuity-based encoding and decoding method according to an embodiment is shown.

[0257] Figure 12A method for performing arithmetic coding / decoding on geometric data and attribute data by setting context continuity related flag information without determining the correlation within the GoF is shown.

[0258] Through the context continuity flags (geometry_context_continue_flag, attribute_context_continue_flag), encoding and decoding can be performed using the context continuity from the previous frame regardless of the inter-frame correlation. Thereby, the time required for correlation search can be reduced.

[0259] Depending on the value of the geometry_context_continue_flag, the application of the attribute context continuity can be determined.

[0260] For example, when a frame is initialized and in a random access period, the geometric entropy context can be initialized before arithmetic coding of the geometric data of the frame without using frame continuity. The attribute entropy context can be initialized before arithmetic coding of the attribute data. Then, the geometric and / or attribute data included in the frame can be encoded and decoded. The entropy context probability applied to the current frame is stored. Then, based on the frame count, it is determined whether all frames in the group of frames have been encoded or decoded, and the process is terminated.

[0261] When the frame is not in a random access period, it is determined whether the value of the geometric context continuous flag is true or false to use frame continuity. When the value of the flag is not true, the geometric entropy context is initialized. When the value of the flag is true, it is determined whether the value of the attribute context continuous flag is true or false. When the value of this flag is not true, the attribute entropy context is initialized. When the value of this flag is true, the geometric data and attribute data of the frame are encoded and decoded. In other words, the geometric entropy context and / or attribute entropy context of the current frame may not be initialized but can be used to perform arithmetic coding and arithmetic decoding on the geometric data and attribute data of the frame. The entropy context probability used is stored, and it is determined whether the program for all frames has been completed. This process is repeated for the next frame after the current frame in the group of frames.

[0262] For example, when a frame is initialized and in a random access period, the geometric entropy context can be initialized before arithmetic coding of the geometric data of the frame without using frame continuity. The attribute entropy context can be initialized before arithmetic coding of the attribute data. Then, the geometric and / or attribute data included in the frame can be encoded and decoded. The entropy context probability applied to the current frame is stored. Then, based on the frame count, it is determined whether all frames in the group of frames have been encoded or decoded, and the process is terminated.

[0263] When the frame is not in the random access period, determine whether the value of the geometric context continuity flag is true or false for using frame continuity. When the value of the flag is not true, initialize the geometric entropy context. In addition, initialize the attribute entropy context. When the value of the flag is true, the geometric data and the attribute data of the frame are encoded and decoded. In other words, the geometric entropy context and / or the attribute entropy context of the current frame may not be initialized but may be used to perform arithmetic coding and arithmetic decoding on the geometric data and the attribute data of the frame. Store the entropy context probability used and determine whether the program for all frames has been completed. Repeat this process for the next frame after the current frame in the group of frames.

[0264] The point cloud data transmitting device or encoder may set the values of the geometric context continuity flag and the attribute context continuity flag. The values of the geometric context continuity flag and the attribute context continuity flag may be set for the frames in the GoF. The values of the geometric context continuity flag and the attribute context continuity flag may be set before applying arithmetic coding. The point cloud data transmitting device or encoder may set the values of the geometric context continuity flag and the attribute context continuity flag based on the conditions for determining the inter-frame correlation. When encoding the GoF, the point cloud data transmitting device or encoder may perform context continuity encoding according to the values indicated by the geometric context continuity flag and the attribute context continuity flag without performing a separate frame correlation determination process.

[0265] In addition, as Figure 12 shown in two examples, the geometric measurement context continuity flag and the attribute context continuity flag may be defined and used separately. Alternatively, only the geometric context continuity flag may be used, and the processing of context continuity for geometry and attributes may be indicated based on the value of the geometric context continuity flag without separately defining or using the attribute context continuity flag.

[0266] The point cloud data receiving device or decoder may receive the geometric context continuity flag and / or the attribute context continuity flag set by the transmitting device or encoder and may perform context continuity-based arithmetic decoding on the geometric data and / or the attribute data.

[0267] Figure 13 A context continuity-based encoding and decoding method according to an embodiment is shown.

[0268] Figure 13 Describe a method of encoding and decoding only geometry based on context continuity after searching for correlation in the GoF.

[0269] When the geometry_context_continue_flag is set, inter-frame correlation can be searched. When the correlation is high, the geometric context from the previous frame can be used to continuously perform encoding / decoding. When there is no correlation, the geometric context can be initialized. The correlation can be determined differently for LiDAR content and object-centered content. For example, for LiDAR content, when it is determined that the center point for capturing the content has moved / rotated beyond a specific threshold, the geometric context from the previous frame can be used to continuously perform encoding / decoding. Depending on whether geometric context continuity is applied, it is determined whether to apply attribute context continuity.

[0270] The method / apparatus according to an embodiment initializes a frame. When it is a random access period, it is not necessary to use inter-frame continuity for random access, and the geometric entropy context is initialized. Then, the attribute entropy context is initialized. The geometric data and / or attribute data of the frame are encoded and decoded. The entropy context probability applied to the current frame is stored. The same process is repeated for the frames in the GoF.

[0271] When the frame is not in a random access period and the value of the geometric context continuous flag is not true, the correlation or similarity between the frames is determined. If the correlation or similarity is not true, the geometric entropy context and the attribute entropy context are initialized. If the correlation or similarity is true, the geometric data and / or attribute data of the frame are encoded and decoded without initializing the entropy context. The entropy context probability applied to the current frame is stored, and the same process is repeated for the frames in the GoF.

[0272] If the value of the geometric context continuous flag is true, the geometric data and / or attribute data of the current frame are encoded and decoded without determining the correlation or similarity. The entropy context probability applied to the current frame is stored. The same process is repeated for the frames in the GoF.

[0273] Figure 14 A context-continuity-based encoding and decoding method according to an embodiment is shown.

[0274] Figure 14 A method of using context continuity to encode and decode only attributes after searching for attribute correlation in the GoF is illustrated.

[0275] When the attribute_context_continue_flag is set, inter-frame correlation can be searched. When the correlation is high, encoding / decoding can be continuously performed using the attribute context from the previous frame. When there is no correlation, the attribute context can be initialized. The correlation can be determined differently for LiDAR content and general content. For example, for LiDAR content, when it is determined that the center point for capturing the content has moved / rotated beyond a specific threshold, encoding / decoding can be continuously performed using the attribute context from the previous frame. Alternatively, depending on whether inter-attribute prediction is applied, the attribute context can be continuously used from the previous frame. The attribute context continue flag can be applied independently of the geometric context continue flag.

[0276] The method / apparatus according to an embodiment initializes a frame and determines whether it is in a random access period. When it is in a random access cycle, the geometric entropy context is initialized without using frame continuity for random access. Then, the attribute entropy context is initialized. The geometric data and / or attribute data of the frame are encoded and decoded. The entropy context probability applied to the current frame is stored. The same process is repeated for the frames in the GoF.

[0277] When the frame is not in a random access cycle and the value of the geometric context continue flag is not true, the correlation or similarity between the frames is determined. If the correlation or similarity is not true, the geometric entropy context and the attribute entropy context are initialized. The geometric data and / or attribute data of the frame are encoded and decoded. The entropy context probability applied to the current frame is stored, and the same process is repeated for the frames in the GoF.

[0278] If the value of the geometric context continue flag is true and the value of the attribute context continue flag is not true, the correlation or similarity between the frames is determined. If the correlation or similarity is not true, the attribute entropy context is initialized. The geometric data and / or attribute data of the frame are encoded and decoded. The entropy context probability applied to the current frame is stored, and the same process is repeated for the frames in the GoF.

[0279] If the value of the geometric context continue flag is true and the value of the attribute context continue flag is also true, the geometric data and / or attribute data of the frame are encoded and decoded based on the geometric entropy context and the attribute entropy context. The entropy context probability applied to the current frame is stored. The same process is repeated for the frames in the GoF.

[0280] Figure 15 The relationship between context-related flags according to an embodiment is shown.

[0281] Figure 15 The relationship between context continuity flags is shown.

[0282] When the geometry_slice_context_continue_flag is set to true, the continuity with the previous frames in the GoF can be ignored.

[0283] If the value of the geometry slice context continue flag is true and the random access period is true, then the geometry entropy context is initialized, and the attribute entropy context is initialized. In addition, the geometry data and the attribute data of the frame are encoded and decoded. The entropy context probabilities applied to the current frame are stored. The same process is repeated for the frames in the GoF.

[0284] If the value of the geometry slice context continue flag is true, the random access period is not true, and the value of the geometry context continue flag is not true, then the geometry entropy context is initialized, and then the attribute entropy context is initialized. The geometry and attribute data of the frame are encoded and decoded. The entropy context probabilities applied to the current frame are stored. The same process is repeated for the frames in the GoF.

[0285] If the value of the geometry context continue flag is true and the value of the attribute context continuity flag is not true, then the attribute entropy context is initialized, and the geometry and attribute data of the frame are encoded and decoded. The entropy context probabilities applied to the current frame are stored. The same process is repeated for the frames in the GoF.

[0286] If the value of the attribute context continuity flag is true, then the geometry and attribute data of the frame are encoded and decoded. The entropy context probabilities applied to the current frame are stored. The same process is repeated for the frames in the GoF.

[0287] The method / apparatus according to an embodiment may signal an index of the continuity information. When the context or probability model of the previous frame is continuously used, the decoder may receive information about which previous frame is used. The index signaled to the decoder may indicate the distance from the current frame, the distance from the I-frame, or the number of frames from the start frame.

[0288] FIG. 16A and FIG. 16B illustrate a point cloud data encoding apparatus according to an embodiment.

[0289] Each component in FIG. 16A and FIG. 16B may correspond to hardware, software, a processor, and / or a combination thereof. The apparatus may include a memory storing instructions for encoding point cloud data and a processor, where the instructions may cause the processor to perform an encoding operation.

[0290] FIG. 16A illustrates an operation of encoding point cloud data based on a single frame when the single frame is input, and FIG. 16B further illustrates an operation of encoding point cloud data across multiple frames in the GoF.

[0291] The data input unit acquires point cloud data.

[0292] The coordinate transformer transforms the coordinate system of the geometric data representing the position of points.

[0293] The quantization / voxelization processor quantizes the geometric data based on quantization parameters and voxelizes the data into voxels.

[0294] The octree occupancy code generator generates occupancy codes indicating whether the points corresponding to the geometric data occupy the nodes in the octree.

[0295] The surface model processor can process the surface of the geometric data using trisoup or the like.

[0296] The geometric context continuity determination / storage section can determine whether to use context continuity before arithmetic coding and store the context used. The geometric context continuity determination / storage section can determine the correlation with the previous frame to determine the frame for which context continuity is to be used. The index of the frame using context continuity can be signaled to the decoder. Context continuity can be applied identically or differently to geometric data and / or attribute data. In the case of different applications, signaling information with different values can be provided for geometric data and / or attribute data to indicate to the decoder the index of the reference frame according to context continuity.

[0297] The arithmetic compiler performs arithmetic compilation on the geometric data.

[0298] When the attribute data, which is an attribute of a point, is color, the color transformation processor can transform the color scheme.

[0299] The attribute transformation processor can sort the points of the attribute data and search for neighbors.

[0300] The prediction / lifting / RAHT transformation processor can transform the attribute data based on prediction, lifting, and / or RAHT schemes.

[0301] The coefficient quantization processor can quantize the coefficients of the attribute data generated by the transformation.

[0302] The attribute context continuity determination / storage section can determine whether to use context continuity before arithmetic coding and store the context used. The attribute context continuity determination / storage section can determine the correlation with the previous frame to determine the frame for which context continuity is to be used. The index of the frame using context continuity can be signaled to the decoder. Context continuity can be applied identically or differently to geometric data and / or attribute data. In the case of different applications, signaling information with different values can be provided for geometric data and / or attribute data to indicate to the decoder the index of the reference frame according to context continuity.

[0303] FIG. 16B shows FIG. 16A in more detail. For the same components in FIGS. 16A and 16B, refer to the description given above.

[0304] The LPU partitioner partitions the point cloud data into largest prediction units (LPUs).

[0305] Depending on whether the frame containing the point cloud data is a P-frame or an I-frame, encoding is performed as described below.

[0306] The motion predictor predicts the motion of the geometric information (geometric data or points) included in a P-frame.

[0307] The motion compensator predicts the motion for a P-frame and compensates for the motion.

[0308] The geometric context continuity determination / storage section determines whether to use the context continuity of geometry or initialize a frame based on the value of the context continuity flag.

[0309] The inter-predictor between geometric information performs inter-frame prediction.

[0310] The geometric information entropy encoder performs arithmetic coding on the geometric data based on inter-frame prediction depending on whether to use context continuity.

[0311] The intra-predictor of geometric information performs intra-prediction on an I-frame.

[0312] The geometric information entropy encoder performs arithmetic coding on the geometric data in an I-frame.

[0313] When lossy coding is applied to the geometric data, the attribute information encoder adjusts the color of the attribute data. Once inter-prediction is performed on the geometric data, inter-prediction is performed on the attribute data. To use the inter-frame reference information, the attribute context continuity determination / storage unit performs the same operation. In this operation, the attribute context continuity determination and storage can be performed according to the value of the attribute context continuity flag. Alternatively, the same operation can be performed according to the value of the geometric context continuity flag instead of the attribute context continuity flag.

[0314] In the case of intra-prediction, the attribute information (attribute data) is predicted within a frame.

[0315] The attribute information entropy encoder performs arithmetic coding on the intra- or inter-predicted attribute information.

[0316] FIGS. 17A and 17B show a point cloud data decoding apparatus according to an embodiment.

[0317] Each component in FIGS. 17A and 17B can correspond to hardware, software, a processor, and / or a combination thereof. The decoder may further include a memory storing instructions for decoding point cloud data and a processor, where the instructions may cause the processor to perform the encoding operations.

[0318] FIG. 17A illustrates an operation of decoding point cloud data based on frame assistance when a single frame is input, while FIG. 17B further illustrates an operation of decoding point cloud data across multiple frames in a GoF.

[0319] The decoding operations in FIGS. 17A and 17B follow a reverse process of the encoding operations in FIGS. 16A and 16B. [[ID=⑥]]

[0320] The receiver receives a bitstream containing geometric data and attribute data.

[0321] The geometric context continuity processor may decode the geometric data based on context continuity using the index of the previously received frame.

[0322] The arithmetic decoder performs arithmetic decoding on the geometric data based on context continuity.

[0323] The octree reconstruction processor based on occupancy codes reconstructs an octree (occupancy tree).

[0324] The surface model processor processes the geometric surface based on triangle-based reconstruction, upsampling, or voxelization.

[0325] The geometric reconstructor reconstructs the decoded geometric data and provides it for attribute decoding.

[0326] The coordinate inverse transformer inversely transforms the coordinates transformed by the encoder.

[0327] The attribute context continuity processor may decode the attribute data based on context continuity using the index of the previously received frame. The attribute reconstruction process may correspond to a reverse process of the operations in the attribute context continuity determination / storage section.

[0328] The arithmetic decoder performs arithmetic decoding on the attribute data based on context continuity.

[0329] The inverse quantization processor inversely quantizes the coefficients of the attribute data, which is an inverse process of the quantization performed by the encoder.

[0330] The prediction / lifting / RAHT transform processor transforms the attribute data using one of the prediction, lifting, and RAHT methods applied by the encoder.

[0331] The attribute reconstructor reconstructs the attribute data in a reverse process of the attribute transformation performed by the encoder.

[0332] The color inverse transformer inversely transforms the color scheme of the attribute data.

[0333] Figure 18 FIG. B illustrates in more detail Figure 18 A.

[0334] The geometric context continuity processor of the geometric decoder determines whether to use geometric context continuity based on the value indicated by the context continuity flag included in the bitstream.

[0335] Based on the context continuity flag, the geometric entropy decoder performs arithmetic decoding on the geometric data.

[0336] For predictive decoding, the geometric data is partitioned into LPUs and / or PUs.

[0337] In the case of intra-geometric coding, the geometric information intra-frame prediction reconstructor predicts the geometric data in the frame.

[0338] The geometric transform inverse quantizer inverse quantizes the predicted geometric data.

[0339] The coordinate inverse transformer transforms the coordinate system of the geometric data by inversely performing the operations of the encoder.

[0340] In the case of inter-geometric compilation, the motion compensator predicts the motion between frames and compensates for the motion.

[0341] The geometric inter-frame prediction reconstructor predicts the geometric data regarding the current frame based on the reference frame.

[0342] The attribute context continuity processor of the attribute decoder in the decoder can decode the attribute value based on context continuity using the index of the previously received frame. The attribute reconstruction process can correspond to the reverse process of the operations of the attribute context continuity determination / storage section.

[0343] Based on the context continuity flag, the attribute residual entropy decoder performs arithmetic decoding on the attribute data.

[0344] Depending on whether intra-attribute coding or inter-coding is performed, the attribute data is predicted and reconstructed within the frame or by referring to the reference frame.

[0345] The color inverse transformer transforms the color scheme of the attribute data by inversely performing the operations of the encoder.

[0346] Figure 18 A bitstream including point cloud data and parameters according to an embodiment is shown.

[0347] A method / apparatus for transmitting point cloud data according to an embodiment (i.e., Figure 1 the transmitting device 10000, the point cloud video encoder 10002, the transmitter 10003 in Figure 2 the obtaining 20000 / encoding 20001 / transmitting 20002 in Figure 3 the encoder in Figure 8 the transmitting device in Figure 10the device in Figure 11 the GoF-based encoding in Figures 12 to 15 the context-based encoding in, the encoder in FIGS. 16A and 16B, and Figure 24 the sending method in Figures 18 to 23 generate the bitstream and parameters as shown in

[0348] The method / device for receiving point cloud data according to the embodiment (i.e., Figure 1 the receiving device 10004, the receiver 10005, the point cloud video decoder 10006 in Figure 2 the sending 20002 / decoding 20003 / rendering 20004 in Figure 7 the decoder in Figure 9 the receiving device in Figure 10 the device in Figure 11 the GoF-based decoding in Figures 12 to 15 the context-based decoding in, the decoder in FIGS. 17A and 17B, and Figure 25 the receiving method in Figures 18 to 23 receive and parse the bitstream and parameters as shown in

[0349] For example, the metadata can be generated and sent by the metadata processor (which can also be called the metadata generator) of the sending device, and can be received and obtained by the metadata parser of the receiving device.

[0350] Each abbreviation has the following meaning. An abbreviation can be referred to by another term within the scope of the equivalent meaning: SPS: Sequence Parameter Set; GPS: Geometric Parameter Set; APS: Attribute Parameter Set; TPS: Tile Parameter Set; Geom: Geometric bitstream = geometric slice header + geometric slice data; Attr: Attribute bitstream = attribute tile header + attribute tile data.

[0351] Slices, tiles, and data units represent the basic units of encoding and decoding. The terms slice, tile, and data unit can correspond to each other.

[0352] Tiles or slices can be provided to divide the point cloud into regions for processing. Each region has a different importance level. Depending on the importance, different filters or filter units can be applied, thus allowing the use of high-complexity, high-quality filtering methods in important regions.

[0353] Depending on the processing capability of the receiver, different filtering methods can be applied to different regions (divided by tiles or slices) instead of using complex filtering methods for the entire point cloud, thereby ensuring better quality for regions important to the user and appropriate latency in the system.

[0354] Thus, when a point cloud is partitioned into tiles, different filters or filter units can be applied to the corresponding tiles.

[0355] When a point cloud is partitioned into slices, different filters or filter units can be applied to the corresponding slices.

[0356] The use of geometric and attribute context continuity information can be signaled to and carried in the bitstream.

[0357] The bitstream can contain parameter sets and can contain slice-based geometric data and one or more attribute data.

[0358] A sequence can correspond to a frame and can be partitioned into multiple tiles. The basic unit for encoding / decoding of a tile can be a slice. The size and position of the bounding box included in each tile can be signaled by the TPS, which can contain a geometric slice header, geometric slice data, an attribute slice header, and attribute slice data for each slice. The slice header carries information related to the slice data. For example, this information can include a parameter set ID, a tile ID, a slice ID, a bounding box position / size, the size of a node, and the number of points included in the node.

[0359] The syntax of the parameters included in the Figure 18 bitstream is described below with reference to the corresponding drawings.

[0360] Figure 19 Illustrated is a sequence parameter set (SPS) in a bitstream according to an embodiment.

[0361] A method / apparatus according to an embodiment has context continuity structure information included in the SPS.

[0362] frameIdx: Signals an index indicating the sequential position of the frame in the current sequence.

[0363] context_continue_flag: Signals whether context continuity is applied to both geometry and attributes: true = the current frame uses the continuous context of the previous frame; false = the current frame does not use the continuous context of the previous frame.

[0364] context_continue_use_frameIdx: Indicates the index of the previous frame in which context continuity was used. The index can be signaled as the distance from the current frame, the distance from the first frame, or an index within a continuous P frame.

[0365] Figure 19 The SPS of

[0366] Figure 20Illustrates the tile parameter set (TPS) in the bitstream according to an embodiment.

[0367] The method / apparatus according to an embodiment may have context continuity structure information included in the TPS.

[0368] tile_ctr: Signals the sequential position of the tile that is the current tile among all tiles.

[0369] tile_context_continue_flag: Signals context continuity based on the tile. It may signal whether context continuity is applied to both geometry and attributes, or separately to: True = the current tile uses the context continued from the previous tile. False = the current tile does not use the context continued from the previous tile.

[0370] tile_context_continue_use_tileIdx: Indicates the index of the previous tile that uses context continuity. The index may be signaled as the distance from the current tile or from the first tile.

[0371] Figure 20 The TPS may also refer to the syntax of the parameters according to the GPCC standard document.

[0372] Figure 21 Illustrates the geometry parameter set (GPS) in the bitstream according to an embodiment.

[0373] The method / apparatus according to an embodiment may have context continuity structure information included in the GPS.

[0374] frameIdx: Signals the index of the frame in the current sequence.

[0375] geometry_context_continuous_flag: Signals context continuity on a geometry basis: True = the geometry of the current frame uses the continuous context of the geometry of the previous frame; False = the geometry of the current frame does not use the continuous context of the geometry of the previous frame.

[0376] geometry_context_continue_use_frameIdx: Indicates the index of the previous frame among the previous frames that uses context continuity. The index may be signaled as the distance from the current frame or from the first frame.

[0377] Figure 21 The GPS may also refer to the syntax of the parameters according to the GPCC standard document.

[0378] Figure 22 Shows the attribute parameter set (APS) in the bitstream according to an embodiment.

[0379] The method / apparatus according to an embodiment may have context continuity structure information included in the APS.

[0380] frameIdx: Signals the index of a frame in the current sequence.

[0381] attribute_context_continue_flag: Signals context continuity on an attribute basis: true = the attributes of the current frame use the continuous context of the attributes of the previous frame; false = the attributes of the current frame do not use the continuous context of the attributes of the previous frame.

[0382] attribute_context_continue_use_frameIdx: Indicates the index of the previous frame among the previous frames that uses context continuity. The index may be signaled as the distance from the current frame or from the first frame.

[0383] Figure 22 The APS may also refer to the syntax of the parameters according to the GPCC standard document.

[0384] The geometry_context_continue_flag and the attribute_context_continue_flag may be defined and signaled as a single context continuity flag without distinguishing between geometry and attributes.

[0385] Figure 23 Shows a geometry slice header (GSH) in a bitstream according to an embodiment.

[0386] The method / apparatus according to an embodiment may have context continuity structure information included in the GSH.

[0387] gsh_slice_id: Signals the index indicating the sequential position of the current slice.

[0388] geometry_slice_context_continue_flag: Signals context continuity on a geometry slice basis: true = the current slice uses the continuous context of the previous slice; false = the current slice does not use the continuous context of the previous slice.

[0389] geometry_slice_context_continue_use_frameIdx: Indicates the index of the previous slice among the previous slices that uses context continuity. The index may be signaled as the distance from the current slice or from the first slice.

[0390] Figure 23The GSH can further refer to the syntax of the parameters according to the GPCC standard document.

[0391] Figure 24 The figure illustrates a method for encoding point cloud data according to an embodiment.

[0392] The method / device for sending point cloud data according to an embodiment (i.e., Figure 1 the sending device 10000, the point cloud video encoder 10002, the transmitter 10003 in Figure 2 the obtaining 20000 / encoding 20001 / sending 20002 in Figure 3 the encoder in Figure 8 the sending device in Figure 10 the device in Figure 11 the GoF-based encoding in Figures 12 to 15 the context-based encoding in Figure 24 and the encoder in FIGS. 16A and 16B) encodes the point cloud data as

[0393] S2400: The method for sending point cloud data according to an embodiment of the present invention may include encoding the point cloud data.

[0394] The encoding operation according to an embodiment includes encoding operations performed by Figure 1 the sending device 10000, the point cloud video encoder 10002, the transmitter 10003 in Figure 2 the obtaining 20000 / encoding 20001 / sending 20002 in Figure 3 the encoder in Figure 8 the sending device, etc. in Figure 11 the GoF-based encoding in Figures 12 to 15 the context-based encoding in Figures 18 to 23 and the operations of the encoder in FIGS. 16A and 16B to encode the point cloud, and generate a bitstream and parameters as

[0395] S2401: The method for sending point cloud data according to an embodiment of the present invention may further include sending a bitstream containing the point cloud data. The sending operation according to an embodiment includes sending the encoded bitstream.

[0396] Referring to FIG. 16, the operation S2400 of encoding the point cloud data may include encoding the geometric data of the point cloud data and encoding the attribute data of the point cloud data. The encoding of the geometric data may include determining the continuity of the context of the geometric data and performing arithmetic encoding on the geometric data based on the context. The encoding of the attribute data may include determining the continuity of the context for the attribute data and performing arithmetic encoding on the attribute data based on the context.

[0397] Referring to Figure 11The operation S2400 of encoding point cloud data may include encoding the point cloud data based on a group of frames (GoF) including the current frame and at least one reference frame of the current frame, where the current frame contains the point cloud data.

[0398] Reference Figure 11 The operation S2400 of encoding point cloud data may further include performing arithmetic coding on the point cloud data based on the correlation with at least one reference frame in the GoF without initializing the context.

[0399] Reference Figure 12 The operation S2400 of encoding point cloud data may further include: based on a first value of a context_continue_flag, initializing an entropy context and performing arithmetic coding on the point cloud data, or based on a second value of the context_continue_flag, performing arithmetic coding on the point cloud data without initializing the entropy context. The first value of the context_continue_flag may indicate that the continuous context of the previous frame is not used for encoding the point cloud data in the current frame, and the second value may indicate that the continuous context of the previous frame is used for encoding the point cloud data in the current frame.

[0400] Reference Figure 13 The operation S2400 of encoding point cloud data may further include: searching for the correlation with at least one reference frame in the GOF based on a geometric context continue flag, and based on the found correlation being greater than or equal to a threshold, performing arithmetic coding on the geometric data of the current frame based on the continuous geometric context for the previous frame, or based on the found correlation being less than the threshold, initializing the geometric context for the previous frame.

[0401] Reference Figure 14 The operation S2400 of encoding point cloud data may further include: searching for the correlation with at least one reference frame in the GOF based on an attribute context continue flag, and based on the found correlation being greater than or equal to a threshold, performing arithmetic coding on the attribute data of the current frame based on the continuous attribute context for the previous frame, or based on the found correlation being less than the threshold, initializing the attribute context for the previous frame.

[0402] Reference Figures 18 to 24 The bitstream may contain at least one of a sequence parameter set, a tile parameter set, a geometry parameter set, an attribute parameter set, or a slice header. At least one of the sequence parameter set, the tile parameter set, the geometry parameter set, the attribute parameter set, or the slice header may include context continuity related information.

[0403] Figure 24 The sending method of Figure 1It is executed by the sending device. A point cloud data sending device may include: an encoder configured to encode point cloud data; and a transmitter configured to transmit a bitstream containing the point cloud data.

[0404] Figure 25 It shows a method for decoding point cloud data according to an embodiment.

[0405] The method / device for receiving point cloud data according to an embodiment (i.e., Figure 1 the receiving device 10004, the receiver 10005, the point cloud video decoder 10006 in Figure 2 the sending 20002 / decoding 20003 / rendering 20004 in Figure 7 the decoder in Figure 9 the receiving device in Figure 10 the device in Figure 11 the GoF-based decoding in Figures 12 to 15 the context-based decoding in Figure 25 and the decoder in FIGS. 17A and 17B) decodes the point cloud data as

[0406] S2500: The method for receiving point cloud data according to an embodiment may include receiving a bitstream containing the point cloud data.

[0407] The receiving operation according to an embodiment includes receiving a bitstream containing the point cloud data and a parameter set.

[0408] S2501: The method for receiving point cloud data according to an embodiment may further include decoding the point cloud data.

[0409] The decoding operation according to an embodiment may include Figure 1 the receiving device 10004, the receiver 10005, the point cloud video decoder 10006 in Figure 2 the sending 20002 / decoding 20003 / rendering 20004 in Figure 7 the decoder in Figure 9 the decoding operation of the receiving device in Figure 11 the GoF-based decoding in Figures 12 to 15 the context-based decoding in Figure 25 The method of Figure 24 may follow the reverse process of the method of

[0410] The operation S2501 of decoding the point cloud data may include decoding the geometric data of the point cloud data and decoding the attribute data of the point cloud data. The decoding of the geometric data may include performing arithmetic decoding on the geometric data based on context. The decoding of the attribute data may include performing arithmetic decoding on the attribute data based on context.

[0411] The operation S2501 of decoding point cloud data may further include decoding the point cloud data based on a group of frames (GoF) including a current frame and at least one reference frame for the current frame, where the current frame contains the point cloud data.

[0412] The operation S2501 of decoding point cloud data may include: initializing an entropy context and performing arithmetic decoding on the point cloud data based on a first value of a context continuity flag, or performing arithmetic decoding on the point cloud data without initializing the entropy context based on a second value of the context continuity flag. The first value of the context continuity flag may indicate that the continuous context of a previous frame is not used for encoding the point cloud data in the current frame, and the second value may indicate that the continuous context of the previous frame is used for encoding the point cloud data in the current frame.

[0413] In the operation S2500 of receiving a bitstream containing point cloud data, the bitstream may include at least one of a sequence parameter set, a tile parameter set, a geometry parameter set, an attribute parameter set, or a slice header, where at least one of the sequence parameter set, the tile parameter set, the geometry parameter set, the attribute parameter set, or the slice header may include context continuity related information.

[0414] Figure 25 The method may be performed by Figure 1 a receiving device. The point cloud data receiving device may include: a receiver configured to receive a bitstream containing point cloud data; and a decoder configured to decode the point cloud data.

[0415] The PCC encoding method, PCC decoding method, and signaling method according to the above embodiments may provide the following effects.

[0416] In an embodiment, geometric and attribute compression performance may be enhanced. In addition, by removing redundant information between highly correlated frames to the greatest extent, the bitstream can be effectively configured. By signaling the applicable range of each of the geometry and attributes, optimal compression efficiency can be achieved. In addition, since the probability model is not initialized for each frame, memory usage and computation time can be reduced.

[0417] The embodiment supports a method of dividing 3D map data captured by a LiDAR device and integrated into a single content segment into slices to apply an angular mode to support efficient geometric compression of the data.

[0418] Therefore, the embodiment may provide a method of dividing point cloud frames captured by a LiDAR device into slices for efficient geometric compression of geometry-based point cloud compression (G-PCC) when the frames are integrated into a single point cloud content, thereby improving geometric compression encoding / decoding efficiency.

[0419] The method / device for sending / receiving point cloud data according to an embodiment may more efficiently compress and reconstruct point cloud data based on an operation of partitioning point cloud data captured by a LiDAR device according to a 3D map and related signaling information.

[0420] Accordingly, the sending method / device according to an embodiment may effectively compress and send point cloud data, and transmit signaling information for the point cloud data, and the receiving method / device according to an embodiment may also effectively decode and reconstruct point cloud data.

[0421] The operation of the sending / receiving device according to the above embodiment may be described in conjunction with the following point cloud compression processing procedure.

[0422] Embodiments have been described in terms of a method and / or a device, and the description of the method and the description of the device may be applied complementarily to each other.

[0423] Although the drawings have been described separately for simplicity, new embodiments may be designed by combining the embodiments shown in the corresponding drawings. Designing a computer-readable recording medium on which those skilled in the art record a program for executing the above embodiments also falls within the scope of the appended claims and their equivalents. The device and method according to an embodiment may not be limited by the configurations and methods of the above embodiments. Various modifications may be made to the embodiments by selectively combining all or some of the embodiments. Although the preferred embodiments have been described with reference to the drawings, those skilled in the art will understand that various modifications and variations may be made to the embodiments without departing from the spirit or scope of the disclosure described in the appended claims. Such modifications should not be understood separately from the technical idea or perspective of the embodiments.

[0424] The various elements of the apparatus of the embodiments can be implemented by hardware, software, firmware, or a combination thereof. The various elements in the embodiments can be implemented by a single chip (e.g., a single hardware circuit). According to an embodiment, the components according to the embodiment can be implemented as separate chips respectively. According to an embodiment, at least one or more components of the apparatus according to the embodiment can include one or more processors capable of executing one or more programs. One or more programs can execute any one or more operations / methods according to the embodiment or include instructions for executing the same. The executable instructions for executing the method / operation of the apparatus according to the embodiment can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transitory CRM or other computer program product configured to be executed by one or more processors. Additionally, the memory according to the embodiment can be used as a concept that covers not only volatile memory (e.g., RAM), but also non-volatile memory, flash memory, and PROM. Additionally, it can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). Additionally, the processor-readable recording medium can be distributed to computer systems connected via a network such that the processor-readable code can be stored and executed in a distributed manner.

[0425] In the present disclosure, " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" can mean "A and / or B". Further, "A, B" can mean "A and / or B". Further, "A / B / C" can mean "at least one of A, B, and / or C". Additionally, "A / B / C" can mean "at least one of A, B, and / or C". Further, in this specification, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" can mean 1) only A, 2) only B, or 3) both A and B. In other words, the term "or" used in this document should be interpreted as indicating "additionally or alternatively".

[0426] Terms such as first and second can be used to describe the various elements of the embodiments. However, the various components according to the embodiment should not be limited by the above terms. These terms are only used to distinguish one element from another. For example, the first user input signal can be called the second user input signal. Similarly, the second user input signal can be called the first user input signal. The use of these terms should be interpreted without departing from the scope of the various embodiments. The first user input signal and the second user input signal are both user input signals, but do not mean the same user input signal unless the context clearly dictates otherwise.

[0427] The terms used to describe the embodiments are used to describe specific embodiments and are not intended to limit the embodiments. As used in the description of the embodiments and the claims, unless the context clearly dictates otherwise, the singular forms include plural referents. The expression "and / or" is used to include all possible combinations of the terms. Terms such as "comprising" or "having" are intended to indicate the presence of elements, quantities, steps, components, and / or assemblies, and it should be understood that the possibility of the presence of additional elements, quantities, steps, components, and / or assemblies is not excluded. As used herein, conditional expressions such as "if" and "when" are not limited to alternative cases, but are intended to perform related operations or interpret related definitions in accordance with specific conditions when the specific conditions are met.

[0428] The operations according to the embodiments described in this specification may be performed by a transmitting / receiving device including a memory and / or a processor according to the embodiments. The memory may store programs for processing / controlling the operations according to the embodiments, and the processor may control the various operations described in this specification. The processor may be referred to as a controller or the like. In an embodiment, the operations may be performed by firmware, software, and / or a combination thereof. The firmware, software, and / or a combination thereof may be stored in the processor or the memory.

[0429] The operations according to the above embodiments may be performed by a transmitting device and / or a receiving device according to the embodiments. The transmitting / receiving device may include: a transmitter / receiver configured to transmit and receive media data; a memory configured to store instructions (program code, algorithms, flowcharts, and / or data) for a process according to the embodiments; and a processor configured to control the operations of the transmitting / receiving device.

[0430] The processor may be referred to as a controller or the like and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above embodiments may be performed by the processor. In addition, the processor may be implemented as an encoder / decoder for the operations of the above embodiments.

[0431] [Mode for disclosure]

[0432] As described above, the relevant details have been described in the best mode for carrying out the embodiments.

[0433] [Industrial applicability]

[0434] As described above, the embodiments are fully or partially applicable to point cloud data transmitting / receiving devices and systems.

[0435] Those skilled in the art may change or modify the embodiments in various ways within the scope of the embodiments.

[0436] The embodiments may include variations / modifications within the scope of the claims and their equivalents.

Claims

1. A method for transmitting point cloud data, the method comprising: Encoding the point cloud data; And Transmitting a bitstream containing the point cloud data.

2. The method according to claim 1, wherein The encoding of the point cloud data includes: Encoding the geometric data of the point cloud data; and Encoding the attribute data of the point cloud data; Wherein, the encoding of the geometric data includes: Determining the continuity of the context for the geometric data; and Performing arithmetic coding on the geometric data based on the context; Wherein, the encoding of the attribute data includes: Determining the continuity of the context for the attribute data; and Performing arithmetic coding on the attribute data based on the context.

3. The method according to claim 1, wherein, The encoding of the point cloud data includes: Encoding the point cloud data based on a group of frames (GoF) including a current frame and at least one reference frame for the current frame, the current frame containing the point cloud data.

4. The method according to claim 3, wherein The encoding of the point cloud data further includes: Performing arithmetic coding on the point cloud data based on the correlation with at least one reference frame in the GoF without initializing the context.

5. The method according to claim 3, wherein, The encoding of the point cloud data further includes: Initializing an entropy context and performing arithmetic coding on the point cloud data based on a first value of a context continuity flag; or Performing arithmetic coding on the point cloud data without initializing the entropy context based on a second value of the context continuity flag, Wherein: The first value of the context continuity flag indicates that the continuous context of the previous frame is not used for encoding the point cloud data in the current frame; and The second value indicates that the continuous context of the previous frame is used for encoding the point cloud data in the current frame.

6. The method according to claim 3, wherein, The encoding of the point cloud data further includes: Searching for the correlation with at least one reference frame in the GOF based on a geometric context continuity flag; and Performing one of the following: Based on the found correlation being greater than or equal to a threshold, performing arithmetic coding on the geometric data of the current frame based on the continuous geometric context for the previous frame; or Based on the found correlation being less than the threshold, initializing the geometric context for the previous frame.

7. The method according to claim 3, wherein The encoding of the point cloud data further includes: Searching for the correlation with at least one reference frame in the GOF based on an attribute context continuity flag; and Performing one of the following: Based on the found correlation being greater than or equal to a threshold, performing arithmetic coding on the attribute data of the current frame based on the continuous attribute context for the previous frame; or Based on the found correlation being less than the threshold, initializing the attribute context for the previous frame.

8. The method according to claim 1, wherein The bitstream includes at least one of a sequence parameter set, a tile parameter set, a geometric parameter set, an attribute parameter set, or a slice header, Wherein, the at least one of the sequence parameter set, the tile parameter set, the geometric parameter set, the attribute parameter set, or the slice header includes context continuity related information.

9. An apparatus for transmitting point cloud data, comprising: An encoder configured to encode point cloud data; And A transmitter configured to transmit a bitstream containing the point cloud data.

10. A method for receiving point cloud data, comprising: Receiving a bitstream containing point cloud data; And Decoding the point cloud data.

11. The method according to claim 10, wherein, The decoding of the point cloud data includes: Decoding the geometric data of the point cloud data; and Decoding the attribute data of the point cloud data, Wherein, the decoding of the geometric data includes: Performing arithmetic decoding on the geometric data based on context; Wherein, the decoding of the attribute data includes: Performing arithmetic decoding on the attribute data based on context.

12. The method according to claim 10, wherein, The decoding of the point cloud data includes: Decoding the point cloud data based on a group of pictures (GoF) including a current frame and at least one reference frame for the current frame, the current frame containing the point cloud data.

13. The method according to claim 11, wherein The decoding of the point cloud data further includes: Initializing an entropy context and performing arithmetic decoding on the point cloud data based on a first value of a context continuity flag; or Performing arithmetic decoding on the point cloud data without initializing the entropy context based on a second value of the context continuity flag, Wherein: The first value of the context continuity flag indicates that the continuous context of a previous frame is not used for encoding the point cloud data in the current frame; and The second value indicates that the continuous context of the previous frame is used for encoding the point cloud data in the current frame.

14. The method according to claim 10, wherein The bitstream contains at least one of a sequence parameter set, a tile parameter set, a geometric parameter set, an attribute parameter set, or a slice header, Wherein, at least one of the sequence parameter set, the tile parameter set, the geometric parameter set, the attribute parameter set, or the slice header includes context continuity related information.

15. A device for receiving point cloud data, the device comprising: A receiver configured to receive a bitstream containing point cloud data; And A decoder configured to decode the point cloud data.

Images