Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Abstract

A decoding method according to an embodiment can include the steps of receiving a bitstream including point cloud data; and decoding the point cloud data. An encoding method according to an embodiment can include the steps of encoding point cloud data; and transmitting a bitstream including the point cloud data.

Description

Technical Field

[0001] The embodiments relate to methods and apparatus for processing point cloud content. Background Technology

[0002] Point cloud content is content represented by a point cloud, which is a collection of points belonging to a coordinate system representing three-dimensional space. Point cloud content can represent media configured in three dimensions and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and autonomous driving services. However, tens of thousands to hundreds of thousands of points are required to represent point cloud content. Therefore, methods for efficiently processing large amounts of point data are needed. Summary of the Invention

[0003] Technical issues

[0004] The embodiments provide an apparatus and method for efficiently processing point cloud data. The embodiments also provide a point cloud data processing method and apparatus for addressing addressing latency and encoding / decoding complexity.

[0005] The technical scope of the embodiments is not limited to the foregoing technical objects, but can be extended to other technical objects that can be inferred by those skilled in the art based on the full content disclosed herein.

[0006] Technical solution

[0007] The decoding method according to an embodiment may include: receiving a bitstream containing point cloud data, and decoding the point cloud data. The encoding method according to an embodiment may include: encoding the point cloud data, and sending a bitstream containing the point cloud data.

[0008] Beneficial effects

[0009] The apparatus and method according to the embodiments can process point cloud data with high efficiency.

[0010] The apparatus and method according to the embodiments can provide high-quality point cloud services.

[0011] The apparatus and method according to the embodiments can provide point cloud content for providing general services such as VR services and autonomous driving services. Attached Figure Description

[0012] The accompanying drawings are included to provide a further understanding of this disclosure and are incorporated in and constitute a part of this application. The drawings illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure. For a better understanding of the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the accompanying drawings. The same reference numerals will be used throughout the drawings to refer to the same or similar components.

[0013] Figure 1An exemplary point cloud content providing system according to an embodiment is shown; Figure 2 This is a block diagram illustrating the operation provided by point cloud content according to an embodiment; Figure 3 The figure illustrates an exemplary point cloud encoder according to an embodiment; Figure 4 An example of an octree and occupancy code according to an embodiment is shown; Figure 5 The illustration shows an example of point configuration in each LOD according to an embodiment; Figure 6 The illustration shows an example of point configuration in each LOD according to an embodiment; Figure 7 The diagram illustrates a point cloud decoder according to an embodiment; Figure 8 The figure shows a transmitting apparatus according to an embodiment; Figure 9 The figure shows a receiving device according to an embodiment; Figure 10 The diagram illustrates an exemplary structure operable with respect to a point cloud data transmission / reception method / apparatus according to an embodiment; Figure 11 The illustration shows the process of encoding, transmitting, and decoding point cloud data according to an embodiment. Figure 12 The illustration shows the transmission and decoding of a portion of a point cloud bitstream according to an embodiment. Figure 13 The illustration shows scalable decoding of point cloud data according to an embodiment; Figure 14 The illustration shows scalable encoding according to an embodiment; Figure 15 The diagram illustrates a layer group for scalable transmission according to an embodiment; Figure 16 The illustration shows partial geometric data and partial attribute data according to an embodiment; Figure 17 The illustration shows partial geometric data and partial attribute data according to an embodiment; Figure 18 The illustration shows partial geometric data and partial attribute data according to an embodiment; Figure 19 The illustration shows some geometric data and some properties according to an embodiment; Figure 20 The illustration shows a bitstream containing point cloud data and parameters according to an embodiment; Figure 21 The illustration shows scalable enhancement of LoD generation information according to an embodiment; Figure 22 The diagram illustrates the attribute parameter set according to an embodiment; Figure 23 The figure shows an encoder (transmitting device) according to an embodiment. Figure 24 The diagram illustrates a decoder (receiving device) according to an embodiment. Figure 25 The illustration shows a scalable scaling effect according to an embodiment; Figure 26 The illustration shows subgroup LoD sampling considering missing layers according to an embodiment; Figure 27 The illustration shows an encoder and decoder use case related to FGS according to an embodiment; Figure 28 The illustration shows an encoder and decoder use case related to FGS according to an embodiment; Figure 29 The diagram illustrates the sequence parameter set according to an embodiment; Figure 30 The diagram illustrates the sequence parameter set according to an embodiment; Figure 31 The diagram illustrates an encoding method according to an embodiment; and Figure 32 The diagram illustrates a decoding method according to an embodiment. Detailed Implementation

[0014] Now, reference will be made in detail to preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The following detailed description, given with reference to the accompanying drawings, is intended to explain exemplary embodiments of the present disclosure and not to illustrate only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0015] While most of the terms used in this disclosure are selected from commonly used terms in the art, the applicant has arbitrarily chosen some terms, and their meanings will be explained in detail in the following description as needed. Therefore, this disclosure should be understood based on the literal meaning of the terms rather than their simple names or connotations.

[0016] Figure 1 An exemplary point cloud content delivery system according to an embodiment is shown.

[0017] Figure 1 The point cloud content providing system shown may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 are capable of transmitting and receiving point cloud data via wired or wireless communication.

[0018] The point cloud data transmitting device 10000 according to an embodiment can acquire and process point cloud video (or point cloud content) and transmit it. According to an embodiment, the transmitting device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device, and / or a server. According to an embodiment, the transmitting device 10000 may include devices configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers.

[0019] According to an embodiment, the transmitting device 10000 includes a point cloud video acquirer 10001, a point cloud video encoder 10002, and / or a transmitter (or communication module) 10003.

[0020] The point cloud video acquirer 10001 according to an embodiment acquires point cloud video through processing procedures such as capture, synthesis, or generation. Point cloud video is point cloud content represented by a point cloud, which is a set of points located in 3D space, and may be referred to as point cloud video data, point cloud data, etc. The point cloud video according to an embodiment may include one or more frames. A frame represents a still image / picture. Therefore, point cloud video may include point cloud images / frames / pictures, and may be referred to as point cloud images, frames, or pictures.

[0021] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 can encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to an embodiment may include geometry-based point cloud compression (G-PCC) coding and / or video-based point cloud compression (V-PCC) coding or next-generation coding. The point cloud compression coding according to an embodiment is not limited to the above embodiments. The point cloud video encoder 10002 can output a bitstream containing the encoded point cloud video data. The bitstream may contain not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0022] According to an embodiment, transmitter 10003 transmits a bitstream containing encoded point cloud video data. The bitstream, according to an embodiment, is encapsulated in a file or segment (e.g., a streaming segment) and transmitted via various networks such as broadcast networks and / or broadband networks. Although not shown in the figures, transmitting device 10000 may include an encapsulator (or encapsulation module) configured to perform encapsulation operations. According to an embodiment, the encapsulator may be included in transmitter 10003. According to an embodiment, the file or segment may be transmitted via a network to receiving device 10004 or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). Transmitter 10003 according to an embodiment is capable of wired / wireless communication with receiving device 10004 (or receiver 10005) via networks such as 4G, 5G, and 6G. Additionally, the transmitter may perform necessary data processing operations depending on the network system (e.g., a 4G, 5G, or 6G communication network system). Transmitting device 10000 can transmit encapsulated data on demand.

[0023] The receiving device 10004 according to an embodiment includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include devices, robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)).

[0024] According to an embodiment, receiver 10005 receives a bitstream containing point cloud video data or a file / segment encapsulated with a bitstream from a network or storage medium. Receiver 10005 may perform necessary data processing according to the network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). According to an embodiment, receiver 10005 may decapsulate the received file / segment and output a bitstream. According to an embodiment, receiver 10005 may include a decapsulator (or decapsulator module) configured to perform a decapsulation operation. The decapsulator may be implemented as a separate element (or component) from receiver 10005.

[0025] The point cloud video decoder 10006 decodes the bitstream containing point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to the method in which the point cloud video data is encoded (e.g., the reverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding (the reverse process of point cloud compression). Point cloud decompression encoding includes G-PCC encoding.

[0026] Renderer 10007 renders decoded point cloud video data. Renderer 10007 can output point cloud content by rendering not only the point cloud video data but also the audio data. According to an embodiment, renderer 10007 may include a display configured to display the point cloud content. According to an embodiment, the display may be implemented as a separate device or component rather than included in renderer 10007.

[0027] The arrows indicated by dashed lines in the diagram represent the transmission path of the feedback information acquired by the receiving device 10004. The feedback information reflects the interactivity of the user consuming the point cloud content and includes information about the user (e.g., header orientation information, viewport information, etc.). Specifically, when the point cloud content is for a service requiring user interaction (e.g., autonomous driving services, etc.), the feedback information may be provided to the content sender (e.g., the sending device 10000) and / or the service provider. According to embodiments, the feedback information may be used in both the receiving device 10004 and the sending device 10000, or it may not be provided.

[0028] According to the embodiment, the head orientation information is information about the user's head position, orientation, angle, movement, etc. The receiving device 10004 according to the embodiment can calculate viewport information based on the head orientation information. The viewport information can be information about the area of ​​the point cloud video that the user is viewing. The viewpoint is the point through which the user views the point cloud video and can refer to the center point of the viewport area. That is, the viewport is the area centered on the viewpoint, and the size and shape of the area can be determined by the field of view (FOV). Therefore, in addition to head orientation information, the receiving device 10004 can extract viewport information based on the vertical or horizontal FOV supported by the device. Furthermore, the receiving device 10004 performs gaze analysis, etc., to examine the way the user consumes the point cloud, the area the user gazes at in the point cloud video, the gaze duration, etc. According to the embodiment, the receiving device 10004 can send feedback information including the gaze analysis results to the transmitting device 10000. The feedback information according to the embodiment can be acquired during rendering and / or display. The feedback information according to the embodiment can be acquired by one or more sensors included in the receiving device 10004. According to an embodiment, feedback information can be obtained by the renderer 10007 or by a separate external element (or device, component, etc.). Figure 1The dashed lines in the diagram represent the process of sending feedback information obtained by the renderer 10007. The point cloud content providing system can process (encode / decode) point cloud data based on the feedback information. Therefore, the point cloud video data decoder 10006 can perform decoding operations based on the feedback information. The receiving device 10004 can send the feedback information to the transmitting device 10000. The transmitting device 10000 (or the point cloud video data encoder 10002) can perform encoding operations based on the feedback information. Therefore, the point cloud content providing system can effectively process necessary data (e.g., point cloud data corresponding to the user's head position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide point cloud content to the user.

[0029] According to the embodiment, the transmitting device 10000 may be referred to as an encoder, transmitting device, transmitter, etc., and the receiving device 10004 may be referred to as a decoder, receiving device, receiver, etc.

[0030] According to the embodiments Figure 1 Point cloud data processed in a point cloud content provision system (through a series of processes including acquisition, encoding, transmission, decoding, and rendering) can be referred to as point cloud content data or point cloud video data. According to embodiments, point cloud content data can be used as a concept encompassing metadata or signaling information related to point cloud data.

[0031] Figure 1 The components of the point cloud content provided by the system can be implemented by hardware, software, processors, and / or combinations thereof.

[0032] Figure 2 This is a block diagram illustrating the point cloud content provisioning operation according to an embodiment.

[0033] Figure 2 The block diagram shows Figure 1 The operation of the point cloud content providing system described herein. As mentioned above, the point cloud content providing system can process point cloud data based on point cloud compression encoding (e.g., G-PCC).

[0034] A point cloud content providing system (e.g., point cloud sending device 10000 or point cloud video acquirer 10001) according to an embodiment can acquire point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system used to represent 3D space. The point cloud video according to an embodiment may include Ply (Polygon file format or Stanford Triangle format) files. When the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of the points. The position of each point may be represented by parameters (e.g., values ​​of the X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include the attributes of the points (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc., of each point). A point has one or more attributes. For example, a point may have a color attribute or two attributes: color and reflectivity. According to embodiments, geometry can be referred to as location, geometric information, geometric data, location information, location data, etc., and attributes can be referred to as attributes, attribute information, attribute data, etc. A point cloud content providing system (e.g., point cloud sending device 10000 or point cloud video acquirer 10001) can obtain point cloud data from information related to the point cloud video acquisition process (e.g., depth information, color information, etc.).

[0035] A point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) according to an embodiment can encode point cloud data (20001). The point cloud content providing system can encode point cloud data based on point cloud compression coding. As described above, point cloud data can include geometric information and attribute information about points. Therefore, the point cloud content providing system can perform geometric coding to encode geometry and output a geometric bitstream. The point cloud content providing system can perform attribute coding to encode attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system can perform attribute coding based on geometric coding. The geometric bitstream and attribute bitstream according to an embodiment can be multiplexed and output as a single bitstream. The bitstream according to an embodiment may also contain signaling information related to geometric coding and attribute coding.

[0036] A point cloud content providing system (e.g., transmitting device 10000 or transmitter 10003) according to an embodiment can transmit encoded point cloud data (20002). Figure 1As shown, encoded point cloud data can be represented by geometric bitstreams and attribute bitstreams. Additionally, the encoded point cloud data can be transmitted as a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometric encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and transmit it as a file or fragment.

[0037] The point cloud content providing system (e.g., receiving device 10004 or receiver 10005) according to the embodiment can receive a bitstream containing encoded point cloud data. Additionally, the point cloud content providing system (e.g., receiving device 10004 or receiver 10005) can demultiplex the bitstream.

[0038] A point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode encoded point cloud data (e.g., geometric bitstream, attribute bitstream) transmitted in a bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode point cloud video data based on signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode the geometric bitstream to reconstruct the location (geometry) of the points. The point cloud content providing system can reconstruct the attributes of the points by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can reconstruct point cloud video based on location according to the reconstructed geometry and the decoded attributes.

[0039] A point cloud content providing system (e.g., receiving device 10004 or renderer 10007) according to an embodiment can render decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) can use various rendering methods to render the geometry and attributes decoded through the decoding process. Points in the point cloud content can be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to the user through a display (e.g., a VR / AR display, a general display, etc.).

[0040] The point cloud content providing system (e.g., receiving device 10004) according to an embodiment can obtain feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. The feedback information and operation of the point cloud content providing system according to an embodiment are related to reference. Figure 1 The feedback information and operation described are the same, so their detailed description is omitted.

[0041] Figure 3 An exemplary point cloud encoder according to an embodiment is shown.

[0042] Figure 3 Show Figure 1 An example of a point cloud video encoder 10002. The point cloud encoder reconstructs and encodes point cloud data (e.g., point locations and / or attributes) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) based on network conditions or applications. When the total size of the point cloud content is large (e.g., providing 60 Gbps of point cloud content for 30 fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system can reconstruct the point cloud content based on a maximum target bitrate to provide the point cloud content according to network conditions, etc.

[0043] For reference Figure 1 and Figure 2 As described, the point cloud encoder can perform both geometric encoding and attribute encoding. Geometric encoding is performed before attribute encoding.

[0044] The point cloud encoder according to the embodiment includes a coordinate transformer (transform coordinates) 30000, a quantizer (quantize and remove points (voxarization)) 30001, an octree analyzer (analyze octrees) 30002, a surface approximation analyzer (analyze surface approximations) 30003, an arithmetic encoder (arithmetic encoding) 30004, a geometry reconstructor (reconstruct geometry) 30005, a color transformer (transform colors) 30006, an attribute transformer (transform attributes) 30007, a RAHT transformer (RAHT) 30008, a LOD generator (generate LODs) 30009, a lift transformer (lift) 30010, a coefficient quantizer (quantize coefficients) 30011, and / or an arithmetic encoder (arithmetic encoding) 30012.

[0045] Coordinate transformer 30000, quantizer 30001, octree analyzer 30002, surface approximation analyzer 30003, arithmetic encoder 30004, and geometric reconstructor 30005 can perform geometric encoding. Geometric encoding according to embodiments may include octree geometric encoding, direct encoding, trisoup geometric encoding, entropy encoding, and / or other geometric encoding tools. Direct encoding and trisoup geometric encoding are applied selectively or in combination. Geometric encoding is not limited to the examples described above.

[0046] As shown in the figure, the coordinate transformer 30000 according to an embodiment receives a position and transforms it into coordinates. For example, the position can be transformed into position information in three-dimensional space (e.g., three-dimensional space represented by the XYZ coordinate system). The position information in three-dimensional space according to an embodiment can be referred to as geometric information.

[0047] According to an embodiment, quantizer 30001 performs geometric quantization. For example, quantizer 30001 may quantize points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). Quantizer 30001 performs a quantization operation: multiplying the difference between the minimum position value and the position value of each point by a preset quantization scaling value, and then finding the nearest integer value by rounding the value obtained through multiplication. Thus, one or more points may have the same quantized position (or position value). According to an embodiment, quantizer 30001 performs voxelization based on the quantized position to reconstruct the quantized points. As in the case of pixels (the smallest unit containing 2D image / video information), points in point cloud content (or 3D point cloud video) according to an embodiment may be included in one or more voxels. As a combination of volume and pixel, the term voxel refers to a 3D cubic space generated when 3D space is divided into units (unit = 1.0) based on axes representing 3D space (e.g., X-axis, Y-axis, and Z-axis). The quantizer 30001 enables the matching of a group of points in 3D space to voxels. According to one embodiment, a voxel may comprise only one point. According to another embodiment, a voxel may include one or more points. To represent a voxel as a point, the location of the voxel's center can be set based on the locations of one or more points included within the voxel. In this case, attributes encompassing all locations within a voxel can be combined and assigned to the voxel.

[0048] According to the embodiment, the octree analyzer 30002 performs octree geometric encoding (or octree coding) to represent voxels in an octree structure. The octree structure represents points based on the matching of octree structures with voxels.

[0049] The surface approximation analyzer 30003 according to the embodiment can analyze and approximate an octree. The octree analysis and approximation according to the embodiment is a process of analyzing a region containing multiple points to efficiently provide an octree and voxelization.

[0050] According to an embodiment, the arithmetic encoder 30004 performs entropy encoding on octrees and / or approximate octrees. For example, the encoding scheme includes arithmetic encoding. As a result of the encoding, a geometric bitstream is generated.

[0051] The attribute encoding is performed by a color transformer 30006, an attribute transformer 30007, a RAHT transformer 30008, a LOD generator 30009, a boosting transformer 30010, a coefficient quantizer 30011, and / or an arithmetic encoder 30012. As described above, a point may have one or more attributes. The attribute encoding according to the embodiment is also applied to the attributes that a point has. However, when an attribute (e.g., color) includes one or more elements, the attribute encoding is applied independently to each element. The attribute encoding according to the embodiment includes color transformation encoding, attribute transformation encoding, region adaptive hierarchical transformation (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) encoding, and interpolation-based hierarchical nearest neighbor prediction (boosting transformation) encoding with update / boosting steps. Depending on the point cloud content, the above-described RAHT encoding, prediction transformation encoding, and boosting transformation encoding may be used selectively, or a combination of one or more encoding schemes may be used. The attribute encoding according to the embodiment is not limited to the examples described above.

[0052] The color converter 30006 according to an embodiment performs color transformation encoding that transforms the color values ​​(or textures) included in the attributes. For example, the color converter 30006 can transform the format of color information (e.g., from RGB to YCbCr). Optionally, the operation of the color converter 30006 according to an embodiment can be applied based on the color values ​​included in the attributes.

[0053] According to the embodiment, the geometry reconstructor 30005 reconstructs (decompresses) octrees and / or approximate octrees. The geometry reconstructor 30005 reconstructs the octree / voxel based on the results of analyzing the point distribution. The reconstructed octree / voxel may be referred to as the reconstructed geometry (recovered geometry).

[0054] According to the embodiment, the attribute transformer 30007 performs attribute transformation to transform attributes based on reconstructed geometry and / or locations without performing geometric encoding. As described above, since attributes depend on geometry, the attribute transformer 30007 can transform attributes based on reconstructed geometric information. For example, based on the position value of a point included in a voxel, the attribute transformer 30007 can transform the attributes of the point at that location. As described above, when the center position of a voxel is set based on the positions of one or more points included in the voxel, the attribute transformer 30007 transforms the attributes of one or more points. When performing trisoup geometric encoding, the attribute transformer 30007 can transform attributes based on the trisoup geometric encoding.

[0055] The attribute transformer 30007 performs attribute transformation by calculating the average of the attributes or attribute values ​​(e.g., color or reflectivity of each point) of neighboring points within a specific location / radius from the center (or location value) of each voxel. The attribute transformer 30007 can apply weights based on the distance from the center to each point when calculating the average. Therefore, each voxel has a location and a calculated attribute (or attribute value).

[0056] The attribute transformer 30007 can search for nearest neighbors within a specific location / radius of the center of each voxel based on a KD-tree or Morton code. A KD-tree is a binary search tree and supports a data structure that allows points to be managed based on location, enabling fast nearest neighbor search (NNS). Morton codes are generated by representing the coordinates (e.g., (x, y, z)) of the 3D location of all points as bit values ​​and mixing the bits. For example, when the coordinates representing the point location are (5, 9, 1), the bit values ​​are (0101, 1001, 0001). Mixing the bit values ​​according to the bit index in the order of z, y, and x produces 010001000111. This value is represented as the decimal number 1095. That is, the Morton code value for the point with coordinates (5, 9, 1) is 1095. The attribute transformer 30007 can sort the points based on their Morton code values ​​and perform NNS using a depth-first traversal process. After an attribute transformation operation, use a KD tree or Morton code when an NNS is needed in another transformation process used for attribute encoding.

[0057] As shown in the figure, the transformation properties are input to the RAHT transformer 30008 and / or the LOD generator 30009.

[0058] According to an embodiment, the RAHT transformer 30008 performs RAHT encoding for predicting attribute information based on reconstructed geometric information. For example, the RAHT transformer 30008 can predict the attribute information of higher-level nodes in an octree based on attribute information associated with lower-level nodes in the octree.

[0059] The LOD generator 30009 according to the embodiment generates a Level of Detail (LOD) to perform predictive transform coding. The LOD according to the embodiment represents the level of detail of the point cloud content. As the LOD value decreases, it indicates a deterioration in the detail of the point cloud content. As the LOD value increases, it indicates an enhancement in the detail of the point cloud content. Points can be classified by LOD.

[0060] The lift transformer 30010 according to the embodiment performs lift transform coding to transform point cloud attributes based on weights. As described above, lift transform coding may optionally be applied.

[0061] According to the embodiment, the coefficient quantizer 30011 quantizes the attribute encoded by the attribute based on the coefficient.

[0062] According to the embodiment, the arithmetic encoder 30012 encodes quantized attributes based on arithmetic coding.

[0063] Although not shown in the figure, Figure 3 The elements of the point cloud encoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above-described... Figure 3 At least one of the operation and / or functions of the elements of the point cloud encoder. Additionally, one or more processors are operable or perform operations for executing... Figure 3 The software program and / or instruction set for the operation and / or function of the elements of the point cloud encoder. One or more memories according to the embodiments may include high-speed random access memory, or may include non-volatile memory (e.g., one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices).

[0064] Figure 4 An example of an octree and occupancy code according to an embodiment is shown.

[0065] For reference Figures 1 to 3 As described, the point cloud content delivery system (point cloud video encoder 10002) or point cloud encoder (e.g., octree analyzer 30002) performs octree geometric encoding (or octree encoding) based on an octree structure to efficiently manage the regions and / or locations of voxels.

[0066] Figure 4 The upper part shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes of a coordinate system (e.g., the X, Y, and Z axes). This is achieved by two poles (0, 0, 0) and (2... d , 2 d , 2 d An octree structure is created by recursively subdividing a bounding box aligned to the cubic axis. Here, 2d can be set as the value of the minimum bounding box that constitutes all points surrounding the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following equation. In the following equation, (x int n , y int n , z int n ) indicates the position (or position value) of the quantized point.

[0067]

[0068] like Figure 4 As shown in the upper center, the entire 3D space can be divided into eight spaces according to partitions. Each partitioned space is represented by a cube with six faces. For example... Figure 4 As shown in the upper right, each of the eight spaces is further subdivided based on a coordinate system axis (e.g., the X, Y, and Z axes). Thus, each space is divided into eight smaller spaces. These smaller spaces are also represented by cubes with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0069] Figure 4 The lower part shows the octree occupancy code. The occupancy code generates the octree to indicate whether each of the eight partitions generated by dividing a space contains at least one point. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of a partitioned space, and each child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to a child node contains at least one point, the node is assigned a value of 1. When the space corresponding to a child node does not contain a point (the space is empty), the node is assigned a value of 0. Since... Figure 4 The occupancy code shown is 00100001, therefore, it indicates that the space corresponding to the third and eighth child nodes among the eight child nodes each contains at least one point. As shown, each of the third and eighth child nodes has eight child nodes, and the child nodes are represented by an 8-bit occupancy code. The figure shows that the occupancy code for the third child node is 10000111, and the occupancy code for the eighth child node is 01001111. A point cloud encoder (e.g., an arithmetic encoder 30004) according to an embodiment can perform entropy coding on the occupancy code. To increase compression efficiency, the point cloud encoder can perform intra-frame / inter-frame coding on the occupancy code. A receiving device (e.g., receiving device 10004 or point cloud video decoder 10006) according to an embodiment reconstructs the octree based on the occupancy code.

[0070] A point cloud encoder according to an embodiment (e.g., Figure 4 A point cloud encoder or octree analyzer (30002) can perform voxelization and octree encoding to store point locations. However, points are not always uniformly distributed in 3D space, so there may be specific regions with fewer points. Therefore, performing voxelization over the entire 3D space is inefficient. For example, when a specific region contains very few points, voxelization is not necessary in that specific region.

[0071] Therefore, for the aforementioned specific region (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment can skip voxelization and perform direct encoding to directly encode the point locations included in the specific region. Direct Encoding Mode (DCM) can be used to directly encode the coordinates of points. The point cloud encoder according to the embodiment can also perform trisoup geometric encoding based on a surface model, which reconstructs the point locations in the specific region (or node) based on voxels. Trisoup geometric encoding is a geometric encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct encoding and trisoup geometric encoding according to the embodiment can be performed selectively. Alternatively, direct encoding and trisoup geometric encoding according to the embodiment can be performed in combination with octree geometric encoding (or octree encoding).

[0072] To perform direct encoding, the option to apply direct encoding using direct mode should be enabled. The node to which direct encoding is applied must not be a leaf node, and there should be fewer than a threshold number of points within that node. Furthermore, the total number of points to which direct encoding is applied should not exceed a preset threshold. When the above conditions are met, the point cloud encoder (or arithmetic encoder 30004) according to the embodiment can perform entropy encoding on the point locations (or location values).

[0073] A point cloud encoder according to an embodiment (e.g., a surface approximation analyzer 30003) can determine a specific level of an octree (a level less than the depth d of the octree) and can begin using a surface model at that level to perform trisoup geometry encoding to reconstruct point locations in a node region based on voxels (trisoup mode). The point cloud encoder according to an embodiment can specify the level at which trisoup geometry encoding is applied. For example, the point cloud encoder does not operate in trisoup mode when the specific level is equal to the depth of the octree. In other words, the point cloud encoder according to an embodiment can operate in trisoup mode only when the specified level is less than the depth value of the octree. The 3D cubic region of a node at a specified level according to an embodiment is called a block. A block may include one or more voxels. A block or voxel may correspond to a cube. Geometry is represented as surfaces within each block. A surface according to an embodiment may intersect each edge of a block at most once.

[0074] A block has 12 edges, therefore a block contains at least 12 intersections. Each intersection is called a vertex. Vertices along an edge are detected when there is at least one occupied voxel adjacent to the edge in all blocks sharing the edge. An occupied voxel, according to an embodiment, refers to a voxel containing a point. The vertex position detected along an edge is the average position of the edges of all voxels adjacent to the edge in all blocks sharing the edge.

[0075] Once a vertex is detected, the point cloud encoder according to the embodiment can perform entropy encoding on the edge's origin (x, y, z), the edge's direction vector (Δx, Δy, Δz), and the vertex position value (relative position value within the edge). When trisoup geometry encoding is applied, the point cloud encoder according to the embodiment (e.g., geometry reconstructor 30005) can generate the restored geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization processes.

[0076] Vertices located at the edges of a block determine the surface passing through the block. According to the embodiment, the surface is a non-planar polygon. During triangle reconstruction, the surface represented by the triangles is reconstructed based on the origin of the edges, the direction vectors of the edges, and the position values ​​of the vertices. The triangle reconstruction process is performed as follows: i) calculating the centroid value of each vertex, ii) subtracting the centroid value from each vertex value, and iii) estimating the sum of squares of the values ​​obtained through the subtraction.

[0077]

[0078] Estimate the minimum value of the sum and perform a projection process based on the axis with the minimum value. For example, when element x is minimum, each vertex is projected onto the x-axis relative to the center of the block, and the projection is onto the (y, z) plane. When the value obtained by the projection onto the (y, z) plane is (ai, bi), the value of θ is estimated by atan2(bi, ai), and the vertices are sorted based on the value of θ. The following shows the vertex combinations for creating triangles based on the number of vertices. Vertices are sorted from 1 to n. The following shows that for four vertices, two triangles can be constructed based on vertex combinations. The first triangle can be composed of vertices 1, 2, and 3 from the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 from the sorted vertices.

[0079] Table 2-1. Triangles formed from vertices sorted by 1, ..., n

[0080] n triangles

[0081] 3 (1,2,3)

[0082] 4 (1,2,3), (3,4,1)

[0083] 5 (1,2,3), (3,4,5), (5,1,3)

[0084] 6 (1,2,3), (3,4,5), (5,6,1), (1,3,5)

[0085] 7 (1,2,3), (3,4,5), (5,6,7), (7,1,3), (3,5,7)

[0086] 8 (1,2,3), (3,4,5), (5,6,7), (7,8,1), (1,3,5), (5,7,1)

[0087] 9 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,1,3), (3,5,7), (7,9,3)

[0088] 10 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,1), (1,3,5), (5,7,9), (9,1,5)

[0089] 11 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,11), (11,1,3), (3,5,7), (7,9,11), (11,3,7)

[0090] 12 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,11), (11,12,1), (1,3,5), (5,7,9), (9,11,1), (1,5,9)

[0091] An upsampling process is performed to add points along the edges of the triangle at the center, and voxelization is then performed. The added points are generated based on the upsampling factor and the width of the block. The added points are called thinned vertices. The point cloud encoder according to an embodiment can voxelize the thinned vertices. Additionally, the point cloud encoder can perform attribute encoding based on the voxelized positions (or position values).

[0092] Figure 5 An example of point configuration in each LOD according to an embodiment is shown.

[0093] For reference Figures 1 to 4 The described approach involves reconstructing (decompressing) the encoded geometry before performing attribute encoding. When direct encoding is applied, the geometry reconstruction operation may include altering the placement of directly encoded points (e.g., placing directly encoded points in front of the point cloud data). When trisoup geometry encoding is applied, the geometry reconstruction process is performed through triangle reconstruction, upsampling, and voxelization. Since attributes depend on geometry, attribute encoding is performed based on the reconstructed geometry.

[0094] A point cloud encoder (e.g., LOD generator 30009) can classify (or reorganize) points according to LOD. The figure shows the point cloud content corresponding to LOD. The leftmost image in the figure represents the original point cloud content. The second image from the left in the figure represents the point distribution in the lowest LOD, and the rightmost image represents the point distribution in the highest LOD. That is, points are sparsely distributed in the lowest LOD and densely distributed in the highest LOD. In other words, as the LOD increases in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between points narrows.

[0095] Figure 6 An example of point configuration for each LOD according to an embodiment is shown.

[0096] For reference Figures 1 to 5 As described, a point cloud content providing system or point cloud encoder (e.g., point cloud video encoder 10002, Figure 3 A point cloud encoder or LOD generator (30009) can generate LODs. LODs are generated by reorganizing points into a set of refined levels based on a set of LOD distance values ​​(or a set of Euclidean distances). The LOD generation process is performed not only by the point cloud encoder but also by the point cloud decoder.

[0097] Figure 6 The upper part shows examples of points (P0 to P9) of point cloud content distributed in 3D space. Figure 6 In this context, the original order represents the order of points P0 to P9 before LOD generation. Figure 6 In this context, LOD-based order represents the order in which points are generated according to their LOD. Points are reorganized by LOD. Additionally, higher LODs include points belonging to lower LODs. For example... Figure 6 As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, the points of LOD1, P9, P8, and P7.

[0098] For reference Figure 3 As described, the point cloud encoder according to the embodiments may selectively or in combination perform predictive transform coding, lifting transform coding, and RAHT transform coding.

[0099] The point cloud encoder according to an embodiment can generate predictors for points to perform predictive transformation coding for setting the predictive attributes (or predictive attribute values) of each point. That is, N predictors can be generated for N points. The predictors according to an embodiment can calculate weights (=1 / distance) based on the LOD value of each point, index information of neighboring points existing within a set distance of each LOD, and the distance to the neighboring points.

[0100] According to an embodiment, the predicted attribute (or attribute value) is set as the average of values ​​obtained by multiplying the attributes (or attribute values) of neighboring points (e.g., color, reflectivity, etc.) set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 30011) according to an embodiment can quantize and inverse quantize the residual (which may be referred to as residual attribute, residual attribute value, or attribute prediction residual, attribute residual) obtained by subtracting the predicted attribute (or attribute value) from the attribute (attribute value) of each point. The quantization process is configured as shown in the table below.

[0101] Table. Pseudocode for Attribute Prediction Residual Quantization

[0102] int PCCQuantization(int value, int quantStep) {

[0103] if (value >= 0) {

[0104] return floor(value / quantStep + 1.0 / 3.0);

[0105] } else {

[0106] return -floor(-value / quantStep+1.0 / 3.0);

[0107] }

[0108] }

[0109] TABLE Attribute prediction residuals inverse quantization pseudo code

[0110] int PCCInverseQuantization(int value, int quantStep) {

[0111] if (quantStep == 0) {

[0112] return value;

[0113] } else {

[0114] return value quantStep;

[0115] }

[0116] }

[0117] When the predictors of each point have neighboring points, the point cloud encoder (e.g., arithmetic encoder 30012) according to the embodiment can perform entropy encoding on the residual values ​​of quantization and inverse quantization as described above. When the predictors of each point do not have neighboring points, the point cloud encoder (e.g., arithmetic encoder 30012) according to the embodiment can perform entropy encoding on the attributes of the corresponding point without performing the above operations.

[0118] The point cloud encoder (e.g., lift transformer 30010) according to an embodiment can generate predictors for each point, set the calculated LOD and register neighboring points in the predictors, and set weights based on the distance to the neighboring points to perform lift transform coding. The lift transform coding according to the embodiment is similar to the predictive transform coding described above, but the difference is that weights are applied cumulatively to the attribute values. The process of cumulatively applying weights to attribute values ​​according to the embodiment is configured as follows.

[0119] 1) Create an array quantized weights (QW) to store the weight values ​​of each point. The initial value of all elements of QW is 1.0. Multiply the QW value of the predictor index of the neighboring nodes registered in the predictor by the weight of the current point's predictor, and add the values ​​obtained by multiplication.

[0120] 2) Improve the prediction process: Subtract the value obtained by multiplying the attribute value of the point by the weight from the existing attribute value to calculate the predicted attribute value.

[0121] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0122] 4) The weights calculated by multiplying the weights computed for all predictors by the weights stored in the QW corresponding to the predictor index are summed with the updateweight array and used as the index of the neighbor node. The values ​​obtained by multiplying the attribute values ​​of the neighbor node indexes by the calculated weights are summed with the update array.

[0123] 5) Improve the update process: Divide the attribute values ​​of the update array of all predictors by the weight values ​​of the updateweight array of the predictor index, and add the existing attribute values ​​to the values ​​obtained by division.

[0124] 6) For all predictors, the predicted attribute is calculated by multiplying the attribute value updated through the boosting update process by the weight updated through the boosting prediction process (stored in QW). The predicted attribute value is quantized by a point cloud encoder (e.g., coefficient quantizer 30011) according to the embodiment. Additionally, the point cloud encoder (e.g., arithmetic encoder 30012) performs entropy encoding on the quantized attribute value.

[0125] A point cloud encoder according to an embodiment (e.g., RAHT transform 30008) can perform RAHT transform coding, where attributes associated with lower-level nodes in an octree are used to predict attributes of higher-level nodes. RAHT transform coding is an example of intra-frame attribute coding performed by scanning backward through an octree. The point cloud encoder according to an embodiment scans the entire region starting from voxels and repeats a merging process at each step, merging voxels into larger blocks, until the root node is reached. The merging process according to the embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on the node directly above an empty node.

[0126] The following equation represents the RAHT transformation matrix. In this equation, Indicates level The average attribute value of the voxels at that location. Based on and To be calculated. Used for and The weight is and .

[0127]

[0128] here, It is a low-pass value and is used in the next higher level of merging. This represents the high-pass coefficient. At each step, the high-pass coefficient is quantized and subjected to entropy encoding (e.g., encoded by an arithmetic encoder 300012). Weights are calculated as follows: .pass and Create the root node as follows.

[0129]

[0130] The value of gDC is also quantized and entropy-encoded, just like the high-pass coefficient.

[0131] Figure 7 A point cloud decoder according to an embodiment is shown.

[0132] Figure 7 The point cloud decoder shown is an example of a point cloud decoder and can perform decoding operations. Figures 1 to 6 The reverse process of the encoding operation of the point cloud encoder is shown.

[0133] For reference Figure 1 and Figure 6 As described, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0134] The point cloud decoder according to the embodiment includes an arithmetic decoder (arithmetic decoding) 7000, an octree synthesizer (synthesized octree) 7001, a surface approximation synthesizer (synthesized surface approximation) 7002, a geometry reconstructor (reconstructed geometry) 7003, an inverse coordinate transformer (inverse coordinate transformation) 7004, an arithmetic decoder (arithmetic decoding) 7005, an inverse quantizer (inverse quantization) 7006, a RAHT transformer 7007, a LOD generator (generated LOD) 7008, an inverse lifter (inverse lift) 7009, and / or a color inverse transformer (inverse color transformation) 7010.

[0135] An arithmetic decoder 7000, an octree synthesizer 7001, a surface approximation synthesizer 7002, a geometry reconstructor 7003, and a coordinate inverse transformer 7004 can perform geometric decoding. Geometric decoding according to embodiments may include direct decoding and trisoup geometric decoding. Direct encoding and trisoup geometric decoding are selectively applied. Geometric decoding is not limited to the examples described above and is provided as a reference. Figures 1 to 6 The reverse process of the described geometric encoding is executed.

[0136] According to an embodiment, the arithmetic decoder 7000 decodes the received geometric bitstream based on arithmetic coding. The operation of the arithmetic decoder 7000 corresponds to the inverse process of the arithmetic encoder 30004.

[0137] The octree synthesizer 7001 according to an embodiment can generate an octree by obtaining a octet code (or information about the geometry obtained as a decoding result) from the decoded geometry bitstream. The octet code is as shown in the reference... Figures 1 to 6 Please describe that configuration in detail.

[0138] When applying trisoup geometry encoding, the surface approximation synthesizer 7002 according to the embodiment can synthesize the surface based on the decoded geometry and / or the generated octree.

[0139] According to an embodiment, the geometry reconstructor 7003 can regenerate geometry based on surface and / or decoded geometry. See also... Figures 1 to 9 As described, direct encoding and trisoup geometric encoding are selectively applied. Therefore, geometry reconstructor 7003 directly imports and sums the positional information of points for which direct encoding has been applied. When trisoup geometric encoding is applied, geometry reconstructor 7003 can reconstruct the geometry by performing reconstruction operations (e.g., triangle reconstruction, upsampling, and voxelization) of geometry reconstructor 30005. Details and references Figure 6 The descriptions are the same for all of them, so their descriptions are omitted. The reconstructed geometry may include point cloud images or frames that do not contain attributes.

[0140] According to the embodiment, the inverse coordinate transformer 7004 can obtain the point position based on the reconstructed geometric transformation coordinates.

[0141] Arithmetic decoder 7005, inverse quantizer 7006, RAHT transformer 7007, LOD generator 7008, inverse booster 7009, and / or color inverse transformer 7010 can perform reference... Figure 6 The attribute decoding described herein includes Region Adaptive Hierarchical Transformation (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) decoding, and interpolation-based hierarchical nearest neighbor prediction (lifting transformation) decoding with update / lifting steps. These three decoding schemes may be used selectively, or a combination of one or more decoding schemes may be used. The attribute decoding according to the embodiments is not limited to the examples described above.

[0142] According to the embodiment, the arithmetic decoder 7005 decodes the attribute bitstream through arithmetic encoding.

[0143] According to the embodiment, the inverse quantizer 7006 inversely quantizes information about the decoded attribute bitstream or the attributes obtained as a decoding result, and outputs the inversely quantized attributes (or attribute values). Inverse quantization can be selectively applied based on the attribute encoding of the point cloud encoder.

[0144] According to an embodiment, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 can handle the reconstructed geometry and inverse quantization attributes. As described above, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 can selectively perform decoding operations corresponding to the encoding of the point cloud encoder.

[0145] According to the embodiment, the color inverse transformer 7010 performs inverse transformation encoding to inversely transform the color values ​​(or textures) included in the decoded attributes. The operation of the color inverse transformer 7010 can be selectively performed based on the operation of the color transformer 30006 of the point cloud encoder.

[0146] Although not shown in the figure, Figure 7 The elements of the point cloud decoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above-described... Figure 7 The point cloud decoder's components include at least one or more operations and / or functions. Additionally, one or more processors are operable or perform operations for executing... Figure 7 The software program and / or instruction set for the operation and / or function of the elements of the point cloud decoder.

[0147] Figure 8A transmitting apparatus according to an embodiment is shown.

[0148] Figure 8 The transmitting device shown is Figure 1 The transmitting device 10000 (or Figure 3 Example of a point cloud encoder. Figure 8 The transmitting device shown can perform the same operation as the reference. Figures 1 to 6 The described point cloud encoder includes one or more of the same or similar operations and methods. The transmitting apparatus according to embodiments may include a data input unit 8000, a quantization processor 8001, a voxelization processor 8002, an octree occupancy code generator 8003, a surface model processor 8004, an intra / inter-frame coding processor 8005, an arithmetic encoder 8006, a metadata processor 8007, a color transformation processor 8008, an attribute transformation processor 8009, a prediction / boosting / RAHT transformation processor 8010, an arithmetic encoder 8011, and / or a transmission processor 8012.

[0149] According to an embodiment, the data input unit 8000 receives or acquires point cloud data. The data input unit 8000 can perform operations and / or acquisition methods similar to those of the point cloud video acquirer 10001 (or refer to...). Figure 2 The described acquisition process (20000) is the same or similar operation and / or acquisition method.

[0150] The data input unit 8000, quantization processor 8001, voxelization processor 8002, octree occupancy code generator 8003, surface model processor 8004, intra / inter-frame coding processor 8005, and arithmetic encoder 8006 perform geometric coding. Geometric coding according to the embodiment and reference... Figures 1 to 9 The geometric codes described are the same or similar, so their detailed descriptions are omitted.

[0151] The quantization processor 8001 according to the embodiment quantizes geometry (e.g., point position values). The operation of the quantization processor 8001 and / or quantization with reference... Figure 3 The operation and / or quantization of the described quantizer 30001 are the same or similar. Details and references Figures 1 to 9 The descriptions are the same.

[0152] According to the embodiment, the voxelization processor 8002 voxels the quantized position values ​​of points. The voxelization processor 8002 can perform operations similar to those described above. Figure 3 The operation and / or voxelization process of the quantizer 30001 described are the same as or similar to the operation and / or process. Details and references Figures 1 to 6 The descriptions are the same.

[0153] According to an embodiment, the octree occupancy code generator 8003 performs octree encoding based on the voxelized positions of points in the octree structure. The octree occupancy code generator 8003 can generate occupancy codes. The octree occupancy code generator 8003 can perform operations similar to those referenced... Figure 3 and Figure 4 The operations and / or methods described are the same as or similar to those of the point cloud encoder (or octree analyzer 30002). Details and references Figures 1 to 6 The descriptions are the same.

[0154] According to an embodiment, the surface model processor 8004 can perform trisoup geometric encoding based on a surface model to reconstruct point positions in a specific region (or node) on a voxel basis. The surface model processor 8004 can perform operations related to reference... Figure 3 The operations and / or methods described are the same as or similar to those of the point cloud encoder (e.g., surface approximation analyzer 30003). Details and references are available. Figures 1 to 6 The descriptions are the same.

[0155] According to an embodiment, the intra / inter-frame coding processor 8005 can perform intra / inter-frame coding on point cloud data. The intra / inter-frame coding processor 8005 can perform operations similar to those described above. Figure 7 The described intra / inter-frame coding is the same or similar. See details and references. Figure 7 Those described are the same. According to an embodiment, the intra / inter-frame coding processor 8005 may be included in the arithmetic encoder 8006.

[0156] According to an embodiment, the arithmetic encoder 8006 performs entropy encoding on octrees and / or approximate octrees of point cloud data. For example, the encoding scheme includes arithmetic encoding. The arithmetic encoder 8006 performs the same or similar operations and / or methods as the arithmetic encoder 30004.

[0157] The metadata processor 8007 according to an embodiment processes metadata (e.g., set values) about point cloud data and provides it to necessary processing procedures such as geometric encoding and / or attribute encoding. Additionally, the metadata processor 8007 according to an embodiment can generate and / or process signaling information related to geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be encoded separately from the geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be interleaved.

[0158] Color transformation processor 8008, attribute transformation processor 8009, prediction / boosting / RAHT transformation processor 8010, and arithmetic encoder 8011 perform attribute encoding. Attribute encoding and reference according to the embodiment. Figures 1 to 6 The attribute codes described are the same or similar, so their detailed descriptions are omitted.

[0159] According to an embodiment, a color transformation processor 8008 performs color transformation encoding to transform color values ​​included in attributes. The color transformation processor 8008 may perform color transformation encoding based on reconstructed geometry. The reconstructed geometry and reference... Figures 1 to 9 The description is the same. Furthermore, its execution is the same as the reference. Figure 3 The operation and / or methods of the described color converter 30006 are the same as or similar to those described. Detailed descriptions are omitted.

[0160] According to an embodiment, the attribute transformation processor 8009 performs attribute transformation to transform attributes based on reconstructed geometry and / or locations where geometric encoding is not performed. The attribute transformation processor 8009 performs transformations with reference to... Figure 3 The operation and / or method of the described attribute transformer 30007 are the same as or similar to those described. Detailed descriptions thereof are omitted. The prediction / boosting / RAHT transformation processor 8010 according to the embodiment can encode the transformed attributes by any one or a combination of RAHT encoding, prediction transformation encoding, and boosting transformation encoding. The prediction / boosting / RAHT transformation processor 8010 performs and references... Figure 3 The RAHT transformer 30008, LOD generator 30009, and boost transformer 30010 described herein operate at least one of the same or similar operations. Furthermore, the predictive transform coding, boost transform coding, and RAHT transform coding are similar to those of the reference transformer. Figures 1 to 9 The descriptions are the same, so their detailed descriptions are omitted.

[0161] The arithmetic encoder 8011 according to the embodiment can encode the attributes of the code based on arithmetic encoding. The arithmetic encoder 8011 performs the same or similar operations and / or methods as the arithmetic encoder 30012.

[0162] The transmission processor 8012 according to an embodiment can transmit individual bitstreams containing encoded geometric and / or encoded attribute and metadata information, or transmit a single bitstream configured to contain encoded geometric and / or encoded attribute and metadata information. When the encoded geometric and / or encoded attribute and metadata information according to an embodiment is configured as a single bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to an embodiment may contain signaling information and slice data. The signaling information includes a sequence parameter set (SPS) for sequence-level signaling, a geometric parameter set (GPS) for signaling for geometric information encoding, an attribute parameter set (APS) for signaling for attribute information encoding, and a tile parameter set (TPS) for tile-level signaling. The slice data may include information about one or more slices. A slice according to an embodiment may include a geometric bitstream Geom0. 0and one or more attribute bitstreams Attr0 0 and Attr1 0 .

[0163] A slice is a series of syntactic elements that represent a point cloud frame, either fully or partially encoded.

[0164] According to an embodiment, the TPS may include information about each tile in one or more tiles (e.g., coordinate information and height / size information about the bounding box). The geometric bitstream may include a header and a payload. The header of the geometric bitstream according to an embodiment may include a geom_parameter_set_id, a geom_tile_id, and a geom_slice_id included in the GPS, as well as information about the data contained in the payload. As described above, the metadata processor 8007 according to an embodiment may generate and / or process signaling information and transmit it to the transmission processor 8012. According to an embodiment, the element performing geometry encoding and the element performing attribute encoding may share data / information with each other, as indicated by the dashed lines. The transmission processor 8012 according to an embodiment may perform the same or similar operations and / or transmission methods as the transmitter 10003. Details and References Figure 1 and Figure 2 The descriptions are the same, so their descriptions are omitted.

[0165] Figure 9 An example of a receiving device according to an embodiment is shown.

[0166] Figure 9 The receiving device shown in the diagram is Figure 1 The receiving device 10004 (or Figure 10 and Figure 11 An example of a point cloud decoder. Figure 9 The receiving device shown in the diagram can perform and reference... Figures 1 to 11 The same or similar one or more operations and methods described in the point cloud decoder.

[0167] The receiving apparatus according to an embodiment may include a receiver 9000, a receiving processor 9001, an arithmetic decoder 9002, an octree reconstruction processor based on occupancy codes 9003, a surface model processor (triangle reconstruction, upsampling, voxelization) 9004, an inverse quantization processor 9005, a metadata parser 9006, an arithmetic decoder 9007, an inverse quantization processor 9008, a prediction / boost / RAHT inverse transform processor 9009, a color inverse transform processor 9010, and / or a renderer 9011. Each decoding element according to an embodiment can perform the inverse process of the operation of the corresponding encoding element according to the embodiment.

[0168] Receiver 9000 according to an embodiment receives point cloud data. Receiver 9000 can perform operations related to... Figure 1 The operation and / or receiving method of the receiver 10005 are the same as or similar to those of the receiver. Detailed description omitted.

[0169] According to an embodiment, the receiving processor 9001 can acquire a geometric bitstream and / or an attribute bitstream from the received data. The receiving processor 9001 may be included in the receiver 9000.

[0170] The arithmetic decoder 9002, the octet-based octree reconstruction processor 9003, the surface model processor 9004, and the inverse quantization processor 9005 can perform geometric decoding. Geometric decoding and reference according to the embodiment... Figures 1 to 10 The described geometric decodings are the same or similar, so their detailed descriptions are omitted.

[0171] The arithmetic decoder 9002 according to the embodiment can decode a geometric bitstream based on arithmetic coding. The arithmetic decoder 9002 performs the same or similar operations and / or encodings as the arithmetic decoder 7000.

[0172] According to an embodiment, the octree reconstruction processor 9003 based on octane codes can reconstruct an octree by obtaining octane codes from the decoded geometric bitstream (or information about the geometry obtained as a decoding result). The octree reconstruction processor 9003 performs the same or similar operations and / or methods as the octree synthesizer 7001 and / or the octree generation method. When applying trisoup geometry encoding, the surface model processor 9004 according to an embodiment can perform trisoup geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on surface modeling methods. The surface model processor 9004 performs the same or similar operations as the surface approximation synthesizer 7002 and / or the geometry reconstructor 7003.

[0173] According to the embodiment, the inverse quantization processor 9005 can inverse quantize the decoded geometry.

[0174] Metadata parser 9006 according to an embodiment can parse metadata (e.g., set values) contained in received point cloud data. Metadata parser 9006 can deliver the metadata to geometry decoder and / or attribute decoder. Metadata and reference Figure 8 The metadata described is the same, so its detailed description is omitted.

[0175] Arithmetic decoder 9007, inverse quantization processor 9008, prediction / boost / RAHT inverse transform processor 9009, and color inverse transform processor 9010 perform attribute decoding. Attribute decoding and referenceFigures 1 to 10 The properties described are decoded the same or similarly, so their detailed descriptions are omitted.

[0176] The arithmetic decoder 9007 according to an embodiment can decode the attribute bitstream via arithmetic coding. The arithmetic decoder 9007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 9007 performs the same or similar operations and / or encodings as the arithmetic decoder 7005.

[0177] According to an embodiment, the inverse quantization processor 9008 can reversibly quantize and decode attribute bitstreams. The inverse quantization processor 9008 performs the same or similar operations and / or methods as the inverse quantizer 7006 and / or the inverse quantization method.

[0178] According to an embodiment, the predictive / boosting / RAHT inverse transform processor 9009 can process reconstructed geometry and inverse quantized attributes. The predictive / boosting / RAHT inverse transform processor 9009 performs one or more operations and / or decodings that are the same as or similar to those of the RAHT transformer 7007, LOD generator 7008, and / or inverse booster 7009. According to an embodiment, the color inverse transform processor 9010 performs inverse transform encoding to inversely transform color values ​​(or textures) included in the decoded attributes. The color inverse transform processor 9010 performs operations and / or inverse transform encodings that are the same as or similar to those of the color inverse transformer 7010. According to an embodiment, the renderer 9011 can render point cloud data.

[0179] Figure 10 An exemplary structure operable in conjunction with a point cloud data transmission / reception method / apparatus according to an embodiment is shown.

[0180] Figure 10 The structure represents a configuration in which at least one of the following components—server 1060, robot 1010, autonomous vehicle 1020, XR device 1030, smartphone 1040, home appliance 1050, and / or head-mounted display (HMD) 1070—is connected to cloud network 1000. Robot 1010, autonomous vehicle 1020, XR device 1030, smartphone 1040, or home appliance 1050 are referred to as devices. Furthermore, XR device 1030 may correspond to a point cloud data (PCC) device according to an embodiment or be operatively connected to a PCC device.

[0181] Cloud Network 1000 can refer to a network that forms part of or exists within a cloud computing infrastructure. Here, Cloud Network 1000 can be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

[0182] Server 1060 can be connected via cloud network 1000 to at least one of robot 1010, self-driving vehicle 1020, XR device 1030, smartphone 1040, home appliance 1050 and / or HMD 1070, and can assist at least a portion of the processing of connected devices 1010 to 1070.

[0183] HMD 1070 represents one of the implementation types of the XR device and / or PCC device according to the embodiments. The HMD-type device according to the embodiments includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0184] Hereinafter, various embodiments of the apparatus 1010 to 1050 that apply the above-described technology will be described. Figure 10 The devices 1010 to 1050 shown are operable to be connected to / coupled to the point cloud data transmitting and receiving devices according to the above embodiments.

[0185] <PCC+XR>

[0186] The XR / PCC device 1030 may employ PCC technology and / or XR (AR+VR) technology, and may be implemented as an HMD, a head-up display (HUD) installed in a vehicle, a television, a mobile phone, a smartphone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a stationary robot, or a mobile robot.

[0187] The XR / PCC device 1030 can analyze 3D point cloud data or image data acquired through various sensors or from external devices and generate positional and attribute data about 3D points. Thus, the XR / PCC device 1030 can acquire information about the surrounding space or real-world objects and render and output XR objects. For example, the XR / PCC device 1030 can match an XR object, including auxiliary information about the identified object, with the identified object and output a matched XR object.

[0188] <PCC+XR+Mobile Phone>

[0189] The XR / PCC device 1030 can be implemented as a smartphone 1040 by applying PCC technology.

[0190] The 1040 smartphone can decode and display point cloud content based on PCC technology.

[0191] <PCC+Self-Driving+XR>

[0192] The self-driving vehicle 1020 can be realized as a mobile robot, vehicle, unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0193] The self-driving vehicle 1020 employing XR / PCC technology can refer to a self-driving vehicle equipped with means for providing XR images, or a self-driving vehicle serving as a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, the self-driving vehicle 1020 can be distinguished from and operatively connected to the XR device 1030.

[0194] The autonomous vehicle 1020, equipped with means for providing XR / PCC images, can acquire sensor information from sensors including cameras and output generated XR / PCC images based on the acquired sensor information. For example, the autonomous vehicle 1020 may have a HUD and output XR / PCC images to it, thereby providing passengers with XR / PCC objects corresponding to real objects or objects presented on a screen.

[0195] When an XR / PCC object is output to a HUD, at least a portion of the XR / PCC object can be output to overlap with the actual object being pointed at by the passenger's eyes. Conversely, when an XR / PCC object is output to a display installed within the autonomous vehicle, at least a portion of the XR / PCC object can be output to overlap with objects on the screen. For example, the autonomous vehicle 1020 can output XR / PCC objects corresponding to objects such as roads, other vehicles, traffic lights, traffic signs, two-wheeled vehicles, pedestrians, and buildings.

[0196] Virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or point cloud compression (PCC) technologies according to the embodiments are applicable to various devices.

[0197] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. AR technology, on the other hand, refers to the technology of displaying virtually created CG images on top of images of real objects. MR technology is similar to AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology differs from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with the same characteristics as real objects. More specifically, an example of MR technology application is holographic services.

[0198] Recently, VR, AR, and MR technologies have often been referred to as scaled reality (XR) technologies rather than being clearly distinguished from each other. Therefore, embodiments of this disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are suitable for such technologies.

[0199] The PCC method / apparatus according to the embodiments can be applied to vehicles that provide autonomous driving services.

[0200] Vehicles providing autonomous driving services connect to the PCC device for wired / wireless communication.

[0201] When the point cloud data (PCC) transmitting / receiving device according to an embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services (which may be provided together with autonomous driving services) and transmit it to the vehicle. When the PCC transmitting / receiving device is installed in the vehicle, it can receive / process content data related to AR / VR / PCC services based on user input signals input through a user interface device and provide it to the user. The vehicle or user interface device according to an embodiment can receive user input signals. User input signals according to an embodiment may include signals indicating autonomous driving services.

[0202] The encoding method / apparatus according to the embodiments includes Figure 1 The transmitting device 10000, the point cloud video encoder 10002, and the transmitter 10003, Figure 2 Get 20000 - Encode 20001 - Send 20002 Figures 11 to 13 encoder, Figures 14 to 19 Partial encoding, Figures 20 to 22 and Figures 29 to 30 Bitstream and parameter generation, Figures 23 to 28 Encoding and Figure 31 The encoding method.

[0203] The decoding method / apparatus according to the embodiments includes Figure 1 The receiving device 10004, receiver 10005, and point cloud video decoder 10006. Figure 2 Sending 20002 - Decoding 20003 - Rendering 20004 Figure 7 decoder Figure 9 The receiving device Figure 10 The device Figures 11 to 13 decoder Figures 14 to 19 Partial decoding, Figures 20 to 22 and Figures 29 to 30 Bitstream and parameter parsing, Figures 23 to 28 Decoding and Figure 32 Decoding method.

[0204] The encoding / decoding method / apparatus according to the embodiments may be simply referred to as the method / apparatus according to the embodiments.

[0205] According to the embodiments, geometric data, geometric information, location information, etc., that constitute point cloud data are interpreted as having the same meaning. Attribute data, attribute information, etc., that constitute point cloud data are interpreted as having the same meaning.

[0206] The encoding / decoding method according to the embodiment includes a method for sampling direction signaling for subgroups of multiple decoders using LoD.

[0207] The embodiments include methods for effectively supporting selective decoding of partial data when such decoding is required during the transmission and reception of point cloud data due to receiver performance or transmission rate. In this case, scalable enhancement methods for scalable coding can be used in attribute coding, which requires additional consideration for use in partial geometry data coding environments.

[0208] The embodiments relate to techniques for constructing data structures composed of point clouds. The embodiments include methods for informing a receiver of the sampling location by taking into account a portion of the geometric data encoding environment in the generation of a scalable LoD for scalable attribute encoding.

[0209] refer to Figure 3 and Figure 7 Point cloud data consists of the location (geometric) (e.g., XYZ coordinates) and attributes (e.g., color, reflectivity, intensity, grayscale, opacity, etc.) of each data point. In point cloud compression (PCC), octree-based compression is performed to effectively compress the non-uniform distribution of data in 3D space, and attribute information is compressed based on this.

[0210] The encoding / decoding method according to the embodiment can increase compression efficiency by alternately applying sampling positions based on the octree depth under the assumption of full octree depth geometry in LoD generation used in scalable uplift, by using attributes for positions closer to the node center. When scalable transmission is supported, partial geometry data is supported. In this case, a mismatch may occur between the encoder and decoder for sampling positions used in scalable uplift LoD generation, which may lead to inaccurate representations of parts of the point cloud. The mismatch between geometry and attributes during partial decoding can be resolved by changing or explicitly signaling the sampling reference positions according to the method of the embodiment.

[0211] The embodiments include a method for considering subgroup LoD sampling direction signaling for multiple decoders of a single bitstream.

[0212] Figure 11 The illustration shows the process of encoding, transmitting, and decoding point cloud data according to an embodiment.

[0213] Point cloud data is compressed and delivered by dividing location information associated with data points and related characteristic information such as color / brightness / reflectivity into geometric and attribute information, respectively. In this regard, point cloud (PC) data can be constructed based on an octree structure with layers according to the degree of detail or based on the level of detail (LoD). This scheme enables scalable point cloud data encoding and representation. In this case, depending on the receiver performance or transmission rate, it is possible to decode or represent only a portion of the point cloud data. However, currently, there is no method to remove unnecessary data beforehand. That is, when only a portion of the scalable PCC bitstream needs to be sent (e.g., only some layers are decoded during scalable decoding), it is not possible to selectively send only the necessary parts. Therefore, two methods are used: 1) re-encoding the necessary parts after decoding; 2) sending the entire bitstream so that the receiver can selectively apply it. However, in the case of method 1), delays may occur due to the time required for decoding and re-encoding. In the case of method 2), bandwidth efficiency may be reduced due to the transmission of unnecessary data. Furthermore, data quality must be degraded to use a fixed bandwidth.

[0214] Figure 12 The illustration shows the transmission and decoding of a portion of the point cloud bitstream according to an embodiment.

[0215] When compressed data is divided and delivered according to layers, only the necessary portions of the compressed data can be selectively delivered at the bitstream level without a separate conversion process. This is efficient in terms of storage space, as each stream requires only one storage space. Furthermore, because only the necessary layers are selected before transmission (via a bitstream selector), efficient transmission in terms of bandwidth is achieved.

[0216] Figure 13 The illustration shows scalable decoding of point cloud data according to an embodiment.

[0217] From the receiver's (decoder's) perspective, if information for reconstructing the entire PCC data is delivered regardless of the receiver's performance, the receiver needs to perform a process of selecting only the data corresponding to the desired layer after reconstructing the point cloud data through decoding (data selection or subsampling). In this case, because the delivered bitstream has already been decoded, it may cause latency in low-latency receivers, or depending on the receiver's performance, decoding may be impossible. However, when the bitstream is divided into slices to be delivered, the receiver can selectively deliver the bitstream to the decoder based on the decoder's performance or application fields according to the density of the point cloud data to be represented. In this case, because selection is performed before decoding, decoder efficiency can be improved, and decoders of various performance levels can be supported.

[0218] Figure 14 The illustration shows scalable encoding according to an embodiment.

[0219] The encoding / decoding method according to the embodiments may include scalable encoding and decoding of point cloud data.

[0220] Scalable encoding: The encoding method according to the embodiments can compress and deliver point cloud data to achieve scalable transmission. First, tools capable of supporting scalability can be used in performing location and attribute compression. Additionally, in delivering the compressed data, slices can be subdivided to achieve partial decoding, and deterministic criteria for selecting each slice before decoding can be signaled. Information for selecting the slices required by the receiver can be delivered. Scalable transmission can refer to a situation where only a portion of the bitstream is delivered or decoded, rather than the entire bitstream, which may result in low-resolution point cloud data.

[0221] When scalable transfer is applied to a geometry bitstream based on an octree (the same as an occupancy tree), point cloud data can be constructed from the bitstream of each octree layer ranging from the root node to the leaf node, based solely on information about the octree layer up to a specific octree layer. Therefore, there is no dependency on lower octree layer information used for the target octree layer (the layer of the occupancy tree). Dependency conditions can be used as a common constraint applied to both geometry and attribute encoding.

[0222] In attribute encoding, when using scalable boosting, partial decoding can be performed based on octree depth, similar to octree-based geometric encoding. Each octree depth can belong to a LoD, and one or more octree depths can match a specific LoD. Figure 14 The illustration shows a method for generating LoD (Level of Detail) when using scalable promotion. It illustrates a method based on a geometric octree structure to select an attribute of a parent node as one of the attributes of its child nodes. Figure 14 As shown, when the octree depth is 4, the attribute of a node belonging to octree depth 3 can be determined by selecting one child node from the child nodes belonging to octree depth 4. Similarly, the attribute of a node belonging to octree depth 2 can be selected from the attributes of the child nodes belonging to octree depth 3. In scalable promotion, attributes can be assigned to nodes at all octree levels by recursively using the attributes of representative attributes based on the attributes of child nodes in the direction from the leaf node to the root node.

[0223] Figure 14The diagram further illustrates how the position of the selected child node varies depending on the octree level. At the leaf node level (octree depth 4), the first node can be used as a property of the parent node during sampling, while at octree depth 3, the last node can be used as a property of the parent node. The node marked by shading (1400) indicates the position selected as a property of the parent node.

[0224] The encoding method according to the embodiments may include encoding the entire attribute data based on the LoD associated with the octree (or occupancy tree). For a node belonging to each depth (or level) of the octree (or occupancy tree), if the index of the depth (or level) is even, the attribute of the first child node belonging to a parent node in sorted order can be set (sampled) as the attribute of the parent node. If the index of the depth (or level) is odd, the attribute of the last child node belonging to a parent node in sorted order can be set (sampled) as the attribute of the parent node.

[0225] The decoding method according to the embodiments may include decoding the entire attribute data based on the LoD associated with the octree (or occupancy tree). For a node belonging to each depth (or level) of the octree (or occupancy tree), if the index of the depth (or level) is even, the attribute of the first child node in sorted order belonging to a parent node can be set (sampled) as the attribute of the parent node. If the index of the depth (or level) is odd, the attribute of the last child node in sorted order belonging to a parent node can be set (sampled) as the attribute of the parent node.

[0226] Figure 15 The illustration shows a layer group for scalable transmission according to an embodiment.

[0227] The encoding method according to the embodiments may include generating a layer group for scalable transmission.

[0228] When geometry and attribute encoding schemes support layer-based scalability and slicing is configured according to the layer structure, scalability before decoding can be supported, as in scalable transport applications. Figure 15 The diagram illustrates a method for configuring layer groups based on an octree structure.

[0229] Layer group: refers to the bundle of layer structure units that appear in G-PCC coding, such as octree layers or LoD layers.

[0230] Subgroup: Refers to a set of neighboring nodes based on positional information used for a layer group. Alternatively, bundling can be configured based on the lowest layer (which can refer to the layer closest to the root; for example, layer 6 in the case of group 3). Bundling of neighboring nodes can be configured according to Morton code order, distance-based adjacency, or coding order. Furthermore, nodes in a parent-child relationship can be defined as existing within a subgroup.

[0231] When defining subgroups, boundaries can appear in the middle of a layer. Regarding continuity at these boundaries, the entropy continuation enabled flag (sps_entropy_continution_enabled_flag) or the entropy continuation flag (gsh_entropy_continution_flag) can signal whether entropy is being used continuously. Continuity with previous slices can be maintained by providing a reference slice ID (ref_slice_id).

[0232] When using layer and subgroup structures, it is necessary to deliver a scalable structure that allows the transmitter / receiver (encoder / decoder) to select scalable layers. In the case of an octree structure, scalable transmission can be supported by all octree layers, or it can be restricted to layers at or below a specific depth. When only some of the octree layers are included, it can be determined whether slicing at the bitstream stage is necessary / unnecessary by signaling the scalable layer to which the slice belongs. For example, starting from the root node, a single scalable layer can be configured without supporting scalable transmission up to a specific depth, and the octree layers below can be configured to have a one-to-one match with the scalable layer. Typically, scalability can be supported for the portions corresponding to leaf nodes. Figure 15 As shown, when a slice includes multiple octree layers, a scalable layer can be configured for those layers.

[0233] Implementations can differentiate between scalable transmission and scalable decoding for their respective purposes. For the purpose of selecting information up to a specific layer at the transmitting / receiving side without the aid of a decoder, implementations can use scalable transmission. For scalable decoding, a specific layer can be selected during encoding. Scalable transmission can support the selection of desired information in a compressed state (at the bitstream stage) without the aid of a decoder, enabling determination at the receiver or transmitter. In cases such as scalable representation, scalable decoding can be used by supporting the encoding / decoding of only the desired portion during the encoding / decoding process.

[0234] The layer configuration used for scalable transport and the layer configuration used for scalable decoding can differ. For example, from the perspective of scalable transport, the lowest three octree layers, including the leaf node layer, can be configured as a single layer. However, from the perspective of scalable decoding, scalable decoding is possible for each of the leaf node layer, leaf node layer-1, and leaf node layer-2 when all layer information is included.

[0235] The encoding / decoding method according to the embodiments may include encoding and decoding point cloud data based on both slice-level scalability and octree-level scalability.

[0236] A layer group can represent a unit used to define a set of one or more octree layers. A layer group can also represent a bundle of octree layers for a slice-by-slice configuration. Spatial scalability based on the actual octree layer (or scalable attribute layer) can be provided for each octree layer. However, when configured at the slice level before bitstream parsing, selection can be performed at the layer group level. Figure 15 As illustrated, when slice-level scalability is used in applications such as scalable transport, the provided scalable steps can include three steps: Group 1, Group 2, and Group 3. Conversely, the scalable steps provided at the decoding stage, based on an octree structure, can include eight steps ranging from the root to the leaves.

[0237] According to an embodiment, when groups 1 to 3 are all configured as individual slices, the receiver or transmitter may: 1) select only group 1, 2) select both group 1 and group 2, or 3) select group 1, group 2 and group 3.

[0238] 1) When only group 1 is selected, the maximum octree level is 4, and a scalable octree level from 0 to 4 can be selected during the decoding process. The receiver (decoder) can treat the node size obtainable through the maximum octree depth as a leaf node. The node size can then be sent to the decoder as signaling information (parameter information).

[0239] 2) When Group 1 and Group 2 are selected, a layer of 5 can be added, and the maximum octree level can be 5. During the decoding process, a scalable layer can be selected from octree layers 0 to 5. In this case, the receiver can treat the node size obtainable through the maximum octree depth as a leaf node. The node size can then be sent to the decoder as signaling information (parameter information).

[0240] 3) When Group 1, Group 2, and Group 3 are selected, layers 6 and 7 can be added, and the maximum octree level can be 7. During the decoding process, a scalable layer can be selected from octree layers 0 to 7. In this case, the receiver can treat the node size obtainable through the maximum octree depth as a leaf node. The node size can then be sent to the decoder as signaling information (parameter information).

[0241] The decoding layers for geometry and attributes can be selected symmetrically or asymmetrically.

[0242] Figure 16 The illustration shows some geometric data and some properties according to an embodiment.

[0243] The encoding method according to an embodiment may include encoding attributes based on partial geometric data. The decoding method according to an embodiment may include decoding attributes based on partial geometric data.

[0244] When using scalable transport, partial decoding can be performed on geometry and attributes, and the encoded octree depth can differ from the decoded octree depth. In the case of scalable upscaling generated from LoD based on the full octree depth, mismatches may occur due to the difference between the encoded and decoded octree depths.

[0245] When performing scalable boosted LoD generation for an octree depth of 4, the selection operation can be alternated: selecting the first child node at the leaf level, the last child node at leaf level +1, and the first child node at leaf level +2. This method of selecting nodes closer to the center of the parent node can improve encoding efficiency. However, because it assumes the full octree depth, it may cause the following problems in partial geometry data scenarios. Figure 16 The illustration depicts a scenario where scalable boosted LoD generation is performed after encoding has been performed at octree depth 4, but partial decoding is being performed up to octree depth 3. Scalable boosted LoD generation alternates between selecting the first child node based on octree level and selecting the last child node, starting from the leaf node level and proceeding to the root node level. Considering the decoded geometric octree depth, the receiver (decoder) selects the first child node at octree level 3 and the last child node at octree level 2 during scalable boosted LoD generation. However, this operation results in the selection of attributes different from those selected during scalable boosted LoD generation based on octree depth 4 at the time of encoding. Therefore, a portion of the point cloud that differs from the encoder's intent is decoded.

[0246] like Figure 16As shown, at all four full octree depths (octree layers or LoD levels), the encoder can sample the attribute of the first child node among the child nodes of a parent node at depth 4 as a representative attribute of the parent node, and sample the attribute of the last child node among the child nodes of a parent node at depth 3 as a representative attribute of the parent node. The decoder can decode the point cloud data associated with octree depths 1 to 3 for partial decoding. Assuming depth 3 is the last layer, the decoder can sample the attribute of the first child node among the child nodes of a parent node at depth 2 as a representative attribute of the parent node. Because the encoder and decoder perform sampling in different order for depth (layer) 3, errors occur in the reconstructed point cloud.

[0247] Figure 17 The illustration shows some geometric data and some properties according to an embodiment.

[0248] As a method to address the mismatch in sampling order between the encoder and decoder during scalable boost LoD generation based on partial geometry data, the encoder can signal the sampling positions to the decoder. Sampling positions can be signaled individually for each level. By additionally sending information about the full octree depth, the correct sampling positions can be used within the partial geometry data environment. Therefore, partial geometry data and attributes can be supported via signaling without modifying the existing encoder / decoder.

[0249] The decoder can partially decode the geometry associated with octree depths (layers or LoD levels) 1 through 3, and can also partially decode the attributes associated with octree depths (layers or LoD levels) 1 through 3. When the encoder encodes attributes for depths 1 through 4, it can compress the point cloud data by sampling the attribute of the first child node among the child nodes of a parent node at depth 4 as a representative attribute of the parent node. When reconstructing the compressed point cloud data, the decoder can partially decode the point cloud data for depths (levels) 1 through 3. When decoding begins at depth 3, the decoder can sample the attribute of the last child node among the child nodes of a parent node at depth 3, rather than the attribute of the first child node, based on the sampling order sent by the encoder as a signal, as a representative attribute of the parent node. The encoder can send the sampling positions for depths 2 through 1 as signaling information (parameter information) to the decoder.

[0250] Figure 18 The illustration shows some geometric data and some properties according to an embodiment.

[0251] The encoding method according to the embodiment may include signaling the sampling position for octree depth 1. Partial geometry data decoding is a method of discarding consecutive octree levels starting from the leaf node level. Therefore, by delivering the sampling information about octree depth 1 that is always decoded, subsequent sampling positions can be applied alternately according to the octree depth. For example, when the sampling position for octree depth 1 is the last child node, the first and last child nodes for subsequent octree depths can be inferred. Therefore, partial geometry data and attributes can be supported via signaling without modifying the existing encoder / decoder. When the sampling position for depth 1 is the last node, the encoder can send information indicating that the sampling position for depth 1 is the last node to the decoder. For depths 2 to 3, the decoder can sequentially sample the first node at depth 2 and the last node at depth 3 as representative attributes of the parent node.

[0252] Figure 19 The illustration shows some geometric data and some properties according to an embodiment.

[0253] The encoding method according to the embodiment may include a fixed sampling position for octree depth 1. Unlike conventional methods that fix sampling positions at the leaf node level, the sampling position can be fixed for octree depth 1. Therefore, even when partial geometry data is encoded, the sampling position used for scalable LoD generation can remain unchanged.

[0254] like Figure 19 As shown, at depth (layer or level) 1, the sampling position can be fixed to the first node among the child nodes belonging to a parent node. Following a predetermined fixed order, the decoder can sample the first node at depth 1. Subsequently, the decoder can infer the sampling position to sample the last node at depth 2 and the first node at depth 3.

[0255] Furthermore, the encoding method according to the embodiment can use a fixed sampling position for all depths. The encoding method according to the embodiment can also use a signal to transmit the use of a point at a specific location.

[0256] [Table 1]

[0257] [Table 2]

[0258] As mentioned above, the sampling positions can be fixed, such as by sampling the first child node at each depth or sampling the last node at each depth. The encoder can send signaling information indicating the use of fixed positions (sequences) to the decoder.

[0259] In applications such as layer group slicing or scalable coding, when skipping layer groups or layers, the sampling direction used by the encoder can be inferred to maintain coding efficiency. Missing layer information can be inferred from layer information, bounding box information, or root node information associated with the actual decoded / encoded subdata, along with total layer information, bounding box information, or root node information associated with the encoder's input point cloud data. The encoder can then estimate these layers. The sampling direction used can be inferred as (octreeNodeSizeLog2 + rootNodeSizeLog2 - rootNodeSizeLog2_coded).

[0260] if (layer_group_enabled_flag) {

[0261] octreeNodeSizeLog2 = lodIndex;

[0262] direction = (octreeNodeSizeLog2 + rootNodeSizeLog2 -rootNodeSizeLog2_coded) & 1;

[0263] subsampleByOctree(pointCloud, packedVoxel, input, octreeNodeSizeLog2,retained, indexes, direction);

[0264] }

[0265] lodIndex: Indicates the index of the current LoD. The index value can increase or decrease sequentially from higher LoD to lower LoD.

[0266] octreeNodeSizeLog2: Log2 scaling value for the current LoD node size.

[0267] rootNodeSizeLog2: A log2 scaling value for the size of the node or bounding box of the entire input point cloud data covering the encoder.

[0268] rootNodeSizeLog2_coded: A log2 scaling value for the size of the node or bounding box that covers the actual encoded point cloud data.

[0269] Figure 20 The illustration shows a bitstream containing point cloud data and parameters according to an embodiment.

[0270] The encoding method according to the embodiments may include sending a bitstream containing information related to the separated slices. Parameter sets and / or SEI messages in the bitstream may include information related to the separated slices. Sequence Parameter Set (SPS), Geometry Parameter Set (GPS), Attribute Parameter Set (APS), and / or Geometry Slice Header (GSH) and / or Attribute Slice Header (ASH) in the bitstream may include information related to the separated slices. Depending on the application or system, information related to the separated slices may be defined at appropriate locations or individual locations within the bitstream, allowing for different application scopes and methods. Signals may have different meanings depending on their location within the bitstream. When a signal is defined in an SPS, it can be applied uniformly throughout the sequence. When a signal is defined in a GPS, this may indicate that the signal is applied to geometric reconstruction. When a signal is defined in an APS, this may indicate that the signal is applied to attribute reconstruction. When a signal is defined in a TPS, this may indicate that the signal is applied only to points within a tile. When a signal is carried in a slice cell, this may indicate that the signal is applied only to the corresponding slice. Additionally, depending on the application or system, signals can be defined at corresponding or individual locations within the bitstream, allowing for different application scopes and methods. Furthermore, when the syntax elements defined below apply to multiple point cloud data streams as well as the current point cloud data stream, information related to the separated slices can be delivered through higher-level parameter sets, etc.

[0271] Each abbreviation has the following meaning. Each abbreviation may be referred to by other terms within the range of equivalent meanings: SPS: Sequence Parameter Set, GPS: Geometric Parameter Set, APS: Attribute Parameter Set, TPS: Tile Parameter Set, Geom (Geometric): Geometric Bit Stream = Geometric Slice Header + Geometric Slice Data, Attr (Attribute): Attribute Bit Stream = Attribute Patch Header + Attribute Patch Data.

[0272] A slice comprises point cloud data and represents the unit of encoding and decoding. A slice can also be called a data unit. A slice represents the geometry and attributes of a point cloud frame used for partial or complete encoding. A slice can also be called a tile. A data unit represents a sequence of bytes carrying a syntax structure of a specific length.

[0273] The implementation defines information independent of the encoding technique, which will be described later. Furthermore, the following information can be defined in conjunction with the encoding method and can be defined in the tile parameter set to support locally different scalability. Additionally, when the syntax elements defined below apply not only to the current point cloud data stream but also to multiple point cloud data streams, they can be delivered through higher-level parameter sets, etc.

[0274] The encoding method according to an embodiment may include defining NAL (Network Abstraction Layer) units, defining relevant information for selecting layers such as layer_id in the bitstream, and delivering that information to the decoder. The decoding method according to an embodiment may include selecting and decoding the bitstream at the system level.

[0275] The parameters according to the embodiments (which may be referred to differently as metadata, signaling information, etc.) can be generated during the process of the transmitter according to the embodiments and can be delivered to the receiver according to the embodiments for use in the reconstruction process.

[0276] For example, the parameters according to the embodiment can be generated by the metadata processor (or metadata generator) of the transmitting device according to the embodiment, and can be obtained by the metadata parser of the receiving device according to the embodiment.

[0277] Figure 21 The illustration shows scalable boosting of LoD generation information according to an embodiment.

[0278] Figure 22 The diagram illustrates the set of attribute parameters according to an embodiment.

[0279] Figure 21 and Figure 22 The illustration is included Figure 20 The syntax of parameter information in the bitstream. Figure 21 and Figure 22 The information in the code can be generated by an encoding device and decoded by a decoding device.

[0280] `top_down_LoD_generation_flag`: When equal to 1, it indicates that when sampling attributes from child nodes used for scalable boosting LoD generation, the sampling position is determined alternately in a top-down direction based on the initial sampling depth (e.g., octree depth 1). When equal to 0, this flag can indicate that the sampling position is determined alternately in a bottom-up direction based on the leaf node level. The reference used to determine the sampling position can be changed from bottom to top. Scalable boosting LoD generation can be performed in a bottom-up manner.

[0281] `first_depth_sampling_direction_present_flag`: When equal to 1, it indicates that the sampling position is explicitly delivered when sampling attributes from child nodes with respect to a reference octree depth (e.g., octree depth 1). When equal to 0, it indicates that the sampling position is implicitly delivered when sampling attributes from child nodes. For example, the sampling position could be the first child node or the last child node.

[0282] `first_depth_sampling_direction_forward_flag`: When equal to 1, it indicates that sampling from child nodes should be performed in the forward direction. That is, it indicates that the first child node should be sampled at the initial sampling depth. When equal to 0, it indicates that sampling from child nodes should be performed in the opposite direction. That is, it indicates that the last child node should be sampled at the initial sampling depth.

[0283] `num_full_octree_depth`: This indicates the octree depth considered during encoding. When `top_down_LoD_generation_flag` equals 0, because the sampling direction used for scalable boost LoD generation is determined to be bottom-up, the decoder can use `num_full_octree_depth`, which indicates the octree depth considered during encoding instead of the depth of the decoded geometry, as a reference for determining the sampling direction at each octree depth.

[0284] `child_node_sampling_forward_direction_flag`: When equal to 1, it indicates that the sampling position is explicitly delivered when sampling attributes from child nodes for each octree depth. When equal to 0, it indicates that the sampling position is implicitly delivered when sampling attributes from child nodes. For example, the sampling position can be the first child node or the last child node.

[0285] `child_node_sampling_position`: This can be directly signaled to indicate the sampling position at each octree level. When equal to 0, it indicates the first node. When equal to 1, it indicates the last node. When equal to one of the values ​​from 2 to 7, it indicates the first through sixth intermediate nodes (or the nearest node). For example, when equal to one of the values ​​from 2 to 7, it can indicate a node between the first and last nodes.

[0286] `lifting_scalability_enabled_flag`: When equal to 1, it indicates that the attribute decoding process allows the pruned octree decoding result for the input geometry points. `lifting_scalability_enabled_flag` equal to 0 indicates that the attribute decoding process requires the complete octree decoding result. When this flag is not present, its value is inferred to be 0. When `trisoup_enabled_flag` equals 0, `lifting_scalability_enabled_flag` is set to 0. When `geom_tree_coded_axis_list_present_flag` equals 1, `lifting_scalability_enabled_flag` is set to 0.

[0287] When geom_scaling_enabled_flag equals 1, lifting_scalability_enabled_flag is set to 0 to ensure partial decoding of the geometric data of the point cloud.

[0288] The increment of `lifting_max_nn_range_minus1` indicates the maximum range of nearest neighbors used to limit the distance to points registered as neighbors. The value of `lifting_max_nn_range` is the number of octree nodes around the point.

[0289] fixed_sampling_direction_forward_flag: When equal to 1, it indicates that a sampling method that selects points in a fixed order is used for the points included in the sampling unit.

[0290] `sample_direction_type`: This indicates the method or order in which points were used for sampling. When equal to 0, it indicates the first point among those sorted using Morton code. When equal to 1, it indicates the last point among those sorted using Morton code. When equal to 2, it indicates the second point was used. When equal to 3, it indicates the third point was used.

[0291] inferred_sampling_direction_enabled_flag: When equal to 1, it indicates that the method used to infer the sampling direction is used.

[0292] rootNodeSizeLog2: This can indicate the log2 scaling value of the size of a node or bounding box that covers the entire input point cloud data of the encoder.

[0293] rootNodeSizeLog2_coded: A log2 scaling value that overwrites the size of the node or bounding box in the actual encoded point cloud data. The encoder can signal the encoded root node size. When this value is not directly signaled, the root node size can be inferred internally within the encoder or decoder based on the decoded geometric point cloud data.

[0294] The encoder and / or decoder can infer the number of missing layers from rootNodeSizeLog2-rootNodeSizeLog2_coded, and if necessary, can directly use the number of missing layers as parameter information by signaling.

[0295] Figure 23 The figure shows an encoder (transmitting device) according to an embodiment.

[0296] Figure 23 Corresponding to Figure 1 The transmitting device 10000, the point cloud video encoder 10002, and the transmitter 10003, Figure 2 Get 20000 - Encode 20001 - Send 20002 Figures 11 to 13 encoder, Figures 14 to 19 Partial encoding, Figures 20 to 22 and Figures 29 to 30 Bitstream and parameter generation, Figures 23 to 28 encoding, Figure 31 Encoding methods, etc.

[0297] The components at the transmitting and receiving sides of this disclosure will be described below. Each component may correspond to a processor, software, or hardware. Additionally, the following components may be combined with the PCC transmitting / receiving side structure and / or signaling information. Each of the encoder (encoding device) and decoder (decoding device) may include a memory and a processor.

[0298] 1) Encoder (coding device): The encoder acquires point cloud data as input. The encoder can process positional information (or geometric data) (e.g., XYZ coordinates, ...) separately. -θ coordinates, etc., and attribute information (or attribute data) (e.g., color, reflectivity, intensity, grayscale, opacity, medium, material, gloss, etc.) are encoded. Compressed data can be divided into units for transmission. Sub-bitstream generators can package information by dividing it into appropriate units to select the necessary information at the bitstream level based on hierarchical structure information.

[0299] Figure 24 The diagram illustrates a decoder (receiving device) according to an embodiment.

[0300] Figure 24 Corresponding toFigure 1 The receiving device 10004, receiver 10005, and point cloud video decoder 10006. Figure 2 Sending 20002 - Decoding 20003 - Rendering 20004 Figure 7 decoder Figure 9 The receiving device Figure 10 The device Figures 11 to 13 decoder Figures 14 to 19 Partial decoding, Figures 20 to 22 and Figures 29 to 30 Bitstream and parameter parsing, Figures 23 to 28 Decoding Figure 32 Decoding methods, etc.

[0301] The decoder receives a bitstream as input from the encoder. The decoder can process the bitstream for location information and the bitstream for attribute information separately. In this operation, a sub-bitstream classifier can deliver the bitstream to the appropriate decoder based on information in the bitstream header. Alternatively, the layer required by the receiver can be selected during this process. For the classified bitstream, the geometry decoder and attribute decoder can reconstruct the geometric and attribute data respectively based on the characteristics of the data, and the renderer can then convert them into a format for the final output.

[0302] Figure 25 The illustration shows a scalable upgrade based on an embodiment.

[0303] The decoding apparatus according to the embodiment can decode point cloud data based on scalable enhancement, such as... Figure 25 As shown in the diagram. When slices are configured in layers for scalable transport, they can be used as follows: Figure 25The diagram illustrates scalable lifting based on metadata. When using scalable lifting, signaling information such as scalable_lifting_lod_generation_info() delivered via SEI messages or APS according to the embodiment can be used. The top-level sampling direction can be explicitly signaled via first_depth_sampling_direction_forward_flag (when first_depth_sampling_direction_present_flag=1) when the top-down LoD generation flag is equal to 1, or the top-level sampling direction defined in the standard documentation can be used (when first_depth_sampling_direction_present_flag=0). When top-down LoD generation is not used, the sampling position can be estimated in the bottom-up direction using the octree depth information (num_full_octree_depth) used during encoding, or the position can be specified based on the sampling position for each octree depth provided by child_node_sampling_forward_direction_flag or child_node_sampling_position.

[0304] In the decoding method according to the embodiment, when scalable boosting is enabled (true) and top-down LoD generation is false, the sampling position is estimated in the bottom-up direction. When top-down LoD generation is enabled (true) and sampling direction existence is false, the sampling position is estimated in the top-down direction. When sampling direction existence is enabled (true), the sampling position is estimated in the top-down direction.

[0305] Figure 26 The illustration shows LoD sampling of subgroups considering missing layers according to an embodiment.

[0306] The method according to the embodiment can change the LoD sampling direction for each layer in the attribute layer group slice. In each layer, child nodes can be arranged in Morton code order, and the attribute of the first or last node can be selected (sampled) as the attribute of the parent node according to the octree layer.

[0307] When the total depth of the octree is delivered to the decoder, the decoder can sample the points (nodes) in the octree in the same LoD sampling direction as the encoder. However, when decoding a subset of the encoded bitstream, the decoder's sampling direction may differ from the encoder's sampling direction because the sampling direction at the lower levels is fixed to the forward direction, and there is a possibility that the sampling direction (first or last node) may vary depending on the coding layer.

[0308] To address this issue, the method according to the embodiment can compensate for the number of missing layers in the subgroup LoD generation as follows: The number of missing layers = the maximum depth of the encoded LoD - the maximum depth of the current subgroup LoD.

[0309] The method according to the embodiment can calculate the number of missing layers based on the difference between the maximum depth of the LoD encoded by the encoder and the maximum depth of the LoD of the current subgroup.

[0310] In addition, the number of missing layers can be derived from two parameters.

[0311] rootNodeSizeLog2 is a parameter that indicates the maximum depth of the encoded LoD.

[0312] rootNodeSizeLog2_coded is the exported value that indicates the maximum depth of the current subgroup LoD.

[0313] These parameters can be used to compensate for octreeNodeSizeLog2 in determining the LoD sampling direction.

[0314] if (layer_group_enabled_flag) {

[0315] int32_t octreeNodeSizeLog2 = lodIndex;

[0316] bool direction = (octreeNodeSizeLog2 + (rootNodeSizeLog2 -rootNodeSizeLog2_coded)) & 1;

[0317] subsampleByOctree(pointCloud, packedVoxel, input, octreeNodeSizeLog2,retained, indexes, direction);

[0318] }

[0319] The sampling direction can be calculated based on the octree node size (octreeNodeSizeLog2), the root node size (rootNodeSizeLog2), and the encoded root node size (rootNodeSizeLog2_coded).

[0320] octreeNodeSizeLog2 can be the value of the maximum depth of the LoD.

[0321] Based on this, the parameter octreeNodeSizeLog2 can be used as follows: if (layer_group_enabled_flag) { int32_t octreeNodeSizeLog2 = lodIndex; bool direction = octreeNodeSizeLog2 & 1; / octreeNodeSizeLog2 =maxDepth – endDepth[layerId] / maxDepth += sps.num_layers_minus1[layerGroupId]+1; subsampleByOctree(pointCloud, packedVoxel, input, octreeNodeSizeLog2,retained, indexes, direction); } The maximum number of layers (max.layer) can be calculated using methods such as maxDepth += sps.num_layers_minus1[layerGroupId]+1.

[0322] The method for deriving the LoD sampling direction is the same in both of the aforementioned subgroup LoD generation schemes. However, the method for calculating the maximum depth of the encoded LoD can differ. One is implemented by deriving the value using a defined parameter for the number of layers in each layer group, and the other sends this value as explicit parameter information via a signal.

[0323] Figure 27 The illustration shows an encoder and decoder use case related to FGS according to an embodiment.

[0324] Figure 28 The illustration shows an encoder and decoder use case related to FGS according to an embodiment.

[0325] In progressive decoding and partial decoding, the target application can be considered as a single decoder. Fine-grained slicing (FGS) according to the embodiments can be used to support multiple decoders with different capabilities due to memory, clock speed, display device, etc.

[0326] For example, when decoders A, B, and C have different capabilities in terms of resolution, one way to support the decoders is to generate different sets of bitstreams based on the requirements of the target device. Figure 27As shown in (b), an effective method is to generate a bitstream with an FGS capable of supporting three different decoders by selecting fine-grained slices.

[0327] For example, the encoder can select sub-layer groups of a bitstream based on requests from the decoder. Without FGS, different bitstreams would be generated by different encoders and selectively stored or transmitted. Figure 28 As shown in (b), a single bitstream with FGS can support multiple decoders without additional encoders and storage.

[0328] To support these use cases, granular slice encoders can generate different parameter sets for each target device. This is used to check decoder capabilities and set decoder parameters. In situations where the syntax definition is unclear, the encoder or transcoder can set different parameter values ​​based on the target device for a subset of the bitstream.

[0329] If necessary, multiple parameter sets can be generated for different target devices, such as SPS, GPS, and APS, and each parameter set can present criteria for selecting different levels of bitstream subsets that can be supported in the bitstream.

[0330] For example, num_layer_group_minus1 can indicate the maximum depth that the target device must support. Because num_layer_group_minus1 has different values ​​for decoders A, B, and C, the bitstream that can be processed by each decoder can be selectively used.

[0331] For example, it can be determined whether a subset is ultimately a displayable subset based on information about the number of points (in a subgroup or layer group).

[0332] Figure 27 (a) The illustration shows a case without FGS, where multiple encoders encode the entire point cloud data by downsampling the data, or encode the entire point cloud data to send bit streams of different sizes, and multiple decoders can decode point cloud data from low resolution to high resolution separately.

[0333] Figure 27 (b) The illustration shows a single encoder with FGS and multiple decoders. When a single encoder encodes point cloud data into multiple FGS for transmission via layer group slicing, multiple decoders can decode point cloud data from low resolution to high resolution based on the FGS.

[0334] Figure 28(a) The illustration shows a scenario without an FGS (Frame Gradient Selection System) and multiple encoders and decoders, where multiple encoders can encode a portion of the point cloud data by downsampling the entire point cloud data, or encode the entire point cloud data. A slice selector can select low-resolution or high-resolution slices, transcode the selected slices, and send them to multiple decoders. Multiple decoders can decode the point cloud data in the received low-resolution or high-resolution slices.

[0335] Figure 28 (b) The illustration shows a single encoder and multiple decoders with an FGS. A single encoder can encode the entire point cloud data based on layer group slices, transcode the low-resolution or high-resolution FGS through a slice selector and send it to the decoders, and multiple decoders can decode the point cloud data in the low-resolution or high-resolution FGS.

[0336] Considering Figures 27 to 28 For use cases, two possible examples of the maximum depth of the encoded LoD are described below.

[0337] Figure 29 The diagram illustrates the sequence parameter set according to an embodiment.

[0338] The encoding method according to the embodiments may include generating Figure 29 The sequence parameter set is included in the bitstream. The decoding method according to the embodiment may include a set of parameters based on... Figure 29 The information in the sequence parameter set is used to decode the point cloud data.

[0339] like Figure 29 As shown, the method according to the embodiment can generate and send FGS-related parameter information through explicit signaling.

[0340] As an explicit method, the encoder can deliver the root_subgroup_bbox_size, which indicates the maximum depth of the LoD in the SPS.

[0341] root_node_size_log2 indicates the log2 value of the cube root node size of the encoded octree.

[0342] root_node_size_log2 can indicate the number of encoded tree layers from the root to the leaf layer.

[0343] In fine-grained slicing, the number of missing layers in a subgroup can be calculated based on the difference between root_node_size_log2 and the maximum depth of the current subgroup. The number of missing layers in a subgroup is used to derive the sampling direction in subgroup LoD generation. The number of missing layers in a subgroup is also used to compensate for the approximate positions of nodes in intermediate layers.

[0344] Figure 30 The diagram illustrates the sequence parameter set according to an embodiment.

[0345] The method according to the embodiments may further include, for example, adding descriptive information to the bitstream in the sequence parameter set.

[0346] For example, constraints can be added to signaling information related to the maximum depth of the encoded LoD. Considering that EESW uses num_layers_minus1 to derive the maximum depth of the encoded LoD, the following semantics can be added.

[0347] `num_layer_groups_minus1`: This value, incremented by 1, indicates the number of layer groups representing consecutive tree layer groups as part of the geometric coding tree structure. `num_layer_groups_minus1` ranges from 0 to the number of coding tree layers. `num_layer_groups_minus1` is the value representing the number of layer groups used in the encoder.

[0348] num_layers_minus1: This value plus 1 indicates the number of encoded layers included in the i-th layer group. The maximum number of encoded layers in the current frame is calculated by accumulating num_layers_minus1[i]+1 for all i in the range from 0 to num_layer_groups_minus1.

[0349] The maximum number of layers derived from num_layers_minus1 is the number of tree layers encoded from the root to the leaf layer.

[0350] In fine-grained slicing, the number of missing layers in a subgroup can be the difference between the maximum number of layers and the maximum depth of the current subgroup. The number of missing layers in a subgroup is used to derive the sampling direction in subgroup LoD generation. The number of missing layers in a subgroup is also used to compensate for the approximate positions of nodes in intermediate layers.

[0351] Regarding the number of layer groups (num_layer_groups_minus1), when the transcoder supports various receivers, num_layer_groups_minus1, which indicates the number of layer groups, can also be changed according to the number of receivers.

[0352] Figure 31 The diagram illustrates an encoding method according to an embodiment.

[0353] The encoding method according to the embodiment may include encoding point cloud data (S3100) and / or transmitting a bit stream containing point cloud data (S3110).

[0354] Let's refer to each other. Figure 23 and Figure 15 The encoding of point cloud data (S3100) may include encoding geometric data and attribute data of the point cloud data. Geometric data may be encoded based on a tree comprising layers from the root to the leaf layers, and attribute data may be encoded based on a tree comprising layers from the root to the leaf layers. The tree for geometric data may include a layer group containing at least one layer, which includes at least one subgroup, and the tree for attribute data may include a layer group containing at least one layer, which includes at least one subgroup. Encoding may further include generating scalable improved level of detail (LoD) information.

[0355] Let's refer to each other. Figure 17 Regarding the signaling of the sampling positions of nodes in the tree, the encoding of point cloud data may include generating information indicating the sampling positions of nodes within a specific depth of the tree used for attribute data, where the sampling positions may all be one of the first or last nodes.

[0356] Let's refer to each other. Figure 18 Regarding the initial position of the sampling of nodes in the tree, the encoding of the point cloud data may include generating information indicating the sampling position of nodes within the initial depth of the tree for attribute data, wherein the sampling position may be either the first node or the last node.

[0357] Let's refer to each other. Figure 19 Regarding the initial sampling position in a fixed tree, the encoding of point cloud data may include sampling nodes based on sampling positions within the initial depth of the tree used for attribute data, where the sampling position may be either the first node or the last node.

[0358] The apparatus according to an embodiment may further include a computer-readable storage medium storing a bitstream generated by the encoding method according to an embodiment.

[0359] The method according to an embodiment may include: acquiring a bitstream for point cloud data, wherein the bitstream is generated based on: encoding geometric data of the point cloud data and encoding attribute data of the point cloud data; and transmitting data including the bitstream.

[0360] Figure 32 The diagram illustrates a decoding method according to an embodiment.

[0361] The decoding method according to the embodiment may include receiving a bit stream containing point cloud data (S3200) and / or decoding the point cloud data (S3210).

[0362] Let's refer to each other. Figure 24Decoding point cloud data (S3210) may include decoding the geometric data of point cloud data and decoding the attribute data of point cloud data.

[0363] Let's take a look together Figure 21 and 22 Regarding `lifting_scalability_enabled_flag` and `scalable_lifting_lod_generation_info()`, the bitstream may contain flags for scalable lift, and this bitstream may further contain scalable lift level of detail (LoD) information based on the values ​​of the flags for scalable lift. The scalable lift LoD information may include at least one of the following: a generation flag for the LoD of attribute data for point cloud data, a sampling direction presence flag, a sampling direction forward flag, or full octree depth information.

[0364] Let's refer to each other. Figure 21 Regarding `top_down_LoD_generation_flag`, `first_depth_sampling_direction_present_flag`, `first_depth_sampling_direction_forward_flag`, and `num_full_octree_depth`, the first value of the generation flags for LoD of attribute data indicates that the sampling position of the nodes of the tree associated with the point cloud data is changed from a specific depth in the tree in a top-down direction, and the second value of the generation flags for LoD of attribute data indicates that the sampling position of the nodes of the tree associated with the point cloud data is changed from the leaf nodes in the tree in a bottom-up direction. The sampling direction presence flag indicates whether the sampling position at a specific depth in the tree associated with the point cloud data is explicitly signaled. The sampling direction forward flag indicates whether the first node at a specific depth in the point cloud data is sampled. The total octree depth information indicates the total depth of the octree associated with the geometry of the point cloud data.

[0365] Let's refer to each other. Figure 21Regarding `fixed_sampling_direction_forward_flag`, `sampling_direction_type`, `inferred_sampling_direction_enabled_flag`, `rootNodeSizeLog2`, and `rootNodeSizeLog2_coded`, scalable improved LoD information may include at least one of the following: a fixed sampling direction forward flag, a sampling direction type, an inferred sampling direction enabled flag, and the root node size of the input point cloud data or the root node size of the encoded point cloud data. The fixed sampling direction forward flag indicates whether the sampling location for nodes in the tree associated with the point cloud data is fixed. The sampling direction type indicates whether the sampling location for nodes in the tree associated with the point cloud data is the first or last node. The inferred sampling direction enabled flag indicates whether the sampling location for nodes in the tree associated with the point cloud data is inferred.

[0366] Let's refer to each other. Figure 29 Regarding root_node_size_log2, the bitstream may contain information related to the size of the root node of the encoded octree of the point cloud data, wherein the information related to the size of the root node of the encoded octree may indicate the number of layers from the root layer to the leaf layer of the encoded octree.

[0367] Let's refer to each other. Figure 30 Regarding num_layer_groups_minus1 and num_layers_minus1, the bitstream may contain at least one of information indicating the number of layer groups or information indicating the number of layers. The information indicating the number of layer groups may indicate a value related to the number of layer groups comprising several consecutive layers of a tree associated with the geometry of the point cloud data. The information indicating the number of layers may indicate a value related to the number of layers included in the layer group.

[0368] This decoding method is performed by the decoding device. (See reference) Figure 1 The decoding apparatus may include: a memory; and at least one processor connected to the memory. The at least one processor may be configured to: receive a bitstream containing point cloud data; and decode the point cloud data.

[0369] The methods and apparatus according to the embodiments can provide the following technical effects.

[0370] The LoD generation used in scalable upscaling assumes full octree depth geometry and can achieve compression efficiency by alternating sampling positions between the first and last child nodes according to the octree depth, thus using attributes located near the center of the node. When scalable transport is supported, partial geometry data is supported. In this case, mismatches in sampling positions used for scalable upscaling LoD generation (due to at least one missing layer) may occur between the encoder and decoder. This can lead to inaccurate representations of parts of the point cloud. According to the method of this embodiment, mismatches between geometry and attributes in partial decoding can be resolved by changing the reference sampling position or explicitly signaling the reference sampling position.

[0371] The method / apparatus for sending and receiving point cloud data according to the embodiments can scalably encode and / or decode point cloud data while taking into account partial geometric data, and can effectively compress and reconstruct point cloud data based on operation and / or related signaling information according to the embodiments.

[0372] The various elements of the embodiments can be implemented by hardware, software, firmware, or a combination thereof. The various elements of the embodiments can be implemented by a single chip, such as a single hardware circuit. According to the embodiments, the elements can be selectively executed by separate chips, respectively. According to the embodiments, at least one of the elements of the embodiments can be executed in one or more processors including instructions for performing the operations according to the embodiments.

[0373] The operations according to the above embodiments can be performed by the transmitting and / or receiving devices according to the embodiments. The transmitting / receiving devices may include a transmitter / receiver configured to transmit and receive media data; a memory configured to store instructions (program code, algorithms, flowcharts, and / or data) for the processes according to the embodiments; and a processor configured to control the operation of the transmitting / receiving devices.

[0374] The processor may be referred to as a controller, etc., and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above embodiments can be performed by the processor. Alternatively, the processor may be implemented as an encoder / decoder for the operations of the above embodiments.

[0375] Embodiments have been described with respect to methods and / or apparatus, and the descriptions of methods and apparatus may be applied complementaryly to each other.

[0376] Although the accompanying drawings have been described separately for simplicity, new embodiments can be designed by combining the embodiments illustrated in the respective drawings. Recording media containing programs for performing the above embodiments, designed to be computer-readable by those skilled in the art, also fall within the scope of the appended claims and their equivalents. The apparatus and methods according to the embodiments are not limited to the configurations and methods of the above embodiments. Various modifications can be made to the embodiments by selectively combining all or some of the embodiments. Although preferred embodiments have been described with reference to the accompanying drawings, those skilled in the art will appreciate that various modifications and variations can be made to the embodiments without departing from the spirit or scope of this disclosure as described in the appended claims. These modifications should not be understood solely from the technical ideas or viewpoints of the embodiments.

[0377] Various elements of the apparatus of the embodiments can be implemented by hardware, software, firmware, or a combination thereof. Various elements of the embodiments can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiments, the components of the embodiments can be implemented as separate chips. According to the embodiments, at least one or more of the components of the apparatus of the embodiments can include one or more processors capable of executing one or more programs. The one or more programs can perform any one or more of the operations / methods according to the embodiments or include instructions for performing them. Executable instructions for performing the methods / operations of the apparatus according to the embodiments can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transient CRM or other computer program product configured to be executed by one or more processors. Additionally, the memory according to the embodiments can be used to encompass not only volatile memory (e.g., RAM) but also the concepts of non-volatile memory, flash memory, and PROM. It can also be implemented in the form of a carrier wave, such as transmission over the Internet. Furthermore, the processor-readable recording medium can be distributed across computer systems connected via a network, such that processor-readable code can be stored and executed in a distributed manner.

[0378] In this disclosure, " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" can mean "A and / or B". Furthermore, "A, B" can mean "A and / or B". Additionally, "A / B / C" can mean "at least one of A, B, and / or C". Also, "A / B / C" can mean "at least one of A, B, and / or C". Furthermore, in this specification, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" can mean 1) only A, 2) only B, or 3) both A and B. In other words, the term "or" as used in this document should be interpreted as indicating "alternatively or alternatively".

[0379] Terms such as "first" and "second" can be used to describe various elements of the embodiments. However, the various components according to the embodiments should not be limited by the terms used above. These terms are used only to distinguish one element from another. For example, a first user input signal can be referred to as a second user input signal. Similarly, a second user input signal can be referred to as a first user input signal. The use of these terms should not be interpreted beyond the scope of the various embodiments. Both a first user input signal and a second user input signal are user input signals, but they do not mean the same user input signal unless the context clearly specifies otherwise.

[0380] The terminology used to describe embodiments is for the purpose of describing particular embodiments and is not intended to limit the embodiments. As used in the description of embodiments and claims, unless the context clearly specifies otherwise, the singular forms “a,” “an,” and “the” include a plural of indicators. The expression “and / or” is used to include all possible combinations of terms. Terms such as “comprising” or “having” are intended to indicate the presence of figures, quantities, steps, elements, and / or components and should be understood to not exclude the possibility of additional figures, quantities, steps, elements, and / or components. As used herein, conditional expressions such as “if” and “when” are not limited to optional cases and are intended to perform related operations or interpret related definitions based on specific conditions when specific conditions are met.

[0381] The operations described in this specification according to embodiments can be performed by a transmitting / receiving device including a memory and / or a processor according to embodiments. The memory may store a program for processing / controlling operations according to embodiments, and the processor may control the various operations described herein. The processor may be referred to as a controller, etc. In embodiments, the operations may be performed by firmware, software, and / or combinations thereof. Firmware, software, and / or combinations thereof may be stored in a processor or memory.

[0382] The operations according to the above embodiments can be performed by the transmitting and / or receiving devices according to the embodiments. The transmitting / receiving devices may include: a transmitter / receiver configured to transmit and receive media data; a memory configured to store instructions (program code, algorithms, flowcharts, and / or data) for the processes according to the embodiments; and a processor configured to control the operation of the transmitting / receiving devices.

[0383] The processor may be referred to as a controller, etc., and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above embodiments can be performed by the processor. Alternatively, the processor may be implemented as an encoder / decoder for the operations of the above embodiments.

[0384] Invention Model

[0385] As described above, the relevant details have been described in the best mode for implementing the embodiments.

[0386] Industrial applicability

[0387] As described above, the embodiments are applicable in whole or in part to point cloud data transmission / reception devices and systems.

[0388] Those skilled in the art can change or modify the embodiments in various ways within the scope of the embodiments.

[0389] Embodiments may include variations / modifications within the scope of the claims and their equivalents.

Claims

1. A decoding method, comprising: Receive a bitstream containing point cloud data; as well as The point cloud data is decoded.

2. The decoding method according to claim 1, wherein, The decoding of the point cloud data includes: Decode the geometric data of the point cloud data; and The attribute data of the point cloud data is decoded.

3. The decoding method according to claim 1, wherein, The bitstream contains flags for scalable boosting. The bitstream further includes scalable boost level of detail (LoD) information based on the value of the flag used for scalable boosting, and The scalable boosted LoD information includes at least one of the following: A flag used to generate the LoD of the attribute data for the point cloud data; The sampling direction is marked; Sampling direction forward indicator; or Information on the depth of the entire octree.

4. The decoding method according to claim 3, wherein, The first value of the generation flag used for the LoD of the attribute data indicates that the sampling position of the node of the tree associated with the point cloud data is changed from a specific depth of the tree in a top-down direction. Wherein, the second value of the generation flag for the LoD of the attribute data indicates that the sampling position of the node of the tree associated with the point cloud data is changed from the leaf node of the tree in the bottom-up direction. The sampling direction is indicated by a flag indicating whether the sampling position at a specific depth of the tree associated with the point cloud data is explicitly signaled. Wherein, the sampling direction forward flag indicates whether the first node at the specific depth associated with the point cloud data has been sampled, and The total octree depth information indicates the total depth of the octrees associated with the geometric data of the point cloud data.

5. The decoding method according to claim 3, wherein, The scalable boosted LoD information includes at least one of the following: A fixed sampling direction forward marker; Sampling direction type; Inferred sampling direction enable flag; Input the root node size of the point cloud data; or The size of the root node of the encoded point cloud data. The fixed sampling direction forward flag indicates whether the sampling position for nodes in the tree associated with the point cloud data is fixed. Wherein, the sampling direction type indicates whether the position used to sample the node in the tree associated with the point cloud data is the first node or the last node, and The inferred sampling direction enable flag indicates whether to infer the position used to sample the node in the tree associated with the point cloud data.

6. The decoding method according to claim 1, wherein, The bitstream contains information related to the size of the root node of the octree encoding the point cloud data. The information related to the size of the root node of the encoded octree indicates the number of layers from the root layer to the leaf layer of the encoded octree.

7. The decoding method according to claim 1, wherein, The bitstream contains at least one of information indicating the number of layer groups or information indicating the number of layers. The information indicating the number of layer groups indicates a value related to the number of layer groups, wherein the layer groups include some consecutive layers of a tree associated with the geometric data of the point cloud data, and The information indicating the number of layers indicates a value related to the number of layers included in the layer group.

8. A decoding device, comprising: Memory; as well as At least one processor is connected to the memory. Wherein, the at least one processor is configured to: Receive a bitstream containing point cloud data; and The point cloud data is decoded.

9. An encoding method, comprising: Encode the point cloud data; as well as Send a bit stream containing the point cloud data.

10. The encoding method according to claim 9, wherein, The encoding of the point cloud data includes: Encode the geometric data of the point cloud data; and The attribute data of the point cloud data is encoded. Specifically, the geometric data is encoded based on a tree consisting of layers from the root to the leaf layers. Specifically, the attribute data is encoded based on a tree consisting of layers from the root to the leaf layers. The tree used for the geometric data includes a layer group comprising at least one layer, and the layer group includes at least one subgroup. The tree for the attribute data includes a layer group comprising at least one layer, and the layer group includes at least one subgroup. The encoding further includes generating scalable improved level of detail (LoD) information.

11. The encoding method according to claim 10, wherein, The encoding of the point cloud data includes generating information indicating the sampling locations of nodes within a specific depth of the tree used for the attribute data. The sampling location is either the first node or the last node.

12. The encoding method according to claim 10, wherein, The encoding of the point cloud data includes generating information indicating the sampling positions of nodes within the initial depth of the tree used for the attribute data. The sampling location is either the first node or the last node.

13. The encoding method according to claim 10, wherein, The encoding of the point cloud data includes sampling nodes based on sampling positions within the initial depth of the tree used for the attribute data. The sampling location is either the first node or the last node.

14. A computer-readable storage medium storing a bit stream generated by the method of claim 9.

15. A method comprising: Obtain a bitstream for point cloud data, wherein the bitstream is generated based on the following: Encode the geometric data of the point cloud data; and Encode the attribute data of the point cloud data; and Send data including the bit stream.

Images