Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Abstract

A point cloud data transmission method according to an embodiment may comprise the steps of: encoding point cloud data; and transmitting the bit stream including the point cloud data. A point cloud data receiving method according to an embodiment may comprise the steps of: receiving a bitstream including point cloud data; and decoding the point cloud data.

Description

Technical Field

[0001] The embodiments relate to a method and apparatus for processing point cloud content. Background Technology

[0002] Point cloud content is content represented by point clouds, which are collections of points belonging to a coordinate system representing three-dimensional space. Point cloud content can represent media configured in three dimensions and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and autonomous driving services. However, tens of thousands to hundreds of thousands of point data points are required to represent point cloud content. Therefore, an efficient method for processing large amounts of point data is needed. Summary of the Invention

[0003] Technical issues

[0004] The embodiments provide an apparatus and method for efficiently processing point cloud data. The embodiments also provide a point cloud data processing method and apparatus for addressing waiting time and encoding / decoding complexity.

[0005] The technical scope of the implementation is not limited to the above-described technical objectives, but can be extended to other technical objectives that can be inferred by those skilled in the art based on the entire content disclosed herein.

[0006] Technical solution

[0007] In one aspect of this disclosure, a method for transmitting point cloud data may include: encoding the point cloud data; and transmitting a bit stream containing the point cloud data. In another aspect of this disclosure, a method for receiving point cloud data may include: receiving a bit stream containing the point cloud data; and decoding the point cloud data.

[0008] Beneficial effects

[0009] The apparatus and method according to the embodiments can efficiently process point cloud data.

[0010] The apparatus and method according to the embodiments can provide high-quality point cloud services.

[0011] The apparatus and method according to the embodiments can provide point cloud content for providing general services such as VR services and autonomous driving services. Attached Figure Description

[0012] The accompanying drawings are included to provide a further understanding of this disclosure and are incorporated into and constitute a part of this application. They illustrate embodiments of the disclosure and, together with this description, serve to illustrate the principles of the disclosure. For a better understanding of the various embodiments described below, reference should be made to the following description of embodiments in conjunction with the accompanying drawings. The same reference numerals will be used throughout the drawings to refer to the same or similar parts.

[0013] Figure 1 An exemplary point cloud content delivery system according to an embodiment is shown;

[0014] Figure 2 This is a block diagram illustrating the operation of providing point cloud content according to an embodiment;

[0015] Figure 3 An exemplary point cloud encoder according to an implementation method is illustrated;

[0016] Figure 4 An example of an octree and occupancy code according to an implementation is shown;

[0017] Figure 5 An example of point configuration in each LOD according to the implementation method is illustrated;

[0018] Figure 6 An example of point configuration in each LOD according to the implementation method is illustrated;

[0019] Figure 7 An example of a point cloud decoder according to an implementation method is shown;

[0020] Figure 8 An example of a transmitting device according to an embodiment is shown;

[0021] Figure 9 An example of a receiving device according to an embodiment is shown;

[0022] Figure 10 An exemplary structure that can be operated in conjunction with a point cloud data transmission / reception method / apparatus according to an embodiment is illustrated;

[0023] Figure 11 The process of encoding, transmitting, and decoding point cloud data according to the implementation method is illustrated;

[0024] Figure 12 The layer-based point cloud data configuration according to the embodiment and the geometry and attribute bitstream structure according to the embodiment are illustrated.

[0025] Figure 13 A bitstream configuration according to an implementation method is illustrated;

[0026] Figure 14 An example of a bitstream sorting method according to an implementation method is shown;

[0027] Figure 15 A method for selecting geometric data and attribute data according to an implementation method is illustrated;

[0028] Figure 16 An example is illustrated of a method for configuring slices containing point cloud data according to an implementation method;

[0029] Figure 17 An example of a geometric coding layer structure according to an implementation method is shown;

[0030] Figure 18 The layer and subgroup structure according to the implementation method is illustrated;

[0031] Figure 19 Examples of multi-resolution, multi-size ROIs according to the implementation method are shown;

[0032] Figure 20 A layer group slice according to an embodiment is illustrated;

[0033] Figure 21 The nearest neighbor search process according to the implementation method is illustrated;

[0034] Figure 22 The nearest neighbor search process according to the implementation method is illustrated;

[0035] Figure 23 The nearest neighbor search process according to the implementation method is illustrated;

[0036] Figure 24 An attribute encoding method according to an implementation method is illustrated;

[0037] Figure 25 An attribute decoding method according to an implementation method is illustrated;

[0038] Figure 26 An example is shown of a bitstream containing parameters and encoded point cloud data according to an implementation method;

[0039] Figure 27 A set of sequence parameters according to an implementation method is illustrated;

[0040] Figure 28 An example is shown of the attribute parameter set and attribute data cell header according to the implementation method;

[0041] Figure 29 An example of a dependency attribute data unit header according to an implementation method is shown;

[0042] Figure 30 The process of sending partial point cloud data according to the implementation method is illustrated;

[0043] Figure 31 A method for sending and receiving point cloud data according to an embodiment is illustrated;

[0044] Figure 32 A method for sending and receiving point cloud data according to an embodiment is illustrated;

[0045] Figure 33 A method for transmitting point cloud data according to an embodiment is illustrated; and

[0046] Figure 34 A method for receiving point cloud data according to an embodiment is illustrated. Detailed Implementation

[0047] Preferred embodiments of the present disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. The detailed description given below with reference to the drawings is intended to illustrate exemplary embodiments of the present disclosure, and not to show only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0048] Although most of the terms used in this disclosure are selected from commonly used terms in the art, some terms have been arbitrarily chosen by the applicant and their meanings are explained in detail in the following description as needed. Therefore, this disclosure should be understood based on the intended meaning of the terms rather than their simple names or meanings.

[0049] Figure 1 An exemplary point cloud content delivery system according to an implementation is shown.

[0050] Figure 1 The point cloud content providing system shown may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 are capable of transmitting and receiving point cloud data via wired or wireless communication.

[0051] The point cloud data transmission device 10000 according to an embodiment can acquire and process point cloud video (or point cloud content) and transmit it. According to an embodiment, the transmission device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device, and / or a server. According to an embodiment, the transmission device 10000 may include devices configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers.

[0052] The transmitting device 10000 according to the embodiment includes a point cloud video acquirer 10001, a point cloud video encoder 10002 and / or a transmitter (or communication module) 10003.

[0053] The point cloud video acquirer 10001 according to an embodiment acquires point cloud video through processing procedures such as capture, synthesis, or generation. Point cloud video is point cloud content represented by a point cloud, which is a set of points located in 3D space, and may be referred to as point cloud video data, point cloud data, etc. The point cloud video according to an embodiment may include one or more frames. A frame represents a still image / scene. Therefore, point cloud video may include point cloud images / frames / scenes, and may be referred to as point cloud images, frames, or scenes.

[0054] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 may encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to an embodiment may include geometry-based point cloud compression (G-PCC) coding and / or video-based point cloud compression (V-PCC) coding or next-generation coding. The point cloud compression coding according to an embodiment is not limited to the above embodiments. The point cloud video encoder 10002 may output a bitstream containing the encoded point cloud video data. The bitstream may contain not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0055] According to an embodiment, transmitter 10003 transmits a bitstream containing encoded point cloud video data. The bitstream, according to an embodiment, is encapsulated in a file or segment (e.g., a streaming segment) and transmitted via various networks such as broadcast networks and / or broadband networks. Although not shown in the figures, transmitting device 10000 may include an encapsulator (or encapsulation module) configured to perform encapsulation operations. According to an embodiment, the encapsulator may be included in transmitter 10003. According to an embodiment, the file or segment may be transmitted via a network to receiving device 10004 or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). Transmitter 10003 according to an embodiment is capable of wired / wireless communication with receiving device 10004 (or receiver 10005) via networks such as 4G, 5G, and 6G. Additionally, the transmitter may perform necessary data processing operations depending on the network system (e.g., a 4G, 5G, or 6G communication network system). Transmitting device 10000 can transmit encapsulated data on demand.

[0056] The receiving device 10004 according to an embodiment includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include devices, robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)).

[0057] According to an embodiment, receiver 10005 receives a bitstream containing point cloud video data or a file / segment encapsulated with a bitstream from a network or storage medium. Receiver 10005 may perform necessary data processing according to the network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). According to an embodiment, receiver 10005 may decapsulate the received file / segment and output a bitstream. According to an embodiment, receiver 10005 may include a decapsulator (or decapsulator module) configured to perform a decapsulation operation. The decapsulator may be implemented as a separate element (or component) from receiver 10005.

[0058] The point cloud video decoder 10006 decodes the bitstream containing point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to the method in which the point cloud video data is encoded (e.g., the reverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding (the reverse process of point cloud compression). Point cloud decompression encoding includes G-PCC encoding.

[0059] Renderer 10007 renders decoded point cloud video data. Renderer 10007 can output point cloud content by rendering not only the point cloud video data but also the audio data. According to one embodiment, renderer 10007 may include a display configured to display the point cloud content. According to another embodiment, the display may be implemented as a separate device or component rather than included in renderer 10007.

[0060] The arrows indicated by dashed lines in the diagram represent the transmission paths of the feedback information acquired by the receiving device 10004. The feedback information reflects the interactivity of the user consuming the point cloud content and includes information about the user (e.g., head orientation information, viewport information, etc.). Specifically, when the point cloud content is for a service requiring user interaction (e.g., autonomous driving services, etc.), the feedback information may be provided to the content sender (e.g., the sending device 10000) and / or the service provider. Depending on the implementation, the feedback information may be used in both the receiving device 10004 and the sending device 10000, or it may not be provided.

[0061] According to the embodiment, the head orientation information is information about the user's head position, orientation, angle, movement, etc. The receiving device 10004 according to the embodiment can calculate viewport information based on the head orientation information. The viewport information can be information about the area of ​​the point cloud video that the user is viewing. The viewpoint is the point through which the user views the point cloud video, and can refer to the center point of the viewport area. That is, the viewport is the area centered on the viewpoint, and the size and shape of the area can be determined by the field of view (FOV). Therefore, in addition to head orientation information, the receiving device 10004 can also extract viewport information based on the vertical or horizontal FOV supported by the device. Furthermore, the receiving device 10004 performs gaze analysis, etc., to examine the way the user consumes the point cloud, the area the user gazes at in the point cloud video, the gaze duration, etc. According to the embodiment, the receiving device 10004 can send feedback information including the gaze analysis results to the transmitting device 10000. The feedback information according to the embodiment can be acquired during rendering and / or display. The feedback information according to the embodiment can be acquired by one or more sensors included in the receiving device 10004. According to the implementation method, the feedback information can be obtained by the renderer 10007 or by a separate external component (or device, component, etc.). Figure 1 The dashed lines in the diagram represent the process of sending feedback information obtained by the renderer 10007. The point cloud content providing system can process (encode / decode) point cloud data based on the feedback information. Therefore, the point cloud video data decoder 10006 can perform decoding operations based on the feedback information. The receiving device 10004 can send feedback information to the sending device 10000. The sending device 10000 (or the point cloud video data encoder 10002) can perform encoding operations based on the feedback information. Therefore, the point cloud content providing system can efficiently process necessary data (e.g., point cloud data corresponding to the user's head position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide point cloud content to the user.

[0062] According to the implementation, the transmitting device 10000 may be referred to as an encoder, transmitting device, transmitter, etc., and the receiving device 10004 may be referred to as a decoder, receiving device, receiver, etc.

[0063] According to the implementation method Figure 1 Point cloud data processed in a point cloud content provision system (through a series of processes including acquisition, encoding, transmission, decoding, and rendering) can be referred to as point cloud content data or point cloud video data. Depending on the implementation, point cloud content data can be used as a concept encompassing metadata or signaling information related to point cloud data.

[0064] Figure 1 The components of the point cloud content provided by the system can be implemented by hardware, software, processors, and / or combinations thereof.

[0065] Figure 2 This is a block diagram illustrating the operation of providing point cloud content according to an embodiment.

[0066] Figure 2 The block diagram shows Figure 1 The operation of the point cloud content providing system described herein. As mentioned above, the point cloud content providing system can process point cloud data based on point cloud compression encoding (e.g., G-PCC).

[0067] A point cloud content providing system according to an embodiment (e.g., point cloud sending device 10000 or point cloud video acquirer 10001) can acquire point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system used to represent 3D space. The point cloud video according to an embodiment may include Ply (Polygon file format or Stanford Triangle format) files. When the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of the points. The position of each point may be represented by parameters (e.g., values ​​of the X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include the attributes of the points (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc., of each point). A point has one or more attributes. For example, a point may have a color attribute or two attributes: color and reflectivity. According to the implementation, geometry can be referred to as location, geometric information, geometric data, location information, location data, etc., and attributes can be referred to as attributes, attribute information, attribute data, etc. A point cloud content providing system (e.g., a point cloud sending device 10000 or a point cloud video acquirer 10001) can acquire point cloud data from information related to the point cloud video acquisition process (e.g., depth information, color information, etc.).

[0068] A point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) according to an embodiment can encode point cloud data (20001). The point cloud content providing system can encode point cloud data based on point cloud compression encoding. As described above, point cloud data can include geometric information and attribute information about points. Therefore, the point cloud content providing system can perform geometric encoding to encode geometry and output a geometric bitstream. The point cloud content providing system can perform attribute encoding to encode attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system can perform attribute encoding based on geometric encoding. The geometric bitstream and attribute bitstream according to an embodiment can be multiplexed and output as a single bitstream. The bitstream according to an embodiment may also contain signaling information related to geometric encoding and attribute encoding.

[0069] A point cloud content providing system according to an embodiment (e.g., transmitting device 10000 or transmitter 10003) can transmit encoded point cloud data (20002). For example... Figure 1 As shown, encoded point cloud data can be represented by geometric bitstreams and attribute bitstreams. Additionally, the encoded point cloud data can be transmitted as a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometric encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and transmit it as a file or fragment.

[0070] The point cloud content providing system (e.g., receiving device 10004 or receiver 10005) according to the embodiment can receive a bitstream containing encoded point cloud data. Additionally, the point cloud content providing system (e.g., receiving device 10004 or receiver 10005) can demultiplex the bitstream.

[0071] A point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode encoded point cloud data (e.g., geometric bitstream, attribute bitstream) transmitted in a bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode point cloud video data based on signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode the geometric bitstream to reconstruct the location (geometry) of the points. The point cloud content providing system can reconstruct the attributes of the points by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can reconstruct point cloud video based on location according to the reconstructed geometry and the decoded attributes.

[0072] A point cloud content providing system (e.g., receiving device 10004 or renderer 10007) according to an embodiment can render decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) can use various rendering methods to render the geometry and attributes decoded through the decoding process. Points in the point cloud content can be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to the user through a display (e.g., a VR / AR display, a general display, etc.).

[0073] The point cloud content providing system (e.g., receiving device 10004) according to the embodiment can acquire feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. The feedback information and operation / reference of the point cloud content providing system according to the embodiment... Figure 1 The feedback information and operation described are the same, so their detailed description is omitted.

[0074] Figure 3 An exemplary point cloud encoder according to an implementation is shown.

[0075] Figure 3 Show Figure 1 An example of a point cloud video encoder 10002. The point cloud encoder reconstructs and encodes point cloud data (e.g., point locations and / or attributes) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) based on network conditions or applications. When the total size of the point cloud content is large (e.g., 60Gbps of point cloud content for 30fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system can reconstruct the point cloud content based on a maximum target bitrate to provide the point cloud content according to network conditions, etc.

[0076] For reference Figure 1 and Figure 2 As described, the point cloud encoder can perform geometric encoding and attribute encoding. Geometric encoding is performed before attribute encoding.

[0077] The point cloud encoder according to the implementation includes a coordinate transformer (transform coordinates) 30000, a quantizer (quantize and remove points (voxarization)) 30001, an octree analyzer (analyze octrees) 30002, a surface approximation analyzer (analyze surface approximations) 30003, an arithmetic encoder (arithmetic encoding) 30004, a geometry reconstructor (reconstruct geometry) 30005, a color transformer (transform colors) 30006, an attribute transformer (transform attributes) 30007, a RAHT transformer (RAHT) 30008, an LOD generator (generate LODs) 30009, a lift transformer (lift) 30010, a coefficient quantizer (quantize coefficients) 30011, and / or an arithmetic encoder (arithmetic encoding) 30012.

[0078] The coordinate transformer 30000, quantizer 30001, octree analyzer 30002, surface approximation analyzer 30003, arithmetic encoder 30004, and geometric reconstructor 30005 are capable of performing geometric coding. Geometric coding according to the implementation may include octree geometric coding, prediction tree geometric coding, direct coding, triplet geometric coding, and entropy coding. Direct coding and triplet geometric coding are applied selectively or in combination. Geometric coding is not limited to the examples described above.

[0079] As shown in the figure, the coordinate transformer 30000 according to the embodiment receives the position and transforms it into coordinates. For example, the position can be transformed into position information in three-dimensional space (e.g., three-dimensional space represented by the XYZ coordinate system). The position information in three-dimensional space according to the embodiment can be referred to as geometric information.

[0080] The quantizer 30001 according to the embodiment quantizes geometry. For example, the quantizer 30001 may quantize points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). The quantizer 30001 performs a quantization operation: multiplying the difference between the minimum position value and the position value of each point by a preset quantization scaling value, and then finding the nearest integer value by rounding the value obtained by multiplication. Thus, one or more points may have the same quantized position (or position value). The quantizer 30001 according to the embodiment performs voxelization based on the quantized position to reconstruct the quantized points. As in the case of pixels (the smallest unit containing 2D image / video information), points in the point cloud content (or 3D point cloud video) according to the embodiment may be included in one or more voxels. As a combination of volume and pixel, the term voxel refers to the 3D cubic space generated when 3D space is divided into units (unit = 1.0) based on axes representing 3D space (e.g., X-axis, Y-axis, and Z-axis). The quantizer 30001 allows a group of points in 3D space to be matched with voxels. In one embodiment, a voxel may include only one point. In another embodiment, a voxel may include one or more points. To represent a voxel as a point, the location of the voxel's center can be set based on the locations of one or more points included in the voxel. In this case, attributes of all locations included in a voxel can be combined and assigned to the voxel.

[0081] The octree analyzer 30002 according to the implementation performs octree geometric encoding (or octree coding) to represent voxels in an octree structure. The octree structure represents points based on the matching of octree structures with voxels.

[0082] The surface approximation analyzer 30003 according to the embodiment can analyze and approximate an octree. The octree analysis and approximation according to the embodiment is a process of analyzing a region containing multiple points to efficiently provide an octree and voxelization.

[0083] The arithmetic encoder 30004 according to the embodiment performs entropy encoding on octrees and / or approximate octrees. For example, the encoding scheme includes arithmetic encoding. As a result of the encoding, a geometric bitstream is generated.

[0084] The attribute encoding is performed by a color transformer 30006, an attribute transformer 30007, a RAHT transformer 30008, a LOD generator 30009, a boosting transformer 30010, a coefficient quantizer 30011, and / or an arithmetic encoder 30012. As described above, a point may have one or more attributes. The attribute encoding according to the embodiment is also applied to the attributes possessed by a point. However, when an attribute (e.g., color) comprises one or more elements, attribute encoding is applied independently to each element. The attribute encoding according to the embodiment includes color transformation encoding, attribute transformation encoding, region adaptive hierarchical transformation (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) encoding, and interpolation-based hierarchical nearest neighbor prediction (boosting transformation) encoding with update / boosting steps. Depending on the point cloud content, the above-described RAHT encoding, prediction transformation encoding, and boosting transformation encoding may be used selectively, or a combination of one or more encoding schemes may be used. The attribute encoding according to the embodiment is not limited to the examples described above.

[0085] The color converter 30006 according to the embodiment performs color transformation encoding that transforms the color values ​​(or textures) included in the attributes. For example, the color converter 30006 can transform the format of color information (e.g., from RGB to YCbCr). Optionally, the operation of the color converter 30006 according to the embodiment can be applied based on the color values ​​included in the attributes.

[0086] The geometry reconstructor 30005, according to the implementation method, reconstructs (decompresses) octrees and / or approximate octrees. The geometry reconstructor 30005 reconstructs the octree / voxel based on the results of analyzing the point distribution. The reconstructed octree / voxel can be referred to as the reconstructed geometry (recovered geometry).

[0087] The attribute transformer 30007 according to the embodiment performs attribute transformation to transform attributes based on reconstructed geometry and / or positions without performing geometric encoding. As described above, since attributes depend on geometry, the attribute transformer 30007 can transform attributes based on reconstructed geometric information. For example, based on the position value of a point included in a voxel, the attribute transformer 30007 can transform the attributes of the point at that position. As described above, when the center position of a voxel is set based on the positions of one or more points included in the voxel, the attribute transformer 30007 transforms the attributes of one or more points. When performing triadic geometric encoding, the attribute transformer 30007 can transform attributes based on the triadic geometric encoding.

[0088] The attribute transformer 30007 performs attribute transformation by calculating the average of the attributes or attribute values ​​(e.g., color or reflectivity of each point) of neighboring points within a specific location / radius from the center (or position value) of each voxel. The attribute transformer 30007 can apply weights based on the distance from the center to each point when calculating the average. Therefore, each voxel has a position and a calculated attribute (or attribute value).

[0089] The attribute transformer 30007 can search for nearest neighbor points within a specific location / radius of the center of each voxel based on a KD-tree or Morton code. A KD-tree is a binary search tree and supports a data structure that allows points to be managed based on location, enabling fast nearest neighbor search (NNS). Morton codes are generated by representing the coordinates (e.g., (x, y, z)) of the 3D location of all points as bit values ​​and mixing the bits. For example, when the coordinates representing the point location are (5, 9, 1), the bit values ​​are (0101, 1001, 0001). Mixing the bit values ​​according to the bit index in the order of z, y, and x produces 010001000111. This value is represented as the decimal number 1095. That is, the Morton code value for the point with coordinates (5, 9, 1) is 1095. The attribute transformer 30007 can sort the points based on their Morton code values ​​and perform NNS using a depth-first traversal process. After an attribute transformation operation, use a KD tree or Morton code when an NNS is needed in another transformation process used for attribute encoding.

[0090] As shown in the figure, the transformation properties are input to the RAHT transformer 30008 and / or the LOD generator 30009.

[0091] According to the implementation, the RAHT transformer 30008 performs RAHT encoding for predicting attribute information based on the reconstructed geometric information. For example, the RAHT transformer 30008 can predict the attribute information of higher-level nodes in an octree based on the attribute information associated with lower-level nodes in the octree.

[0092] The LOD generator 30009 according to the embodiment generates a Level of Detail (LOD) to perform predictive transform coding. The LOD according to the embodiment represents the level of detail of the point cloud content. As the LOD value decreases, it indicates a deterioration in the detail of the point cloud content. As the LOD value increases, it indicates an enhancement in the detail of the point cloud content. Points can be classified according to LOD.

[0093] The lift transformer 30010 according to the embodiment performs lift transform coding to transform point cloud attributes based on weights. As described above, lift transform coding may optionally be applied.

[0094] According to the implementation method, the coefficient quantizer 30011 quantizes the attribute encoding based on the coefficient.

[0095] According to the implementation method, the arithmetic encoder 30012 encodes quantized attributes based on arithmetic coding.

[0096] Although not shown in the figure, Figure 3 The elements of the point cloud encoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above-described... Figure 3 At least one of the operations and / or functions of the elements of the point cloud encoder. Additionally, one or more processors may operate or perform operations for executing... Figure 3 The software program and / or instruction set for the operation and / or function of the elements of the point cloud encoder. One or more memories according to the embodiment may include high-speed random access memory, or include non-volatile memory (e.g., one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices).

[0097] Figure 4 An example of an octree and occupancy code according to an implementation is shown.

[0098] For reference Figures 1 to 3 As described, the point cloud content delivery system (point cloud video encoder 10002) or point cloud encoder (e.g., octree analyzer 30002) performs octree geometric encoding (or octree encoding) based on an octree structure to efficiently manage the regions and / or locations of voxels.

[0099] Figure 4 The upper part shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes of a coordinate system (e.g., the X, Y, and Z axes). This is achieved by using two poles (0,0,0) and (2... d ,2 d ,2 d An octree structure is created by recursively subdividing the bounding box aligned to the cubic axis. Here, 2d can be set as the value of the minimum bounding box that constitutes all points surrounding the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following formula. In the following formula, (x int n ,y int n ,z int n ) indicates the position (or position value) of the quantized point.

[0100] d=Ceil(Log2(Max(x_n^int,y_n^int,z_n^in,n=1,…,N)+1))

[0101] like Figure 4 As shown in the upper center, the entire 3D space can be divided into eight spaces according to partitions. Each partitioned space is represented by a cube with six faces. For example... Figure 4 As shown in the upper right, each of the eight spaces is further subdivided based on a coordinate system axis (e.g., the X, Y, and Z axes). Thus, each space is divided into eight smaller spaces. These smaller spaces are also represented by cubes with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0102] Figure 4 The lower part shows the octree occupancy code. The occupancy code generates the octree to indicate whether each of the eight partitions generated by dividing a space contains at least one node. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of a partitioned space, and each child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to a child node contains at least one node, the node is assigned a value of 1. When the space corresponding to a child node does not contain a node (the space is empty), the node is assigned a value of 0. Since... Figure 4 The occupancy code shown is 00100001, thus indicating that the spaces corresponding to the third and eighth child nodes among the eight child nodes each contain at least one point. As shown, each of the third and eighth child nodes has eight child nodes, and the child nodes are represented by an 8-bit occupancy code. The attached figure shows that the occupancy code for the third child node is 10000111, and the occupancy code for the eighth child node is 01001111. A point cloud encoder (e.g., an arithmetic encoder 30004) according to an embodiment can perform entropy encoding on the occupancy code. To increase compression efficiency, the point cloud encoder can perform intra-frame / inter-frame encoding on the occupancy code. A receiving device (e.g., receiving device 10004 or point cloud video decoder 10006) according to an embodiment reconstructs the octree based on the occupancy code.

[0103] A point cloud encoder according to an implementation method (e.g., Figure 4 A point cloud encoder or octree analyzer (30002) can perform voxelization and octree encoding to store point locations. However, points are not always uniformly distributed in 3D space, so there may be specific regions with fewer points. Therefore, performing voxelization over the entire 3D space is inefficient. For example, when a specific region contains very few points, voxelization is not necessary in that specific region.

[0104] Therefore, for the aforementioned specific region (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment can skip voxelization and perform direct encoding to directly encode the point positions included in the specific region. The coordinates of the directly encoded points according to the embodiment are called the Direct Encoding Mode (DCM). The point cloud encoder according to the embodiment can also perform triadic geometry encoding based on the surface model, which reconstructs the point positions in the specific region (or node) based on voxels. Triadic geometry encoding is a geometry encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct encoding and triadic geometry encoding according to the embodiment can be performed selectively. In addition, direct encoding and triadic geometry encoding according to the embodiment can be performed in combination with octree geometry encoding (or octree encoding).

[0105] To perform direct encoding, the option to apply direct encoding using direct mode should be enabled. The node to which direct encoding is applied must not be a leaf node, and there should be fewer than a threshold number of points within that node. Furthermore, the total number of points to which direct encoding is applied should not exceed a preset threshold. When the above conditions are met, the point cloud encoder (or arithmetic encoder 30004) according to the implementation method can perform entropy encoding on the point locations (or location values).

[0106] A point cloud encoder according to an embodiment (e.g., a surface approximation analyzer 30003) can determine a specific level of the octree (a level less than the depth d of the octree) and can start using a surface model from that level to perform triadic geometry encoding to reconstruct the point positions in the node region based on voxels (triadic mode). The point cloud encoder according to an embodiment can specify the level to which triadic geometry encoding is to be applied. For example, when the specific level is equal to the depth of the octree, the point cloud encoder does not operate in triadic mode. In other words, the point cloud encoder according to an embodiment can operate in triadic mode only when the specified level is less than the depth value of the octree. The 3D cubic region of a node at a specified level according to an embodiment is called a block. A block may include one or more voxels. A block or voxel may correspond to a cube. Geometry is represented as surfaces within each block. A surface according to an embodiment may intersect each edge of a block at most once.

[0107] A block has 12 edges, therefore a block contains at least 12 intersections. Each intersection is called a vertex. Vertices along an edge are detected when there is at least one occupied voxel adjacent to the edge in all blocks sharing the edge. An occupied voxel, according to the implementation, refers to a voxel containing a point. The vertex position detected along an edge is the average position of the edges of all voxels adjacent to the edge in all blocks sharing the edge.

[0108] Once a vertex is detected, the point cloud encoder according to the embodiment can perform entropy encoding on the edge's origin (x, y, z), the edge's direction vector (Δx, Δy, Δz), and the vertex position value (relative position value within the edge). When applying triad geometry encoding, the point cloud encoder according to the embodiment (e.g., geometry reconstructor 30005) can generate the restored geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization processes.

[0109] Vertices located at the edges of a block determine the surface passing through the block. The surface, according to the implementation, is a non-planar polygon. During triangle reconstruction, the surface represented by triangles is reconstructed based on the origin of the edges, the direction vectors of the edges, and the position values ​​of the vertices. The triangle reconstruction process is performed as follows: i) calculating the centroid value of each vertex, ii) subtracting the centroid value from each vertex value, and iii) estimating the sum of squares of the values ​​obtained through the subtraction.

[0110] ① ② ③

[0111] Estimate the minimum value of the sum and perform a projection process based on the axis with the minimum value. For example, when element x is minimum, each vertex is projected onto the x-axis relative to the center of the block, and the projection is onto the (y,z) plane. When the value obtained by the projection onto the (y,z) plane is (ai,bi), the value of θ is estimated by atan2(bi,ai), and the vertices are sorted based on the value of θ. The following shows the vertex combinations for creating triangles based on the number of vertices. Vertices are sorted from 1 to n. The following shows that for four vertices, two triangles can be constructed based on the vertex combinations. The first triangle can be composed of vertices 1, 2, and 3 from the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 from the sorted vertices.

[0112] Table 2-1. Triangles formed from vertices sorted by 1, ..., n

[0113] n triangles

[0114] 3(1,2,3)

[0115] 4(1,2,3),(3,4,1)

[0116] 5(1,2,3),(3,4,5),(5,1,3)

[0117] 6(1,2,3),(3,4,5),(5,6,1),(1,3,5)

[0118] 7(1,2,3),(3,4,5),(5,6,7),(7,1,3),(3,5,7)

[0119] 8(1,2,3),(3,4,5),(5,6,7),(7,8,1),(1,3,5),(5,7,1)

[0120] 9(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,1,3),(3,5,7),(7,9,3)

[0121] 10(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,1),(1,3,5),(5,7,9),(9,1,5)

[0122] 11(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,11),(11,1,3),(3,5,7),(7,9,11),(11,3,7)

[0123] 12(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,11),(11,12,1),(1,3,5),(5,7,9),(9,11,1),(1,5,9)

[0124] An upsampling process is performed to add points along the edges of the triangle at the center, and voxelization is then performed. The added points are generated based on the upsampling factor and the width of the block. The added points are called thinned vertices. A point cloud encoder according to an implementation can voxelize the thinned vertices. Additionally, the point cloud encoder can perform attribute encoding based on the voxelized positions (or position values).

[0125] Figure 5 Examples of point configurations in various Levels of Detail (LODs) according to the implementation method are shown.

[0126] For reference Figures 1 to 4 The described approach involves reconstructing (decompressing) the encoded geometry before performing attribute encoding. When direct encoding is applied, the geometry reconstruction operation may include altering the placement of directly encoded points (e.g., placing directly encoded points in front of the point cloud data). When triadic geometry encoding is applied, the geometry reconstruction process is performed through triangle reconstruction, upsampling, and voxelization. Since attributes depend on geometry, attribute encoding is performed based on the reconstructed geometry.

[0127] A point cloud encoder (e.g., LOD generator 30009) can classify (or reorganize) points according to LOD. The figure shows the point cloud content corresponding to LOD. The leftmost view in the figure represents the original point cloud content. The second view from the left in the figure represents the point distribution in the lowest LOD, and the rightmost view represents the point distribution in the highest LOD. That is, points are sparsely distributed in the lowest LOD and densely distributed in the highest LOD. In other words, as the LOD increases in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between points narrows.

[0128] Figure 6 An example of point configuration for each LOD according to an implementation method is shown.

[0129] For reference Figures 1 to 5 As described, a point cloud content providing system or point cloud encoder (e.g., point cloud video encoder 10002, Figure 3 A point cloud encoder or LOD generator (30009) can generate LODs. LODs are generated by reorganizing points into a set of refined levels based on a set of LOD distance values ​​(or a set of Euclidean distances). The LOD generation process is performed not only by the point cloud encoder but also by the point cloud decoder.

[0130] Figure 6 The upper part shows examples of points (P0 to P9) of point cloud content distributed in 3D space. Figure 6 In this context, the original order represents the order of points P0 to P9 before LOD generation. Figure 6 In this context, LOD-based order represents the order in which points are generated according to their LOD. Points are reorganized by LOD. Additionally, higher LODs include points belonging to lower LODs. For example... Figure 6 As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, the points of LOD1, P9, P8, and P7.

[0131] For reference Figure 3 As described, the point cloud encoder according to the implementation can selectively or in combination perform predictive transform coding, lifting transform coding, and RAHT transform coding.

[0132] The point cloud encoder according to the embodiment can generate predictors for points to perform predictive transformation encoding for setting the predictive attributes (or predictive attribute values) of each point. That is, N predictors can be generated for N points. The predictors according to the embodiment can calculate weights (=1 / distance) based on the LOD value of each point, index information of neighboring points existing within a set distance of each LOD, and the distance to the neighboring points.

[0133] According to the implementation, the predicted attribute (or attribute value) is set as the average of the values ​​obtained by multiplying the attributes (or attribute values) of neighboring points (e.g., color, reflectivity, etc.) set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 30011) according to the implementation can quantize and inverse quantize the residuals (which may be referred to as residual attributes, residual attribute values, or attribute prediction residuals, or attribute residuals) obtained by subtracting the predicted attribute (attribute value) from the attribute (attribute value) of each point. The quantization process is configured as shown in the table below.

[0134] Table. Pseudocode for Attribute Prediction Residual Quantization

[0135] int PCCQuantization(int value,int quantStep){

[0136] if(value>=0){

[0137] return floor(value / quantStep+1.0 / 3.0);

[0138] }else{

[0139] return-floor(-value / quantStep+1.0 / 3.0);

[0140] }

[0141] }

[0142] Table. Pseudocode for Inverse Quantization of Attribute Prediction Residuals

[0143] int PCCInverseQuantization(intvalue,int quantStep){

[0144] if(quantStep==0){

[0145] return value;

[0146] }else{

[0147] return value * quantStep;

[0148] }

[0149] }

[0150] When the predictors of each point have neighboring points, the point cloud encoder (e.g., arithmetic encoder 30012) according to the embodiment can perform entropy encoding on the residual values ​​of quantization and inverse quantization as described above. When the predictors of each point do not have neighboring points, the point cloud encoder (e.g., arithmetic encoder 30012) according to the embodiment can perform entropy encoding on the attributes of the corresponding point without performing the above operations.

[0151] The point cloud encoder (e.g., lift transformer 30010) according to the embodiment can generate predictors for each point, set the calculated LOD and register neighboring points in the predictors, and set weights based on the distance to the neighboring points to perform lift transform coding. The lift transform coding according to the embodiment is similar to the prediction transform coding described above, but the difference is that the weights are applied cumulatively to the attribute values. The process of applying weights cumulatively to the attribute values ​​according to the embodiment is configured as follows.

[0152] 1) Create an array quantized weights (QW) to store the weight values ​​of each point. The initial value of all elements of QW is 1.0. Multiply the QW value of the predictor index of the neighboring nodes registered in the predictor by the weight of the current point's predictor, and add the values ​​obtained by multiplication.

[0153] 2) Improve the prediction process: Subtract the value obtained by multiplying the attribute value of the point by the weight from the existing attribute value to calculate the predicted attribute value.

[0154] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0155] 4) The weights calculated by multiplying the weights computed for all predictors by the weights stored in the QW corresponding to the predictor index are summed with the updateweight array and used as the index of the neighbor node. The values ​​obtained by multiplying the attribute values ​​of the neighbor node indexes by the calculated weights are summed with the update array.

[0156] 5) Improve the update process: Divide the attribute values ​​of the update array of all predictors by the weight values ​​of the updateweight array of the predictor index, and add the existing attribute values ​​to the values ​​obtained by division.

[0157] 6) For all predictors, the predicted attribute is calculated by multiplying the attribute value updated through the boosting update process by the weight updated through the boosting prediction process (stored in QW). The predicted attribute value is quantized by a point cloud encoder (e.g., coefficient quantizer 30011) according to the implementation. In addition, the point cloud encoder (e.g., arithmetic encoder 30012) performs entropy encoding on the quantized attribute value.

[0158] A point cloud encoder according to an embodiment (e.g., RAHT transformer 30008) can perform RAHT transform coding, where attributes associated with lower-level nodes in an octree are used to predict attributes of higher-level nodes. RAHT transform coding is an example of intra-frame attribute coding performed by scanning backward through an octree. The point cloud encoder according to an embodiment scans the entire region starting from voxels and repeats a merging process of merging voxels into larger blocks at each step until the root node is reached. The merging process according to the embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on the node directly above an empty node.

[0159] The following equation represents the RAHT transformation matrix. In this equation, This represents the average attribute value of the voxels at level l. Based on and To calculate. and The weight is and

[0160]

[0161] Here, It is a low-pass value and is used in the next higher level of merging. This represents the high-pass coefficient. The high-pass coefficient at each step is quantized and subjected to entropy encoding (e.g., encoded by an arithmetic encoder 300012). Weights are calculated as follows: pass and Create the root node as follows.

[0162]

[0163] The value of gDC is also quantized and subjected to entropy decoding, just like the high-pass coefficient.

[0164] Figure 7 A point cloud decoder according to an embodiment is shown.

[0165] Figure 7 The point cloud decoder shown is an example of a point cloud decoder and can perform decoding operations. Figures 1 to 6 The reverse process of the encoding operation of the point cloud encoder is shown.

[0166] For reference Figure 1 and Figure 6 As described, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0167] The point cloud decoder according to the implementation includes an arithmetic decoder (arithmetic decoding) 7000, an octree synthesizer (synthesized octree) 7001, a surface approximation synthesizer (synthesized surface approximation) 7002, a geometry reconstructor (reconstructed geometry) 7003, an inverse coordinate transformer (inverse coordinate transformation) 7004, an arithmetic decoder (arithmetic decoding) 7005, an inverse quantizer (inverse quantization) 7006, a RAHT transformer 7007, an LOD generator (generated LOD) 7008, an inverse lifter (inverse lift) 7009, and / or a color inverse transformer (inverse color transformation) 7010.

[0168] An arithmetic decoder 7000, an octree synthesizer 7001, a surface approximation synthesizer 7002, a geometry reconstructor 7003, and a coordinate inverse transformer 7004 can perform geometric decoding. Geometric decoding according to the embodiment may include direct decoding and triplet geometric decoding. Direct encoding and triplet geometric decoding are selectively applied. Geometric decoding is not limited to the examples described above and is provided as a reference. Figures 1 to 6 The reverse process of the described geometric encoding is executed.

[0169] The arithmetic decoder 7000 according to the embodiment decodes the received geometric bitstream based on arithmetic coding. The operation of the arithmetic decoder 7000 corresponds to the inverse process of the arithmetic encoder 30004.

[0170] The octree synthesizer 7001 according to the embodiment can generate an octree by obtaining occupancy codes (or information about the geometry obtained as a decoding result) from the decoded geometry bitstream. The occupancy codes are as follows: Figures 1 to 6 Please describe that configuration in detail.

[0171] When applying triplet geometry encoding, the surface approximation synthesizer 7002 according to the implementation can synthesize the surface based on the decoded geometry and / or the generated octree.

[0172] According to the embodiment, the geometry reconstructor 7003 can regenerate geometry based on surface and / or decoded geometry. See also... Figures 1 to 9 As described, direct encoding and triadic geometric encoding are selectively applied. Therefore, the geometry reconstructor 7003 directly imports and sums the positional information of points for which direct encoding has been applied. When triadic geometric encoding is applied, the geometry reconstructor 7003 can reconstruct the geometry by performing the reconstruction operations (e.g., triangle reconstruction, upsampling, and voxelization) of the geometry reconstructor 30005. Details and references Figure 6 The descriptions are the same for all of them, so their descriptions are omitted. The reconstructed geometry may include point cloud images or frames that do not contain attributes.

[0173] According to the implementation method, the inverse coordinate transformer 7004 can obtain the point position based on the reconstructed geometric transformation coordinates.

[0174] The arithmetic decoder 7005, inverse quantizer 7006, RAHT transformer 7007, LOD generator 7008, inverse booster 7009, and / or color inverse transformer 7010 are executable references. Figure 6 The attribute decoding described herein includes Region Adaptive Hierarchical Transformation (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) decoding, and interpolation-based hierarchical nearest neighbor prediction (lifting transformation) decoding with update / lifting steps. These three decoding schemes may be used selectively, or a combination of one or more decoding schemes may be used. The attribute decoding according to the embodiments is not limited to the examples described above.

[0175] According to the implementation method, the arithmetic decoder 7005 decodes the attribute bitstream by arithmetic encoding.

[0176] The inverse quantizer 7006 according to the implementation dequantizes information about the decoded attribute bitstream or the attributes obtained as a decoding result, and outputs the dequantized attributes (or attribute values). Inverse quantization can be selectively applied based on the attribute encoding of the point cloud encoder.

[0177] According to the implementation, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 can handle the reconstructed geometry and inverse quantization attributes. As described above, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 can selectively perform decoding operations corresponding to the encoding of the point cloud encoder.

[0178] According to the implementation, the color inverse transformer 7010 performs inverse transformation encoding to inversely transform the color values ​​(or textures) included in the decoded attributes. The operation of the color inverse transformer 7010 can be selectively performed based on the operation of the color transformer 30006 of the point cloud encoder.

[0179] Although not shown in the figure, Figure 7 The elements of the point cloud decoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above-described... Figure 7 The point cloud decoder's components include at least one or more operations and / or functions. Additionally, one or more processors are operable or perform operations for executing... Figure 7 The software program and / or instruction set for the operation and / or function of the elements of the point cloud decoder.

[0180] Figure 8 A transmitting device according to an embodiment is shown.

[0181] Figure 8The transmitting device shown is Figure 1 The transmitting device 10000 (or Figure 3 Example of a point cloud encoder. Figure 8 The transmitting device shown can perform the same operation as the reference. Figures 1 to 6 The described point cloud encoder includes one or more of the same or similar operations and methods. The transmitting apparatus according to the embodiment may include a data input unit 8000, a quantization processor 8001, a voxelization processor 8002, an octree occupancy code generator 8003, a surface model processor 8004, an intra / inter-frame coding processor 8005, an arithmetic encoder 8006, a metadata processor 8007, a color transformation processor 8008, an attribute transformation processor 8009, a prediction / boosting / RAHT transformation processor 8010, an arithmetic encoder 8011, and / or a transmission processor 8012.

[0182] According to the embodiment, the data input unit 8000 receives or acquires point cloud data. The data input unit 8000 can perform operations and / or acquisition methods related to the point cloud video acquirer 10001 (or refer to...). Figure 2 The described acquisition process (20000) is the same as or similar to the operation and / or acquisition method.

[0183] The data input unit 8000, quantization processor 8001, voxelization processor 8002, octree occupancy code generator 8003, surface model processor 8004, intra / inter-frame coding processor 8005, and arithmetic encoder 8006 perform geometric coding. Geometric coding according to the implementation method and reference... Figures 1 to 9 The geometric codes described are the same or similar, so their detailed descriptions are omitted.

[0184] The quantization processor 8001 according to the implementation quantizes geometry (e.g., point position values). The operation of the quantization processor 8001 and / or quantization with reference... Figure 3 The operation and / or quantization of the described quantizer 30001 are the same or similar. Details and references Figures 1 to 9 The descriptions are the same.

[0185] According to the embodiment, the voxelization processor 8002 voxels the quantized position values ​​of points. The voxelization processor 8002 can execute and reference... Figure 3 The operation and / or voxelization process of the quantizer 30001 described are the same as or similar to the operation and / or process. Details and references Figures 1 to 6 The descriptions are the same.

[0186] According to the implementation method, the octree occupancy code generator 8003 performs octree encoding based on the voxelized positions of points in the octree structure. The octree occupancy code generator 8003 can generate occupancy codes. The octree occupancy code generator 8003 can execute and reference... Figure 3 and Figure 4 The operations and / or methods described are the same as or similar to those of the point cloud encoder (or octree analyzer 30002). Details and references Figures 1 to 6 The descriptions are the same.

[0187] According to the implementation, the surface model processor 8004 can perform triadic geometry encoding based on a surface model to reconstruct point positions in a specific region (or node) based on voxels. The surface model processor 8004 can perform operations with reference to... Figure 3 The operations and / or methods described are the same as or similar to those of the point cloud encoder (e.g., surface approximation analyzer 30003). Details and references are available. Figures 1 to 6 The descriptions are the same.

[0188] The intra / inter-frame coding processor 8005 according to the embodiment can perform intra / inter-frame coding on point cloud data. The intra / inter-frame coding processor 8005 can perform the same operations as referenced... Figure 7 The described intra / inter-frame coding is the same or similar. Details and references Figure 7 The descriptions are the same. According to an implementation, the intra / inter-frame coding processor 8005 may be included in the arithmetic encoder 8006.

[0189] The arithmetic encoder 8006 according to the embodiment performs entropy encoding on the octree and / or approximate octree of point cloud data. For example, the encoding scheme includes arithmetic encoding. The arithmetic encoder 8006 performs the same or similar operations and / or methods as the arithmetic encoder 30004.

[0190] The metadata processor 8007 according to an embodiment processes metadata (e.g., set values) about point cloud data and provides it to necessary processing procedures such as geometric encoding and / or attribute encoding. Additionally, the metadata processor 8007 according to an embodiment can generate and / or process signaling information related to geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be encoded separately from the geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be interleaved.

[0191] The color transformation processor 8008, attribute transformation processor 8009, prediction / boosting / RAHT transformation processor 8010, and arithmetic encoder 8011 perform attribute encoding. Attribute encoding and reference according to the implementation method. Figures 1 to 6 The attribute codes described are the same or similar, so their detailed descriptions are omitted.

[0192] According to the implementation, the color transformation processor 8008 performs color transformation encoding to transform color values ​​included in attributes. The color transformation processor 8008 may perform color transformation encoding based on reconstructed geometry. The reconstructed geometry and reference... Figures 1 to 9 The description is the same. Furthermore, its execution is the same as the reference. Figure 3 The operation and / or methods of the described color converter 30006 are the same as or similar to those described. Detailed descriptions are omitted.

[0193] According to the implementation, the attribute transformation processor 8009 performs attribute transformation to transform attributes based on reconstructed geometry and / or locations where geometric encoding is not performed. The attribute transformation processor 8009 performs transformations with reference to... Figure 3 The operation and / or method of the described attribute transformer 30007 are the same as or similar to those described. Detailed descriptions are omitted. The prediction / boosting / RAHT transformation processor 8010 according to the embodiment can encode the transformed attributes through any one or a combination of RAHT encoding, prediction transformation encoding, and boosting transformation encoding. The prediction / boosting / RAHT transformation processor 8010 performs and references... Figure 3 The RAHT transformer 30008, LOD generator 30009, and boost transformer 30010 described herein operate at least one of the same or similar operations. Furthermore, the predictive transform coding, boost transform coding, and RAHT transform coding are similar to those of the reference transformer. Figures 1 to 9 The descriptions are the same, so their detailed descriptions are omitted.

[0194] The arithmetic encoder 8011 according to the embodiment can encode the attributes of the code based on arithmetic encoding. The arithmetic encoder 8011 performs the same or similar operations and / or methods as the arithmetic encoder 300012.

[0195] The transmission processor 8012 according to an embodiment can transmit individual bitstreams containing encoded geometric and / or encoded attribute and metadata information, or transmit a single bitstream configured with encoded geometric and / or encoded attribute and metadata information. When the encoded geometric and / or encoded attribute and metadata information according to an embodiment is configured as a single bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to an embodiment may include signaling information and slice data. The signaling information includes a sequence parameter set (SPS) for sequence-level signaling, a geometric parameter set (GPS) for geometric information encoded signaling, an attribute parameter set (APS) for attribute information encoded signaling, and a tile parameter set (TPS) for tile-level signaling. The slice data may include information about one or more slices. A slice according to an embodiment may include a geometric bitstream Geom00 and one or more attribute bitstreams Attr00 and Attr10.

[0196] A slice is a series of syntactic elements that represent all or part of a encoded point cloud frame.

[0197] The TPS according to an embodiment may include information about each tile in one or more tiles (e.g., coordinate information and height / size information about the bounding box). The geometric bitstream may include a header and a payload. The header of the geometric bitstream according to an embodiment may include a geom_parameter_set_id, a geom_tile_id, and a geom_slice_id included in the GPS, as well as information about the data contained in the payload. As described above, the metadata processor 8007 according to an embodiment may generate and / or process signaling information and transmit it to the transmission processor 8012. According to an embodiment, the element performing geometry encoding and the element performing attribute encoding may share data / information with each other, as indicated by the dashed lines. The transmission processor 8012 according to an embodiment may perform the same or similar operations and / or transmission methods as the transmitter 10003. Details and References Figure 1 and Figure 2 The descriptions are the same, so their descriptions are omitted.

[0198] Figure 9 A receiving device according to an embodiment is shown.

[0199] Figure 9 The receiving device shown is Figure 1 The receiving device 10004 (or Figure 10 and Figure 11 Example of a point cloud decoder. Figure 9 The receiving device shown can perform the same operation as the reference. Figures 1 to 11 The same or similar one or more operations and methods described in the point cloud decoder.

[0200] The receiving apparatus according to the embodiments may include a receiver 9000, a receiving processor 9001, an arithmetic decoder 9002, an octree reconstruction processor based on occupancy codes 9003, a surface model processor (triangle reconstruction, upsampling, voxelization) 9004, an inverse quantization processor 9005, a metadata parser 9006, an arithmetic decoder 9007, an inverse quantization processor 9008, a prediction / boost / RAHT inverse transform processor 9009, a color inverse transform processor 9010, and / or a renderer 9011. Each decoding element according to the embodiments can perform the inverse process of the operation of the corresponding encoding element according to the embodiments.

[0201] Receiver 9000, according to an embodiment, receives point cloud data. Receiver 9000 can perform operations related to... Figure 1The operation and / or receiving method of the receiver 10005 are the same as or similar to those of the receiver. Detailed description omitted.

[0202] The receiving processor 9001 according to the embodiment can acquire a geometric bit stream and / or an attribute bit stream from the received data. The receiving processor 9001 may be included in the receiver 9000.

[0203] Arithmetic decoder 9002, octet-based octree reconstruction processor 9003, surface model processor 9004, and inverse quantization processor 9005 are capable of performing geometric decoding. Geometric decoding and reference according to the implementation method. Figures 1 to 10 The described geometric decodings are the same or similar, so their detailed descriptions are omitted.

[0204] The arithmetic decoder 9002 according to the embodiment can decode a geometric bitstream based on arithmetic coding. The arithmetic decoder 9002 performs the same or similar operations and / or encodings as the arithmetic decoder 7000.

[0205] According to an embodiment, the octree reconstruction processor 9003 based on occupancy codes can reconstruct an octree by obtaining occupancy codes from the decoded geometric bitstream (or information about the geometry obtained as a decoding result). The octree reconstruction processor 9003 based on occupancy codes performs the same or similar operations and / or methods as the octree synthesizer 7001 and / or the octree generation method. When applying triad geometry encoding, the surface model processor 9004 according to an embodiment can perform triad geometry decoding and related geometric reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on surface model methods. The surface model processor 9004 performs the same or similar operations as the surface approximation synthesizer 7002 and / or the geometry reconstructor 7003.

[0206] The geometry of reversible quantization decoding according to the implementation of the inverse quantization processor 9005.

[0207] The metadata parser 9006 according to the implementation can parse metadata (e.g., set values) contained in received point cloud data. The metadata parser 9006 can pass the metadata to geometry decoding and / or attribute decoding. Metadata and reference Figure 8 The metadata described is the same, so its detailed description is omitted.

[0208] Arithmetic decoder 9007, inverse quantization processor 9008, prediction / boost / RAHT inverse transform processor 9009, and color inverse transform processor 9010 perform attribute decoding. Attribute decoding and reference Figures 1 to 10 The properties described are decoded the same or similarly, so their detailed descriptions are omitted.

[0209] The arithmetic decoder 9007 according to the embodiment can decode the attribute bitstream via arithmetic coding. The arithmetic decoder 9007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 9007 performs the same or similar operations and / or encodings as the arithmetic decoder 7005.

[0210] The inverse quantization processor 9008 according to the embodiment can reversibly quantize and decode the attribute bitstream. The inverse quantization processor 9008 performs the same or similar operations and / or methods as the inverse quantizer 7006 and / or the inverse quantization method.

[0211] According to an embodiment, the prediction / boosting / RAHT inverse transform processor 9009 can process reconstructed geometry and inverse quantization attributes. The prediction / boosting / RAHT inverse transform processor 9009 performs one or more operations and / or decodings that are the same as or similar to those of the RAHT transformer 7007, LOD generator 7008, and / or inverse booster 7009. According to an embodiment, the color inverse transform processor 9010 performs inverse transform encoding to inverse transform color values ​​(or textures) included in the decoded attributes. The color inverse transform processor 9010 performs operations and / or inverse transform encodings that are the same as or similar to those of the color inverse transformer 7010. According to an embodiment, the renderer 9011 can render point cloud data.

[0212] Figure 10 An exemplary structure that can be combined with a point cloud data transmission / reception method / apparatus according to an embodiment is shown.

[0213] Figure 10 The structure represents a configuration in which at least one of the following components—server 1060, robot 1010, autonomous vehicle 1020, XR device 1030, smartphone 1040, home appliance 1050, and / or head-mounted display (HMD) 1070—is connected to cloud network 1000. Robot 1010, autonomous vehicle 1020, XR device 1030, smartphone 1040, or home appliance 1050 are referred to as devices. Furthermore, XR device 1030 may correspond to a point cloud data (PCC) device according to an embodiment or be operatively connected to a PCC device.

[0214] Cloud network 1000 can refer to a network that forms part of or exists within a cloud computing infrastructure. Here, cloud network 1000 can be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

[0215] Server 1060 can be connected via cloud network 1000 to at least one of robot 1010, self-driving vehicle 1020, XR device 1030, smartphone 1040, home appliance 1050 and / or HMD 1070, and can assist at least a portion of the processing of connected devices 1010 to 1070.

[0216] HMD 1070 represents one of the implementation types of an XR device and / or PCC device according to an embodiment. An HMD-type device according to an embodiment includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0217] Hereinafter, various embodiments of the apparatus 1010 to 1050 that apply the above-described technology will be described. Figure 10 The devices 1010 to 1050 shown are operable to be connected to / coupled to the point cloud data transmitting and receiving devices according to the above embodiments.

[0218] <PCC+XR>

[0219] The XR / PCC device 1030 may employ PCC technology and / or XR (AR+VR) technology, and may be implemented as an HMD, a head-up display (HUD) installed in a vehicle, a television, a mobile phone, a smartphone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a stationary robot, or a mobile robot.

[0220] The XR / PCC device 1030 can analyze 3D point cloud data or image data acquired through various sensors or from external devices and generate positional and attribute data about 3D points. Thus, the XR / PCC device 1030 can acquire information about the surrounding space or real-world objects and render and output XR objects. For example, the XR / PCC device 1030 can match an XR object, including auxiliary information about the identified object, with the identified object and output the matched XR object.

[0221] <PCC+XR+Mobile Phone>

[0222] The XR / PCC device 1030 can be implemented as a mobile phone 1040 by applying PCC technology.

[0223] Mobile phone 1040 can decode and display point cloud content based on PCC technology.

[0224] <PCC+Autopilot+XR>

[0225] The self-driving vehicle 1020 can be realized as a mobile robot, vehicle, unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0226] The self-driving vehicle 1020 employing XR / PCC technology can refer to a self-driving vehicle equipped with means for providing XR images, or a self-driving vehicle serving as a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, the self-driving vehicle 1020 can be distinguished from and operatively connected to the XR device 1030.

[0227] The autonomous vehicle 1020, equipped with means for providing XR / PCC images, can acquire sensor information from sensors including cameras and output generated XR / PCC images based on the acquired sensor information. For example, the autonomous vehicle 1020 may have a HUD and output XR / PCC images to it, thereby providing passengers with XR / PCC objects corresponding to real objects or objects presented on the screen.

[0228] When an XR / PCC object is output to a HUD, at least a portion of the XR / PCC object can be output to overlap with the actual object being pointed at by the passenger's eyes. Conversely, when an XR / PCC object is output to a display installed within the autonomous vehicle, at least a portion of the XR / PCC object can be output to overlap with objects on the screen. For example, the autonomous vehicle 1220 can output XR / PCC objects corresponding to objects such as roads, other vehicles, traffic lights, traffic signs, two-wheeled vehicles, pedestrians, and buildings.

[0229] Virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or point cloud compression (PCC) technologies according to the implementation methods are applicable to various devices.

[0230] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. On the other hand, AR technology refers to the technology of displaying virtually created CG images on top of images of real objects. MR technology is similar to AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology differs from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with the same characteristics as real objects. More specifically, an example of MR technology application is holographic services.

[0231] Recently, VR, AR, and MR technologies have often been referred to as extended reality (XR) technologies rather than being explicitly distinguished from each other. Therefore, embodiments of this disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are suitable for such technologies.

[0232] The PCC method / apparatus according to the embodiments can be applied to vehicles that provide self-driving services.

[0233] Vehicles providing autonomous driving services connect to the PCC device for wired / wireless communication.

[0234] When the point cloud data (PCC) transmitting / receiving device according to the embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services (which may be provided together with autonomous driving services) and transmit it to the vehicle. When the PCC transmitting / receiving device is installed in the vehicle, it can receive / process content data related to AR / VR / PCC services based on user input signals input through a user interface device and provide it to the user. The vehicle or user interface device according to the embodiment can receive user input signals. User input signals according to the embodiment may include signals indicating autonomous driving services.

[0235] The point cloud data transmission method / apparatus according to the embodiments can be interpreted as involving... Figure 1 Transmitting device 10000 Figure 1 Point cloud video encoder 10002 Figure 1 Transmitter 10003 Figure 2 Acquisition 20000 / Encoding 20001 / Transmission 20002 Figure 3 encoder, Figure 8 The transmitting device Figure 10 The device Figure 11 , Figure 20 , Figures 30 to 32 encoder, Figure 33 Terms related to sending methods, etc.

[0236] The point cloud data transmission method / apparatus according to the embodiments can be interpreted as involving... Figure 1 Transmitting device 10000 Figure 1 Point cloud video encoder 10002 Figure 1 Transmitter 10003 Figure 2 Acquisition 20000 / Encoding 20001 / Transmission 20002 Figure 3 encoder, Figure 8 The transmitting device Figure 10 The device Figure 11 , Figure 20 and Figures 30 to 32 encoder, Figure 34 Terms related to sending methods, etc.

[0237] Furthermore, the point cloud data transmission / reception method / apparatus according to the embodiments can be simply referred to as the method / apparatus.

[0238] According to the implementation method, the geometric data, geometric information, location information, and geometry constituting point cloud data will be interpreted as having the same meaning. Similarly, the attribute data and attribute information constituting point cloud data will be interpreted as having the same meaning.

[0239] The method / apparatus according to the implementation may include and perform a predictive transform for partial coding of G-PCC.

[0240] Implementations include a method for efficiently supporting selective decoding of portions of data when necessary, due to receiver performance or transmission rates of sending and receiving point cloud data. Implementations also include a method for selecting necessary information or removing unnecessary information within bitstream units by dividing geometric and attribute data delivered per data unit into semantic units such as geometric octrees and Level of Detail (LoD).

[0241] The implementation includes a method for configuring a data structure composed of point clouds. More specifically, the implementation discloses a method for efficiently transmitting layer-configured point cloud compression (PCC) data through packing and signaling, and includes a method for applying the packing and signaling method to scalable PCC-based services. In particular, the implementation includes a method for configuring segments and sending and receiving the segments to be more suitable for scalable PCC services when a direct compression mode is used for location compression. Specifically, the implementation can provide a compressed structure for efficiently storing and transmitting large point cloud data with wide distribution and high point density.

[0242] Reference Figure 3 Point cloud data consists of the location (geometric) (e.g., XYZ coordinates) and attributes (e.g., color, reflectivity, intensity, grayscale, opacity, etc.) of each data point. In point cloud compression (PCC), octree-based compression is performed to efficiently compress the non-uniform distribution characteristics in 3D space, and attribute information is compressed accordingly. Figure 3 This is a flowchart of the transmitting and receiving sides of the PCC. The operations according to the implementation can be handled by corresponding components on the transmitting and receiving sides of the PCC.

[0243] Figure 11 The process of encoding, transmitting, and decoding point cloud data according to the implementation method is illustrated.

[0244] Figure 11 Each component can correspond to hardware, software, processor, and / or a combination thereof.

[0245] The point cloud encoder 15000 is a transmission device that performs the transmission method according to the embodiment, and can scalably encode and transmit point cloud data.

[0246] The point cloud decoder 15010 is a receiving device that performs the receiving method according to the embodiment and can scalably decode point cloud data.

[0247] The source data received by encoder 15000 may include geometric data and / or attribute data.

[0248] The encoder 15000 scalably encodes point cloud data but does not immediately generate a partial PCC bitstream. Instead, when it receives full geometry and full attribute data, it stores the data in a storage device connected to the encoder. The encoder can then perform transcoding on the partial encoding, generating and sending a partial PCC bitstream. The decoder 15010 can receive and decode the partial PCC bitstream to reconstruct partial geometry and / or partial attributes.

[0249] Upon receiving the full geometry and all attributes, encoder 15000 can store the data in a storage device connected to the encoder and transcode the point cloud data using a low quantization parameter (QP) to generate and transmit a complete PCC bitstream. Decoder 15010 can receive and decode the complete PCC bitstream to reconstruct the full geometry and / or all attributes. Decoder 15010 can select partial geometry and / or partial attributes from the complete PCC bitstream via data selection.

[0250] The method / apparatus according to this embodiment compresses and transmits point cloud data by dividing the location information of data points, which are point cloud data, and feature information such as color / brightness / reflectivity into geometric information and attribute information. In this case, an octree structure with layers can be configured according to the level of detail, or PCC data can be configured according to the level of detail (LoD). Scalable point cloud data encoding and representation can then be performed based on the configured structure or data. In this case, due to the performance of the receiver or the transmission rate, only a portion of the point cloud data can be decoded or represented.

[0251] In this process, the method / apparatus according to the implementation can remove unnecessary data in advance. In other words, when only a portion of the scalable PCC bitstream needs to be sent (i.e., only some layers are decoded in scalable decoding), it is not possible to select and send only the necessary portion. Therefore, 1) the necessary portion needs to be re-encoded after decoding (15020), or 2) the receiver must selectively apply the operation after all the data has been transmitted to the receiver (15030). However, in case 1), a delay may occur due to the time spent on decoding and re-encoding (15020). In case 2), bandwidth efficiency may be reduced due to the transmission of unnecessary data. Furthermore, when using fixed bandwidth, it may be necessary to reduce the data quality for transmission (15030).

[0252] Therefore, the method / apparatus according to the implementation can define a slice segmentation structure for point cloud data and signal the scalable layer and slice structure for scalable transmission.

[0253] In implementation, to ensure efficient bitstream delivery and decoding, the bitstream can be divided into specific units to be processed.

[0254] For octree-based geometric compression, the methods / apparatus implemented may use both entropy-based encoding and direct encoding. In this case, a slice configuration is needed to efficiently utilize scalability.

[0255] The units according to the implementation may be referred to as LODs, layers, slices, etc. LOD is the same term as LOD in attribute data encoding, but can refer to a data unit used for a hierarchical structure of bitstreams. It can be a concept of a hierarchical structure based on point cloud data corresponding to a bundle of one or more depths, such as the depth (level) of an octree or multiple trees. Similarly, layers are units that provide a way to generate sub-bitstreams, and a layer is a bundle corresponding to one or more depths and can correspond to one or more LODs. Furthermore, a slice is a unit used to configure sub-bitstream units and can correspond to one depth, a portion of a depth, or two or more depths. Moreover, it can correspond to one LOD, a portion of an LOD, or two or more LODs. According to the implementation, LODs, layers, and slices can correspond to each other, or one of LODs, layers, and slices can be included in another. Furthermore, the units according to the implementation may include LODs, layers, slices, layer groups, or subgroups, and can be referred to as complementary to each other.

[0256] Furthermore, in the case of large point clouds with wide distribution and high density of points, latency issues may arise due to the large number of bitstreams that need to be processed to access regions of interest. The method / apparatus according to the embodiments can solve this latency problem. For the hierarchical point cloud structure for compressing large point cloud data with low latency according to the embodiments, tile / brick / slice layer groups / slice subgroups, tile / slice layer groups / slice subgroups, slice layer groups / slice subgroups, or layer group bbox->subgroup bbox relationships can be used.

[0257] Figure 12 The layer-based point cloud data configuration and the structure of the geometry and attribute bitstreams according to an implementation method are shown.

[0258] The transmission method / apparatus according to the implementation method can be configured as follows: Figure 12 The layer-based point cloud data shown is used for encoding and decoding of point cloud data.

[0259] The implementation aims to achieve efficient transmission and decoding by selectively transmitting and decoding data in bitstream units of layered point cloud data.

[0260] Depending on the application domain, point cloud data can be layered in terms of SNR, spatial resolution, color, temporal frequency, bit depth, etc., and layers can be constructed in the direction of increasing data density based on octree or LoD structure.

[0261] The method / apparatus according to the embodiments can be based on, for example... Figure 16 The layers shown are used to configure, encode, and decode geometric bitstreams and attribute bitstreams.

[0262] According to an embodiment of the present invention, the bit stream obtained by the transmitting device / encoder through point cloud compression can be divided into geometric data bit stream and attribute data bit stream according to the data type and then transmitted.

[0263] Each bitstream according to the implementation method can be composed of slices. Regardless of layer information or LoD information, the geometric data bitstream and the attribute data bitstream can each be configured as a slice and delivered. In this case, when only a portion of the layer or LoD is used, the following should be performed: 1) decode the bitstream, 2) select only the desired portion and remove unnecessary portions, and 3) re-encode only based on the necessary information.

[0264] Figure 13 A bitstream configuration according to an implementation method is shown.

[0265] The transmission method / apparatus according to the implementation method can generate, for example... Figure 13 The bitstream shown, and the receiving method / apparatus according to the embodiment can be used for, for example... Figure 13 The point cloud data included in the bitstream shown is decoded.

[0266] Bitstream configuration according to the implementation method

[0267] In implementation, to avoid unnecessary intermediate processing, the bitstream can be divided into layers (or LoD) and sent.

[0268] For example, in a LoD-based PCC structure, lower LoDs are included in higher LoDs. Information included in the current LoD but not in previous LoDs (i.e., newly included information in each LoD) can be called R (the remainder). Figure 17 As shown, the initial LoD information and the new information R included in each LoD can be divided and sent in each independent unit.

[0269] The transmission method / apparatus according to the implementation can encode geometric data and generate a geometric bitstream. The geometric bitstream can be configured for each LOD or layer. The geometric bitstream can include a header (geometric header) for each LOD or layer. The header can include reference information for the next LOD or layer. The current LOD (layer) can also include information R (geometric data) not included in previous LODs (layers).

[0270] The receiving method / apparatus according to the embodiments can encode attribute data and generate an attribute bitstream. The attribute bitstream can be configured for each LOD or layer, and the attribute bitstream can include a header (attribute header) for each LOD or layer. The header can include reference information for the next LOD or layer. The current LOD (layer) can also include information R (attribute data) not included in previous LODs (layers).

[0271] The receiving method / apparatus according to the embodiments can receive bitstreams composed of LODs or layers, and efficiently decode only the necessary data without complex intermediate processing.

[0272] Figure 14 An example of a bitstream sorting method according to an implementation method is given.

[0273] like Figure 14 As shown, the method / apparatus according to the embodiment can be used for... Figure 13 The bitstream is sorted.

[0274] Bitstream sorting method according to the implementation method

[0275] like Figure 14 As shown, when transmitting a bit stream, the transmission method / apparatus according to the implementation can transmit geometry and attributes serially. In this case, depending on the data type, the entire geometry information (geometric data) can be transmitted first, followed by the attribute information (attribute data). In this case, the geometry information can be quickly reconstructed based on the transmitted bit stream information.

[0276] exist Figure 14 In (a), for example, the layer (LOD) containing geometric data can be positioned first in the bitstream, and the layer (LOD) containing attribute data can be positioned after the geometric layer. Since the attribute data depends on the geometric data, the geometric layer can be placed first. Furthermore, the positions can be changed differently depending on the implementation. References can also be made between geometric headers and between attribute headers and geometric headers.

[0277] Reference Figure 14(b) allows for the collection and delivery of bitstreams comprising both geometric and attribute data within the same layer. In this case, the decoding execution time can be reduced by using compression techniques capable of decoding geometry and attributes in parallel. Furthermore, the information requiring initial processing (lower LoD, where geometry must precede attributes) can be placed first.

[0278] The first layer 1800 includes the geometric and attribute data corresponding to the lowest LOD0 (layer 0) along with their respective headers. The second layer 1810 includes LOD0 (layer 0) and also includes the geometric and attribute data of points from a new and more detailed layer 1 (LOD1) as information R1, which is not included in LOD0 (layer 0). The third layer 1820 can then be placed in a similar manner.

[0279] The transmitting / receiving method / apparatus according to the embodiments can efficiently select the desired layer (or LoD) in the application domain at the bitstream level when transmitting and receiving bitstreams. In the bitstream sequencing method according to the embodiments, geometric information is collected and transmitted ( Figure 18 Empty portions can be created in the middle after bitstream level selection. In this case, bitstream rearrangement may be necessary. This is especially true when considering the geometry and properties of each layer's binding and delivery. Figure 14 Unnecessary information can be selectively removed based on the application area, as follows.

[0280] Figure 15 A method for selecting geometric data and attribute data according to an implementation method is illustrated.

[0281] The bit stream selection according to the method / apparatus of the implementation method is performed as follows.

[0282] When it is necessary to select a bitstream as described above, the bitstream-level data can be selected according to the method / apparatus of the implementation: 1) symmetric selection of geometry and attributes; 2) asymmetric selection of geometry and attributes; or 3) a combination of the above two methods.

[0283] 1) Symmetrical selection of geometry and attributes

[0284] Reference Figure 15 This illustrates the case where only up to LoD1 (LOD0+R1) of LoDs are selected (19000) and sent or decoded, while the information corresponding to R2 (the new part in LOD2) in the upper layer is removed for sending / decoding.

[0285] 2) Asymmetric selection of geometry and attributes

[0286] The method / apparatus according to the implementation can transmit geometry and attributes asymmetrically. Only the attributes of the upper layer (attribute R2) are removed (19001), and the full geometry (from level 0 (root level) to level 7 (leaf level) in the triangle octree structure) can be selected and transmitted / decoded (19011).

[0287] Reference Figure 12 When point cloud data is represented in an octree structure and is hierarchically divided into LODs (or layers), scalable encoding / decoding (scalability) can be supported.

[0288] The scalability features, depending on the implementation, may include slice-level scalability and / or octree-level scalability.

[0289] According to the implementation, the LoD (Level of Detail) can be used as a unit to represent a collection of one or more octree layers. Furthermore, it can refer to a bundle of octree layers to be configured as slices.

[0290] In attribute encoding / decoding, the LOD, depending on the implementation, can be extended in a broader sense and used as a unit for detailed data partitioning.

[0291] In other words, spatial scalability of the actual octree layer (or scalable attribute layer) can be provided for each octree layer. However, when scalability is configured per slice before bitstream parsing, the choice can be made in LoD depending on the implementation.

[0292] In an octree structure, LOD0 can correspond to the root level to level 4, LOD1 can correspond to the root level to level 5, and LOD2 can correspond to the root level to level 7, with level 7 being the leaf level.

[0293] In other words, such as Figure 12 As shown, when scalability is utilized by slice, such as in the case of scalable transmission, the provided scalable step size can correspond to three steps: LoD0, LoD1, and LoD2. In the decoding operation, the scalable step size provided by the octree structure can correspond to eight steps from the root to the leaf.

[0294] According to the implementation method, for example, in Figure 12 In the process, when LoD0 to LoD2 are configured as the corresponding slices, the transcoder of the receiver or transmitter ( Figure 11 The transcoder 15040 can be selected as 1) LoD0 only, 2) LoD0 and LoD1, or 3) LoD0, LoD1 and LoD2 for scalable processing.

[0295] Example 1: When only LoD0 is selected, the maximum octree level can be 4, and a scalable layer can be selected from octree levels 0 to 4 during decoding. In this case, the receiver can treat the node size that can be obtained through the maximum octree depth as a leaf node and can send that node size via signaling information.

[0296] Example 2: When LoD0 and LoD1 are selected, layer 5 can be added. Therefore, the maximum octree level can be 5, and a scalable layer can be selected from octree layers 0 to 5 during decoding. In this case, the receiver can treat the node size that can be obtained through the maximum octree depth as a leaf node and can send that node size via signaling information.

[0297] According to the implementation method, the octree depth, octree level, and octree grade can be units that divide the data in detail.

[0298] Example 3: When LoD0, LoD1, and LoD2 are selected, layers 6 and 7 can be added. Therefore, the maximum octree level can be 7, and a scalable layer can be selected from octree layers 0 to 7 during decoding. In this case, the receiver can treat the node size that can be obtained through the maximum octree depth as a leaf node and can send that node size via signaling information.

[0299] Figure 16 An example is given of a method for slicing point cloud data according to an implementation method.

[0300] Slicing configuration according to the implementation method

[0301] According to the implementation method / apparatus / encoder, the G-PCC bitstream can be configured by segmenting the bitstream into a slice structure. The data unit used for detailed data representation can be a slice.

[0302] According to the implementation method, a slice can refer to a data unit used to partition point cloud data. That is, a slice represents a portion of point cloud data. A slice can be referred to as a term representing a part or unit.

[0303] For example, one or more octree layers can be matched with a slice.

[0304] According to the transmission method / apparatus of the embodiment, such as an encoder, a bit stream based on slice 2001 can be configured by scanning the nodes (points) included in the octree in the direction of scan sequence 2000.

[0305] exist Figure 16 In (a), some nodes in the octree layer can be included in a slice.

[0306] An octree level (e.g., level 0 to level 4) can form a slice 2002.

[0307] Partial data from an octree level (e.g., level 5) can form each slice 2003, 2004, 2005.

[0308] Partial data from an octree level (e.g., level 6) can form each slice.

[0309] exist Figure 16 In (b) and (c) of 16, when multiple octree layers match a slice, only some nodes from each layer may be included. In this way, when multiple slices constitute a geometry / attribute frame, the necessary information for configuring the layers can be delivered to the receiver. This information may include information about the layers included in each slice and information about the nodes included in each layer.

[0310] exist Figure 16 In (b), a portion of the data in the octree layers (e.g., levels 0 to 3) and level 4 can be configured as a slice.

[0311] An octree level (e.g., partial data at level 4 and partial data at level 5) can be configured as a slice.

[0312] An octree level (e.g., partial data at level 5 and partial data at level 6) can be configured as a slice.

[0313] An octree level (e.g., a portion of the data at level 6) can be configured as a slice.

[0314] exist Figure 16 In (c), the octree layer (e.g., data from level 0 to level 4) can be configured as a slice.

[0315] Partial data from each of the octree levels 5, 6, and 7 can be configured as a slice.

[0316] The encoder and the corresponding device according to the embodiment can encode point cloud data and can generate and transmit a bit stream including encoded data and parameter information related to the point cloud data.

[0317] Furthermore, when generating the bitstream, it can be based on the bitstream structure according to the implementation method (for example, see...). Figures 12 to 16 The receiving device, decoder, and corresponding device according to the embodiment can receive and parse a bitstream configured for selective partial data decoding, and partially decode and efficiently provide point cloud data (see [link to embodiment]). Figure 11 ).

[0318] Scalable transmission according to the implementation method

[0319] The point cloud data transmission method / apparatus according to the embodiments can transmit bit streams including point cloud data in a scalable manner, and the point cloud data reception method / apparatus according to the embodiments can receive and decode bit streams in a scalable manner.

[0320] When according to Figures 12 to 16 When the bitstream of the illustrated implementation is used for scalable transmission, information needed to select the desired slice for the receiver can be sent to the receiver. Scalable transmission can mean sending or decoding only a portion of the bitstream, rather than decoding the entire bitstream, and the result can be low-resolution point cloud data.

[0321] When applying scalable transport to octree-based geometry bitstreams, point cloud data may need to be configured with octree data only at each octree layer from the root node to the leaf node. Figure 16 The information within a specific octree layer of the bitstream.

[0322] Therefore, the target octree layer should not depend on information about lower octree layers. This can be a constraint applied jointly to both geometric encoding and attribute encoding.

[0323] Furthermore, in scalable transmission, a scalable structure is required for the transmitter / receiver to select scalable layers. Considering the octree structure according to the implementation, all octree layers can support scalable transmission, or scalable transmission can be allowed only for specific octree layers or lower layers. When a slice includes some layers in the octree, it can be indicated that the scalable layers included in the slice are scalable layers. Therefore, it can be determined whether the slice is necessary / unnecessary in the bitstream stage. Figure 16 In example (a), the yellow portion starting from the root node constitutes a scalable layer, but does not support scalable transport. Subsequent octree layers can be matched with scalable layers in a one-to-one correspondence. Typically, the portion corresponding to a leaf node can support scalability. When multiple octree layers are included in a slice, a scalable layer can be defined that should be configured for these layers.

[0324] In this context, scalable transmission and scalable decoding can be used separately, depending on the purpose. Scalable transmission can be used on the send / receive side to select information up to a specific layer without involving the decoder. Scalable decoding is used to select a specific layer during encoding. That is, scalable transmission can support the selection of necessary information in a compressed state (at the bitstream stage) without involving the decoder, allowing the information to be sent or determined by the receiver. On the other hand, scalable decoding can support encoding / decoding only the required portion of data during the encoding / decoding process, and therefore can be used as a scalable representation in this case.

[0325] In this context, the layer configuration for scalable transport can differ from the layer configuration for scalable decoding. For example, three bottom octree layers including leaf nodes can constitute a single layer in terms of scalable transport. However, when all layer information is included in terms of scalable decoding, scalable decoding can be performed for each of leaf node layer n, leaf node layer n-1, and leaf node layer n-2.

[0326] The method / apparatus according to the embodiments can perform fine-grained slicing.

[0327] When fine-grained slicing is enabled, the G-PCC bitstream (see [reference]) can be used in the encoding operation of the point cloud transmission method according to the implementation method. Figure 24 (etc.) is split into multiple sub-bitstreams. To efficiently utilize the layered structure of G-PCC, each slice can include encoded data for a portion of the coding layer or a portion of the region. Using partitions of slices paired with the coding layer structure, scalable transport or spatial random access use cases can be supported in an efficient manner.

[0328] Figure 17 An example of a geometric coding layer structure according to an implementation method is shown.

[0329] A point cloud data transmission method / apparatus according to an embodiment (i.e., Figure 1 Transmitting device 10000 Figure 1 Point cloud video encoder 10002 Figure 1 Transmitter 10003 Figure 2 Acquisition 20000 / Encoding 20001 / Transmission 20002 Figure 3 encoder, Figure 8 The transmitting device Figure 10 The device Figure 12 , Figure 21 , Figure 28 and Figure 29 encoder, Figure 23 and Figure 30 Methods such as transmission can encode point clouds into layered structures to generate layer-based bitstreams, such as... Figure 17 As shown.

[0330] A point cloud data receiving method / apparatus according to an embodiment (i.e., Figure 1 The receiving device 10004 Figure 1 Receiver 10005 Figure 1 Point cloud video decoder 10006 Figure 2 Transmission 20002 / Decoding 20003 / Rendering 20004 Figure 7 decoder Figure 9 The receiving device Figure 10 The device [[ID= ,​ , ​ and ​ decoder ​ and ​ (The receiving method, etc.) can receive layer-based point cloud data and bitstreams, and selectively decode the data, such as... ​ As shown.

[0331] The bitstream and point cloud data according to the implementation method can be generated based on slicing according to the coding layer. By slicing the bitstream at the end of the coding layer of the coding process, the method / apparatus according to the implementation method can select relevant slices, thereby supporting scalable transmission or partial decoding.

[0332] ​ (a) illustrates a geometric coding layer structure with eight layers, where each slice corresponds to a layer group. Layer group 13900 includes coding layers 0 through 4. Layer group 23901 includes coding layer 5. Layer group 33902 is a group for coding layers 6 and 7. When the geometry (or attribute) has a tree structure with eight levels (depths), the bitstream can be configured hierarchically by grouping data corresponding to one or more levels (depths). Each group can be included in a slice.

[0333] ​ (b) illustrates the decoded output when two slices are selected from three groups. When the decoder selects both group 1 and group 2, it can select partial layers of the tree at levels (depths) 0 to 5. That is, partial decoding of the coding layers can be supported by using slices of the layer group structure without accessing the entire bitstream.

[0334] For partial decoding processing according to the implementation method, the encoder can generate three slices based on the layer group structure. The decoder can perform partial decoding by selecting two slices from the three slices.

[0335] Bitstream according to the implementation method ( ​ The receiving method / apparatus may include layer-based groups / slices 3903. Each slice may include a header containing signaling information related to the point cloud data (geometric data and / or attribute data) included in the slice. The receiving method / apparatus according to the embodiment may select partial slices and decode the point cloud data included in the slice's payload based on the headers included in the slices.

[0336] In addition to the layer group structure, the method / apparatus according to the implementation can also consider the use case of spatial random access, dividing the layer group into several subgroups. Subgroups are mutually exclusive, and the set of subgroups can be the same as the layer group. Since the points of each subgroup form boundaries in the spatial domain, the subgroup can be represented by subgroup bounding box information. Based on spatial information, the layer group and subgroup structure can support spatial access. By efficiently comparing the region of interest (ROI) with the bounding box information about each slice, spatial random access within a frame or tile can be supported.

[0337] According to the method / apparatus of the embodiment, layer group 13900, layer group 23901 and layer group 33902 can be divided into one or more subgroups.

[0338] Although ​ The geometric coding layer structure is shown as an example, but the attribute coding layer structure can be generated in a similar manner.

[0339] The method / apparatus according to the implementation can perform layer-group-based slice segmentation. In fine-grained slices, each slice can contain encoded data from layer groups as defined below.

[0340] A layer group is defined as a set of consecutive tree layers whose start and end depths can be any number of tree depths, and whose start depth is less than its end depth. The order of the encoded data in a slice can be the same as the order of the encoded data in a single slice.

[0341] For example, imagine a geometric coding layer structure with eight layers, such as ​ As shown in (a). In this example, there are three layer groups, each matching a different slice: layer group 1 for coding layers 0 through 4, layer group 2 for coding layer 5, and layer group 3 for coding layer 6. When the first two slices are sent or selected, the decoded output will be a portion of layers 0 through 5, as shown in (a). ​ As shown in (b), by using slices in the layer group structure, partial decoding of the coding layer can be supported even without accessing the entire bitstream.

[0342] A subgroup is a subset of a layer group, where points within the subgroup are adjacent to each other. Subgroups of a layer group are mutually exclusive, and the set of points within a subgroup can be the same as the set of points within the layer group. Points within each subgroup are defined in a spatial region, and therefore, subgroup bounding box information can be used to describe the boundaries of the subgroup. Using spatial information, layer and subgroup structures can support efficient access to the Region of Interest (ROI) by selecting slices that cover the ROI.

[0343] ​ The layer group structure and subgroup structure according to the implementation method are illustrated.

[0344] ​The layer-based point cloud data and bitstream shown can be represented as follows: ​ The delimiting box shown.

[0345] The subgroup structure and corresponding bounding boxes are shown. Layer group 2 is divided into two subgroups (group 2-1 and group 2-2), and layer group 3 is divided into four subgroups (group 3-1, group 3-2, group 3-3, and group 3-4). Subgroups of layer group 2 and layer group 3 are included in different slices. Given slices of a layer group and subgroups with bounding box information, 1) the bounding box of each slice can be compared with the ROI, and 2) slices with subgroup bounding boxes associated with the ROI can be selected and spatial access can be performed. Then, 3) the selected slices are selected. When considering the ROI in region 3-3, slices 1, 3, and 6 are selected as the subgroup bounding boxes of layer group 1, subgroup 2-2, and subgroup 3-3 to cover the ROI region. For efficient spatial access, it is assumed that there are no dependencies between subgroups within the same layer group. In real-time streaming or low-latency use cases, selection and decoding operations can be performed when each slice is received to improve time efficiency.

[0346] The method / apparatus according to the embodiment can represent data as multiple layers (which may be referred to as depth or level) as a layer tree 4000 during geometric and / or attribute encoding. Point cloud data corresponding to layers (depth / level) can be grouped into layer groups (or groups) 4001, as shown in Figure 39. Each layer group can be further divided (segmented) into subgroups 4002. A bitstream can be generated by configuring each subgroup as a slice. The receiving apparatus according to the embodiment can receive the bitstream, select a specific slice, decode the subgroups included in the slice, and decode the bounding boxes corresponding to the subgroups. For example, when slice 1 is selected, the bounding boxes 4003 corresponding to group 1 can be decoded. Group 1 may be data corresponding to the largest region. When the user wishes to view the refined regions of group 1 in detail, the method / apparatus according to the embodiment can select slice 3 and / or slice 6, and the bounding boxes (point cloud data) of groups 2-2 and / or groups 3-3 can be accessed partially and hierarchically for the refined regions included in the region of group 1.

[0347] ​ Examples of multi-resolution, multi-size ROIs according to the implementation method are shown.

[0348] Layer slicing provides efficient access to large-scale or dense point cloud data based on scalability and spatial access capabilities. Rendering or displaying content can be time-consuming due to the large number of points and the large data size. As an alternative, the level of detail can be adjusted based on the viewer's interest. For example, structural or global region information is more important than local details when the viewer moves away from the scene or object. Conversely, more detailed information about the ROI is needed when the viewer moves closer to a specific region or object. Using adaptive methods, renderers can efficiently provide viewers with data of sufficient quality. ​ Examples of incremental detail changes at three viewing distance levels are illustrated, where the viewing distance varies based on the ROI. For example, the viewing distance could be 1) high-level view (coarse detail), 2) mid-level view (medium detail), and 3) low-level view (fine detail).

[0349] ​ A layer group slice is illustrated according to an implementation method.

[0350] A point cloud data transmission method / apparatus according to an embodiment (i.e., ​ Transmitting device 10000 ​ Point cloud video encoder 10002 ​ Transmitter 10003 ​ Acquisition 20000 / Encoding 20001 / Transmission 20002 ​ encoder, ​ The transmitting device ​ The device ​ , ​ , ​ and ​ encoder, ​ and ​ The sending method, etc., can be used to slice point cloud data based on layer groups, such as... ​ As shown.

[0351] A point cloud data receiving method / apparatus according to an embodiment (i.e., ​ The receiving device 10004 ​ Receiver 10005 ​ Point cloud video decoder 10006 ​ Transmission 20002 / Decoding 20003 / Rendering 20004 ​ decoder ​ The receiving device ​ The device ​ , ​ , ​ and ​ decoder ​ and ​(Methods such as receiving methods) can receive and decode layer-based segmented data, such as... ​ As shown.

[0352] The method / apparatus according to the implementation can support high-resolution ROI based on the scalability of hierarchical slicing and spatial accessibility.

[0353] Reference ​ The encoder can generate bitstream slices of octree layer groups or spatial subgroups of each layer group. Slices matching the ROI for each resolution can be selected and sent upon request. Since details other than the requested ROI are not included in the bitstream, the overall bitstream size can be smaller than that in tile-based methods. The receiver's decoder can combine slices to generate three outputs. For example, 1) a high-level view output can be generated from layer group slice 1; 2) a mid-level view output can be generated from layer group slice 1 and selected subgroups of layer group 2; 3) a low-level view with detailed output can be generated from layer group 1 and selected subgroups of layer groups 2 and 3. Because the output can be generated progressively, the receiver can provide viewing experiences such as zooming in / out. The resolution can gradually increase from the high-level view to the low-level view.

[0354] According to the implementation, encoder 60000 can correspond to a geometric encoder and an attribute encoder as a point cloud encoder. The encoder can slice the point cloud data based on layer groups (or subgroups). This layer can be referred to as the depth of the tree, the level of the layer, etc. As shown in section 60000-1, the depth of the geometric octree and / or the level of the attribute layer can be divided into layer groups (or subgroups).

[0355] The slice selector 60001, in conjunction with the encoder 60000, can select partitioned slices (or sub-slices) to selectively and partially transmit data, such as layer group 1 to layer group 3.

[0356] Decoder 6002 can decode point cloud data that is selectively and partially transmitted. For example, it can decode a high-level view of layer group 1 (which has a high depth / layer / level or index 0, or is close to the root). Subsequently, a mid-level view can be decoded by individually increasing the depth / level index above layer group 1 and layer group 2. Low-level views can be decoded based on layer groups 1 through 3.

[0357] ​ The nearest neighbor search process according to the implementation method is illustrated.

[0358] ​ An example is illustrated of a process for predicting the current point by finding the nearest neighbor when encoding and decoding a point using a prediction method / apparatus based on a transmission / reception method / apparatus, according to an embodiment.

[0359] The encoding / decoding process according to the implementation method can be configured as follows.

[0360] To encode and decode points, the encoding and decoding method according to the implementation can perform a nearest neighbor (NN) search to find the nearest neighbors similar to the current point and perform a prediction transformation. The nearest neighbor search can be referred to as an NN search, etc.

[0361] For example, the proposed G-PCCv1-based NN search method also finds prediction candidates from neighboring nodes with higher LoD and the encoding node with the current LoD. For fast search, the search range is limited by the cubic boundary and the number of points in the Morton encoding order. Considering subgroup boundaries, when neighboring candidates are in the same layer group as the current node, the implementation can use additional constraints called intra-layer group search boundaries.

[0362] Within a layer group, when a neighbor candidate is in the same layer group as the current node, the use of neighbor nodes in the same subgroup boundary is restricted.

[0363] In other words, neighboring nodes outside the subgroup bounding box cannot be considered as neighbor candidates for the current node, such as... ​ As shown. For example, when the current encoding / decoding target is the first subgroup 2100, the first subgroup 2100 includes nodes (points) belonging to LoD N-1 and LoD N. To predict the current node 2101, a prediction value can be generated by referring to the current node's parent node or predecessor node. Nodes belonging to the second subgroup 2102, which is different from the first subgroup 2100 to which the current node 2101 belongs, are excluded from the neighbor candidates of the current node 2101.

[0364] Furthermore, the implementation can use layer group adaptation positions in subgroup estimation and neighbor distance calculation. Therefore, considering the case of missing subgroups, discrepancies between the encoder and decoder can be prevented.

[0365] Layer group adaptation position: Indicates the number of LoDs of the descendant subgroups to the right of the encoding position, and then the number of LoDs of the descendant subgroups and skipped descendant subgroups to the left.

[0366] ​ An example of the NN search process according to an implementation method is shown.

[0367] ​ Examples similar to ​ The NN search process in [the context of the NN search].

[0368] When the current LoD is at the coarsest level of the current layer group, neighbor candidates can be in different layer groups. In this case, when a node is at the upper subgroup boundary, it can search for neighbors outside the subgroup boundary.

[0369] Search boundaries between layers: such as ​ As shown, when a neighbor candidate is in a higher-level group, the use of neighbor nodes at the boundary of the upper subgroup is restricted.

[0370] ​ An example of the NN search process according to an implementation method is shown.

[0371] ​ Examples similar to ​ and ​ The NN search process in [the context of the NN search].

[0372] When using layer group adaptation, the encoding of the node location from the parent node may differ when the node is within the bounding box of the current node's subgroup.

[0373] The layer group adaptation position can be configured as follows according to the implementation method.

[0374] For nodes within the subgroup bounding box: the encoding position is shifted to the right by the number of LoDs of the current subgroup's encoded subgroup, and then shifted to the left by the number of LoDs of the current subgroup's encoded subgroup and the number of skipped subgroups.

[0375] Nodes outside the subgroup bounding box: The encoding position is shifted to the right by the number of LoDs in the encoded subgroup of the upper subgroup, and then shifted to the left by the number of LoDs in the encoded subgroup of the upper subgroup and the number of skipped subgroups.

[0376] In other words, when a neighbor candidate is within the subgroup bounding box, details of the geometric position up to the highest level of the subgroup are considered. Conversely, when a neighbor candidate is outside the subgroup bounding box, details of the geometric position up to the highest level of the parent-child group are considered.

[0377] Reference ​ In this implementation, to generate a predicted value for the current node in the first subgroup 2100, reference can be made to nodes with higher LoD levels included in the first subgroup 2100 to which the current node belongs and / or previous nodes with the current LoD level in the first subgroup 2100 to which the current node belongs. Nodes not in the first subgroup but in the second subgroup 2102 are excluded from the candidates used to predict the current node.

[0378] Reference ​In this implementation, a second subgroup to which the current node does not belong cannot be used as a neighbor candidate for predicting the current node. The first and second subgroups can be demarcated by a boundary. However, a parent-child group including the first and second subgroups can be used as a neighbor candidate for predicting the current node. Nodes belonging to a parent-child group with a higher LoD level than the current node in the first subgroup can be neighbor candidates for the current node.

[0379] Reference ​ More specifically, the geometric location (geometric information, geometric shape, etc.) used to predict the current node can be considered. The geometric location values ​​of nodes belonging to the first subgroup 2100, which includes the current node 2101, and located at a lower LoD level than the current node, can be candidates for the geometric location of the current node's neighbors. That is, geometric locations can be selected from the range of nodes included in the subgroup bounding box. Nodes belonging to a second subgroup different from the first subgroup to which the current node belongs cannot be candidates for the current node's neighbors. However, when the second subgroup is a sibling subgroup of a parent-child group that is the same as the first subgroup to which the current node belongs, nodes belonging to the parent-child group can be candidates for the current node's neighbors. When selecting nodes belonging to the parent-child group, the geometric location of nodes in the parent-child group may not be defined in detail to the range of the subgroup, and the geometric location may only be determined to the LoD level to which the parent-child group belongs.

[0380] For example, the NN search according to the implementation method can be represented by the following pseudocode.

[0381]

[0382]

[0383]

[0384]

[0385]

[0386]

[0387]

[0388]

[0389]

[0390]

[0391]

[0392]

[0393] The point cloud data transmission / reception method / apparatus according to the implementation can search for nearest neighbors and perform atlas shifting.

[0394] When using upper (parent) child groups with additional missing nodes, the differences in node positions between the parent-child group and the child-child group can be significant, so neighbors may not be found in the current attribute atlas. If the current point does not exist in the current atlas, an atlas shifting process can be performed to align the starting positions of nodes in the parent and child LoDs.

[0395] For example, atlas shifting can be represented in pseudocode as follows.

[0396]

[0397] ​ The starting node included in a subgroup may differ from the starting node included in the parent group. In this case, the nodes in the parent group may not match the nodes in the subgroup, resulting in incorrectly encoded attribute values. To avoid this, a process can be added to shift the atlas of nodes (points) included in the parent group so that it matches the atlas of the starting nodes of the subgroup.

[0398] Compare the atlas ID of the current point, which is the target of the current attribute encoding, with the current atlas ID.

[0399] When the current atlas ID is different from the atlas ID of the current point, update the current atlas ID to the atlas ID of the current point.

[0400] When the atlas ID of a node belonging to the upper part is different from the current atlas ID (it can be smaller than the current atlas ID because atlas IDs are sorted in ascending order), a point index called the cube index can be added to select only points belonging to the current atlas and exclude points that do not belong to the current atlas.

[0401] For nodes (points) in a subgroup that belong to the current atlas, the current atlas information can be updated based on a combination of the point index (called the cube index) and the atlas index.

[0402] The process involves constructing an atlas by selecting points belonging to the child group's bounding box (bbox) from among the points belonging to the parent group when the child group's bounding box (bbox) is included in the parent group's bounding box (bbox).

[0403] When the current LoD is at the Nth level, the retained LoD is at the N - 1th level. Therefore, in "while(packedVoxel[retained[cubeIndex]].mortonCode>>atlasBoundaryBit)<curAtlasId)++cubeIndex;", this process finds the initial starting point of the nodes in the parent subgroup included in the bounding box range of the child subgroup by excluding the points that do not belong to the current atlas ID (curAtlasId) from the points included in LoD N - 1. Then, through the second while loop, the retained points that belong to the current atlas ID (curAtlasId) are registered into the atlas.

[0404] Regarding the relationship between atlas shifting and NN search, when selecting a node in the parent subgroup as a neighbor candidate to predict the attributes of a node in the child subgroup, the atlas of the parent subgroup can be shifted based on the atlas ID of the child subgroup.

[0405] The point cloud data sending / receiving method / apparatus according to the embodiment can search for the nearest neighbor, shift the atlas (when necessary), and derive subgroup weights.

[0406] When predicting the transformation, quantization weights are used to assign credit to the nodes used in the prediction. In G - PCCv1, the weights are derived as the sum of the weights of the relevant nodes that reference the current node as a neighbor. Based on this concept, the following constraints can be used to calculate the weights of the nodes in the subgroup.

[0407] Subgroup weight derivation: When the current node and its neighbors belong to the same subgroup, the sum of the weights of the relevant nodes that reference the current node as a neighbor is generated.

[0408] Refer to ​ For each point (node) in the subgroup, weights for compensating for the loss in neighbor selection can be assigned to the nodes that are frequently selected as neighbors in order to reduce the loss.

[0409] Weights are used for quantization.

[0410] The process of generating weights is configured as follows:

[0411] weight(m)+=alpha*numRefNodes(m) / maxNumRefNodes+beta

[0412] numRefNodes: Indicates the number of times this node is used as a neighbor at a specific node m.

[0413] maxNumRefNodes: Indicates the maximum number of nodes in the parent group that use the current node as a neighbor.

[0414] alpha and beta are variables.

[0415] The process of calculating the quantization weights can be represented by the following pseudocode.

[0416]

[0417]

[0418] This method ensures that the decoder targets the entire fine-grained slice (FGS) (see reference). ​ The layer-based slices (i.e., the term referring to slice granularity units) and the missing FGS both generate the same subgroup weights as the encoder. However, the weights may be significantly smaller than the expected weights, and the loss may be exacerbated at higher LoDs. Implementations may include performing subgroup weight adjustments for each subgroup and each node to mitigate the loss.

[0419] The subgroup weights are adjusted as follows.

[0420] weight(m)+=alpha*numRefNodes(m) / maxNumRefNodes+beta

[0421] weight(m) is the subgroup weight of the m-th node.

[0422] numRefNodes(m) is the number of related nodes of the m-th node.

[0423] maxNumRefNodes is the maximum number of related nodes in the current subgroup.

[0424] alpha and beta are weight adjustment coefficients that can be derived from the encoder.

[0425] As a method for adaptively correcting the weights of each node during the subgroup weight adjustment process, the extent to which weights are used for correction can be determined based on the number of times each node within the subgroup is referenced by neighboring nodes, as described above. In addition to the above method, another approach can be used for adaptive node correction weight calculation. Indicators indicating the importance of each node (such as proportional weights based on weight size, subgroup weights, and the number of subgroup nodes) can be used to improve the correction accuracy of important nodes (i.e., nodes frequently referenced by other nodes).

[0426] The process of weight utilization and correction according to the implementation method can be represented by pseudocode as follows.

[0427]

[0428]

[0429]

[0430]

[0431]

[0432]

[0433] The encoder can calculate alpha + beta as the maximum value of the subgroup outside weights, and set beta as the average value of the subgroup outside weights at the leaf nodes of the subgroup. For the outside of the subgroup, the outside weights can be estimated from the current subgroup.

[0434] Furthermore, during the encoding process, it can be determined whether each node is included in a subgroup based on subgroup bounding box information. When a subgroup changes, the encoder, context, zero-run information, and arithmetic encoder for each subgroup are updated. Zero-run (a method to improve compression efficiency by sending a certain number of consecutive zeros) can lead to discontinuities within LoDs and reduce compression efficiency due to subgroup partitioning. According to implementations, more efficient compression can be achieved by concatenating the end of the previous LoD and the beginning of the next LoD within the same subgroup.

[0435] This process can be represented by the following pseudocode.

[0436]

[0437]

[0438]

[0439]

[0440]

[0441]

[0442]

[0443]

[0444]

[0445]

[0446]

[0447]

[0448]

[0449] ​ An attribute encoding method according to an implementation method is illustrated.

[0450] according to ​ The point cloud data transmission method / apparatus of the implementation method corresponds to ​ Transmitting device 10000 ​ Point cloud video encoder 10002 ​ Transmitter 10003 ​ Acquisition 20000 / Encoding 20001 / Transmission 20002 ​ encoder, ​ The transmitting device ​ The device ​ , ​ , ​ and ​ encoder, ​ Sending methods, etc.

[0451] ​ This is an exemplary flowchart of a layer group slice encoder for LoD-based attribute encoding. First, parameters for SPS, GPS, APS, LGSI, etc., are generated, and the layer group structure is configured. The layer group structure can be configured during or before generating the layer group slice list (LGSI). After performing geometric encoding, LoD generation is performed based on the geometric nodes. Attribute encoding layers (e.g., LoD, RAHT encoding layers) are matched with the layer group, and quantization weights are obtained based on the LoD. Then, for each attribute encoding layer, it is checked whether the layer group has changed, and for each node in the attribute encoding layer, it is checked whether the subgroup has changed. When a subgroup or layer group changes, the attribute encoder used for the previous subgroup can be stored, and the attribute encoder used for the current subgroup can be used to ensure continuity of context, zero-run, etc., within the subgroup and to ensure independence from neighboring subgroups. After compressing nodes belonging to all attribute encoding layers, the encoded bitstream of each subgroup is packaged into each slice.

[0452] Although the example described above is based on LoD-based attribute encoding, it can also be applied to other attribute encodings included in the implementation, such as RAHT.

[0453] Reference ​ The encoder can be based on ​The flowchart describes how point cloud data is encoded. Depending on the encoder's encoding configuration, parameter values ​​can be generated. Here, parameters such as SPS, GPS, APS, LGSI, etc., can be generated along with the encoded point cloud data and sent in the bitstream. The encoder can first encode the geometric data of the point cloud data using a geometry encoder. After geometric encoding, the encoder can generate a Level of Detail (LOD) using an attribute encoder. During LOD generation, attributes can be represented in a layer group structure (which may be called a layer group including subgroups, an FGS including subgroups, etc.) according to the above implementation. The LOD can be mapped to layer groups. One or more indices of the LOD (which may be called layers, levels, depths, etc.) can be grouped (or mapped) into layer groups. Weights can be assigned to each layer group. For example, when the highest-level layer group is important, it can be assigned a larger weight. On the other hand, when the lowest-level layer group is important, it can be assigned a larger weight. The weights are variable depending on the desired configuration, such as random access to data about the level of detail and partially encoded data. Layer groups can be determined. Here, the determined layer group can be a layer group to be partially encoded or partially transmitted. Subgroups can be determined within subgroups belonging to the layer group. The determined subgroups can be subgroups to be partially encoded or partially transmitted. Check if the subgroup has changed. If the subgroup has changed, context information can be saved when encoding the subgroup, and necessary context information can be loaded. Node attributes are encoded. If the subgroup has not changed, node attributes can be encoded immediately. When the end of the node and the end of the LOD are reached, attribute data unit headers can be generated. The attribute bitstream can be generated and transmitted together with the geometric bitstream. Additionally, predictive coding can be performed based on the subgroup, such as... ​ exemplified in .

[0454] ​ An attribute decoding method according to an implementation method is illustrated.

[0455] according to ​ The point cloud data receiving method / apparatus of the implementation method corresponds to ​ The receiving device 10004 ​ Receiver 10005 ​ Point cloud video decoder 10006 ​ Transmission 20002 / Decoding 20003 / Rendering 20004 ​ decoder ​ The receiving device ​ The device ​ , ​ , ​ and ​ decoder ​ The receiving method, etc.

[0456] The attribute decoder takes as input the attribute fine-grained slice bitstream, decoded geometry, and layer group structure. After geometry decoding, LoD generation is performed based on the provided decoded geometry, and weight derivation is performed according to the node-based relationships of the LoD. Information generated through these operations is used in subsequent attribute decoding operations. The first attribute slice can be operated on in the same way as the previous attribute slice. Then, when layer group slicing is enabled, the dependent attribute data cell header can be parsed, and information from `layer_group_id` and `subgroup_id` can be retrieved. Information about the LoD can be signaled when necessary. Based on this information, layer groups matching the current LoD can be searched, and parent and child groups can be selected. When the layer group structure is applied to geometry and attributes in the same manner, the reference and parent information used in geometry decoding can be used. Based on this information, information in the dependent attribute data cells can be decoded, and the point cloud data can be reconstructed.

[0457] During decoding, the LoD generation and weight derivation methods used by the encoder can be employed. Therefore, LoDs can be mapped to layer groups.

[0458] Reference ​ The decoder can be based on ​ The flowchart shown illustrates the decoding of point cloud data. Parameters contained in the bitstream, such as SPS, GPS, APS, and LGSI, can be parsed. The decoder's geometry decoder can decode the geometry data. The decoder's attribute decoder can generate a Level of Detail (LOD). When generating the LOD, a layer group structure (FGS) according to the implementation can be used. LOD-specific weights can be derived. These weights can be applied to attributes in the reverse process of the encoder's operation. Attribute data unit headers in the bitstream can be parsed. Attribute data units in the bitstream can be decoded. It is checked whether layer group slicing is enabled. When layer group slicing is enabled, dependent attribute data unit headers are parsed. LODs can be mapped to layer groups. A higher subgroup of the current subgroup of the current layer group can be selected. A higher subgroup can refer to the parent-child group of the current subgroup, the parent-parent-child group, etc. Dependent attribute data units can be decoded. It is checked whether the end of the geometry bitstream has been reached. The attribute bitstream corresponding to the geometry bitstream can be decoded. Both geometry and attributes are decoded, and a point cloud output is generated. Then, the decoding process is terminated. When layer group slicing is not enabled, decoding can be terminated without any layer group-based operations. Additionally, predictive decoding can be performed based on subgroups, such as... ​ exemplified in .

[0459] ​ An example is shown of a bitstream containing parameters and encoded point cloud data according to an implementation method.

[0460] A point cloud data transmission method / apparatus according to an embodiment (which corresponds to)​ Transmitting device 10000 ​ Point cloud video encoder 10002 ​ Transmitter 10003 ​ Acquisition 20000 / Encoding 20001 / Transmission 20002 ​ encoder, ​ The transmitting device ​ The device ​ , ​ , ​ and ​ encoder, ​ (Sending methods, etc.) can generate, for example, ​ The bit stream shown.

[0461] according to ​ The point cloud data receiving method / apparatus of the implementation method (which corresponds to) ​ The receiving device 10004 ​ Receiver 10005 ​ Point cloud video decoder 10006 ​ Sending 20002 / Decoding 20003 / Rendering 20004 ​ decoder ​ The receiving device ​ The device ​ , ​ , ​ and ​ decoder ​ (The receiving method, etc.) can be parsed as follows: ​ The bit stream shown.

[0462] Information regarding separated slices (referring to layer groups, subgroups, FSGs, etc.) according to the implementation method can be defined in the parameter set and SEI message as follows. It can be defined in the sequence parameter set, geometric parameter set, attribute parameter set, geometric slice header, and attribute slice header, and can be defined at corresponding locations or individual locations depending on the application or system, to use different application scopes, different application methods, etc. In other words, it can have different meanings depending on the location carrying the signal. When information is defined in SPS, it can be applied equally throughout the entire sequence. When defined in GPS, it can indicate that the information is used for location reconstruction. When defined in APS, it can indicate that the information is applied to attribute reconstruction. When defined in TPS, it can indicate that the signaling is applied only to points within a tile. When delivered on a slice-based basis, it can indicate that the signaling is applied only to the corresponding slice. Furthermore, it can be defined at corresponding locations or individual locations depending on the application and system, to use different application scopes, different application methods, etc. Additionally, when the syntax elements defined below apply to multiple point cloud data streams as well as the current point cloud data stream, this information can be carried in the parent parameter set, etc.

[0463] Each abbreviation has the following meaning. Each abbreviation can be referred to by another term within the equivalent range: SPS: Sequence Parameter Set; GPS: Geometric Parameter Set; APS: Attribute Parameter Set; TPS: Tile Parameter Set; Geom: Geometric Bitstream = Geometric Tile Header + Geometric Tile Data; Attr: Attribute Bitstream = Attribute Brick Header + Attribute Brick Data.

[0464] Although defining information independently of encoding techniques has been described, it is possible to define information in conjunction with encoding methods. It can be defined within a tile parameter set to support varying scalability across regions. Furthermore, when the syntax elements defined below apply to multiple point cloud data streams as well as the current point cloud data stream, this information can be carried in higher-level parameter sets, etc.

[0465] Alternatively, a Network Abstraction Layer (NAL) unit can be defined, and relevant information for selecting a layer (such as layer_id) can be delivered, allowing the bitstream to be selected at the system level.

[0466] The parameters (which may be referred to as metadata, signaling information, etc.) according to the embodiments can be generated in the processing of the transmitter according to the embodiments described below and delivered to the receiver according to the embodiments for use in the reconstruction process.

[0467] For example, the parameters according to the embodiment can be generated in the metadata processor (or metadata generator) of the transmitting device according to the embodiment described below, and obtained by the metadata parser of the receiving device according to the embodiment.

[0468] Generate a transmission method / apparatus according to the implementation method, such as ​ The information shown, and send the message containing that information. ​ The bitstream.

[0469] ​ An example of a sequence parameter set according to an implementation method is shown.

[0470] ​ Examples are included in ​ The set of sequence parameters in the bit stream.

[0471] sps_extension_flag: Indicates whether the sequence parameter set is extended. Depending on whether the set is extended, it contains the following elements:

[0472] layer_group_enabled_flag: Indicates whether the bitstream included in the sequence is encoded based on layer groups (e.g., ​ ).

[0473] num_layer_groups_minus1: Indicates the number of layer groups contained in the bitstream.

[0474] layer_group_id[i]: Identifies each layer group.

[0475] num_layers_minus1[i]: Indicates the number of layers included in the layer group.

[0476] subgroup_enabled_flag[i]: Indicates whether the subgroup includes subgroups, such as ​ As shown.

[0477] subgroup_bbox_origin_bits_minus1: Indicates the origin position of the bounding box of the subgroup.

[0478] subgroup_bbox_size_bits_minus1: Indicates the size (width, height, and depth) of the bounding box of the subgroup.

[0479] root_subgroup_bbox_origin: Indicates the origin of the bounding box that is the root of the subgroup's bounding box.

[0480] root_subgroup_bbox_size: Indicates the size (width, height, depth) of the bounding box that is the root of the subgroup's bounding box.

[0481] root_subgroup_bbox_origin indicates the origin of the bounding box of the subgroup of the root subgroup.

[0482] root_subgroup_bbox_size indicates the size of the bounding box of the subgroup of the root subgroup.

[0483] ​ An example of the attribute parameter set and attribute data unit header according to the implementation method is shown.

[0484] ​ Examples are included in ​ The attribute parameter set and attribute data unit header in the bitstream.

[0485] `aps_attr_ref_id_present_flag`: When equal to 1, it indicates the context reference of the attribute slice based on `attr_ref_layer_group_id` and `attr_ref_subgroup_id`. It indicates the context state of subsequent attribute slices, as indicated by `attr_context_reference_indication_flag`. When `aps_attr_ref_id_present_flag` equals 0, the context reference and context state are inherited from the geometry slice corresponding to the `layer_group_id` and `subgroup_id` of the currently dependent attribute slice.

[0486] The increments of subgroup_weight_adj_coeff_a_bits_minus1 and subgroup_weight_adj_coeff_b_bits_minus1 by 1 indicate the sizes of subgroup_weight_adj_coeff_a and subgroup_weight_adj_coeff_b, respectively.

[0487] subgroup_weight_adj_coeff_a and subgroup_weight_adj_coeff_b indicate the coefficients used for weight adjustment in the current subgroup.

[0488] ​ An example of a dependency attribute data unit header according to an implementation method is shown.

[0489] ​ Examples are included in ​ The header of the dependency attribute data unit in the bitstream.

[0490] The `dadu_attribute_parameter_set_id` of the dependent attribute data cell indicates the active attribute parameter set (APS) indicated by `aps_attr_parameter_set_id`. The value of `dgdu_attribute_parameter_set_id` of the dependent geometry data cell is equal to the value of `adu_geometry_parameter_set_id` of the attribute data cell of the corresponding slice.

[0491] dadu_sps_attr_idx identifies the attribute that is encoded as an index of the list of active SPS attributes.

[0492] dadu_slice_id indicates the attribute fragment to which the current dependent attribute data unit belongs.

[0493] `dadou_layer_group_id` is an indicator of the layer group of the slice. `dadou_layer_group_id` is in the range of 0 to `num_layer_groups_minus1`. If it does not exist, it is inferred to be 0.

[0494] `dadou_subgroup_id` indicates the subgroup identifier of the layer group referenced by `dadu_layer_group_id`. `dadou_subgroup_id` is in the range of 0 to `num_subgroups_minus1[dadu_layer_group_id]`. Here, `dadu_subgroup_id` indicates the order of slices within the same `dadu_layer_group_id`. If it does not exist, `dadu_subgroup_id` is inferred to be 0.

[0495] `attr_ref_layer_group_id` is an indicator that specifies the layer group identifier of the context reference of the current dependent attribute data unit. `attr_ref_layer_group_id` is in the range of 0 to `attr_layer_group_id` for the current dependent attribute data unit.

[0496] `attr_ref_subgroup_id` is an indicator of the reference subgroup of the layer group indicated by `attr_ref_layer_group_id`. Within the layer group indicated by `attr_ref_layer_group_id`, `attr_ref_subgroup_id` is in the range of 0 to `num_subgroup_id_minus1`.

[0497] When attr_context_reference_indication_flag equals 1, it indicates that the context state of the current dependency slice is inherited by one or more subsequent dependency slices. When attr_context_reference_indication_flag equals 0, it indicates that the context state of the current dependency slice is not inherited by subsequent dependency slices.

[0498] The decoder can use `attr_context_reference_indication_flag` to manage the context buffer. When `attr_context_reference_indication_flag` equals 1, the context state of the current dependent slice is stored in the context buffer at the end of decoding. When `attr_context_reference_indication_flag` equals 0, the context state of the current dependent slice is not stored in the context buffer.

[0499] When `subgroup_weight_adj_coeff_present_flag` equals 1, it indicates that `subgroup_weight_adj_coeff_a` and `subgroup_weight_adj_coeff_b` exist for the current subgroup corresponding to the current slice. When `subgroup_weight_adj_coeff_present_flag` equals 0, it indicates that the subgroup weight adjustment coefficients do not exist. The subgroup weight adjustment coefficients (`subgroup_weight_adj_coeff_a` and `subgroup_weight_adj_coeff_b`) are inferred to be 0.

[0500] Figure 30 The process of sending partial point cloud data according to the implementation method is illustrated.

[0501] A point cloud data transmission method / apparatus according to an embodiment (which corresponds to) Figure 1 Transmitting device 10000 Figure 1 Point cloud video encoder 10002 Figure 1 Transmitter 10003 Figure 2 Acquisition 20000 / Encoding 20001 / Transmission 20002 Figure 3 encoder, Figure 8 The transmitting device Figure 10 The device Figure 11 , Figure 20 , Figure 30 and Figure 32 encoder, Figure 33 The transmission method (e.g., the transmission method) can scalably encode the entire point cloud data, store it in a storage device, perform transcoding for partial encoding, and then transmit the partial bit stream, such as... Figure 30 As shown.

[0502] A point cloud data receiving method / apparatus according to an embodiment (which corresponds to) Figure 1 The receiving device 10004 Figure 1 Receiver 10005 Figure 1 Point cloud video decoder 10006 Figure 2 Transmission 20002 / Decoding 20003 / Rendering 20004 Figure 7 decoder Figure 9 The receiving device Figure 10 The device Figure 11 , Figure 20 , Figure 30 and Figure 32 decoder Figure 34 (The receiving method, etc.) can receive and decode bit streams of partial point cloud data to reconstruct partial geometry and / or partial attributes.

[0503] The implementation includes a method for partitioning and transmitting compressed data according to specific criteria used for point cloud data. When using layered coding, compressed data can be partitioned and transmitted according to layers, which can improve the efficiency at the receiving end.

[0504] The geometry and attributes of point cloud data can be compressed to provide services. In PCC-based services, the compression rate or quantity of data can be adjusted for transmission based on receiver performance or transmission environment.

[0505] When configuring point cloud data on a slice-by-slice basis, receiver performance or transmission environments may change. In this case, it is necessary to either 1) pre-transcode the bitstream into a form suitable for each environment, store the bitstream separately, and select a portion for transmission, or 2) perform transcoding before transmission. In this case, if the number of receiver environments to be supported increases or the transmission environment changes frequently, it may cause storage space-related problems or latency caused by transcoding.

[0506] Figure 31 A method for sending and receiving point cloud data according to an embodiment is illustrated.

[0507] A point cloud data transmission method / apparatus according to an embodiment (which corresponds to) Figure 1Transmitting device 10000 Figure 1 Point cloud video encoder 10002 Figure 1 Transmitter 10003 Figure 2 Acquisition 20000 / Encoding 20001 / Transmission 20002 Figure 3 encoder, Figure 8 The transmitting device Figure 10 The device Figure 11 , Figure 20 , Figure 30 and Figure 32 encoder, Figure 33 (Sending methods, etc.) can be like Figure 30 The diagram shows scalable encoding and storage of the entire point cloud data in a storage device (storage can be skipped), and the option to select a bitstream for immediate transmission of a portion of the point cloud, which is consistent with... Figure 30 Different from China.

[0508] A point cloud data receiving method / apparatus according to an embodiment (which corresponds to) Figure 1 The receiving device 10004 Figure 1 Receiver 10005 Figure 1 Point cloud video decoder 10006 Figure 2 Transmission 20002 / Decoding 20003 / Rendering 20004 Figure 7 decoder Figure 9 The receiving device Figure 10 The device Figure 11 , Figure 20 , Figure 30 and Figure 32 decoder Figure 34 (e.g., receiving methods) can immediately receive and decode bitstreams of partial point cloud data to reconstruct partial geometry and / or partial attributes.

[0509] According to the implementation, when compressed data is partitioned and transmitted according to layers, only the necessary portions of the pre-compressed data can be selectively transmitted in the bitstream step without a separate transcoding operation. This is efficient even in terms of storage space, as each stream requires only one storage space. Furthermore, since only the necessary layers are selectively transmitted, efficient transmission can be performed even in terms of bandwidth (bitstream selector).

[0510] The point cloud data receiving method / apparatus according to embodiments of the present invention can provide the following effects:

[0511] Figure 32 A method for sending and receiving point cloud data according to an embodiment is illustrated.

[0512] A point cloud data transmission method / apparatus according to an embodiment (which corresponds to) Figure 1Transmitting device 10000 Figure 1 Point cloud video encoder 10002 Figure 1 Transmitter 10003 Figure 2 Acquisition 20000 / Encoding 20001 / Transmission 20002 Figure 3 encoder, Figure 8 The transmitting device Figure 10 The device Figure 11 , Figure 20 , Figure 30 and Figure 32 encoder, Figure 33 The transmission method (e.g., encoding the entire point cloud data in a scalable manner and storing it in a storage device (storage can be skipped), performing transcoding, and then transmitting the entire point cloud, such as... Figure 31 As shown.

[0513] A point cloud data receiving method / apparatus according to an embodiment (which corresponds to) Figure 1 The receiving device 10004 Figure 1 Receiver 10005 Figure 1 Point cloud video decoder 10006 Figure 2 Transmission 20002 / Decoding 20003 / Rendering 20004 Figure 7 decoder Figure 9 The receiving device Figure 10 The device Figure 11 , Figure 20 , Figure 30 and Figure 32 decoder Figure 34 (The receiving method, etc.) can receive the entire point cloud, decode it in a scalable manner, and select data by subsampling to perform methods for reconstructing the entire geometry / attributes or reconstructing a portion of the geometry / attributes.

[0514] In other words, when transmitting layered point cloud data, beneficial effects can be achieved on both the transmitting and receiving sides. In this case, if information enabling the reconstruction of the entire PCC data is transmitted regardless of the receiver's performance, the receiver needs to select the data corresponding to the desired layer after reconstructing the point cloud data through decoding (i.e., perform data selection or subsampling). In this situation, since the delivered bitstream has already been decoded, delays may occur in receivers targeting low latency, or decoding may not be performed depending on the receiver's performance.

[0515] According to the implementation, when the bitstream is divided into slices and delivered, the receiver can selectively deliver the bitstream to the decoder based on the decoder performance or the density of the point cloud data to be represented by the application domain. In this case, by performing the selection before decoding, decoder efficiency can be improved and decoders with various performance characteristics can be supported.

[0516] Figure 33 A method for sending point cloud data according to an implementation method is illustrated.

[0517] S3300: The point cloud data transmission method according to the embodiment may include encoding the point cloud data.

[0518] The encoding operation according to the implementation method may include Figure 1 The transmitting device 10000, the point cloud video acquirer 10001, and the point cloud video encoder 10002, Figure 2 Obtaining 20000 / encoding 20001 Figure 3 encoder, Figure 8 encoder, Figure 10 XR device 1030, Figure 11 Encoder 15000, Figure 20 Encoder 60000 and slice selector 60001, Figure 24 Parameter generation and geometry / attribute encoding, Figures 26 to 29 Bitstream and parameter generation, and Figures 30 to 32 Scalable encoders and bitstream selectors.

[0519] S3301: The point cloud data transmission method according to the embodiment may further include transmitting a bit stream containing point cloud data.

[0520] The sending operation according to the implementation method may include Figure 1 Transmitting device 10000, transmitter 10003, Figure 2 Transmission 20002, Figure 3 Bit stream transmission Figure 8 Bit stream transmission Figure 11 Partial or complete PCC bitstream transmission Figure 20 Layer-based bitstream transmission, and Figures 30 to 32 Sending part or all of the bit stream.

[0521] Figure 34 A method for receiving point cloud data according to an embodiment is illustrated.

[0522] S3400: The point cloud data receiving method according to the embodiment may include receiving a bit stream containing point cloud data.

[0523] The receiving operation according to the implementation method may include Figure 1 The receiving device 10004 and receiver 10005 Figure 2 Transmission 20002, Figure 7 and Figure 9 Bit stream reception Figure 10 XR device 1030, Figure 11 Reception of complete or partial bitstreams Figure 20 Layer-based bitstream reception Figure 25 Bitstream and parameter parsing Figures 26 to 29 Parameters and bitstream reception, and Figures 30 to 32 Receive the complete or partial bitstream.

[0524] S3401: The point cloud data receiving method according to the embodiment may further include decoding the point cloud data.

[0525] The decoding operation according to the implementation method may include Figure 1 The receiving device 10004, the point cloud video decoder 10006, and the receiver 10007 Figure 2 Decoding 20003 / Rendering 20004 Figure 7 and Figure 9 decoder Figure 10 XR device 1030, Figure 11 Decoder, scalable decoder, data selection, Figure 20 Decoder 60002, Figure 25 Parameter decoding and geometry / attribute decoding, Figures 26 to 29 Parameters and bitstream decoding, and Figures 30 to 32 The decoder, scalable decoder, and data selector / subsampler.

[0526] Reference Figure 1 The transmission method according to the implementation may include encoding point cloud data and transmitting a bit stream containing point cloud data.

[0527] Reference Figure 18 , Figure 25 and Figure 21 Regarding attributes, Level of Detail (LOD), layer groups (subgroups), nearest neighbor search, and intra-layer group search, the encoding of point cloud data can include encoding the attributes of the point cloud data. Attribute encoding can include generating layer groups based on the level of detail (LOD) for the attribute, and searching for neighbor candidates for the attribute based on the neighbor nodes of the subgroups within the layer group.

[0528] Reference Figure 22 Regarding inter-layer group search, attribute encoding can include searching for neighbor candidates for an attribute based on the neighbor nodes of the parent and child groups of the subgroups within the layer group.

[0529] Reference Figure 23 Regarding the adaptation position of layer groups, nodes within the bounding box of subgroups, and nodes outside the bounding box of subgroups, the geometric position of neighbor candidates can be considered based on the subgroup if the neighbor candidate is within the subgroup. If the neighbor candidate is outside the subgroup, the geometric position of the neighbor candidate can be considered based on the parent-child group of the subgroup.

[0530] Reference Figure 23 Regarding atlas shifting, the encoding of attributes may also include aligning the starting position of nodes included in a subgroup with the starting position of nodes included in the parent-child group of the subgroup, where atlas information can be shifted to attributes.

[0531] Reference Figure 22 Regarding the derivation of subgroup weights, the encoding of attributes may also include performing predictive transformations on attributes based on quantized weights, where the quantized weights can be generated based on whether the current node is referenced as a neighboring node.

[0532] Reference Figure 22 Regarding the weight generation method ((weight(m)+=alpha*numRefNodes(m) / maxNumRefNodes+beta)), the quantized weight of node m can be generated based on the number of nodes that refer to the current node as a neighbor node (numRefNodes(m)), the maximum number of related nodes in the current subgroup (maxNumRefNodes), and the coefficients (alpha and beta) related to the weight.

[0533] Reference Figures 27 to 29 Regarding the subgroup weight adjustment coefficients (subgroup_weight_adj_coeff_a and subgroup_weight_adj_coeff_b), the bitstream may contain coefficient information related to the weights for subgroups that include point cloud data.

[0534] The point cloud data transmission method can be performed by a transmitting device. The transmitting device may include an encoder configured to encode point cloud data, and a transmitter configured to transmit the point cloud data. The transmitting device may include a memory storing encoding instructions and a processor configured to perform encoding operations.

[0535] The method for receiving point cloud data can be executed in the reverse order of the method for sending point cloud data. (See reference...) Figure 1 The receiving method may include receiving a bit stream containing point cloud data and decoding the point cloud data.

[0536] Decoding point cloud data can include decoding the attributes of the point cloud data. Attribute decoding can include generating a layer group based on the LOD for the attribute, and searching for neighbor candidates for the attribute based on the neighbor nodes of the subgroups in the layer group.

[0537] Decoding an attribute can include searching for neighbor candidates for the attribute based on the neighbor nodes of the parent and child groups of the subgroups in the layer group.

[0538] Decoding attributes may also include aligning the starting position of nodes included in a subgroup with the starting position of nodes included in the parent-child groups of the subgroup, where atlas information can be shifted to attributes.

[0539] The point cloud data receiving method can be performed by a receiving device. The receiving device may include a receiver configured to receive a bit stream containing point cloud data and a decoder configured to decode the point cloud data.

[0540] Therefore, point cloud data can be compressed and reconstructed with high compression efficiency. Furthermore, spatial random access and scalable coding can be achieved.

[0541] Implementations have been described in terms of methods and / or apparatus, and the descriptions of methods and apparatus may be applied complementaryly to each other.

[0542] Although the accompanying drawings are described separately for simplicity, new embodiments can be designed by combining the embodiments shown in the corresponding figures. Recording media containing programs for executing the above embodiments, designed to be computer-readable by those skilled in the art, also fall within the scope of the appended claims and their equivalents. The apparatus and methods according to the embodiments are not limited to the configurations and methods of the above embodiments. Various modifications can be made to the embodiments by selectively combining all or some of the embodiments. Although preferred embodiments have been described with reference to the accompanying drawings, those skilled in the art will understand that various modifications and variations can be made to the embodiments without departing from the spirit or scope of this disclosure as described in the appended claims. These modifications should not be understood solely from the technical concept or viewpoint of the embodiments.

[0543] Various elements of the apparatus according to the embodiments can be implemented by hardware, software, firmware, or a combination thereof. Various elements in the embodiments can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiments, the components according to the embodiments can be implemented as separate chips. According to the embodiments, at least one or more components of the apparatus according to the embodiments can include one or more processors capable of executing one or more programs. One or more programs can perform any one or more operations / methods according to the embodiments or include instructions for performing them. Executable instructions for performing the methods / operations of the apparatus according to the embodiments can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transient CRM or other computer program product configured to be executed by one or more processors. Additionally, the memory according to the embodiments can be used as a concept encompassing not only volatile memory (e.g., RAM) but also non-volatile memory, flash memory, and PROM. It can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, the processor-readable recording medium can be distributed to computer systems connected via a network, such that processor-readable code can be stored and executed in a distributed manner.

[0544] In this disclosure, " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" can mean "A and / or B". Furthermore, "A, B" can mean "A and / or B". Additionally, "A / B / C" can mean "at least one of A, B, and / or C". Also, "A / B / C" can mean "at least one of A, B, and / or C". Furthermore, in this specification, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" can mean 1) only A, 2) only B, or 3) both A and B. In other words, the term "or" as used in this document should be interpreted as indicating "additionally or alternatively".

[0545] Terms such as "first" and "second" can be used to describe various elements of the embodiments. However, the various components according to the embodiments should not be limited by the terms used above. These terms are used only to distinguish one element from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should not be interpreted outside the scope of the various embodiments. Both the first user input signal and the second user input signal are user input signals, but they do not mean the same user input signal unless the context clearly specifies otherwise.

[0546] The terminology used to describe embodiments is for the purpose of describing particular embodiments and is not intended to limit the embodiments. As used in the description of embodiments and claims, the singular form includes the plural of the referred objects unless the context clearly specifies otherwise. The expression “and / or” is used to include all possible combinations of terms. Terms such as “comprising” or “having” are intended to indicate the presence of figures, quantities, steps, elements, and / or components and should be understood not to exclude the possibility of additional figures, quantities, steps, elements, and / or components. As used herein, conditional expressions such as “if” and “when” are not limited to optional cases and are intended to perform related operations or interpret related definitions based on specific conditions when those conditions are met.

[0547] The operations described in this specification according to embodiments can be performed by a transmitting / receiving device including a memory and / or a processor according to embodiments. The memory may store a program for processing / controlling operations according to embodiments, and the processor may control the various operations described in this specification. The processor may be referred to as a controller, etc. In embodiments, operations may be performed by firmware, software, and / or combinations thereof. Firmware, software, and / or combinations thereof may be stored in a processor or memory.

[0548] The operations according to the above embodiments can be performed by the transmitting and / or receiving devices according to the embodiments. The transmitting / receiving devices may include a transmitter / receiver configured to transmit and receive media data, a memory configured to store instructions (program code, algorithms, flowcharts, and / or data) for the processes according to the embodiments, and a processor configured to control the operation of the transmitting / receiving devices.

[0549] The processor may be referred to as a controller, etc., and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above embodiments can be performed by the processor. Alternatively, the processor may be implemented as an encoder / decoder for the operations of the above embodiments.

[0550] Invention Model

[0551] As described above, the relevant details are presented in the best mode for implementing the method.

[0552] Industrial applicability

[0553] As described above, the implementation methods can be applied in whole or in part to point cloud data transmission / reception devices and systems.

[0554] Within the scope of the implementation method, technicians may change or modify the implementation method in various ways.

[0555] Implementation may include variations / modifications within the scope of the claims and their equivalents.

Claims

1. A method for sending point cloud data, the method comprising: Encode point cloud data; as well as A bitstream containing the point cloud data is sent.

2. The method according to claim 1, wherein: The encoding of the point cloud data includes: Encoding the attributes of the point cloud data, The encoding of the attribute includes: generating a layer group based on the level of detail (LOD) for the attribute; and A neighbor candidate of the attribute is searched based on neighbor nodes of the subgroup in the layer group.

3. The method according to claim 2, wherein: The encoding of the attribute also includes: The neighbor candidates of the attribute are searched based on neighbor nodes of a parent subgroup of the subgroup in the layer group.

4. The method according to claim 3, wherein: Based on the neighbor candidate being in the subgroup, considering the geometric position of the neighbor candidate based on the subgroup; and Based on the neighbor candidate being outside the sub-group, the geometric position of the neighbor candidate is considered based on the parent sub-group of the sub-group.

5. The method according to claim 4, wherein: The encoding of the attribute also includes: aligning the start position of the node included in the child group with the start position of the node included in the parent child group of the child group, Therein, the atlas information is shifted to the attribute.

6. The method according to claim 4, wherein: The encoding of the attribute also includes: performing a predictive transformation on the attributes based on the quantized weights, The quantization weight is generated based on whether the current node is referenced as a neighbor node.

7. The method according to claim 6, wherein: The quantized weight is generated based on the number of nodes that refer to the current node as the neighbor node, the maximum number of related nodes in the current subgroup, and a coefficient associated with the weight.

8. The method according to claim 1, wherein: The bitstream contains coefficient information associated with the weights for a subset comprising the point cloud data.

9. A device for transmitting point cloud data, the device comprising: an encoder configured to encode point cloud data; as well as A transmitter is configured to transmit the point cloud data.

10. A method for receiving point cloud data, the method comprising: receiving a bitstream containing point cloud data; as well as The point cloud data is decoded.

11. The method according to claim 10, wherein: The decoding of the point cloud data comprises: Decoding the attributes of the point cloud data, Wherein, the decoding of the attribute comprises: generating a layer group based on the level of detail (LOD) for the attribute; and A neighbor candidate of the attribute is searched based on neighbor nodes of the subgroup in the layer group.

12. The method according to claim 11, wherein: The decoding of the attribute further comprises: The neighbor candidates of the attribute are searched based on neighbor nodes of a parent subgroup of the subgroup in the layer group.

13. The method according to claim 12, wherein: Based on the neighbor candidate being in the subgroup, considering the geometric position of the neighbor candidate based on the subgroup; and Based on the neighbor candidate being outside the sub-group, the geometric position of the neighbor candidate is considered based on the parent sub-group of the sub-group.

14. The method according to claim 13, wherein: The decoding of the attribute further comprises: aligning the start position of the node included in the child group with the start position of the node included in the parent child group of the child group, Therein, the atlas information is shifted to the attribute.

15. A device for receiving point cloud data, the device comprising: a receiver configured to receive a bitstream containing point cloud data; as well as A decoder is configured to decode the point cloud data.

Images