Point cloud data transmitting device, point cloud data transmitting method, point cloud data receiving device, and point cloud data receiving method

Abstract

A point cloud data transmission method according to an embodiment includes the following steps: encoding point cloud data, the point cloud data including geometric data and attribute data; and transmitting a bit stream including the point cloud data. A point cloud data reception method according to an embodiment includes the following steps: receiving a bit stream including point cloud data, the point cloud data including geometric data and attribute data; and decoding the point cloud data.

Description

Technical Field

[0001] The embodiments relate to a method and apparatus for processing point cloud content. Background Technology

[0002] Point cloud content is content represented by a point cloud, which is a collection of points belonging to a coordinate system representing three-dimensional space. Point cloud content can represent media configured in three dimensions and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and autonomous driving services. However, tens of thousands to hundreds of thousands of point data points are required to represent point cloud content. Therefore, a method for efficiently processing large amounts of point data is needed. Summary of the Invention

[0003] Technical issues

[0004] The embodiments provide an apparatus and method for efficiently processing point cloud data. The embodiments also provide a point cloud data processing method and apparatus for addressing latency and encoding / decoding complexity.

[0005] The technical scope of the embodiments is not limited to the foregoing technical objectives, but can be extended to other technical objectives that can be inferred by those skilled in the art based on the entire content disclosed herein.

[0006] Technical solution

[0007] To achieve these objectives and other advantages, and in accordance with the purposes of this disclosure, in some embodiments, a method for transmitting point cloud data may include: encoding point cloud data including geometric information and attribute information, and transmitting a bit stream including the encoded point cloud data.

[0008] In some embodiments, a method for receiving point cloud data may include: receiving a bit stream including point cloud data, and decoding the point cloud data including geometric information and attribute information.

[0009] Beneficial effects

[0010] The device and method according to the embodiments can efficiently process point cloud data.

[0011] The device and method according to the embodiments can provide high-quality point cloud services.

[0012] The devices and methods according to the embodiments can provide point cloud content to offer general services such as VR services and autonomous driving services. Attached Figure Description

[0013] The accompanying drawings are included to provide a further understanding of this disclosure and are incorporated in and constitute a part of this application. The drawings illustrate embodiments of the disclosure and, together with the description, serve to illustrate the principles of the disclosure. For a better understanding of the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the accompanying drawings. In the drawings:

[0014] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown.

[0015] Figure 2 This is a block diagram illustrating the operation of providing point cloud content according to an embodiment.

[0016] Figure 3 The illustration shows an exemplary processing of captured point cloud video according to an embodiment.

[0017] Figure 4 The figure illustrates an exemplary point cloud encoder according to an embodiment.

[0018] Figure 5 An example of a voxel according to an embodiment is shown.

[0019] Figure 6 An example of an octree and occupancy code according to an embodiment is shown.

[0020] Figure 7 An example of a neighbor node pattern according to an embodiment is shown.

[0021] Figure 8 The illustration shows an example of point configuration in each LOD according to an embodiment.

[0022] Figure 9 The illustration shows an example of point configuration in each LOD according to an embodiment.

[0023] Figure 10 The illustration shows an exemplary point cloud decoder according to an embodiment.

[0024] Figure 11 The illustration shows an exemplary point cloud decoder according to an embodiment.

[0025] Figure 12 The illustration shows an exemplary transmitting device according to an embodiment.

[0026] Figure 13 The illustration shows an exemplary receiving device according to an embodiment.

[0027] Figure 14 An exemplary structure is shown that is operatively connected to a method / device for sending and receiving point cloud data according to an embodiment.

[0028] Figure 15 This is a flowchart illustrating a point cloud data encoder and a point cloud data decoder according to an embodiment.

[0029] Figure 16 The configuration of segments 16020 to 16050 of the bitstream according to an embodiment is shown.

[0030] Figure 17 The diagram illustrates bitstream segment matching based on a PCC layer structure according to an embodiment.

[0031] Figure 18 The illustration shows the decoding of bitstream segments and layer matching of the PCC structure according to an embodiment.

[0032] Figure 19 A PCC NAL (Network Abstraction Layer) unit according to an embodiment is shown.

[0033] Figure 20 The header of the NAL unit according to an embodiment is shown.

[0034] Figure 21 The type of NAL unit according to an embodiment is shown.

[0035] Figure 22 The set of sequence parameters according to an embodiment is shown.

[0036] Figure 23 The set of parameters (geometry_parameter_set) according to an embodiment is shown.

[0037] Figure 24 The attribute parameter set (attribute_parameter_set()) according to an embodiment is shown.

[0038] Figure 25 The figure shows the encoder of a point cloud data transmission device according to an embodiment.

[0039] Figure 26 The diagram illustrates a decoder for a point cloud data receiving device according to an embodiment.

[0040] Figure 27 The diagram illustrates the process of sending and receiving point cloud data according to an embodiment.

[0041] Figure 28 The diagram illustrates the process of sending and receiving point cloud data according to an embodiment.

[0042] Figure 29 The diagram illustrates a method for transmitting point cloud data according to an embodiment; and

[0043] Figure 30 The illustration shows a method for receiving point cloud data according to an embodiment. Detailed Implementation

[0044] Preferred embodiments of the present disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. The detailed description given below with reference to the accompanying drawings is intended to explain exemplary embodiments of the present disclosure and not to illustrate only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0045] Although most of the terms used in this disclosure have been selected from commonly used terms in the art, some terms have been arbitrarily chosen by the applicant and their meanings are explained in detail in the following description as needed. Therefore, this disclosure should be understood based on the intended meaning of the terms rather than their simple names or meanings.

[0046] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown.

[0047] Figure 1 The point cloud content providing system illustrated may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 are capable of transmitting and receiving point cloud data via wired or wireless communication.

[0048] The point cloud data transmitting device 10000 according to an embodiment can securely acquire and process point cloud video (or point cloud content) and transmit it. According to an embodiment, the transmitting device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device and / or a server. According to an embodiment, the transmitting device 10000 may include devices configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers.

[0049] According to an embodiment, the transmitting device 10000 includes a point cloud video acquirer 10001, a point cloud video encoder 10002, and / or a transmitter (or communication module) 10003.

[0050] The point cloud video acquirer 10001 according to an embodiment acquires point cloud video through processing procedures such as capture, synthesis, or generation. Point cloud video is point cloud content represented by a point cloud, which is a collection of points located in 3D space, and may be referred to as point cloud video data. The point cloud video according to an embodiment may include one or more frames. A frame represents a still image / picture. Therefore, point cloud video may include point cloud images / frames / pictures, and may be referred to as point cloud images, frames, or pictures.

[0051] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 can encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to an embodiment may include geometry-based point cloud compression (G-PCC) coding and / or video-based point cloud compression (V-PCC) coding or next-generation coding. The point cloud compression coding according to an embodiment is not limited to the above embodiments. The point cloud video encoder 10002 can output a bitstream containing encoded point cloud video data. The bitstream may contain not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0052] According to an embodiment, transmitter 10003 transmits a bitstream containing encoded point cloud video data. The bitstream, according to an embodiment, is encapsulated in a file or segment (e.g., a streaming segment) and transmitted over various networks such as broadcast networks and / or broadband networks. Although not shown in the figures, transmitting device 10000 may include an encapsulator (or encapsulation module) configured to perform encapsulation operations. According to an embodiment, the encapsulator may be included in transmitter 10003. According to an embodiment, the file or segment can be transmitted over a network to receiving device 10004 or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). Transmitter 10003, according to an embodiment, is capable of wired / wireless communication with receiving device 10004 (or receiver 10005) via networks such as 4G, 5G, and 6G. Additionally, the transmitter can perform necessary data processing operations depending on the network system (e.g., a 4G, 5G, or 6G communication network system). Transmitting device 10000 can transmit encapsulated data on demand.

[0053] According to an embodiment, the receiving device 10004 includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include devices, robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)).

[0054] According to an embodiment, receiver 10005 receives a bitstream containing point cloud video data or a file / segment encapsulating the bitstream from a network or storage medium. Receiver 10005 can perform necessary data processing according to the network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). According to an embodiment, receiver 10005 can decapsulate the received file / segment and output the bitstream. According to an embodiment, receiver 10005 may include a decapsulator (or decapsulator module) configured to perform a decapsulation operation. The decapsulator may be implemented as a separate element (or component) from receiver 10005.

[0055] The point cloud video decoder 10006 decodes the bitstream containing point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to the method in which the point cloud video data is encoded (e.g., the inverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding, which is the inverse process of point cloud compression. Point cloud decompression encoding includes G-PCC encoding.

[0056] Renderer 10007 renders decoded point cloud video data. Renderer 10007 can output point cloud content by rendering not only the point cloud video data but also the audio data. According to an embodiment, renderer 10007 may include a display configured to display the point cloud content. According to an embodiment, the display may be implemented as a separate device or component rather than included in renderer 10007.

[0057] The arrows indicated by dashed lines in the diagram represent the transmission path of the feedback information acquired by receiving device 10004. The feedback information reflects the interactivity of a user consuming the point cloud content and includes information about the user (e.g., head orientation information, viewport information, etc.). Specifically, when the point cloud content is for a service requiring user interaction (e.g., autonomous driving services, etc.), the feedback information can be provided to the content sender (e.g., sending device 10000) and / or the service provider. According to embodiments, the feedback information may be used in both receiving device 10004 and sending device 10000, or it may not be provided.

[0058] According to an embodiment, head orientation information is information about the user's head position, orientation, angle, movement, etc. According to an embodiment, receiving device 10004 can calculate viewport information based on the head orientation information. Viewport information can be information about the area of ​​the point cloud video the user is viewing. The viewpoint is the point through which the user views the point cloud video and can refer to the center point of the viewport area. That is, the viewport is the area centered on the viewpoint, and the size and shape of the area can be determined by the field of view (FOV). Therefore, in addition to head orientation information, receiving device 10004 can also extract viewport information based on the vertical or horizontal FOV supported by the device. Furthermore, receiving device 10004 performs gaze analysis, etc., to examine the way the user consumes the point cloud, the area the user gazes at in the point cloud video, the gaze duration, etc. According to an embodiment, receiving device 10004 can send feedback information including the gaze analysis results to transmitting device 10000. According to an embodiment, the feedback information can be acquired during rendering and / or display. According to an embodiment, the feedback information can be acquired by one or more sensors included in receiving device 10004. According to an embodiment, feedback information can be obtained by the renderer 10007 or by a separate external component (or device, component, etc.). Figure 1 The dashed lines in the diagram represent the process of sending feedback information obtained by the renderer 10007. The point cloud content providing system can process (encode / decode) point cloud data based on the feedback information. Therefore, the point cloud video data decoder 10006 can perform decoding operations based on the feedback information. The receiving device 10004 can send the feedback information to the sending device 10000. The sending device 10000 (or the point cloud video data encoder 10002) can perform encoding operations based on the feedback information. Therefore, the point cloud content providing system can effectively process necessary data (e.g., point cloud data corresponding to the user's head position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide the point cloud content to the user.

[0059] According to the embodiment, the transmitting device 10000 may be referred to as an encoder, transmitting device, transmitter, etc., and the receiving device 10004 may be referred to as a decoder, receiving device, receiver, etc.

[0060] According to the embodiments Figure 1 Point cloud data processed in a point cloud content provision system (through a series of processes including acquisition, encoding, transmission, decoding, and rendering) can be referred to as point cloud content data or point cloud video data. According to embodiments, point cloud content data can be used as a concept encompassing metadata or signaling information related to point cloud data.

[0061] Figure 1 The components of the point cloud content providing system illustrated can be implemented by hardware, software, processors, and / or combinations thereof.

[0062] Figure 2 This is a block diagram illustrating the operation of providing point cloud content according to an embodiment.

[0063] Figure 2 The block diagram shows Figure 1 The operation of the point cloud content providing system described herein. As mentioned above, the point cloud content providing system can process point cloud data based on point cloud compression encoding (e.g., G-PCC).

[0064] A point cloud content providing system (e.g., point cloud sending device 10000 or point cloud video acquirer 10001) according to an embodiment can acquire point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system used to represent 3D space. The point cloud video according to an embodiment may include Ply (Polygon file format or Stanford Triangle format) files. When the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of the points. The position of each point may be represented by parameters (e.g., values ​​of the X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include the attributes of the points (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc., of each point). A point has one or more attributes. For example, a point may have an attribute as color or two attributes: color and reflectivity. According to embodiments, geometry can be referred to as location, geometric information, geometric data, etc., and attributes can be referred to as attributes, attribute information, attribute data, etc. A point cloud content providing system (e.g., point cloud transmitting device 10000 or point cloud video acquirer 10001) can obtain point cloud data from information related to the point cloud video acquisition process (e.g., depth information, color information, etc.).

[0065] A point cloud content providing system (e.g., transmitting device 10000 or point cloud video encoder 10002) according to an embodiment can encode point cloud data (20001). The point cloud content providing system can encode point cloud data based on point cloud compression encoding. As described above, point cloud data can include the geometry and attributes of points. Therefore, the point cloud content providing system can perform geometry encoding to encode the geometry and output a geometry bitstream. The point cloud content providing system can perform attribute encoding to encode the attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system can perform attribute encoding based on geometry encoding. The geometry bitstream and attribute bitstream according to an embodiment can be multiplexed and output as a single bitstream. The bitstream according to an embodiment may also contain signaling information related to geometry encoding and attribute encoding.

[0066] A point cloud content providing system according to an embodiment (e.g., transmitting device 10000 or transmitter 10003) can transmit encoded point cloud data (20002). For example... Figure 1 As illustrated, encoded point cloud data can be represented by geometric bitstreams and attribute bitstreams. Additionally, the encoded point cloud data can be transmitted as a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometric and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and transmit it as a file or fragment.

[0067] The point cloud content providing system (e.g., receiving device 10004 or receiver 10005) according to an embodiment can receive a bitstream containing encoded point cloud data. Additionally, the point cloud content providing system (e.g., receiving device 10004 or receiver 10005) can demultiplex the bitstream.

[0068] A point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode encoded point cloud data (e.g., geometric bitstream, attribute bitstream) transmitted in a bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode point cloud video data based on signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode the geometric bitstream to reconstruct the location (geometry) of the points. The point cloud content providing system can reconstruct the attributes of the points by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can reconstruct point cloud video based on location according to the reconstructed geometry and the decoded attributes.

[0069] A point cloud content providing system (e.g., receiving device 10004 or renderer 10007) according to an embodiment can render decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) can use various rendering methods to render the geometry and attributes decoded through the decoding process. Points in the point cloud content can be rendered as vertices with a certain thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to the user through a display (e.g., a VR / AR display, a general display, etc.).

[0070] The point cloud content providing system (e.g., receiving device 10004) according to an embodiment can obtain feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. The feedback information and operation / reference of the point cloud content providing system according to an embodiment... Figure 1 The feedback information and operations described are the same, and therefore their detailed descriptions are omitted.

[0071] Figure 3 The illustration shows an exemplary process for capturing point cloud video according to an embodiment.

[0072] Figure 3 Illustrated reference Figures 1 to 2 The described point cloud content provides an exemplary point cloud video capture process for the system.

[0073] Point cloud content includes point cloud videos (images and / or videos) representing objects and / or environments located in various 3D spaces (e.g., 3D spaces representing real environments, 3D spaces representing virtual environments, etc.). Therefore, the point cloud content providing system according to embodiments can use one or more cameras (e.g., infrared cameras capable of acquiring depth information, RGB cameras capable of extracting color information corresponding to the depth information, etc.), projectors (e.g., infrared pattern projectors acquiring depth information), LiDAR, etc., to capture point cloud videos. The point cloud content providing system according to embodiments can extract the geometry composed of points in 3D space from the depth information and extract the attributes of each point from the color information to obtain point cloud data. Images and / or videos according to embodiments can be captured based on at least one of inward-facing and outward-facing techniques.

[0074] Figure 3 The left side of the diagram illustrates inward-facing technology. Inward-facing technology refers to the technique of capturing images of a central object by positioning one or more cameras (or camera sensors) around it. Inward-facing technology can be used to generate point cloud content that provides users with 360-degree images of key objects (e.g., VR / AR content that provides users with 360-degree images of objects such as characters, players, objects, or actors).

[0075] Figure 3 The right side of the diagram illustrates outward-facing technology. Outward-facing technology refers to the technique of capturing images of the environment of a central object, rather than the central object itself, using one or more cameras (or camera sensors) positioned around it. Outward-facing technology can be used to generate point cloud content that provides a representation of the surrounding environment from the user's perspective (e.g., content representing the external environment that can be provided to users of self-driving vehicles).

[0076] As shown in the figure, point cloud content can be generated based on the capture operations of one or more cameras. In this case, the coordinate systems between the cameras can be different, and therefore the point cloud content providing system can calibrate one or more cameras to set the global coordinate system before the capture operation. Additionally, the point cloud content providing system can generate point cloud content by compositing arbitrary images and / or videos with images and / or videos captured using the aforementioned capture techniques. The point cloud content providing system may not perform [specific steps] when generating point cloud content representing virtual space. Figure 3 The capture operations described herein. The point cloud content providing system according to an embodiment can perform post-processing on the captured images and / or videos. In other words, the point cloud content providing system can remove unwanted areas (e.g., background), identify spaces to which the captured images and / or videos are connected, and perform a space-filling operation when spatial holes exist.

[0077] A point cloud content delivery system can generate point cloud content by performing coordinate transformations on points in point cloud video captured from each camera. The system can perform these transformations based on the position coordinates of each camera. Therefore, the system can generate content representing a wide range or point cloud content with high point density.

[0078] Figure 4 The figure illustrates an exemplary point cloud encoder according to an embodiment.

[0079] Figure 4 Show Figure 1 An example of a point cloud video encoder 10002. The point cloud encoder reconstructs and encodes point cloud data (e.g., point locations and / or attributes) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) based on network conditions or applications. When the total size of the point cloud content is large (e.g., 60Gbps of point cloud content for 30fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system can reconstruct the point cloud content based on a maximum target bitrate to provide the point cloud content according to network conditions, etc.

[0080] For reference Figure 1 and Figure 2 The point cloud encoder described herein can perform geometric encoding and attribute encoding. Geometric encoding is performed before attribute encoding.

[0081] The point cloud encoder according to the embodiment includes a coordinate transformer (transform coordinates) 40000, a quantizer (quantize and remove points (voxarization)) 40001, an octree analyzer (analyze octrees) 40002, a surface approximation analyzer (analyze surface approximations) 40003, an arithmetic encoder (arithmetic encoding) 40004, a geometry reconstructor (reconstruct geometry) 40005, a color transformer (transform colors) 40006, an attribute transformer (transform attributes) 40007, a RAHT transformer (RAHT) 40008, an LOD generator (generate LODs) 40009, a lift transformer (lift) 40010, a coefficient quantizer (quantize coefficients) 40011, and / or an arithmetic encoder (arithmetic encoding) 40012.

[0082] Coordinate transformer 40000, quantizer 40001, octree analyzer 40002, surface approximation analyzer 40003, arithmetic encoder 40004, and geometric reconstructor 40005 can perform geometric encoding. Geometric encoding according to embodiments may include octree geometric encoding, direct encoding, trisoup geometric encoding, and entropy encoding. Direct encoding and trisoup geometric encoding are applied selectively or in combination. Geometric encoding is not limited to the examples described above.

[0083] As shown in the figure, the coordinate transformer 40000 according to an embodiment receives a position and transforms it into coordinates. For example, the position can be transformed into position information in three-dimensional space (e.g., three-dimensional space represented by the XYZ coordinate system). The position information in three-dimensional space according to the embodiment can be referred to as geometric information.

[0084] According to an embodiment, quantizer 40001 performs geometric quantization. For example, quantizer 40001 may quantize points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). Quantizer 40001 performs a quantization operation: multiplying the difference between the minimum position value and the position value of each point by a preset quantization scaling value, and then finding the nearest integer value by rounding the value obtained through multiplication. Thus, one or more points may have the same quantized position (or position value). According to an embodiment, quantizer 40001 performs voxelization based on the quantized position to reconstruct the quantized points. As in the case of a pixel, which is the smallest unit containing 2D image / video information, points in point cloud content (or 3D point cloud video) according to an embodiment may be included in one or more voxels. The term voxel, as a composite of volume and pixel, refers to the 3D cubic space generated when 3D space is divided into units (unit = 1.0) based on axes representing 3D space (e.g., X-axis, Y-axis, and Z-axis). The quantizer 40001 can match a group of points in 3D space to voxels. According to an embodiment, a voxel may include only one point. According to an embodiment, a voxel may include one or more points. To represent a voxel as a point, the location of the voxel's center can be set based on the locations of one or more points included in the voxel. In this case, attributes included in all locations within a voxel can be combined and assigned to the voxel.

[0085] According to the embodiment, the octree analyzer 40002 performs octree geometric encoding (or octree coding) to represent voxels in an octree structure. The octree structure represents points based on the matching of octree structures with voxels.

[0086] The surface approximation analyzer 40003 according to the embodiment can analyze and approximate an octree. The octree analysis and approximation according to the embodiment is a process of analyzing a region containing multiple points to efficiently provide an octree and voxelization.

[0087] According to an embodiment, the arithmetic encoder 40004 performs entropy encoding on octrees and / or approximate octrees. For example, the encoding scheme includes arithmetic encoding. As a result of the encoding, a geometric bitstream is generated.

[0088] The attribute encoding is performed by a color transformer 40006, an attribute transformer 40007, a RAHT transformer 40008, an LOD generator 40009, a boosting transformer 40010, a coefficient quantizer 40011, and / or an arithmetic encoder 40012. As described above, a point can have one or more attributes. The attribute encoding according to the embodiments is also applied to the attributes that a point has. However, when an attribute (e.g., color) includes one or more elements, attribute encoding is applied independently to each element. The attribute encoding according to the embodiments includes color transformation encoding, attribute transformation encoding, region adaptive hierarchical transformation (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) encoding, and interpolation-based hierarchical nearest neighbor prediction encoding with an update / boosting step (boosting transformation). Depending on the point cloud content, the above-described RAHT encoding, prediction transformation encoding, and boosting transformation encoding can be used selectively, or a combination of one or more encoding schemes can be used. The attribute encoding according to the embodiments is not limited to the examples described above.

[0089] The color converter 40006 according to the embodiment performs color transformation encoding that transforms the color values ​​(or textures) included in the attributes. For example, the color converter 40006 can transform the format of color information (e.g., from RGB to YCbCr). The operation of the color converter 40006 according to the embodiment can be optionally applied based on the color values ​​included in the attributes.

[0090] According to the embodiment, the geometry reconstructor 40005 reconstructs (decompresses) octrees and / or approximate octrees. The geometry reconstructor 40005 reconstructs the octree / voxel based on the results of analyzing the distribution of points. The reconstructed octree / voxel can be referred to as the reconstructed geometry (recovered geometry).

[0091] According to the embodiment, the attribute transformer 40007 performs attribute transformation to transform attributes based on reconstructed geometry and / or locations where geometric encoding is not performed. As described above, because attributes depend on geometry, the attribute transformer 40007 can transform attributes based on reconstructed geometric information. For example, based on the position value of a point included in a voxel, the attribute transformer 40007 can transform the attributes of the point at that position. As described above, when the center position of a voxel is set based on the positions of one or more points included in the voxel, the attribute transformer 40007 transforms the attributes of one or more points. When performing triadic geometric encoding, the attribute transformer 40007 can transform attributes based on triadic geometric encoding.

[0092] The attribute transformer 40007 performs attribute transformation by calculating the average of the attributes or attribute values ​​(e.g., color or reflectivity of each point) of neighboring points within a specific location / radius from the center of each voxel. The attribute transformer 40007 can apply weights based on the distance from the center to each point when calculating the average. Therefore, each voxel has a location and a calculated attribute (or attribute value).

[0093] The attribute transformer 40007 can search for nearest neighbors within a specific location / radius of the center of each voxel based on a KD-tree or Morton code. A KD-tree is a binary search tree and supports a data structure that allows points to be managed based on location, enabling fast nearest neighbor search (NNS). Morton codes are generated by representing the coordinates (e.g., (x, y, z)) of the 3D location of all points as bit values ​​and mixing the bits. For example, when the coordinates representing the point location are (5, 9, 1), the bit values ​​are (0101, 1001, 0001). Mixing the bit values ​​according to the bit index in the order of z, y, and x produces 010001000111. This value is expressed as the decimal number 1095. That is, the Morton code value for the point with coordinates (5, 9, 1) is 1095. The attribute transformer 40007 can sort the points based on their Morton code values ​​and perform NNS using a depth-first traversal process. After an attribute transformation operation, if an NNS is needed in another transformation process used for attribute encoding, use a KD tree or Morton code.

[0094] As shown in the figure, the transformation properties are input to the RAHT transformer 40008 and / or the LOD generator 40009.

[0095] According to an embodiment, the RAHT transformer 40008 performs RAHT encoding for predicting attribute information based on the reconstructed geometric information. For example, the RAHT transformer 40008 can predict the attribute information of higher-level nodes in an octree based on the attribute information associated with lower-level nodes in the octree.

[0096] The LOD generator 40009 according to the embodiment generates a Level of Detail (LOD) to perform predictive transform coding. The LOD according to the embodiment represents the level of detail of the point cloud content. As the LOD value decreases, it indicates a deterioration in the detail of the point cloud content. As the LOD value increases, it indicates an enhancement in the detail of the point cloud content. Points can be classified by LOD.

[0097] The lift transformer 40010 according to the embodiment performs lift transform coding to transform point cloud attributes based on weights. As described above, lift transform coding may optionally be applied.

[0098] According to the embodiment, the coefficient quantizer 40011 quantizes the attribute encoded by the attribute based on the coefficient.

[0099] According to the embodiment, the arithmetic encoder 40012 encodes quantized attributes based on arithmetic encoding.

[0100] Although not shown in the figure, Figure 4 The elements of the point cloud encoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above... Figure 4 At least one of the operations and / or functions of the elements of the point cloud encoder. Additionally, one or more processors can operate or perform operations for executing... Figure 4 The software program and / or instructions for the operation and / or function of the elements of the point cloud encoder. One or more memories according to the embodiments may include high-speed random access memory, or may include non-volatile memory (e.g., one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices).

[0101] Figure 5 An example of a voxel according to an embodiment is shown.

[0102] Figure 5 This shows a voxel positioned in 3D space, represented by a coordinate system consisting of three axes: X, Y, and Z. (See reference...) Figure 4 The point cloud encoder (e.g., quantizer 40001) can perform voxelization. A voxel refers to the 3D cubic space generated when the 3D space is divided into cells (unit = 1.0) based on axes representing the 3D space (e.g., X-axis, Y-axis, and Z-axis). Figure 5 An example of voxels generated via an octree structure is shown, where a cubic axis-aligned bounding box defined by two poles (0,0,0) and (2d,2d,2d) is recursively subdivided. A voxel comprises at least one point. The spatial coordinates of a voxel can be estimated from its positional relationship to a group of voxels. As mentioned above, voxels possess properties similar to pixels in a 2D image / video, such as color or reflectivity. Details and references of voxels are provided. Figure 4 The descriptions are the same, and therefore their descriptions are omitted.

[0103] Figure 6 An example of an octree and occupancy code according to an embodiment is shown.

[0104] For reference Figures 1 to 4As described, the point cloud content delivery system (point cloud video encoder 10002) or point cloud encoder (e.g., octree analyzer 40002) performs octree geometric encoding (or octree encoding) based on an octree structure to efficiently manage the regions and / or locations of voxels.

[0105] Figure 6 The upper part shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes of a coordinate system (e.g., the X, Y, and Z axes). This is achieved by using two poles (0,0,0) and (2... d ,2 d ,2 d An octree structure is created by recursively subdividing a bounding box with a defined cubic axis. Here, 2 d This can be set to the value of the minimum bounding box that constitutes all points surrounding the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following equation. In the following equation, (x int n ,y int n ,z int n ) indicates the position (or position value) of the quantized point.

[0106]

[0107] like Figure 6 As shown in the upper center, the entire 3D space can be divided into eight spaces according to partitions. Each partitioned space is represented by a cube with six faces. For example... Figure 6 As shown in the upper right, each of the eight spaces is further subdivided based on a coordinate system axis (e.g., the X, Y, and Z axes). Thus, each space is divided into eight smaller spaces. These smaller spaces are also represented by cubes with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0108] Figure 6 The lower part shows the octree occupancy code. The occupancy code generates the octree to indicate whether each of the eight partitions generated by dividing a space contains at least one point. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of a partitioned space, and each child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to a child node contains at least one point, the node is assigned a value of 1. When the space corresponding to a child node does not contain a point (the space is empty), the node is assigned a value of 0. Because Figure 6The occupancy code shown is 00100001, indicating that the space corresponding to the third and eighth child nodes among the eight child nodes each contains at least one point. As shown, each of the third and eighth child nodes has eight child nodes, and the child nodes are represented by an 8-bit occupancy code. The attached figure shows that the occupancy code for the third child node is 10000111, and the occupancy code for the eighth child node is 01001111. A point cloud encoder (e.g., an arithmetic encoder 40004) according to an embodiment can perform entropy coding on the occupancy code. To increase compression efficiency, the point cloud encoder can perform intra-frame / inter-frame coding on the occupancy code. A receiving device (e.g., receiving device 10004 or point cloud video decoder 10006) according to an embodiment reconstructs the octree based on the occupancy code.

[0109] A point cloud encoder according to an embodiment (e.g., Figure 4 A point cloud encoder or octree analyzer (40002) can perform voxelization and octree encoding to store point locations. However, points are not always uniformly distributed in 3D space, and therefore there may be specific regions where fewer points are present. Therefore, performing voxelization over the entire 3D space is inefficient. For example, when a specific region contains very few points, voxelization is not necessary in that specific region.

[0110] Therefore, for the specific region mentioned above (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment can skip voxelization and perform direct encoding to directly encode the point positions included in the specific region. The coordinates of the directly encoded points according to the embodiment are called the Direct Encoding Mode (DCM). The point cloud encoder according to the embodiment can also perform triadic geometry encoding based on the surface model, which reconstructs the point positions in the specific region (or node) based on voxels. Triadic geometry encoding is a geometry encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct encoding and triadic geometry encoding according to the embodiment can be performed selectively. In addition, direct encoding and triadic geometry encoding according to the embodiment can be performed in combination with octree geometry encoding (or octree encoding).

[0111] To perform direct encoding, the option to use direct mode to apply direct encoding should be activated. The node to which direct encoding is to be applied is not a leaf node, and there should be fewer than a threshold number of points within that node. Additionally, the total number of points to which direct encoding is to be applied should not exceed a preset threshold. When the above conditions are met, the point cloud encoder (or arithmetic encoder 40004) according to the embodiment can perform entropy encoding on the point locations (or location values).

[0112] A point cloud encoder (e.g., a surface approximation analyzer 40003) according to an embodiment can determine a specific level of an octree (a level less than the depth d of the octree) and can start from that level using a surface model to perform triadic geometry encoding to reconstruct point locations in a node region based on voxels (triadic mode). The point cloud encoder according to an embodiment can specify the level to which triadic geometry encoding is applied. For example, the point cloud encoder does not operate in triadic mode when the specific level is equal to the depth of the octree. In other words, the point cloud encoder according to an embodiment can operate in triadic mode only when the specified level is less than the depth value of the octree. The 3D cubic region of a node at a specified level according to an embodiment is called a block. A block may include one or more voxels. A block or voxel may correspond to a brick. Geometry is represented by surfaces within each block. A surface according to an embodiment may intersect each edge of a block at most once.

[0113] A block has 12 edges, and therefore there are at least 12 intersections within a block. Each intersection is called a vertex (or highest point). Vertices existing along an edge are detected when there is at least one occupied voxel adjacent to the edge in all blocks sharing the edge. According to the embodiment, an occupied voxel refers to a voxel containing a point. The vertex position detected along the edge is the average position of the edges of all voxels adjacent to the edge in all blocks sharing the edge.

[0114] Once a vertex is detected, the point cloud encoder according to the embodiment can perform entropy encoding on the edge's origin (x, y, z), the edge's direction vector (Δx, Δy, Δz), and the vertex position value (relative position value within the edge). When applying triad geometry encoding, the point cloud encoder according to the embodiment (e.g., geometry reconstructor 40005) can generate the restored geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization processes.

[0115] Vertices located at the edges of a block define the surface traversing the block. According to the embodiment, the surface is a non-planar polygon. During triangle reconstruction, the surface represented by triangles is reconstructed based on the origin of the edges, the direction vectors of the edges, and the position values ​​of the vertices. The triangle reconstruction process is performed by: 1) calculating the centroid value of each vertex, 2) subtracting the centroid value from each vertex value, and 3) estimating the sum of squares of the values ​​obtained through the subtraction.

[0116] 1)

[0117] Estimate the minimum value of the sum and perform a projection process based on the axis with the minimum value. For example, when element x is at its minimum, each vertex is projected onto the x-axis relative to the center of the block, and the projection is onto the (y,z) plane. When the value obtained by the projection onto the (y,z) plane is (ai,bi), the value of θ is estimated by atan2(bi,ai), and the vertices are sorted based on the value of θ. The following shows the vertex combinations for creating triangles based on the number of vertices. Vertices are sorted from 1 to n. The following shows that for four vertices, two triangles can be constructed based on vertex combinations. The first triangle can be composed of vertices 1, 2, and 3 from the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 from the sorted vertices.

[0118] Table 2-1. Triangles formed from vertices ordered 1, ..., n

[0119]

[0120] An upsampling process is performed to add points along the edges of the triangle at the center, and voxelization is then performed. The added points are generated based on the upsampling factor and the width of the block. These added points are called refined vertices. According to an embodiment, the point cloud encoder can voxelize the refined vertices. Additionally, the point cloud encoder can perform attribute encoding based on the voxelized positions (or position values).

[0121] Figure 7 An example of a neighbor node pattern according to an embodiment is shown.

[0122] To increase the compression efficiency of point cloud videos, the point cloud encoder according to the embodiment can perform entropy coding based on context-adaptive arithmetic coding.

[0123] For reference Figures 1 to 6 As described, a point cloud content providing system or point cloud encoder (e.g., point cloud video encoder 10002, Figure 4 The point cloud encoder or arithmetic encoder (40004) can immediately perform entropy coding on the occupancy code. Alternatively, the point cloud content providing system or point cloud encoder can perform entropy coding (intra-frame coding) based on the occupancy code of the current node and the occupancy of neighboring nodes, or entropy coding (inter-frame coding) based on the occupancy code of a previous frame. According to the embodiment, a frame represents a collection of simultaneously generated point cloud videos. The compression efficiency of intra-frame coding / inter-frame coding according to the embodiment can depend on the number of neighboring nodes referenced. As the number of bits increases, the computation becomes more complex, but coding can be biased to one side, which can increase compression efficiency. For example, when given a 3-bit context, 8 (=2) bits are required. 3 There are several methods to perform encoding. The division of the encoding affects the implementation complexity. Therefore, it is necessary to meet an appropriate level of compression efficiency and complexity.

[0124] Figure 7 The illustration shows the process of obtaining an occupancy pattern based on the occupancy of neighboring nodes. According to an embodiment, a point cloud encoder determines the occupancy of neighboring nodes for each node in an octree and obtains the value of the neighboring node pattern. The neighboring node patterns are used to infer the occupancy pattern of a node. Figure 7 The left side of the diagram shows the cube corresponding to the node (the cube positioned in the middle) and six cubes (neighboring nodes) that share at least one face with it. The nodes shown in the diagram are nodes at the same depth. The numbers shown in the diagram represent the weights associated with the six nodes (1, 2, 4, 8, 16, and 32). Weights are assigned sequentially based on the position of neighboring nodes.

[0125] Figure 7 The right side of the diagram shows the neighbor node pattern values. The neighbor node pattern value is the sum of values ​​multiplied by the weights of the occupying neighbor nodes (neighbor nodes with points). Therefore, the neighbor node pattern values ​​range from 0 to 63. When the neighbor node pattern value is 0, it indicates that there are no occupying nodes among the node's neighbors. When the neighbor node pattern value is 63, it indicates that all neighbor nodes are occupying nodes. As shown in the diagram, because the neighbor nodes assigned weights 1, 2, 4, and 8 are considered occupying nodes, the neighbor node pattern value is 15, which is the sum of 1, 2, 4, and 8. The point cloud encoder can perform encoding based on the neighbor node pattern values ​​(e.g., when the neighbor node pattern value is 63, 64 types of encoding can be performed). According to embodiments, the point cloud encoder can reduce encoding complexity by changing the neighbor node pattern values ​​(e.g., based on a table that changes 64 to 10 or 6).

[0126] Figure 8 The illustration shows an example of point configuration in each LOD according to an embodiment.

[0127] For reference Figures 1 to 7 The description states that the encoded geometry is reconstructed (decompressed) before attribute encoding is performed. When direct encoding is applied, the geometry reconstruction operation may include changing the placement of the directly encoded points (e.g., placing the directly encoded points in front of the point cloud data). When triadic geometry encoding is applied, the geometry reconstruction process is performed through triangle reconstruction, upsampling, and voxelization. Because attributes depend on the geometry, attribute encoding is performed based on the reconstructed geometry.

[0128] A point cloud encoder (e.g., LOD generator 40009) can classify (reorganize) points by LOD. The figure shows the point cloud content corresponding to LOD. The leftmost image in the figure represents the original point cloud content. The second image from the left in the figure represents the point distribution in the lowest LOD, and the rightmost image represents the point distribution in the highest LOD. That is, points in the lowest LOD are sparsely distributed, and points in the highest LOD are densely distributed. In other words, as the LOD increases in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between points narrows.

[0129] Figure 9 The illustration shows an example of point configuration for each LOD according to an embodiment.

[0130] For reference Figures 1 to 8 As described, a point cloud content providing system or point cloud encoder (e.g., point cloud video encoder 10002, Figure 4 A point cloud encoder or LOD generator (40009) can generate LODs. LODs are generated by reorganizing points into a set of refinements based on a set LOD distance value (or a set of Euclidean distances). The LOD generation process is performed not only by the point cloud encoder but also by the point cloud decoder.

[0131] Figure 9 The upper part shows examples of points (P0 to P9) of point cloud content distributed in 3D space. Figure 9 In this context, the original order represents the order of points P0 to P9 before LOD generation. Figure 9 In this context, LOD-based order represents the order in which points are generated according to their LOD. Points are reorganized by LOD. Additionally, higher LODs include points belonging to lower LODs. For example... Figure 9 As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, the points of LOD1, P9, P8, and P7.

[0132] For reference Figure 4 As described, the point cloud encoder according to the embodiments can selectively or in combination perform predictive transform coding, lifting transform coding, and RAHT transform coding.

[0133] The point cloud encoder according to an embodiment can generate predictors for points to perform predictive transformation coding for setting the predictive attributes (or predictive attribute values) for each point. That is, N predictors can be generated for N points. The predictors according to an embodiment can calculate weights (= 1 / distance) based on the LOD value of each point, index information about neighboring points existing within a set distance of each LOD, and the distance to the neighboring points.

[0134] According to an embodiment, the predicted attribute (or attribute value) is set as the average of values ​​obtained by multiplying the attributes (or attribute values) of neighboring points (e.g., color, reflectivity, etc.) set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 40011) according to an embodiment can quantize and inverse quantize the residual (which may be referred to as residual attribute, residual attribute value, or attribute prediction residual) obtained by subtracting the predicted attribute (attribute value) from the attribute (attribute value) of each point. The quantization process is configured as shown in the table below.

[0135] Pseudocode for Residual Quantization of Table Attribute Prediction

[0136] int PCCQuantization(int value,int quantStep){

[0137] if(value>=0){

[0138] return floor(value / quantStep+1.0 / 3.0);

[0139] }else{

[0140] return-floor(-value / quantStep+1.0 / 3.0);

[0141] }

[0142] }

[0143] Pseudocode for inverse quantization of residuals in table attribute prediction

[0144] int PCCInverseQuantization(int value,int quantStep){

[0145] if(quantStep==0){

[0146] return value;

[0147] }else{

[0148] return value * quantStep;

[0149] }

[0150] }

[0151] When the predictor for each point has neighboring points, the point cloud encoder (e.g., arithmetic encoder 40012) according to the embodiment can perform entropy encoding on the residual values ​​of quantization and inverse quantization as described above. When the predictor for each point has no neighboring points, the point cloud encoder (e.g., arithmetic encoder 40012) according to the embodiment can perform entropy encoding on the attributes of the corresponding point without performing the above operations.

[0152] The point cloud encoder (e.g., lift transformer 40010) according to an embodiment can generate a predictor for each point, set the calculated LOD and register neighboring points in the predictor, and set weights based on the distance to the neighboring points to perform lift transform coding. The lift transform coding according to the embodiment is similar to the predictive transform coding described above, but differs in that weights are applied cumulatively to attribute values. The process of cumulatively applying weights to attribute values ​​according to the embodiment is configured as follows.

[0153] 1) Create an array quantized weights (QW) to store the weight values ​​for each point. All elements of the QW are initialized to 1.0. Multiply the QW values ​​of the predictor indices of the neighboring nodes registered in the predictor by the weights of the current point's predictor, and sum the values ​​obtained through multiplication.

[0154] 2) Improve the prediction process: Subtract the value obtained by multiplying the attribute value of the point by the weight from the existing attribute value to calculate the predicted attribute value.

[0155] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0156] 4) The weights calculated by multiplying the weights computed for all predictors by the weights stored in the QW corresponding to the predictor index are accumulated into the updateweight array as the index of the neighbor node. The values ​​obtained by multiplying the attribute values ​​of the neighbor node indices by the calculated weights are accumulated into the update array.

[0157] 5) Improve the update process: Divide the attribute values ​​of the update array of all predictors by the weight values ​​of the updateweight array of the predictor index, and add the existing attribute values ​​to the values ​​obtained by division.

[0158] 6) For all predictors, the predicted attribute is calculated by multiplying the attribute value updated through the boosting update process by the weight updated through the boosting prediction process (stored in the QW). The predicted attribute value is quantized by a point cloud encoder (e.g., coefficient quantizer 40011) according to the embodiment. Additionally, the point cloud encoder (e.g., arithmetic encoder 40012) performs entropy encoding on the quantized attribute value.

[0159] A point cloud encoder according to an embodiment (e.g., RAHT transform 40008) can perform RAHT transform coding, where attributes associated with lower-level nodes in an octree are used to predict attributes of higher-level nodes. RAHT transform coding is an example of intra-frame attribute coding via backward scanning of an octree. The point cloud encoder according to an embodiment scans the entire region starting from voxels and repeats a merging process at each step, merging voxels into larger blocks, until the root node is reached. The merging process according to the embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on the node immediately above an empty node.

[0160] The following equation represents the RAHT transformation matrix. In this equation, This represents the average attribute value of the voxels at level l. Based on and To calculate.

[0161] and The weight is and

[0162]

[0163] here, It is a low-pass value and is used in the next higher-level merge process. This represents the high-pass coefficient. The high-pass coefficient at each step is quantized and subjected to entropy encoding (e.g., encoded by an arithmetic encoder 400012). Weights are calculated as follows: pass and Create the root node as follows.

[0164]

[0165] Like the high-pass coefficient, the value of gDC is also quantized and subjected to entropy encoding.

[0166] Figure 10 The diagram illustrates a point cloud decoder according to an embodiment.

[0167] Figure 10 The point cloud decoder shown in the diagram is Figure 1 The example of the point cloud video decoder 10006 described in [the document], and it can perform [operations] with [other functions]. Figure 1The point cloud video decoder 10006 illustrated operates in the same or similar manner. As shown, the point cloud decoder can receive a geometry bitstream and an attribute bitstream contained in one or more bitstreams. The point cloud decoder includes a geometry decoder and an attribute decoder. The geometry decoder performs geometry decoding on the geometry bitstream and outputs the decoded geometry. The attribute decoder performs attribute decoding based on the decoded geometry and the attribute bitstream and outputs the decoded attributes. The decoded geometry and decoded attributes are used to reconstruct the point cloud content (the decoded point cloud).

[0168] Figure 11 The diagram illustrates a point cloud decoder according to an embodiment.

[0169] Figure 11 The point cloud decoder shown in the figure is Figure 10 The illustrated example of a point cloud decoder, capable of performing decoding operations, is a reference. Figures 1 to 9 The reverse process of the encoding operation of the point cloud encoder described.

[0170] For reference Figure 1 and Figure 10 As described, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0171] The point cloud decoder according to the embodiment includes an arithmetic decoder (arithmetic decoding) 11000, an octree synthesizer (synthesized octree) 11001, a surface approximation synthesizer (synthesized surface approximation) 11002, a geometry reconstructor (reconstructed geometry) 11003, an inverse coordinate transformer (inverse coordinate transformation) 11004, an arithmetic decoder (arithmetic decoding) 11005, an inverse quantizer (inverse quantization) 11006, a RAHT transformer 11007, a LOD generator (generated LOD) 11008, an inverse lifter (inverse lift) 11009, and / or a color inverse transformer (inverse color transformation) 11010.

[0172] Arithmetic decoder 11000, octree synthesizer 11001, surface approximation synthesizer 11002, geometric reconstructor 11003, and coordinate inverse transformer 11004 can perform geometric decoding. Geometric decoding according to embodiments may include direct encoding and triplet geometric decoding. Direct encoding and triplet geometric decoding are selectively applied. Geometric decoding is not limited to the examples described above and is provided for reference only. Figures 1 to 9 The reverse process of the described geometric encoding is executed.

[0173] According to an embodiment, the arithmetic decoder 11000 decodes the received geometric bitstream based on arithmetic coding. The operation of the arithmetic decoder 11000 corresponds to the inverse process of the arithmetic encoder 40004.

[0174] The octree synthesizer 11001 according to an embodiment can generate an octree by obtaining a octet code (or information about the geometry obtained as a decoding result) from a decoded geometry bitstream. The octet code is as shown in the reference... Figures 1 to 9 Please describe that configuration in detail.

[0175] When applying triplet geometry encoding, the surface approximation synthesizer 11002 according to the embodiment can synthesize the surface based on the decoded geometry and / or the generated octree.

[0176] According to an embodiment, the geometry reconstructor 11003 can regenerate geometry based on surface and / or decoded geometry. See reference... Figures 1 to 9 As described, direct encoding and triadic geometric encoding are selectively applied. Therefore, the geometry reconstructor 11003 directly imports and sums the positional information of the points to which direct encoding is applied. When triadic geometric encoding is applied, the geometry reconstructor 11003 can reconstruct the geometry by performing reconstruction operations of the geometry reconstructor 40005, such as triangle reconstruction, upsampling, and voxelization. Details and References Figure 6 The descriptions are the same, and therefore their descriptions are omitted. The reconstructed geometry may include point cloud images or frames that do not contain attributes.

[0177] According to the embodiment, the inverse coordinate transformer 11004 can obtain the point position based on the reconstructed geometric transformation coordinates.

[0178] Arithmetic decoder 11005, inverse quantizer 11006, RAHT transformer 11007, LOD generator 11008, inverse booster 11009, and / or color inverse transformer 11010 can perform reference... Figure 10 The attribute decoding described herein includes Region Adaptive Hierarchical Transformation (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) decoding, and interpolation-based hierarchical nearest neighbor prediction (lifting transformation) decoding with update / lifting steps. These three decoding schemes may be used selectively, or a combination of one or more decoding schemes may be used. The attribute decoding according to the embodiments is not limited to the examples described above.

[0179] According to the embodiment, the arithmetic decoder 11005 decodes the attribute bitstream by arithmetic encoding.

[0180] According to the embodiment, the inverse quantizer 11006 inverse quantizes information about the decoded attribute bitstream or the attributes obtained as a decoding result, and outputs the inverse-quantized attributes (or attribute values). Inverse quantization can be selectively applied based on the attribute encoding of the point cloud encoder.

[0181] According to the embodiments, the RAHT transformer 11007, LOD generator 11008, and / or inverse lifter 11009 can process the reconstructed geometry and inverse quantization attributes. As described above, the RAHT transformer 11007, LOD generator 11008, and / or inverse lifter 11009 can selectively perform decoding operations corresponding to the encoding of the point cloud encoder.

[0182] According to the embodiment, the color inverse transformer 11010 performs inverse transform encoding to inverse transform the color values ​​(or textures) included in the decoded attributes. The operation of the color inverse transformer 11010 can be selectively performed based on the operation of the color transformer 40006 of the point cloud encoder.

[0183] Although not shown in the figure, Figure 11 The elements of the point cloud decoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above... Figure 11 The point cloud decoder's components include at least one or more operations and / or functions. Additionally, one or more processors can operate or perform operations for executing... Figure 11 A set of software programs and / or instructions for the operation and / or function of the elements of a point cloud decoder.

[0184] Figure 12 The illustration shows an exemplary transmitting device according to an embodiment.

[0185] Figure 12 The transmitting device shown is Figure 1 Transmitting device 10000 (or Figure 4 Example of a point cloud encoder. Figure 12 The transmitting device illustrated can perform and reference Figures 1 to 9 The described point cloud encoder includes one or more of the same or similar operations and methods. The transmitting device according to the embodiment may include a data input unit 12000, a quantization processor 12001, a voxelization processor 12002, an octree occupancy code generator 12003, a surface model processor 12004, an intra / inter-frame coding processor 12005, an arithmetic encoder 12006, a metadata processor 12007, a color transformation processor 12008, an attribute transformation processor 12009, a prediction / boosting / RAHT transformation processor 12010, an arithmetic encoder 12011, and / or a transmitting processor 12012.

[0186] According to an embodiment, the data input unit 12000 receives or acquires point cloud data. The data input unit 12000 can perform operations and / or acquisition methods similar to those of the point cloud video acquirer 10001 (or refer to...). Figure 2The described acquisition process (20000) is the same as or similar to the operation and / or acquisition method.

[0187] The data input unit 12000, quantization processor 12001, voxelization processor 12002, octree occupancy code generator 12003, surface model processor 12004, intra / inter-frame coding processor 12005, and arithmetic encoder 12006 perform geometric coding. Geometric coding and reference according to the embodiment. Figures 1 to 9 The geometric codes described are the same or similar, and therefore their detailed descriptions are omitted.

[0188] The quantization processor 12001 according to the embodiment quantizes geometry (e.g., point position values). The operation of the quantization processor 12001 and / or quantization with reference... Figure 4 The operation and / or quantization of the described quantizer 40001 are the same or similar. Details and references Figures 1 to 9 The descriptions are the same.

[0189] According to the embodiment, the voxelization processor 12002 voxels the quantized position values ​​of points. The voxelization processor 120002 can perform operations related to reference... Figure 4 The operation and / or voxelization process of the described quantizer 40001 is the same as or similar to the operation and / or process. Details and references Figures 1 to 9 The descriptions are the same.

[0190] According to the embodiment, the octree occupancy code generator 12003 performs octree encoding based on the voxelized positions of points in the octree structure. The octree occupancy code generator 12003 can generate occupancy codes. The octree occupancy code generator 12003 can perform operations similar to those referenced... Figure 4 and Figure 6 The operations and / or methods described are the same as or similar to those of the point cloud encoder (or octree analyzer 40002). Details and references Figures 1 to 9 The descriptions are the same.

[0191] According to an embodiment, the surface model processor 12004 can perform triadic geometry encoding based on a surface model to reconstruct point positions in a specific region (or node) based on voxels. The surface model processor 12004 can perform operations related to reference... Figure 4 The operations and / or methods described are the same as or similar to those of the point cloud encoder (e.g., surface approximation analyzer 40003). Details and references Figures 1 to 9 The descriptions are the same.

[0192] According to an embodiment, the intra / inter-frame coding processor 12005 can perform intra / inter-frame coding on point cloud data. The intra / inter-frame coding processor 12005 can perform operations similar to those referenced... Figure 7The described intra / inter-frame coding is the same or similar. Details and references. Figure 7 The descriptions are the same. According to an embodiment, the intra / inter-frame coding processor 12005 may be included in the arithmetic encoder 12006.

[0193] According to an embodiment, the arithmetic encoder 12006 performs entropy encoding on octrees and / or approximate octrees of point cloud data. For example, the encoding scheme includes arithmetic encoding. The arithmetic encoder 12006 performs the same or similar operations and / or methods as the arithmetic encoder 40004.

[0194] The metadata processor 12007 according to an embodiment processes metadata about point cloud data, such as setting values, and provides it to necessary processing procedures such as geometric encoding and / or attribute encoding. Additionally, the metadata processor 12007 according to an embodiment can generate and / or process signaling information related to geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be encoded separately from the geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be interleaved.

[0195] The color transformation processor 12008, attribute transformation processor 12009, prediction / boosting / RAHT transformation processor 12010, and arithmetic encoder 12011 perform attribute encoding. Attribute encoding and reference according to the embodiment. Figures 1 to 9 The attributes described are coded the same or similarly, and therefore their detailed descriptions are omitted.

[0196] According to an embodiment, the color transformation processor 12008 performs color transformation encoding to transform color values ​​included in attributes. The color transformation processor 12008 can perform color transformation encoding based on reconstructed geometry. The reconstructed geometry is related to a reference... Figures 1 to 9 The description is the same. Furthermore, its execution is the same as the reference. Figure 4 The operation and / or methods of the described color converter 40006 are the same as or similar to those described. Detailed descriptions are omitted.

[0197] According to an embodiment, the attribute transformation processor 12009 performs attribute transformations to transform attributes based on the reconstructed geometry and / or locations where geometric encoding is not performed. The attribute transformation processor 12009 performs and references... Figure 4 The operation and / or method of the described attribute transformer 40007 are the same as or similar to those described. Detailed descriptions thereof are omitted. The prediction / boosting / RAHT transformation processor 12010 according to the embodiment can encode the transformed attributes by any one or a combination of RAHT encoding, prediction transformation encoding, and boosting transformation encoding. The prediction / boosting / RAHT transformation processor 12010 performs and references... Figure 4The RAHT transformer 40008, LOD generator 40009, and lift transformer 40010 described herein operate at least one of the same or similar operations. Furthermore, the predictive transform coding, lift transform coding, and RAHT transform coding are similar to those described in the reference... Figures 1 to 9 The descriptions are the same, and therefore their detailed descriptions are omitted.

[0198] The arithmetic encoder 12011 according to the embodiment can encode the attributes of the encoding based on arithmetic encoding. The arithmetic encoder 12011 performs the same or similar operations and / or methods as the arithmetic encoder 400012.

[0199] The transmitting processor 12012 according to an embodiment can transmit each bitstream containing encoded geometric and / or encoded attribute and metadata information, or transmit a bitstream configured with encoded geometric and / or encoded attribute and metadata information. When the encoded geometric and / or encoded attribute and metadata information according to an embodiment is configured as a bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to an embodiment may include signaling information and slice data, the signaling information including a sequence parameter set (SPS) for sequence-level signaling, a geometric parameter set (GPS) for geometric information encoded signaling, an attribute parameter set (APS) for attribute information encoded signaling, and a tile parameter set (TPS) for tile-level signaling. Slice data may include information about one or more slices. A slice according to an embodiment may include a geometric bitstream Geom0. 0 and one or more attribute bitstreams Attr0 0 and Attr1 0 .

[0200] A slice is a series of syntax elements that represent some or all of an encoded point cloud frame.

[0201] According to an embodiment, the TPS may include information about each tile in one or more tiles (e.g., coordinate information and height / size information about the bounding box). The geometric bitstream may include a header and a payload. The header of the geometric bitstream according to an embodiment may include a geom_parameter_set_id, a geom_tile_id, and a geom_slice_id included in the GPS, as well as information about the data included in the payload. As described above, the metadata processor 12007 according to an embodiment may generate and / or process signaling information and send it to the transmit processor 12012. According to an embodiment, the element performing geometry encoding and the element performing attribute encoding may share data / information with each other, as indicated by the dashed lines. The transmit processor 12012 according to an embodiment may perform the same or similar operations and / or transmission methods as the transmitter 10003. Details and References Figure 1 and Figure 2 The descriptions are the same, and therefore their descriptions are omitted.

[0202] Figure 13 The illustration shows an exemplary receiving device according to an embodiment.

[0203] Figure 13 The receiving device shown in the diagram is Figure 1 The receiving device 10004 (or Figure 10 and Figure 11 Example of a point cloud decoder. Figure 13 The receiving device illustrated can perform and reference Figures 1 to 11 The same or similar one or more operations and methods described in the point cloud decoder.

[0204] The receiving device according to an embodiment includes a receiver 13000, a receiving processor 13001, an arithmetic decoder 13002, an octree reconstruction processor based on occupancy codes 13003, a surface model processor (triangle reconstruction, upsampling, voxelization) 13004, an inverse quantization processor 13005, a metadata parser 13006, an arithmetic decoder 13007, an inverse quantization processor 13008, a prediction / boost / RAHT inverse transform processor 13009, a color inverse transform processor 13010, and / or a renderer 13011. Each decoding element according to an embodiment can perform the inverse process of the operation of the corresponding encoding element according to an embodiment.

[0205] Receiver 13000 according to an embodiment receives point cloud data. Receiver 13000 can perform operations related to... Figure 1 The operation and / or receiving method of the receiver 10005 are the same as or similar to those of the receiver. Detailed description omitted.

[0206] According to an embodiment, the receiving processor 13001 can acquire geometric bitstreams and / or attribute bitstreams from the received data. The receiving processor 13001 may be included in the receiver 13000.

[0207] The arithmetic decoder 13002, the octet-based octree reconstruction processor 13003, the surface model processor 13004, and the inverse quantization processor 13005 can perform geometric decoding. Geometric decoding and reference according to the embodiment. Figures 1 to 10 The described geometric decodings are the same or similar, and therefore their detailed descriptions are omitted.

[0208] The arithmetic decoder 13002 according to the embodiment can decode a geometric bitstream based on arithmetic coding. The arithmetic decoder 13002 performs the same or similar operations and / or encodings as the arithmetic decoder 11000.

[0209] According to an embodiment, the octree reconstruction processor 13003 based on occupancy codes can reconstruct an octree by obtaining occupancy codes from the decoded geometric bitstream (or information about the geometry obtained as a decoding result). The octree reconstruction processor 13003 performs operations and / or methods identical or similar to those of the octree synthesizer 11001 and / or the octree generation method. When applying triad geometry encoding, the surface model processor 13004 according to an embodiment can perform triad geometry decoding and related geometric reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on surface modeling methods. The surface model processor 13004 performs operations identical or similar to those of the surface approximation synthesizer 11002 and / or the geometry reconstructor 11003.

[0210] According to the embodiment, the inverse quantization processor 13005 can inverse quantize the decoded geometry.

[0211] Metadata parser 13006 according to an embodiment can parse metadata, such as set values, contained in received point cloud data. Metadata parser 13006 can pass the metadata to geometry decoder and / or attribute decoder. Metadata and reference Figure 12 The metadata described is the same, and therefore its detailed description is omitted.

[0212] The arithmetic decoder 13007, inverse quantization processor 13008, prediction / boost / RAHT inverse transform processor 13009, and color inverse transform processor 13010 perform attribute decoding. Attribute decoding and reference Figures 1 to 10 The properties described are decoded the same or similarly, and therefore their detailed descriptions are omitted.

[0213] The arithmetic decoder 13007 according to an embodiment can decode the attribute bitstream via arithmetic coding. The arithmetic decoder 13007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 13007 performs the same or similar operations and / or encodings as the arithmetic decoder 11005.

[0214] According to the embodiment, the inverse quantization processor 13008 can inverse quantize the decoded attribute bitstream. The inverse quantization processor 13008 performs the same or similar operations and / or methods as the inverse quantizer 11006 and / or the inverse quantization method.

[0215] According to an embodiment, the prediction / boosting / RAHT inverse transform processor 13009 can process reconstructed geometry and inverse quantized attributes. The prediction / boosting / RAHT inverse transform processor 13009 performs one or more operations and / or decodings that are the same as or similar to those of the RAHT transformer 11007, LOD generator 11008, and / or inverse booster 11009. According to an embodiment, the color inverse transform processor 13010 performs inverse transform encoding to inverse transform color values ​​(or textures) included in the decoded attributes. The color inverse transform processor 13010 performs operations and / or inverse transform encodings that are the same as or similar to those of the color inverse transformer 11010. According to an embodiment, the renderer 13011 can render point cloud data.

[0216] Figure 14 An exemplary structure is shown that is operatively connectable to a method / device for sending and receiving point cloud data according to an embodiment.

[0217] Figure 14 The structure represents a configuration in which at least one of server 1460, robot 1410, autonomous vehicle 1420, XR device 1430, smartphone 1440, home appliance 1450, and / or HMD 1470 is connected to cloud network 1400. Robot 1410, autonomous vehicle 1420, XR device 1430, smartphone 1440, or home appliance 1450 are referred to as devices. Furthermore, XR device 1430 may correspond to a point cloud data (PCC) device according to an embodiment or be operatively connected to a PCC device.

[0218] Cloud network 1400 can refer to a network that forms part of or exists within a cloud computing infrastructure. Here, cloud network 1400 can be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

[0219] Server 1460 may be connected to at least one of the robot 1410, the self-driving vehicle 1420, the XR device 1430, the smart phone 1440, the household appliance 1450, and / or the HMD 1470 via the cloud network 1400, and may assist in the processing of at least a part of the connected devices 1410 to 1470.

[0220] The HMD 1470 represents one of the implementation types of the XR device and / or the PCC device according to an embodiment. According to an embodiment, the HMD-type device includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0221] Hereinafter, various embodiments of the devices 1410 to 1450 to which the above technologies are applied will be described. Figure 14 The illustrated devices 1410 to 1450 may be operably connected / coupled to the point cloud data sending / receiving device according to the above embodiment.

[0222] <PCC+XR>

[0223] The XR / PCC device 1430 may adopt PCC technology and / or XR (AR+VR) technology, and may be implemented as an HMD, a head-up display (HUD) provided in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a household appliance, a digital sign, a vehicle, a stationary robot, or a mobile robot.

[0224] The XR / PCC device 1430 may analyze 3D point cloud data or image data obtained through various sensors or from external devices and generate position data and attribute data regarding 3D points. Thus, the XR / PCC device 1430 may obtain information about the surrounding space or real objects, and render and output XR objects. For example, the XR / PCC device 1430 may cause an XR object including auxiliary information about the identified object to match the identified object and output the matched XR object.

[0225] <PCC+self-driving+XR>

[0226] The self-driving vehicle 1420 may be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0227] The self-driving vehicle 1420 to which XR / PCC technology is applied may represent an autonomous vehicle provided with a device for providing an XR image, or an autonomous vehicle as a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, the self-driving vehicle 1420 may be distinguished from the XR device 1430 and may be operably connected thereto.

[0228] The self-driving vehicle 1420, equipped with a device for providing XR / PCC images, can acquire sensor information from sensors including cameras and output generated XR / PCC images based on the acquired sensor information. For example, the self-driving vehicle 1420 may have a HUD and output XR / PCC images to it to provide passengers with XR / PCC objects corresponding to real objects or objects presented on a screen.

[0229] In this scenario, when an XR / PCC object is output to the HUD, at least a portion of the XR / PCC object can be output to overlap with the real object being pointed at by the passenger's eyes. Conversely, when an XR / PCC object is output to a display provided within the autonomous vehicle, at least a portion of the XR / PCC object can be output to overlap with an object on the screen. For example, the autonomous vehicle 1220 can output XR / PCC objects corresponding to objects such as roads, other vehicles, traffic lights, traffic signs, two-wheeled vehicles, pedestrians, and buildings.

[0230] Virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or point cloud compression (PCC) technologies according to the embodiments are applicable to various devices.

[0231] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. On the other hand, AR technology refers to the technology of displaying virtually created CG images on top of images of real objects. MR technology is similar to AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology differs from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with the same characteristics as real objects. More specifically, the application of MR technology is holographic services.

[0232] Recently, VR, AR, and MR technologies have sometimes been referred to as extended reality (XR) technologies rather than being clearly distinguished from each other. Therefore, embodiments of this disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are applicable to such technologies.

[0233] The PCC method / device according to the embodiments can be applied to vehicles that provide autonomous driving services.

[0234] Vehicles providing autonomous driving services connect to PCC devices for wired / wireless communication.

[0235] When the point cloud data (PCC) transmitting / receiving device according to an embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services that can be provided with autonomous driving services and transmit it to the vehicle. If the PCC transmitting / receiving device is installed in the vehicle, it can receive / process content data related to AR / VR / PCC services based on user input signals input through a user interface device and provide it to the user. The vehicle or user interface device according to an embodiment can receive user input signals. User input signals according to an embodiment may include signals indicating autonomous driving services.

[0236] Figure 15 This is a flowchart illustrating a point cloud data encoder and a point cloud data decoder according to an embodiment.

[0237] In this disclosure, the method / apparatus according to the embodiments can be interpreted as including both a transmitting method / apparatus and a receiving method / apparatus. The transmitting method / apparatus according to the embodiments may represent a point cloud data encoder or point cloud data encoder, and the receiving method / apparatus according to the embodiments may represent a point cloud data decoder or point cloud data decoder.

[0238] According to the embodiment, the point cloud source is represented as a bitstream and is a collection of points in decoding order. The point cloud may consist of one or more sample arrays and may include geometric information (geometric data) (which is coordinate information), attribute information such as brightness, reflectivity and color values, and arrays of attribute samples representing frame indices, transparency, etc.

[0239] Frames of point cloud data can be divided into tiles and slices.

[0240] The encoded and transmitted point cloud data may include sequence parameter sets, tile inventory, geometric parameter sets, attribute parameter sets, geometric data, and attribute data.

[0241] Geometric data can consist of a header and data (payload). The header may include a geometry parameter set ID / tile, slice, or frame ID. The payload may include node information for each geometric octree depth.

[0242] Attribute data can consist of a header and data (payload). The header may include a parameter set ID, attribute ID, and attribute geometry slice ID. The payload may include information such as the prediction index, residual values, and attribute codes for each point.

[0243] Figure 15 The encoders 15000 and 15010, and the decoders 15020 and 1530 can respectively correspond to Figure 1 Encoder 1002 and decoder 1006, Figure 2 Encoder / encode 20001 and decoder / decode 20003, Figure 4 PCC encoder Figure 11 PCC decoder Figure 12 Geometric encoder and attribute encoder diagrams Figure 13 geometry decoder and attribute decoder, or Figure 14 XR device 1730.

[0244] The embodiments propose a method to effectively support selective decoding when selective decoding of partial data is required due to receiver performance or transmission speed in transmitting and receiving point cloud data. The proposed method includes selecting necessary information or removing unnecessary information from bitstream units by dividing geometric and attribute data, typically delivered as data units, into semantic units such as geometric octrees and LoD (Level of Detail). Here, Network Abstraction Layer (NAL) units are defined as units for selecting information, and High-Level Syntax (HLS) is defined for PCC structure reconstruction.

[0245] Therefore, the embodiments address techniques for constructing data structures composed of point clouds. Specifically, methods for efficiently delivering layer-configured PCC data packaging and signaling will be described, and a method based on this approach will be proposed for application to scalable PCC-based services.

[0246] Point cloud data includes location (geometric (e.g., XYZ coordinates)) and attributes (e.g., color, reflectivity, intensity, grayscale, opacity, etc.). In point cloud compression (PCC), octree-based compression is performed to effectively compress non-uniformly distributed features in 3D space, based on their compressed attribute information. The G-PCC encoder and decoder have been referenced respectively... Figure 4 and 11 This has been described. Thus, the operation of the embodiment can be handled by the respective components on the transmitting and receiving sides of the PCC.

[0247] The transmitting device according to an embodiment compresses and transmits point cloud data by dividing location information (geometric data) and attribute information such as color / brightness / reflectivity (attribute data) about the data points into geometry and attributes. In this case, according to an embodiment, the PC data can be configured according to an octree structure with layers or level of detail (LoD), and scalable point cloud data encoding and representation can be performed based on this. In this operation, only a portion of the point cloud data may be decoded or represented due to the performance of the receiver or the transmission rate. However, there is currently no method to remove unwanted data in advance. That is, when only a portion of the scalable PCC bitstream needs to be transmitted (e.g., when only some layers are decoded in scalable decoding), it may not be permissible to select and transmit only the necessary portions. Therefore, 1) only the necessary portions should be re-encoded after decoding, or 2) after all data has been transmitted, the receiver should selectively apply the necessary operations. However, in case 1), delays may occur due to the time required for decoding and re-encoding. In case 2), bandwidth efficiency may degrade due to the transmission of unnecessary data. Furthermore, when using fixed bandwidth, it may be necessary to reduce the quality of the data used for transmission.

[0248] Variations and combinations of embodiments of this disclosure are possible. The terminology used in this disclosure can be understood based on the intended meaning of terms commonly used in the relevant art.

[0249] This paper proposes a bitstream packing method for effectively performing scalable representation, subsampling, subset extraction, etc., based on layer-by-layer configuration of point cloud data, using bitstreams as the unit.

[0250] Encoder 15000 and encoder 15010 are encoders of a point cloud data transmitting device according to an embodiment. Decoder 15020 and decoder 15030 are decoders of a point cloud data receiving device according to an embodiment.

[0251] The encoder 15000 can encode geometric data and attribute data. The data input to the encoder can be called source data. For example, source geometry is geometric input data, and source attributes are attribute input data. The encoder 15000 can encode the entire source geometry data and source attribute data to generate a complete PCC bitstream. The decoder 15020 can receive the complete PCC bitstream. The decoder 15020 can decode (or reconstruct) the complete source geometry data and source attribute data by decoding the data encoded and transmitted on the transmitting side.

[0252] Compared to encoder 15000, encoder 15010 may not fully encode the source data, but may encode only a portion of the data based on subsampling and / or downsampling techniques. Therefore, decoder 15030 can receive a portion of the PCC bitstream. Decoder 15030 can decode the portion of the PCC bitstream in the reverse process of encoder 15010 to reconstruct partial geometric and attribute data.

[0253] In this disclosure, a transmission / reception method / device according to embodiments is proposed to configure PC data according to an octree structure or a LoD (Level of Detail) having layers corresponding to the level of detail, define PCC NAL units, transmit the geometry / attribute layer structure with signals through a parameter set, pre-remove unnecessary data, and transmit only a portion of the data (partial PCC bitstream).

[0254] Figure 16 The configuration of segments 16020 to 16050 of the bitstream according to an embodiment is shown.

[0255] Figure 16 The illustration shows a transmission device according to an embodiment that divides a bitstream obtained through point cloud compression into a geometric data bitstream 16060 and an attribute data bitstream 16070 based on the data type to be transmitted. Here, each bitstream can be configured and delivered on a slice-by-slice basis. Each of the geometric data bitstream 16060 and the attribute data bitstream 16070 can be configured as a unit and delivered independently of layer information or LoD information.

[0256] Figure 16 The geometric bitstreams 16060, 16020, and 16040, and the attribute bitstreams 16070, 16030, and 06050 can be derived from... Figure 15 The encoders 15000 and 15010 generate / encode, and are generated by... Figure 15 Decoders 15020 and 15030 for decoding / parsing.

[0257] A slice refers to a series of syntax elements that represent the whole or part of an encoded point cloud frame.

[0258] In this scenario, the information used to reconstruct geometry and attributes resides within a single slice. Therefore, the likelihood of reconstruction errors due to information loss is likely low, and the bitstream can be easily configured. However, to use only some information from the bitstream, the entire bitstream needs to be decoded. Furthermore, when a subset of point cloud data is delivered or used in an application domain based on scalable coding, information reconstruction is required. Therefore, to address these issues, the method / apparatus according to the embodiments can perform scalable coding schemes 16000 and 16010 on the point cloud data.

[0259] To efficiently select information at the bitstream level, the transmitting / receiving device according to the embodiment can segment the geometric bitstream and the attribute bitstream separately. In this case, the data can be partitioned based on PCC hierarchies.

[0260] A layer refers to a unit of partitioning a bitstream for scalable decoding. A bitstream can be divided into segments. Segments correspond to layers. Layers can be included within NAL units. An NAL unit can include one or more layers. According to embodiments, various criteria can exist for partitioning a bitstream into layers, and related data within the bitstream can belong to the same layer. Furthermore, the bitstream can be partitioned into layers based on units such as slices / tiles that can be partitioned from point cloud data.

[0261] For example, an encoder can compress geometric data based on an octree structure. In this case, multiple pieces of information belonging to the same octree depth level can be bundled and delivered as a single unit. Therefore, devices in application domains that only require a subset of the information can selectively reconstruct the necessary information on a bitstream basis.

[0262] When the encoder performs RAHT encoding on attributes, it can distinguish attribute data based on the octree depth level, just as it would in the case of geometry.

[0263] When the encoder performs predictive-lifting encoding on attributes, the attribute data can be partitioned based on the LoD layer.

[0264] In partitioning the geometric bitstream into bitstream units based on the octree depth level, when encoding attributes based on RAHT, bitstream segments can be generated in the same units as the geometry (see [link to RAHT]). Figure 15 (Synchronization partition 16000).

[0265] As in 1) Figure 16 In the 16000 synchronization partition, signaling can be executed to make the geometric bitstream and attribute bitstream structurally similar.

[0266] As in 2) Figure 16 In the asynchronous partitioning of the 16010, when attributes are encoded via prediction boosting, LoD can be configured independently of the geometric octree structure, and attribute bitstream fragments can be configured in a format different from the geometric one.

[0267] In this case, bitstream segments 16020 to 16050 can be distinguished based on layers or using another method, depending on the application domain. When the above distinction method affects decoding, information about it can be sent separately as a signal. Furthermore, each bitstream segment can match only one layer of information, or multiple layers or some layers of information.

[0268] When the encoder of the transmitting device according to the embodiment generates a geometry bitstream 16060 and an attribute bitstream 16070, encoding / decoding (reconstruction) requires both bitstreams because the attributes depend on the geometry. The structure of the geometry bitstream 16060 and the attribute bitstream 16070 facilitates the reconstruction of data included in a single unit.

[0269] Furthermore, the encoder of the transmitting device according to the embodiment can generate geometric bitstreams 16020 and 16040 and attribute bitstreams 16030 and 16050. Geometric bitstream 16020 and attribute bitstream 16030 can be generated based on bitstream segments. For each layer, each bitstream can be partitioned into one or more segments, and each segment of the geometry and each segment of the attributes can be synchronized or desynchronized with each other according to the attribute encoding scheme. As mentioned above, information matching between bitstream segments and layers can be singular / plural matching. The bitstream segment processing method according to the embodiment will be further described below.

[0270] The geometric data includes layers generated based on an octree structure, and the attribute data includes layers generated based on LOD.

[0271] Furthermore, performing synchronous partitioning means that the number of points in the geometry is the same as the number of points in the attributes up to the selective scalable decoding of a specific layer.

[0272] When the geometric bitstream and attribute bitstream are divided into, as follows Figure 16 When the layer shown is used, scalable decoding can be performed efficiently and quickly, without the receiver needing to determine whether the received point cloud data needs to be decoded to the slice level.

[0273] Accordingly, a layer according to an embodiment can be a partitioning unit for a bitstream that can be scalably decoded.

[0274] Furthermore, the layers (fragments) according to embodiments of this disclosure are not limited to scalable decoding, but can be units that can be partitioned / classified according to various characteristics of geometry / attributes.

[0275] For example, even without scalable decoding, the geometric depth used in geometric encoding can have layers. Additionally, direct encoded points generated via IDCM and bitstreams based on octree compression can be partitioned into their respective layers. Similarly, aside from the case of scalable decoding, attribute layers used in attribute encoding can exist within the attribute data. Attribute data can be partitioned into layers based on the similarity between attributes.

[0276] Figure 17 The diagram illustrates bitstream segment matching based on a PCC layer structure according to an embodiment.

[0277] In the following text, a bitstream segment matching method based on a PCC layer structure will be described from the perspective of a point cloud data encoder, according to an embodiment.

[0278] Figure 17 Explain how to match Figure 16 The geometric / attribute data and geometric / attribute bitstream segments described in the document. Figure 15 The encoder can be Figure 17 The process generates / matches / encodes bitstream segments and sends them.

[0279] Figure 17 The diagram illustrates the correlation between the proposed bitstream segment configuration method and the layer structure of the actual geometric and attribute data. First, it is assumed that the geometry consists of three octree depth levels 17000, with the root to leaf numbered from 0 to 3, and the geometric bitstream segments 17010 matching their respective octree layers can also be numbered in the same way.

[0280] For example, segmentation / numbering can be performed such that segment 0 (NAL 0) matches the root node at depth level 0, segment 1 (NAL1) matches the node at depth level 1, segment 2 (NAL2) matches the node at depth level 2, and segment 3 (NAL3) matches the leaf node at depth level 3.

[0281] In this configuration, each bitstream segment can be packaged and delivered within a unit called the Network Abstraction Layer (NAL). Furthermore, by defining the bitstream information included in the NAL header (data type: geometric data; layer number = octree depth), it can be determined whether information should be retrieved from a NAL unit, even if the bitstream has not been parsed to a lower level.

[0282] When attributes are encoded using LoD-based prediction boosting (17030), the information newly included in each LoD can be assumed to constitute the information of each layer, such as... Figure 17 As shown in the image.

[0283] Attribute bitstream segment 17020 can be packed into attribute NAL units. For example, attribute NAL unit 0 can be information constituting LoD 0, attribute NAL units 0 and 1 can be information constituting LoD 1, and attribute NAL units 0, 1, and 2 can be information constituting LoD 2.

[0284] In this case, asynchronous partitioning can be applied because the bitstream fragment configuration method differs between geometry and attributes.

[0285] When the same bitstream segment configuration method is used for geometry and attributes Figure 16 Synchronized partitions can be applied.

[0286] According to the embodiments, the matching between segments (units) of the geometric bitstream and the deep layers of the geometric octree structure can be a one-to-one matching (packaging) or a many-to-many matching (packaging).

[0287] According to the embodiments, the matching between the fragments (units) of the attribute bitstream and the layers of the attribute LOD structure can be a one-to-one matching (packaging) or a many-to-many matching (packaging).

[0288] According to an embodiment, depth and LOD can be references used to distinguish layers, or other references can be units of layers.

[0289] Figure 18 The illustration shows the decoding of bitstream segments and layer matching of the PCC structure according to an embodiment.

[0290] refer to Figure 18 Decoding will be described from the perspective of the receiving device (decoder) according to the embodiments. Figure 17 The process of generating the bitstream segment structure shown is illustrated.

[0291] As a corresponding process to the above embodiments, the point cloud receiving device or decoder according to the embodiments can decode the bit stream segments, and then match the bit stream segments with the layers in the PCC structure.

[0292] Figure 18 Illustration Figure 15 How does the decoder receive and decode? Figure 16 The geometric / attribute bitstream segment. That is, Figure 18 This diagram illustrates how the decoder matches received geometry / attribute bitstream segments with geometry / attribute data.

[0293] The receiver can selectively decode / parse bit stream segments partitioned and transmitted by the PCC layer structure based on the information in the NAL unit header.

[0294] Therefore, the amount of data sent to the decoder of the receiving device can be reduced efficiently / effectively beforehand. According to the embodiment, the decoder can select the layer to be decoded for each of the geometric bitstream and the attribute bitstream.

[0295] For example, the layer selection method can be defined by the receiver system, or the transmitter can deliver selection-related information to the receiver based on the decoder's performance.

[0296] The decoder can remove information following the layer based on information in the NAL unit header. Figure 17 In this context, it is assumed that all four levels (octree depth levels 0 to 3) are defined for use with the geometric octree level, but levels up to LoD1 are defined for use with attributes. In this case, it is not necessary to base the definition on the spatial_id in the attribute NAL cell header (see reference). Figure 20 The information is used to select / use the NAL 2 attribute. The selected information can be matched with the geometric octree layer and the LoD layer.

[0297] In this scenario, the entire layer configuration information and the information for matching bitstream segments can be used as reference information for selecting bitstream segments. This information can be sent via parameter sets such as SPS, GPS, and APS, or via information such as SEI messages (signaling information).

[0298] According to the embodiments, the point cloud receiving device or decoder can receive bit streams and perform bit stream extraction / selection based on NAL units.

[0299] 1) When nal_unit_type equals 0 (SPS_NUT), the receiving device or decoder parses the NAL unit and determines whether the point cloud data or bitstream is segmented based on slices (e.g., for data (bitstream) that is segmented into slices, sps_slice_segmentation_flag can be 1). Then, the receiving device, decoder, or geometry decoder can perform the geometry bitstream selection / extraction process as follows.

[0300] 2) For slice-based segmented bitstreams, when nal_unit_type equals 2 (GPS_NUT), the receiving device, decoder, or geometry decoder parses the NAL unit.

[0301] The receiving device, decoder, or geometry decoder can obtain geometry layer information that matches the nal_spatial_id in the NAL unit header. For example, according to an embodiment, when gps_max_spatial_id is 3, gps_max_geom_layer_idx is 3, and nal_spatial_id is 0, gps_geom_layer_idx can be 0 (18000). When nal_spatial_id is 1, gps_geom_layer_idx can be 1 (18010). When nal_spatial_id is 2, gps_geom_layer_idx can be 2 (18020). When nal_spatial_id is 3, gps_geom_layer_idx can be 3 (18030). That is, the geometry layer can be selected based on the nal_spatial_id of the geometry bitstream segment.

[0302] It can be based on Figure 20 The nal_spatial_id is obtained from the information sent by the signal in the NAL unit header.

[0303] When a geometric layer exists that is targeted by the receiving device, geometric NAL cells that match the layer and have a value greater than nal_spatial_id can be discarded. Therefore, selective resolution of NAL cells can be performed.

[0304] 3) For selected / extracted bitstream segments, the receiving device, decoder, or geometry decoder can obtain the geometry slice segment by parsing the NAL units. The decoder obtains the slice layer's original byte sequence payload (slice_layer_rbsp()) by parsing all selected slices according to the transmission order (or according to the order in which they were sent individually with signals). In this case, the order of data reconstruction is crucial because PCC transmission / reception performs predictions based on the similarity between layers.

[0305] 4) The decoded geometric data can be reconstructed based on the layer structure identified in operation 2).

[0306] The receiving device, decoder, or attribute decoder can perform attribute bitstream selection / extraction as follows.

[0307] 5) When the sps_synchronous_geom_attr_segment_flag included in the SPS is 0 (or 1, where the value can be changed according to the embodiment) (i.e., this flag indicates synchronization), it can indicate that the attribute NAL unit (16000) is configured in the same way as the geometric NAL unit (e.g., sps_synchronous_geom_attr_segment_flag is 1). In this case, the attribute target layer can be set to the value of the geometric target layer, and the bitstream of a specific layer and lower layers can be extracted and selected through the nal_spatial_id of the attribute NAL unit.

[0308] 6) When `sps_synchronous_geom_attr_segment_flag` equals 1 (or 0, where the value can be changed according to the embodiment) (i.e., when this flag indicates an asynchronous case), the attribute NAL unit can be configured differently from the geometric NAL unit (16010). In this case, information corresponding to the case where `nal_unit_type` is 3 (APS_NUT) is parsed. Here, attribute layer information matching `nal_spatial_id` in the attribute NAL unit header can be obtained.

[0309] For example, according to an embodiment, when `sps_synchromous_geom_attr_segment_flag` = 0 (i.e., the asynchronous case), assume `aps_max_spatial_id` is 3 and `aps_max_attr_layer_idx` is 2. In this case, when `nal_spatial_id` is 0, `aps_attr_layer_idx` can be 0 (18040). When `nal_spatial_id` is 1, `aps_attr_layer_idx` can be 1 (18050). When `nal_spatial_id` is 2, `aps_attr_layer_idx` can be 2. The structure of the attribute layer can be identified based on `nal_spatial_id` and `aps_attr_layer_idx` as described above. In the presence of a target geometry layer, attribute NAL units with values ​​greater than `nal_spatial_id` that match the corresponding layer can be discarded. In an embodiment, layer 2 may not be used. In this case, the information corresponding to LoD1 (18060) can be decoded only by discarding the case where nal_spatial_id=2.

[0310] 7) For selected / extracted bitstream segments, the decoder can obtain attribute slice segments by parsing NAL units. The decoder obtains slice_layer_rbsp() by parsing all selected slices according to the transmission order (or according to the order in which they were sent individually with signals). In this case, the order of data reconstruction is important because PCC performs predictions based on inter-layer similarity. Furthermore, the reconstructed geometric data can be used in attribute decoding if necessary.

[0311] 8) The decoded data can be reconstructed based on the layer structure identified in operation 6).

[0312] In short, in order to support Figure 15 Scalable / partial encoding / decoding allows embodiments to configure geometry / attribute bitstreams into fragments. In this case, NAL units can be used. Furthermore, when the geometry / attribute bitstream is represented as... Figure 16 When considering the fragments shown, one can take into account cases where geometry and attributes are synchronized and / or cases where geometry and attributes are not synchronized. For example... Figure 17 As shown, the encoder of the point cloud data transmitting device and / or the transmitting device according to the embodiment can generate geometric bitstream segments by matching geometric bitstream segments with the level (depth / layer) of octree nodes, and generate attribute bitstream segments by matching attribute bitstream segments with the LOD (layer) of attribute data. Similarly, as Figure 18As shown, based on signaling information, the decoder of the receiving device and / or the point cloud data receiving device according to the embodiment can perform scalable (partial) decoding of the geometric bitstream according to the matching / segment sent with the signal / NAL segment / octree level (depth / layer), and perform scalable (partial) decoding of the attribute bitstream segment according to the matching / segment sent with the signal / NAL segment / LOD level.

[0313] Decoding according to an embodiment may include matching units of the geometry bitstream to an octree layer of the geometry data and mapping units of the attribute bitstream to a level of detail (LOD) of the attribute data.

[0314] Figure 19 A PCC NAL (Network Abstraction Layer) unit according to an embodiment is shown.

[0315] As used herein, terminology, grammar, and semantics may be understood solely based on the intended meaning of the term and may be referred to as other terms within the equivalent scope.

[0316] NAL units can be derived from Figure 15 It is generated using encoders such as 15000 and 15010.

[0317] According to the embodiments, the point cloud data transmitting device and / or encoder can define information about the slices, geometric slice headers, and attribute slice headers into which the bit stream is segmented in the sequence parameter set.

[0318] In addition, by defining the above segmentation-related information in the corresponding or separate locations according to the application or system, different application scopes, application methods, etc. can be used.

[0319] Furthermore, the method for defining information independently of attribute encoding techniques according to the embodiments can be defined in conjunction with attribute encoding techniques.

[0320] The above segmentation-related information can be defined in the geometry parameter set used for geometric scalability. Additionally, when Figure 19 When the syntax elements shown apply not only to the current point cloud data stream but also to multiple point cloud data streams, information can be delivered through a higher-level parameter set.

[0321] nal_unit specifies the NAL unit that contains the bitstream of point cloud data.

[0322] The nal_unit_header is the header of a NAL unit. A NAL unit can consist of a header and a payload.

[0323] NumBytesInNalUnit specifies the number of bytes in a NAL unit.

[0324] `rbsp_byte[i]` indicates the i-th byte of the RBSP. RBPS represents the payload of the original byte sequence. The payload of a NAL cell includes RBPS. RBPS can include parameter sets, such as SPS, and slice data RBPS.

[0325] The value of emulation_prevention_three_byte is 0x03. This value must not be used during the decoding process when it exists.

[0326] A NAL element consists of a NAL element header and a NAL element payload, such as Figure 19 As shown in the diagram, the NAL cell payload can be configured in RBPS bytes.

[0327] The embodiment defines the PCC Network Abstraction Layer (NAL) unit as a method to increase the efficiency of bitstream unit selection when the bitstream is divided into segments. Here, the NAL unit can be divided into geometric NAL units (nal_unit_type = 16, 17) and attribute NAL units (nal_unit_type = 18, 19) according to nal_unit_type. In addition, non-coding layer information such as parameter sets or SEI messages can be distinguished.

[0328] The units that make up a bitstream include information about the structure of the units in the bitstream.

[0329] Figure 20 The header of the NAL unit according to an embodiment is shown.

[0330] When configuring PCC bitstreams on a spatial layer basis, `nuh_spatial_id_plus1` can be used to distinguish spatial layers within a bitstream unit by subtracting 1 from `nuh_spatial_id_plus1`. When considering spatial layer slicing, the values ​​of `gsh_slice_id` and `ash_slice_id` can be defined to be linked with `nuh_spatial_id_plus1`.

[0331] Figure 17 and Figure 18 The matching between the described PCC bitstream segments and PCC layers can be based on... Figure 20 The execution is based on the ID information corresponding to the ID information in the original unit header.

[0332] A spatial layer can be a unit of partitioning a PCC bitstream. The bitstream can be partitioned layer by layer based on the spatial ID information in the NAL unit header of the bitstream.

[0333] When slices are associated with spatial layers, the slice ID of each geometry and attribute can be linked to the spatial ID of the NAL unit, allowing the slice ID value to match the spatial ID value. In other words, geometry can be partitioned into layers based on an octree structure, and attributes can be partitioned into layers based on LOD (Level of Detail).

[0334] `nal_unit_type` specifies the NAL unit type. Details of the type value will be found in [reference needed]. Figure 21 Describe it.

[0335] Subtracting 1 from nuh_layer_id_plus1 indicates the layer represented by information from CL (coding layer) NAL units or non-CL NAL units.

[0336] Figure 21 The type of NAL unit according to an embodiment is shown.

[0337] Various types of NAL units can be transmitted via signals based on the value of nal_unit_type. The transmission method / device (point cloud data transmission method / device) according to the embodiment can generate and transmit NAL units, and the receiving method according to the embodiment can receive and parse information according to the type of NAL unit. Thus, reference information can be obtained. Figures 15 to 18 The technical purpose and effects described.

[0338] Figure 19 The NAL unit consists of a header and a payload. Figure 20 Describe the syntax of the header, and Figure 21 Shown according to Figure 20 The header is configured with the unit type of payload sent by the signal.

[0339] When nal_unit_type is 0, the NAL unit is SPS_NUT, the content of the NAL unit and the RBSP syntax structure are sequence parameter sets (seq_parameter_set_rbsp()), and the NAL unit type category is non-CL (coding layer).

[0340] When nal_unit_type is 1, the NAL unit is TPS_NUT, the content and RBSP syntax structure of the NAL unit are tile parameter set (tile_parameter_set_rbsp()), and the NAL unit type category is non-CL.

[0341] When nal_unit_type is 2, the NAL unit is GPS_NUT, the content and RBSP syntax structure of the NAL unit are geometric parameter set (geom_parameter_set_rbsp()), and the NAL unit type category is non-CL.

[0342] When nal_unit_type is 3, the NAL unit is APS_NUT, the content and RBSP syntax structure of the NAL unit are attribute parameter set (attribute_parameter_set_rbsp()), and the NAL unit type category is non-CL.

[0343] When nal_unit_type is 4, the NAL unit is AUD_NUT, the RBSP syntax structure of the NAL unit content is access unit delimiter (access_unit_delimiter_rbsp()), and the NAL unit type category is non-CL.

[0344] When nal_unit_type is 5, the NAL unit is PREFIX_SEI_NUT, the content of the NAL unit and the RBSP syntax structure are supplementary enhancement information (sei_rbsp()), and the NAL unit type category is non-CL.

[0345] When nal_unit_type is 6, the NAL unit is SUFFIX_SEI_NUT, the content of the NAL unit and the RBSP syntax structure are supplementary enhancement information (sei_rbsp()), and the NAL unit type category is non-CL.

[0346] When nal_unit_type is 7, the NAL unit is EOS_NUT, the content of the NAL unit and the RBSP syntax structure are end of sequence (end_of_seq_rbsp()), and the NAL unit type category is non-CL.

[0347] When nal_unit_type is 8, the NAL unit is EOB_NUT, the content of the NAL unit and the RBSP syntax structure are end of sequence (end_of_bitstream_rbsp()), and the NAL unit type category is non-CL.

[0348] When nal_unit_type is 9 to 15, the NAL unit is RSV_NCL9 to RSV_NCL15. The content and RBSP syntax structure of the NAL unit are reserved, and the NAL unit type category is non-CL.

[0349] When nal_unit_type is 16, the NAL unit is IDG_NUT, the content of the NAL unit and the RBSP syntax structure are coded slices of independently decodable geometry (slice_layer_rbsp()), and the NAL unit type category is GCL (geometric coding layer).

[0350] When nal_unit_type is 17, the NAL unit is DG_NUT, the content of the NAL unit and the RBSP syntax structure are geometry-dependent encoded slices (slice_layer_rbsp()), and the NAL unit type category is GCL.

[0351] When nal_unit_type is 18, the NAL unit is IDA_NUT, the content of the NAL unit and the RBSP syntax structure are coded slices of independently decodable attributes (slice_layer_rbsp()), and the NAL unit type category is ACL (attribute coding layer).

[0352] When nal_unit_type is 19, the NAL unit is DA_NUT, and the content and RBSP syntax structure of the NAL unit are attribute-dependent encoded slices (slice_layer_rbsp()), and the NAL unit type category is ACL.

[0353] When nal_unit_type is 20 to 27, the NAL unit is RSV_NVCL20 to RSV_NVCL27. The content and RBSP syntax structure of the NAL unit are reserved, and the NAL unit type category is CL (coding layer).

[0354] When nal_unit_type is 28 to 31, the NAL unit is UNSPEC28 to UNSPEC31, the content and RBSP syntax structure of the NAL unit are unspecified, and the NAL unit type category is CL.

[0355] Figure 22 The set of sequence parameters according to an embodiment is shown.

[0356] Figure 22 This illustrates the configuration of payload information for NAL units that may be included based on the type indicated by the header of the NAL unit.

[0357] A value of 1 for sps_slice_segment_flag indicates that slices are divided. When the flag is 0, it indicates that each of the geometry and attributes is configured in a slice.

[0358] A value of 1 for `sps_synchronous_geom_attr_segment_flag` indicates that geometry slices and attribute slices are divided with the same structure. A value of 0 indicates that geometry slices and attribute slices are divided with independent structures. When `sps_slice_segment_flag` is equal to 0, `sps_synchronous_geom_attr_segment_flag` should be 0.

[0359] A profile_compatibility_flags[j] equal to 1 indicates that the bitstream conforms to the profile indicated by profile_idc equal to j. For any j value not specified as a profile_idc value, the value of profile_compatibility_flag[j] can be equal to 0.

[0360] `level_idc` indicates the level to which the bitstream belongs. The bitstream should not contain any other value for `level_idc`. Other values ​​for `level_idc` are reserved for future use by ISO / IEC.

[0361] A `sps_bounding_box_present_flag` value of 1 indicates the source bounding box offset, and its size information is signaled in SPS. A `sps_bounding_box_present_flag` value of 0 indicates that source bounding box information is not signaled.

[0362] sps_bounding_box_offset_x indicates the x-offset of the source bounding box in Cartesian coordinates. If it does not exist, the value of sps_bounding_box_offset_x is assumed to be 0.

[0363] sps_bounding_box_offset_y indicates the y-offset of the source bounding box in Cartesian coordinates. If it does not exist, the value of sps_bounding_box_offset_y is assumed to be 0.

[0364] sps_bounding_box_offset_z indicates the z-offset of the source bounding box in Cartesian coordinates. If it does not exist, the value of sps_bounding_box_offset_z is assumed to be 0.

[0365] sps_bounding_box_scale_factor indicates the scaling factor of the source bounding box in Cartesian coordinates. If it does not exist, the value of sps_bounding_box_scale_factor is inferred to be 1.

[0366] sps_bounding_box_size_width indicates the width of the source bounding box in Cartesian coordinates. If it does not exist, the value of sps_bounding_box_size_width is inferred to be 1.

[0367] sps_bounding_box_size_height indicates the height of the source bounding box in Cartesian coordinates. If it does not exist, the value of sps_bounding_box_size_height is inferred to be 1.

[0368] sps_bounding_box_size_depth indicates the depth of the source bounding box in Cartesian coordinates. If it does not exist, the value of sps_bounding_box_size_depth is inferred to be 1.

[0369] sps_source_scale_factor indicates the scaling factor of the source point cloud.

[0370] sps_seq_parameter_set_id provides an identifier for SPS to reference by other syntax elements. In a specific version of the bitstream conforming to an embodiment, the value of sps_seq_parameter_set_id can be 0. Other values ​​may be reserved for future use by ISO / IEC.

[0371] sps_num_attribute_sets indicates the number of encoded attributes in the bitstream. The value of sps_num_attribute_sets can be in the range of 0 to 63.

[0372] attribute_dimension[i] specifies the number of components of the i-th attribute.

[0373] attribute_instance_id[i] specifies the instance ID of the i-th attribute.

[0374] attribute_bitdepth[i] specifies the bit depth of the i-th attribute signal.

[0375] attribute_cicp_colour_primaries[i] indicates the chromaticity coordinates of the primary color source of the i-th attribute.

[0376] `attribute_cicp_transfer_characteristics[i]` indicates the reference photoelectric transfer characteristic function of the color attribute, as a function of the source input linear light intensity Lc with a nominal real value range of 0 to 1, or indicates the reciprocal of the reference photoelectric transfer characteristic function as a function of the output linear light intensity Lo with a nominal real value range of 0 to 1.

[0377] The attribute_cicp_matrix_coeffs[i] describes the matrix coefficients used to derive luminance and chromaticity signals from the green, blue, and red or Y, Z, and X primary colors.

[0378] The attribute_cicp_video_full_range_flag[i] indicates the black level and range of the luminance and chrominance signals derived from the real-valued component signals of E'Y, E'PB, and E'PR or E'R, E'G, and E'B.

[0379] `known_attribute_label_flag[i]` equal to 1 specifies the `known_attribute_label` of the i-th attribute sent by a signal. `known_attribute_label_flag[i]` equal to 0 specifies the `attribute_label_four_bytes` of the i-th attribute sent by a signal.

[0380] A known_attribute_label[i] equal to 0 specifies that the attribute is color. A known_attribute_label[i] equal to 1 specifies that the attribute is reflectivity. A known_attribute_label[i] equal to 2 specifies that the attribute is frame index.

[0381] The attribute_label_four_bytes[i] indicates the known attribute type with 4 bytes.

[0382] `attribute_label_four_bytes[i]` equal to 0 indicates that the attribute type is color. `attribute_label_four_bytes[i]` equal to 1 indicates that the attribute type is reflectance. `attribute_label_four_bytes[i]` equal to 0xffffffff indicates that no attribute type is specified.

[0383] A value of 1 for `sps_extension_present_flag` indicates that the `sps_extension_data` syntax structure exists within the SPS syntax structure. A value of 0 for `sps_extension_present_flag` indicates that this syntax structure does not exist. When it does not exist, the value of `sps_extension_present_flag` is inferred to be 0.

[0384] sps_extension_data_flag can have any value. Its presence and value do not affect the decoder conforming to the version according to the embodiment.

[0385] The syntax and semantics described in this disclosure can provide the effects of the methods / apparatus according to the embodiments.

[0386] Figure 23 The geometry_parameter_set according to an embodiment is shown.

[0387] `gps_max_spatial_id` indicates the maximum value of the spatial ID used for the current geometry. `gps_max_spatial_id` can have values ​​between 0 and the maximum value of `nuh_spatial_id_plus1` given when the NAL cell type category is GCL (Geometry Coding Layer). See reference... Figure 18 As described, this information is used when the decoder performs a matching operation according to the embodiment.

[0388] `gps_max_geom_layer_idx` indicates the maximum value of the geometric layer defined in the geometry encoding. For example, for geometry configured in an octree structure, it might have a maximum octree depth level value. (See reference...) Figure 18 As described, this information is used when the decoder performs a matching operation according to the embodiment.

[0389] OCTREE refers to an octree representing the 3D geometry of a point cloud. An encoder encoding geometric data according to an embodiment can use a geometric octree. The octree structure is generated through recursive subdivision.

[0390] `gps_geom_layer_idx[i]` indicates the geometry layer that matches the `i`th `spatial_id`. For example, for geometry configured in an octree structure, it can indicate the octree depth level that matches the `spatial_id`.

[0391] Send gps_geom_layer_idx[i] in the geometry parameter set as many times as the value of gps_max_spatial_id. See reference Figure 18As described, this information is used when the decoder performs a matching operation according to the embodiment.

[0392] The `gps_geom_parameter_set_id` provides an identifier for GPS that other syntax elements can reference. The value of `gps_seq_parameter_set_id` can be in the range of 0 to 15 (inclusive).

[0393] The `gps_seq_parameter_set_id` specifies the value of `sps_seq_parameter_set_id` for the active SPS. The value of `gps_seq_parameter_set_id` can be in the range of 0 to 15 (inclusive).

[0394] A value of 1 for `gps_box_present_flag` specifies that additional bounding box information is provided in the geometry header referencing the current GPS. A value of 0 for `gps_bounding_box_present_flag` specifies that additional bounding box information is not sent as a signal in the geometry header.

[0395] A `gps_gsh_box_log2_scale_present_flag` value of 1 specifies that `gsh_box_log2_scale` is signaled in the header of each geometry slice referenced to the current GPS. A `gps_gsh_box_log2_scale_present_flag` value of 0 specifies that `gsh_box_log2_scale` is not signaled in the header of each geometry slice, but a common scale for all slices is signaled in the current GPS's `gps_gsh_box_log2_scale`. When `gps_box_present_flag` is 1, `gps_gsh_box_log2_scale_present_flag` is signaled.

[0396] gps_gsh_box_log2_scale indicates the common scaling factor for the origin of the bounding box of all slices of the current GPS.

[0397] A unique_geometry_points_flag value of 1 indicates that all output points have a unique location. A unique_geometry_points_flag value of 0 indicates that two or more output points may have the same location.

[0398] A neighbor_context_restriction_flag value of 0 indicates that the octree encoding uses the context determined from the six neighboring parent nodes. A neighbor_context_restriction_flag value of 1 indicates that the octree encoding uses only the context determined from the sibling nodes.

[0399] An inferred_direct_coding_mode_enabled_flag value of 1 indicates that the direct_mode_flag may exist in the geometry node syntax. A value of 0 indicates that the direct_mode_flag does not exist in the geometry node syntax.

[0400] A bitwise_occupancy_coding_flag of 1 indicates that geometry node occupancy is encoded using bitwise contextualization of the syntax element occupancy_map. A bitwise_occupancy_coding_flag of 0 indicates that geometry node occupancy is encoded using the dictionary-encoded syntax element occypancy_byte.

[0401] An adjacent_child_contextualization_enabled_flag of 1 indicates that the adjacent child nodes of the neighboring octree node are used for bit-based occupancy contextualization. An adjacent_child_contextualization_enabled_flag of 0 indicates that the child nodes of the neighboring octree node are not used for occupancy contextualization.

[0402] log2_neighbor_avail_boundary specifies the value of the variable NeighbAvailBoundary. This value is used during the decoding process as follows.

[0403] NeighbAvailBoundary=2"log2_neighbor_avail_boundary"

[0404] When `neighbour_context_restriction_flag` equals 1, `NeighbAvailabilityMask` is set to 1. Conversely, when `neighbour_context_restriction_flag` equals 0, `NeighbAvailabilityMask` is set to 1. <log2_neighbor_avail_boundary。

[0405] log2_intra_pred_max_node_size specifies the appropriate octree node size for intra-frame prediction.

[0406] Trisoup node size (log2_trisoup_node_size): The variable TrisoupNodeSize is specified as the size of the triangle node as follows:

[0407] TrisoupNodeSize = 1 <log2_trisoup_node_size

[0408] When log2_trisoup_node_size equals 0, the geometry bitstream only includes octree-coded syntax. When log2_trisoup_node_size is greater than 0, there are bitstream consistency requirements: inferred_direct_coding_mode_enabled_flag must be equal to 0, and unique_geometry_points_flag must be equal to 1.

[0409] A value of 1 for `gps_extension_present_flag` indicates that the `gps_extension_data` syntax structure exists within the GPS syntax structure. A value of 0 for `gps_extension_present_flag` indicates that this syntax structure does not exist. When it does not exist, the value of `gps_extension_present_flag` is inferred to be 0.

[0410] The `gps_extension_data_flag` can have any value. According to the embodiment, its presence and value do not affect the decoder conforming to the specification. According to the embodiment, the decoder conforms to the specification.

[0411] Figure 24 The attribute parameter set (attribute_parameter_set()) according to an embodiment is shown.

[0412] `aps_max_spatial_id` indicates the maximum value of the `spatial_id` for the current attribute. `aps_max_spatial_id` can have values ​​between 0 and the maximum value of `nuh_spatial_id_plus1` given when the NAL cell type class is ACL (Attribute Encoding Layer). When the geometry / attributes are out of sync according to `sps_synchronous_geom_attr_segment_flag`, `aps_max_spatial_id`, `aps_max_attr_layer_idx`, and `aps_attr_layer_idx_[i]` can be signaled. `aps_attr_layer_idx_[i]` can be signaled for each `aps_max_spatial_id`.

[0413] `aps_max_attr_layer_idx` indicates the maximum value of the attribute layer that makes up the attribute. For example, when attribute encoding is performed based on LoD, it may have the maximum value of the LoD layer.

[0414] `aps_attr_layer_idx[i]` indicates the attribute layer that matches the `i`th `spatial_id`. For example, when performing attribute encoding based on a LoD structure, it can indicate the LoD layer that matches the `spatial_id` of the attribute layer.

[0415] aps_attr_parameter_set_id provides an identifier for APS to reference by other syntax elements. The value of aps_attr_parameter_set_id can be in the range of 0 to 15 (inclusive).

[0416] aps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id used for active SPS. The value of aps_seq_parameter_set_id can be in the range of 0 to 15 (inclusive).

[0417] `attr_coding_type` indicates the encoding type of the attribute used for a given value of `attr_coding_type`. The value of `attr_coding_type` can be 0, 1, or 2. Other values ​​of `attr_coding_type` are reserved for future use by ISO / IEC. Decoders can ignore reserved values ​​of `attr_coding_type`.

[0418] For example, when attr_coding_type equals 0, the encoding type is PredictingWeight Lifting. When attr_coding_type equals 1, the encoding type is Region Adaptive Hierarchical Transformation (RAHT). When attr_coding_type equals 2, the encoding type is Fixed Weight Lifting.

[0419] `aps_attr_initial_qp` specifies the initial value of the variable `SliceQp` for each slice of the reference APS. The initial value of `SliceQp` is changed / modified at the attribute slice fragment layer when a non-zero value of `slice_qp_delta_luma` or `slice_qp_delta_luma` is decoded. The value of `aps_attr_initial_qp` can be in the range of 0 to 52 (inclusive).

[0420] aps_attr_chroma_qp_offset specifies the offset of the initial quantization parameters sent by the syntax aps_attr_initial_qp using signals.

[0421] A value of 1 for `aps_slice_qp_delta_present_flag` indicates that the `ash_attr_qp_delta_luma` and `ash_attr_qp_delta_luma` syntax elements exist in ASH. A value of 0 for `aps_slice_qp_present_flag` indicates that the `ash_attr_qp_delta_luma` and `ash_attr_qp_delta_luma` syntax elements do not exist in ASH.

[0422] `lifting_num_pred_nearest_neighbours` specifies the maximum number of nearest neighbors to be used for prediction. The value of `lifting_num_pred_nearest_neighbours` can be in the range of 1 to xx.

[0423] `lifting_max_num_direct_predictors` specifies the maximum number of predictors to be used for direct prediction. The value of `lifting_max_num_direct_predictors` can range from 0 to `lifting_num_pred_nearest_neighbours`.

[0424] The value of the variable MaxNumPredictors used in the decoding process is given as follows: MaxNumPredictors = lifting_max_num_direct_predictors + 1

[0425] The lifting_search_range specifies the range used to determine the nearest neighbors for prediction and building a distance-based, detailed search range.

[0426] A value of 1 for `lifting_lod_regular_sampling_enabled_flag` specifies that a regular sampling strategy should be used to build the level of detail. A value of 0 for `lifting_lod_regular_sampling_enabled_flag` specifies that a distance-based sampling strategy should be used instead.

[0427] `lifting_num_detail_levels_minus1` specifies the number of detail levels used for attribute encoding. The value of `lifting_num_detail_levels_minus1` can be in the range of 0 to xx.

[0428] The values ​​of `lifting_num_pred_nearest_neighbours`, `lifting_max_num_direct_predictors`, `lifting_search_range`, `lifting_lod_regular_sampling_enabled_flag`, and `lifting_num_detail_levels_minus1` are signaled based on the value of `isLifting`. `isLifting` is 1 when `attr_coding_type` is 0 or 2. Otherwise, it is 0.

[0429] `lifting_sampling_period[idx]` specifies the sampling period for the detail level `idx`. The value of `lifting_sampling_period[]` can be in the range of 0 to `xx`.

[0430] lifting_sampling_period and lifting_sampling_distance_squared[idx] are signaled according to lifting_lod_decimation_enabled_flag, and can be signaled for each index as many as the value of num_detail_levels_minus1.

[0431] `lifting_sampling_distance_squared[idx]` specifies the square of the sampling distance used for the detail level `idx`. The value of `lifting_sampling_distance_squared[]` can be in the range of 0 to `xx`.

[0432] `lifting_adaptive_prediction_threshold` specifies the threshold for enabling adaptive prediction. The value of `lifting_adaptive_prediction_threshold[]` can be in the range of 0 to xx.

[0433] `lifting_intra_lod_prediction_num_layers` specifies the number of LoD layers, where the predicted value of the target point can be generated by referencing decoded points within the same LoD layer. `lifting_intra_lod_prediction_num_layers` equals `num_detail_levels_minus1` plus 1, indicating that the target point can reference decoded points within the same LoD layer from all LoD layers. `lifting_intra_lod_prediction_num_layers` equals 0, indicating that the target point cannot reference decoded points within the same LoD layer from any LoD layer. `lifting_intra_lod_prediction_num_layers` can range from 0 to `lifting_num_detail_levels_minus1` plus 1.

[0434] When attr_coding_type equals 0, lifting_adaptive_prediction_threshold and lifting_intra_lod_prediction_num_layers are sent by signal.

[0435] `aps_extension_present_flag` equal to 1 indicates that the `aps_extension_data` syntax structure exists in the APS syntax structure. `aps_extension_present_flag` equal to 0 indicates that this syntax structure does not exist. When it does not exist, the value of `aps_extension_present_flag` is inferred to be equal to 0.

[0436] The `aps_extension_data_flag` can have any value. According to the embodiment, its presence and value do not affect the decoder conforming to the profile.

[0437] Figure 25 The figure shows the encoder of a point cloud data transmission device according to an embodiment.

[0438] Hereinafter, components of the transmitting terminal and receiving terminal according to embodiments will be described. Each component may correspond to a processor, software, or hardware. Furthermore, the components described below may be combined with the structure and / or signaling information of the PCC transmitting terminal and receiving terminal.

[0439] Figure 25 The illustration shows an embodiment of a detailed functional configuration for encoding and transmitting PCC data according to an embodiment. When point cloud data is input to encoders 25000 and 25010, encoders 25000 and 25010 provide positional information (geometric data (e.g., XYZ coordinates, phi-theta coordinates, etc.)) and attribute information (attribute data (e.g., color, reflectivity, intensity, grayscale, opacity, medium, material, gloss, etc.)). The compressed data is divided into multiple units for transmission. The data can be divided and packaged by packaging module 25020 in units of NAL according to hierarchical structure information.

[0440] In other words, the geometry encoder 25000 receives point cloud data and encodes the geometric data of the point cloud data. The encoded geometric data can be in the form of a geometric bitstream.

[0441] The attribute encoder 25010 receives point cloud data and encodes the attribute data of the point cloud data. The encoded attribute data can be in the form of an attribute bitstream.

[0442] The NAL cell packer 25020 receives geometry / attribute bitstreams and / or metadata, and stores the above references. Figures 15 to 17 as well as Figures 19 to 24 The described bitstream partitions and / or packs them into NAL units for layer structures. The NAL unit packer 25020 can exist as a separate component or can be included in the geometry / attribute encoders 25000 and 25010 to perform the corresponding operations.

[0443] Metadata generator 25030 generates metadata, which is signaling information related to the operation of geometry encoding 25000 and / or attribute encoding 25010. For example, refer to Figures 19 to 24 The described SPS, GPS, APS, TPS, etc. can be generated and provided to the NAL unit packer 25020.

[0444] The multiplexer 25040 can receive and multiplex sub-bit streams from each layer.

[0445] Transmitter 25050 can send point cloud bit streams.

[0446] According to the above embodiments, the point cloud data transmission device can increase storage and transmission efficiency by partitioning / compressing and transmitting PCC data.

[0447] Figure 26 The diagram illustrates a decoder for a point cloud data receiving device according to an embodiment.

[0448] Figure 26 The illustration shows an embodiment of a detailed functional configuration for receiving / decoding PCC data according to an embodiment. When a bitstream is input to receiver 26000, receiver 26000 can divide it into a bitstream for location information and a bitstream for attribute information and process the divided bitstream. In this case, NAL unit parser 26020 can distinguish between no-CL (non-coded layer) information such as SPS, GPS, and APS and CL (coded layer) information such as geometric NAL units and attribute NAL units. Metadata parser 26030 can identify the geometric and attribute layer structure of point cloud data based on the information carried in SPS, GPS, and APS, and set the target layer to be decoded. NAL unit selection 26020 can be performed to remove layers excluded from decoding in NAL units based on nal_spatial_id. Geometric decoder 26040 and attribute decoder 26050 can reconstruct geometric data and attribute data from the classified bitstreams according to the characteristics of the data, respectively, and then renderer 26060 can convert the data into a format for the final output.

[0449] In other words, receiver 26000 can receive point cloud bit streams transmitted by the transmitting device according to the embodiment.

[0450] The demultiplexer 26010 can demultiplex point cloud bit streams.

[0451] After receiving the demultiplexed point cloud bitstream, the NAL cell parser / selector 26020 can be based on the reference... Figures 16 to 20 NAL unit analysis reference described Figures 19 to 24 Describe the metadata, geometric data, and / or attribute data, and select the data to be decoded.

[0452] Metadata parser 26030 can parse metadata in point cloud data bitstream. It can provide metadata to NAL unit parser / selector 26020. Based on the parsed metadata, the operations of decoders 26040 and 26050 can be performed.

[0453] The geometry decoder 26040 decodes geometric data.

[0454] The attribute decoder 26050 decodes attribute data.

[0455] Renderer 26060 can render geometry / attribute data.

[0456] Figure 26 Each process in the process can handle, for example... Figure 15 The illustrated complete and / or partial PCC bitstream can be provided to the user in a scalable and efficient manner. Therefore, the decoder or receiver can partition and process data according to layers, thereby increasing decoder performance.

[0457] Figure 27 The diagram illustrates the process of sending and receiving point cloud data according to an embodiment.

[0458] Figure 27 The illustration shows the effect of the transmitting terminal according to an embodiment. Figure 27 and 28 The illustrations show various usage scenarios based on the embodiments.

[0459] This embodiment provides a method for transmitting point cloud data by dividing compressed data according to predetermined criteria. In particular, when using layered encoding, compressed data can be divided and transmitted according to layers, which can increase the storage and transmission efficiency of the transmitting end.

[0460] like Figure 27 As illustrated, the geometry and attributes of point cloud data can be compressed and provided. In PCC-based services, the compression ratio or the amount of data to be transmitted can be adjusted based on receiver performance or transmission environment. When point cloud data is bundled into a single slice unit, if receiver performance or transmission environment changes, 1) a bitstream suitable for each environment can be pre-converted, stored separately, and selected (27020, 27040) when to be transmitted, or 2) a transcoding process (27030) is required before transmission. In this case, problems related to storage space or latency depending on transcoding may arise when the number of receiver environments to be supported increases or transmission environments change frequently.

[0461] like Figure 15 As shown, point cloud data can be subsampled based on the performance of various decoders before data transmission, and then encoded and stored.

[0462] Furthermore, when compressed data is divided and sent in units of NAL units according to layers as presented in the embodiments, only the necessary portions of the pre-compressed data can be selectively sent in the bitstream step using the information in the NAL unit header, without the need for a separate transformation process (see [link]). Figures 19-20 ).

[0463] This approach may even be efficient in terms of storage space, as each stream requires only one storage space. Furthermore, because only the necessary layers are selected before transmission, efficient transmission in terms of bandwidth (via the bitstream selector 27040) can be achieved. The bitstream selector 27040 can be included in the transmitting device or connected to an encoder, transmitter, etc. The scalable encoder 27000 and the bitstream selector 27040 can encode and selectively transmit layer-based portions of the PCC bitstream (see [link to documentation]). Figures 16 to 18 ).

[0464] Encoder 27000 may correspond to the encoder according to the above embodiment. The geometry / attribute data as source input can be scalably encoded but not immediately transmitted. Instead, the encoded data can be stored in storage space 27020. Storage space 27020 may be included in the point cloud data transmitting device according to the embodiment and may be connected to encoder 10002 or transmitter 10003. The transmitting device may selectively transmit the point cloud data stored in storage space 27020. Alternatively, the transmitting device may perform transcoding 27030 on the point cloud data for partial encoding and then transmit the data.

[0465] Decoder 27010 may correspond to the decoder according to the above embodiment. It can partially reconstruct geometric / attribute data from the received bitstream.

[0466] Figure 28 The diagram illustrates the process of sending and receiving point cloud data according to an embodiment.

[0467] Figure 28 The diagram illustrates the process of the transmitting and receiving devices transmitting and receiving point cloud data according to the above embodiments.

[0468] The transmitting / receiving device according to the embodiment can transmit point cloud data by dividing compressed data according to predetermined criteria. When layered encoding according to the embodiment is used, the point cloud data can be divided and transmitted in compressed data form according to layers. In this case, the efficiency of the receiving side can be increased.

[0469] In particular, Figure 28 The diagram illustrates the operation of the transmitting and receiving devices when transmitting point cloud data consisting of layers. In this scenario, when information sufficient to reconstruct the entire PCC data is delivered regardless of receiver performance, the receiver needs to perform a process of selecting only the data corresponding to the necessary layers after reconstructing the point cloud data through decoding (data selection (subsampling) 28000). In this case, because the delivered bitstream has already been decoded, a receiver designed for low latency may experience delays or be unable to perform decoding depending on receiver performance.

[0470] The data selector (or subsampler) 28000 can be included in the receiving device and can be connected to the decoder, etc.

[0471] The transmitting device can use storage space 28010 to transmit the complete PCC bitstream via transcoding for low QP. The receiving device can receive the bitstream and reconstruct the complete geometry / attribute data. Furthermore, the geometry / attribute data can be partially selected and reconstructed via data selector 28000 or based on subsampling.

[0472] Furthermore, when the geometry / attribute bitstream is sliced ​​and delivered according to the above embodiments, the receiver can selectively decode the bitstream fields based on the density of the point cloud data to be represented according to the decoder performance or application domain, and deliver the decoded bitstream to the decoder. In this case, the information of the layer to be selected can be selected at the NAL unit stage through the nal_spatial_id and nal_data_type in the NAL unit header. In this case, because the selection is performed before decoding, the decoder efficiency can be increased, and decoders of various performance levels can be supported through a single bitstream.

[0473] For example, the scalable encoder 28020 corresponds to the encoder according to the above embodiment. The encoder 28020 can scalably encode geometric and / or attribute data as source input, and can then generate a complete PCC bitstream and send it or store it in storage for future transmission. Alternatively, it can generate a partial PCC bitstream and send that bitstream or store it in storage for future transmission.

[0474] Decoder 28040 corresponds to the decoder according to the above embodiment. Prior to decoder 28040, bitstream selector 28030 can selectively decode desired portions of data from a complete or partial PCC bitstream based on layer-based scalable coding using layer-specific NAL units. Decoder 28040 can decode and reconstruct partial geometry / attribute data.

[0475] Figure 29 The illustration shows a method for sending point cloud data according to an embodiment.

[0476] S29000: The method for transmitting point cloud data according to the embodiment includes encoding point cloud data including geometric data and attribute data. The encoding operation according to the embodiment may include... Figure 1 Encoder 1002, Figure 2 The code 20001 Figure 4 encoder, Figure 12 Encoders 12001 to 12011 Figure 14 XR equipment 1430, Figure 15Encoders 15000 and 15010, Figure 25 The encoders 25000 and 25010, the cell packer 25020, and the metadata generator 25030, Figure 27 The scalable encoder 27000 and bitstream selector 27040, and Figure 28 Operation of the scalable encoder 28020.

[0477] S29010: The point cloud data transmission method according to the embodiment further includes transmitting a bit stream including point cloud data. The transmission operation according to the embodiment includes... Figure 1 transmitter 10003 Figure 2 Transmission 20002, Figure 4 Transmission of geometric bitstreams and / or attribute bitstreams Figure 12 Transmitter 12012 Figure 14 XR equipment 1430, Figure 15 Transmission of complete or partial PCC bit streams Figure 16 Bitstream segmentation Figure 25 Multiplexer 25040 and transmitter 25050, Figure 27 Transcoding transmission and partial PCC bitstream transmission, and Figure 28 The operation of transcoding transmission and complete PCC bit stream transmission.

[0478] Figure 30 The illustration shows a method for receiving point cloud data according to an embodiment.

[0479] S30000: A method for receiving point cloud data according to an embodiment includes receiving a bit stream comprising point cloud data, the point cloud data including geometric data and attribute data. The receiving operation according to an embodiment may include... Figure 1 The operation and response of receiver 10005 Figure 2 The transmission of 20002 and the reception of the receiver Figure 4 and Figure 11 Geometry / attribute bitstream, Figure 13 Receiver 13000 and receiver processor 13001 Figure 14 XR device 1430, receiver Figure 15 Complete PCC bitstream and partial PCC bitstream Figure 16 Bitstream segment layer reception, Figures 19 to 24 The NAL unit receives, Figure 26 Receiver 26000 and demultiplexer 26010, Figure 27 Reception of part of the PCC bit stream Figure 28 Operations such as receiving a complete PCC bitstream.

[0480] S30010: The point cloud data receiving method according to the embodiment further includes decoding the point cloud data. Decoding according to the embodiment may include... Figure 1 Decoder 10006 Figure 2 Decoding 20003 Figure 10 geometry / attribute decoder Figure 11 geometry / attribute decoder Figure 13 Decoders 13002 to 13010 Figure 14 XR equipment 1430, Figure 15 Decoders 15020 and 15030, Figure 16 Decoding of bitstream fragment layers Figure 18 Layer matching of fragments (units), Figures 19 to 24 Syntax parsing / decoding Figure 26 Geometry / attribute decoders 26040 and 26050, metadata parser 26030, and cell parser / selector 26020, Figure 27 Decoder 27010, Figure 28 Operation of scalable decoder / data selector (subsampler) 28000, bitstream selector 28030 and decoder 28040, etc.

[0481] All or part of the operations in the point cloud data receiving method according to the embodiment can follow the reverse process of the sending method.

[0482] Through point cloud compression processing and the combination of PCC NAL units and geometry / attribute layer structures, a PCC transmitter or receiver according to embodiments of this disclosure can utilize the aforementioned PCC NAL units. Furthermore, geometry / attribute layer structures included in the aforementioned parameter set can be used. Therefore, the PCC transmitter can increase storage and transmission efficiency, while the PCC receiver can reduce latency and increase decoding efficiency.

[0483] In this disclosure, “A / B” is used to include “A and B” and “A or B”.

[0484] Embodiments have been described in terms of methods and / or apparatus, and the descriptions of methods and apparatus may be applied complementaryly to each other.

[0485] Although the accompanying drawings have been described separately for simplicity, new embodiments can be designed by combining the embodiments shown in the respective drawings. Recording media containing programs for performing the above embodiments, designed to be computer-readable and designed according to the needs of those skilled in the art, also fall within the scope of the appended claims and their equivalents. The apparatus and methods according to the embodiments are not limited to the configurations and methods of the above embodiments. Various modifications can be made to the embodiments by selectively combining all or some of the embodiments. Although preferred embodiments have been described with reference to the accompanying drawings, those skilled in the art will understand that various modifications and variations can be made to the embodiments without departing from the spirit or scope of this disclosure as described in the appended claims. These modifications should not be understood solely from the technical ideas or viewpoints of the embodiments.

[0486] Various elements of the device according to the embodiments can be implemented by hardware, software, firmware, or a combination thereof. Various elements of the embodiments can be implemented by a single chip, for example, a single hardware circuit. According to the embodiments, components according to the embodiments can be implemented as separate chips. According to the embodiments, at least one or more components of the device according to the embodiments can include one or more processors capable of executing one or more programs. One or more programs can perform any one or more operations / methods according to the embodiments or include instructions for performing them. Executable instructions for performing the methods / operations of the device according to the embodiments can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transient CRM or other computer program product configured to be executed by one or more processors. Additionally, the memory according to the embodiments can be used as a concept encompassing not only volatile memory (e.g., RAM) but also non-volatile memory, flash memory, and PROM. It can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, the processor-readable recording medium can be distributed to computer systems connected via a network, such that processor-readable code can be stored and executed in a distributed manner.

[0487] In this disclosure, the terms “ / ” and “,” should be interpreted as indicating “and / or”. For example, the expression “A / B” can mean “A and / or B”. Furthermore, “A, B” can mean “A and / or B”. Additionally, “A / B / C” can mean “at least one of A, B, and / or C”. Also, “A / B / C” can mean “at least one of A, B, and / or C”. Furthermore, in this specification, the term “or” should be interpreted as indicating “and / or”. For example, the expression “A or B” can mean 1) only A, 2) only B, or 3) both A and B. In other words, the term “or” as used in this document should be interpreted as indicating “additionally or alternatively”.

[0488] Terms such as "first" and "second" are used to describe various elements of the embodiments. However, the various components according to the embodiments should not be limited by the foregoing terms. These terms are used only to distinguish one element from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should not be interpreted beyond the scope of the various embodiments. Both the first user input signal and the second user input signal are user input signals, but they do not mean the same user input signal unless the context clearly specifies otherwise.

[0489] The terminology used to describe embodiments is for the purpose of describing particular embodiments and is not intended to limit the embodiments. As used in the description of embodiments and claims, unless the context clearly specifies otherwise, the singular forms “a,” “an,” and “the” include a plural of the referred objects. The expression “and / or” is used to include all possible combinations of terms. Terms such as “comprising” or “having” are intended to indicate the presence of figures, quantities, steps, elements, and / or components and should be understood not to exclude the possibility of additional figures, quantities, steps, elements, and / or components. As used herein, conditional expressions such as “if” and “when” are not limited to optional cases and are intended to be interpreted as performing the relevant operation when a particular condition is met, or to interpret the relevant definition based on a particular condition.

[0490] Operations according to the embodiments described herein can be performed by a transmitting / receiving device including a memory and / or processor according to the embodiments. The memory may store programs for processing / controlling the operations according to the embodiments, and the processor may control the various operations described herein. The processor may be referred to as a controller, etc. In the embodiments, operations may be performed by firmware, software, and / or combinations thereof. Firmware, software, and / or combinations thereof may be stored in a processor or memory.

[0491] Embodiments of the present invention

[0492] As described above, the relevant content has been described in the best mode for implementing the embodiments.

[0493] Industrial applicability

[0494] As described above, the embodiments are applicable in whole or in part to point cloud data sending / receiving devices and systems.

[0495] Those skilled in the art can change or modify the embodiments in various ways within the scope of the embodiments.

[0496] Embodiments may include variations / modifications within the scope of the claims and their equivalents.

Claims

1. A method for transmitting point cloud data, the method comprising: The geometric data of the point cloud data is encoded based on multiple geometric layers in segmented geometric slices; The attribute data of the point cloud data is encoded based on multiple attribute layers in the segmented attribute slices. as well as Send a bitstream comprising the point cloud data, the point cloud data including the geometric data in the segmented geometric slices and the attribute data in the segmented attribute slices. The bitstream includes information representing identifiers of the plurality of geometric layers, information representing identifiers of the plurality of attribute layers, information related to the number of the plurality of geometric layers, information regarding the maximum number of levels in the octree, and information regarding layer configurations associated with the plurality of geometric layers and the plurality of attribute layers. The plurality of geometric layers are associated with the levels of the octree. The segmented geometric slices are mapped to the plurality of geometric layers, and The segmented attribute slices are mapped to the multiple attribute layers.

2. The method according to claim 1, in, The method further includes: generating a geometric bitstream including the geometric data and an attribute bitstream including the attribute data; and Based on layers, the geometric bitstream and the attribute bitstream are segmented. Wherein: layers for the geometric bitstream are generated based on an octree structure; and Layers for the attribute bitstream are generated based on the level of detail (LOD).

3. An apparatus for transmitting point cloud data, the apparatus comprising: Memory; as well as A processor, the processor being linked to the memory, wherein the processor is configured to: The geometric data of the point cloud data is encoded based on multiple geometric layers in segmented geometric slices; The attribute data of the point cloud data is encoded based on multiple attribute layers in the segmented attribute slices. as well as Send a bitstream comprising the point cloud data, the point cloud data including the geometric data in the segmented geometric slices and the attribute data in the segmented attribute slices. The bitstream includes information representing identifiers of the plurality of geometric layers, information representing identifiers of the plurality of attribute layers, information related to the number of the plurality of geometric layers, information regarding the maximum number of levels in the octree, and information regarding layer configurations associated with the plurality of geometric layers and the plurality of attribute layers. The plurality of geometric layers are associated with the levels of the octree. The segmented geometric slices are mapped to the plurality of geometric layers, and The segmented attribute slices are mapped to the multiple attribute layers.

4. The apparatus according to claim 3, wherein, The processor is further configured to: Generate a geometry bitstream including the geometry data and an attribute bitstream including the attribute data; and Based on layers, the geometric bitstream and the attribute bitstream are segmented.

5. The apparatus according to claim 4, wherein: Layers for the geometric bitstream are generated based on an octree structure; and Layers for the attribute bitstream are generated based on the level of detail (LOD).

6. A method for receiving point cloud data, the method comprising: Receive a bitstream including point cloud data, the point cloud data including geometric data in segmented geometric slices and attribute data in segmented attribute slices; Based on multiple geometric layers in the segmented geometric slices, the geometric data of the point cloud data is decoded; The attribute data of the point cloud data is decoded based on multiple attribute layers in the segmented attribute slices. as well as The bitstream includes information representing identifiers of the plurality of geometric layers, information representing identifiers of the plurality of attribute layers, information related to the number of the plurality of geometric layers, information regarding the maximum number of levels in the octree, and information regarding layer configurations associated with the plurality of geometric layers and the plurality of attribute layers. The plurality of geometric layers are associated with the levels of the octree. The segmented geometric slices are mapped to the plurality of geometric layers, and The segmented attribute slices are mapped to the multiple attribute layers.

7. The method according to claim 6, in: The geometric data includes layers generated based on an octree structure; and The attribute data includes layers generated based on the level of detail (LOD). The bitstream includes a geometric bitstream and an attribute bitstream; the geometric bitstream includes the geometric data, and the attribute bitstream includes the attribute data. The geometric bitstream and the attribute bitstream are segmented based on layers.

8. An apparatus for receiving point cloud data, the apparatus comprising: Memory; as well as A processor, connected to the memory, wherein the processor is configured to: Receive a bitstream including point cloud data, the point cloud data including geometric data in segmented geometric slices and attribute data in segmented attribute slices; Based on multiple geometric layers in the segmented geometric slices, the geometric data of the point cloud data is decoded; The attribute data of the point cloud data is decoded based on multiple attribute layers in the segmented attribute slices. as well as The bitstream includes information representing identifiers of the plurality of geometric layers, information representing identifiers of the plurality of attribute layers, information related to the number of the plurality of geometric layers, information regarding the maximum number of levels in the octree, and information regarding layer configurations associated with the plurality of geometric layers and the plurality of attribute layers. The plurality of geometric layers are associated with the levels of the octree. The segmented geometric slices are mapped to the plurality of geometric layers, and The segmented attribute slices are mapped to the multiple attribute layers.

9. The apparatus according to claim 8, in, The bitstream includes a geometric bitstream and an attribute bitstream; the geometric bitstream includes the geometric data, and the attribute bitstream includes the attribute data. The geometric bitstream and the attribute bitstream are segmented based on layers.

10. The apparatus according to claim 9, wherein: The geometric data includes layers generated based on an octree structure; and The attribute data includes layers generated based on the level of detail (LOD). The processor is further configured to: Match the units of the geometric bitstream with the octree layers of the geometric data; and Map the cells of the attribute bitstream to the level of detail (LOD) of the attribute data. The unit contains information about the units of the bit stream.

Images