Point cloud data transmitting device, point cloud data transmitting method, point cloud data receiving device, and point cloud data receiving method.

Abstract

The point cloud data receiving device according to the embodiment includes a receiver for receiving a bitstream including geometry data of the point cloud data and / or attribute data of the point cloud data and / or a decoder for decoding the point cloud data in the bitstream. The point cloud data transmitting device according to the embodiment includes an encoder for encoding the point cloud data and / or a transceiver for transmitting a bitstream including the point cloud data and / or signaling information related to the point cloud data.

Description

[Technical field]

[0001] The embodiments relate to a method and apparatus for processing Point Cloud Content. [Background technology]

[0002] Point cloud content is content represented by a point cloud, which is a collection of points belonging to a coordinate system that represents a three-dimensional space. Point cloud content can represent three-dimensional media and is used to provide various services such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality) and autonomous driving services. However, to represent point cloud content, tens of thousands to hundreds of thousands of point data are required. Therefore, a method for efficiently processing a huge amount of point data is required.

[0003] In order to meet the requirement that the transmission, encoding, decoding and rendering processes be performed in real time with low latency, the point cloud data can be divided into point cloud frames or point cloud data in units of tiles or slices. Summary of the Invention [Problem to be solved by the invention]

[0004] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments provide an apparatus and method for efficiently processing point cloud data.The embodiments provide a method and apparatus for processing point cloud data to address latency and encoding / decoding complexity.

[0005] However, the scope of the invention is not limited to the above-mentioned technical problems, and may be extended to other technical problems that a person skilled in the art can derive based on all the contents described herein. [Means for solving the problem]

[0006] To achieve the technical objective, a point cloud data transmission method according to an embodiment includes: a step of encoding point cloud data and / or a step of transmitting a bitstream including the point cloud data and signaling information related to the point cloud data.

[0007] The encoding step according to the embodiment includes encoding geometry data of the points of the point cloud data and encoding attribute data of the points of the point cloud data.

[0008] Further, the point cloud data transmission method according to the embodiment further includes dividing the point cloud data into one or more slices, and the encoding step encodes the point cloud data for each slice.

[0009] Further, the dividing step according to the embodiment divides the point cloud data based on information about the maximum number of points included in a slice.

[0010] Further, the signaling information according to the embodiment includes information indicating whether the splitting step is performed and, if the splitting step is performed, information indicating a method of performing the splitting step, where the splitting step divides the point cloud data into a plurality of slices including a first slice and a second slice, and the first slice and the second slice include overlapping regions in a three-dimensional space including the point cloud data.

[0011] Furthermore, the method of division according to the embodiment is either a method of division based on an octree of the point cloud data or a method of division based on the length of an axis of a three-dimensional space including the point cloud data.

[0012] Furthermore, in the point cloud data transmission method according to the embodiment, if the number of points in the point cloud data is equal to or less than the maximum number information, the point cloud data transmission method does not divide the point cloud data into one or more slices.

[0013] To achieve the technical objective, a point cloud data receiving method according to an embodiment includes a step of receiving a bitstream including geometry data of the point cloud data and attribute data of the point cloud data, and a step of decoding the point cloud data in the bitstream.

[0014] The decoding step according to an embodiment includes decoding geometry data of the point cloud data and / or decoding quality data of the point cloud data.

[0015] The bitstream according to the embodiment includes information indicating whether the point cloud data is divided into one or more slices, and if the received point cloud data includes multiple slices, the decoder decodes the received point cloud data slice by slice, and the number of points included in the slice is determined based on the maximum number of points information.

[0016] According to an embodiment, the bitstream includes information indicating how to divide the slices, the slices according to the embodiment include a first slice and a second slice, the first slice and the second slice including overlapping regions in a three-dimensional space that include point cloud data.

[0017] Further, the division method according to the embodiment is either a method of dividing the point cloud data based on an octree or a method of dividing the point cloud data based on the length of an axis of a three-dimensional space including the point cloud data.

[0018] According to an embodiment, the division method is either a method of dividing based on an octree of the point cloud data or a method of dividing based on the length of an axis of a three-dimensional space including the point cloud data. Effect of the Invention

[0019] The apparatus and method according to the embodiment can process point cloud data with high efficiency.

[0020] The apparatus and method according to the embodiment can provide a high-quality point cloud service.

[0021] The apparatus and method according to the embodiment can provide point cloud content for providing general-purpose services such as VR services and autonomous driving services.

[0022] The apparatus and method for transmitting and receiving point cloud data according to the embodiment can improve the coding performance of the point cloud by performing a slice division method and signaling data required therefor.

[0023] The apparatus and method for transmitting and receiving point cloud data according to the embodiment can provide improved parallel processing and scalability by performing spatial adaptive partitioning for independent encoding / decoding of point cloud data.

[0024] The apparatus and method for transmitting and receiving point cloud data according to the embodiment can provide an effect of re-segmenting points that are spatially close to each other when performing independent data segmentation.

[0025] The point cloud data transmission and reception device and method according to the embodiment divides the point cloud data into tiles, slices and / or bricks as shown in Figures 18 and 19, thereby enabling random access and parallel coding in the three-dimensional space occupied by the point cloud data.

[0026] The apparatus and method for transmitting and receiving point cloud data according to the embodiment performs encoding and decoding independently in slice units, thereby preventing overlapping errors during encoding and decoding. [Brief description of the drawings]

[0027] The accompanying drawings, which are provided for a clearer understanding of the embodiments, and which, together with the associated description of the embodiments, illustrate the embodiments. For a clearer understanding of the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the drawings, in which like reference numerals correspond to corresponding parts in the following drawings.

[0028]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

[0029] Preferred embodiments will be described in detail with reference to the accompanying drawings. The following detailed description with reference to the accompanying drawings is intended to illustrate preferred embodiments rather than to illustrate only embodiments that may be implemented by the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments may be practiced without such details.

[0030] Most of the terms used in the examples are common and widely used in the relevant field, but some are arbitrarily selected by the applicant, and their meanings will be explained in detail below as necessary. Therefore, the examples should be understood based on the intended meaning of the terms, rather than the simple names and meanings of the terms.

[0031] FIG. 1 is a diagram illustrating an example of a point cloud content providing system according to an embodiment.

[0032] The point cloud content providing system shown in Fig. 1 includes a transmission device 10000 and a reception device 10004. The transmission device 10000 and the reception device 10004 are capable of wired or wireless communication to transmit and receive point cloud data.

[0033] The transmitting device 10000 according to the embodiment secures, processes, and transmits a point cloud video (or point cloud content). In the embodiment, the transmitting device 10000 includes a fixed station, a base transceiver system (BTS), a network, an AI (Artificial Intelligence) device and / or system, a robot, an AR / VR / XR device and / or server, etc. In the embodiment, the transmitting device 10000 includes a device that communicates with a base station and / or other wireless devices using a wireless connection technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)), a robot, a vehicle, an AR / VR / XR device, a mobile device, a home appliance, an IoT (Internet of Things) device, an AI device / server, etc.

[0034] The transmitting device 10000 according to the embodiment includes a Point Cloud Video Acquisition unit 10001, a Point Cloud Video Encoder 10002, and / or a Transmitter (or a communication module) 10003.

[0035] The point cloud video acquisition unit 10001 according to the embodiment acquires a point cloud video through a process such as capture, synthesis, or generation. The point cloud video is a point cloud content represented by a point cloud, which is a collection of points located in a three-dimensional space, and is also called point cloud video data. The point cloud video according to the embodiment includes one or more frames. One frame represents a still image / picture. Therefore, the point cloud video includes a point cloud image / frame / picture, and is called any of a point cloud image, a frame, and a picture.

[0036] The point cloud video encoder 10002 according to the embodiment encodes the secured point cloud video data. The point cloud video encoder 10002 encodes the point cloud video data based on point cloud compression coding. The point cloud compression coding according to the embodiment includes Geometry-based Point Cloud Compression (G-PCC) coding and / or Video based Point Cloud Compression (V-PCC) coding or next-generation coding. Note that the point cloud compression coding according to the embodiment is not limited to the above-mentioned embodiment. The point cloud video encoder 10002 can output a bitstream including encoded point cloud video data. The bitstream includes not only encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0037] The transmitter 10003 according to the embodiment transmits a bitstream including encoded point cloud video data. The bitstream according to the embodiment is encapsulated into a file or a segment (e.g., a streaming segment) and transmitted via various networks such as a broadcast network and / or a broadband network. Although not shown, the transmitter 10000 includes an encapsulation unit (or an encapsulation module) that performs an encapsulation operation. In the embodiment, the encapsulation unit is included in the transmitter 10003. In the embodiment, the file or segment is transmitted to the receiving device 10004 via a network or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitter 10003 according to the embodiment can perform wired or wireless communication with the receiving device 10004 (or a receiver 10005) via a network such as 4G, 5G, or 6G. The transmitter 10003 can also perform a required data processing operation via a network system (e.g., a communication network system such as 4G, 5G, or 6G). The transmitting device 10000 can also transmit encapsulated data by an on-demand method.

[0038] The receiving device 10004 according to the embodiment includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. In the embodiment, the receiving device 10004 includes a device, a robot, a vehicle, an AR / VR / XR device, a mobile device, a home appliance, an IoT (Internet of Things) device, an AI device / server, etc., that communicates with a base station and / or other wireless devices using a wireless connection technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)).

[0039] The receiver 10005 according to the embodiment receives a bitstream including point cloud video data or a file / segment in which the bitstream is encapsulated from a network or storage medium. The receiver 10005 performs a data processing operation required by a network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). The receiver 10005 according to the embodiment decapsulates the received file / segment and outputs a bitstream. In the embodiment, the receiver 10005 also includes a decapsulation unit (or a decapsulation module) for performing the decapsulation operation. The decapsulation unit is also embodied as an element (or component) separate from the receiver 10005.

[0040] The point cloud video decoder 10006 decodes a bitstream including point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data in the manner in which it was encoded (e.g., the reverse process of the operation of the point cloud video encoder 10002). Thus, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud reconstruction coding, which is the reverse process of point cloud compression. The point cloud reconstruction coding includes G-PCC coding.

[0041] The renderer 10007 renders the decoded point cloud video data. The renderer 10007 renders not only the point cloud video data but also audio data to output point cloud content. In an embodiment, the renderer 10007 includes a display for displaying the point cloud content. In an embodiment, the display is not included in the renderer 10007 and is embodied by a separate device or component.

[0042] In the drawing, the dotted arrow indicates a transmission path of feedback information obtained by the receiving device 10004. The feedback information is information for reflecting interaction with a user consuming point cloud content, and includes user information (e.g., head orientation information, viewport information, etc.). In particular, when the point cloud content is for a service requiring interaction with a user (e.g., an autonomous driving service, etc.), the feedback information may be transmitted to a content transmitting side (e.g., the transmitting device 10000) and / or a service provider. In an embodiment, the feedback information may be used not only by the transmitting device 10000 but also by the receiving device 10004, or may not be provided.

[0043] According to an embodiment, the head orientation information is information regarding the position, direction, angle, movement, etc. of the user's head. According to an embodiment, the receiving device 10004 calculates viewport information based on the head orientation information. The viewport information is information regarding the area of ​​the point cloud video that the user is looking at. The viewpoint is the point at which the user is looking at the point cloud video, and means the center of the viewport area. That is, the viewport is an area centered on the viewpoint, and the size, shape, etc. of the area are determined by the FOV (Field Of View). Therefore, the receiving device 10004 can extract viewport information based on the vertical or horizontal FOV supported by the device in addition to the head orientation information. In addition, the receiving device 10004 performs gaze analysis, etc. to confirm the user's point cloud consumption manner, the point cloud video area that the user is gazing at, the gaze time, etc. In an embodiment, the receiving device 10004 can transmit feedback information including the result of the gaze analysis to the transmitting device 10000. According to an embodiment, the feedback information is obtained during the rendering and / or display process. According to the embodiment, the feedback information is obtained by one or more sensors included in the receiving device 10004. Also, in the embodiment, the feedback information is obtained by the renderer 10007 or another external element (or device, component, etc.). The dotted lines shown in FIG. 1 indicate the process of transmitting the feedback information obtained by the renderer 10007. The point cloud content providing system processes (encodes / decodes) point cloud data based on the feedback information. Thus, the point cloud video data decoder 10006 can perform a decoding operation based on the feedback information. Also, the receiving device 10004 can transmit the feedback information to the transmitting device 10000. The transmitting device 10000 (or the point cloud video data encoder 10002) can perform an encoding operation based on the feedback information.Therefore, the point cloud content providing system does not process (encode / decode) all point cloud data, but efficiently processes necessary data (e.g., point cloud data corresponding to the user's head position) based on feedback information, and can provide point cloud content to the user.

[0044] In an embodiment, the sending device 10000 may be referred to as an encoder, a sending device, a transmitter, etc., and the receiving device 10004 may be referred to as a decoder, a receiving device, a receiver, etc.

[0045] The point cloud data processed (processed in a series of steps of acquisition / encoding / transmission / decoding / rendering) in the point cloud content providing system of FIG. 1 according to the embodiment is also called point cloud content data or point cloud video data. In the embodiment, the point cloud content data can be used as a concept including metadata or signaling information related to the point cloud data.

[0046] The elements of the point cloud content providing system illustrated in FIG. 1 may be implemented in hardware, software, a processor, and / or a combination thereof.

[0047] FIG. 2 is a block diagram illustrating operations for providing point cloud content according to an embodiment.

[0048] Figure 2 is a block diagram showing the operation of the point cloud content providing system described in Figure 1. As described above, the point cloud content providing system processes point cloud data based on point cloud compression coding (eg, G-PCC).

[0049] In a point cloud content providing system (e.g., a point cloud transmitting device 10000 or a point cloud video acquiring unit 10001) according to an embodiment, a point cloud video is acquired (20000). The point cloud video is represented by a point cloud belonging to a coordinate system representing a three-dimensional space. The point cloud video according to an embodiment includes a Ply (Polygon File format or the Stanford Triangle format) file. When the point cloud video has one or more frames, the acquired point cloud video includes one or more Ply files. A Ply file includes point cloud data such as the geometry and / or attributes of a point. The geometry includes the position of the point. The position of each point is represented by parameters (e.g., values ​​of each of the X-axis, Y-axis, and Z-axis) indicating a three-dimensional coordinate system (e.g., a coordinate system consisting of XYZ axes). The attributes include the attributes of the point (e.g., texture information of each point, hue (YCbCr or RGB), reflectance (r), transparency, etc.). One point has one or more attributes (or attributes). For example, one point may have one attribute of hue, or two attributes of hue and reflectance. In the embodiment, geometry may be referred to as position, geometry information, geometry data, etc., and attributes may be referred to as attributes, attribute information, attribute data, etc. In addition, the point cloud content providing system (e.g., the point cloud transmitting device 10000 or the point cloud video acquiring unit 10001) may obtain point cloud data from information related to the point cloud video acquiring process (e.g., depth information, hue information, etc.).

[0050] A point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) according to an embodiment encodes point cloud data (20001). The point cloud content providing system encodes point cloud data based on point cloud compression coding. As described above, point cloud data includes geometry and attributes of points. Thus, the point cloud content providing system can perform geometry encoding to encode the geometry and output a geometry bitstream. The point cloud content providing system can perform attribute encoding to encode the attributes and output an attribute bitstream. In an embodiment, the point cloud content providing system can perform attribute encoding based on the geometry encoding. The geometry bitstream and the attribute bitstream according to the embodiment are multiplexed and output as one bitstream. The bitstream according to the embodiment further includes signaling information related to the geometry encoding and the attribute encoding.

[0051] A point cloud content providing system (e.g., transmitting device 10000 or transmitter 10003) according to an embodiment transmits encoded point cloud data (20002). As described in FIG. 1, the encoded point cloud data is represented by a geometry bitstream and a feature bitstream. The encoded point cloud data is transmitted in the form of a bitstream together with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometry encoding and feature encoding). The point cloud content providing system also encapsulates the bitstream for transmitting the encoded point cloud data and transmits it in the form of a file or segment.

[0052] A point cloud content providing system (e.g., receiving device 10004 or receiver 10005) according to an embodiment receives a bitstream including encoded point cloud data, and the point cloud content providing system (e.g., receiving device 10004 or receiver 10005) can demultiplex the bitstream.

[0053] The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) decodes the encoded point cloud data (e.g., geometry bitstream, attribute bitstream) transmitted in a bitstream. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) decodes the point cloud video data based on signaling information related to the encoding of the point cloud video data included in the bitstream. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) decodes the geometry bitstream to restore the position (geometry) of the point. The point cloud content providing system decodes the attribute bitstream based on the restored geometry to restore the attribute of the point. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) restores the point cloud video based on the position according to the restored geometry and the decoded attribute.

[0054] A point cloud content providing system (e.g., receiving device 10004 or renderer 10007) according to an embodiment renders the decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) renders the geometry and characteristics decoded during the decoding process using various rendering methods. Points of the point cloud content are rendered as a fixed point with a certain thickness, a cube with a predetermined minimum size centered on the position of the fixed point, or a circle centered on the position of the fixed point. All or a part of the region of the rendered point cloud content is provided to a user via a display (e.g., a VR / AR display, a general display, etc.).

[0055] The point cloud content providing system according to the embodiment (e.g., receiving device 10004) can obtain feedback information (20005). The point cloud content providing system encodes and / or decodes point cloud data based on the feedback information. The feedback information and the operation of the point cloud content providing system according to the embodiment are the same as the feedback information and the operation described in FIG. 1, so a detailed description will be omitted.

[0056] FIG. 3 illustrates an example of a point cloud video capture process according to an embodiment.

[0057] FIG. 3 illustrates an example of a point cloud video capture process of the point cloud content providing system described in FIG. 1 and FIG.

[0058] Point cloud content includes point cloud videos (images and / or videos) showing objects and / or environments located in various 3D spaces (e.g., a 3D space showing a real environment, a 3D space showing a virtual environment, etc.). Thus, the point cloud content providing system according to the embodiment captures point cloud videos using one or more cameras (e.g., an infrared camera capable of obtaining depth information, an RGB camera capable of extracting color information corresponding to the depth information, etc.), projectors (e.g., an infrared pattern projector for obtaining depth information, etc.), LiDAR, etc. to generate point cloud content. The point cloud content providing system according to the embodiment extracts a geometric form composed of points in a 3D space from the depth information, and extracts characteristics of each point from the color information to obtain point cloud data. The images and / or videos according to the embodiment are captured based on either an inward-facing approach or an outward-facing approach.

[0059] The inward-looking method is shown on the left side of Figure 3. The inward-looking method is a method in which one or more cameras (or camera sensors) positioned around a central object capture the central object. The inward-looking method is used to generate point cloud content that provides the user with a 360-degree image of the core object (e.g., VR / AR content that provides the user with a 360-degree image of an object (e.g., a core object such as a character, player, item, actor, etc.)).

[0060] The outward-looking approach is shown on the right side of Figure 3. In the outward-looking approach, one or more cameras (or camera sensors) positioned around a central object capture the environment of the central object that is not the central object. The outward-looking approach is used to generate point cloud content to provide the surrounding environment from a user's perspective (e.g., content showing the external environment provided to a user of an autonomous vehicle).

[0061] As shown in the figure, the point cloud content is generated based on the capture operation of one or more cameras. In this case, since the coordinate systems of the respective cameras are different, the point cloud content providing system performs calibration of one or more cameras to set a global coordinate system before the capture operation. The point cloud content providing system also generates point cloud content by combining the image and / or video captured by the above-mentioned capture method with an arbitrary image and / or video. When generating point cloud content representing a virtual space, the point cloud content providing system does not perform the capture operation described in FIG. 3. The point cloud content providing system according to the embodiment can also perform post-processing on the captured image and / or video. That is, the point cloud content providing system can remove undesired areas (e.g., background) or can recognize a space where the captured images and / or videos are connected and fill in a spatial hole if one exists.

[0062] The point cloud content providing system can also generate one point cloud content by performing coordinate system transformation on the points of the point cloud video acquired from each camera. The point cloud content providing system performs coordinate system transformation of the points based on the position coordinates of each camera. In this way, the point cloud content providing system can generate content showing a wide range or generate point cloud content with a high point density.

[0063] FIG. 4 is a diagram illustrating an example of a point cloud encoder according to an embodiment.

[0064] FIG. 4 shows an example of the point cloud video encoder 10002 of FIG. 1. The point cloud encoder performs encoding by reconstructing point cloud data (e.g., point positions and / or characteristics) to adjust the quality of the point cloud content (e.g., lossless, lossy, near-lossless) according to the network condition or application. If the overall size of the point cloud content is large (e.g., point cloud content of 60 Gbps for 30 fps), the point cloud content providing system cannot stream the corresponding content in real time. Therefore, the point cloud content providing system can reconstruct the point cloud content based on the maximum target bit rate to provide it according to the network environment, etc.

[0065] As shown in Figures 1 and 2, a point cloud encoder can perform geometry encoding and feature encoding, where the geometry encoding is performed before the feature encoding.

[0066] The point cloud encoder according to the embodiment includes a coordinate system transformation unit (Transformation Coordinates) 40000, a quantization unit (Quantize and Remove Points (Voxelize)) 40001, an octree analysis unit (Analyze Octree) 40002, a surface approximation analysis unit (Analyze Surface Approximation) 40003, an arithmetic encoder (Arithmetic Encode) 40004, a geometry reconstruction unit (Reconstruct Geometry) 40005, a color transformation unit (Transform Colors) 40006, a characteristic transformation unit (Transfer Attributes) 40007, a RAHT transformation unit 40008, an LOD generation unit (Generated LOD) 40009, a lift transformation unit (Lifting) 40010, a coefficient quantization unit (Quantize Coefficients) 40011 and / or an arithmetic encoder (Arithmetic Encode) 40012.

[0067] The coordinate system conversion unit 40000, the quantization unit 40001, the octree analysis unit 40002, the surface approximation analysis unit 40003, the arithmetic encoder 40004, and the geometry reconstruction unit 40005 can perform geometry encoding. The geometry encoding according to the embodiment includes octree geometry coding, direct coding, trisoup geometry encoding, and entropy coding. The direct coding and trisoup geometry encoding are applied selectively or in combination. Note that the geometry encoding is not limited to the above examples.

[0068] As shown in the figure, the coordinate system conversion unit 40000 according to the embodiment receives a position and converts it into a coordinate system. For example, the position is converted into position information in a three-dimensional space (e.g., a three-dimensional space expressed in an XYZ coordinate system). The position information in the three-dimensional space according to the embodiment is also referred to as geometry information.

[0069] The quantizer 40001 according to the embodiment quantizes the geometry. For example, the quantizer 40001 quantizes the points based on the minimum position value of all points (for example, the minimum value on each axis for the X-axis, Y-axis, and Z-axis). The quantizer 40001 performs a quantization operation by multiplying the difference between the minimum position value and the position value of each point by a predetermined quantization scale value, and then rounding down or up to find the closest integer value. Therefore, one or more points can have the same quantized position (or position value). The quantizer 40001 according to the embodiment performs voxelization based on the quantized position to reconstruct the quantized points. The smallest unit including 2D image / video information is a pixel, and the points of the point cloud content (or 3D point cloud video) according to the embodiment are included in one or more voxels. A voxel is a combination of the words volume and pixel, and refers to a three-dimensional cubic space that is generated when a three-dimensional space is divided into units (unit=1.0) based on an axis (e.g., X-axis, Y-axis, Z-axis) that represents the three-dimensional space. The quantization unit 40001 can match a group of points in the three-dimensional space with a voxel. In an embodiment, one voxel can include only one point. In an embodiment, one voxel includes one or more points. In addition, to represent one voxel with one point, the position of the center point of the corresponding voxel can be set based on the position of one or more points included in the voxel. In this case, the characteristics of all positions included in one voxel are combined and assigned to the corresponding voxel.

[0070] The octree analysis unit 40002 according to the embodiment performs octree geometry coding (or octree coding) to represent the voxels in an octree structure, which represents points matched to the voxels based on an octet structure.

[0071] The surface approximation analysis unit 40003 according to the embodiment analyzes and approximates the octree. The octree analysis and approximation according to the embodiment is a process of analyzing a region containing a large number of points to voxelize it in order to efficiently provide an octree and voxelization.

[0072] The arithmetic encoder 40004 according to the embodiment entropy encodes the octree and / or the approximated octree. For example, the encoding method includes an arithmetic encoding method. As a result of the encoding, a geometry bitstream is generated.

[0073] The color transform unit 40006, the feature transform unit 40007, the RAHT transform unit 40008, the LOD generating unit 40009, the lift transform unit 40010, the coefficient quantization unit 40011 and / or the arithmetic encoder 40012 perform feature coding. As described above, one point has one or more features. The feature coding according to the embodiment is equally applied to the features of one point. However, when one feature (e.g., hue) includes one or more elements, an independent feature coding is applied to each element. The feature coding according to the embodiment includes color transformation coding, feature transformation coding, RAHT (Region Adaptive Hierarchial Transform) coding, Interpolarization-based hierarchical nearest-neighbour prediction-Prediction Transform coding, and Lift transform (interpolation-based hierarchical nearest-neighbour prediction with an update / lifting step (Lifting Transform)) coding. Depending on the point cloud content, the above-mentioned RAHT coding, predictive transform coding, and lift transform coding may be selectively used, or a combination of one or more coding may be used. Furthermore, the feature coding according to the embodiment is not limited to the above examples.

[0074] The color converter 40006 according to the embodiment performs color conversion coding to convert the color values ​​(or textures) included in the characteristics. For example, the color converter 40006 converts the format of the hue information (e.g., converts from RGB to YCbCr). The operation of the color converter 40006 according to the embodiment is optionally applied depending on the color values ​​included in the characteristics.

[0075] The geometry reconstruction unit 40005 according to the embodiment reconstructs (restores) the octree and / or the approximated octree. The geometry reconstruction unit 40005 reconstructs the octree / voxel based on the result of analyzing the distribution of points. The reconstructed octree / voxel is also called the reconstructed geometry (or restored geometry).

[0076] The feature converter 40007 according to the embodiment performs feature conversion to convert features based on a position where geometry encoding has not been performed and / or a reconstructed geometry. As described above, features depend on geometry, so the feature converter 40007 can convert features based on reconstructed geometry information. For example, the feature converter 40007 can convert features of a point included in a voxel based on a position value of the point. As described above, when the position of the center point of a voxel is set based on the positions of one or more points included in the voxel, the feature converter 40007 converts features of one or more points. When trisoup geometry encoding is performed, the feature converter 40007 can convert features based on the trisoup geometry encoding.

[0077] The attribute conversion unit 40007 performs attribute conversion by calculating the average value of the attributes or attribute values ​​(e.g., the hue or reflectance of each point) of adjacent points within a certain position / radius from the position (or position value) of the center point of each voxel. When calculating the average value, the attribute conversion unit 40007 applies a weighting value according to the distance from the center point to each point. Thus, each voxel has a position and a calculated attribute (or attribute value).

[0078] The feature conversion unit 40007 searches for adjacent points within a specific position / radius from the position of the center point of each voxel based on the KD tree or Moulton code. The KD tree supports a data structure that manages points based on their position so that a Nearest Neighbor Search (NNS) can be performed quickly using a binary search tree. The Moulton code is generated by mixing bits and expressing the coordinate values ​​(e.g., (x, y, z)) indicating the three-dimensional position of all points. For example, if the coordinate value indicating the position of a point is (5, 9, 1), the bit value of the coordinate value is (0101, 1001, 0001). If the bit values ​​are mixed according to the bit index in the order of z, y, and x, the result is 010001000111. This value is expressed as 1095 in decimal. In other words, the Moulton code value of a point with coordinate values ​​(5, 9, 1) is 1095. The feature converter 40007 aligns points based on the Moulton code value and performs nearest neighbor search (NNS) through a depth-first traversal process. If nearest neighbor search (NNS) is required in other conversion processes for feature coding after the feature conversion operation, a KD tree or Moulton code is used.

[0079] As shown, the transformed attributes are input to a RAHT transformer 40008 and / or an LOD generator 40009 .

[0080] The RAHT converter 40008 according to the embodiment performs RAHT coding to predict feature information based on the reconstructed geometry information. For example, the RAHT converter 40008 can predict feature information of a node at a higher level of the octree based on feature information associated with a node at a lower level of the octree.

[0081] The LOD generating unit 40009 according to the embodiment generates LOD (Level of Detail) to perform predictive conversion coding. The LOD according to the embodiment indicates the degree of detail of the point cloud content, and it is indicated that the smaller the LOD value, the lower the detail of the point cloud content, and the larger the LOD value, the higher the detail of the point cloud content. Points can be classified according to the LOD.

[0082] The lift transform unit 40010 according to the embodiment performs lift transform coding, which transforms the characteristics of the point cloud based on weights. As described above, the lift transform coding is selectively applied.

[0083] The coefficient quantization unit 40011 according to the embodiment quantizes the feature-coded feature based on the coefficients.

[0084] The arithmetic encoder 40012 according to the embodiment encodes the quantized feature based on arithmetic coding.

[0085] The elements of the point cloud encoder of FIG. 4 may be implemented in hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits (not shown) configured to communicate with one or more memories included in the point cloud providing apparatus. The one or more processors may perform any one of the operations and / or functions of the elements of the point cloud encoder of FIG. 4 described above. The one or more processors may also operate or execute a software program and / or set of instructions to perform the operations and / or functions of the elements of the point cloud encoder of FIG. 4. According to an embodiment, the one or more memories may include high speed random access memory or may include non-volatile memory (e.g., one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices, etc.).

[0086] FIG. 5 is a diagram showing an example of a voxel according to the embodiment.

[0087] FIG. 5 shows a voxel located in a three-dimensional space expressed by a coordinate system consisting of three axes, the X-axis, the Y-axis, and the Z-axis. As shown in FIG. 4, a point cloud encoder (e.g., the quantization unit 40001, etc.) performs voxelization. A voxel means a three-dimensional cubic space that is generated when the three-dimensional space is divided into units (unit=1.0) based on the axes (e.g., the X-axis, the Y-axis, and the Z-axis) that represent the three-dimensional space. FIG. 5 shows two extreme points (0,0,0) and (2 d , 2 d , 2 d ) is generated by an octree structure that recursively subdivides a bounding box (cubical axis-aligned bounding box) defined by . A voxel contains at least one point. The spatial coordinates of a voxel can be estimated from its positional relationship with a voxel group. As mentioned above, a voxel has characteristics (such as hue or reflectance) just like a pixel in a 2D image / video. A detailed description of voxels is omitted here as it has been described in FIG. 4.

[0088] FIG. 6 is a diagram illustrating an example of an octree and occupancy code according to an embodiment.

[0089] As shown in Figures 1 to 4, a point cloud content providing system (point cloud video encoder 10002) or a point cloud encoder (e.g., octree analysis unit 40002) performs octree geometry coding (or octree coding) based on an octree structure to efficiently manage the area and / or position of voxels.

[0090] The upper part of FIG. 6 shows an octree structure. The three-dimensional space of the point cloud content according to the embodiment is represented by the axes of a coordinate system (e.g., X-axis, Y-axis, and Z-axis). The octree structure is represented by two extreme points (0,0,0) and (2 d , 2 d , 2 d ) is generated by recursively subdividing the bounding box defined by (x int n , y int n , z int n ) indicates the quantized position (or position value) of the point.

[0091] d=

[0092] TIFF0007640526000001.tif13147

[0093] As shown in the upper center of Figure 6, the entire 3D space is divided into eight spaces through division. Each divided space is represented as a cube with six sides. As shown in the upper right of Figure 6, each of the eight spaces is again divided by the axes of the coordinate system (e.g., X-axis, Y-axis, Z-axis). Therefore, each space is again divided into eight smaller spaces. The divided smaller spaces are also represented as cubes with six sides. This division method is applied until the leaf nodes of the octree become voxels.

[0094] The lower part of FIG. 6 shows an occupancy code of an octree. The occupancy code of an octree is generated to indicate whether each of the eight divided spaces generated by dividing one space contains at least one point. Therefore, one occupancy code is expressed by eight child nodes. Each child node indicates the occupancy of the divided space, and has a 1-bit value. Therefore, the occupancy code is expressed by an 8-bit code. That is, if the space corresponding to a child node contains at least one point, the corresponding node has a value of 1. If the space corresponding to a node does not contain a point (empty), the corresponding node has a value of 0. The occupancy code shown in FIG. 6 is 00100001, which indicates that the spaces corresponding to the third and eighth child nodes of the eight child nodes each contain at least one point. As shown in the figure, the third and eighth child nodes each have eight child nodes, and each child node is expressed by an 8-bit occupancy code. The figure shows that the occupied code of the third child node is 10000111, and the occupied code of the eighth child node is 01001111. A point cloud encoder according to the embodiment (e.g., the arithmetic encoder 40004) can entropy code the occupied code. To improve compression efficiency, the point cloud encoder can also intra / inter code the occupied code. A receiving device according to the embodiment (e.g., the receiving device 10004 or the point cloud video decoder 10006) reconstructs an octree based on the occupied code.

[0095] A point cloud encoder according to an embodiment (e.g., the point cloud encoder or octree analyzer 40002 in FIG. 4) performs voxelization and octree coding to store the positions of points. However, since points in a 3D space are not always uniformly distributed, there may be certain areas where there are not many points. Therefore, it is inefficient to perform voxelization on the entire 3D space. For example, if there are almost no points in a certain area, there is no need to perform voxelization on that area.

[0096] Therefore, the point cloud encoder according to the embodiment does not perform voxelization for the above-mentioned specific region (or nodes other than the leaf nodes of the octree), but performs direct coding, which directly codes the positions of points included in the specific region. The coordinates of the direct coding points according to the embodiment are called a direct coding mode (DCM). The point cloud encoder according to the embodiment can also perform trisoup geometry encoding, which reconstructs the positions of points in a specific region (or node) based on voxels based on a surface model. Trisoup geometry encoding is a geometry encoding that represents an object as a series of triangle meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. The direct coding and trisoup geometry encoding according to the embodiment are selectively performed. The direct coding and trisoup geometry encoding according to the embodiment can also be combined with octree geometry coding (or octree coding).

[0097] To perform direct coding, an option to use direct mode for applying direct coding must be activated, and the node to which direct coding is applied must not be a leaf node, and points below a threshold must exist within a specific node. Also, the total number of points to be subject to direct coding must not exceed a predetermined threshold. If the above conditions are met, the point cloud encoder (or the computation encoder 40004) according to the embodiment can entropy code the positions (or position values) of the points.

[0098] The point cloud encoder (e.g., the surface approximation analysis unit 40003) according to the embodiment can determine a specific level of the octree (if the level is smaller than the depth d of the octree), and from that level, can perform trisoup geometry encoding, which uses a surface model to reconstruct the positions of points in the node area based on voxels (trisoup mode). The point cloud encoder according to the embodiment can specify the level to which the trisoup geometry encoding is applied. For example, if the specified level is equal to the depth of the octree, the point cloud encoder does not operate in trisoup mode. That is, the point cloud encoder according to the embodiment can operate in trisoup mode only when the specified level is smaller than the depth value of the octree. The 3D cubic area of ​​the node at a specified level according to the embodiment is called a block. One block includes one or more voxels. A block or a voxel can also correspond to a brick. Within each block, the geometry is represented as a surface. The surface according to the embodiment can intersect each edge of the block at most once.

[0099] Since one block has 12 edges, there are at least 12 intersections in one block. Each intersection is called a vertex. A vertex along an edge is detected if there is at least one occupied voxel adjacent to the edge among all blocks that share the edge. In this embodiment, an occupied voxel means a voxel that contains a point. The position of the detected vertex along the edge is the average position along the edge of all voxels adjacent to the edge among all blocks that share the edge.

[0100] When a vertex is detected, the point cloud encoder according to the embodiment can entropy code the edge start point (x, y, z), the edge direction vector (Δx, Δy, Δz), and the vertex position value (relative position value within the edge). When trisoup geometry encoding is applied, the point cloud encoder according to the embodiment (e.g., the geometry reconstruction unit 40005) can generate a restored geometry (reconstructed geometry) by performing triangle reconstruction, up-sampling, and voxelization processes.

[0101] TIFF0007640526000002.tif37170

[0102] TIFF0007640526000003.tif28163

[0103] The minimum of the added values ​​is found, and a projection process is performed along the axis where the minimum value is. For example, if the x element is the minimum, each vertex is projected onto the x-axis based on the center of the block, and then onto the (y,z) plane. If the value obtained by projecting onto the (y,z) plane is (ai,bi), the θ value is found using atan2(bi,ai) and the vertices are aligned based on the θ value. The following table shows the combination of vertices to generate a triangle depending on the number of vertices. The vertices are aligned in order from 1 to n. The following table shows that for four vertices, two triangles are formed by combining the vertices. The first triangle is formed from the 1st, 2nd, and 3rd vertices of the aligned vertices, and the second triangle is formed from the 3rd, 4th, and 1st vertices of the aligned vertices.

[0104] Table.Triangles formed from vertices ordered 1

[0105] n Triangles

[0106] 3 (1,2,3)

[0107] 4 (1,2,3)、(3,4,1)

[0108] 5 (1,2,3)、(3,4,5)、(5,1,3)

[0109] 6 (1,2,3)、(3,4,5)、(5,6,1)、(1,3,5)

[0110] 7 (1,2,3)、(3,4,5)、(5,6,7)、(7,1,3)、(3,5,7)

[0111] 8 (1,2,3)、(3,4,5)、(5,6,7)、(7,8,1)、(1,3,5)、(5,7,1)

[0112] 9 (1,2,3)、(3,4,5)、(5,6,7)、(7,8,9)、(9,1,3)、(3,5,7)、 (7,9,3)

[0113] 10 (1,2,3)、(3,4,5)、(5,6,7)、(7,8,9)、(9,10,1)、(1,3,5)、 (5,7,9)、(9,1,5)

[0114] 11 (1,2,3)、(3,4,5)、(5,6,7)、(7,8,9)、(9,10,11)、 (11,1,3)、 (3,5,7)、(7,9,11)、(11,3,7)

[0115] 12 (1,2,3)、(3,4,5)、(5,6,7)、(7,8,9)、(9,10,11)、 (11,12,1)、(1,3,5)、(5,7,9)、(9,11,1)、(1,5,9)

[0116] The upsampling process is performed to add intermediate points along the edges of triangles for voxelization. The additional points are generated based on the upsampling factor and the block width. The additional points are called refined vertices. The point cloud encoder according to the embodiment can voxelize the refined vertices. The point cloud encoder can also perform feature encoding based on the voxelized positions (or position values).

[0117] FIG. 7 is a diagram illustrating an example of an adjacent node pattern according to the embodiment.

[0118] To increase the compression efficiency of the point cloud video, the point cloud encoder according to the embodiment performs entropy coding based on context adaptive arithmetic coding.

[0119] As described in FIG. 1 to FIG. 6, a point cloud content providing system or a point cloud encoder (e.g., a point cloud video encoder 10002, a point cloud encoder or an arithmetic encoder 40004 in FIG. 4) can immediately perform entropy coding on the occupancy code. In addition, the point cloud content providing system or the point cloud encoder can perform entropy coding (intra coding) based on the occupancy code of the current node and the occupancy rate of the neighboring node, or can perform entropy coding (inter coding) based on the occupancy code of the previous frame. A frame according to the embodiment means a collection of point cloud videos generated at the same time. The compression efficiency of intra coding / inter coding according to the embodiment varies depending on the number of neighboring nodes to be referenced. Although the complexity increases as the number of bits increases, the compression efficiency can be improved by leaning to one side. For example, if there is a 3-bit context, coding is performed in 8 ways, which is 2 to the power of 3. The part to be coded separately affects the complexity of the implementation. Therefore, it is necessary to match the compression efficiency and the appropriate level of the complexity.

[0120] FIG. 7 illustrates a process of determining an occupancy pattern based on the occupancy of adjacent nodes. The point cloud encoder according to the embodiment obtains a neighbor pattern value by determining the occupancy of adjacent nodes of each node of an occupancy tree. The neighbor pattern is used to infer the occupancy pattern of the corresponding node. The left side of FIG. 7 illustrates a cube corresponding to the node (the cube located in the middle) and six cubes (neighboring nodes) that share at least one side with the corresponding cube. The illustrated nodes are nodes at the same depth. The illustrated numbers indicate the weights (1, 2, 4, 8, 16, 32, etc.) associated with each of the six nodes. Each weight is assigned in order according to the position of the neighboring node.

[0121] The right side of FIG. 7 shows the adjacent node pattern value. The adjacent node pattern value is the sum of values ​​multiplied by the weight values ​​of occupied adjacent nodes (adjacent nodes having points). Therefore, the adjacent node pattern value has a value from 0 to 63. An adjacent node pattern value of 0 means that there is no node (occupied node) that has points among the adjacent nodes of the corresponding node. An adjacent node pattern value of 63 means that all adjacent nodes are occupied nodes. As shown in the figure, adjacent nodes to which weight values ​​1, 2, 4, and 8 are assigned are occupied nodes, so the adjacent node pattern value is 15, which is a value obtained by adding 1, 2, 4, and 8. The point cloud encoder can perform coding according to the adjacent node pattern value (e.g., if the adjacent node pattern value is 63, 64 coding is performed). In an embodiment, the point cloud encoder can reduce the coding complexity by changing the adjacent node pattern value (e.g., based on a table that changes 64 to 10 or 6).

[0122] FIG. 8 is a diagram illustrating an example of a point configuration for each LOD according to the embodiment.

[0123] As described in Figures 1 to 7, before feature coding is performed, the coded geometry is reconstructed (restored). When direct coding is applied, the operation of geometry reconstruction includes changing the placement of the direct coded points (e.g., placing the direct coded points in front of the point cloud data). When trisoup geometry coding is applied, the process of geometry reconstruction includes the processes of triangulation, upsampling, and voxelization. Since features are dependent on geometry, feature coding is performed based on the reconstructed geometry.

[0124] A point cloud encoder (e.g., LOD generator 40009) reorganizes points by LOD. The diagram shows point cloud content corresponding to LOD. The left side of the diagram shows the original point cloud content. The second from the left in the diagram shows the distribution of points with the lowest LOD, and the rightmost side shows the distribution of points with the highest LOD. That is, the points with the lowest LOD have a sparse distribution, and the points with the highest LOD have a fine distribution. That is, as the LOD increases along the direction of the arrow shown at the bottom of the diagram, the spacing (or distance) between points becomes shorter.

[0125] FIG. 9 is a diagram illustrating an example of a point configuration for each LOD according to the embodiment.

[0126] As described in FIG. 1 to FIG. 8, a point cloud content providing system or point cloud encoder (e.g., point cloud video encoder 10002, point cloud encoder or LOD generator 40009 in FIG. 4) generates LOD. The LOD is generated by realigning points with a set of refinement levels according to a set LOD distance value (or a set of Euclidean Distances). The LOD generation process is performed not only in the point cloud encoder but also in the point cloud decoder.

[0127] The upper part of Figure 9 shows an example of points (P0 to P9) of point cloud content distributed in 3D space. The original order in Figure 9 shows the order of points P0 to P9 before LOD generation. The LOD based order in Figure 9 shows the order of points after LOD generation. Points are re-ordered for each LOD. Also, higher LODs include points that belong to lower LODs. As shown in Figure 9, LOD0 includes P0, P5, P4, and P2. LOD1 includes points of LOD0, P1, P6, and P3. LOD2 includes points of LOD0, points of LOD1, and P9, P8, and P7.

[0128] As described in FIG. 4, the point cloud encoder according to the embodiment can selectively or in combination perform predictive transform coding, lift transform coding, and RAHT transform coding.

[0129] The point cloud encoder according to the embodiment generates a predictor for each point and performs predictive conversion coding to set a prediction characteristic (or a prediction characteristic value) for each point. That is, N predictors are generated for N points. The predictor according to the embodiment can calculate a weight (=1 / distance) based on the LOD value of each point, index information for adjacent points within a distance set for each LOD, and a distance value to the adjacent point.

[0130] The predicted feature (or feature value) according to the embodiment is set as an average value of the feature (or feature value, e.g., hue, reflectance, etc.) of the neighboring points set in the predictor of each point multiplied by the weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., the coefficient quantization unit 40011) according to the embodiment may quantize and inverse quantize residuals (also called residual features, residual feature values, feature predicted residual values, etc.) obtained by subtracting the predicted feature (feature value) from the feature (feature value) of each point. The quantization process is as shown in the following table.

[0131] Attribute prediction residuals quantization pseudo code

[0132] int PCCQuantization(int value、int quantStep) {

[0133] if( value >=0) {

[0134] return floor(value / quantStep + 1.0 / 3.0);

[0135] } else {

[0136] return -floor(-value / quantStep + 1.0 / 3.0);

[0137] }

[0138] }

[0139] Attribute prediction residuals inverse quantization pseudo Code

[0140] int PCCInverseQuantization(int value、int quantStep) {

[0141] if( quantStep ==0) {

[0142] return value;

[0143] } else {

[0144] return value * quantStep;

[0145] }

[0146] }

[0147] The point cloud encoder (e.g., the arithmetic encoder 40012) according to the embodiment performs entropy coding on the quantized and dequantized residual values ​​as described above if there are adjacent points in the predictor of each point. If there are no adjacent points in the predictor of each point, the point cloud encoder (e.g., the arithmetic encoder 40012) according to the embodiment does not perform the above process and performs entropy coding on the characteristics of the corresponding point.

[0148] The point cloud encoder (e.g., lift transform unit 40010) according to the embodiment generates a predictor for each point, sets the calculated LOD in the predictor, registers adjacent points, and sets weights according to the distance to the adjacent points to perform lift transform coding. The lift transform coding according to the embodiment is similar to the above-mentioned point transform coding, but differs in that weights are cumulatively applied to feature values. The process of cumulatively applying weights to feature values ​​according to the embodiment is as follows.

[0149] 1) Create an array QW (QuantizationWeight) to store the weight value of each point. The initial value of all elements of QW is 1.0. The QW value of the predictor index of the adjacent node registered in the predictor is multiplied by the predictor weight value of the current point and added.

[0150] 2) Lift prediction process: To calculate the predicted attribute value, the attribute value of the point is multiplied by the weighting value and subtracted from the existing attribute value.

[0151] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to 0.

[0152] 4) The calculated weights for all predictors are multiplied by the weights stored in the QW corresponding to the predictor index, and the calculated weights are accumulated and summed in the update weight array as the index of the adjacent node. The update array is accumulated and summed by multiplying the attribute values ​​of the index of the adjacent node by the calculated weights.

[0153] 5) Lift update process: For every predictor, divide the feature value in the update array by the weight value in the update weight array for the predictor index, and then add the divided value back to the existing feature value.

[0154] 6) For all predictors, the feature values ​​updated in the lift update process are further multiplied by the weights updated in the lift prediction process (stored in the QW) to calculate predicted feature values. A point cloud encoder (e.g., coefficient quantizer 40011) according to the embodiment quantizes the predicted feature values. In addition, a point cloud encoder (e.g., arithmetic encoder 40012) entropy codes the quantized feature values.

[0155] The point cloud encoder (e.g., the RAHT converter 40008) according to the embodiment performs RAHT transform coding, which predicts the features of higher level nodes using features associated with lower level nodes of the octree. RAHT transform coding is an example of feature intra-coding using octree backward scan. The point cloud encoder according to the embodiment scans the entire region from the voxel, and repeats the merging process up to the root node while combining the voxels into larger blocks at each step. The merging process according to the embodiment is performed only for occupied nodes. The merging process is not performed for empty nodes, but is performed for the nodes immediately above the empty nodes.

[0156] The following formula shows the RAHT transformation matrix: TIFF0007640526000004.tif1023 shows the average feature value of voxels at level l. TIFF0007640526000005.tif922 TIFF0007640526000006.tif1031 and Calculated from TIFF0007640526000007.tif1137. TIFF0007640526000008.tif1135 and The weighting for TIFF0007640526000009.tif1130 is TIFF0007640526000010.tif944 and The file is TIFF0007640526000011.tif1048.

[0157] TIFF0007640526000012.tif3993

[0158] TIFF0007640526000013.tif826 is the low-pass value that will be used in the next higher level merging process. TIFF0007640526000014.tif1025 is the high-pass coefficients, and the high-pass coefficients at each step are quantized and entropy coded (e.g., the coding of the computational encoder 400012). The weights are The root node is calculated using the last TIFF0007640526000016.tif1122 and TIFF0007640526000017.tif1021 generates the following:

[0159] TIFF0007640526000018.tif2899

[0160] The gDC values ​​are also quantized and entropy coded like the high-pass coefficients.

[0161] FIG. 10 is a diagram illustrating an example of a point cloud decoder according to an embodiment.

[0162] The point cloud decoder shown in FIG. 10 is an example of the point cloud video decoder 10006 shown in FIG. 1, and performs the same or similar operations as the point cloud video decoder 10006 described in FIG. 1. As shown, the point cloud decoder receives a geometry bitstream and an attribute bitstream included in one or more bitstreams. The point cloud decoder includes a geometry decoder and an attribute decoder. The geometry decoder performs geometry decoding on the geometry bitstream and outputs a decoded geometry. The attribute decoder performs attribute decoding based on the decoded geometry and attribute bitstream and outputs decoded attributes. The decoded geometry and decoded attributes are used to restore the point cloud content (decoded point cloud).

[0163] FIG. 11 is a diagram illustrating an example of a point cloud decoder according to an embodiment.

[0164] The point cloud decoder shown in FIG. 11 is an example of the point cloud decoder described in FIG. 10, and performs a decoding operation which is the reverse process of the encoding operation of the point cloud encoder described in FIGS. 1 to 9.

[0165] As explained in Figures 1 and 10, the point cloud decoder performs geometry decoding and feature decoding. Geometry decoding is performed before feature decoding.

[0166] The point cloud decoder according to the embodiment includes an arithmetic decoder 11000, an octree synthesis unit 11001, a surface approximation synthesis unit 11002, a reconstruct geometry unit 11003, an inverse transform coordinates unit 11004, an arithmetic decoder 11005, an inverse quantize unit 11006, a RAHT transform unit 11007, a generate LOD unit 11008, an inverse lifting unit 11009 and / or an inverse transform colors unit 11010.

[0167] The arithmetic decoder 11000, the octree synthesis unit 11001, the surface approximation synthesis unit 11002, the geometry reconstruction unit 11003, and the coordinate system inverse transformation unit 11004 perform geometry decoding. The geometry decoding according to the embodiment includes direct coding and trisoup geometry decoding. The direct coding and trisoup geometry decoding are selectively applied. Furthermore, the geometry decoding is not limited to the above examples, and is performed in the reverse process of the geometry encoding described in FIG. 1 to FIG. 9.

[0168] The arithmetic decoder 11000 according to the embodiment decodes the received geometry bitstream based on arithmetic coding. The operation of the arithmetic decoder 11000 corresponds to the inverse process of the arithmetic encoder 40004.

[0169] The octree synthesis unit 11001 according to the embodiment obtains an exclusive code from the decoded geometry bitstream (or from the decoded result, information on the secured geometry) to generate an octree. The exclusive code is described in detail in FIG. 1 to FIG. 9.

[0170] The surface approximation synthesis unit 11002 according to the embodiment synthesizes a surface based on the decoded geometry and / or the generated octree if trisoup geometry encoding is applied.

[0171] The geometry reconstruction unit 11003 according to the embodiment regenerates geometry based on the surface and / or the decoded geometry. As described in FIG. 1 to FIG. 9, direct coding and trisoup geometry coding are selectively applied. Therefore, the geometry reconstruction unit 11003 directly brings and adds position information of the point to which the direct coding is applied. Also, when the trisoup geometry coding is applied, the geometry reconstruction unit 11003 restores the geometry by performing the reconstruction operation of the geometry reconstruction unit 40005, for example, triangulation, upsampling, and voxelization operations. The detailed contents are omitted since they are the same as those described in FIG. 6. The restored geometry includes a point cloud picture or frame that does not include features.

[0172] A coordinate system inverse transformation unit 11004 according to the embodiment transforms the coordinate system based on the reconstructed geometry to obtain the position of the point.

[0173] The arithmetic decoder 11005, the inverse quantization unit 11006, the RAHT transform unit 11007, the LOD generating unit 11008, the inverse lift unit 11009 and / or the color inverse transform unit 11010 perform the feature decoding described in FIG. 10. The feature decoding according to the embodiment includes RAHT (Region Adaptive Hierarchial Transform) decoding, Interpolarization-based hierarchical nearest-neighbour prediction-Prediction Transform decoding, and Lift Transform (interpolation-based hierarchical nearest-neighbour prediction with an update / lifting step (Lifting Transform)) decoding. The above three decodings are used selectively, or one or more combinations of decodings are used. In addition, the feature decoding according to the embodiment is not limited to the above examples.

[0174] An arithmetic decoder 11005 according to an embodiment decodes the attribute bitstream into arithmetic coding.

[0175] The inverse quantization unit 11006 according to the embodiment inverse quantizes the decoded feature bitstream or the information on the feature acquired as a result of decoding, and outputs the inverse quantized feature (or feature value). The inverse quantization is selectively applied based on the feature coding of the point cloud encoder.

[0176] In an embodiment, the RAHT transformer 11007, LOD generator 11008 and / or inverse lifter 11009 process the reconstructed geometry and dequantized features. As described above, the RAHT transformer 11007, LOD generator 11008 and / or inverse lifter 11009 selectively perform decoding operations corresponding to the encoding of the point cloud encoder.

[0177] The color inverse converter 11010 according to the embodiment performs inverse coding to inversely convert color values ​​(or textures) included in the decoded features. The operation of the color inverse converter 11010 is selectively performed based on the operation of the color converter 40006 of the point cloud encoder.

[0178] The elements of the point cloud decoder of Figure 11 may be implemented in hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits (not shown) configured to communicate with one or more memories included in the point cloud providing device. The one or more processors perform any of the operations and / or functions of the elements of the point cloud decoder of Figure 11 described above. Additionally, the one or more processors operate or execute a software program and / or set of instructions to perform the operations and / or functions of the elements of the point cloud decoder of Figure 11.

[0179] FIG. 12 illustrates an example of a transmitting device according to an embodiment.

[0180] The transmitting device shown in FIG. 12 is an example of the transmitting device 10000 in FIG. 1 (or the point cloud encoder in FIG. 4). The transmitting device shown in FIG. 12 performs any of the same or similar operations and methods as the operations and encoding methods of the point cloud encoder described in FIG. 1 to FIG. 9. The transmitting device according to the embodiment includes a data input unit 12000, a quantization processing unit 12001, a voxelization processing unit 12002, an octree occupation code generation unit 12003, a surface model processing unit 12004, an intra / inter coding processing unit 12005, an arithmetic coder 12006, a metadata processing unit 12007, a hue conversion processing unit 12008, a characteristic conversion processing unit (or attribute conversion processing unit) 12009, a prediction / lift / RAHT conversion processing unit 12010, an arithmetic coder 12011 and / or a transmission processing unit 12012.

[0181] According to the embodiment, the data input unit 12000 receives or acquires point cloud data. The data input unit 12000 performs an operation and / or an acquisition method that is the same as or similar to the operation and / or acquisition method of the point cloud video acquisition unit 10001 (or the acquisition process 20000 shown in FIG. 2).

[0182] Geometry encoding is performed by a data input unit 12000, a quantization processing unit 12001, a voxelization processing unit 12002, an octree occupation code generation unit 12003, a surface model processing unit 12004, an intra / inter coding processing unit 12005, and an arithmetic coder 12006. The geometry encoding according to the embodiment is the same as or similar to the geometry encoding described with reference to Figures 1 to 9, and therefore a detailed description thereof will be omitted.

[0183] The quantization unit 12001 according to the embodiment quantizes geometry (e.g., position values ​​of points). The operation and / or quantization of the quantization unit 12001 is the same as or similar to the operation and / or quantization of the quantization unit 40001 shown in Fig. 4. The specific description is as described in Figs. 1 to 9.

[0184] The voxelization processing unit 12002 according to the embodiment voxels the position values ​​of the quantized points. The voxelization processing unit 120002 performs operations and / or processes that are the same as or similar to the operations and / or voxelization processes of the quantization unit 40001 shown in Fig. 4. The details are as described in Figs. 1 to 9.

[0185] The octree occupation code generator 12003 according to the embodiment performs octree coding on the positions of the voxelized points based on the octree structure. The octree occupation code generator 12003 generates an occupation code. The octree occupation code generator 12003 performs operations and / or methods that are the same as or similar to the operations and / or methods of the point cloud encoder (or the octree analysis unit 40002) described in Figures 4 and 6. The details are as described in Figures 1 to 9.

[0186] The surface model processing unit 12004 according to the embodiment performs trisoup geometry encoding to reconstruct the positions of points within a specific region (or node) on a voxel basis based on a surface model. The surface model processing unit 12004 performs operations and / or methods that are the same as or similar to those of the point cloud encoder (e.g., the surface approximation analysis unit 40003) shown in Fig. 4. The details are as described in Figs. 1 to 9.

[0187] The intra / inter coding unit 12005 according to the embodiment performs intra / inter coding of the point cloud data. The intra / inter coding unit 12005 performs coding that is the same as or similar to the intra / inter coding described in FIG. 7. The details are as described in FIG. 7. In the embodiment, the intra / inter coding unit 12005 is included in the arithmetic coder 12006.

[0188] According to an embodiment, the arithmetic coder 12006 entropy codes the octree and / or the approximated octree of the point cloud data. For example, the encoding method includes an arithmetic encoding method. The arithmetic coder 12006 performs operations and / or methods that are the same as or similar to those of the arithmetic encoder 40004.

[0189] The metadata processing unit 12007 according to the embodiment processes metadata related to point cloud data, such as setting values, and provides the metadata to necessary processing such as geometry encoding and / or feature encoding. The metadata processing unit 12007 according to the embodiment also generates and / or processes signaling information related to geometry encoding and / or feature encoding. The signaling information according to the embodiment is encoded and processed separately from the geometry encoding and / or feature encoding. The signaling information according to the embodiment may also be interleaved.

[0190] The hue conversion processor 12008, the feature conversion processor 12009, the prediction / lift / RAHT conversion processor 12010, and the arithmetic coder 12011 perform feature coding. The feature coding according to the embodiment is the same as or similar to the feature coding described in Figs. 1 to 9, so a detailed description thereof will be omitted.

[0191] The hue conversion processing unit 12008 according to the embodiment performs hue conversion coding to convert a hue value included in a feature. The hue conversion processing unit 12008 performs hue conversion coding based on the reconstructed geometry. The reconstructed geometry is as described in FIG. 1 to FIG. 9. The hue conversion processing unit 12008 performs operations and / or methods that are the same as or similar to the operations and / or methods of the color conversion unit 40006 described in FIG. 4. Detailed description will be omitted.

[0192] The feature conversion processing unit 12009 according to the embodiment performs feature conversion to convert features based on positions where geometry coding has not been performed and / or reconstructed geometry. The feature conversion processing unit 12009 performs the same or similar operation and / or method as the feature conversion unit 40007 described in FIG. 4. Detailed description will be omitted. The prediction / lift / RAHT conversion processing unit 12010 according to the embodiment codes the converted features by one or a combination of RAHT coding, predictive conversion coding, and lift conversion coding. The prediction / lift / RAHT conversion processing unit 12010 performs one of the same or similar operations as the RAHT conversion unit 40008, the LOD generating unit 40009, and the lift conversion unit 40010 described in FIG. 4. In addition, the predictive conversion coding, lift conversion coding, and RAHT conversion coding are the same as those described in FIG. 1 to FIG. 9, so detailed description will be omitted.

[0193] The arithmetic coder 12011 according to the embodiment encodes the coded attribute based on arithmetic coding. The arithmetic coder 12011 performs operations and / or methods that are the same as or similar to the operations and / or methods of the arithmetic encoder 400012.

[0194] The transmission processing unit 12012 according to the embodiment transmits each bitstream including the coded geometry and / or coded attribute and metadata information, or transmits the coded geometry and / or coded attribute and metadata information in one bitstream. When the coded geometry and / or coded attribute and metadata information according to the embodiment is configured in one bitstream, the bitstream includes one or more sub-bitstreams. The bitstream according to the embodiment includes signaling information including a Sequence Parameter Set (SPS) for sequence level signaling, a Geometry Parameter Set (GPS) for signaling geometry information coding, an Attribute Parameter Set (APS) for signaling attribute information coding, and a Tile Parameter Set (TPS) for tile level signaling, and slice data. The slice data includes information about one or more slices. One slice according to the embodiment includes one geometry bitstream (Geometry Parameter Set (GPS) for signaling geometry information coding, an Attribute Parameter Set (APS) for signaling attribute information coding, and a Tile Parameter Set (TPS) for tile level signaling. 0 ) and one or more attribute bit streams (Attr0 0 , Attr1 0 ).

[0195] According to an embodiment, a TPS includes information about one or more tiles (eg, bounding box coordinate information and height / size information, etc.).

[0196] The geometry bitstream includes a header and a payload. The header of the geometry bitstream according to the embodiment includes identification information of a parameter set included in the GPS (geom_parameter_set_id), a tile identifier (geom_tile_id), a slice identifier (geom_slice_id), and information about data included in the payload. As described above, the metadata processing unit 12007 according to the embodiment can generate and / or process signaling information and transmit it to the transmission processing unit 12012. In the embodiment, the element performing geometry coding and the element performing attribute coding can share data / information with each other as shown by the dotted line. The transmission processing unit 12012 according to the embodiment performs an operation and / or a transmission method that is the same as or similar to the operation and / or transmission method of the transmitter 10003. A detailed description is omitted as it is the same as that described in FIG. 1 and FIG. 2.

[0197] FIG. 13 illustrates an example of a receiving device according to the embodiment.

[0198] The receiving device shown in Fig. 13 is an example of the receiving device 10004 in Fig. 1 (or the point cloud decoder in Figs. 10 and 11). The receiving device shown in Fig. 13 performs any of the same or similar operations and methods as the operations and decoding methods of the point cloud decoder described in Figs. 1 to 11.

[0199] The receiving device according to the embodiment includes a receiving unit 13000, a receiving processing unit 13001, an arithmetic decoder 13002, an occupancy code based octree reconstruction processing unit 13003, a surface model processing unit (triangle reconstruction, upsampling, voxelization) 13004, an inverse quantization processing unit 13005, a metadata analysis 13006, an arithmetic decoder 13007, an inverse quantization processing unit 13008, a prediction / lift / RAHT inverse transform processing unit 13009, a hue inverse transform processing unit 13010, and / or a renderer 13011. Each component of the decoding according to the embodiment performs the inverse process of the component of the encoding according to the embodiment.

[0200] The receiving unit 13000 according to the embodiment receives point cloud data. The receiving unit 13000 performs an operation and / or a receiving method that is the same as or similar to the operation and / or the receiving method of the receiver 10005 of Fig. 1. A detailed description will be omitted.

[0201] The receiving processor 13001 according to the embodiment obtains a geometry bit stream and / or an attribute bit stream from the received data. The receiving processor 13001 is included in the receiving unit 13000.

[0202] Geometry decoding is performed by an arithmetic decoder 13002, an exclusive code based octree reconstruction processor 13003, a surface model processor 13004, and an inverse quantization processor 13005. The geometry decoding according to the embodiment is the same as or similar to the geometry decoding described in Figures 1 to 10, so a detailed description will be omitted.

[0203] The computation decoder 13002 according to the embodiment decodes the geometry bitstream based on computation coding. The computation decoder 13002 performs operations and / or coding that are the same as or similar to those of the computation decoder 11000.

[0204] The exclusive code based octree reconstruction processor 13003 according to the embodiment obtains an exclusive code from the decoded geometry bitstream (or from the decoded result, information on the secured geometry) and reconstructs an octree. The exclusive code based octree reconstruction processor 13003 performs an operation and / or method identical or similar to the operation of the octree synthesis unit 11001 and / or an octree generation method. The surface model processor 13004 according to the embodiment performs trisoup geometry decoding and associated geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on a surface model scheme when trisoup geometry encoding is applied. The surface model processor 13004 performs an operation identical or similar to the operation of the surface approximation synthesis unit 11002 and / or the geometry reconstruction unit 11003.

[0205] The inverse quantization unit 13005 according to the embodiment inverse quantizes the decoded geometry.

[0206] According to an embodiment, the metadata analysis 13006 analyzes metadata, such as setting values, included in the received point cloud data. The metadata analysis 13006 transmits the metadata to the geometry decoder and / or the feature decoder. A detailed description of the metadata is omitted here as it has been described in FIG. 12.

[0207] The arithmetic decoder 13007, the inverse quantization processor 13008, the prediction / lift / RAHT inverse transform processor 13009, and the hue inverse transform processor 13010 perform feature decoding. The feature decoding is the same as or similar to the feature decoding described in FIG. 1 to FIG. 10, so a detailed description thereof will be omitted.

[0208] The computation decoder 13007 according to the embodiment decodes the attribute bitstream into computation coding. The computation decoder 13007 performs decoding of the attribute bitstream based on the reconstructed geometry. The computation decoder 13007 performs operations and / or coding that are the same or similar to those of the computation decoder 11005.

[0209] The inverse quantization unit 13008 according to the embodiment inversely quantizes the decoded characteristic bit stream. The inverse quantization unit 13008 performs the same or similar operation and / or method as the inverse quantization unit 11006.

[0210] The prediction / lift / RAHT inverse transform processing unit 13009 according to the embodiment processes the reconstructed geometry and the inverse quantized features. The prediction / lift / RAHT inverse transform processing unit 13009 performs any of the same or similar operations and / or decoding as those of the RAHT transform unit 11007, the LOD generating unit 11008, and / or the inverse lift unit 11009. The hue inverse transform processing unit 13010 according to the embodiment performs inverse transform coding for inversely transforming color values ​​(or textures) included in the decoded features. The hue inverse transform processing unit 13010 performs the same or similar operations and / or inverse transform coding as those of the color inverse transform unit 11010. The renderer 13011 according to the embodiment renders the point cloud data.

[0211] FIG. 14 illustrates an example of a structure that can be linked to a point cloud data transmitting / receiving method / apparatus according to an embodiment.

[0212] 14 illustrates a configuration in which any of a server 1460, a robot 1410, an autonomous vehicle 1420, an XR device 1430, a smartphone 1440, a home appliance 1450, and / or an HMD 1470 are connected to a cloud network 1410. The robot 1410, the autonomous vehicle 1420, the XR device 1430, the smartphone 1440, or the home appliance 1450 may also be referred to as an apparatus. The XR device 1430 may correspond to a point cloud data (PCC) device or may be linked to a PCC device according to an embodiment.

[0213] Cloud network 1400 refers to a network that constitutes a part of a cloud computing infrastructure or exists within a cloud computing infrastructure, where cloud network 1400 is configured using a 3G network, a 4G or LTE network, a 5G network, or the like.

[0214] The server 1460 may be coupled to any of the robot 1410, autonomous vehicle 1420, XR device 1430, smartphone 1440, home appliance 1450, and / or HMD 1470 via the cloud network 1400 and may assist with at least some of the processing of the coupled devices 1410-1470.

[0215] The HMD (Head-Mount Display) 1470 represents any of the types of XR devices and / or PCC devices according to the embodiment. The HMD type device according to the embodiment includes a communications unit, a control unit, a memory unit, an I / O unit, a sensor unit, and a power supply unit.

[0216] Various embodiments of the devices 1410 to 1450 to which the above-mentioned technology is applied will be described below. Here, the devices 1410 to 1450 shown in Fig. 14 can be linked / coupled to the point cloud data transmitting / receiving device according to the above-mentioned embodiment.

[0217] <PCC+XR>

[0218] The XR / PCC device 1430 may be embodied in a Head-Mount Display (HMD), a Head-Up Display (HUD) installed in a vehicle, a TV, a mobile phone, a smartphone, a computer, a wearable device, a home appliance, a digital sign, a vehicle, a fixed robot, a mobile robot, etc. by applying PCC and / or XR (AR+VR) technology.

[0219] The XR / PCC device 1430 may obtain information about the surrounding space or real objects by analyzing 3D point cloud data or image data obtained by various sensors or from an external device to generate position data and attribute data for 3D points, and may render and output the XR object to be output. For example, the XR / PCC device 1430 may output an XR object including additional information about the recognized object in correspondence with the corresponding recognized object.

[0220] <PCC+XR+Mobile Phone>

[0221] The XR / PCC device 1430 is embodied as a mobile phone 1440 to which PCC technology is applied.

[0222] The mobile phone 1440 can decode and display point cloud content based on PCC technology.

[0223] <PCC+Autonomous Driving+XR>

[0224] The autonomous vehicle 1420 is realized as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0225] The autonomous vehicle 1420 to which the XR / PCC technology is applied refers to an autonomous vehicle equipped with a means for providing an XR image, an autonomous vehicle that is the target of control / interaction within the XR image, etc. In particular, the autonomous vehicle 1420 that is the target of control / interaction within the XR image may be separated from the XR device 1430 and may be linked to each other.

[0226] The autonomous vehicle 1420 equipped with a means for providing an XR / PCC image obtains sensor information from sensors including a camera, and outputs an XR / PCC image generated based on the obtained sensor information. For example, the autonomous vehicle 1420 may provide the passenger with an XR / PCC object corresponding to a real object or an object on a screen by outputting an XR / PCC image through a HUD.

[0227] At this time, when the XR / PCC object is output to the HUD, at least a part of the XR / PCC object is output to overlap with an actual object toward which the passenger's gaze is directed. On the other hand, when the XR / PCC object is output to a display provided in the autonomous vehicle, at least a part of the XR / PCC object is output to overlap with an object in the screen. For example, the autonomous vehicle 1220 may output XR / PCC objects corresponding to objects such as a road, another vehicle, a traffic light, a traffic sign, a motorcycle, a pedestrian, a building, etc.

[0228] The VR (Virtual Reality) technology, the AR (Augmented Reality) technology, the MR (Mixed Reality) technology, and / or the PCC (Point Cloud Compression) technology according to the embodiment can be applied to various devices.

[0229] That is, VR technology is a display technology that provides real objects and backgrounds only in CG images. On the other hand, AR technology is a technology that displays virtual CG images on top of images of real things. Also, MR technology is similar to the AR technology in that it mixes virtual objects into the real world. However, AR technology clearly distinguishes between real objects and virtual objects made of CG images, and uses virtual objects to complement real objects, while MR technology is different from AR technology in that virtual objects and real objects are considered to have the same characteristics. More specifically, for example, the MR technology is applied to hologram services.

[0230] However, recently, rather than clearly distinguishing between VR, AR, and MR technologies, they are also referred to as XR (extended reality) technology. Therefore, the embodiments of the present invention can be applied to any of VR, AR, MR, and XR technologies. These technologies are applied with encoding / decoding based on PCC, V-PCC, and G-PCC technologies.

[0231] The PCC method / apparatus according to the embodiment can be applied to vehicles providing autonomous driving services.

[0232] Vehicles providing autonomous driving services are connected to the PCC device via wired or wireless communication.

[0233] When the point cloud data (PCC) transceiver according to the embodiment is connected to a vehicle for wired or wireless communication, it can receive / process AR / VR / PCC service related content data that can be provided along with an autonomous driving service and transmit it to the vehicle. Also, when the point cloud data transceiver is installed in a vehicle, the point cloud transceiver can receive / process AR / VR / PCC service related content data according to a user input signal inputted to a user interface device and provide it to a user. The vehicle or user interface device according to the embodiment receives a user input signal. The user input signal according to the embodiment includes a signal instructing an autonomous driving service.

[0234] FIG. 15 illustrates a point cloud encoder according to an embodiment.

[0235] The point cloud encoder 15000 according to the embodiment receives point cloud data (PCC data, 15000a) and encodes them. The point cloud encoder according to the embodiment outputs a geometric information bitstream 15000b and an attribute information bitstream 15000c. The point cloud encoder 15000 according to the embodiment includes a spatial division unit 15001, a geometric information encoding unit 15002, and / or an attribute information encoding unit 15003.

[0236] The point cloud encoder of FIG. 15 corresponds to or is coupled to the transmitting device 1000 of FIG. 1, the point cloud video encoder 10002, the encoding 18001 of FIG. 2, the encoder of FIG. 4, the transmitting device of FIG. 12, and the XR device 1430 of FIG. 14, etc.

[0237] The spatial division unit 15001 of the point cloud encoder receives point cloud data (PCC data, 15000a) and divides the point cloud data into one or more 3D spaces. The spatial division unit 15001 receives the point cloud data and spatially divides the point cloud data into 3D blocks. The point cloud data includes geometric information and / or attribute information of a point (or multiple points). The spatial division unit 15001 spatially divides the point cloud data (PCC data) based on a bounding box and / or a sub-bounding box, etc. The method / apparatus according to the embodiment performs encoding / decoding based on the divided units (boxes).

[0238] The transmitting device (e.g., a space dividing unit) according to the embodiment can divide the point cloud data in a three-dimensional space into units that can be processed in parallel in order to compress the point cloud data. The space dividing unit 15001 according to the embodiment can divide the point cloud data into slice units so that the receiving device can support a user's scalability and / or region of interest (ROI) functions.

[0239] The transmitting device according to the embodiment may divide the point cloud data according to the embodiment into slices (also called bricks) or tiles in order to achieve low latency, low complexity, temporal scalability, spatial scalability, region of interest-based spatial scalability, quality scalability, parallel encoding, and parallel decoding by the receiving device. Therefore, the transmitting device according to the embodiment needs to perform an operation to support scalability adjustment, quantum value adjustment, etc. in slice units. Therefore, the point cloud data transmitting device according to the embodiment may configure overlapped slices so that overlapping areas between slices are included. A method in which the point cloud data transmitting device according to the embodiment divides the point cloud data into one or more slices (or overlapped slices) will be described in detail below.

[0240] A slice means a unit (or a minimum unit) for the point cloud data transmitting device according to the embodiment to encode and / or decode point cloud data, and the point cloud data transmitting device and / or receiving device can independently encode and / or decode point cloud data in slice units. A slice according to the embodiment is also called a brick, a block, or a macroblock.

[0241] The space division unit 15001 can perform part or all of the operations of the point cloud acquisition 10001 in FIG. 1, the acquisition 20000 in FIG. 2, the operations according to FIGS. 3 to 5, and the data input unit 12000 in FIG.

[0242] The geometric information encoder 15002 receives and encodes geometry information of point cloud data (PCC data) according to an embodiment. The geometry information means position information of points included in the point cloud data. The geometric information encoder 15002 encodes the geometry information and outputs a geometric information bitstream. The geometric information encoder 15002 can reconstruct the position information of the points and output restored geometric information 15002a. The geometric information encoder 15002 transmits the restored geometric information to the attribute information encoder 15002.

[0243] The geometric information encoding unit 15002 performs some / all of the operations of the Point Cloud Video Encoder 10002 in Figure 1, the encoding 20001 in Figure 2, the coordinate system conversion unit 40000, quantization 40001, octree analysis unit 40002, surface approximation analysis unit 40003, arithmetic encoder 40004, geometry reconstruction unit 40005 in Figure 4, and the quantization processing unit 12001, voxelization processing unit 12002, octree occupancy code generation unit 12003, surface model processing unit 12004, intra / inter coding processing unit 12005 and / or arithmetic coder 12006 in Figure 12.

[0244] The attribute information encoding unit 15003 receives attribute information of the point cloud data according to an embodiment, and encodes the attribute information using the restored geometric information received from the geometric information encoding unit 15003. The attribute information encoding unit 15003 encodes the attribute information and outputs an attribute information bit stream 15000c. The attribute information encoding unit 15003 performs, for example, a prediction transform, a lifting transform, and / or a Region Adaptive Hierarchical Transform (RAHT) transform according to an embodiment. The attribute information encoding unit 15003 performs, for example, a prediction lifting transform. The prediction lifting transform means a combination of some or all of the detailed operations of the prediction transform and / or the lifting transform according to an embodiment.

[0245] The point cloud encoder according to the embodiment performs encoding using some, all and / or a combination of a prediction transform, a lifting transform and / or a Region Adaptive Hierarchical Transform (RAHT) transform according to the embodiment.

[0246] The attribute information encoding unit 15003 performs all / part of the operations of the point cloud video encoder 10002 in Figure 1, the encoding 20001 in Figure 2, the color conversion unit 40006, attribute conversion unit 40007, RATH conversion unit 40008, LOD generation unit 40009, lift conversion unit 40010, coefficient quantization unit 40011 and / or the arithmetic encoding unit 40012 in Figure 4, and the operations of the hue conversion processing unit 12008, attribute conversion processing unit 12009, prediction / lift / RAHT conversion processing unit 12010 and arithmetic coder 12011 in Figure 12.

[0247] Here, the reconstructed geometric information 15002c means an octree reconstructed and / or an approximated octree by the geometry reconstruction unit 40005 described in Fig. 4. The reconstructed geometric information means an octree structure or an exclusive code described in Fig. 6. The reconstructed geometric information may also mean an octree exclusive code generated by the octree exclusive code generation unit 12003 described in Fig. 12.

[0248] The attribute information encoding unit 15003 encodes attribute information of point cloud data according to an embodiment. Here, the encoding unit 15003 according to an embodiment encodes the attribute information using reconstructed geometric information (or reconstructed geometry information) according to an embodiment. The attribute information encoding unit 15003 can encode received data to generate a bitstream including attribute information (or attribute information).

[0249] The attribute information encoding unit 15003 according to the embodiment includes a hue conversion unit 40006, an attribute transmission unit 40007, a RAHT conversion unit 40008, an LOD generation unit 40009, a lift unit 40010, a quantization unit 40011 and / or a arithmetic encoding unit 400012 of FIG.

[0250] Point cloud data according to the embodiment is classified into Category 1 and Category 3 according to the characteristics of the data. Category 1 data is static data consisting of one frame. Category 3 data is dynamic data consisting of N frames. A ply file, which is the file format of point cloud data, consists of multiple points depending on the data acquisition method.

[0251] The points according to the embodiment include attribute information (characteristic information) including position information (geometry information) of the corresponding point, color information, reflectance information, time information, normal vector information, etc. The points according to the embodiment include various information depending on the conditions to be expressed. Category 1 and Category 3 data consisting of points include frames including a large amount of points. However, when the point cloud data transmitting device according to the embodiment receives and encodes frames including such points, there is a problem in that delay time and unnecessary resources are used.

[0252] Therefore, in the point cloud data transmission device according to the embodiment, the point cloud data transmission operation, the encoding operation, the receiving device's decoding operation, and the receiving device's rendering processing operation are performed in real time, and the point cloud data is divided into a plurality of regions in order to process with low delay. In the point cloud data transmission device according to the embodiment, a frame of the point cloud data is divided into tiles, slices, and / or bricks.

[0253] Therefore, in the point cloud data transmitting device according to the embodiment, the point cloud data can be divided by an independent data division method for improving the parallel processing ability and scalability of 3D point cloud data. When performing independent data division, a spatially adaptive division method can be included by supporting a method in which points close in space are re-divided.

[0254] The point cloud data transmission device according to the embodiment provides the effect of dividing the point cloud data and encoding each of the divided data independently, thereby enabling random access and parallel encoding in the three-dimensional space occupied by the point cloud data.

[0255] FIG. 16 illustrates a point cloud encoder according to an embodiment.

[0256] The point cloud encoder according to the embodiment includes a data characteristic investigation unit 16000, a brick tiling unit 16001, a divided data input unit 16002, a coordinate conversion unit 16003, a quantization / voxelization processing unit 16004, an octree occupancy code generation unit 16005, a surface model processing unit 16006, a first arithmetic coder 16007, a geometry reconstruction unit 16008, a hue conversion processing unit 16009, an attribute conversion processing unit 16010, a prediction / lift / RAHT conversion processing unit 16011, a coefficient quantization processing unit 16012, a second arithmetic coder 16013 and / or a bitstream combination unit 16014.

[0257] The point cloud encoder shown in FIG. 16 corresponds to or is coupled to the transmitting device 1000, point cloud video encoder 10002, encoding 18001 in FIG. 2, encoder in FIG. 4, transmitting device in FIG. 12, XR device 1430 in FIG. 14, etc.

[0258] The data characteristic investigation unit 16000, brick tiling unit 16001, and divided data input unit 16002 of Figure 16 are components included in the point cloud video acquisition 10001 and point cloud video encoder 10002 of Figure 1, acquisition 18000 and encoding 18002 of Figure 2, data input unit 12000 of Figure 12, and spatial division unit 15001 of Figure 15.

[0259] The coordinate conversion unit 16003 is the conversion coordinate unit 38000 in Fig. 4 or the quantization processing unit 12001 in Fig. 12. The quantization / voxelization processing unit 16004 is the quantization and point removal unit (voxelization unit) 38001 in Fig. 4 and the voxelization processing unit 12002 in Fig. 12, and performs the operations in Figs. 5 to 7. The octree occupation code generation unit 16005 is the octree analysis unit 38002 in Fig. 4 and the octree occupation code generation unit 12003 in Fig. 4, and performs the operations in Figs. 6 and 7. The surface model processing unit 16006 is the surface approximation analysis unit 38003 in Fig. 4 and the surface model processing unit 12004 in Fig. 12. The first arithmetic coder 16007 and the geometry reconstruction unit 16008 are the arithmetic encoding unit 38004 in Fig. 4 and the arithmetic coder 12006 in Fig. 12. The hue conversion processing unit 16009 is the hue conversion unit 38006 in Fig. 4 and the hue conversion processing unit 12008 in Fig. 12. The attribute conversion processing unit 16010 is the characteristic transmission unit 38007 in Fig. 4 and the attribute conversion processing unit 12009 in Fig. 12. The prediction / lift / RAHT conversion processing unit 16011 is all or a part of the RAHT 38008, LOD generation unit 38009, and lift unit 40010 in Fig. 4, and is the prediction / lift / RAHT conversion processing unit 12010 in Fig. 12. The coefficient quantization processing unit 16012 and the second arithmetic coder 16013 are the coefficient quantization unit 40011 in Fig. 4, the arithmetic encoding unit 40012, and the arithmetic coder 12011 in Fig. 12.

[0260] The coordinate conversion unit 16003, the quantization / voxelization processing unit 16004, the octree occupation code generation unit 16005, the surface model processing unit 16006, the first arithmetic coder 16007, and the geometry reconstruction unit 16008 in Fig. 16 are included in the geometric information encoding unit 15002 in Fig. 15. The geometry reconstruction unit 16008, the hue conversion processing unit 16009, the attribute conversion processing unit 16010, the prediction / lift / RAHT conversion processing unit 16011, the coefficient quantization processing unit 16012, and the second arithmetic coder 16013 in Fig. 16 are included in the attribute information encoding unit 15003 in Fig. 15.

[0261] The point cloud data encoder according to the embodiment shows an example of a point cloud data encoder 15000 according to the embodiment in Fig. 15. The encoder and the encoder according to the embodiment are called an encoder, and the decoder and the decoder are called a decoder.

[0262] The data characteristic examination unit 16000 receives point cloud data according to the embodiment and examines the characteristics of the received point cloud data. For example, the data characteristic examination unit 16000 examines whether points of the point cloud data are densely distributed in a specific area and sparsely distributed in a specific area. The data characteristic examination unit 16000 examines whether the point cloud data is composed of a point cloud sequence including a plurality of frames, such as LiDAR data or data for autonomous driving. The data characteristic examination unit 16000 examines whether point cloud data in one frame is concentrated in a specific area (for example, when consecutive attribute information values ​​are concentrated on the left side, right side, upper side, lower side, front side, or rear side). The data characteristic examination unit 16000 examines whether objects represented by the point cloud data are sparsely present from each other. The data characteristic examination unit 16000 examines whether a three-dimensional space occupied by the point cloud data extends in one direction (for example, when the length of a specific axis is longer than the length of other axes).

[0263] The tiling unit 16001 receives point cloud data according to an embodiment, and generates one or more tiles, one or more slices, or one or more bricks from the received point cloud data. That is, the brick tiling unit 16001 divides the point cloud data into tiles, slices, or bricks. The brick tiling unit 16001 outputs the point cloud data divided into tiles, slices, or bricks.

[0264] A tile refers to a portion of a three-dimensional space occupied by point cloud data according to an embodiment. A tile according to an embodiment includes one or more slices. A tile according to an embodiment is divided into one or more slices, so that a point cloud data encoder can encode point cloud data in parallel. In a tile according to an embodiment, the point cloud data encoder performs different quantization and / or conversion for each of one or more slices according to characteristics of the data.

[0265] A slice refers to a unit of data that a point cloud data transmitting device and a receiving device according to an embodiment encode and decode. A slice refers to a collection of data in a three-dimensional space occupied by point cloud data, or a collection of a portion of data among point cloud data. A slice refers to an area of ​​points or a collection of points included in a tile according to an embodiment. A tile according to an embodiment is divided into one or more slices based on the number of points included in a tile. For example, a tile refers to a collection of points divided by the number of points. A tile according to an embodiment is divided into slices based on the number of points, and some data is split or merged during the division process.

[0266] A block refers to a unit into which point cloud data included in a slice is divided by points. For example, a block refers to a part of a three-dimensional space occupied by cloud data in one slice, and refers to a detailed unit in which a point cloud data transmitting device and a receiving device according to an embodiment perform encoding and decoding.

[0267] Brick refers to a block or slice according to the embodiment. That is, a brick can also refer to a region or a set of points that divides a slice or point cloud data. In the embodiment, a brick is also called a region.

[0268] Brick tiling according to the embodiment means that the point cloud data transmitting device divides the point cloud data into one or more bricks (slices). Brick tiling is also called slice partitioning.

[0269] The brick tiling unit 16001 according to the embodiment may perform slice distribution based on various variations of methods for dividing according to data characteristics, such as point density-based brick tiling (slice distribution), circular sector brick tiling (slice distribution), layer brick tiling (slice distribution), spherical / elliptical area tiling (slice distribution), attribute-based brick tiling (slice distribution), and axis-based adaptive division method. The division method will be described later with reference to Figures 19 to 31.

[0270] According to the embodiment, point density-based brick tiling is also called point density-based slice distribution, circular sector brick tiling is also called circular sector slice distribution, layer brick tiling is also called layer slice distribution, spherical / elliptical region tiling is also called spherical / elliptical slice distribution, and attribute-based brick tiling is also called attribute-based slice distribution.

[0271] For example, if the data characteristics investigation unit 16000 determines that points of the point cloud data are densely distributed in certain areas and sparsely distributed in other areas, the brick tiling unit 16001 performs division based on the point density-based brick tiling (slice distribution) method.

[0272] For example, if the data characteristics investigation unit 16000 determines that the point cloud data is composed of a point cloud sequence including multiple frames, such as LiDAR data or data for autonomous driving, the brick tiling unit 16001 performs division using a circular sector brick tiling (slice distribution) method. If the data characteristics investigation unit 16000 determines that the point cloud data in one frame is concentrated in a specific area, the brick tiling unit 16001 performs division using a layer brick tiling (slice distribution) method. If the data characteristics investigation unit 16000 determines that the objects represented by the point cloud data are sparsely present, the brick tiling unit 16001 performs division based on spherical / elliptical area tiling. If the three-dimensional space occupied by the point cloud data extends in one direction, the data characteristics investigation unit 16000 performs division using an axis-based adaptive division method.

[0273] According to the embodiment, the point cloud data may be divided into a plurality of slices. There are various methods for dividing the point cloud data into a plurality of slices. According to the embodiment, the point cloud data included in the slice is independently encoded. The point cloud data transmitting and receiving devices according to the embodiment independently encode and decode the data in slice units, thereby preventing repeated errors during the encoding and decoding process.

[0274] A slice refers to a unit into which a point cloud frame in the point cloud data is divided for encoding. The point cloud data is divided into a number of slices according to a slice partitioning scheme.

[0275] According to an embodiment of the method for dividing point cloud data into a plurality of slices, the method divides the point cloud data based on variance information of color information to ensure color continuity. For example, if the variance information of color information in a specific point cloud or a region of the point cloud data is greater than a first threshold, the corresponding data or region is regarded as a non-smooth block, and if the weight of the non-smooth block in the entire point cloud data is greater than a second threshold, the block corresponding to the non-smooth block (or some points included in the non-smooth block) is extracted and separated as a first slice, and the remaining unseparated cloud data is separated as a second slice. According to the embodiment, the variance information of color information is calculated to separate the first slice and the second slice, and the variance information of color information can be obtained by using a kd-tree.

[0276] A kd-tree according to an embodiment is a type of binary tree that represents a hierarchy for partitioning a k-dimensional space. The kd-tree scheme represents a geometry-adaptive method for partitioning point cloud data into slices.

[0277] According to embodiments, slices may be referred to by various terms such as bricks, blocks, macroblocks, and / or tiles, etc. Brick tiling according to embodiments may also be referred to as slice partitioning.

[0278] The divided data input unit 16002 receives the divided point cloud data output by the brick tiling unit 16001. The divided data input unit 16002 outputs position information (geometry information) of points in each brick (or slice or tile) and attribute information (characteristic information) of the points. The divided data input unit 16002 transmits the position information of the points to the coordinate conversion unit 16003 on a brick basis (or slice or tile basis), and transmits the attribute information of the points to the hue conversion processing unit 16009.

[0279] The data characteristics investigation section 16000, brick tiling section 16001 and / or partitioned data input section 16002 according to an embodiment are included within the spatial division section 15001 of FIG.

[0280] The coordinate system conversion unit 16003 receives position information (geometry information) of points according to the embodiment and converts the coordinate information. The coordinate system conversion unit 16003 transmits the converted coordinate information to the quantization / voxelization processing unit 16004. The coordinate system conversion unit 16003 performs the operation of the coordinate system conversion unit 40000 in FIG.

[0281] The quantization / voxelization processing unit 16004 performs quantization and / or voxelization on the position information of the points using the coordinate conversion unit 16003. The quantization / voxelization processing unit 16004 transmits the position information (geometry information) of the points that have been quantized and / or voxelized to an octree occupation code generation unit 16005. The quantization / voxelization processing unit 16004 performs the operations of the quantization and voxelization processing unit 40001 in FIG. 4 and the voxelization processing unit 12002 in FIG.

[0282] The octree occupancy code generator 16005 receives quantized and / or voxelized geometry information and generates an octree occupancy code based on the quantized and / or voxelized geometry information. The octree occupancy code includes a data structure representing position information of a point according to an embodiment. The octree occupancy code generator 16005 transmits the octree occupancy code to the first arithmetic coder 16007, the surface model processor 16006, and / or the geometry reconstruction unit 16008. The octree occupancy code generator 16005 performs the operations of the octree analysis unit 40002 in FIG. 4 and the octree occupancy code generator 12003 in FIG. 12.

[0283] The surface model processing unit 16006 receives the octree occupation code according to the embodiment and processes the surface model. The surface model processing unit 16006 outputs geometry information that has been subjected to the surface model processing and transmits it to the first arithmetic coder 16007. The surface model processing unit 16006 performs the operations of the surface approximation analysis unit 40003 in FIG. 4 and the surface model processing unit 12004 in FIG.

[0284] The first arithmetic coder 16007 receives the octree occupancy code according to the embodiment and / or the octree occupancy code subjected to the surface model processing, and encodes them. The first arithmetic coder 16007 outputs a geometry bitstream. The first arithmetic coder 16007 transmits the geometry bitstream to the bitstream combiner. The first arithmetic coder 16007 performs the operation of the arithmetic encoding 40004 in Fig. 4 and the operation of the arithmetic coder 12006 in Fig. 12.

[0285] The first arithmetic coder 16007 transmits information related to whether or not to encode geometry information for a corresponding brick (or slice or tile) to the divided data input unit 16002.

[0286] The geometry reconstruction unit 16008 receives the octree occupancy code according to the embodiment and / or the octree occupancy code that has been subjected to surface model processing, and reconstructs geometry information based on them. The geometry reconstruction unit 16008 generates the Moulton code according to the embodiment, and reconstructs geometry information based on them. The geometry reconstruction unit 16008 performs the operation of the geometry reconstruction unit 40005 in FIG. 4.

[0287] A hue conversion processing unit 16009 receives attribute information of points according to the embodiment and converts the hue information of the attribute information. The hue conversion processing unit outputs the converted attribute information and transmits it to an attribute conversion processing unit 16010. The hue conversion processing unit 16009 performs the operations of the hue conversion unit 40006 in FIG. 4 and the hue conversion processing unit 12008 in FIG. 12.

[0288] An attribute conversion processing unit 16010 performs attribute conversion processing by mapping position information of points according to the embodiment, attribute information subjected to hue conversion processing by a hue conversion processing unit 16009 according to the embodiment, and / or geometry information reconstructed by a geometry reconstruction unit 16008. The attribute conversion processing unit 16010 performs the operations of the attribute transfer unit 40007 and the attribute conversion processing unit 12009 in FIG.

[0289] The prediction / lift / RAHT conversion processing unit 16011 converts the attribute information converted by the attribute conversion processing unit 16010 using the prediction, lifting and / or RAHT methods. The prediction / lift / RAHT conversion processing unit 16011 outputs the attribute information converted using the prediction, lifting and / or RAHT methods and transmits them to the coefficient quantization processing unit 16012. The prediction / lift / RAHT conversion processing unit 16011 performs the operations of the RAHT 40008, the LOD generation unit 40009 and / or lift 40010 in Fig. 4, and the prediction / lift / RAHT conversion processing unit 12010 in Fig. 12.

[0290] The coefficient quantization processing unit 16012 receives the attribute information converted by the prediction / lift / RAHT conversion processing unit 16011 and quantizes it. The coefficient quantization processing unit 16012 transmits the quantized attribute information to a second operation coder 16013. The coefficient quantization processing unit 16012 performs the operation of the coefficient quantization unit 40011 in FIG.

[0291] The second operation coder 16013 receives the attribute information quantized by the coefficient quantization unit 16012 and encodes the received attribute information. The second operation coder 16013 outputs an attribute bitstream including the encoded attribute information. The second operation coder 16013 outputs the attribute bitstream and transmits it to the bitstream combination unit 16014.

[0292] The second arithmetic coder 16013 transmits information related to whether or not to encode characteristic information of a corresponding brick (or slice or tile) to the divided data input unit 16002. The second arithmetic coder 16013 performs the operations of the arithmetic encoding unit 40012 of FIG. 4 and the arithmetic coder 12011 of FIG.

[0293] The bitstream combining unit 16014 combines a geometry bitstream and / or an attribute information bitstream according to an embodiment. The bitstream combining unit 16014 combines the geometry bitstream and the attribute bitstream in units of bricks (or slices or tiles) according to an embodiment.

[0294] The point cloud data transmitting apparatus according to the embodiment includes a data characteristic checking unit 16000, a brick tiling unit 16001, a divided data input unit 16002, and a bit stream combining unit 16014. The point cloud data transmitting apparatus according to the embodiment divides point cloud data into one or more bricks in the brick tiling unit according to, for example, brick_tiling_filter information. The point cloud data according to the embodiment is divided into N regions by the brick tiling unit 16001 and input to the divided data input unit 16002. For each of the divided N bricks, the point cloud data in each brick is divided into point position information and point attribute information and encoded. When the encoding process is performed for each of the N bricks, the encoded point position information and point attribute information are combined into one bit stream in the bit stream combining unit 16014 according to the embodiment.

[0295] The point cloud data transmitting device according to the embodiment divides point cloud data into one or more brick units using the brick tiling unit 16001, and performs encoding for each divided brick unit, thereby performing encoding for each region. In this case, data of a region necessary for the point cloud data receiving device according to the embodiment can be separated from less important regions and encoded efficiently. With this configuration, the point cloud data transmitting device can increase the encoding efficiency.

[0296] A point cloud data encoder according to an embodiment may divide point cloud data into one or more tiles, slices, blocks, and / or bricks, and set different quantization for each divided unit. By dividing the point cloud data in this manner, the point cloud data encoder may differentially encode points having different importance. In addition, a point cloud data receiving device may efficiently decode the differentially encoded data to quickly provide important information to a user.

[0297] The point cloud data transmission device provides an encoding method optimized for data characteristics by dividing point cloud data into one or more brick units and encoding each divided brick unit. For example, in the case of point cloud data including points densely distributed in a specific area, the area where the points are densely distributed is divided into small parts and encoded to reduce errors that may occur during encoding. For example, in a data sequence consisting of multiple frames such as LiDAR data or autonomous driving, the point cloud data is divided and encoded using a circular sector brick tiling method described later to reduce encoding errors for each frame.

[0298] The point cloud data transmitting and receiving devices according to the embodiment can prevent repeated errors during the encoding and decoding process by independently performing encoding and decoding on a slice-by-slice basis.

[0299] In the point cloud data transmission device according to the embodiment, point cloud content capable of real-time processing, low latency, and parallel processing is provided by performing division operations based on a division method that is adaptively performed for each requirement that may arise in various processing situations such as transmission, encoding / decoding, and rendering of point cloud data.

[0300] FIG. 17 illustrates a bitstream combining unit of a point cloud data transmitting apparatus according to an embodiment.

[0301] The bitstream combination unit 17000 shown in Figure 17 corresponds to or is combined with the transmitting device 10000, transmitter 10003, point cloud video encoder 10002, encoding 18001, transmitting unit 18002, encoder of Figure 4, transmitting device of Figure 12, XR device 1430 of Figure 14, etc.

[0302] The bitstream combiner 17000 in Figure 17 is the bitstream combiner 16014 in Figure 16, or is included in the transmission processing unit 12012 in Figure 12, the transmitter 10003 in Figure 1, or the transmission unit 18002 in Figure 2. That is, the geometry bitstream 17000a and the attribute bitstream 17000b in Figure 17 correspond to the geometry bitstream and the attribute bitstream received by the bitstream combiner 16014 in Figure 16, or are the geometry information bitstream 15000b and the attribute information bitstream 15000c in Figure 15, the geometry bitstream output by the arithmetic encoder 38004 and the attribute bitstream output by the arithmetic encoder 40012 in Figure 4, the bitstream output by the arithmetic coder 12006 in Figure 12, and the bitstream output by the arithmetic coder 12011 in Figure 12.

[0303] The bitstream combiner 17000 is shown. The bitstream combiner 17000 receives a geometry bitstream 17000a and / or a feature bitstream 17000b. The bitstream combiner 17000 combines the received geometry bitstream 17000a and feature bitstream 17000b to output a bitstream 17000c.

[0304] The bitstream combiner 17000 receives a geometry bitstream 17000a encoded in a region unit and a feature bitstream 17000b encoded in a region unit and combines them. The bitstream combiner 17000 outputs a region-based bitstream 17000c. According to an embodiment, the bitstream combiner 17000 receives bitstreams 17000a and 17000b in which point cloud data corresponding to one or more regions divided according to Figures 19 to 25 is encoded, and combines them. Thus, the bitstream combiner 17000 can output a bitstream 17000c in a region unit according to Figures 19 to 25.

[0305] A bitstream combination unit 17000 means the bitstream combination unit 16014 in Fig. 19. A geometry bitstream 17000a and an attribute bitstream 17000b mean the geometry bitstream and the attribute bitstream in Fig. 19. A geometry bitstream 17000a and an attribute bitstream 17000b mean the geometry bitstream 1500b and the attribute information bitstream 15000c in Fig. 18.

[0306] The bitstream combining unit 17000 includes a combining unit 17001 for combining the geometry bitstream 17000a and the attribute bitstream 17000b according to an embodiment. The combining unit 17001 receives the geometry bitstream 17000a and the attribute bitstream 17000b, combines them for each region, and outputs a bitstream 17000c.

[0307] The bitstream combining unit 17000 determines in what order the geometry bitstream 17000a and the attribute bitstream 17000b are combined and how the combination is performed. The combination method includes multiplexing, MVC bitstream combination, sequential combination, importance-based combination, etc. The combination method according to the embodiment is determined or selected according to the identification of the received geometry bitstream 17000a and / or attribute bitstream 17000b (e.g., bitstreams of various attributes, presence or absence of attribute bitstreams, degree of quantization of geometry and attribute bitstreams, length of bitstreams, presence or absence of tiling, etc.). The point cloud data encoder generates information (combining_filter_index) on which combination method was used in the encoder to transmit the combined geometry bitstream and attribute bitstream, and transmits them in the form of a bitstream. According to the embodiment, a specific combination method can be fixedly used according to the transmission characteristics of the bitstream.

[0308] The point cloud data transmitting apparatus according to the embodiment can transmit the bit stream generated by encoding the divided regions shown in Figures 19 to 25 for each region by configuring the bit stream combining unit 17000. Therefore, the point cloud data receiving apparatus according to the embodiment can efficiently decode the point cloud data received for each region together with the signaling information.

[0309] FIG. 18 is a diagram illustrating a method in which a point cloud data transmitting device according to an embodiment divides point cloud data into one or more regions.

[0310] A brick according to the embodiment corresponds to a slice, which is also referred to as a slice in this specification.

[0311] The operation shown in Figure 18 is performed by a combination of the transmitting device 1000, transmitter 10003, point cloud video encoder 10002, encoding 18001, transmitting unit 18002, encoder of Figure 4, transmitting device of Figure 12, and XR device 1430 of Figure 14, etc.

[0312] According to an embodiment, point cloud data can be obtained by dividing the point cloud data into one or more slices.

[0313] The operation shown in FIG. 18 is performed by, for example, the space division unit 15001 in FIG. 15, the data characteristic examination unit 16000 in FIG. 16, the brick tiling unit 16001, and the data input unit 12000 in FIG.

[0314] The left image 18000 of FIG. 18 shows point cloud data according to an embodiment. Point cloud data can be classified into multiple types of point cloud data (e.g., LiDAR data, etc.). The types of point cloud data can be classified according to whether the points of the point cloud data are densely or sparsely distributed.

[0315] The right image 18001 in Fig. 18 illustrates the operation of splitting the point cloud data of the left image 18000 in Fig. 20 into three bricks. The operation of splitting the point cloud data into three bricks according to the embodiment is performed by the space splitting unit 15001 in Fig. 15 and the brick tiling unit 16001 in Fig. 16.

[0316] Referring to the right image 18001 of Fig. 18, the brick tiling unit 16001 according to the embodiment divides the point cloud data of the left image 18000 of Fig. 18 into a first brick 18001a, a second brick 18001b, and a third brick 18001c. The brick tiling unit 19001 according to the embodiment divides the point cloud data based on the type of the point cloud data. The bricks according to the embodiment may be called by other names within the scope of equivalent meaning.

[0317] The point cloud data transmitting device according to the embodiment provides the effect of enabling random access and parallel encoding in the three-dimensional space occupied by the point cloud data by dividing the point cloud data into tiles, slices and / or bricks as described in Figures 18 and 19.

[0318] FIG. 19 is a diagram illustrating a method in which a point cloud data transmitting device according to an embodiment divides point cloud data into one or more regions.

[0319] The operations according to the division method shown in Fig. 19 are performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14 in combination with each other. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0320] Point cloud data according to the embodiment can be obtained and the obtained point cloud data can be divided into one or more slices as shown in Fig. 19. The operation shown in Fig. 19 can be performed, for example, by the space division unit 15001 in Fig. 15, the data characteristic examination unit 16000 in Fig. 16, the brick tiling unit 16001, and the data input unit 12000 in Fig. 12.

[0321] 19(a) shows that a three-dimensional space 19A00 occupied by point cloud data is divided based on the length of an axis of the three-dimensional space 19A00. The point cloud data transmission device according to the embodiment encodes the point cloud data according to the embodiment. The point cloud data transmission device according to the embodiment divides the points or three-dimensional space 19A00 according to the embodiment into one or more divided regions 19A01a to 19A01c, and can independently perform encoding according to the embodiment for the points included in the divided regions.

[0322] The three-dimensional space 19A00 occupied by the point cloud data includes the point cloud data according to the embodiment. The three-dimensional space 19A00 includes a plurality of points representing the point cloud data. The plurality of points are units constituting the point cloud data according to the embodiment. The point cloud data transmitting device according to the embodiment can encode the position information and attribute information of each of the plurality of points.

[0323] The divided regions 19A01a-19A01c include a portion of the point 19A00 of the point cloud data according to the embodiment. The point cloud data transmission device according to the embodiment divides the point cloud data into one or more divided regions 19A01a-19a01c based on the characteristics of the point cloud data. The divided regions 19A01a-19A01c according to the embodiment correspond to or are called tiles, slices, bricks, or blocks.

[0324] The point cloud data transmission device according to the embodiment divides a three-dimensional space 19A00 including points based on the length of an axis of the three-dimensional space. The three-dimensional space 19A00 including points is a bounding box or a tile. The three-dimensional space (e.g., a bounding box or a tile) according to the embodiment is represented by an x-axis, a y-axis, and a z-axis.

[0325] In the point cloud data according to the embodiment, for example, the length of any one of the x-axis, y-axis, and z-axis is shorter than the remaining two. For example, there is point cloud data representing a terrain that is widely distributed along two axes except one axis. If the point cloud data transmitting device performs encoding on points within a bounding box regardless of the axis length, the encoding efficiency decreases. Therefore, the point cloud data transmitting device needs to divide the data into one or more regions according to the axis length of the three-dimensional space. Therefore, the point cloud data transmitting device according to the embodiment calculates the relative lengths of the x-axis, y-axis, and z-axis, and divides the point cloud data into one or more regions based on the calculated relative lengths.

[0326] Therefore, the point cloud data transmission device according to the embodiment checks the length of the shortest axis and the length of the longest axis among the x-axis, y-axis, and z-axis, and divides the longest axis into units of the length of the shortest axis. For example, as shown in FIG. 19(a), the longest axis of the bounding box is the y-axis 30, and the shortest axis is the z-axis 10. Therefore, the point cloud data transmission device according to the embodiment divides the Y-axis into 10 units, which is the length of the Z-axis, and divides the point cloud data into three slices 19A01a to 19A01c.

[0327] The point cloud data transmitting device according to the embodiment can also divide the divided slices again. For example, one divided slice 19A01a can be divided again into units of the length of the longest axis and the shortest axis. In this case, two slices with lengths of 10 on the X-axis, Y-axis, and Z-axis are generated.

[0328] The point cloud data transmission device according to the embodiment divides point cloud data based on a slice partition scheme of Uniform-Geometry partition along the longest edge, as shown in Fig. 19(a). Uniform-Geometry partition along the longest edge means a method of partitioning as follows:

[0329] 1) Assume the length of the longest edge and the shortest edge in the 3D space as maxEdge and minEdge, slice number as sliceNum, slice size as sliceSize. The default value of sliceNum is set as maxEdge / minEdge and sliceSize is set as minEdge.

[0330] 2) Divide the point cloud data by sliceNum using the uniform-geometry partitioning scheme according to the embodiment.

[0331] FIG. 19(b) illustrates partitioning a three-dimensional space 19B00 occupied by point cloud data based on octree information regarding the points contained within the three-dimensional space 19B00.

[0332] The octree information according to the embodiment means an octree structure for indicating geometry information of a point according to the embodiment. The point cloud data transmitting device according to the embodiment can divide a slice so that a three-dimensional space corresponds to one or more spaces represented by an octree.

[0333] That is, the point cloud data transmitting device according to the embodiment can perform Uniform-Geometry partitioning using Octree. The Uniform-Geometry partitioning method using Octree is as follows.

[0334] 1) Set the depth of octree partition depOctree=1 by default. The depth of octree partition means the level of the octree structure.

[0335] 2) The point cloud data is divided into 8^depOctree slices using an octree distribution structure.

[0336] For example, when depOctree is 1, the point cloud data transmission device according to the embodiment divides the point cloud data into eight slices including points for eight spaces.

[0337] By dividing the bounding box in this manner, the point cloud data transmission device of the embodiment can encode each non-overlapping slice individually, and similarly the receiving device can decode the point cloud data for each slice individually.

[0338] The point cloud data transmitting apparatus according to the embodiment divides the 3D space into one or more slices based on the color information or reflectance information, which is characteristic data of the points in any tile. For example, the point cloud data transmitting apparatus according to the embodiment may divide the slice into points aligned by the value of the characteristic data after sorting the characteristic data according to the embodiment in ascending order.

[0339] In the point cloud data transmitting apparatus according to the embodiment, when dividing points in this manner, the division is based on ascending sorting, so that bounding boxes can overlap between slices, and slices are divided in a form in which multiple slices share one space. Therefore, the point cloud data transmitting apparatus according to the embodiment can support spatial scalability and region of interest based spatial scalability, which are requirements for point cloud compression.

[0340] FIG. 20 illustrates how a point cloud data transmission device according to an embodiment divides point cloud data into one or more regions.

[0341] The operations according to the division method shown in Fig. 20 are performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14 in combination with each other. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0342] The operation shown in FIG. 20 is performed by, for example, the space division unit 15001 in FIG. 15, the data characteristic examination unit 16000 in FIG. 16, the brick tiling unit 16001, and the data input unit 12000 in FIG.

[0343] 20(a) shows a three-dimensional space 20A00 occupied by point cloud data. The point cloud data transmission device according to the embodiment encodes point cloud data according to the embodiment. The point cloud data transmission device according to the embodiment divides a point 20A01 or a three-dimensional space 20A00 according to the embodiment into one or more divided regions 20A02, and performs encoding according to the embodiment independently for the points included in the divided regions.

[0344] The three-dimensional space 20A00 occupied by the point cloud data includes the point cloud data according to the embodiment. The three-dimensional space 20A00 includes a plurality of points 20A01 representing the point cloud data.

[0345] The points 20A01 are units constituting the point cloud data according to the embodiment. The point cloud data transmission device according to the embodiment encodes the position information and attribute information of each of the points 20A01.

[0346] The divided region 20A02 includes a portion of the points 20A01 of the point cloud data according to the embodiment. The point cloud data transmission device according to the embodiment divides the point cloud data into one or more divided regions 20A02 based on the characteristics of the point cloud data. The divided region 20A02 according to the embodiment corresponds to or is referred to as a tile, a slice, a brick, or a block.

[0347] The characteristics of the point cloud data are related to the density of the points 20A01. For example, the point cloud data according to the embodiment is data in which points are densely distributed in the direction of the arrow 20A03. Therefore, the point cloud data transmitting device according to the embodiment can divide the point cloud data into one or more divided regions 20A02 based on the density of the points.

[0348] For example, the point cloud data transmission device according to the embodiment can divide the point cloud data into a divided region 20A02 including a first brick 20A02-1 and a second brick 20A02-2. The first brick 20A02-1 is a region including points that are relatively sparsely distributed among a plurality of points. The second brick 20A02-2 is a region including points that are relatively densely distributed among a plurality of points. According to the embodiment, the second brick 20A02-2, in which the points are densely distributed, has a larger brick size than the first brick 20A02-1, in which the points are sparsely distributed.

[0349] Figure 20(b) shows a method of dividing a three-dimensional space 20A00 occupied by point cloud data into one or more regions and a process of encoding each brick. Specifically, Figure 20(b) shows dividing point cloud data into one or more bricks based on the density of points in the point cloud data. Such a method of dividing point cloud data is called point density-based brick tiling.

[0350] Point density-based brick tiling according to an embodiment refers to dividing point cloud data according to an embodiment into one or more bricks according to a point density or a point distance according to an embodiment. According to an embodiment, a divided brick is composed of one 3D block or a bundle of two or more blocks.

[0351] The operation of Fig. 20(b) is performed by the spatial division unit 15001 of Fig. 15 and the brick tiling unit 16001 of Fig. 16. The process shown in Fig. 20(b) includes a step of receiving point cloud data (20B00), a step of determining whether or not to perform brick division (20B01), a step of determining the number of bricks (20B02), a step of calculating point density (20B03), a step of determining a quantization coefficient according to the point density for each brick (20B04), a step of encoding for each brick (20B05), and / or a step of outputting a bitstream for each brick (20B06).

[0352] In the step of receiving point cloud data (20B00), point cloud data according to an embodiment is received.

[0353] The step of determining whether or not to perform brick division (20B01) is a step of determining whether or not to divide the received point cloud data into one or more bricks (or slices or tiles).

[0354] The step (20B01) of determining whether or not to perform brick division is a step of determining whether or not to perform brick division based on x-coordinate values, y-coordinate values ​​and / or z-coordinate values ​​of points in the received point cloud data.

[0355] In the step of determining whether or not to perform brick division (20B01), it is determined whether or not to perform brick division based on the type of received point cloud data. The step of determining whether or not to perform brick division (20B01) is a step of generating brick_tiling_filter information indicating whether or not to perform brick division.

[0356] In the step of determining the number of bricks (20B02), if brick division is determined in the step of determining whether or not to perform brick division, the number of bricks into which the point cloud data is divided is determined. In the step of determining the number of bricks (20B02), the number of bricks to be divided is determined based on the type of the received point cloud data.

[0357] The step of calculating the density of points (20B03) is a step of calculating the density of points according to an embodiment and dividing the point cloud data into one or more bricks based on the calculated density. According to an embodiment, the density of points is defined based on the number of points per volume occupied by the points, the average distance between the points, etc. According to an embodiment, the density of points for each brick is variably determined.

[0358] In the step of determining a quantization coefficient according to a density of points for each brick (20B04), the point cloud data transmitting apparatus according to the embodiment determines a quantization coefficient according to density in order to perform variable decoding according to the density of points.

[0359] The step (20B05) of encoding for each brick means an operation of the point cloud data transmitting device according to the embodiment encoding the point cloud data for each brick. According to the embodiment, each divided brick is dynamically changed according to the setting of the point cloud data transmitting device. The point cloud data transmitting device generates information such as information for notifying division information (e.g., brick_unit_information, brick_density_quantization_step).

[0360] The step of outputting a bitstream for each brick (20B06) refers to an operation of the point cloud data transmitting device outputting a bitstream in which the point cloud data is encoded in units of each brick.

[0361] The point cloud data transmitting apparatus according to the embodiment can effectively encode a sequence in which points are distributed at different densities in different regions within a frame by performing point density-based tiling according to the embodiment.

[0362] The point cloud data transmitting device according to the embodiment performs tiling based on point density according to the embodiment, and can increase the compression rate of geometry information by lowering the scaling factor for low-density bricks according to point density, and can provide an effect of expressing location information precisely by increasing the scaling factor for bricks including high-density points.

[0363] FIG. 21 is a diagram illustrating a method in which a point cloud data transmitting device according to an embodiment divides point cloud data into one or more regions.

[0364] The operations according to the division method shown in Fig. 21 are performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, the XR device 1430 in Fig. 14, and the like, in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0365] The operation shown in FIG. 21 is performed by, for example, space division unit 15001 in FIG. 15, data characteristic examination unit 16000 in FIG. 16, brick tiling unit 16001, and data input unit 12000 in FIG.

[0366] 21 shows that the point cloud data transmitting device divides the three-dimensional space or point cloud data 21A00 occupied by the point cloud data into one or more circular sectors 21A02. The point cloud data transmitting device according to the embodiment encodes the point cloud data according to the embodiment for each divided circular sector 21A02. The operation of dividing the point cloud data shown in FIG. 21 is performed by the space dividing unit 15001 in FIG. 15 and the brick tiling unit 16001 in FIG. 16.

[0367] Sector 21A02 according to the embodiment may correspond to or be referred to as a tile, a slice, a brick, or a block according to the embodiment.

[0368] The point cloud data transmitting device according to the embodiment divides the point cloud data 21A00 into one or more circular sectors 21A02 based on a reference axis 21A01. The operation of dividing the point cloud data into bricks in the form of one or more circular sectors is also called circular sector brick tiling.

[0369] The point cloud data 21A00 refers to point cloud data acquired by a point cloud data transmitting device. The point cloud data 21A00 includes a plurality of points.

[0370] The reference axis 21A01 refers to an axis that is a reference for dividing the point cloud data into one or more circular sectors 21A02. The reference axis 21A01 indicates the center of the three-dimensional space (e.g., a bounding box) that the point cloud data occupies. For example, the reference axis is the center of the x-axis and y-axis of the three-dimensional space, and is an axis that is vertical in the xy plane. For example, the reference axis is the center of the y-axis and z-axis of the three-dimensional space, and is an axis that is vertical in the yz plane. For example, the reference axis is the center of the x-axis and z-axis of the three-dimensional space, and is an axis that is vertical in the xz plane.

[0371] According to an embodiment, the point cloud data is divided into one or more circular sectors based on a reference axis 21A01 including the origin of the circular sector. According to an embodiment, when one or more circular sectors are determined, each unit may contain points including position information or attribute information.

[0372] The reference axis 21A01 according to the embodiment is determined based on information indicating the central point information. The central point information according to the embodiment is, for example, central_node_xyz. The reference axis 21A01 according to the embodiment is determined through the central point information in the direction of the x-axis, y-axis, or z-axis in the three-dimensional space, which has the shortest axis length.

[0373] The circular sector 21A02 represents a brick that divides the point cloud data according to the embodiment. The circular sector 21A02 includes points. The circular sector 21A00 represents an area formed in an arc shape that radiates from the reference axis at a central angle 21A02-1.

[0374] According to an embodiment, each divided circular sector 21A02 has a radius 21A02-1 that is the same as or different from each other. The radius 21A02-1 of the circular sector 21A02 is determined based on the density of the point cloud data. According to an embodiment, the radius 21A02-1 is indicated by bounding_box_radius.

[0375] The point cloud data transmission device according to the embodiment is divided into circular sectors 21A02 having various central angles 21A02-2. The point cloud data transmission device according to the embodiment determines the central angle of each circular sector. The central angle of the circular sector is determined based on the density of the point cloud data.

[0376] When point cloud data is composed of multiple frames, the point cloud data transmitting apparatus according to the embodiment determines the pattern of the divided regions based on the moving direction 21A03 of the object included in the point cloud data. When point cloud data is composed of multiple frames, the moving direction of the object included in the point cloud data (e.g., the direction of the blue arrow) can be defined. The moving direction of the object according to the embodiment is indicated by moving_direction_vector.

[0377] In the embodiment, the bounding box information (3D space occupied by point cloud data) is defined for each frame and is determined by the coordinate information (e.g., x, y, z information) of each frame. In the embodiment, when the same bounding box information is applied to all frames, the bounding box information flag (use_identical_bounding_box_information_flag) is sent.

[0378] According to an embodiment, the point cloud data transmitting device encodes each frame by different quantization coefficients, geometry compression methods, attribute compression methods, and single attribute / multiple attributes.

[0379] The point cloud data transmission device according to the embodiment can perform circular sector brick tiling to effectively encode data sequences including multiple frames, such as LiDAR data and data for autonomous driving.

[0380] The point cloud data transmission device according to the embodiment can signal bricks using only the radius information and angle of the bricks by performing circular sector brick tiling, thereby reducing the complexity of the receiving device. In addition, by performing circular sector brick tiling, it is possible to improve coding by location information prediction when the distribution of points within the same circular sector is the same.

[0381] FIG. 22 is a diagram illustrating how a point cloud data transmitting device according to an embodiment divides point cloud data into one or more regions.

[0382] 22 shows that the point cloud data transmission device divides the three-dimensional space occupied by the point cloud data or the point cloud data 23A00 into one or more layers 22A01 or 22A02. The point cloud data transmission device according to the embodiment can encode the point cloud data according to the embodiment for each divided layer 22A02.

[0383] The operations according to the division method shown in Fig. 22 are performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, the XR device 1430 in Fig. 14, and the like, in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0384] The operation shown in FIG. 22 is performed by, for example, the space division unit 15001 in FIG. 15, the data characteristic examination unit 16000 in FIG. 16, the brick tiling unit 16001, and the data input unit 12000 in FIG.

[0385] The point cloud data transmission device according to the embodiment divides the point cloud data 22A00 into one or more layers 22A01. The method of dividing the point cloud data 22A01 according to the embodiment into one or more layers 22A01 is also called layer brick tiling.

[0386] Layer 22A01 according to the embodiment may correspond to or be referred to as a tile, a slice, a brick, or a block according to the embodiment.

[0387] The point cloud data transmitting apparatus according to the embodiment sorts the point cloud data before dividing the point cloud data into one or more layers. The point cloud data transmitting apparatus sorts the point cloud data in ascending order, descending order, or Moulton code order. The point cloud data sorting operation according to the embodiment is performed by the space division unit 15001 of Fig. 15, the brick tiling unit 16001 of Fig. 16, or the data characteristic inspection unit 16000. The point cloud data transmitting apparatus according to the embodiment generates information (e.g., enable_sorting_flag) indicating a sorting method of the point cloud data according to the embodiment, and transmits it in the form of a bit stream.

[0388] The layers 22A01 and 22A02 refer to a collection of point cloud data included in a certain range of the x-axis, y-axis, or z-axis of the three-dimensional space occupied by the point cloud data. In other words, the layers 22A01 and 22A02 are composed of a collection of points.

[0389] The layer 22A01 shown in the left image of Fig. 22 means a collection of all point cloud data points having y coordinate information in a certain range, as in the left image of Fig. 22. Therefore, according to the embodiment, the point cloud data is composed of multiple layers in the range of y coordinate information of the points. Such a layer shown in the left image of Fig. 22 is also called an xz layer (or a y direction layer).

[0390] The layer 22A02 shown in the right image of Fig. 22 means a collection of all point cloud data points having a certain range of x coordinate information, as in the right image of Fig. 22. Therefore, according to the embodiment, the point cloud data is composed of multiple layers in the range of x coordinate information of the points. Such a layer shown in the right image of Fig. 22 is also called a yz layer (or an x ​​direction layer).

[0391] Similarly, the point cloud data is composed of multiple layers in the range of z-coordinate information of the points. The layer shown in the left image of Fig. 22 is also called an xy layer (or a z-direction layer). According to an embodiment, the point cloud data is divided into one or more x-direction layers, y-direction layers, or z-direction layers.

[0392] Point cloud data according to an embodiment may also be organized into multiple layers formed based on the chronological movement of points.

[0393] The point cloud data transmitting apparatus according to the embodiment may perform progressive coding on one or more layers. Progressive coding according to the embodiment means performing coding progressively on an aligned layer basis.

[0394] The point cloud data transmission device according to the embodiment divides the point cloud data into one or more layers, so that compression is effectively applied to data characteristics when the point cloud data within a frame is concentrated in a specific region (e.g., when consecutive attribute information values ​​are concentrated on the left side, right side, upper side, lower side, front side or rear side).

[0395] The point cloud data transmission device according to the embodiment can improve the correlation of position information within a brick and increase encoding efficiency by dividing the point cloud data into one or more layers.

[0396] FIG. 23 is a diagram illustrating how a point cloud data transmitting device according to an embodiment divides point cloud data into one or more regions.

[0397] Fig. 23 shows that the point cloud data transmitting device divides the three-dimensional space occupied by the point cloud data or the point cloud data into one or more spherical / elliptical regions 23001a to 23001f. The point cloud data transmitting device according to the embodiment encodes the point cloud data according to the embodiment for each of one or more spherical / elliptical regions 23001a to 23001f. The operation of dividing the point cloud data shown in Fig. 23 is performed by the space dividing unit 15001 in Fig. 15 and the brick tiling unit 16001 in Fig. 16. The operation shown in Fig. 23 is performed by, for example, the data input unit 12000 in Fig. 12.

[0398] The operations according to the division method shown in Fig. 23 are performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, the XR device 1430 in Fig. 14, and the like, in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0399] In an embodiment, the spherical / elliptical regions 23001a-23001f may correspond to or be referred to as tiles, slices, bricks, or blocks, depending on the embodiment.

[0400] The point cloud data transmitting device according to the embodiment accurately encodes points located in important areas in the point cloud data 23000, and appropriately encodes unimportant areas. For example, when the point cloud data 23000 is data 23000 indicating a specific city as shown in FIG. 23, there may be some parts that require accurate encoding in the encoding process of the point cloud data transmitting device. For example, the areas 23001a to 23001f shown in the circular or elliptical shapes in FIG. 23 may be parts that require accurate encoding in the encoding process. In particular, while the point cloud data is huge, if the area that requires accurate encoding is specified as a specific area, a delay may occur in the transmission and reception process if all the point cloud data is encoded.

[0401] Therefore, in order to efficiently encode point cloud data, the point cloud data transmission device of the embodiment divides the point cloud data 2300 into one or more spherical regions (or elliptical regions) 23001a to 23001f, and performs encoding for the divided spherical regions 23001a to 23001f.

[0402] Point cloud data 23000 is data that includes one or more points. Point cloud data 23000 includes one or more regions that require encoding in a specific manner (or regions that require precise encoding in encoding). Regions that require encoding in a specific manner may be formed into a circular (or spherical) shape as shown in 23001a and 23001b of FIG. 23, or into an elliptical (or squashed sphere) shape as shown in 23001c to 23001f of FIG. 23. Regions that are formed into a circular (or spherical) shape as in the former are also called spherical regions (or circular bricks). Regions that are formed into an elliptical (or squashed sphere) shape as in the latter are also called elliptical regions (or spheroidal regions).

[0403] The method in which point cloud data is divided into one or more spherical or elliptical regions is also called spherical / elliptical region tiling.

[0404] According to an embodiment, if the point cloud data is represented based on a spherical coordinate system or a geographic coordinate system, the point cloud data can be divided into spherical regions. According to an embodiment, different coordinate systems (such as a cylindrical coordinate system) are used depending on the shape of the bricks.

[0405] The point cloud data transmitting device according to the embodiment can divide the point cloud data into one or more spherical regions 23001a, 23001b or elliptical regions 23001c to 23001f. The point cloud data transmitting device according to the embodiment generates a spherical / elliptical region usage flag (sphere_or_oval_flag) indicating whether a brick of the point cloud data is a spherical region or an elliptical region, and transmits the flag by including it in the bitstream.

[0406] The point cloud data transmission device according to the embodiment generates information about each of the spherical and / or elliptical regions and transmits the information in a bit stream. For example, the point cloud data includes information related to the pattern of each of the spherical or elliptical regions.

[0407] The spherical regions 23001a and 23001b refer to bricks configured in a spherical shape. The spherical regions 23001a and 23001b refer to a collection of points of point cloud data within a circular or spherical region within a bounding box. The point cloud data transmitting device generates center point information (e.g., center_point_xyz) and radius information (radius_distance) for each of the spherical regions 23001a and 23001b and transmits them in the form of a bit stream to each of the bricks.

[0408] The elliptical regions 23001c to 23001f refer to bricks configured in an elliptical or prolate shape. The elliptical regions 23001c to 23001f refer to a collection of points of point cloud data within a circular or spherical region within a bounding box. The point cloud data transmission device generates center point information (e.g., center_point_xyz), radius information (radius_distance), and azimuth angle information (azimuth_angle) for each of the elliptical regions 23001c to 23001f and transmits them in the form of a bit stream to each of the bricks.

[0409] The point cloud data transmitting device according to the embodiment uses spherical / elliptical region tiling to effectively encode the LiDAR data in a sequence in which objects are sparsely present and it is clear which parts of the data have objects and which do not have objects in space.

[0410] The point cloud data transmission device according to the embodiment can provide a high compression rate for data in which position information is distributed circularly, such as VP-tree, by using spherical / elliptical region tiling.

[0411] FIG. 24 is a diagram illustrating how a point cloud data transmitting device according to an embodiment divides point cloud data into one or more regions.

[0412] The operations according to the division method shown in Fig. 24 are performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, the XR device 1430 in Fig. 14, and the like, in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0413] 24 illustrates a point cloud data transmitting apparatus dividing a three-dimensional space occupied by point cloud data or point cloud data 24000 into attribute-based regions 24001 to 24003. The point cloud data transmitting apparatus according to the embodiment can encode the point cloud data according to the embodiment for each of one or more attribute-based regions 24001 to 24003. The operation of dividing the point cloud data illustrated in FIG. 24 is performed by the space dividing unit 15001 of FIG. 15 and the brick tiling unit 16001 of FIG. 16. The operation illustrated in FIG. 24 is performed, for example, by the data input unit 12000 of FIG. 12.

[0414] The point cloud data transmitting device according to the embodiment divides the point cloud data 24000 based on attribute information of points. For example, the point cloud data transmitting device forms one or more regions according to attributes (hue, reflectance, transparency, time information, object index, etc.) constituting the point cloud data. The method of dividing the region based on attribute information according to the embodiment is also called attribute-based region tiling (e.g., attribute-based brick tiling).

[0415] The left image of Fig. 24 shows a plurality of points 24000a to 24000c in a bounding box. The plurality of points includes, for example, points (P0, P2, P4, P5) having a reddish hue, points (P7, P8, P9) having a bluedish hue, and points (P1, P3, P6) having a greenish hue. The point cloud data transmitting device according to the embodiment can divide the point cloud data into one or more regions (e.g., bricks) according to the hue of the points.

[0416] The right image of Figure 24 shows a number of points 24000a to 24000c split into three bricks 24001, 24002, 24003. For example, points (P0, P2, P4, P5) having reddish hues are split into the first brick 24001, points (P7, P8, P9) having blue hues are split into the second brick 24002, and points (P1, P3, P6) having green hues are split into the third brick 24003.

[0417] In the point cloud data according to the embodiment, attribute information of a plurality of points is examined before dividing the plurality of points into one or more regions based on the attribute information. The operation of examining the attribute information of a plurality of points is performed by a data characteristic examination unit 16000 in FIG. 16. For example, the point cloud data according to the embodiment can examine reddish points among the points. For example, the operation of examining reddish points is performed based on the values ​​of RGB information of the points.

[0418] The point cloud data transmitting device (transmitting device) according to the embodiment may transmit, in a bit stream form, information indicating whether or not attribute-based area tiling according to the embodiment has been performed on the point cloud data. In addition, the point cloud data transmitting device (transmitting device) according to the embodiment may transmit, in a bit stream form, an attribute type (referenced_attribute_type) that is a division criterion when attribute-based area tiling has been performed. The division criterion according to the embodiment includes a plurality of attribute types, and the criterion is an attribute information value range (referenced_attribute_type_range) or an attribute information value (referenced_attribute_type_value).

[0419] The point cloud data transmitting device according to the embodiment can perform effective encoding on a sequence containing a large amount of geometry information corresponding to one or similar attribute information value by using attribute-based region tiling according to the embodiment.

[0420] FIG. 25 is a diagram illustrating how a point cloud data transmitting device according to an embodiment divides point cloud data into one or more regions.

[0421] The operations according to the division method shown in Fig. 25 are performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, the XR device 1430 in Fig. 14, and the like, in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0422] FIG. 25 illustrates a point cloud data transmitting device dividing a three-dimensional space occupied by point cloud data or point cloud data 25000 into regions 25001 and 25002 divided based on the axis lengths of bounding boxes. The point cloud data transmitting device according to the embodiment can encode the point cloud data according to the embodiment for each of the regions 25001 and 25002 divided based on the axis lengths of one or more bounding boxes. The operation of dividing the point cloud data illustrated in FIG. 25 is performed by the space dividing unit 15001 of FIG. 15 and the brick tiling unit 16001 of FIG. 16. According to the embodiment, the operation of dividing into the regions 25001 and 25002 (e.g., bricks) divided based on the axis lengths of the bounding boxes is also called adaptive axis length based region tiling (or adaptive axis length based brick tiling). The operation illustrated in FIG. 25 is performed, for example, by the data input unit 12000 of FIG. 12.

[0423] The point cloud data 25000 includes one or more points. The point cloud data is contained within a three-dimensional space (e.g., a bounding box). In one embodiment, the three-dimensional space (e.g., a bounding box) is represented by an x-axis, a y-axis, and a z-axis.

[0424] In the point cloud data 25000, for example, the length of any one of the x-axis, y-axis, and z-axis is shorter or significantly shorter than the lengths of the remaining two axes. For example, there is point cloud data expressing a terrain with a shape that is widely distributed on two axes except one axis. If the point cloud data transmitting device performs encoding on points within a bounding box regardless of the length of the axis, the encoding efficiency decreases. Therefore, the point cloud data transmitting device needs to divide the data into one or more regions according to the length of the axis of the three-dimensional space. Therefore, the point cloud data transmitting device according to the embodiment can calculate the relative lengths of the x-axis, y-axis, and z-axis, and divide the point cloud data into one or more regions based on the calculated relative lengths.

[0425] A point cloud data transmitting device (e.g., tile dividing unit 16000 in FIG. 16) according to an embodiment can divide the length of the longest axis (e.g., X-axis) of the x-axis length 25000x, the y-axis length 25000y, and the z-axis length 25000z in a three-dimensional space into the length of the shortest axis (e.g., Z-axis) to form one or more slices (or first bricks, 25001). Data divided into one or more slices (or first bricks) is a unit that can be encoded and decoded in parallel.

[0426] A point cloud data transmitting device according to an embodiment (for example, the tile dividing unit 18000 in FIG. 18 or the tiling unit 19001 in FIG. 19) can perform subdivision on a part of the first bricks, 25001. In the subdivision according to the embodiment, for some of the first bricks, the axis with the longest length (i.e., the y-axis, 25000y in the case of the 25001 brick in FIG. 25) is divided into units of the length of the axis with the shortest length (for example, the x-axis or z-axis) to form one or more detailed slices (or second bricks, 25002). Data divided into one or more detailed slices (or second bricks) is a unit that can be encoded and decoded in parallel.

[0427] The point cloud data transmission device according to the embodiment can also perform division based on the min / max information of points, regardless of the spatial data distribution, using the maximum and minimum numbers (maxPointNum, minPointNum) of points in the first brick.

[0428] The point cloud data transmitting device according to the embodiment determines whether to re-divide the first brick or the second brick based on the number of points in the region, the size of the space allocated to each region, and the density of points included in the region. According to the embodiment, when re-division is performed in space, it can be performed space adaptively, and slices can be divided by a method such as re-division based on a median axis other than a method of simply dividing along a specific axis.

[0429] The point cloud data transmitting apparatus according to the embodiment can effectively encode a sequence having a spatial unit correlation within one frame of point cloud data including a large terrain feature by using adaptive axis length based region tiling according to the embodiment. The point cloud data receiving apparatus according to the embodiment can effectively decode point cloud data including a large terrain feature.

[0430] The transmitting device according to the embodiment may divide the point cloud data (or points in a tile) into one or more slices. The slices according to the embodiment are configured such that points included in each slice do not overlap with each other in three-dimensional space. Conversely, the slices according to the embodiment may be configured such that points included in each slice have an area where they overlap with each other in three-dimensional space. In this manner, the transmitting device may configure slices to have an overlapping area in three-dimensional space, and such slices are called overlapping slices. The transmitting device according to the embodiment may encode point cloud data for real-time processing / low latency, scalability, and parallel processing by configuring overlapping slices.

[0431] The transmitting device according to the embodiment can configure overlapped slices according to the embodiment while dividing the point cloud data. The transmitting device according to the embodiment divides the point cloud data (original data) into tiles that can have bounding box information before encoding the point cloud data. A tile is the smallest possible spatial unit for the transmitting device to encode in parallel or for the receiving device to decode.

[0432] In the embodiment, the tile is further composed of slices, which are the smallest unit capable of defining a data structure. A slice is a set of points that constitute point cloud data, and is composed of a maximum of 1.1 million points and a minimum of 550,000 points. In the embodiment, the slice includes spatial regions that overlap each other.

[0433] In practice, before the encoding / decoding process is performed, the point cloud data according to the embodiment is divided into N tiles, and each tile is composed of N slices. The operation of dividing the point cloud data into one or more tiles is performed by a tile composition unit included in the transmitting device, and the operation of dividing one tile into one or more slices is performed by a slice composition unit of the transmitting device. At this time, there are various methods for dividing the point cloud data into one or more slices so as to include overlapping spatial regions.

[0434] The transmitting device (e.g., a slice construction unit) can divide the point cloud data (or points within a tile) into one or more overlapping slices based on a multi-point sampling based overlapping slice construction method, an importance based overlapping slice construction method, an attribute range based overlapping slice construction method, or a Moulton code order based overlapping slice construction method depending on the characteristics of the point cloud data.

[0435] The slice constructor according to the embodiment includes any one of a multi-point sampling based overlap slice constructor, an importance based overlap slice constructor, an attribute range based overlap slice constructor, and a Moulton code order based overlap slice constructor. The above-mentioned method will be described in detail with reference to Figures 26 to 30.

[0436] In addition, the transmitting apparatus according to the embodiment includes a sequential input slice constructing unit for each tile that divides slices in sequence in the tile output unit so that the slice divider does not wait for the time for other tiles to be divided when constructing overlapped slices. The above-mentioned method is described in detail with reference to FIG.

[0437] In addition, when constructing overlapping slices, a sequential input slice constructing unit for each tile is included, which divides slices in sequence in the tile output unit so that the corresponding slice divider does not have to wait for the time for other tiles to be divided. Various slice construction methods can be used for each tile.

[0438] FIG. 26 illustrates a division unit of a point cloud data transmission device according to an embodiment.

[0439] The division unit 26000 shown in Fig. 26 is performed by combining the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0440] The point cloud data transmission device according to the embodiment includes a division unit 26000. The division unit 26000 according to the embodiment means the spatial division unit 15001 in Fig. 15 and the brick tiling unit 16001 in Fig. 16. The division unit 26000 shown in Fig. 26 is, for example, a configuration included in the data input unit 12000 in Fig. 12.

[0441] The splitter 26000 according to the embodiment receives geometry data and / or attribute data according to the embodiment. The splitter 26000 can split the received geometry data and / or attribute data into one or more slices and output geometry data (bit stream) for each slice and / or attribute data (bit stream) for each slice.

[0442] The division unit 26000 according to the embodiment includes a tile division execution unit 26001 and / or a slice division execution unit 26002. The point cloud data transmission device according to the embodiment can divide the point cloud data into one or more tiles based on geometry data and characteristic data before the slice division according to the embodiment is performed.

[0443] The dividing unit 26000 according to the embodiment divides or distributes points of the point cloud data according to the embodiment into one or more slices. The slices according to the embodiment include overlapping areas in the 3D space. Therefore, the slice dividing unit of the point cloud data transmitting device according to the embodiment is also called an overlapping slice dividing unit 26002.

[0444] The tile division execution unit 26001 divides the point cloud data obtained according to the embodiment into one or more tiles.

[0445] The slice division execution unit 26002 partitions (or divides) points of point cloud data in a tile into one or more slices according to an embodiment. For example, if there are 100 points in one tile, the slice division unit 26002 can divide the corresponding tile into 10 slices including 10 points each. In this case, each slice consisting of 10 points can overlap with each other. That is, a portion of one or more slices includes an overlapping area.

[0446] The slice division execution unit 26002 according to the embodiment divides one tile or points of point cloud data into one or more slices, and outputs a geometry bitstream and a feature bitstream for each slice. After overlapped slice division according to the embodiment is performed, the transmission device according to the embodiment outputs a pair of geometry bitstream and feature bitstream independently for each slice, and can decode them in a combination unit.

[0447] The dividing unit 26000 according to the embodiment divides the points so that the slice dividing unit 26002 according to the embodiment includes an overlapping area between slices in the tile to form slices. The dividing unit 26000 or the slice dividing unit 26002 according to the embodiment generates overlapping slice compose information (or a flag) (overlapping_slice_compose_flag) indicating whether or not to form a slice to include an overlapping area.

[0448] When the value of the overlap slice compose information according to the embodiment indicates that overlapping slices are not to be constructed (also called non-overlapping slice division), the division unit according to the embodiment divides the points based on the axis-based slice construction method or the octree-based slice construction method described above.

[0449] When the value of the overlap slice compose information according to the embodiment indicates that overlap slices are to be constructed (also called overlap slice division), the division unit according to the embodiment divides the points based on a multi-point sampling based overlap slice construction method, an importance based overlap slice construction method, an attribute range based overlap slice construction method, or a Moulton code order based overlap slice construction method.

[0450] When the slice division execution unit 26002 of the embodiment performs slice division for each tile without going through a waiting state, if sequential input slice division for each tile is performed, the slice division execution unit 26002 signals information (sequential_input_slice_configuration_in_tile_flag) indicating whether or not division is performed based on such a method to the receiving device.

[0451] The slice division execution unit 26002 according to the embodiment transmits information (overlapping_slice_compose_method) on which division method is used in the encoder to transmit the geometry bitstream and the attribute bitstream to the encoding unit according to the embodiment. Various division methods can be used within one tile. In this case, the division method is signaled for each slice.

[0452] A method for dividing points in a tile or points of point cloud data into one or more slices to include overlapping regions (i.e., overlapping slice division method) will be described in detail from FIG.

[0453] The slice division unit 26000 according to the embodiment is also called a slice construction unit. The slice construction unit according to the embodiment constructs slices according to the embodiment based on a division method such as a multi-point sampling based overlap slice construction method, an importance based overlap slice construction method, a feature data range based overlap slice construction method, a sequential input based overlap slice construction method, or a Moulton code order based overlap slice construction method according to the characteristics of the point cloud data. According to the embodiment, slices divided according to various slice construction methods can be included in one tile.

[0454] FIG. 27 illustrates an overlapping slice division method according to an embodiment.

[0455] Figure 27 shows a method of constructing overlapping slices based on multiple points, which is one example of the overlapping slice division method according to the embodiment. The slice division method shown in Figure 27 is performed by the division unit 26000 or the slice division execution unit 26002 of Figure 26. The slice division method shown in Figure 27 is also performed by the spatial division unit 15001 of Figure 15 or the brick tiling unit 16001 of Figure 16. The operation shown in Figure 27 is performed, for example, by the data input unit 12000 of Figure 12.

[0456] The operation according to the division method shown in Fig. 27 is performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14, etc., in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0457] According to an embodiment, overlapped slices based on multi-point sampling refer to a method of sampling any N points within one tile and constructing one or more slices from the sampled points, as shown in FIG. 27.

[0458] The transmitting device according to the embodiment may sample using points close to the edges of the x, y, and z axes in the tile 27000 to divide the tile according to the embodiment into one or more slices. For example, the transmitting device according to the embodiment samples a point 27001a-2 closest to the horizontal axis (e.g., x axis) in the tile 27001 and a point 27001a-1 closest to the vertical axis (e.g., y axis) in the tile to configure a first slice 27001a. A method of deriving a point close to the edges of the x, y, and z axes in the tile 27000 includes a method of calculating a point with the minimum or maximum x, y, and z coordinate values ​​in a three-dimensional space (bounding box). The transmitting device according to the embodiment configures a slice to include the two sampled points 27001a-1 and 27001a-2.

[0459] The transmitting device according to the embodiment samples arbitrary points 27001b-1, 27001b-2 in the tile 27000 to divide the tile according to the embodiment into one or more slices. The transmitting device according to the embodiment configures a slice 27001b to include the two sampled points 27001b-1, 27001b-2.

[0460] The transmitting device according to the embodiment uses a method of deriving N points that are separated by a specific distance as a method of deriving sampling points from among the points according to the embodiment. The transmitting device according to the embodiment generates slices using two or more points and sets a bounding box for the slice. In the transmitting device according to the embodiment, the points sampled to form a bounding box in one slice can be reused to form a bounding box for another slice, and overlapping slices are then formed.

[0461] The transmitting device according to the embodiment can generate a set including identifiers of the points included in each slice by constructing slices composed of the sampled points. At this time, although there may be points that do not fall within a slice, the receiving device can reconstruct one or more slices using only those points. Alternatively, the receiving device can construct a slice by adding those points to the slice with the fewest number of points.

[0462] According to the embodiment, the receiving device may check whether the minimum number of points (e.g., 550,000) and the maximum number of points (e.g., 1,100,000) are distributed for each slice configured with sampling points. If the corresponding slice does not satisfy the above-mentioned point number distribution, the receiving device may sample other sampling points among the points in the tile to generate a new bounding box, and then check whether the slice satisfies the next point number distribution.

[0463] With this configuration, the receiving device can quantize, align, offset adjust, and scale points belonging to different slices within one slice, thereby providing scalability for point cloud compression and effectively conveying information about a user's region of interest.

[0464] FIG. 28 illustrates an overlapping slice division method according to an embodiment.

[0465] The operation according to the division method shown in Fig. 28 is performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14, etc., which are combined together. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0466] Fig. 28 shows an importance-based overlap slice configuration method, which is one embodiment of the overlap slice division method according to the embodiment. The slice division method shown in Fig. 28 is performed by the division unit 26000 or the slice division execution unit 26002 in Fig. 26. Also, the slice division method shown in Fig. 27 is performed by the spatial division unit 15001 in Fig. 15 or the brick tiling unit 16001 in Fig. 16. The operation shown in Fig. 28 is performed, for example, by the data input unit 12000 in Fig. 12.

[0467] The importance-based overlap slice construction method according to the embodiment refers to a method of sampling one or more points with high importance within a tile and constructing one or more slices using the sampled points, as shown in FIG. 28.

[0468] The transmitting device according to the embodiment samples points of high importance within one tile and configures one or more slices based on the samples, as shown in Fig. 28. The method by which the transmitting device according to the embodiment classifies points of high importance means a hue value or reflectance value selected as a representative value among characteristic values ​​within one tile.

[0469] The importance-based representative value according to the embodiment is also called a representative point.

[0470] For example, since there may be points with similar hues in a particular space (or adjacent spaces), the transmitting device according to the embodiment may specify a point having a representative value of the hue (or the value itself) for each tile or for each three-dimensional space. The transmitting device according to the embodiment generates a slice having a bounding box of a three-dimensional shape centered on a point designated as a representative value. The three-dimensional shape centered on a point designated as a representative value may be, for example, a cubic shape, a spherical shape, a rectangular parallelepiped shape, or a cylindrical shape. The transmitting device according to the embodiment may configure points included within a range of a three-dimensional shape centered on a point designated as a representative value as one slice.

[0471] There are various methods for deriving or finding a point having a representative value based on importance according to an embodiment. For example, the transmitting device according to an embodiment derives a distribution of characteristic values, and then determines one or more points having the most distributed hue as a point having a representative value based on importance. For example, the lower part of FIG. 28 shows a reflectance value distribution 28002 showing a distribution of characteristic values ​​based on the reflectance values ​​of points in one 3D space or tile.

[0472] Referring to the reflectance value distribution 28002, for example, the x-axis indicates the reflectance value 28002b, and the y-axis indicates the number of points having the corresponding reflectance value 28002a. For example, referring to the reflectance value distribution chart 28002 shown in FIG. 28, there are many points having reflectance values ​​between 32 and 50 in the corresponding 3D space or tile, and there are less than 200 points each having a reflectance value of 140 or more. Therefore, in the transmitting device according to the embodiment, referring to the reflectance value distribution chart 28002, for example, a point having a reflectance value of 47 is a point having a representative value based on importance according to the embodiment.

[0473] Also, for example, referring to the reflectance value distribution chart 28002 shown in FIG. 28, there may be a large number of points for a particular reflectance value in a corresponding 3D space or tile depending on the characteristics of the data. For example, there are more points having reflectance values ​​of 0, 15, 28, 50, 62, etc. than other reflectance values. Such reflectance values ​​are called peak values. In this case, the transmitting device according to the embodiment may determine such reflectance values ​​as points having a representative value based on importance.

[0474] The transmitting device according to the embodiment may designate a point having a peak value among the characteristic data according to the embodiment as the representative point. In addition, the transmitting device according to the embodiment may designate a point having a characteristic data value most similar to the average value of the characteristic data as the representative point other than the representative point. In addition, the transmitting device according to the embodiment may designate a point having an average value of the characteristic data, a point having a characteristic value similar to the average value, a minimum value / maximum value of the characteristic data, etc. as the representative point.

[0475] The transmitting device according to the embodiment may configure slices including points within a cubic or rectangular bounding box that occupies a certain distance above, below, left, right, front, and back with the representative point as its midpoint. The transmitting device according to the embodiment may also configure slices including points within a spherical bounding box that occupies a certain distance from the representative point as its midpoint. According to the embodiment, points that do not belong to a slice belong to the nearest slice, and the transmitting device according to the embodiment may configure another slice using such points.

[0476] In the embodiment, when point cloud data is divided based on the method shown in FIG. 28, the transmitting device can signal this using signaling information (e.g., value_based_overlapping_slice_partition) and transmit it to the transmitting device.

[0477] With this configuration, the receiving device can quantize, align, offset adjust, and scale points belonging to different slices within one slice, thereby achieving scalability for point cloud compression and effectively transmitting information about a user's area of ​​interest.

[0478] FIG. 29 illustrates an overlapping slice division method according to an embodiment.

[0479] The operation according to the division method shown in Fig. 29 is performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14 in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0480] Figure 29 shows a method of constructing overlap slices based on a characteristic data range, which is one example of an overlap slice division method according to an embodiment. The slice division method shown in Figure 29 is performed by the division unit 26000 or the slice division execution unit 26002 of Figure 26. Also, the slice division method shown in Figure 27 is performed by the space division unit 15001 of Figure 15 or the brick tiling unit 16001 of Figure 16. The operation shown in Figure 29 is performed, for example, by the data input unit 12000 of Figure 12.

[0481] The method of constructing overlapping slices based on a characteristic data range according to the embodiment means a method of deriving (sampling) one or more points having characteristic values ​​within a specific range within a tile to construct one or more slices, as shown in FIG.

[0482] A point cloud data transmission device according to an embodiment can sort (ascending or descending order) points belonging to one tile or three-dimensional space based on characteristic data values ​​and divide them into ranges of characteristic data values ​​to create one or more slices.

[0483] The transmitting device according to the embodiment does not perform ascending sorting according to the embodiment on the characteristic data of points of the point cloud data, but instead configures slices in a manner that assigns a range of characteristic data values ​​for each slice and assigns points to slices based on the assigned range of characteristic data values.

[0484] For example, referring to FIG. 29, the transmitting device according to the embodiment allocates a range of characteristic data values ​​(e.g., hue values) to each of N slices. The transmitting device according to the embodiment checks the characteristic data of the points and compares the corresponding characteristic data value (e.g., hue value) range for each point. The transmitting device according to the embodiment configures the slices by inserting the index of the point into the slice. According to the embodiment, the characteristic data value (hue value) range is specifically based on the hue value and reflectance value, etc. According to the embodiment, one or more slices are distributed based on the characteristic data value, so that each slice can overlap based on the bounding box. This method is used in a scenario where a specific characteristic data range value must be decoded with priority when the point cloud data according to the embodiment is decoded.

[0485] In addition to the method of allocating the range of the characteristic data value, the transmitting device according to the embodiment can automatically determine the range of the characteristic data value according to the distribution degree of the characteristic data and perform distribution. The distribution degree of the characteristic data according to the embodiment can be divided based on the characteristic value of the representative point, similar to the reflectivity distribution degree of the overlapping slice based on importance. The representative point corresponds to the average value of the characteristic data, a similar attribute value, etc.

[0486] According to the embodiment, when the transmitting device divides slices based on the attribute data value, the range of the attribute data value is smaller than the entire sequence, so when selecting adjacent nodes, the residual value tends to be smaller. Therefore, when distributing slices based on the range of the attribute data value according to the embodiment, the compression rate is high.

[0487] In the embodiment, when point cloud data is divided by the method shown in FIG. 29, the transmitting device signals the divided point cloud data using signaling information (e.g., attribute_range_based_slice_partition) and transmits the signaling information to the transmitting device.

[0488] FIG. 30 illustrates an overlapping slice division method according to an embodiment.

[0489] The operation according to the division method shown in Fig. 30 is performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14, etc., in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0490] Figure 30 shows a Morton code-based overlap slice construction method, which is one example of overlap slice division methods according to an embodiment. The slice division method shown in Figure 30 is performed by the division unit 26000 or slice division execution unit 26002 of Figure 26. The slice division method shown in Figure 27 is performed by the spatial division unit 15001 of Figure 15 or the brick tiling unit 16001 of Figure 16. The operation shown in Figure 30 is performed, for example, by the data input unit 12000 of Figure 12.

[0491] The Morton code based overlap slice construction method according to the embodiment means a method of deriving a Morton code of a point in one tile and constructing one or more slices based on the derived Morton code, as shown in Fig. 30. The Morton code according to the embodiment is generated in a transmission device according to the embodiment, for example, in the spatial division unit 15001 of Fig. 15 or the brick tiling unit 16001 of Fig. 16.

[0492] The Morton code refers to a code generated by bit-wise interleaving coordinate values ​​(e.g., x, y, z coordinate values) of point position information (present in geometry data). According to an embodiment, each point of the point cloud data has a different Morton code.

[0493] Fig. 30(a) shows a formula for generating a Morton code using coordinate values ​​of a point according to an embodiment. The Morton code according to the embodiment includes, for example, a step of converting the coordinate values ​​of a point into binary numbers as shown in 30A00 of Fig. 30(a) and / or a step of interleaving bits of the coordinate values ​​expressed in binary numbers as shown in 30A01 of Fig. 30(a).

[0494] In FIG. 30(a), x_k, y_k, and z_k respectively mean the k-th digit bit of the x-coordinate value, y-coordinate value, and z-coordinate value. In FIG. 30(a), N means the total number of digits of the coordinate value expressed in binary. x_i, y_i, and z_i mean the i-th point among the points of the point cloud data according to the embodiment. MC(x, y, z) means the Morton code of a specific point according to the embodiment. M means the total number of digits of the coordinate value of the point according to the embodiment.

[0495] The point cloud data transmission device generates a Morton code by interleaving bits of coordinate values ​​of a point. For example, the transmission device according to the embodiment performs an operation of converting coordinate values ​​of a point for each axis into binary numbers (30A00). The point cloud data transmission device according to the embodiment generates a Morton code by interleaving the coordinate values ​​for each axis in sequence (30A01).

[0496] For example, assume that a first point having coordinate values ​​of (3,4,5) as position information is to be interleaved. The transmitting device can derive the binary number of the coordinate values ​​for each axis of the first point (011,100,101) (30A00). The transmitting device sequentially arranges the most significant bit (e.g., 0) of the x value of the coordinate value of the first point, the most significant bit (e.g., 1) of the y value of the coordinate value of the first point, and the most significant bit (e.g., 1) of the z value of the coordinate value of the first point (e.g., 011). The transmitting device sequentially arranges the second most significant bit (e.g., 1) of the x value of the coordinate value of the first point, the second most significant bit (e.g., 0) of the y value of the coordinate value of the first point, and the second most significant bit (e.g., 0) of the z value of the coordinate value of the first point (e.g., 100). The Morton code according to the embodiment includes an arrangement of the second most significant bits of the coordinate values ​​arranged consecutively to the arrangement of the first most significant bits of the coordinate values ​​(e.g., 011100). The point cloud data transmitting device (or Morton code generating unit) sequentially arranges the least significant bit (e.g., 1) of the x value of the coordinate value of the first point, the least significant bit (e.g., 0) of the y value of the coordinate value of the first point, and the least significant bit (e.g., 1) of the z value of the coordinate value of the first point. The Morton code according to the embodiment includes an arrangement of least significant bits consecutively arranged in an arrangement of the first most significant bit of the coordinate value and an arrangement of the second most significant bit of the coordinate value (e.g., 011100101) (30A01). The Morton code (e.g., 01110010) represented in binary for the first point according to the embodiment can be represented in decimal (299).

[0497] The transmitting device according to the embodiment uses a Moulton Code order search (also called Z-order search) of points of the point cloud data to construct one or more slices according to the embodiment. Searching in Moulton Code order means a method of searching in ascending or descending order of the Morton Code when the position information of the points according to the embodiment is converted to the Morton Code.

[0498] 30(b) shows a method for dividing one or more slices by a transmitting device according to an embodiment, which means a method using a Morton code of points according to an embodiment. FIG 30(b) shows a method for constructing overlapping slices based on a Morton code according to an embodiment.

[0499] The Morton code-based overlap slice construction method shown in Figure 30(b) includes a step of sorting points in ascending or descending order based on the Morton code value (30B00), a step of dividing the sorted points into slices having a number of points ranging from the minimum number of points (minPoint) to the maximum number of points (maxPoint) (30B01), and / or a step of merging and redividing slices in which no points exist for a number ranging from the minimum number of points (minPoint) to the maximum number of points (maxPoint) (30B02).

[0500] The transmitting device according to the embodiment generates one Morton code for each point from a method for deriving a Morton code according to the embodiment (30B00). The transmitting device according to the embodiment sorts the Morton codes for each point in ascending or descending order. Referring to 30B00 in FIG. 30(b), TilePointNum means the number of points in a tile according to the embodiment. MC_i means the Morton code of the i-th point. sort(MC_i) means a function for sorting points in ascending order and / or descending order based on the Morton code according to the embodiment. The sorting method according to the embodiment (e.g., whether sorted in ascending order or descending order) is conveyed by signaling information transmitted from the transmitting device to the receiving device.

[0501] The transmitting device according to the embodiment divides the point cloud data into one or more slices such that points are distributed within a range of a minimum number of points to a maximum number of points within one slice using the aligned Morton code according to the embodiment (30B01). For example, the transmitting device according to the embodiment inserts point indices into the slices such that only the range of the minimum number of points to the maximum number of points is included within one slice.

[0502] Referring to 30B01 in FIG. 30(b), minPoint means the minimum number of points present in one slice, and maxPoint means the maximum number of points present in one slice. minPoint and maxPoint mean the minimum number of points and the maximum number of points according to the embodiment. sliceN means the Nth slice according to the embodiment. For example, a transmitting device according to the embodiment generates N slices.

[0503] Referring to 30B02 in FIG. 30(b), the transmitting device can merge some of the generated N slices with other slices. The transmitting device can split (e.g., splitHalf) some of the generated N slices. For example, N-1 slices are slices that include (maximum number of points-1) points. Also, the remaining slice is a slice that includes fewer points than (minimum number of points). The remaining slice is merged with some of the N-1 slices according to the embodiment, or is divided by the transmitting device and merged with each of the N-1 slices.

[0504] In the embodiment, when point cloud data is divided based on the method shown in FIG. 30, the transmitting device signals the divided data using signaling information (e.g., morton_order_based_slice_partition) and transmits the signaling information to the transmitting device.

[0505] According to the embodiment, the Morton code searches all points existing in the 3D space based on the proximity, and therefore, the transmitting device according to the embodiment has an advantage that it can search for points close to a specific point with a small amount of calculation by using the overlapping slice construction method based on the Morton code.

[0506] In addition, by using the Morton code-based overlapping slice construction method according to the embodiment, the transmitter according to the embodiment can derive the Morton code to perform calculations much faster than space division calculations using an octree, and can achieve efficient point cloud data encoding by minimizing overlapping space during space division.

[0507] FIG. 31 illustrates an overlapping slice division method according to an embodiment.

[0508] The operation according to the division method shown in Fig. 31 is performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14, etc., which are combined together. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0509] Fig. 31 shows a slice construction method for each tile, which is one embodiment of the overlapping slice division method according to the embodiment. The sequential input slice construction method for each tile shown in Fig. 31 is performed by the division unit 26000 or slice division execution unit 26002 in Fig. 26. Also, the sequential input slice construction method for each tile shown in Fig. 27 is performed by the spatial division unit 15001 in Fig. 15 or the brick tiling unit 16001 in Fig. 16. The operation shown in Fig. 31 is performed, for example, by the data input unit 12000 in Fig. 12.

[0510] The tile-by-tile sequential input slice construction method according to the embodiment means a method of dividing point cloud data according to the embodiment into one or more tile units, as shown in FIG. 31, and dividing each divided tile into one or more slices.

[0511] FIG. 31(a) shows an operation of a transmitting device according to an embodiment, which divides point cloud data into tiles and divides the divided tiles into slices.

[0512] As shown in FIG. 31(a), the transmitting device according to the embodiment divides point cloud data into one or more tiles based on the number of points, etc. (or data size), stores them in a buffer, and sequentially divides the data in the tiles stored in the buffer into one or more slices and transmits them to the encoder according to the embodiment.

[0513] Thus, a transmitting device according to an embodiment (or a spatial division unit according to FIG. 15) includes, for example, a tile input unit 31A00, a tile composition unit 31A01, a tile output unit 31A02, a slice input unit 31A03, a slice composition unit 31A04 and / or a slice output unit 31A05.

[0514] The point cloud data obtained according to the embodiment is input to the tile input unit 31A00.

[0515] The tile construction unit 31A01 receives point cloud data input by the tile input unit 31A00 according to the embodiment, and divides the point cloud data into one or more tiles. The tile construction unit 31A01 according to the embodiment first waits 31A01a until the input tile is generated before outputting the tile.

[0516] The tile output unit 31A02 outputs one or more tiles generated by dividing the point cloud data.

[0517] The slice input unit 31A03 receives data for one tile and inputs the points contained therein.

[0518] The slice construction unit 31A04 generates (or divides) one or more slices using points contained in one tile input by the slice input unit 31A03.

[0519] The slice output unit 31A05 outputs the generated one or more slices, which are then transmitted to an encoder according to the embodiment, slice by slice.

[0520] The one or more slices divided according to the embodiment may or may not overlap in space with each other.

[0521] FIG. 31(b) shows an operation of a transmitting device according to an embodiment, which divides point cloud data into tiles and divides the divided tiles into slices.

[0522] As shown in FIG. 31(b), the transmitting device according to the embodiment can divide point cloud data into one or more tiles based on the number of points, etc. (or data size), store them in a buffer, and sequentially divide the data in the tiles stored in the buffer into one or more slices and transmit them to the encoder according to the embodiment.

[0523] Thus, a transmitting device according to an embodiment (or a spatial division unit according to FIG. 15) includes, for example, a tile input unit 31B00, a tile composition unit 31B01, a tile output unit 31B02, a slice input unit 31B02, a slice composition unit 31B03 and / or a slice output unit 31B04.

[0524] Point cloud data obtained according to the embodiment is input to tile input unit 31B00.

[0525] The tile construction unit 31B01 receives point cloud data input to the tile input unit 31B00 according to an embodiment and divides the point cloud data into one or more tiles.

[0526] The tile output unit or slice input unit 31A02 generates one or more tiles by dividing the point cloud data, and receives data relating to the points contained therein.

[0527] The slice construction unit 31B03 generates (or divides) one or more slices using points contained in one tile input by the slice input unit 31A02.

[0528] The slice output unit 31B04 outputs the generated one or more slices. The output slices are transmitted to an encoder according to the embodiment for each slice.

[0529] According to an embodiment, the one or more divided slices may or may not overlap in space with each other.

[0530] The transmitting device according to the embodiment divides the point cloud data into one or more slices as shown in Fig. 31(b), so that while the transmitting device according to the embodiment configures tiles for each tie size, slices can be divided separately for other tiles. Therefore, with the configuration as shown in Fig. 31(b), the transmitting device can efficiently divide points into multiple slices quickly.

[0531] In the embodiment, when point cloud data is divided based on the method shown in FIG. 30, the transmitting device can signal the divided data using signaling information (e.g., sequential_input_slice_configuration_in_tile) and transmit the signal to the transmitting device.

[0532] FIG. 32 illustrates an operation of a point cloud data transmission device according to an embodiment for determining whether to divide point cloud data into one or more slices.

[0533] The operation of determining whether or not to divide shown in Fig. 32 is performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14 in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0534] The operation shown in Fig. 32 is performed by the point cloud data transmission device according to the embodiment, the spatial division unit 15001 shown in Fig. 15, the data characteristic examination unit 16000 shown in Fig. 16, the brick tiling unit 16001, the divided data input unit 16002, and the division unit 26000 shown in Fig. 26. The point cloud data transmission device (or the spatial division unit) according to the embodiment divides the point cloud data into one or more slices. The operation shown in Fig. 32 is performed, for example, by the data input unit 12000 in Fig. 12.

[0535] The point cloud data transmitting device according to the embodiment checks whether or not the division operation is to be performed by first checking the conditions for performing the division operation of the slice before performing the division (distribution) according to the embodiment.

[0536] In the embodiment, before the splitting operation is performed, if the number of points contained in one frame of point cloud data is less than the maximum number of points (MaxPointNum) (Yes), the transmitting device in the embodiment immediately performs encoding using the encoding unit, excluding the splitting operation and the refining operation (or merging operation) in the embodiment (32000a) (Yes).

[0537] The maximum number of points (MaxPointNum) according to the embodiment is predetermined information (value) in the point cloud data transmitting device or a value input and set by a user. The maximum number of points according to the embodiment is determined based on the characteristics of the point cloud data. The maximum number of points according to the embodiment can be transmitted to the receiving device according to the embodiment by signaling information.

[0538] The transmitting device according to the embodiment can increase the encoding efficiency of the encoder of the transmitting device for point cloud data encoded in slice units by setting the number of maximum points according to the embodiment. The transmitting device according to the embodiment can increase the encoding efficiency adaptively to the characteristics of the data by setting the number of maximum points based on the characteristics of the point cloud data.

[0539] Therefore, the method includes a step 32000a for checking the number of points currently inserted in one slice.

[0540] If the number of points currently inserted in the corresponding slice is greater than the maximum number of points (MaxPointNum), the point cloud data transmitting device (or space dividing unit) according to the embodiment may divide 32001 the point cloud data according to the embodiment and / or refine 32002 the point cloud data according to the embodiment.

[0541] If the number of points currently inserted in the corresponding slice is less than the maximum number of points (MaxPointNum), the point cloud data transmitting device (or spatial division unit) according to the embodiment may not divide 32001 or refine 32002 the point cloud data according to the embodiment, and the point cloud data is encoded by the encoder according to the embodiment.

[0542] The operation of dividing 32001 the point cloud data means an operation of dividing the point cloud data (or points in a tile) described in Figures 15 to 31. The point cloud data transmitting device (or the space dividing unit) may further include a point cloud data dividing unit that performs the operation of dividing 32001 the point cloud data.

[0543] The operation of refining the point cloud data 32002 refers to the operation of refining points in the divided slices according to the embodiment. The point cloud data transmission device (or the space division unit) further includes a point cloud data refinement unit that performs the operation of refining the point cloud data 32002.

[0544] A refinement operation according to an embodiment refers to a merger or split operation of one or more slices according to an embodiment. For example, a transmitting device may merge some of the slices for a specific purpose to generate one slice. Similarly, a transmitting device may re-divide some of the slices to generate more slices. A transmitting device according to an embodiment may check the number of points of adjacent slices of a particular slice for refinement.

[0545] For example, if the number of points in one slice is greater than the maximum number of points (MaxPointNum) according to an embodiment, the transmitting device divides the corresponding slice into two or more slices. As another example, if the number of points in one slice is less than the minimum number of points (MinPointNum) according to an embodiment, the transmitting device merges the corresponding slice with another adjacent slice.

[0546] For example, the point cloud data transmitting device according to the embodiment, after performing a division operation according to the embodiment (i.e., after slice partition), splits a slice having a number of points greater than MaxPointNum according to the embodiment into one or more sub-slice, or merges slices having a number of points smaller than MaxPointNum into one slice.

[0547] Splitting is an example of an operation of the refining step according to an embodiment, where if a point count (Asize) is greater than MaxPointNum, the slice containing that point is split into n partitions. A partition can also be a slice. For example, n=ceil(Asize / MaxPointNum).

[0548] Merging means that if the number of points in a slice is less than MinPointNum, it can be merged with the previous slice or the next slice. The previous slice is the slice with a slice index that is one smaller than the current slice, and the next slice is the slice with a slice index that is one larger than the current slice.

[0549] Merging can be performed, for example, based on the following operations:

[0550] 1) If the current slice is the first slice, the direction of the slice to be merged is the direction of the next slice.

[0551] 2) If the current slice is the last slice, the direction of the slice to be merged is the direction of the prior slice.

[0552] 3) There may be cases where the current slice is neither the first slice nor the last slice. Let the sum of the points of the current slice and the prior slice be SumFront, and the sum of the points of the current slice and the next slice be SumNext.

[0553] a) If (SumFront > MaxPointNum and SumNext > MaxPointNum) or (SumFront < MaxPointNum and SumNext < MaxPointNum), the direction of the slice to be merged is the direction of merging with the slice having more points.

[0554] b) If it is not the case in a) above, the direction of the slice to be merged is the direction of merging with the slice having fewer points.

[0555] The transmission device according to the embodiment traverses all the slices merged after merging and compares SubMerged with MaxPointNum (SubMerged indicates the number of points of the merged slice). If SubMerged < MinPointNum, the transmission device continues to perform the merging of the corresponding slice. If SubMerged > MaxPointNum, the transmission device splits the merged slice at the gap of MaxPointNum. If MinPointNum < SumMerged < MaxPointNum, the transmission device maintains the corresponding slice and checks the next slice.

[0556] When the operation shown in FIG. 32 is represented in pseudo code, it is as follows.

[0557] std::vector <partition>curSlices;

[0558] switch (params→partition.method || inputPointCloud.getPointCount()

[0559] <params→partition.sliceMaxPoints) {

[0560] case PartitionMethod::kNone:

[0561] return 1

[0562] case PartitionMethod::kUniformGeom:

[0563] partitionByUniformGeom()

[0564] case PartitionMethod::kOctreeUniform:

[0565] partitionByOctreeDepth()

[0566] params->partition.method indicates the partition method of the partition operation according to the embodiment. The partition method is any of the methods described in Figures 26 to 31.

[0567] inputPointCloud.getPointCount() refers to a method for returning the number of points in point cloud data according to an embodiment.

[0568] params→partition.sliceMaxPoints means the maximum number of points included in one slice (or tile). That is, referring to the pseudo code, if the number of points of the input point cloud data is less than params→partition.sliceMaxPoints (or MaxPointNum), partitionByUniform() and partitionByOctreeDepth() are not performed and a single slice is output, as shown in the pseudo code. Also, referring to the pseudo code, if the number of points of the input point cloud data is more than params→partition.sliceMaxPoints, the operation of dividing the point cloud data according to the embodiment can be performed by performing one of the syntaxes partitionByUniform() and partitionByOctreeDepth(). For example, if a signal is received indicating that the partition operation is to be performed based on the slice partition scheme of Uniform-Geometry partition along the longest edge shown in Fig. 19 (params->partition.method == PartitionMethod::kUniformGeom), distribution is performed based on this partition scheme (partitionByUniformGeom()). For example, if a signal is received indicating that the partition operation is to be performed based on the slice partition scheme of Uniform-Geometry partitioning Using Octree shown in Fig. 19 (params->partition.method == PartitionMethod::kOctreeUniform), distribution is performed based on this partition scheme (partitionByOctreeDepth()).

[0569] Therefore, the point cloud data transmitting and receiving device according to the embodiment determines whether to perform data division and / or data refinement based on the maximum number of points (MaxPointNum). When the number of points in the point cloud data is less than a certain value, a separate process of dividing or merging other points is not performed, and the points can be quickly compressed or decoded.

[0570] Similarly, if the number of points in the point cloud data is greater than a certain value, further operations such as data splitting and / or data refinement are performed to process in efficient data units to improve encoding / decoding performance.

[0571] In this embodiment, the value of the maximum number of points (MaxPointNum) is input by the user.

[0572] In the illustrative embodiment, the maximum point number value (MaxPointNum) is generated by a system setting.

[0573] In this embodiment, the maximum point number value (MaxPointNum) is transmitted as parameter information within the point cloud bitstream (see FIG. 38).

[0574] The transmitting device according to the embodiment is not limited to the above pseudo code, and can perform the division method according to the embodiment based on the division method described with reference to FIGS.

[0575] FIG. 33 illustrates an operation of a point cloud data transmission device according to an embodiment for dividing point cloud data into one or more slices.

[0576] The division operation shown in Fig. 33 is performed by the transmitting device 1000, the transmitter 10003, the point cloud video encoder 10002, the encoding 18001, the transmitting unit 18002, the encoder in Fig. 4, the transmitting device in Fig. 12, and the XR device 1430 in Fig. 14 in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0577] When a transmitting device (or a spatial division unit included in a transmitting device) according to an embodiment performs division using a division method according to an embodiment, an object to be divided is specified first. The object to be divided according to an embodiment is various data.

[0578] Regarding the input point cloud data according to the embodiment, the point cloud data that has not been quantized is input to the division unit, or the quantized point cloud data is input to the division unit. Regarding the input point cloud data according to the embodiment, the point cloud data to which an offset is applied or to which an offset is not applied may be input to the division unit.

[0579] Therefore, the point cloud data transmission device (or the division unit) according to the embodiment further performs an operation of quantizing the point cloud data before dividing the point cloud data. The point cloud data transmission device (or the space division unit) further includes a point cloud data quantization unit 33000 that performs an operation of quantizing the point cloud data.

[0580] The point cloud data transmission device (or division unit) according to the embodiment does not quantize the point cloud data, and can divide it into one or more slices according to the embodiment.

[0581] The point cloud data transmitting device (or the dividing unit) according to the embodiment generates and transmits signaling information indicating whether or not the quantization operation according to the embodiment has been performed. Therefore, the point cloud data receiving device according to the embodiment can perform decoding according to the embodiment based on the signaling information indicating whether or not the quantization operation according to the embodiment has been performed.

[0582] The point cloud data quantization unit 33000 according to the embodiment performs some or all of the operations of the quantization unit (Quantize and Remove Points (Voxelize), 40001) shown in Fig. 4. For example, the quantization unit 33000 performs voxelization based on the quantized positions to reconstruct the quantized points.

[0583] Before being encoded (compressed), the point positions are internally represented as non-negative d-bit integers. To obtain integers according to the embodiment, the transmitting device (or quantizer, 33000) according to the embodiment can round the point positions in the internal coordinate system. If X_n^inT is the point position in the internal coordinate system, the representation of a non-negative d-bit integer is as follows: Round() is a function that rounds the components of a vector to the nearest integer.

[0584] TIFF0007640526000019.tif946

[0585] According to the embodiment, when the transmitting device (or quantization unit, 33000) quantizes point cloud data, multiple points may exist at the same position, and such multiple points are called duplicate points. The process of removing such duplicate points is optionally performed by the transmitting device (or quantization unit, 33000). If the transmitting device (or quantization unit, 33000) removes duplicate points, the transmitting device (or quantization unit, 33000) can remove points having identical quantized coordinates. In order for the transmitting device to detect duplicate points according to the embodiment, an STL set data structure is used.

[0586] According to the embodiment of the quantizer 33000, points having the same quantization position and different characteristic data are merged into one point.

[0587] All or part of the operation of the quantization unit 33000 according to the embodiment, the operation of removing duplicate points, and / or the operation of assigning characteristic data to the remaining points, are also called voxelization. According to the embodiment, voxelization refers to a process of grouping points into one or more voxels. A set of voxels according to the embodiment is a unit cube that fills the range of [i-0.5,i+0.5)x[j-0.5,j+0.5)x[k-0.5,k+0.5) for integers i, j, and k having values ​​from 0 to 2^d-1. According to the embodiment, the positions of all points in a voxel are quantized to the center of the voxel, and the characteristic data of all points in the voxel are combined or averaged and assigned to the voxel. According to the embodiment, if there is any point of point cloud data in a voxel, the voxel is said to be occupied.

[0588] The point cloud data division unit 33001 and the point cloud data refinement unit 33002 according to the embodiment mean the point cloud data division unit 32001 and the point cloud data refinement unit 32002 described in Fig. 32. The operation of Fig. 33 according to the embodiment is expressed in pseudo code as follows.

[0589] quantizedInputCloud=quantization(inputPointCloud)

[0590] switch(params→partition.method)

[0591] refineSlices(quantizedInputCloud)

[0592] The point cloud data transmitting device according to the embodiment performs operations such as data quantization, data division, and data refinement during the encoding process.

[0593] The point cloud data transmitting apparatus according to the embodiment may skip the data quantization 33000 operation for efficient compression performance depending on the data type, attribute, etc. Also, information regarding whether the data quantization operation is performed can be included in the bit stream (see FIG. 38) as signaling information and transmitted. The point cloud data receiving apparatus according to the embodiment can check whether data quantization is performed based on the information included in the bit stream, and restore the point cloud data with low delay by performing a reverse process on the transmitting side.

[0594] According to the embodiment, quantization(inputPointCloud) refers to an operation of quantizing point cloud data according to the embodiment by the quantization unit 33000 according to the embodiment.

[0595] switch(param→partition.method) refers to an operation of dividing the quantized point cloud data according to an embodiment. For example, switch(param→partition.method) refers to dividing the quantized point cloud data into one or more slices by a slice partitioning method. The operation of dividing the quantized point cloud data refers to the operation described in FIG. 32.

[0596] refineSlices(quantizedInputCloud) indicates an operation of the transmission device according to the embodiment refining point cloud data divided by the switch(param→partition.method) syntax. The operation of refining point cloud data means the operation described in FIG.

[0597] FIG. 34 shows a point cloud decoder according to an embodiment.

[0598] The division operation shown in Fig. 34 is performed by the receiving device 10004, the receiving unit 10005, the point cloud video decoder 10006, the decoding 18003, the transmitting unit 18002, the decoders in Fig. 10 and Fig. 11, the receiving device in Fig. 13, and the XR device 1430 in Fig. 14, etc., in combination. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0599] The point cloud decoder 34000 according to the embodiment includes a geometric information decoding unit 34001 and / or an attribute information decoding unit 34002. According to the embodiment, the point cloud decoder may also be referred to as a PCC decoder, a PCC decoding unit, a point cloud decoder, a point cloud decoding unit, a PCC decoder, etc.

[0600] The geometric information decoding unit 34001 receives the geometric information bitstream 34000a of the point cloud data. The geometric information decoding unit 34001 decodes the geometric information bitstream 34000a of the point cloud data and outputs attribute information of the restored point cloud data 34000c. The geometric information decoding unit 34001 reconstructs the geometric information bitstream with geometry information and outputs restored geometric information 34001. The geometric information bitstream 34000a refers to the geometric information bitstream and geometry bitstream in FIG. 15. The attribute information bitstream 34000b refers to the attribute information bitstream and attribute bitstream in FIG. 15.

[0601] The geometric information decoder 34001 decodes the input geometric information bit stream to restore geometric information. The restored geometric information is input to the attribute information decoder. The attribute information decoder 34002 restores the input attribute information using the input attribute information bit stream and the restored geometric information input from the geometric information decoder. The restored geometric information means the geometry reconstructed by the geometry reconstruction unit 11003 described in FIG. 11. The restored geometric information may also mean the octree occupation code reconstructed by the occupation code based octree reconstruction processor 13003 described in FIG. 13.

[0602] The geometric information decoder 34001 receives a geometric information bit stream received by a receiving device according to the embodiment. The geometric information decoder 34001 can decode the geometric information bit stream.

[0603] The geometric information decoding unit 34001 performs all / part of the operations of the point cloud video decoder in Figure 1, the decoder 20003 in Figure 2, the geometry decoder in Figure 10, and the operations of the arithmetic decoder 11000, octree synthesis unit 11001, surface approximation synthesis unit 11002, geometry reconstruction unit 11003 and / or coordinate system inverse transformation unit 11004 described in Figure 11.

[0604] The attribute information decoding unit 34002 receives the attribute information bit stream 34000b of the point cloud data. The attribute information decoding unit 34002 decodes the attribute information bit stream 34000b of the point cloud data and outputs characteristic information of the restored point cloud data 34000c. The attribute information decoding unit 34002 decodes the attribute information bit stream based on the restored geometric information 34001a generated by the geometric information decoding unit 34001.

[0605] The attribute information decoder 34002 receives an attribute information bitstream received by a receiving device according to an embodiment. The attribute information decoder decodes attribute information of the attribute information bitstream based on the restored geometric information. The geometric information and / or attribute information included in the point cloud data may be decoded and restored PCC data.

[0606] The attribute information decoding unit 34002 performs some or all of the operations of the point cloud video decoder in Figure 1, the decoder 20003 in Figure 2, the operation of the attribute decoder described in Figure 10, the operations of the inverse quantization unit 11006, RAHT 11007, LOD generation unit 11008, inverse lift unit 11009 and / or hue inverse transformation unit 11010 in Figure 11, and the operations of the calculation decoder 13007, inverse quantization processing unit 13008, prediction / lift / RAHT inverse transformation processing unit 13009, hue inverse transformation processing unit 13010 and / or renderer 13011 described in Figure 13.

[0607] FIG. 35 illustrates a point cloud decoder according to an embodiment.

[0608] A part or all of the point cloud data decoder according to the embodiment shown in Fig. 35 is performed by combining the receiving device 10004, the receiving unit 10005, the point cloud video decoder 10006, the decoding 18003, the transmitting unit 18002, the decoders in Fig. 10 and Fig. 11, the receiving device in Fig. 13, the XR device 1430 in Fig. 14, etc. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0609] The point cloud decoder according to the embodiment of Fig. 35 means the point cloud data decoder 34000 according to the embodiment of Fig. 34. The point cloud decoder according to the embodiment includes a receiving unit 35000, a divided data input unit 35001, a first arithmetic decoder 35002, a proprietary code based octree reconstruction processing unit 35003, a surface model processing unit 35004, a geometry reconstruction unit 35005, a coordinate inverse transformation unit 35006, a second arithmetic decoder 35007, an inverse quantization processing unit 35008, a prediction / lift / RAHT inverse transformation processing unit 35009, an attribute reconstruction unit 35010, a hue inverse transformation unit 35011 and / or a point combination unit 35012.

[0610] The components of the point cloud data decoder according to the embodiment shown in FIG. 35 are implemented in units of bricks, and can also be implemented independently in units of bricks.

[0611] The receiver 35000 receives a bit stream including point cloud data according to the embodiment. The receiver 35000 transmits the received bit stream to the divided data input unit 35001. The receiver 35000 performs the operations of the receiver 10007 in FIG. 1, the transmitter 20002 in FIG. 2, and the receiver 13000 in FIG.

[0612] The divided data input unit 35001 receives a bitstream including point cloud data according to the embodiment in units of bricks, and outputs a geometry bitstream and an attribute bitstream of the point cloud data in units of bricks for decoding in units of bricks. The divided data input unit 35001 transmits the brick-based geometry bitstream to the first calculation decoder 35002, and transmits the brick-based attribute bitstream to the second calculation decoder 35002.

[0613] In the case where signaling information (e.g., bistream_brick_decoding_flag) indicating that the bitstream received by the receiving unit 35000 has been coded in units of bricks is present (or when coding is indicated in units of bricks), the divided data input unit 35001 performs N decoding operations on the received bitstream in the point cloud data receiving device according to the embodiment. The divided data input unit 35001 performs the operations of reception 10007 in FIG. 1, transmission 20002 in FIG. 2, and reception processing unit 13001 in FIG. 13.

[0614] The first computation decoder 35002 receives the geometry bitstream on a brick-by-brick basis. The first computation decoder 35002 decodes the geometry bitstream. The first computation decoder 35002 outputs the decoded geometry information.

[0615] The first arithmetic decoder 35002 performs the operations of the point cloud video decoder 10006 in FIG. 1, the decoder 20003 in FIG. 2, the arithmetic decoder 11000 in FIG. 11, and the arithmetic decoder 13002 in FIG.

[0616] The occupancy-based octree reconstruction processor 35003 receives the decoded geometry information. The occupancy-based octree reconstruction processor 35003 reconstructs an occupancy-based octree based on the brick-based geometry bitstream. The occupancy-based octree reconstruction processor 35003 transmits the reconstructed octree to the surface model processor 29004.

[0617] The proprietary code-based octree reconstruction processing unit 35003 performs the operations of the point cloud video decoder 10006 in FIG. 1, the decoder 20003 in FIG. 2, the octree analysis unit 11001 in FIG. 11, and the proprietary code-based octree reconstruction processing unit 13003 in FIG.

[0618] The surface model processor 35004 performs surface model processing of the geometry information based on the geometry information decoded by the first arithmetic decoder 35002 and / or the octree reconstructed by the proprietary code based octree reconstruction processor 35003. The surface model processing of the geometry information includes, for example, a triangle reconstruction operation, an up-sampling operation, and a voxelization operation.

[0619] The surface model processing unit 35004 performs the operations of the point cloud video decoder 10006 in FIG. 1, the decoder 20003 in FIG. 2, the surface approximation analysis unit 11002 in FIG. 11, and the surface model processing unit 13004 in FIG.

[0620] The geometry reconstruction unit 35005 receives geometry information that has been subjected to surface model processing by the surface model processing unit 35004. The geometry reconstruction unit reconstructs the geometry information that has been subjected to surface model processing. The geometry reconstruction unit 35005 transmits the reconstructed geometry information to the prediction / lift / RAHT inverse transformation processing unit 35009 and / or the coordinate inverse transformation unit 35006.

[0621] The geometry reconstruction unit 35005 performs the operations of the point cloud video decoder 10006 in FIG. 1, the decoder 20003 in FIG. 2, and the geometry reconstruction unit 11003 in FIG.

[0622] The coordinate inverse conversion unit 35006 inversely converts the coordinate information of the geometry information generated by the geometry reconstruction unit 35005. The coordinate inverse conversion unit 35006 inversely converts the geometry information and outputs position information of points of the point cloud data. The coordinate inverse conversion unit 35006 transmits the position information of the points to the point connection unit 35012.

[0623] The coordinate inverse transformation unit 35006 transmits to the divided data input unit 35001 information indicating that geometry decoding has been performed for the brick corresponding to the point.

[0624] The coordinate inverse transformation unit 35006 performs the operations of the point cloud video decoder 10006 in FIG. 1, the decoder 20003 in FIG. 2, and the coordinate inverse transformation unit 11004 in FIG.

[0625] The second computation decoder 35007 receives the geometry bitstream on a brick-by-brick basis. The second computation decoder 35007 decodes the attribute bitstream (characteristic bitstream). The second computation decoder 35007 outputs the decoded characteristic information.

[0626] The second arithmetic decoder 35007 performs the operations of the point cloud video decoder 10006 in FIG. 1, the decoder 20003 in FIG. 2, the arithmetic decoder 11005 in FIG. 11, and the arithmetic decoder 13007 in FIG.

[0627] The inverse quantization processing unit 35008 receives the decoded characteristic information generated by the second operation decoder 35007. The inverse quantization processing unit 35008 inverse quantizes the received decoded characteristic information. The inverse quantization processing unit 35008 outputs the inverse quantized characteristic information and transmits the inverse quantized characteristic information to the prediction / lift / RAHT inverse transform processing unit 35009.

[0628] The inverse quantization processing unit 35008 performs the operations of the point cloud video decoder 10006 in FIG. 1, the decoder 20003 in FIG. 2, the inverse quantization unit 11006 in FIG. 11, and the arithmetic decoder 13007 in FIG.

[0629] The prediction / lift / RAHT inverse transform processing unit 35009 receives the geometry information dequantized by the inverse quantization processing unit 35008 and inverse transforms them. The prediction / lift / RAHT inverse transform processing unit 35009 inverse transforms the dequantized feature information based on one of a prediction method, a lifting method, and a RAHT method. The prediction / lift / RAHT inverse transform processing unit 35009 inverse transforms the feature information based on the geometry information reconstructed by the geometry reconstruction unit 35005 (or restored geometry information).

[0630] The prediction / lift / RAHT inverse transform processing unit 35009 performs the operations of the point cloud video decoder 10006 in FIG. 1, the decoder 20003 in FIG. 2, the RAHT 11007, the LOD generation unit 11008, and the inverse lift unit 11009 in FIG.

[0631] The attribute reconstruction unit 35010 reconstructs attribute information based on the characteristic information inversely transformed by the prediction / lift / RAHT inverse transformation processing unit 29009. The attribute reconstruction unit outputs the reconstructed attribute information and transmits it to the hue inverse transformation processing unit 35011.

[0632] The attribute reconstruction unit 35010 performs the operations of the point cloud video decoder 10006 in FIG. 1, the decoder 20003 in FIG. 2, the hue inverse conversion unit 11010 in FIG. 11, and the hue inverse conversion processing unit 13010 in FIG.

[0633] The hue inverse conversion unit 35011 receives the geometry information reconstructed by the attribute reconstruction unit 35010. The hue inverse conversion unit 35011 performs inverse hue conversion on the reconstructed geometry information. The hue inverse conversion unit 35011 outputs attribute information of the points of the point cloud data.

[0634] The hue inverse transform unit 35006 transmits information indicating that feature decoding has been performed on the brick corresponding to the point to the divided data input unit 35001. The attribute reconstruction unit 35010 performs the operations of the point cloud video decoder 10006 in Fig. 1, the decoder 20003 in Fig. 2, the hue inverse transform unit 11010 in Fig. 11, and the hue inverse transform processing unit 13010 in Fig. 13.

[0635] The point combining unit 35012 receives point geometry information (position information) and / or point characteristic information (attribute information) of point cloud data according to an embodiment. The point combining unit 35012 receives point geometry information (position information) and / or point characteristic information (attribute information) of point cloud data in brick units, and combines them in brick units. The point combining unit 35012 outputs point cloud data in brick units including point position information and attribute information.

[0636] FIG. 36 illustrates a point cloud data receiving device according to an embodiment.

[0637] The point cloud data receiving device 36000 according to the embodiment includes a point cloud data decoder 36002 (or decoder) according to the embodiment.

[0638] A part or whole of the point cloud data receiving device according to the embodiment shown in Fig. 35 is performed by combining the receiving device 10004, the receiving unit 10005, the point cloud video decoder 10006, the decoding 18003, the transmitting unit 18002, the decoders in Fig. 10 and Fig. 11, the receiving device in Fig. 13, the XR device 1430 in Fig. 14, etc. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0639] The point cloud data receiving device 36000 receives a bitstream 36001 including point cloud data encoded for each slice. The bitstream 36001 includes geometry information and / or attribute information regarding the points included in each slice. The bitstream 36001 is input from a point cloud data decoder 36002 according to an embodiment.

[0640] According to an embodiment, the bitstream 36001 is data in which point cloud data is divided into one or more tiles and / or slices. A slice according to an embodiment means a divided slice as described in FIG. 21 to FIG. 31.

[0641] The decoder 36002 according to the embodiment decodes the geometry bitstream and the attribute bitstream included in the bitstream 36001. The decoder 36002 according to the embodiment performs decoding based on information indicating a division method according to the embodiment or information indicating a superimposed slice division method signaled from a transmitting device.

[0642] FIG. 37 illustrates the point combining portion of a point cloud decoder according to an embodiment.

[0643] A part or all of the point combining unit 37000 according to the embodiment shown in Fig. 37 is performed by combining the receiving device 10004, the receiving unit 10005, the point cloud video decoder 10006, the decoding 18003, the transmitting unit 18002, the decoders in Fig. 10 and Fig. 11, the receiving device in Fig. 13, the XR device 1430 in Fig. 14, etc. Also, the bricks described in this figure correspond to slices, and vice versa. In this specification, bricks are also referred to as slices.

[0644] The point combination unit 37000 is shown. The point combination unit 37000 receives point geometry information 37000a and / or point characteristic information 37000b. The point combination unit 37000 combines the received point geometry information 37000a and point characteristic information 37000b to output point cloud data 37000c. The point combination unit 37000 includes a combination execution unit 37001.

[0645] The join execution unit 37001 receives point geometry information and point attribute information for each brick or tile. The point join unit 37000 receives point geometry information and point attribute information for each brick or tile and joins them based on a join method.

[0646] The combination method according to the embodiment means an operation based on the inverse process of any of the methods of multiplexing, MVC bitstream combination, sequential combination, importance-based combination, etc. When the point cloud decoder according to the embodiment decodes the position values ​​(geometry information) of the points and the attribute values ​​(characteristic information) of the points, it determines which combination method to use to combine the position values ​​and the attribute values.

[0647] The merger executor 37001 according to the embodiment can map one attribute value (characteristic information) to one position value (geometry information) in a 1:1 manner. The merger executor 37001 according to the embodiment uses a lossy merge method when it is not possible to map one attribute value (characteristic information) to one position value (geometry information) in a 1:1 manner. For example, an attribute value (characteristic information) that exists in a specific position value (geometry information) may not exist, and multiple attribute values ​​may exist in one position value. The lossy merger method according to the embodiment can configure point information using a duplicated attribute value indication (duplicated_attribute_index) when multiple characteristic information is mapped.

[0648] A point cloud decoder according to an embodiment (e.g., a point combining unit 37000, a combining execution unit 37001) can combine point geometry information and point attribute information based on information indicating a combining method (e.g., combining_filter_index) included in a bitstream received by a point cloud data receiving device.

[0649] The point geometry information 37000a refers to a geometry bitstream decoded by a point cloud decoder according to an embodiment. The point geometry information 37000a refers to point geometry information output from the coordinate inverse conversion unit 29006 in Fig. 29, or decoded geometry information output from the geometric information decoding unit 28001 in Fig. 28.

[0650] The point attribute information 37000b refers to a feature bit stream decoded by the point cloud decoder according to the embodiment. The point attribute information 37000b refers to the point attribute information output from the hue inverse conversion processing unit 35011 in Fig. 35, or the decoded attribute information output from the attribute information decoding unit 34002 in Fig. 34.

[0651] FIG. 38 illustrates an example of a bitstream structure of point cloud data according to an embodiment.

[0652] Fig. 38 is an example of a bit stream structure 38000 of point cloud data according to an embodiment. The bit stream structure 38000 of point cloud data shown in Fig. 38 is generated by the transmitter 10003 of Fig. 1, the transmission unit 18002 of Fig. 2, the transmission processing unit 12012 of Fig. 12, the bit stream combining unit 16014 of Fig. 16, and the bit stream combining unit 17000 of Fig. 17 of the transmitting device according to the embodiment.

[0653] The bit stream structure 38000 of point cloud data shown in Fig. 38 is a bit stream received by the receiving unit 10005 in Fig. 1, the decoding unit 18003 in Fig. 2, the receiving unit 13000 in Fig. 13, and the receiving unit 35000 in Fig. 35 of the receiving device according to the embodiment. The bit stream structure 38000 of point cloud data shown in Fig. 38 means the geometry and attribute bit stream 36001 for each slice in Fig. 36.

[0654] Some or all of the embodiment of bitstream structure 38000 shown in Figure 38 may be generated by XR device 1430 of Figure 14. Also, the bricks described in this figure correspond to slices and vice versa. In this specification, bricks are also referred to as slices.

[0655] The point cloud data transmitting apparatus according to the embodiment transmits a bitstream 38000 having a bitstream structure as shown in FIG. 31. The point cloud data bitstream 38000 includes a sequential parameter set (SPS) 38001, a geometry parameter set (GPS) 38002, an attribute parameter set (APS) 38003, a tile parameter set (TPS) 38004, and one or more slices (slice 0, slice 1 ... slice n) 38004. The point cloud data bitstream 38000 includes one or more tiles. A tile according to the embodiment is a group of slices including one or more slices. The bitstream 38000 according to the embodiment provides tiles or slices so that the point cloud data can be divided and processed by region. Each region of the bitstream 38000 according to the embodiment has a different importance. Therefore, when the point cloud data is divided into tiles, a different filter (encoding method) and a different filter unit can be applied to each tile. If the point cloud is divided into slices, a different filter, with a different filter unit, can be applied to each slice.

[0656] The point cloud data transmitting device according to the embodiment can provide a method for applying different encoding operations depending on importance and using high-quality encoding methods for important areas by transmitting point cloud data according to the structure of bitstream 38000 as shown in FIG.

[0657] The point cloud data receiving device according to the embodiment applies different filtering (decoding methods) to each region (regions divided into tiles or slices) instead of using complex decoding (filtering) methods on the entire point cloud data depending on the processing capacity of the receiving device, thereby ensuring better image quality and appropriate latency in the system in areas that are important to the user.

[0658] The SPS (Sequence Parameter Set) 38001 is a syntax structure containing syntax elements that apply to zero or more entire CVSs as determined by the content of a syntax element found in the PPS referred to by a syntax element found in each slice segment header. The SPS contains sequence information of the point cloud data bitstream according to the embodiment.

[0659] GPS (Geometry Parameter Set) 38002 refers to a syntax structure including syntax elements to which zero or more overall geometries (or encoded geometries) are applied. GPS 38002 according to an embodiment includes information on how to encode attribute information of point cloud data included in one or more slices 38004. GPS 38002 includes SPS identifier information indicating which SPS 38001 according to an embodiment includes geometry parameters related to the SPS 38001, and GPS identifier information identifying the corresponding GPS.

[0660] APS (Attribute Parameter Set) 38003 refers to a syntax structure including syntax elements to which zero or more overall attributes (or coded attributes) are applied. APS 38003 according to an embodiment includes information on how to code attribute information of point cloud data included in one or more slices 38004. APS 38003 includes SPS identifier information indicating which SPS 38001 according to an embodiment includes geometry parameters related to the SPS 38001, and GPS identifier information to identify the corresponding APS.

[0661] TPS (Tile Parameter Set) 38004 refers to a syntax structure including syntax elements to which zero or more whole tiles (or coded tiles) are applied. The tile inventory includes information about zero or more tiles included in a point cloud data bitstream according to an embodiment. The tile inventory is also called TPS (Tile Parameter Set) according to an embodiment.

[0662] TPS (Tile Parameter Set) 38004 includes identifier information for identifying one or more tiles and information indicating the range of one or more tiles (i.e., the bounding box of the tile). The information indicating the range of one or more tiles (i.e., the bounding box of the tile) includes coordinate information of the reference point of the bounding box indicated by the corresponding tile (e.g., Tile(n).tile_bounding_box_xyz0) and information regarding the width, height, and depth of the corresponding bounding box (e.g., Tile(n).tile_bounding_box_whd). When multiple tiles exist, Tile Inventory 33004 includes information indicating the bounding box for each of the tiles. For example, if each tile is represented by 0 to n based on tile identifier information, information indicating the bounding box of each tile is expressed as Tile(0).tile_bounding_box_xyz0, Tile(0).tile_bounding_box_whd, Tile(1).tile_bounding_box_xyz0, Tile(1).tile_bounding_box_whd..., etc.

[0663] The slice 38004 means a unit for encoding point cloud data by a point cloud data transmitting device according to an embodiment. The slice 38004 according to an embodiment means a unit including one geometry bit stream (Geom00) 38004a and one or more attribute bit streams (Attr00, Attr10) 38004b, 38004b.

[0664] The slice 38004 includes a geometry slice (Geom) 38004a indicating geometry information of the point cloud data contained in the corresponding slice, and one or more attribute slices (Attr) 38004b, 38004c indicating attribute information of the point cloud data contained in the corresponding slice.

[0665] The geometry slice (Geometry slice, Geom) 38004a includes geometry slice data (Geom_slice_data) 38004a-2 including geometry information of the point cloud data, and a geometry slice header (Geom_slice_header, GSH) 38004a-1 including information about the geometry slice data.

[0666] The geometry slice header 38004a-1 includes information about the geometry slice data 38004a-2 in the corresponding slice. For example, the geometry slice header 38004a-1 includes a geometry parameter set identifier (geom_geom_parameter_set_id) for identifying which GPS 38002 indicates the geometry information of the corresponding slice, a geometry slice identifier (geom_slice_id) for identifying the corresponding geometry slice, geometry box origin information (geomBoxOrigin) indicating the box origin of the corresponding geometry slice data, information indicating the log scale of the geometry slice (geom_box_log2_scale), and information related to the number of points of the corresponding geometry slice (geom_num_points).

[0667] When a point cloud data bitstream according to an embodiment includes one or more tiles, a header of a geometry bitstream according to an embodiment further includes information (geom_tile_id) for identifying a tile including the corresponding geometry bitstream.

[0668] The attribute slices (Attr) 38004b, 38004c include attribute slice data (Attr_slice_data) containing attribute information about the point cloud data and an attribute slice header (Attr_slice_header, ASH) 33705c containing information about the attribute slice data.

[0669] According to the embodiment, the parameters required for point cloud encoding are newly defined as a parameter set and header information of the point cloud. For example, when performing attribute information encoding, they can be added to the attribute parameter set RBSP syntax, and when performing tile-based encoding, they can be added to the tile_header syntax.

[0670] According to an embodiment, the transmitting device may generate a different slice configuration header unit per tile or per slice to signal the configuration of the other overlapping slices to the embodiment.

[0671] In the point cloud data transmission and reception method according to the embodiment, by providing such a bit stream structure, the receiver can improve the decoding performance of the attribute information of the point cloud data.

[0672] FIG. 39 shows a sequence parameter set (SPS) of a bitstream according to an embodiment.

[0673] The parameters shown in Fig. 39 are included in the SPS (Sequence Parameter Set) described in Fig. 38. The SPS according to the embodiment includes slice division (slice distribution) according to the embodiment, signaling information related to a slice tiling method, and / or related signaling information.

[0674] profile_idc means information indicating a profile of a bitstream that conforms to Annex A of the H.264 standard document. Other values ​​of profile_idc may be used in the future by ISO / IEC (indicates a profile to which the bitstream conforms as specified in Annex A. Bitstreams shall not contain values ​​of profile_idc other than those specified in Annex A. Other values ​​of profile_idc are reserved for future use by ISO / IEC.).

[0675] profile_compatibility_flags equal to 1 indicates that the bitstream conforms to the profile indicated by profile_idc equal to j as specified in Annex A. The value of profile_compatibility_flag[j] shall be equal to 0 for any value of j that is not specified as an allowed value of profile_idc in Annex A.

[0676] level_idc indicates a level to which the bitstream conforms as specified in Annex A. Bitstreams may not contain values ​​of level_idc other than those specified in Annex A. Other values ​​of level_idc are reserved for future use by ISO / IEC.

[0677] sps_bounding_box_present_flag is 1 if bounding box offset and size information is signalled (equal to 1 specifies the bounding box offset and size information is signalled. sps_bounding_box_present_flag equal to 0 specifies).

[0678] When sps_bounding_box_present_flag has a true value, the SPS according to the embodiment further includes sps_bounding_box_offset_x, sps_bounding_box_offset_y, sps_bounding_box_offset_z, sps_bounding_box_scale_factor, sps_bounding_box_size_width, sps_bounding_box_size_height and sps_bounding_box_size_depth.

[0679] sps_bounding_box_offset_x indicates the x offset of the source bounding box in the cartesian coordinates. When not present, the value of sps_bounding_box_offset_x is inferred to be 0.

[0680] sps_bounding_box_offset_y indicates the y offset of the source bounding box in the cartesian coordinates. When not present, the value of sps_bounding_box_offset_y is inferred to be 0.

[0681] sps_bounding_box_offset_z indicates the z offset of the source bounding box in the Cartesian coordinates. When not present, the value of sps_bounding_box_offset_x is inferred to be 0.

[0682] sps_bounding_box_scale_factor indicates the scale factor the source bounding box in the Cartesian coordinates. When not present, the value of sps_bounding_box_scale_factor is inferred to be 1. Indicates. When not present, the value of sps_bounding_box_scale_factor is inferred to be 0.

[0683] sps_bounding_box_size_width indicates the width of the source bounding box in the Cartesian coordinates. … When not present, the value of sps_bounding_box_size_width is inferred to be a specific value (such as 10).

[0684] sps_bounding_box_size_height indicates the height of the source bounding box in the Cartesian coordinates. When not present, the value of sps_bounding_box_size_height is inferred to be 1. When not present, the value of sps_bounding_box_size_height is inferred to be 0.

[0685] sps_bounding_box_size_depth indicates the depth of the source bounding box in the Cartesian coordinates. When not present, the value of sps_bounding_box_size_depth is inferred to be 1. When not present, the value of sps_bounding_box_size_depth is inferred to be 0.

[0686] sps_source_scale_factor indicates the scale factor of the source point cloud.

[0687] sps_seq_parameter_set_id indicates ID information for the SPS referenced by other syntax elements. sps_seq_parameter_set_id is set to a value from 0 to 15 within the range that satisfies the conditions in the specification of the relevant version. Other information other than 0 for sps_seq_parameter_set_id may be used later by ISO / IEC. (Provides an identifier for the SPS for reference by other syntax elements. In The value of SPS_seq_parameter_set_id may be in the range of 0 to 15, inclusive in bitstreams conforming to this version of this Specification. The value other than 0 for sps_seq_parameter_set_id is reserved for future use by ISO / IEC.)

[0688] sps_num_attribute_sets indicates the number of coded attributes in the bitstream. The value of sps_num_attribute_sets may be in the range of 0 to 64.

[0689] An SPS according to an embodiment includes sps_num_attribute_sets only of attribute_dimension[i], attribute_instance_id[i], attribute_bitdepth[i], attribute_cicp_colour_primaries[i], attribute_cicp_transfer_characteristics[i], attribute_cicp_matrix_coeffs[i], attribute_cicp_video_full_range_flag[i] and / or known_attribute_label_flag[i].

[0690] attribute_dimension[i] specifies the number of components of the i-th attribute.

[0691] attribute_instance_id[i] specifies the attribute instance id.

[0692] attribute_bitdepth[i] specifies the bitdepth of the i-th attribute signal(s).

[0693] attribute_cicp_colour_primaries[i] indicates the chromaticity coordinates of the colour attribute source primaries.

[0694] attribute_cicp_transfer_characteristics[i] indicates the reference opto-electronic transfer characteristic function of the colour attribute as a function of a source input linear optical intensity Lc with a nominal real-valued range of 0 to 1 or indicates the inverse of the reference electro-optical transfer characteristic function as a function of an output linear optical intensity Lo with a nominal real-valued range of 0 to 1.

[0695] attribute_cicp_matrix_coeffs[i] describes the matrix coefficients used in deriving luma and chroma signals from the green, blue, and red, or Y, Z, and X primaries.

[0696] attribute_cicp_video_full_range_flag[i] specifies indicates the black level and range of the luma and chroma signals as derived from E'Y, E'PB, and E'PR or E'R, E'G, and E'B real-valued component signals.

[0697] known_attribute_label_flag[i] equal to 1 specifies know_attribute_label is signalled for the i-th attribute. known_attribute_label_flag[i] equal to 0 specifies attribute_label_four_bytes is signalled for the i-th attribute.

[0698] When known_attribute_label_flag[i] is 1 (true), the SPS according to the embodiment further includes known_attribute_label[i].

[0699] If known_attribute_label[i] is 0 it specifies the attribute is colour. If it is 1 it specifies the attribute is reflectance. If it is 2 it specifies the attribute is a frame index. (equal to 0 specifies the attribute is colour. known_attribute_label[i] equal to 1 specifies the attribute is reflectance. known_attribute_label[i] equal to 2 specifies the attribute is firme index.)

[0700] The SPS according to the embodiment further includes a partitioning_enabling_flag.

[0701] The partitioning_enabling_flag indicates information (or a flag) indicating whether the point cloud transmitting device according to the embodiment divides the point cloud data into slices or bricks.

[0702] A point cloud data receiving device according to an embodiment (for example, the receiving unit 35000 and the divided data input unit 35001 in Figure 35) determines whether the point cloud data in the received bit stream is divided into slice units based on partitioning_enabling_flag, and receives the point cloud data in the bit stream in slice units based on partitioning_enabling_flag.

[0703] split_recursive_enabling_flag indicates information (or a flag) indicating whether the point cloud transmitting device according to the embodiment has further divided a divided slice or brick.

[0704] According to an embodiment, when partitioning_enabling_flag is 1 (true), the SPS further includes split_recursive_enabling_flag, multiple_brick_partitioning_flag, partitioning_unit_size, and / or partitioning_type_index.

[0705] A point cloud data receiving device according to an embodiment (e.g., the receiving unit 35000 and the divided data input unit 35001 in FIG. 35) determines whether the slices of point cloud data in the received bitstream are divided into fine division units based on split_recursive_enabling_flag.

[0706] The partitioning_unit_size is information indicating a division unit of a slice or a brick divided by the point cloud transmitting device according to the embodiment. For example, the partitioning_unit_size indicates a unit size of a slice or a brick according to the embodiment.

[0707] A point cloud data receiving device according to an embodiment (for example, the receiving unit 35000 and the partitioned data input unit 35001 in FIG. 35) can determine the partitioning unit of the slices of the point cloud data of the received bit stream based on the partitioning_unit_size, and decode only the bit stream corresponding to the partitioning unit.

[0708] The multiple_brick_partitioning_flag is information (or a flag) indicating whether or not to divide the point cloud data according to the embodiment based on multiple slice division methods. In other words, this parameter is information that notifies whether or not the partitioning type used for slice division uses various division methods.

[0709] A point cloud data receiving device according to an embodiment (e.g., the receiving unit 35000 and the divided data input unit 35001 in Figure 35) determines whether the point cloud data has been divided based on one or more slice division methods based on multiple_brick_partitioning_flag.

[0710] partitioning_type_index indicates a method of dividing point cloud data into slice units according to an embodiment. The method of dividing into slice units according to an embodiment includes a point density-based slice division method according to an embodiment, a circular sector slice division method according to an embodiment, a layer slice division method according to an embodiment, a spherical / elliptical region slice division method according to an embodiment, an attribute-based slice division method according to an embodiment, and an adaptive axis length-based slice division method according to an embodiment. For example, partitioning_type_index is determined as follows.

[0711] partitioning_type_index=1 indicates that point density-based slice distribution (brick tiling) is performed. Point density-based slice distribution (brick tiling) means dividing slices according to the method shown in FIG.

[0712] partitioning_type_index=2 indicates that circular sector slice partitioning (brick tiling) is performed. Circular sector slice partitioning (brick tiling) means the division of bricks by the method shown in FIG.

[0713] partitioning_type_index=3 indicates that layer slice distribution (brick tiling) has been performed. Layer slice distribution (brick tiling) means the division of slices by the method shown in FIG.

[0714] partitioning_type_index=4 indicates that spherical / elliptical slice partitioning (brick tiling) was performed. Spherical / elliptical region tiling means the division of bricks (slices) according to the method shown in FIG. 24.

[0715] partitioning_type_index=5 indicates that attribute-based slice distribution (brick tiling) is performed. Attribute-based slice distribution (brick tiling) means dividing the bricks (slices) according to the method shown in FIG.

[0716] partitioning_type_index=6 indicates that adaptive axis length-based slice distribution (brick tiling) is performed. Adaptive axis length-based slice distribution (brick tiling) means dividing the bricks (slices) according to the method shown in FIG. 26.

[0717] partitioning_type_index=7 indicates that a user-defined slice distribution (brick tiling) method other than the above-mentioned methods is performed. According to the embodiment, the long axis is divided based on the short axis of the bounding box coordinate axis, and the divided slices can be divided again. In this case, a process of re-dividing the middle axis based on the short axis can be performed.

[0718] A point cloud data receiving device according to an embodiment (for example, the receiving unit 35000 and the partitioned data input unit 35001 in FIG. 35) can decode point cloud data in slice units based on the partitioning_type_index.

[0719] The SPS according to the embodiment further includes partition_current_unit when split_recursive_enabling_flag is 1 (true).

[0720] When the point cloud transmitting device according to the embodiment re-divides a slice or a brick, partition_current_unit is information indicating a division unit of the re-divided slice or brick. For example, partition_current_unit indicates a unit size of the re-divided slice or brick according to the embodiment.

[0721] When a point cloud data receiving device according to an embodiment (e.g., the receiving unit 35000 and the divided data input unit 35001 of Figure 35) confirms that a slice of point cloud data of a bitstream received in response to split_recursive_enabling_flag is divided into fine division units, it can determine the division unit of the re-divided slice based on partition_current_unit.

[0722] The combining_filter_index indicates information about which combining method is used by the encoder to transmit the combined geometry bitstream and attribute bitstream. According to an embodiment, the combining method is signaled according to the transmission environment and characteristics of the bitstream, so that the compression rate can be improved.

[0723] If combining_filter_index=0 it indicates that a combining method based on a multiplexing method was used. According to an embodiment, the multiplexing method can sequence the individual bitstreams into groups of bits or bytes.

[0724] If combining_filter_index=1, it indicates that a combining method based on the sequential combining method is used. According to an embodiment, the sequential combining method combines the geometry bitstream and the attribute bitstream in sequence.

[0725] If combining_filter_index=2, it indicates that a combining method based on importance-based combining is used. According to an embodiment, importance-based combining can prioritize combining for bitstreams with high importance.

[0726] The point cloud data receiving apparatus according to the embodiment (e.g., the point combining unit 35012 of FIG. 35 and the point combining unit 37000 of FIG. 37) may extract a geometry bitstream and an attribute bitstream from a received bitstream based on combining_filter_index. For example, when combining_filter_index is 0, the point cloud data receiving apparatus according to the embodiment extracts a geometry bitstream and an attribute bitstream from a received bitstream based on an inverse process of the multiplexing method. For example, when combining_filter_index is 1, the geometry bitstream and the attribute bitstream from a received bitstream are extracted based on an inverse process of the sequential combining method. For example, when combining_filter_index is 2, the geometry bitstream and the attribute bitstream from a received bitstream are extracted based on an inverse process of the importance-based combining.

[0727] The duplicated_attribute_index indicates that the geometry contains duplicated attribute information when multiple feature information is mapped to one geometry information. The attribute information of the duplicated points can apply duplicated attributes.

[0728] If duplicated_attribute_index=0, it means that the hue is duplicated. When multiple hue values ​​are mapped to one geometry information, the duplicated attribute can be used.

[0729] If duplicated_attribute_index=1, it means that the reflectance is duplicated. When multiple reflectance values ​​are mapped to one geometry information, the duplicated attribute can be used.

[0730] The point cloud data receiving device according to the embodiment can determine attribute information whose geometry is duplicated based on the duplicated_attribute_index.

[0731] When sps_extension_present_flag is 1, it indicates that sps_extension_data is present in the SPS RBSP syntax structure. When this parameter is 0, it indicates that this syntax structure is not present. When not present, the value of sps_extension_present_flag is inferred to be equal to 0.

[0732] sps_extension_data_flag may have any value. Its presence and value do not affect decoder conformance to profiles specified in Annex A Decoders conforming to a profile specified in Annex A.

[0733] The SPS according to the embodiment further includes sequential_input_slice_configuration_in_tile_flag, overlapping_slice_compose_flag, and overlapping_slice_compose_method[i] information.

[0734] The sequential_input_slice_configuration_in_tile_flag indicates information indicating whether the transmitting device according to the embodiment performs sequential input slice division for each tile without a wait state for the slice division operation for each tile.

[0735] The overlapping_slice_compose_flag indicates information indicating whether the transmitting device according to the embodiment uses an overlapping slice division method.

[0736] When the overlapping_slice_compose_flag information is 1 (true), the SPS according to the embodiment further includes overlapping_slice_compose_method[i]. The SPS includes overlapping_slice_compose_method[i] information as many times as the number of slices transmitted by the transmitting device according to the embodiment.

[0737] overlapping_slice_compose_method[i] means information indicating the slice division method of the i-th slice. The slice division method of the i-th slice and the corresponding overlapping_slice_compose_method[i] value are as follows:

[0738] 0 = multiple_point_sampling_based_overlapping_slice_parity_method: The slice construction method is used as the overlapping slice construction method based on multiple point sampling according to the embodiment.

[0739] 1 = value_based_overlapping_slice_partition_method: The slice construction method is used as the overlapping slice construction method based on importance according to the embodiment.

[0740] 2 = attribute_range_based_slice_partition_method: The slice construction method is used as the attribute range-based overlap slice construction method according to the embodiment.

[0741] 3 = morton_order_based_slice_partition_method: the slice construction method is used as the order-based overlap slice construction method of the Morton code according to the embodiment.

[0742] 4~=other_slice_partition_method: Use other partitioning methods

[0743] FIG. 40 shows a Tile Parameter Set (TPS) of a bitstream according to an embodiment.

[0744] The parameters shown in Fig. 40 are included in the Tile Parameter Set (TPS) described in Fig. 38. The TPS according to the embodiment includes signaling information related to the division of slices (slice distribution) according to the embodiment, the slice tiling method, and / or related signaling information.

[0745] num_tiles represents the number of tiles signalled for the bitstream. If no tiles are present in the bitstream, num_tiles is inferred to be 0.

[0746] According to the embodiment, the TPS includes signaling information up to the value indicated in the number of tiles (num_tiles).

[0747] tile_bounding_box_offset_x[i] indicates the x offset of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_offset_x[0] is inferred to be sps_bounding_box_offset_x.

[0748] tile_bounding_box_offset_y[i] indicates the y offset of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_offset_y[0] is inferred to be sps_bounding_box_offset_y.

[0749] tile_bounding_box_offset_z[i] indicates the z offset of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_offset_z[0] is inferred to be sps_bounding_box_offset_z.

[0750] tile_bounding_box_scale_factor[i] indicates the scale factor the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_scale_factor[0] is inferred to be sps_bounding_box_scale_factor.

[0751] tile_bounding_box_size_width[i] indicates the width of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_size_width[0] is inferred to be sps_bounding_box_size_width.

[0752] tile_bounding_box_size_height[i] indicates the height of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_size_height[0] is inferred to be sps_bounding_box_size_height.

[0753] tile_bounding_box_size_depth[i] indicates the depth of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_size_depth[0] is inferred to be sps_bounding_box_size_depth.

[0754] The TPS according to the embodiment includes a partitioning_enable_flag.

[0755] The partitioning_enabling_flag indicates information (or a flag) indicating whether the point cloud transmitting device according to the embodiment divides the point cloud data into slices or bricks. The partitioning_enable_flag indicates the same information as the partitioning_enabling_flag described in FIG.

[0756] A point cloud data receiving device according to an embodiment (for example, the receiving unit 35000 and the divided data input unit 35001 in Figure 35) determines whether the point cloud data in the received bit stream is divided into slice units based on partitioning_enabling_flag, and receives the point cloud data in the bit stream in slice units based on partitioning_enabling_flag.

[0757] According to an embodiment, the TPS further includes multiple_brick_partitioning_flag, partitioning_unit_size, and / or partitioning_type_index when partitioning_enable_flag is 1 (true).

[0758] The multiple_brick_partitioning_flag means the multiple_brick_partitioning_flag included in the SPS described in FIG 39. The partitioning_unit_size means the partitioning_unit_size included in the SPS described in FIG 39. The partitioning_type_index means the partitioning_type_index included in the SPS described in FIG 39.

[0759] The TPS according to the embodiment further includes brick_tiling_type information.

[0760] brick_tiling_type indicates how a transmitting device in accordance with an embodiment divides point cloud data into one or more bricks (also called slices).

[0761] The brick_tiling_type indicates the same information as the partitioning_type_index described in Fig. 39. That is, the brick_tiling_type is also called the partitioning_type_index.

[0762] If the brick_tiling_type information is 1 (indicating point density-based brick tiling (slice distribution)), the TPS according to the embodiment further includes brick_unit_information and brick_density_quantiza...

Claims

1. A method for processing point cloud data by a transmission system, comprising: selectively dividing the point cloud data into one or more slices, where if a number of points in the point cloud data is greater than a maximum number information representing a maximum number of points in a slice, dividing the point cloud data into one or more slices is performed based on an octree of the point cloud data, and if a number of points in the point cloud data is less than the maximum number information, dividing the point cloud data into one or more slices is not performed; encoding the point cloud data as to whether the segmentation is performed; transmitting a bitstream including the encoded point cloud data and signaling information for the point cloud data.

2. The encoding step includes: encoding geometry data of points of the point cloud data; and encoding attribute data of the points of the point cloud data.

3. The signaling information is The method of claim 1 , including information indicating whether the partitioning has been performed and, if so, how the partitioning has been performed.

4. The method of claim 1, wherein the one or more slices include a first slice and a second slice, the first slice and the second slice comprising overlapping regions of three-dimensional space containing the point cloud data.

5. A transmission system for processing point cloud data, comprising: a division unit configured to selectively divide the point cloud data into one or more slices, where if a number of points in the point cloud data is greater than maximum number information representing a maximum number of points in a slice, dividing the point cloud data into one or more slices is performed based on an octree of the point cloud data, and if a number of points in the point cloud data is less than the maximum number information, dividing the point cloud data into one or more slices is not performed; an encoder for encoding the point cloud data according to whether the segmentation is performed or not; a transmitter for transmitting a bitstream including the encoded point cloud data and signaling information for the point cloud data.

6. The encoder comprises: The transmission system of claim 5 , further comprising: encoding geometry data of the points of the point cloud data; and encoding attribute data of the points of the point cloud data.

7. The transmission system of claim 5 , wherein the signaling information includes information indicating whether the division is performed and, if so, information indicating the manner of the division.

8. The transmission system of claim 5, wherein the one or more slices include a first slice and a second slice, the first slice and the second slice comprising overlapping regions of three-dimensional space including the point cloud data.

9. A receiving system for receiving point cloud data, comprising: a receiver for receiving a bitstream including point cloud data and signaling information for the point cloud data; a decoder for decoding the point cloud data, the point cloud data being selectively divided into one or more slices by a transmission system, and if a number of points in the point cloud data is greater than maximum number information for representing a maximum number of points in a slice, the point cloud data being divided into one or more slices based on an octree of the point cloud data, and if a number of points in the point cloud data is less than the maximum number information, the point cloud data is not divided into one or more slices; A point cloud data receiving system including a renderer for rendering the decoded point cloud data.

10. The decoder comprises: a geometry decoder for decoding geometry data of the point cloud data; and a feature decoder for decoding feature data of the point cloud data.

11. The receiving system of claim 9 , wherein the signaling information includes information indicating whether the point cloud data is divided into one or more slices and information indicating a division method for the one or more slices.

12. The receiving system of claim 9, wherein the one or more slices include a first slice and a second slice, the first slice and the second slice comprising overlapping regions of three-dimensional space containing the point cloud data.

13. A method for processing point cloud data by a receiving system, comprising: receiving a bitstream including point cloud data and signaling information for the point cloud data; Decoding the point cloud data, the point cloud data is selectively divided into one or more slices by a transmission system, and if a number of points in the point cloud data is greater than maximum number information for representing a maximum number of points in a slice, the point cloud data is divided into one or more slices based on an octree of the point cloud data, and if a number of points in the point cloud data is less than the maximum number information, the point cloud data is not divided into one or more slices; and rendering the decoded point cloud data.

14. The step of decoding comprises: decoding geometry data of the point cloud data; and decoding attribute data of the point cloud data.

15. The method for processing point cloud data described in claim 13, wherein the signaling information includes information indicating whether the point cloud data is divided into one or more slices and information indicating a division method for the one or more slices.

16. The method of processing point cloud data as described in claim 13, wherein the one or more slices include a first slice and a second slice, the first slice and the second slice comprising overlapping regions of three-dimensional space containing the point cloud data.

Images