Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Abstract

A method, according to embodiments, may comprise the steps of: decoding geometry data of point cloud data in a bitstream; and decoding attribute data of the point cloud data. A method, according to embodiments, may comprise the steps of: encoding geometry data of point cloud data; and encoding attribute data of the point cloud data.

Description

Point cloud data transmission device, point cloud data transmission method, point cloud data reception device and point cloud data reception method

[0001] The embodiments relate to a method and apparatus for processing point cloud content.

[0002] Point cloud content is content represented as a point cloud, which is a set of points belonging to a coordinate system that represents three-dimensional space. Point cloud content can represent three-dimensional media and is used to provide various services such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and autonomous driving services. However, representing point cloud content requires tens of thousands to hundreds of thousands of point data points. Therefore, a method is required to efficiently process a vast amount of point data.

[0003] The embodiments provide an apparatus and a method for efficiently processing point cloud data. The embodiments provide a method and apparatus for processing point cloud data to address latency and encoding / decoding complexity.

[0004] However, the scope of rights of the embodiments is not limited to the technical problems described above, and may be extended to other technical problems that a person skilled in the art can infer based on the entire content described.

[0005] A method for transmitting point cloud data according to embodiments may include the step of encoding point cloud data and the step of transmitting a bitstream containing point cloud data. A method for receiving point cloud data according to embodiments may include the step of receiving a bitstream containing point cloud data and the step of decoding point cloud data.

[0006] The device and method according to the embodiments can process point cloud data with high efficiency.

[0007] The device and method according to the embodiments can provide a high-quality point cloud service.

[0008] The device and method according to the embodiments can provide point cloud content for providing general-purpose services such as VR services and autonomous driving services.

[0009] Drawings are included to further understand the embodiments, and the drawings illustrate the embodiments along with descriptions related to the embodiments. For a better understanding of the various embodiments described below, one must refer to the description of the embodiments below in relation to the following drawings, which include parts corresponding to similar reference numerals throughout the drawings.

[0010] FIG. 1 shows an example of a point cloud content provision system according to embodiments.

[0011] FIG. 2 is a block diagram illustrating a point cloud content provision operation according to embodiments.

[0012] FIG. 3 shows an example of a point cloud encoder according to embodiments.

[0013] FIG. 4 shows examples of octree and occupancy codes according to embodiments.

[0014] Figure 5 shows an example of a point configuration by LOD according to embodiments.

[0015] Figure 6 shows an example of a point configuration by LOD according to embodiments.

[0016] FIG. 7 shows an example of a point cloud decoder according to embodiments.

[0017] FIG. 8 is an example of a transmission device according to embodiments.

[0018] FIG. 9 is an example of a receiving device according to embodiments.

[0019] FIG. 10 shows an example of a structure that can be linked with a point cloud data transmission / reception method / device according to embodiments.

[0020] FIG. 11 shows an example of an atlas search according to embodiments.

[0021] FIG. 12 shows Morton codes in a 2D plane according to embodiments.

[0022] FIG. 13 shows an example of inter-atlas search according to embodiments.

[0023] FIG. 14 shows an example of choosing same-index reference point for atlas search in 2D plane according to embodiments.

[0024] FIG. 15 shows an example of choosing the closest reference point for atlas search in a 2D plane according to embodiments.

[0025] FIG. 16 illustrates a center-based atlas search range management method according to embodiments.

[0026] FIG. 17 illustrates a planar atlas search range management method according to embodiments.

[0027] FIG. 18 shows a 2D example of index-based atlas search range management according to embodiments.

[0028] FIG. 19 shows a 2D example of enlarged atlas search range management according to embodiments.

[0029] FIG. 20 shows an encoder according to embodiments.

[0030] FIG. 21 shows the encoding of attribute data according to embodiments.

[0031] FIG. 22 shows a decoder according to embodiments.

[0032] FIG. 23 illustrates the decoding of attribute data according to embodiments.

[0033] FIG. 24 shows the generation of LoD (Level of Detail) according to the embodiments.

[0034] FIG. 25 illustrates the nearest neighbor point search according to the embodiments.

[0035] FIGS. 26a and FIGS. 26b illustrate the generation of a predictor based on search area-related information according to embodiments.

[0036] FIGS. 27a and FIGS. 27b illustrate the generation of a predictor based on search area-related information according to embodiments.

[0037] FIG. 28 shows a bitstream including geometry data, attribute data, and parameter information according to embodiments.

[0038] FIG. 29 shows an attribute parameter set (APS) within a bitstream according to embodiments.

[0039] FIG. 30 shows an attribute data unit header within a bitstream according to embodiments.

[0040] FIG. 31(a) shows an axis-based search according to embodiments, and FIG. 31(b) shows a multi-axis-based search according to embodiments.

[0041] FIG. 32 shows an example of setting a search area when searching the z-axis according to embodiments.

[0042] FIG. 33 illustrates an inter-atlas search according to embodiments.

[0043] FIG. 34 shows an attribute parameter set (APS) within a bitstream according to embodiments.

[0044] FIG. 35 illustrates a encoding method according to embodiments.

[0045] FIG. 36 illustrates a decoding method according to embodiments.

[0046] Preferred embodiments of the embodiments are described in detail, and examples thereof are shown in the accompanying drawings. The detailed description below, with reference to the accompanying drawings, is intended to describe preferred embodiments of the embodiments rather than merely embodiments that may be implemented according to the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it is obvious to those skilled in the art that the embodiments can be practiced without these details.

[0047] Most terms used in the embodiments are selected from those commonly used in the field, but some terms are chosen at the applicant's discretion, and their meanings are described in detail in the following description as necessary. Accordingly, the embodiments should be understood based on the intended meaning of the terms, rather than their mere names or meanings.

[0048] FIG. 1 shows an example of a point cloud content provision system according to embodiments.

[0049] The point cloud content providing system illustrated in FIG. 1 may include a transmission device (10000) and a reception device (10004). The transmission device (10000) and the reception device (10004) can communicate via wired or wireless means to transmit and receive point cloud data.

[0050] A transmission device (10000) according to embodiments can acquire, process, and transmit point cloud video (or point cloud content). According to embodiments, the transmission device (10000) may include a fixed station, a base transceiver system (BTS), a network, an AI (Artificial Intelligence) device and / or system, a robot, an AR / VR / XR device and / or server, etc. Additionally, according to embodiments, the transmission device (10000) may include a device that communicates with a base station and / or other wireless devices using wireless access technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)), a robot, a vehicle, an AR / VR / XR device, a mobile device, a home appliance, an IoT (Internet of Things) device, an AI device / server, etc.

[0051] A transmission device (10000) according to embodiments includes a point cloud video acquisition unit (10001), a point cloud video encoder (10002), and / or a transmitter (or communication module), 10003.

[0052] A point cloud video acquisition unit (10001) according to the embodiments acquires a point cloud video through processing steps such as capture, synthesis, or generation. The point cloud video is a point cloud content represented as a point cloud, which is a set of points located in a three-dimensional space, and may be referred to as point cloud video data, etc. The point cloud video according to the embodiments may include one or more frames. A frame represents a still image / picture. Accordingly, the point cloud video may include a point cloud image / frame / picture and may be referred to as any one of a point cloud image, a frame, and a picture.

[0053] A point cloud video encoder (10002) according to the embodiments encodes the obtained point cloud video data. The point cloud video encoder (10002) can encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to the embodiments may include Geometry-based Point Cloud Compression (G-PCC) coding and / or Video-based Point Cloud Compression (V-PCC) coding or next-generation coding. Furthermore, the point cloud compression coding according to the embodiments is not limited to the embodiments described above. The point cloud video encoder (10002) can output a bitstream containing the encoded point cloud video data. The bitstream may include not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0054] A transmitter (10003) according to the embodiments transmits a bitstream containing encoded point cloud video data. The bitstream according to the embodiments is encapsulated into a file or segment (e.g., a streaming segment) and transmitted through various networks such as a broadcast network and / or a broadband network. Although not illustrated in the drawings, the transmission device (10000) may include an encapsulation unit (or encapsulation module) that performs an encapsulation operation. Additionally, according to the embodiments, the encapsulation unit may be included in the transmitter (10003). According to the embodiments, the file or segment may be transmitted to a receiving device (10004) via a network or stored on a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitter (10003) according to the embodiments can communicate wired or wirelessly with the receiving device (10004) (or receiver (10005)) via a network such as 4G, 5G, or 6G. Additionally, the transmitter (10003) can perform necessary data processing operations according to a network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). Additionally, the transmission device (10000) can transmit encapsulated data according to an on-demand method.

[0055] A receiving device (10004) according to embodiments includes a receiver (10005), a point cloud video decoder (10006), and / or a renderer (10007). According to embodiments, the receiving device (10004) may include a device, robot, vehicle, AR / VR / XR device, mobile device, home appliance, IoT (Internet of Thing) device, AI device / server, etc., that communicates with a base station and / or other wireless device using wireless access technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)).

[0056] A receiver (10005) according to the embodiments receives a bitstream containing point cloud video data or a file / segment containing the bitstream from a network or a storage medium. The receiver (10005) can perform necessary data processing operations according to a network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). The receiver (10005) according to the embodiments can output a bitstream by decapsulating the received file / segment. Additionally, according to the embodiments, the receiver (10005) may include a decapsulation unit (or decapsulation module) for performing a decapsulation operation. Additionally, the decapsulation unit may be implemented as an element (or component) separate from the receiver (10005).

[0057] A point cloud video decoder (10006) decodes a bitstream containing point cloud video data. The point cloud video decoder (10006) can decode the point cloud video data according to the way the point cloud video data is encoded (e.g., the reverse process of the operation of a point cloud video encoder (10002)). Accordingly, the point cloud video decoder (10006) can decode the point cloud video data by performing point cloud decompression coding, which is the reverse process of point cloud compression. Point cloud decompression coding includes G-PCC coding.

[0058] The renderer (10007) renders the decoded point cloud video data. The renderer (10007) can render not only the point cloud video data but also audio data to output point cloud content. According to embodiments, the renderer (10007) may include a display for displaying the point cloud content. According to embodiments, the display may not be included in the renderer (10007) but may be implemented as a separate device or component.

[0059] The arrows indicated by dotted lines in the drawing represent the transmission path of feedback information obtained from the receiving device (10004). The feedback information is information intended to reflect interaction with a user consuming point cloud content, and includes user information (e.g., head orientation information), viewport information, etc. In particular, if the point cloud content is content for a service requiring interaction with a user (e.g., autonomous driving service, etc.), the feedback information may be transmitted to the content transmitting side (e.g., the transmitting device (10000)) and / or the service provider. Depending on the embodiments, the feedback information may be used in the receiving device (10004) as well as the transmitting device (10000), or it may not be provided.

[0060] Head orientation information according to the embodiments is information regarding the user's head position, direction, angle, movement, etc. The receiving device (10004) according to the embodiments can calculate viewport information based on the head orientation information. Viewport information is information about the area of ​​the point cloud video that the user is looking at. The viewpoint refers to the point where the user is looking at the point cloud video, and may mean the exact center point of the viewport area. That is, the viewport is an area centered on the viewpoint, and the size and shape of the area can be determined by the Field Of View (FOV). Therefore, the receiving device (10004) can extract viewport information based on the vertical or horizontal FOV supported by the device in addition to the head orientation information. In addition, the receiving device (10004) performs gaze analysis, etc., to check the user's point cloud consumption method, the point cloud video area the user is looking at, the gaze time, etc. According to embodiments, the receiving device (10004) may transmit feedback information including gaze analysis results to the transmitting device (10000). According to embodiments, the feedback information may be obtained during the rendering and / or display process. According to embodiments, the feedback information may be obtained by one or more sensors included in the receiving device (10004). Also, according to embodiments, the feedback information may be obtained by the renderer (10007) or a separate external element (or device, component, etc.). The dotted line in FIG. 1 indicates the process of transmitting the feedback information obtained from the renderer (10007). The point cloud content providing system may process (encode / decode) point cloud data based on the feedback information. Accordingly, the point cloud video data decoder (10006) may perform a decoding operation based on the feedback information.Additionally, the receiving device (10004) can transmit feedback information to the transmitting device (10000). The transmitting device (10000) (or the point cloud video data encoder (10002)) can perform an encoding operation based on the feedback information. Thus, the point cloud content providing system can efficiently process necessary data (e.g., point cloud data corresponding to the user's head position) based on the feedback information without processing (encoding / decoding) all point cloud data, and provide point cloud content to the user.

[0061] According to embodiments, the transmission device (10000) may be referred to as an encoder, transmission device, transmitter, etc., and the receiving device (10004) may be referred to as a decoder, receiving device, receiver, etc.

[0062] Point cloud data processed in the point cloud content providing system of FIG. 1 according to embodiments (processed through a series of processes of acquisition / encoding / transmission / decoding / rendering) may be referred to as point cloud content data or point cloud video data. According to embodiments, point cloud content data may be used as a concept including metadata or signaling information related to point cloud data.

[0063] The elements of the point cloud content delivery system illustrated in FIG. 1 can be implemented using hardware, software, processors, and / or combinations thereof.

[0064] FIG. 2 is a block diagram illustrating a point cloud content provision operation according to embodiments.

[0065] The block diagram of FIG. 2 illustrates the operation of the point cloud content provision system described in FIG. 1. As described above, the point cloud content provision system can process point cloud data based on point cloud compression coding (e.g., G-PCC).

[0066] A point cloud content providing system according to the embodiments (e.g., a point cloud transmission device (10000) or a point cloud video acquisition unit (10001)) can acquire a point cloud video (20000). The point cloud video is represented as a point cloud belonging to a coordinate system representing a three-dimensional space. The point cloud video according to the embodiments may include a Ply (Polygon File format or the Stanford Triangle format) file. If the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as the geometry and / or attributes of the points. The geometry includes the positions of the points. The position of each point may be represented by parameters (e.g., values ​​of the X-axis, Y-axis, and Z-axis, respectively) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of XYZ axes). Attributes include attributes of points (e.g., texture information, color (YCbCr or RGB), reflectance (r), transparency, etc. of each point). A point has one or more attributes (or properties). For example, a point may have one attribute which is color, or two attributes which are color and reflectance. According to embodiments, geometry may be referred to as positions, geometry information, geometry data, etc., and attributes may be referred to as attributes, attribute information, attribute data, etc.In addition, a point cloud content provision system (e.g., a point cloud transmission device (10000) or a point cloud video acquisition unit (10001)) can obtain point cloud data from information related to the acquisition process of point cloud video (e.g., depth information, color information, etc.).

[0067] A point cloud content providing system (e.g., a transmission device (10000) or a point cloud video encoder (10002)) according to embodiments can encode point cloud data (20001). The point cloud content providing system can encode point cloud data based on point cloud compression coding. As described above, point cloud data may include geometry and attributes of points. Accordingly, the point cloud content providing system can output a geometry bitstream by performing geometry encoding to encode geometry. The point cloud content providing system can output an attribute bitstream by performing attribute encoding to encode attributes. According to embodiments, the point cloud content providing system can perform attribute encoding based on geometry encoding. The geometry bitstream and attribute bitstream according to embodiments can be multiplexed and output as a single bitstream. The bitstream according to the embodiments may further include signaling information related to geometry encoding and attribute encoding.

[0068] A point cloud content providing system according to embodiments (e.g., a transmission device (10000) or a transmitter (10003)) can transmit encoded point cloud data (20002). As described in FIG. 1, the encoded point cloud data can be represented as a geometry bitstream and an attribute bitstream. Additionally, the encoded point cloud data can be transmitted in the form of a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometry encoding and attribute encoding). Additionally, the point cloud content providing system can encapsulate the bitstream transmitting the encoded point cloud data and transmit it in the form of a file or segment.

[0069] A point cloud content providing system according to embodiments (e.g., a receiving device (10004) or a receiver (10005)) can receive a bitstream containing encoded point cloud data. Additionally, the point cloud content providing system (e.g., a receiving device (10004) or a receiver (10005)) can demultiplex the bitstream.

[0070] A point cloud content providing system (e.g., a receiving device (10004) or a point cloud video decoder (10005)) can decode encoded point cloud data (e.g., a geometry bitstream, an attribute bitstream) transmitted as a bitstream. A point cloud content providing system (e.g., a receiving device (10004) or a point cloud video decoder (10005)) can decode point cloud video data based on signaling information related to the encoding of point cloud video data included in the bitstream. A point cloud content providing system (e.g., a receiving device (10004) or a point cloud video decoder (10005)) can decode the geometry bitstream to restore the positions (geometry) of the points. A point cloud content providing system can decode the attribute bitstream based on the restored geometry to restore the attributes of the points. A point cloud content delivery system (e.g., a receiving device (10004) or a point cloud video decoder (10005)) can restore a point cloud video based on positions according to the restored geometry and decoded attributes.

[0071] A point cloud content providing system according to embodiments (e.g., a receiving device (10004) or a renderer (10007)) can render decoded point cloud data (20004). The point cloud content providing system (e.g., a receiving device (10004) or a renderer (10007)) can render geometry and attributes decoded through a decoding process according to various rendering methods. Points of the point cloud content may be rendered as vertices having a certain thickness, cubes having a specific minimum size with the vertex location as the center, or circles with the vertex location as the center, etc. All or part of the rendered point cloud content is provided to a user through a display (e.g., a VR / AR display, a general display, etc.).

[0072] A point cloud content providing system (e.g., a receiving device (10004)) according to the embodiments can obtain feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. Since the feedback information and the operation of the point cloud content providing system according to the embodiments are the same as the feedback information and operation described in FIG. 1, a detailed description is omitted.

[0073] FIG. 3 shows an example of a point cloud encoder according to embodiments.

[0074] FIG. 3 shows an example of the point cloud video encoder (10002) of FIG. 1. The point cloud encoder reconstructs point cloud data (e.g., positions and / or attributes of points) and performs encoding operations to adjust the quality of point cloud content (e.g., lossless, lossy, near-lossless) according to network conditions or applications. If the total size of the point cloud content is large (e.g., point cloud content of 60 Gbps in the case of 30 fps), the point cloud content delivery system may not be able to stream the content in real time. Therefore, the point cloud content delivery system may reconstruct the point cloud content based on a maximum target bitrate to provide it according to the network environment.

[0075] As described in FIGS. 1 and 2, the point cloud encoder can perform geometry encoding and attribute encoding. Geometry encoding is performed before attribute encoding.

[0076] The point cloud encoder according to the embodiments comprises a coordinate system transformation unit (Transformation Coordinates, 30000), a quantization unit (Quantize and Remove Points (Voxelize), 30001), an octree analysis unit (Analyze Octree, 30002), a surface approximation analysis unit (Analyze Surface Approximation, 30003), an arithmetic encoder (Arithmetic Encode, 30004), a geometry reconstruction unit (Reconstruct Geometry, 30005), a color transformation unit (Transform Colors, 30006), an attribute transformation unit (Transfer Attributes, 30007), a RAHT transformation unit (30008), an LOD generation unit (Generated LOD, 30009), a lifting transformation unit (Lifting) (30010), and a coefficient quantization unit (Quantize Coefficients, 30011). Includes an and / or arithmetic encoder (Arithmetic Encode, 30012).

[0077] The coordinate system transformation unit (30000), quantization unit (30001), octree analysis unit (30002), surface approximation analysis unit (30003), arismetic encoder (30004), and geometry reconstruction unit (30005) can perform geometry encoding. Geometry encoding according to the embodiments may include octree geometry coding, direct coding, trisoup geometry encoding, and entropy encoding. Direct coding and trisoup geometry encoding are applied optionally or in combination. Additionally, geometry encoding is not limited to the above examples.

[0078] As illustrated in the drawings, the coordinate system conversion unit (30000) according to the embodiments receives positions and converts them into a coordinate system. For example, the positions can be converted into position information in a three-dimensional space (e.g., a three-dimensional space expressed in an XYZ coordinate system). The position information in the three-dimensional space according to the embodiments may be referred to as geometry information.

[0079] The quantization unit (30001) according to the embodiments quantizes the geometry. For example, the quantization unit (30001) can quantize points based on the minimum position values ​​of all points (e.g., minimum values ​​on each axis for the X-axis, Y-axis, and Z-axis). The quantization unit (30001) performs a quantization operation to find the nearest integer value by multiplying the difference between the minimum position value and the position value of each point by a preset quantization scale value and then performing rounding down or rounding up. Thus, one or more points may have the same quantized position (or position value). The quantization unit (30001) according to the embodiments performs voxelization based on the quantized positions to reconstruct the quantized points. Just as the minimum unit containing 2D image / video information is a pixel, the points of the point cloud content (or 3D point cloud video) according to the embodiments may be contained in one or more voxels. A voxel is a combination of volume and pixel, and refers to a three-dimensional cubic space that is generated when a three-dimensional space is divided into units (unit=1.0) based on axes representing the three-dimensional space (e.g., X-axis, Y-axis, Z-axis). The quantization unit (40001) can match groups of points in the three-dimensional space to voxels. According to embodiments, a single voxel may contain only one point. According to embodiments, a single voxel may contain one or more points. In addition, to represent a single voxel as a single point, the position of the center of the voxel can be set based on the positions of one or more points included in the voxel. In this case, the attributes of all positions included in the voxel can be combined and assigned to the voxel.

[0080] The octree analysis unit (30002) according to the embodiments performs octree geometry coding (or octree coding) to represent the voxels in an octree structure. The octree structure represents points matched to the voxels based on an octree structure.

[0081] The surface approximation analysis unit (30003) according to the embodiments can analyze and approximate an octree. The octree analysis and approximation according to the embodiments is a process of analyzing to voxelize an area containing multiple points in order to efficiently provide octree and voxelization.

[0082] An arithmetic encoder (30004) according to the embodiments entropy-encodes an octree and / or an approximated octree. For example, the encoding method includes an arithmetic encoding method. As a result of the encoding, a geometry bitstream is generated.

[0083] The color conversion unit (30006), attribute conversion unit (30007), RAHT conversion unit (30008), LOD generation unit (30009), lifting conversion unit (30010), coefficient quantization unit (30011) and / or arismetic encoder (30012) perform attribute encoding. As described above, a point may have one or more attributes. The attribute encoding according to the embodiments is applied equally to the attributes of a point. However, if a single attribute (e.g., color) includes one or more elements, independent attribute encoding is applied to each element. The attribute encoding according to the embodiments may include color conversion coding, attribute conversion coding, Region Adaptive Hierarchial Transform (RAHT) coding, prediction transformation (Interpolaration-based hierarchical nearest-neighbour prediction-Prediction Transform) coding, and lifting transformation (interpolation-based hierarchical nearest-neighbour prediction with an update / lifting step (Lifting Transform)) coding. Depending on the point cloud content, the above-described RAHT coding, prediction transformation coding, and lifting transformation coding may be used optionally, or a combination of one or more of the codings may be used. Furthermore, the attribute encoding according to the embodiments is not limited to the examples described above.

[0084] The color conversion unit (30006) according to the embodiments performs color conversion coding that converts color values ​​(or textures) included in attributes. For example, the color conversion unit (30006) can convert the format of color information (e.g., convert from RGB to YCbCr). The operation of the color conversion unit (30006) according to the embodiments may be applied optionally depending on the color values ​​included in attributes.

[0085] The geometry reconstruction unit (30005) according to the embodiments reconstructs (decompresses) an octree and / or an approximated octree. The geometry reconstruction unit (30005) reconstructs an octree / voxel based on the results of analyzing the distribution of points. The reconstructed octree / voxel may be referred to as the reconstructed geometry (or restored geometry).

[0086] The attribute transformation unit (30007) according to the embodiments performs attribute transformation that transforms attributes based on positions where geometry encoding has not been performed and / or reconstructed geometry. As described above, since attributes are dependent on geometry, the attribute transformation unit (30007) can transform attributes based on reconstructed geometry information. For example, the attribute transformation unit (30007) can transform the attributes of a point at a position based on the position value of a point included in a voxel. As described above, when the position of the center point of a voxel is set based on the positions of one or more points included in a voxel, the attribute transformation unit (30007) transforms the attributes of one or more points. When trisoop geometry encoding is performed, the attribute conversion unit (30007) can convert attributes based on the trisoop geometry encoding.

[0087] The attribute transformation unit (30007) can perform attribute transformation by calculating the average value of attributes or attribute values ​​(e.g., the color or reflectance of each point) of neighboring points within a specific location / radius from the position (or position value) of the center point of each voxel. The attribute transformation unit (30007) can apply a weight based on the distance from the center point to each point when calculating the average value. Thus, each voxel has a position and a calculated attribute (or attribute value).

[0088] The attribute conversion unit (30007) can search for neighboring points within a specific location / radius from the position of the center point of each voxel based on a KD tree or a Molton code. A KD tree is a binary search tree that supports a data structure capable of managing points based on their positions to enable rapid Nearest Neighbor Search (NNS). A Molton code is generated by representing the coordinate values ​​(e.g., (x, y, z)) representing the 3D positions of all points as bit values ​​and mixing the bits. For example, if the coordinate values ​​representing the position of a point are (5, 9, 1), the bit values ​​of the coordinate values ​​are (0101, 1001, 0001). When the bit values ​​are mixed according to the bit indices in the order of z, y, and x, it becomes 010001000111. When this value is represented in decimal, it becomes 1095. That is, the Molton code value of the point with coordinates (5, 9, 1) is 1095. The attribute transformation unit (30007) sorts the points based on the Molton code value and can perform shortest neighbor search (NNS) through a depth-first traversal process. After the attribute transformation operation, if shortest neighbor search (NNS) is required in other transformation processes for attribute coding, a KD tree or Molton code is utilized.

[0089] As shown in the drawing, the converted attributes are input to the RAHT conversion unit (30008) and / or LOD generation unit (30009).

[0090] The RAHT transformation unit (30008) according to the embodiments performs RAHT coding to predict attribute information based on reconstructed geometry information. For example, the RAHT transformation unit (30008) can predict attribute information of a node at an upper level of the octree based on attribute information associated with a node at a lower level of the octree.

[0091] The LOD generation unit (30009) according to the embodiments generates a Level of Detail (LOD) to perform predictive transformation coding. The LOD according to the embodiments represents the degree of detail of the point cloud content, and indicates that the smaller the LOD value, the lower the detail of the point cloud content, and the larger the LOD value, the higher the detail of the point cloud content. Points can be classified according to the LOD.

[0092] The lifting transformation unit (30010) according to the embodiments performs lifting transformation coding that transforms the attributes of the point cloud based on weights. As described above, the lifting transformation coding may be applied optionally.

[0093] The coefficient quantization unit (30011) according to the embodiments quantizes attribute-coded attributes based on coefficients.

[0094] An arismetic encoder (30012) according to the embodiments encodes quantized attributes based on arismetic coding.

[0095] The elements of the point cloud encoder of FIG. 3 may be implemented in hardware, software, firmware, or a combination thereof, comprising one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device, although not illustrated in the drawing. One or more processors may perform at least one of the operations and / or functions of the elements of the point cloud encoder of FIG. 3 described above. Additionally, one or more processors may operate or execute a set of software programs and / or instructions for performing the operations and / or functions of the elements of the point cloud encoder of FIG. 3. One or more memories according to the embodiments may include high-speed random access memory and may include non-volatile memory (e.g., one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0096] FIG. 4 shows examples of octree and occupancy codes according to embodiments.

[0097] As described in FIGS. 1 to 3, a point cloud content providing system (point cloud video encoder (10002)) or a point cloud encoder (e.g., an octree analysis unit (30002)) performs octree geometry coding based on an octree structure (or octree coding) to efficiently manage the area and / or position of a voxel.

[0098] The top of FIG. 4 shows an octree structure. The three-dimensional space of the point cloud content according to the embodiments is represented by the axes of the coordinate system (e.g., X-axis, Y-axis, Z-axis). The octree structure has two poles (0,0,0) and (2 d , 2 d , 2 d It is generated by recursively subdividing the bounding box (cubical axis-aligned bounding box) defined by ). 2d can be set to the value that constitutes the smallest bounding box enclosing all points of the point cloud content (or point cloud video). d represents the depth of the octree. The value of d is determined according to the following equation. In the equation below, (x int n , y int n , z int n ) represents the positions (or position values) of quantized points.

[0099] d =Ceil(Log2(Max(x_n^int,y_n^int,z_n^int,n=1,…,N)+1))

[0100] As illustrated in the middle of the top of Fig. 4, the entire three-dimensional space can be divided into eight spaces according to the division. Each divided space is represented as a cube having six faces. As illustrated in the right of the top of Fig. 4, each of the eight spaces is further divided based on the axes of the coordinate system (e.g., X-axis, Y-axis, Z-axis). Thus, each space is again divided into eight smaller spaces. The divided smaller spaces are also represented as cubes having six faces. This division method is applied until the leaf nodes of the octree become voxels.

[0101] The bottom of Fig. 4 shows the occupancy code of an octree. The occupancy code of an octree is generated to indicate whether each of the eight partitioned spaces resulting from the partitioning of a single space contains at least one point. Therefore, one occupancy code is represented by eight child nodes. Each child node represents the occupancy of the partitioned space, and the child node has a value of 1 bit. Thus, the occupancy code is represented as an 8-bit code. That is, if the space corresponding to the child node contains at least one point, the node has a value of 1. If the space corresponding to the child node does not contain a point (empty), the node has a value of 0. Since the occupancy code shown in Fig. 4 is 00100001, it indicates that the spaces corresponding to the 3rd and 8th child nodes among the eight child nodes each contain at least one point. As illustrated in the drawing, the 3rd child node and the 8th child node each have 8 child nodes, and each child node is represented by an 8-bit Occupancy code. The drawing indicates that the Occupancy code of the 3rd child node is 10000111 and the Occupancy code of the 8th child node is 01001111. A point cloud encoder according to the embodiments (e.g., an arismetic encoder (30004)) can entropy-encode the Occupancy code. Additionally, to increase compression efficiency, the point cloud encoder can intra- / inter-encode the Occupancy code. A receiving device according to the embodiments (e.g., a receiving device (10004) or a point cloud video decoder (10006)) reconstructs the octree based on the Occupancy code.

[0102] A point cloud encoder according to the embodiments (e.g., the point cloud encoder of FIG. 3, or the octree analysis unit (30002)) can perform voxelization and octree coding to store the positions of the points. However, since points in a three-dimensional space are not always evenly distributed, there may be specific areas where few points exist. Therefore, performing voxelization on the entire three-dimensional space is inefficient. For example, if there are almost no points in a specific area, there is no need to perform voxelization up to that area.

[0103] Accordingly, the point cloud encoder according to the embodiments can perform direct coding, which directly codes the positions of points included in a specific region (or nodes excluding leaf nodes of an octree) without performing voxelization on the aforementioned specific region. The coordinates of the points directly coded according to the embodiments are referred to as the Direct Coding Mode (DCM). Additionally, the point cloud encoder according to the embodiments can perform trisoup geometry encoding, which reconstructs the positions of points within a specific region (or node) based on voxels using a surface model. Trisoup geometry encoding is a geometry encoding that represents an object as a series of triangle meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct coding and trisoup geometry encoding according to the embodiments may be performed optionally. In addition, direct coding and trisoop geometry encoding according to the embodiments can be performed in combination with octree geometry coding (or octree coding).

[0104] To perform direct coding, the option to use direct mode for applying direct coding must be enabled, the node to which direct coding is to be applied must not be a leaf node, and there must be points within a specific node that are below a threshold. In addition, the total number of points subject to direct coding must not exceed a preset threshold. If the above conditions are satisfied, the point cloud encoder (or arismetic encoder (30004)) according to the embodiments can entropy-code the positions (or position values) of the points.

[0105] A point cloud encoder according to the embodiments (e.g., a surface approximation analysis unit (30003)) can determine a specific level of an octree (where the level is smaller than the depth d of the octree) and, starting from that level, perform trisoop geometry encoding to reconstruct the position of points within a node region based on voxels using a surface model (trisoop mode). The point cloud encoder according to the embodiments can specify the level to which trisoop geometry encoding is applied. For example, if the specified level is equal to the depth of the octree, the point cloud encoder does not operate in trisoop mode. That is, the point cloud encoder according to the embodiments can operate in trisoop mode only when the specified level is smaller than the depth value of the octree. A three-dimensional cubic region of nodes at a specified level according to the embodiments is referred to as a block. A block may include one or more voxels. A block or a voxel may correspond to a brick. Within each block, geometry is represented as a surface. A surface according to the embodiments may intersect each edge of the block at most once.

[0106] Since one block has 12 edges, there are at least 12 intersection points within one block. Each intersection point is referred to as a vertex. A vertex along an edge is detected if there is at least one occupied voxel adjacent to that edge among all blocks sharing that edge. An occupied voxel according to the embodiments means a voxel containing a point. The position of a vertex detected along an edge is the average position along the edge of all voxels adjacent to that edge among all blocks sharing that edge.

[0107] When a vertex is detected, the point cloud encoder according to the embodiments can entropy-code the edge start point (x, y, z), edge direction vector (Δx, Δy, Δz), and vertex position value (relative position value within the edge). When trisoop geometry encoding is applied, the point cloud encoder according to the embodiments (e.g., geometry reconstruction unit (30005)) can generate restored geometry (reconstructed geometry) by performing triangle reconstruction, up-sampling, and voxelization processes.

[0108] The vertices located on the edges of the block determine the surface passing through the block. The surface according to the embodiments is a non-planar polygon. The triangle reconstruction process reconstructs the surface represented by triangles based on the edge start point, the edge direction vector, and the vertex position value. The triangle reconstruction process is as follows: ① calculate the centroid value of each vertex, ② subtract the centroid value from each vertex value, ③ square the result, and add all the result together.

[0109]

[0110] The minimum sum is calculated, and a projection process is performed along the axis where the minimum value is located. For example, if the x-element is at its minimum, each vertex is projected along the x-axis relative to the center of the block and onto the (y, z) plane. If the value obtained from projecting onto the (y, z) plane is (ai, bi), the θ value is calculated using atan2(bi, ai), and the vertices are aligned based on the θ value. The table below shows the combinations of vertices to generate triangles depending on the number of vertices. The vertices are aligned in order from 1 to n. The table below indicates that for four vertices, two triangles can be formed depending on the combination of vertices. The first triangle can be formed from the 1st, 2nd, and 3rd vertices among the aligned vertices, and the second triangle can be formed from the 3rd, 4th, and 1st vertices among the aligned vertices.

[0111] Table 2-1. Triangles formed from vertices ordered 1,… ,n

[0112] n triangles

[0113] 3 (1,2,3)

[0114] 4 (1,2,3), (3,4,1)

[0115] 5 (1,2,3), (3,4,5), (5,1,3)

[0116] 6 (1,2,3), (3,4,5), (5,6,1), (1,3,5)

[0117] 7 (1,2,3), (3,4,5), (5,6,7), (7,1,3), (3,5,7)

[0118] 8 (1,2,3), (3,4,5), (5,6,7), (7,8,1), (1,3,5), (5,7,1)

[0119] 9 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,1,3), (3,5,7), (7,9,3)

[0120] 10 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,1), (1,3,5), (5,7,9), (9,1,5)

[0121] 11 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,11), (11,1,3), (3,5,7), (7,9,11), (11,3,7)

[0122] 12 (1,2,3), (3,4,5), (5,6,7), (7,8,9), (9,10,11), (11,12,1), (1,3,5), (5,7,9), (9,11,1), (1,5,9)

[0123] The upsampling process is performed to voxelize by adding intermediate points along the edges of the triangle. Additional points are generated based on the upsampling factor value and the width of the block. The additional points are referred to as refined vertices. A point cloud encoder according to the embodiments can voxelize the refined vertices. Additionally, the point cloud encoder can perform attribute encoding based on the voxelized positions (or position values).

[0124] Figure 5 shows an example of a point configuration by LOD according to embodiments.

[0125] As described in FIGS. 1 to 4, the encoded geometry is reconstructed (decompressed) before attribute encoding is performed. When direct coding is applied, the geometry reconstruction operation may include changing the arrangement of the direct-coded points (e.g., placing the direct-coded points at the front of the point cloud data). When trisoop geometry encoding is applied, the geometry reconstruction process involves triangle reconstruction, upsampling, and voxelization. Since attributes depend on geometry, attribute encoding is performed based on the reconstructed geometry.

[0126] A point cloud encoder (e.g., an LOD generation unit (30009)) can reorganize points by LOD. The drawing shows point cloud content corresponding to the LOD. The left side of the drawing shows the original point cloud content. The second figure from the left of the drawing shows the distribution of points of the lowest LOD, and the rightmost figure of the drawing shows the distribution of points of the highest LOD. That is, the points of the lowest LOD are sparsely distributed, while the points of the highest LOD are densely distributed. In other words, according to the direction of the arrow indicated at the bottom of the drawing, as the LOD increases, the spacing (or distance) between points becomes shorter.

[0127] Figure 6 shows an example of a point configuration by LOD according to embodiments.

[0128] As described in FIGS. 1 to 5, a point cloud content providing system or a point cloud encoder (e.g., a point cloud video encoder (10002), the point cloud encoder of FIG. 3, or an LOD generation unit (30009)) can generate an LOD. The LOD is generated by reorganizing points into a set of refinement levels according to a set LOD distance value (or a set of Euclidean distances). The LOD generation process is performed in a point cloud decoder as well as a point cloud encoder.

[0129] The top of Fig. 6 shows examples of points (P0 to P9) of point cloud content distributed in three-dimensional space. The Original Order in Fig. 6 represents the order of points P0 to P9 prior to LOD generation. The LOD-based Order in Fig. 6 represents the order of points following LOD generation. Points are rearranged by LOD. Additionally, higher LODs include points belonging to lower LODs. As illustrated in Fig. 6, LOD0 includes P0, P5, P4, and P2. LOD1 includes the points of LOD0 and P1, P6, and P3. LOD2 includes the points of LOD0, the points of LOD1, and P9, P8, and P7.

[0130] As described in FIG. 3, the point cloud encoder according to the embodiments can perform predictive transform coding, lifting transform coding, and RAHT transform coding selectively or in combination.

[0131] The point cloud encoder according to the embodiments can generate predictors for points and perform predictive transformation coding to set the predicted attribute (or predicted attribute value) of each point. That is, N predictors can be generated for N points. The predictor according to the embodiments can calculate a weight (=1 / distance) value based on the LOD value of each point, indexing information for neighboring points within a set distance per LOD, and the distance value to the neighboring points.

[0132] According to the embodiments, the predicted attribute (or attribute value) is set as the average value of the values ​​obtained by multiplying the attributes (or attribute values, e.g., color, reflectance, etc.) of neighboring points set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder according to the embodiments (e.g., coefficient quantization unit (30011)) can quantize and inverse quantize the residual values ​​(which may be referred to as residual attributes, residual attribute values, attribute prediction residual values, etc.) obtained by subtracting the predicted attribute (attribute value) from the attribute (attribute value) of each point. The quantization process is as shown in the following table.

[0133] graph. Attribute prediction residuals quantization pseudo code

[0134] int PCCQuantization(int value, int quantStep) {

[0135] if( value >=0) {

[0136] return floor(value / quantStep + 1.0 / 3.0);

[0137] } else {

[0138] return -floor(-value / quantStep + 1.0 / 3.0);

[0139] }

[0140] }

[0141] graph. Attribute prediction residuals inverse quantization pseudo code

[0142] int PCCInverseQuantization(int value, int quantStep) {

[0143] if( quantStep ==0) {

[0144] return value;

[0145] } else {

[0146] return value * quantStep;

[0147] }

[0148] }

[0149] A point cloud encoder according to the embodiments (e.g., an arismetic encoder (30012)) can entropy-code the quantized and inversely quantized residual values ​​as described above when there are neighboring points in the predictor of each point. A point cloud encoder according to the embodiments (e.g., an arismetic encoder (30012)) can entropy-code the attributes of the corresponding point without performing the process described above when there are no neighboring points in the predictor of each point.

[0150] A point cloud encoder according to the embodiments (e.g., a lifting transformation unit (30010)) can perform lifting transformation coding by generating a predictor for each point, setting the LOD calculated in the predictor, registering neighboring points, and setting weights based on the distance to neighboring points. The lifting transformation coding according to the embodiments is similar to the prediction transformation coding described above, but differs in that weights are cumulatively applied to attribute values. The process of cumulatively applying weights to attribute values ​​according to the embodiments is as follows.

[0151] 1) Create an array QW (QuantizationWight) to store the weight values ​​of each point. The initial value of all elements in QW is 1.0. Add the value obtained by multiplying the current point's predictor weight by the QW value of the predictor index of the neighboring node registered in the predictor.

[0152] 2) Lift prediction process: To calculate the predicted attribute value, the value obtained by multiplying the point's attribute value by a weight is subtracted from the existing attribute value.

[0153] 3) Create temporary arrays named updateweight and update, and initialize the temporary arrays to 0.

[0154] 4) For all predictors, the calculated weight is additionally multiplied by the weight stored in the QW corresponding to the predictor index, and the resulting weight is accumulated in the update weight array with the neighbor node index. In the update array, the value obtained by multiplying the attribute value of the neighbor node index by the calculated weight is accumulated.

[0155] 5) Lift update process: For all predictors, the attribute value of the update array is divided by the weight value of the update weight array at the predictor index, and the original attribute value is added back to the divided value.

[0156] 6) For all predictors, the predicted attribute value is calculated by additionally multiplying the attribute value updated through the lift update process by the weight (stored in QW) updated through the lift prediction process. A point cloud encoder according to the embodiments (e.g., coefficient quantizer (30011)) quantizes the predicted attribute value. Additionally, a point cloud encoder (e.g., arismetic encoder (30012)) entropies the quantized attribute value.

[0157] A point cloud encoder according to the embodiments (e.g., a RAHT transform unit (30008)) can perform RAHT transform coding to predict attributes of upper-level nodes using attributes associated with nodes at lower levels of the octree. RAHT transform coding is an example of attribute intra-coding through octree backward scanning. A point cloud encoder according to the embodiments scans from voxels to the entire region and repeats the merging process up to the root node, merging voxels into larger blocks at each step. The merging process according to the embodiments is performed only on occupied nodes. The merging process is not performed on empty nodes, and the merging process is performed on the node immediately above the empty node.

[0158] The following equation represents the RAHT transformation matrix. g l x, y, z represents the average attribute value of the voxels at level l. g l x, y, z can be calculated from gl+1 2x, y, z and gl+1 2x+1, y, z. The weights of gl 2x, y, z and gl 2x+1, y, z are w1=wl 2x, y, z and w2=wl 2x+1, y, z.

[0159]

[0160] gl-1 x, y, z are low-pass values ​​used in the merging process at the next higher level. hl-1 x, y, z are high-pass coefficients, and the high-pass coefficients at each step are quantized and entropy-coded (e.g., encoding of an arismetic encoder (400012)). The weights are calculated as wl-1 x, y, z = wl 2x, y, z + wl 2x + 1, y, z. The root node is the last g 1 0, 0, 0 and g 1 0, 0, 1 It is generated as follows through.

[0161]

[0162] The gDC value is also quantized and entropy-coded, just like the high-pass coefficient.

[0163] FIG. 7 shows an example of a point cloud decoder according to embodiments.

[0164] The point cloud decoder illustrated in FIG. 7 is an example of a point cloud decoder and can perform a decoding operation, which is the reverse process of the encoding operation of the point cloud encoder described in FIG. 1 to 6.

[0165] As described in Fig. 1, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0166] A point cloud decoder according to the embodiments comprises an arithmetic decoder (7000), a synthesize octree (7001), a synthesize surface approximation (7002), a reconstruct geometry (7003), an inverse transform coordinates (7004), an arithmetic decoder (7005), an inverse quantize (7006), a RAHT transform (7007), an LOD generater (7008), an inverse lifting (7009), and / or an inverse transform colors (7010).

[0167] An arismetic decoder (7000), an octree composite unit (7001), a surface offset composite unit (7002), a geometry reconstruction unit (7003), and a coordinate system inverse transformation unit (7004) can perform geometry decoding. Geometry decoding according to the embodiments may include direct coding and trisoup geometry decoding. Direct coding and trisoup geometry decoding are applied optionally. Additionally, geometry decoding is not limited to the above examples and is performed as the reverse process of geometry encoding described in FIGS. 1 through 6.

[0168] The arismetic decoder (7000) according to the embodiments decodes the received geometry bitstream based on arismetic coding. The operation of the arismetic decoder (7000) corresponds to the reverse process of the arismetic encoder (30004).

[0169] The octree synthesis unit (7001) according to the embodiments can generate an octree by obtaining an Occupancy code from a decoded geometry bitstream (or information regarding the geometry obtained as a result of decoding). A specific description of the Occupancy code is as described in FIGS. 1 to 6.

[0170] The surface off-relation synthesis unit (7002) according to the embodiments can synthesize a surface based on the decoded geometry and / or the generated octree when trisoop geometry encoding is applied.

[0171] The geometry reconstruction unit (7003) according to the embodiments can regenerate geometry based on a surface and / or decoded geometry. As described in FIGS. 1 through 6, direct coding and trisoop geometry encoding are applied optionally. Accordingly, the geometry reconstruction unit (7003) directly retrieves and adds position information of points to which direct coding has been applied. In addition, when trisoop geometry encoding is applied, the geometry reconstruction unit (7003) can restore geometry by performing reconstruction operations of the geometry reconstruction unit (30005), such as triangle reconstruction, up-sampling, and voxelization operations. Specific details are omitted as they are the same as those described in FIG. 4. The restored geometry may include a point cloud picture or frame that does not contain attributes.

[0172] The coordinate system inverse transformation unit (7004) according to the embodiments can obtain the positions of the points by transforming the coordinate system based on the restored geometry.

[0173] The arismetic decoder (7005), inverse quantization unit (7006), RAHT transformation unit (7007), LOD generation unit (7008), inverse lifting unit (7009), and / or color inverse transformation unit (7010) can perform attribute decoding as described in FIG. 10. Attribute decoding according to the embodiments may include Region Adaptive Hierarchial Transform (RAHT) decoding, Interpolaration-based hierarchical nearest-neighbour prediction-Prediction Transform) decoding, and interpolation-based hierarchical nearest-neighbour prediction with an update / lifting step (Lifting Transform) decoding. The three decodings described above may be used optionally, or a combination of one or more decodings may be used. Furthermore, attribute decoding according to the embodiments is not limited to the examples described above.

[0174] The arismetic decoder (7005) according to the embodiments decodes the attribute bitstream into arismetic coding.

[0175] The inverse quantization unit (7006) according to the embodiments inverse quantizes information about the decoded attribute bitstream or the attribute obtained as a result of decoding and outputs the inverse quantized attributes (or attribute values). Inverse quantization may be optionally applied based on the attribute encoding of the point cloud encoder.

[0176] According to embodiments, the RAHT transformation unit (7007), LOD generation unit (7008), and / or inverse lifting unit (7009) can process the reconstructed geometry and inverse quantized attributes. As described above, the RAHT transformation unit (7007), LOD generation unit (7008), and / or inverse lifting unit (7009) can optionally perform a corresponding decoding operation according to the encoding of the point cloud encoder.

[0177] The color inverse conversion unit (7010) according to the embodiments performs inverse conversion coding to inversely convert the color value (or texture) included in the decoded attributes. The operation of the color inverse conversion unit (7010) may be selectively performed based on the operation of the color conversion unit (30006) of the point cloud encoder.

[0178] The elements of the point cloud decoder of FIG. 7 may be implemented in hardware, software, firmware, or a combination thereof, comprising one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device, although not illustrated in the drawing. One or more processors may perform at least one of the operations and / or functions of the elements of the point cloud decoder of FIG. 7 described above. Additionally, one or more processors may operate or execute a set of software programs and / or instructions for performing the operations and / or functions of the elements of the point cloud decoder of FIG. 7.

[0179] FIG. 8 is an example of a transmission device according to embodiments.

[0180] The transmission device illustrated in FIG. 8 is an example of the transmission device (10000) of FIG. 1 (or the point cloud encoder of FIG. 3). The transmission device illustrated in FIG. 8 can perform at least one of the same or similar operations and methods as the operations and encoding methods of the point cloud encoder described in FIG. 1 to 6. A transmission device according to embodiments may include a data input unit (8000), a quantization processing unit (8001), a voxelization processing unit (8002), an octree occupancy code generation unit (8003), a surface model processing unit (8004), an intra / inter coding processing unit (8005), an arithmetic coder (8006), a metadata processing unit (8007), a color conversion processing unit (8008), an attribute conversion processing unit (or attribute conversion processing unit) (8009), a prediction / lifting / RAHT conversion processing unit (8010), an arithmetic coder (8011) and / or a transmission processing unit (8012).

[0181] The data input unit (8000) according to the embodiments receives or acquires point cloud data. The data input unit (8000) may perform an operation and / or acquisition method identical or similar to the operation and / or acquisition method of the point cloud video acquisition unit (10001) (or the acquisition process (20000) described in FIG. 2).

[0182] The data input unit (8000), quantization processing unit (8001), voxelization processing unit (8002), octree occupancy code generation unit (8003), surface model processing unit (8004), intra / inter coding processing unit (8005), and arithmetic coder (8006) perform geometry encoding. Since the geometry encoding according to the embodiments is identical or similar to the geometry encoding described in FIGS. 1 to 6, a detailed description is omitted.

[0183] The quantization processing unit (8001) according to the embodiments quantizes geometry (e.g., location values ​​of points, or position values). The operation and / or quantization of the quantization processing unit (8001) is the same or similar to the operation and / or quantization of the quantization unit (30001) described in FIG. 3. The specific description is the same as that described in FIG. 1 through 6.

[0184] The voxelization processing unit (8002) according to the embodiments voxelizes the position values ​​of the quantized points. The voxelization processing unit (80002) may perform the same or similar operation and / or process as the operation and / or voxelization process of the quantization unit (30001) described in FIG. 3. The specific description is the same as that described in FIG. 1 to 6.

[0185] The octree occupancy code generation unit (8003) according to the embodiments performs octree coding on the positions of voxelized points based on an octree structure. The octree occupancy code generation unit (8003) can generate an occupancy code. The octree occupancy code generation unit (8003) can perform operations and / or methods identical or similar to the operations and / or methods of the point cloud encoder (or octree analysis unit (30002)) described in FIGS. 3 and 4. The specific description is the same as that described in FIGS. 1 through 6.

[0186] The surface model processing unit (8004) according to the embodiments can perform trisup geometry encoding that reconstructs the positions of points within a specific region (or node) based on a voxel based on a surface model. The surface model processing unit (8004) can perform operations and / or methods identical or similar to the operations and / or methods of the point cloud encoder (e.g., surface approximation analysis unit (30003)) described in FIG. 3. The specific description is the same as that described in FIG. 1 through 6.

[0187] According to the embodiments, the intra / inter coding processing unit (8005) can intra / inter code point cloud data. The intra / inter coding processing unit (8005) can perform coding identical or similar to the intra / inter coding described in FIG. 7. The specific description is the same as that described in FIG. 7. According to the embodiments, the intra / inter coding processing unit (8005) may be included in an arismetic coder (8006).

[0188] An arismetic coder (8006) according to the embodiments entropy-encodes an octree and / or approximated octree of point cloud data. For example, the encoding method includes an arismetic encoding method. The arismetic coder (8006) performs the same or similar operation and / or method as the arismetic encoder (30004).

[0189] A metadata processing unit (8007) according to the embodiments processes metadata regarding point cloud data, such as setting values, and provides it to necessary processing processes such as geometry encoding and / or attribute encoding. Additionally, a metadata processing unit (8007) according to the embodiments may generate and / or process signaling information related to geometry encoding and / or attribute encoding. The signaling information according to the embodiments may be encoded separately from geometry encoding and / or attribute encoding. Additionally, the signaling information according to the embodiments may be interleaved.

[0190] The color conversion processing unit (8008), attribute conversion processing unit (8009), prediction / lifting / RAHT conversion processing unit (8010), and arithmetic coder (8011) perform attribute encoding. Since the attribute encoding according to the embodiments is identical or similar to the attribute encoding described in FIGS. 1 to 6, a detailed description is omitted.

[0191] The color conversion processing unit (8008) according to the embodiments performs color conversion coding that converts color values ​​included in attributes. The color conversion processing unit (8008) may perform color conversion coding based on reconstructed geometry. The description of the reconstructed geometry is the same as that described in FIGS. 1 through 6. In addition, it performs the same or similar operation and / or method as the operation and / or method of the color conversion unit (30006) described in FIG. 3. A detailed description is omitted.

[0192] The attribute transformation processing unit (8009) according to the embodiments performs attribute transformation that transforms attributes based on positions where geometry encoding has not been performed and / or reconstructed geometry. The attribute transformation processing unit (8009) performs operations and / or methods identical or similar to the operations and / or methods of the attribute transformation unit (30007) described in FIG. 3. A detailed description is omitted. The prediction / lifting / RAHT transformation processing unit (8010) according to the embodiments may code the transformed attributes by RAHT coding, prediction transformation coding, and lifting transformation coding, or a combination thereof. The prediction / lifting / RAHT transformation processing unit (8010) performs at least one of operations identical or similar to the operations of the RAHT transformation unit (30008), LOD generation unit (30009), and lifting transformation unit (30010) described in FIG. 3. In addition, the descriptions of predictive transformation coding, lifting transformation coding, and RAHT transformation coding are the same as those described in Figures 1 to 6, so a detailed description is omitted.

[0193] The arismetic coder (8011) according to the embodiments can encode coded attributes based on arismetic coding. The arismetic coder (8011) performs the same or similar operation and / or method as the operation and / or method of the arismetic encoder (300012).

[0194] A transmission processing unit (8012) according to embodiments may transmit each bitstream containing encoded geometry and / or encoded attributes and metadata information, or may transmit the encoded geometry and / or encoded attributes and metadata information by configuring them into a single bitstream. When the encoded geometry and / or encoded attributes and metadata information according to embodiments is configured into a single bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to embodiments may include signaling information and slice data, including SPS (Sequence Parameter Set) for sequence-level signaling, GPS (Geometry Parameter Set) for signaling of geometry information coding, APS (Attribute Parameter Set) for signaling of attribute information coding, and TPS (Tile Parameter Set) for tile-level signaling. The slice data may include information regarding one or more slices. One slice according to embodiments is one geometry bitstream (Geom0 0 ) and one or more attribute bitstreams (Attr0 0 , Attr1 0 It may include ).

[0195] A slice refers to a series of syntax elements representing all or part of a coded point cloud frame.

[0196] According to the embodiments, the TPS may include information regarding each tile (e.g., coordinate value information of a bounding box and height / size information, etc.) for one or more tiles. The geometry bitstream may include a header and a payload. The header of the geometry bitstream according to the embodiments may include identification information of a parameter set included in the GPS (geom_parameter_set_id), a tile identifier (geom_tile_id), a slice identifier (geom_slice_id), and information regarding data included in the payload, etc. As described above, the metadata processing unit (8007) according to the embodiments may generate and / or process signaling information and transmit it to the transmission processing unit (8012). According to the embodiments, the elements performing geometry encoding and the elements performing attribute encoding may share data / information with each other as indicated by the dotted lines. The transmission processing unit (8012) according to the embodiments may perform an operation and / or transmission method identical or similar to the operation and / or transmission method of the transmitter (10003). A detailed explanation is omitted as it is the same as that described in FIGS. 1 and 2.

[0197] FIG. 9 is an example of a receiving device according to embodiments.

[0198] The receiving device illustrated in FIG. 9 is an example of the receiving device (10004) of FIG. 1 (or the point cloud decoder of FIG. 10 and FIG. 11). The receiving device illustrated in FIG. 9 can perform at least one of the same or similar operations and methods as the operations and decoding methods of the point cloud decoder described in FIG. 1 to FIG. 11.

[0199] A receiving device according to the embodiments may include a receiving unit (9000), a receiving processing unit (9001), an arithmetic decoder (9002), an occupancy code-based octree reconstruction processing unit (9003), a surface model processing unit (triangle reconstruction, up-sampling, voxelization) (9004), an inverse quantization processing unit (9005), a metadata parser (9006), an arithmetic decoder (9007), an inverse quantization processing unit (9008), a prediction / lifting / RAHT inverse transformation processing unit (9009), a color inverse transformation processing unit (9010), and / or a renderer (9011). Each component of the decoding according to the embodiments may perform the inverse process of the components of the encoding according to the embodiments.

[0200] A receiver (9000) according to the embodiments receives point cloud data. The receiver (9000) may perform an operation and / or a receiving method identical or similar to the operation and / or receiving method of the receiver (10005) of FIG. 1. A detailed description is omitted.

[0201] A receiving processing unit (9001) according to the embodiments can obtain a geometry bitstream and / or an attribute bitstream from the received data. The receiving processing unit (9001) may be included in the receiving unit (9000).

[0202] The arismetic decoder (9002), the Occupancy code-based octree reconstruction processing unit (9003), the surface model processing unit (9004), and the inverse quantization processing unit (9005) can perform geometry decoding. Since the geometry decoding according to the embodiments is identical or similar to the geometry decoding described in FIGS. 1 to 10, a detailed description is omitted.

[0203] The arismetic decoder (9002) according to the embodiments can decode a geometry bitstream based on arismetic coding. The arismetic decoder (9002) performs the same or similar operation and / or coding as the operation and / or coding of the arismetic decoder (7000).

[0204] According to the embodiments, the Occupancy code-based octree reconstruction processing unit (9003) can reconstruct an octree by obtaining an Occupancy code from a decoded geometry bitstream (or information regarding geometry obtained as a result of decoding). The Occupancy code-based octree reconstruction processing unit (9003) performs the same or similar operations and / or methods as the octree synthesis unit (7001) and / or octree generation method. According to the embodiments, the surface model processing unit (9004) can perform trisup geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, up-sampling, voxelization) based on the surface model method when trisup geometry encoding is applied. The surface model processing unit (9004) performs the same or similar operations as the surface offset synthesis unit (7002) and / or geometry reconstruction unit (7003).

[0205] The inverse quantization processing unit (9005) according to the embodiments can inverse quantize the decoded geometry.

[0206] A metadata parser (9006) according to the embodiments can parse metadata included in the received point cloud data, such as setting values, etc. The metadata parser (9006) can pass the metadata to geometry decoding and / or attribute decoding. A specific description of the metadata is omitted as it is the same as described in FIG. 8.

[0207] The arismetic decoder (9007), inverse quantization processing unit (9008), prediction / lifting / RAHT inverse transformation processing unit (9009), and color inverse transformation processing unit (9010) perform attribute decoding. Since attribute decoding is identical or similar to the attribute decoding described in FIGS. 1 to 10, a detailed description is omitted.

[0208] The arismetic decoder (9007) according to the embodiments can decode an attribute bitstream into arismetic coding. The arismetic decoder (9007) can perform decoding of the attribute bitstream based on reconstructed geometry. The arismetic decoder (9007) performs the same or similar operation and / or coding as the operation and / or coding of the arismetic decoder (7005).

[0209] The inverse quantization processing unit (9008) according to the embodiments can inverse quantize the decoded attribute bitstream. The inverse quantization processing unit (9008) performs the same or similar operation and / or method as the operation and / or inverse quantization method of the inverse quantization unit (7006).

[0210] According to the embodiments, the prediction / lifting / RAHT inverse transformation processing unit (9009) can process the reconstructed geometry and inverse quantized attributes. The prediction / lifting / RAHT inverse transformation processing unit (9009) performs at least one of the same or similar operations and / or decodings as the operations and / or decodings of the RAHT transformation unit (7007), LOD generation unit (7008), and / or inverse lifting unit (7009). According to the embodiments, the color inverse transformation processing unit (9010) performs inverse transformation coding to inversely transform the color values ​​(or textures) included in the decoded attributes. The color inverse transformation processing unit (9010) performs the same or similar operations and / or inverse transformation coding as the operations and / or inverse transformation coding of the color inverse transformation unit (7010). A renderer (9011) according to the embodiments can render point cloud data.

[0211] FIG. 10 shows an example of a structure that can be linked with a point cloud data transmission / reception method / device according to embodiments.

[0212] The structure of FIG. 10 represents a configuration in which at least one of a server (1060), a robot (1010), an autonomous vehicle (1020), an XR device (1030), a smartphone (1040), a home appliance (1050) and / or an HMD (1070) is connected to a cloud network (1010). The robot (1010), the autonomous vehicle (1020), the XR device (1030), the smartphone (1040), or the home appliance (1050) are referred to as devices. Additionally, the XR device (1030) may correspond to a point cloud data (PCC) device according to the embodiments or may be linked with a PCC device.

[0213] The cloud network (1000) may refer to a network that constitutes part of the cloud computing infrastructure or exists within the cloud computing infrastructure. Here, the cloud network (1000) may be configured using a 3G network, a 4G or LTE (Long Term Evolution) network, or a 5G network, etc.

[0214] The server (1060) is connected to at least one of a robot (1010), an autonomous vehicle (1020), an XR device (1030), a smartphone (1040), a home appliance (1050) and / or an HMD (1070) via a cloud network (1000) and can assist in at least some of the processing of the connected devices (1010 to 1070).

[0215] The HMD (Head-Mount Display) (1070) represents one of the types in which an XR device and / or PCC device according to the embodiments may be implemented. A device of the HMD type according to the embodiments includes a communication unit, a control unit, a memory unit, an I / O unit, a sensor unit, and a power supply unit, etc.

[0216] Hereinafter, various embodiments of the device (1010 to 1050) to which the above-described technology is applied are described. Here, the device (1010 to 1050) illustrated in FIG. 10 may be linked / coupled with a point cloud data transmission / reception device according to the above-described embodiments.

[0217] <PCC+XR>

[0218] The XR / PCC device (1030) may be implemented as a Head-Mount Display (HMD), a Head-Up Display (HUD) equipped in a vehicle, a television, a mobile phone, a smartphone, a computer, a wearable device, a home appliance, digital signage, a vehicle, a stationary robot, or a mobile robot by applying PCC and / or XR (AR+VR) technology.

[0219] The XR / PCC device (1030) can obtain information about surrounding space or real objects by analyzing 3D point cloud data or image data obtained through various sensors or from an external device to generate position data and attribute data for 3D points, and can render and output an XR object to be output. For example, the XR / PCC device (1030) can output an XR object containing additional information about a recognized object by associating it with the recognized object.

[0220] <PCC+XR+모바일폰>

[0221] The XR / PCC device (1030) can be implemented as a mobile phone (1040) or the like by applying PCC technology.

[0222] The mobile phone (1040) can decode and display point cloud content based on PCC technology.

[0223] <PCC+자율주행+XR>

[0224] The autonomous vehicle (1020) can be implemented as a mobile robot, vehicle, unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0225] An autonomous vehicle (1020) equipped with XR / PCC technology may refer to an autonomous vehicle equipped with means for providing XR images, or an autonomous vehicle that is the subject of control / interaction within the XR images. In particular, the autonomous vehicle (1020) that is the subject of control / interaction within the XR images is distinguished from the XR device (1030) and can be interconnected with it.

[0226] An autonomous vehicle (1020) equipped with means for providing XR / PCC images can acquire sensor information from sensors including cameras and output XR / PCC images generated based on the acquired sensor information. For example, the autonomous vehicle (1020) can provide an XR / PCC object corresponding to a real object or an object in the screen to the occupant by providing an XR / PCC object by outputting an XR / PCC image with a HUD.

[0227] At this time, when the XR / PCC object is displayed on the HUD, at least a portion of the XR / PCC object may be displayed so as to overlap with the actual object to which the occupant's gaze is directed. On the other hand, when the XR / PCC object is displayed on a display provided inside the autonomous vehicle, at least a portion of the XR / PCC object may be displayed so as to overlap with an object on the screen. For example, the autonomous vehicle (1220) may display XR / PCC objects corresponding to objects such as lanes, other vehicles, traffic lights, traffic signs, motorcycles, pedestrians, buildings, etc.

[0228] VR (Virtual Reality) technology, AR (Augmented Reality) technology, MR (Mixed Reality) technology and / or PCC (Point Cloud Compression) technology according to the embodiments can be applied to various devices.

[0229] In other words, VR technology is a display technology that provides real-world objects or backgrounds solely as CG images. On the other hand, AR technology refers to a technology that displays virtual CG images alongside images of real objects. Furthermore, MR technology is similar to the aforementioned AR technology in that it mixes and combines virtual objects with the real world. However, it is distinguished from AR technology in that while AR technology maintains a clear distinction between real-world objects and virtual objects created from CG images, using virtual objects to complement real-world objects, MR technology regards virtual objects as having the same nature as real-world objects. To give a more specific example, the aforementioned MR technology is applied in hologram services.

[0230] However, recently, rather than clearly distinguishing between VR, AR, and MR technologies, they are also referred to as XR (extended Reality) technology. Therefore, embodiments of the present invention are applicable to all VR, AR, MR, and XR technologies. These technologies may utilize encoding / decoding based on PCC, V-PCC, and G-PCC technologies.

[0231] The PCC method / device according to the embodiments can be applied to a vehicle providing autonomous driving services.

[0232] Vehicles providing autonomous driving services are connected to PCC devices to enable wired / wireless communication.

[0233] When a point cloud data (PCC) transceiver according to the embodiments is connected to a vehicle for wired or wireless communication, it can receive and process content data related to AR / VR / PCC services that can be provided along with an autonomous driving service, and transmit it to the vehicle. Additionally, when the point cloud data transceiver is mounted on a vehicle, the point cloud transceiver can receive and process content data related to AR / VR / PCC services according to a user input signal received through a user interface device and provide it to the user. A vehicle or a user interface device according to the embodiments can receive a user input signal. The user input signal according to the embodiments may include a signal indicating an autonomous driving service.

[0234] The encoding method and apparatus according to the embodiments may include and perform the transmission device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 10, encoding of FIG. 11 to 19, transmission device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.

[0235] The decoding method and apparatus according to the embodiments may include and perform the following: a receiving device (10004) of FIG. 1, a receiving device (20002) to a rendering device (20004) of FIG. 2, a decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, a receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.

[0236] The encoding / decoding method and apparatus according to the embodiments may be referred to simply as the method and apparatus according to the embodiments. The method and apparatus according to the embodiments may include and perform a nearest neighbor search range management method for faster LoD generation.

[0237] The present embodiments relate to a method for improving attribute speed in Geometry-based Point Cloud Compression (G-PCC) for 3D point cloud data compression, and propose a nearest neighbor search area setting management method to reduce the time required for the encoding / decoding process of attribute information when generating a Level of Detail (hereinafter referred to as LOD) during the G-PCC attribute encoding and decoding process. The present embodiments include proposals for a nearest neighbor search area setting management method and a signaling method.

[0238] The present embodiments relate to a method for increasing the compression speed of Geometry-based Point Cloud Compression (G-PCC) for compressing three-dimensional point cloud data. Hereinafter, the encoder and encoder will be referred to as encoders, and the decoder and decoder will be referred to as decoders. A point cloud is composed of a set of points, and each point may have geometry information and attribute information, the geometry information is three-dimensional position (XYZ) information, and the attribute information is color (RGB, YUV, etc.) or / and reflection values.

[0239] The G-PCC encoding process can be composed of dividing a point cloud into tiles according to region, dividing each tile into slices for parallel processing, compressing geometry on a slice-by-slice basis, and compressing attribute information based on reconstructed geometry (decoded geometry) using location information changed through compression. The G-PCC decoding process can be composed of receiving a geometry bitstream and an attribute bitstream on an encoded slice-by-slice basis, decoding the geometry, and decoding attribute information based on the reconstructed geometry through the decoding process.

[0240] For geometry information compression, octree-based, predictive tree-based, or trisoup-based compression techniques may be used, and for attribute information compression, predicting transform-based, lifting transform-based, or RAHT transform-based compression techniques may be used. The present embodiments relate to a technique for obtaining an improvement in encoding / decoding speed by generating Level of Detail (LOD) through an appropriate nearest neighbor search order in the generation of LOD for lifting transform used for attribute information compression of point cloud content captured by LiDAR, RGB-D camera, or LiDAR equipment.

[0241] FIG. 11 shows an example of an atlas search according to embodiments.

[0242] The encoding method and apparatus according to the embodiments (transmitting device (10000) in FIG. 1, acquisition (20000) to transmission (20002) in FIG. 2, FIG. 3, FIG. 8 encoder, each device in FIG. 10, encoding in FIG. 11 to 19, transmitting device in FIG. 20 to 21, encoding in FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation in FIG. 28 to 30, encoding in FIG. 31 to 32, encoding in FIG. 33, parameter generation in FIG. 34, method in FIG. 35, etc.) can perform encoding of attribute data by generating a LoD as in FIG. 11 and predicting attribute data.

[0243] The decoding method and apparatus according to the embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform decoding of attribute data by generating a LoD as in FIG. 11 and predicting attribute data.

[0244] An atlas according to the embodiments may refer to a spatial region in which point cloud data (points) are distributed. An atlas search according to the embodiments may mean searching a region for point(s) for attribute prediction coding.

[0245] The LOD generation method may be (i) a distance-based method that classifies LODs based on the distance between points, (ii) a sampling-based method that sorts points using Morton codes and classifies every X-th corresponding point as a sub-LOD that can be a candidate for the neighbor set, or (iii) a method that constructs an octree for the points and selects points close to the center based on the node level of the octree to classify them as sub-LODs.

[0246] Points sampled by any one of the three methods described above search for neighboring points within their respective LODs to generate a predictor, continuing the search until three nearest neighbor points are found. Search methods for finding nearest neighbor points include Atlas search, which specifies an area surrounding each point and searches only the points within that area, and Full search, which searches all surrounding points. These two searches can be performed sequentially, with Atlas search executed first and Full search proceeding only if three nearest neighbor points are not found during the Atlas search. In this case, since the Atlas search prioritizes searching a narrow area that is close in three dimensions, the Full search process can be omitted if three nearest neighbor points are found during the Atlas search phase, thereby reducing the time required for nearest neighbor search and LOD generation.

[0247] The above process can be performed in both intra-prediction and inter-prediction, and inter-prediction can be distinguished from intra-prediction in that the neighbor search is performed in a reference frame rather than the current frame.

[0248] However, atlas search has a problem in that when the density of the point cloud is high, an excessive number of points are distributed within the atlas search area, which can significantly prolong the search time. In particular, in the case of inter-atlas search, unlike intra-atlas search, the search may be performed in unsampled, i.e., high-density areas, which can further increase the search time. Therefore, in these embodiments, to achieve fast and accurate encoding / decoding in point cloud compression, we aim to increase compression speed by compensating for the shortcomings of the nearest neighbor search process, and to support a method for accelerating the encoding / decoding process through a technique that sets the intra / inter-atlas search areas according to the characteristics of the content.

[0249] FIG. 11 illustrates an example of an atlas search. In these embodiments, a predefined atlas search range is established around a reference point trying to find nearest neighbors, and a neighbor search can be performed targeting only candidate points included within the atlas search range. For example, the atlas search range can be set as a certain range within a three-dimensional space including the reference point, and can be represented in a form divided into a plurality of grids / sub-regions as shown in FIG. 11. Accordingly, the reference point can select the nearest neighbors based on distance criteria among the points included in the atlas search range, and the search range can be limited by excluding points located outside the atlas search range from the search target.

[0250] Example 1

[0251] The embodiments may be applied by modifying and / or combining with one another. The terms used in this document have their common meanings widely used in the art and may be understood based on their intended meanings unless otherwise defined in the context. Additionally, the improved nearest neighbor search method may be performed and applied in both the Point Cloud Compression (PCC) encoder and / or decoder.

[0252] The encoding / decoding method according to the embodiments may include an intra-atlas search method (see FIG. 12) and / or an inter-atlas search method (see FIG. 13). The inter-atlas search method may include a method for selecting an atlas search reference point in a reference frame (see FIG. 14 to 15), and / or a Morton code scaling-based atlas search method.

[0253] The encoding / decoding method according to the embodiments may include an atlas search area setting method. The atlas search area setting method may include a center-based atlas search area setting method (see FIG. 16), a planar atlas search area setting method (see FIG. 17), a vertical atlas search area setting method, an index-based atlas search area setting method (see FIG. 18), and an extended atlas search area setting method (see FIG. 19).

[0254] FIG. 12 shows Morton codes in a 2D plane according to embodiments.

[0255] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, bitstream and parameter generation of FIG. 3, encoder of FIG. 8, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by searching a region (atlas) based on a Morton code as in FIG. 12 and predicting attribute data.

[0256] The decoding method and apparatus according to the embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform decoding of attribute data by searching a region (atlas) based on a Morton code as in FIG. 12 and predicting attribute data.

[0257] Intra-atlas search may be a method of searching by establishing an area containing a reference point (the gray area in FIG. 11) based on a reference point to find the nearest neighbor point, as shown in the example of FIG. 11, and expanding an area of ​​the same size as the reference point in all directions. In these embodiments, when the area of ​​the size to which the reference point belongs is referred to as a mini-cube, a total of 27 mini-cubes, including the mini-cube to which the reference point belongs, may be included in the search target, and the area defined by these 27 mini-cubes may be referred to as the atlas search range. Generally, when the size of the mini-cube is set to 2x2x2, a maximum of 8 points may be included within the mini-cube, and in this case, the size of the atlas search range may be defined as 6x6x6. Additionally, when the size of the mini-cube is 2Nx2Nx2N, the size of the atlas search range may be defined as 6Nx6Nx6N.

[0258] When establishing an atlas search area, the Morton code possessed by each point can be utilized. As illustrated in FIG. 12, the Morton code is one of the space filling curves. Generally, in attribute compression of a G-PCC encoder / decoder, a Morton code is assigned using the 3D coordinates of each point, and attribute encoding / decoding can be performed by aligning the points in the order of the Morton codes. Based on such Morton codes, the relative positions and directions of surrounding points can be determined, thereby establishing an atlas search area and including points belonging to said area as targets for atlas search. Additionally, there may be no points within the atlas search area.

[0259] In atlas search, the criterion for determining the nearest neighbor point can be distance. For example, if the current point's coordinates are (x, y, z) and the coordinates of the searched neighbor point are In this case, the distance D between two points can be calculated according to Equation 1. The point with the smallest distance D, i.e., the point closest to the current point, can be determined as the nearest neighbor point, and in atlas search, three nearest neighbor points can be searched within the atlas search area. However, if there are fewer than three points within the atlas search area, three nearest neighbor points cannot be secured, so the search can then be performed by switching to the full search step.

[0260] The distance calculation formula is as follows:

[0261]

[0262] FIG. 12 illustrates an example of a Morton code in a two-dimensional plane. As shown in FIG. 12, a Morton code can be sequentially assigned to each grid position (coordinate) in a grid space defined by the x-axis and y-axis, and the Morton code can be defined to visit adjacent positions in a specific order along the path of a space filling curve. The Morton code assigned in this way can be used as a criterion for determining the relative position and direction of each point, and can be used for searching for surrounding points and setting search areas by aligning points according to the order of the Morton code.

[0263] FIG. 13 shows an example of inter-atlas search according to embodiments.

[0264] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, bitstream and parameter generation of FIG. 3, encoder of FIG. 8, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by searching a region (atlas) based on inter-atlas search as in FIG. 13 and predicting attribute data.

[0265] The decoding method and apparatus according to the embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform decoding of attribute data by searching a region (atlas) based on inter-atlas search as in FIG. 13 and predicting attribute data.

[0266] When performing inter-prediction-based point cloud compression for multi-frame point cloud compression, inter-prediction can be utilized in atlas search. For example, the atlas search area described in the 'Intra-Atlas Search Method' can be set in the same way based on a reference frame reference point as shown in Fig. 13, and subsequently, points existing in the reference frame can be used as neighbor points to generate a predictor for points in the current frame. At this time, the reference frame reference point can be set through the method presented in the 'Method for Selecting an Atlas Search Reference Point in a Reference Frame', and the atlas search process can be performed in the same way as the 'Intra-Atlas Search Method'. In addition, the atlas search method performed in the reference frame and the search area set by it can be referred to as the Inter-Atlas Search Method and the Inter-Atlas Search Range, respectively.

[0267] FIG. 13 illustrates an example of an inter-atlas search. In the present embodiments, to generate a predictor for a reference point (point trying to find nearest neighbors from reference frame) included in the current frame, a reference point can be set in the reference frame, and an inter-atlas search range can be set centered on the reference point. As shown in FIG. 13, points included within the inter-atlas search range of the reference frame can be included as search targets as neighbor point candidates, and the nearest neighbor point to the reference point can be selected from among the search targets. Accordingly, the reference point of the current frame can perform a neighbor point search using the points included in the inter-atlas search range set in the reference frame.

[0268] FIG. 14 shows an example of choosing same-index reference point for atlas search in 2D plane according to embodiments.

[0269] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, FIG. 8, encoder, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by predicting attribute data by selecting an atlas search reference point in a reference frame as in FIG. 14.

[0270] The decoding method and apparatus according to the embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform decoding of attribute data by predicting attribute data by selecting an atlas search reference point in a reference frame as in FIG. 14.

[0271] When performing an Inter Atlas search, a reference point is required to establish the Inter Atlas search range. However, since a point in the reference frame may not exist at an exact location corresponding to a point in the current frame, a method is required to select a reference point in the reference frame that will serve as the basis for setting the search range. Furthermore, because the current frame and the reference frame may have different indices referring to points in similar locations due to factors such as different alignment methods, a method for selecting a reference point is needed that can determine the reference point to be referenced even in situations of index mismatch.

[0272] FIG. 14 illustrates an example of a method for selecting an atlas search reference point in a reference frame using the same index in a 2D plane. In the present embodiments, in order to determine a reference point in a reference frame corresponding to a current point in a current frame, a point having the same index as the index of the current point in the current frame can be selected in the reference frame and set as the chosen point. As shown in FIG. 14, when a current point in a current frame is identified by a specific index, a reference point in the reference frame for the current point in the current frame can be determined by selecting a point that is identified by the same index in the reference frame.

[0273] As shown in FIG. 14, it may be effective to set a point in a reference frame that has the same index as a point in the current frame as a reference point. For example, when using the same index, the reference point in the reference frame corresponding to a point in the current frame can be determined without a separate search process, so the amount of computation involved in distance calculations for determining the reference point is reduced, which may be advantageous in terms of encoding / decoding speed. In addition, when global motion compensation is applied to the reference frame, the positions of the points in the reference frame may become similar to those in the current frame, so selecting a point with the same index as a reference point may be advantageous for performing atlas search in the reference frame.

[0274] FIG. 15 shows an example of choosing the closest reference point for atlas search in a 2D plane according to embodiments.

[0275] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, FIG. 8, encoder, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by predicting attribute data by selecting an atlas search reference point in a reference frame as in FIG. 15.

[0276] The decoding method and apparatus according to the embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform decoding of attribute data by predicting attribute data by selecting an atlas search reference point in a reference frame as in FIG. 15.

[0277] As shown in Fig. 15, it may be effective to set the point in the reference frame closest to the point in the current frame as the reference point. For example, if a point in the reference frame that is spatially close to the point in the current frame is selected as the reference point, it may be advantageous in terms of the accuracy of the reference atlas search area set around said reference point. In addition, since the characteristics between points with the same index in the current frame and the reference frame may differ when global motion compensation is not applied to the reference frame, it may be advantageous to select the point in the reference frame closest to the point in the current frame as the reference point through distance calculation.

[0278] FIG. 15 illustrates an example of a method for selecting the spatially closest point in a reference frame as the chosen point for the current point of the current frame in a 2D plane. In the present embodiments, to determine the chosen point of the reference frame for the current point, the distance from the current point among multiple points in the reference frame is calculated, and based on the result of the distance calculation, the point with the minimum distance from the current point can be selected as the chosen point. As shown in FIG. 15, the chosen point in the reference frame can subsequently be used as a criterion for setting the reference atlas search area and can be utilized as a reference point in the process of performing neighbor point search in the reference frame to generate a predictor for the current point.

[0279] In an atlas search method based on Morton code scaling, it may be efficient to apply down-scaling to location information when calculating Morton codes. For example, when assigning a Morton code to a point with coordinates (x, y, z), if the Morton code scaling factor is N, the Morton code can be assigned after down-scaling the point's coordinates by N. When the function for assigning Morton codes is expressed as mortonAddr(), the Morton code assigned through the above Morton code scaling technique can be expressed as Equation 2. In this case, the coordinate values ​​representing the actual locations of the points are not changed; rather, down-scaled coordinates are used during the process of assigning Morton codes. In the equation below, N corresponds to the numerator and d corresponds to the exponent of 2 in the denominator; this can be understood as integer down-scaling without decimal points. Down-scaling can be performed to have the same effect as multiplying the coordinate values ​​by N / 2^d and discarding the decimal point.

[0280] The Morton code scaling technique is as follows:

[0281]

[0282] As described in 'Intra-Atlas Search Method', the atlas search area can be set based on Morton codes, and when Morton code scaling is applied, the number of points included within the atlas search area may increase. If more points are included within the atlas search area, the number of nearest neighbor candidates increases during the atlas search, which increases the likelihood of securing three nearest neighbors without switching to the full search step. Accordingly, the total encoding / decoding time can be reduced by avoiding the time-consuming full search process. Furthermore, it can compensate for the problem of selecting inaccurate neighbors when there are insufficient nearest neighbor candidates, and as a result, the accuracy of the predictor generated thereafter is improved, thereby enhancing the performance of attribute compression.

[0283] Atlas search offers the advantage of fast search speeds by rapidly searching for neighboring points within a narrow area, and this applies to both intra-atlas search and inter-atlas search. However, in the case of high-density point clouds, an excessive number of points may be distributed within the atlas search area even if it is narrow. In particular, since inter-atlas search is performed on unsampled reference frames, points may be more densely distributed within the inter-atlas search area, potentially prolonging the search time. Conversely, in the case of low-density point clouds, no points may exist within the atlas search area. In this scenario, failing to find appropriate neighboring points during atlas search necessitates a transition to the next step, full search, which can also increase the time required for full search. Therefore, when performing intra / inter-atlas search for LOD generation, it may be efficient to adaptively modify the search area by considering the characteristics of the point cloud.

[0284] In the following, a method for setting an atlas search area within the encoding / decoding of attribute data of point cloud data is described. The method for setting an atlas search area may include a center-based atlas search area setting method, a planar atlas search area setting method, a vertical atlas search area setting method, an index-based atlas search area setting method, and an extended atlas search area setting method.

[0285] FIG. 16 illustrates a center-based atlas search range management method according to embodiments.

[0286] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, FIG. 8, encoder, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by predicting attribute data by setting a center-based atlas search area as in FIG. 16.

[0287] Decoding method and apparatus according to embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can be performed by predicting attribute data by setting a center-based atlas search area as in FIG. 16.

[0288] In the center-based atlas search area setting method, it may be effective to set only the one central minicube among the 27 minicubes as the atlas search area as shown in FIG. 16 by changing the atlas search area. This setting may be advantageous for reducing search time by limiting the number of points included within the atlas search area when the density of the point cloud is high, and accordingly, it may be efficient to determine whether to apply the technology based on the density of the point cloud.

[0289] In these embodiments, the user may signal the decoder to use the center-based atlas search area setting technique in advance, and may signal differently for intra-atlas search and inter-atlas search, respectively. Additionally, the encoder may determine whether to use the center-based atlas search area setting technique by comparing the density of the input point cloud content with a preset threshold, and signal the result to the decoder. Here, the criterion for determining density may be the number of points within a specific area, or the average of the distances between all points.

[0290] To maximize the efficiency of the center-based atlas search area setting method, a Morton code scaling-based atlas search method can be used in conjunction. For example, by applying Morton code scaling to a low-density point cloud to include accurate neighbor point candidates within the atlas search area, and then searching only for key central neighbor points according to the center-based atlas search area setting, accurate and high-speed neighbor point search can be performed.

[0291] FIG. 16 illustrates an example of a center-based atlas-search range management method. As shown in FIG. 16, in conventional atlas search, an area consisting of a total of 27 mini-cubes in all directions centered around a mini-cube containing a reference point can be set as the original atlas search range. On the other hand, in the center-based atlas search range management method according to the present embodiments, only one mini-cube located in the center among the 27 mini-cubes is set as the modified atlas search range, thereby limiting the search targets to points included in the central mini-cube. Accordingly, the atlas search range can be managed to perform neighbor point search only on the core area around the reference point.

[0292] FIG. 17 illustrates a planar atlas search range management method according to embodiments.

[0293] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, FIG. 8, encoder, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by predicting attribute data by setting a planar atlas search area as in FIG. 17.

[0294] A decoding method and apparatus according to embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform encoding of attribute data by predicting attribute data by setting a planar atlas search area as in FIG. 17.

[0295] In the method for setting a planar atlas search area, as shown in FIG. 17, it may be effective to set only the 9 minicubes horizontally positioned in the center among the 27 minicubes as the atlas search area by changing the atlas search area. This setting can function efficiently because it can reflect the characteristic that the atlas search targets are distributed along the plane when the shape of the point cloud is planar, and accordingly, a method for determining whether to apply the above technique based on the direction of the normal vector of the current point may be efficient.

[0296] In these embodiments, the user may signal the decoder to use the planar atlas search area setting technique in advance, and may signal differently for intra-atlas search and inter-atlas search, respectively. Additionally, the encoder may calculate the magnitude of the angle with respect to the Z-axis using the normal vector of the input point, compare the magnitude of the angle with a preset threshold to determine whether to use the planar atlas search area setting technique, and then signal the result of the determination to the decoder.

[0297] FIG. 17 illustrates an example of a planar atlas-search range management method. As shown in FIG. 17, in conventional atlas search, an area consisting of a total of 27 mini-cubes in all directions centered around a mini-cube containing a reference point can be set as the original atlas search range. On the other hand, in the planar atlas search range management method according to the present embodiments, only a planar area consisting of 9 mini-cubes arranged horizontally in the center among the 27 mini-cubes is set as the modified atlas search range, thereby limiting the search targets to points included in the planar area. Accordingly, when a point cloud is distributed in a planar manner, the atlas search range can be managed by reflecting the distribution characteristics.

[0298] In the method for setting a vertical atlas search area, it may be effective to set the atlas search area to nine minicubes, including the central minicube, among the 27 minicubes arranged in a direction perpendicular to a specific axis. For example, if nine minicubes are selected in a direction perpendicular to the Z-axis, it may be the same as the method for setting a planar atlas search area (see 4.1.2.2), and the method for setting a vertical atlas search area can be effective when the point cloud shows a tendency to be perpendicular to the X-axis or Y-axis. Additionally, a method for determining whether to apply the above technique based on the direction of the normal vector of the current point may be efficient.

[0299] In these embodiments, the user may signal the decoder to use the vertical atlas search area setting technique in advance, and when the user signals, may also signal whether to perform the technique for the X-axis or Y-axis. At this time, different signals may be given for intra-atlas search and inter-atlas search, respectively. Additionally, the encoder may calculate the magnitude of the angle with respect to the X-axis (or Y-axis) using the normal vector of the input point, compare the magnitude of the angle with a preset threshold to determine whether to use the vertical atlas search area setting technique, and then signal the result of the determination to the decoder.

[0300] FIG. 18 shows a 2D example of index-based atlas search range management according to embodiments.

[0301] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, FIG. 8, encoder, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by predicting attribute data by setting an index-based atlas search area as in FIG. 18.

[0302] A decoding method and apparatus according to embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform encoding of attribute data by predicting attribute data by setting an index-based atlas search area as in FIG. 18.

[0303] In the index-based atlas search area setting method, it may be effective to determine which minicubes to include in the search area by using the index of the minicubes as shown in FIG. 18 by changing the atlas search area. In the embodiments, the index order of the minicubes can be set in advance through a look-up table (LUT), and the LUT may be information that is commonly known by the encoder and the decoder without separate transmission and reception.

[0304] The user can signal to the decoder in advance an array of indices of minicubes to be included within the atlas search area, and can signal differently for intra-atlas search and inter-atlas search, respectively. Meanwhile, the index-based atlas search area setting method can be used in conjunction with the center-based atlas search area setting method (4.1.2.1), the planar atlas search area setting method (4.1.2.2), or the vertical atlas search area setting method (4.1.2.3), thereby reducing the amount of information being signaled.

[0305] FIG. 18 illustrates a two-dimensional example of an index-based atlas-search range management method. In these embodiments, indices (Index 0 to Index 8) may be assigned to a plurality of minicubes, including a central minicube containing a reference point, and the atlas-search range may be defined by the minicubes corresponding to a selected index among the indices. As shown in FIG. 18, for example, by including minicubes corresponding to a specific index (Index 0, Index 4, Index 5, etc.) in the atlas-search range and excluding minicubes corresponding to other indices from the search target, the search range can be selectively set on an index basis. Accordingly, atlas-search can be performed on points included within the minicubes corresponding to the selected index.

[0306] FIG. 19 shows a 2D example of enlarged atlas search range management according to embodiments.

[0307] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, FIG. 8, encoder, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by predicting attribute data by setting an extended atlas search area as in FIG. 19.

[0308] Decoding method and apparatus according to embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can be performed by predicting attribute data by setting an extended atlas search area as in FIG. 19.

[0309] In the method for setting an expanded atlas search area, it may be effective to expand the atlas search area by including additional mini-cubes in addition to the existing 27 mini-cubes by changing the atlas search area. A 2D example of setting an expanded atlas search area is shown in FIG. 19, and in the case of 3D, the number of mini-cubes to be searched can be expanded from the existing 27 to 125. In addition, when the density of the point cloud is very low, it may be efficient to perform the expansion at least one step, and when the expansion step is denoted as N, it may be effective to set a total of (3+2N)^3 mini-cubes as search targets based on the central mini-cube.

[0310] In these embodiments, the user may signal the decoder in advance to use the extended atlas search area setting technique, and when the technique is used, the expansion step may also be signaled. At this time, different signals may be given for intra-atlas search and inter-atlas search, respectively. Additionally, the encoder may determine whether to use the extended atlas search area setting technique by reflecting the density of the input point cloud content and signal the result of that determination to the decoder, and may also signal the expansion step to the decoder after the encoder determines it. Here, the criterion for determining density may be the number of points within a specific area or the average of the distances between all points.

[0311] FIG. 19 is a diagram illustrating a two-dimensional example of an enlarged / extended atlas-search range management method. As shown in FIG. 19, in conventional atlas search, a search range can be set with a limited number of minicubes centered on an original atlas search range that includes a reference point trying to find nearest neighbors. On the other hand, in the enlarged atlas search range management method according to the present embodiments, an extended atlas search range can be set by additionally including surrounding minicubes in the original atlas search range. Accordingly, the extended atlas search range includes a wider range of minicubes than the original atlas search range, and neighbor point search for the reference point can be performed on points included within the extended atlas search range.

[0312] FIG. 20 shows an encoder according to embodiments.

[0313] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, encoder of FIG. 3 and FIG. 8, each device of FIG. 10, encoding of FIG. 11 to FIG. 19, transmitting device of FIG. 20 to FIG. 21, encoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to FIG. 30, encoding of FIG. 31 to FIG. 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) may be configured as in FIG. 20.

[0314] A VPCC data encoder is configured to perform encoding of PCC data and may have a block diagram as illustrated in the drawing. Each component may be implemented in hardware, software, a processor, and / or a combination thereof. PCC data is input to the encoder, and as the PCC data is encoded, a geometry information bitstream and an attribute information bitstream may be output.

[0315] The point cloud data encoding device (or encoder) illustrated in the drawing may include a data input unit, a coordinate system conversion unit, a geometry information conversion quantization processing unit, a spatial partitioning unit, a geometry encoding unit, a color conversion processing unit, a color re-adjustment unit, an attribute information encoding unit, and a reference frame generation unit.

[0316] The data input unit can read input data (e.g., PLY, configuration file, etc.) and configure setting parameters required for encoding (e.g., coordinate system type, scale value, division unit, etc.). The coordinate system transformation unit can perform coordinate transformations to support coordinate system changes of the point cloud, such as changing the order of the xyz axes or converting an xyz Cartesian coordinate system to a spherical coordinate system.

[0317] The geometry information conversion quantization processing unit can adjust geometry position values ​​(e.g., enlarge / reduce) by multiplying the geometry position components x, y, and z of a point cloud point by a scale according to a scale setting corresponding to the geometry quantization value. The spatial partitioning unit can divide the point cloud (or geometry data) into tiles or slices to support region-by-region access or parallel processing of the content. The geometry encoding unit can generate a geometry bitstream by encoding the spatially partitioned geometry information.

[0318] The color conversion processing unit can perform color conversions, such as converting the RGB color space to the YUV color space, to support attribute type conversion. The color recalibration unit can predict and / or correct (recalibrate) attribute values ​​suitable for the changed geometry position so that when a scale is applied to the geometry and the position information value is changed, the attribute value corresponding to the changed position is naturally maintained.

[0319] The attribute information encoding unit receives the original attribute information that has been recolored, the restored geometry information, and the reference frame as input, and can generate an attribute information bitstream by encoding the attribute information using these. The reference frame generation unit stores the restored geometry and the restored attribute information in a reference frame buffer and, if necessary, provides the reference frame data from the reference frame to another module so that it can be used as reference information in the geometry / attribute encoding of subsequent frames (or subsequent tiles / slices).

[0320] FIG. 20 is a functional block diagram of a point cloud data encoding device (or encoder). In embodiments, the data input unit receives input data including geometry data, attribute data and parameters, and can perform loading and setting in a form available for use in subsequent modules.

[0321] The coordinate system transformation unit can transform the coordinate representation of the input geometry (e.g., changing the xyz axes, transforming from a Cartesian coordinate system to a spherical coordinate system, etc.), and the geometry information transformation quantization processing unit can perform position control based on geometry normalization / quantization by applying a scale to the position components (x, y, z) of a point according to a set scale (geometry quantization value).

[0322] The spatial partitioning unit can divide the geometry and / or attribute source data into tiles or slices for parallel processing or area-by-area access.

[0323] The geometry information encoding unit can encode the divided geometry information to generate a geometry information bitstream, and also output the geometry information restored during the encoding process to provide for subsequent attribute processing.

[0324] The divided attribute original data can be provided to the recoloring unit after undergoing color space conversion (e.g., RGB*?*UV, etc.) in the color conversion processing unit, and the recoloring unit can improve the spatial consistency of the attributes by predicting or correcting attribute values ​​corresponding to the changed positions, taking into account the situation where position information is scaled / converted by the restored geometry information.

[0325] The attribute information encoding unit can encode attribute information using color-restored attribute information, restored geometry information, and a reference frame as inputs, and generate an attribute information bitstream.

[0326] A reference frame buffer stores restored geometry information and restored attribute information, and a reference frame generation unit generates a reference frame using the restored information stored in the buffer and provides it to an attribute information encoding unit, thereby improving attribute encoding efficiency based on temporal / spatial prediction.

[0327] FIG. 21 shows the encoding of attribute data according to embodiments.

[0328] FIG. 21 further illustrates the attribute data encoding of the encoding step of FIG. 20.

[0329] FIG. 21 is a block diagram of an attribute information encoding unit. Each component may correspond to hardware, software, a processor, and / or a combination thereof.

[0330] The attribute information encoding unit receives segmented point cloud data as input and can encode attribute information by selectively performing a Level of Detail (LOD) based transformation or a Region Adaptive Hierarchical Transform (RAHT) based transformation.

[0331] The LOD generation unit can generate LODs by receiving segmented point cloud data as input, and point-specific neighbors used in lifting transformations and predictive transformations can be determined. In this case, the LOD generation unit can be executed when LOD parameters are present. Additionally, when point-specific neighbors are determined by the LOD generation unit, an attribute reference frame can be received as input to utilize points within the attribute reference frame for more efficient neighbor search.

[0332] The lifting transform unit can generate transform coefficients by performing a frequency transform on attribute information through a prediction and update process for each LOD level using the generated LOD. The generated transform coefficients can be quantized and transmitted to the attribute information entropy encoding unit. Additionally, the lifting transform unit can output restored attribute information by internally performing an inverse transform.

[0333] A predictive transform can be performed to predict the current point using neighboring points determined during the LOD generation process. The difference between the predicted attribute information and the original attribute information can be used as a transform coefficient, which can be quantized and transmitted to the attribute information entropy encoding unit. Additionally, attribute information can be restored and output through inverse quantization and inverse predictive transform.

[0334] RAHT can be performed when there are no LOD parameters. RAHT can receive segmented point cloud data as input and perform RAHT (Region Adaptive Hierarchical Transform) to convert it into the frequency domain and generate transformation coefficients, and then the transformation coefficients can be quantized and transmitted to the attribute information entropy encoding unit.

[0335] FIG. 22 shows a decoder according to embodiments.

[0336] A decoding method and apparatus according to embodiments (a receiving device (10004) in FIG. 1, a receiving (20002) to a rendering (20004) in FIG. 2, a decoder in FIG. 7 and FIG. 9, each device in FIG. 10, decoding in FIG. 11 to FIG. 19, a receiving device in FIG. 22 to FIG. 23, decoding in FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition in FIG. 28 to FIG. 30, decoding in FIG. 31 to FIG. 32, decoding in FIG. 33, parameter acquisition in FIG. 34, method in FIG. 36, etc.) may be configured with FIG. 22.

[0337] FIG. 22 is a block diagram of a PCC data decoder. Each component may correspond to hardware, software, a processor, and / or a combination thereof. An encoded geometry information bitstream and an attribute information bitstream may be input to the decoder, and said bitstreams may be decoded and output as restored PCC data.

[0338] A PCC data decoder may be configured to receive a geometry information bitstream and an attribute information bitstream as inputs and decode them to output restored PCC data. To this end, it may include a geometry information decoder, a coordinate system inverse transformation unit, a geometry information transformation inverse quantization processing unit, an attribute residual information entropy decoder, an attribute information decoder, a color inverse transformation processing unit, and a reference frame generation unit.

[0339] The geometry information decoding unit can restore geometry information by receiving and decoding a geometry information bitstream. The coordinate system inverse transformation unit can restore the changed xyz axes or inversely transform the transformed coordinate system into an xyz orthogonal coordinate system. The geometry information transformation inverse quantization processing unit can restore geometry position values ​​by restoring the signaled scale (scale = geometry quantization value) and applying it to the geometry position components x, y, and z of the restored point.

[0340] The attribute residual information entropy decoding unit can entropy decode the attribute bitstream. The attribute information decoding unit can receive the attribute information bitstream as input, decode it, and restore the attribute information. The color inverse conversion processing unit can restore the converted attributes to RGB colors. The reference frame generation unit can store the restored geometry and restored attribute information in a reference frame buffer and transmit the reference frame data from the reference frame to another module.

[0341] FIG. 22 is a block diagram according to one embodiment of a PCC data decoder, illustrating a configuration in which geometry information bitstream and attribute information bitstream are each decoded to generate restored geometry information and restored attribute information, and restored PCC data is output using the same.

[0342] The geometry information decoding unit can generate restored geometry information by receiving a geometry information bitstream and performing decoding. The coordinate system inverse transformation unit can output geometry information by performing coordinate system inverse transformations on the geometry information as a result of decoding, such as restoring the changed xyz axes or inversely transforming the transformed coordinate system into an xyz orthogonal coordinate system.

[0343] The attribute information decoding unit can generate restored attribute information by receiving an attribute information bitstream and performing decoding, and the color inverse conversion processing unit can output attribute information by performing a color inverse conversion to restore the restored attribute information to RGB colors, etc.

[0344] A reference frame buffer can store restored geometry information and restored attribute information, and a reference frame generation unit can generate a reference frame based on the information stored in the reference frame buffer and provide it to an attribute information decoding unit. Accordingly, the attribute information decoding unit can use the reference frame to restore attribute information, and the restored attribute information can be stored back in the reference frame buffer and used as reference information in subsequent decoding.

[0345] FIG. 23 illustrates the decoding of attribute data according to embodiments.

[0346] FIG. 23 further illustrates the decoding of attribute data of FIG. 22.

[0347] This is a block diagram of the attribute information decoding unit. Each component may correspond to hardware, software, a processor, and / or a combination thereof.

[0348] The attribute information decoding unit may be configured to receive an attribute information bitstream and restore attribute information by performing entropy decoding, inverse transformation, and / or predictive restoration.

[0349] The attribute information entropy decoding unit can recover transformation coefficients by receiving an attribute information bitstream and performing decoding. The LOD generation unit can generate a Level of Detail (LOD) by receiving the recovered geometry information, and point-by-point neighbors used for lifting transformation and predictive transformation can be determined. In this case, the LOD generation unit can be executed when LOD parameters are present. Additionally, when point-by-point neighbors are determined in the LOD generation unit, an attribute reference frame can be received as input to utilize points of the attribute reference frame for more efficient neighbor search.

[0350] The lifting inverse transform can restore attribute information by inverse transforming the input transformation coefficients and performing a frequency inverse transform through the generated LOD and the update and prediction processes for each LOD level. The prediction inverse transform can restore attributes by calculating the predicted value of the current point using neighbor points determined during the LOD generation process and using the sum of the predicted value and the value obtained by inverse transforming the input transformation coefficients.

[0351] Inverse RAHT can restore attribute information by receiving restored geometry information and transformation coefficients as input and performing the inverse process of RAHT (Region Adaptive Hierarchical Transform).

[0352] FIG. 23 is a block diagram illustrating an example of an attribute information decoding process. As illustrated, an attribute information bitstream can be input to an attribute information entropy decoding unit, and the attribute information entropy decoding unit can decode the attribute information bitstream to recover conversion coefficients. Subsequently, the decoding process can be branched depending on the presence or absence of LOD parameters (LoD params).

[0353] If LOD parameters exist (Yes), the LOD generation unit can generate an LOD using restored geometry information and / or attribute reference frames, and determine point-specific neighbors used for lifting transformation and predictive transformation. In this case, the lifting inverse transformation can restore attribute information by inversely transforming the input transformation coefficients and performing frequency inverse transformation through the generated LOD and LOD level-specific update and prediction processes. Additionally, the predictive inverse transformation can obtain a predicted value of the current point using neighbor points determined during the LOD generation process, and restore attributes using the sum of the predicted value and the value obtained by inversely transforming the input transformation coefficients.

[0354] In the case where LOD parameters do not exist (No), Inverse RAHT can restore attribute information by receiving restored geometry information and transformation coefficients as input and performing the inverse process of RAHT. As a result of the branching process as described above, restored attribute information can be output.

[0355] FIG. 24 shows the generation of LoD (Level of Detail) according to the embodiments.

[0356] Figure 24 shows the generation of LoD of the attribute decoding of Figure 23.

[0357] The LOD generation unit may be configured to determine point-by-point neighbor relationships used in the prediction and update process of LOD-based transformations (e.g., lifting transformations) by performing alignment, subsampling, and neighbor search on reference frames and / or current points.

[0358] The Morton code assignment and alignment unit can assign a Morton code to the reference frame and then align the points according to a defined alignment method. The subsampling unit can perform subsampling on the index list to classify the current points into LODs.

[0359] The nearest neighbor search unit can search for one or more neighboring points based on geometry for each point for the prediction and update process of lifting transformations for points classified by LOD, and store the index of the corresponding point. At this time, points in the reference frame may also be used to find neighboring points. In addition, the nearest neighbor search unit can perform intra-atlas search, intra-full search, inter-atlas search, and inter-full search.

[0360] FIG. 24 illustrates a processing flow (or block diagram) according to one embodiment of an LOD generation unit. As illustrated, the LOD generation unit receives restored geometry information as input and can generate and output an LOD index list by sequentially performing a Morton code assignment and alignment unit, a subsampling unit, and a nearest neighbor search unit.

[0361] The Morton code assignment and sorting unit assigns a Morton code to the input restored geometry information (or the corresponding points) and sorts the points according to a defined sorting method. The subsampling unit can perform subsampling on the index list to classify the current points into LODs. The nearest neighbor search unit can construct an LOD index list by searching for one or more neighbor points for each point based on geometry for the points classified as LODs, and storing the index of the neighbor point.

[0362] FIG. 25 illustrates the nearest neighbor point search according to the embodiments.

[0363] FIG. 25 further illustrates the nearest neighbor point search of the attribute decoding of FIG. 23.

[0364] FIG. 25 is a block diagram of a nearest neighbor point search unit. Each component may correspond to hardware, software, a processor, and / or a combination thereof. The nearest neighbor point search unit may be included in or operate in conjunction with an LOD generation unit, and the LOD generation unit may receive restored geometry information and output a multi-layered LOD.

[0365] The nearest neighbor point search unit may be configured to search for neighbor points for points by LOD level based on restored geometry information and to store the search results (e.g., index of neighbor points), and to this end, it may include an intra-atlas search unit, an intra-full search unit, an inter-atlas search unit, and an inter-full search unit.

[0366] The intra-atlas search unit can perform intra-atlas search and change the atlas search area (see 'Atlas Search Area Setting Method'). Information on the method of change can be transmitted to the decoder via signaling, or the decoder can determine whether to set the atlas search area based on a threshold received from the decoder. The intra-wide search unit can perform neighbor point search across the entire point cloud range if three neighbor points are not found in the intra-atlas search unit.

[0367] The inter-atlas search unit can perform inter-atlas search and change the inter-atlas search area (see 'Method for Setting Atlas Search Area'). Information on the method of change can be transmitted to the decoder via signaling, or the decoder can determine whether to perform atlas search area setting based on a threshold received from the decoder. The inter-total search unit searches for neighbor points according to Morton code order rather than in 3D space (see 'Intra-atlas Search Method'), and the search range can be a pre-set range or the number of points, and can also perform nearest neighbor search for the entire point cloud.

[0368] FIG. 25 illustrates a processing flow (or block diagram) according to one embodiment of a nearest neighbor point search unit. As illustrated, the nearest neighbor point search unit can generate and output an LOD index list by sequentially performing an intra-atlas search unit, an intra-full search unit, an inter-atlas search unit, and an inter-full search unit, using subsampled point information of the current frame and point information of the reference frame as inputs.

[0369] The intra-atlas search unit can perform intra-atlas search based on subsampled point information of the current frame and can change the atlas search area (see 'Atlas Search Area Setting Method'). Information regarding the method of change can be transmitted to the decoder via signaling, or the decision on whether to set the atlas search area can be made based on a threshold received from the decoder.

[0370] The intra-atlas search unit can perform neighbor point search across the entire point cloud range if three neighbor points are not found in the intra-atlas search unit.

[0371] The inter-atlas search unit can perform inter-atlas search using point information of a reference frame and can change the inter-atlas search area (see 'Method for Setting the Atlas Search Area'). Information regarding the method of change can be transmitted to the decoder via signaling, or the determination of whether to set the atlas search area can be made based on a threshold received from the decoder.

[0372] The intra-atlas search unit searches for neighbor points according to Morton code order rather than in 3D space (see 'Intra-atlas Search Method'), and the search range can be a preset range or the number of points, and can also perform nearest neighbor search for the entire point cloud. Based on the search results, the nearest neighbor search unit can output an LOD index list.

[0373] The intra-search unit (or intra-atlas search unit) of the nearest neighbor search unit can perform the nearest neighbor search by setting the atlas search area in various ways when performing the intra-atlas search.

[0374] The intra-atlas search unit can perform nearest neighbor search by setting only one central minicube as the atlas search area through the center-based atlas search area setting method (see 'Center-based atlas search area setting method'). Information regarding whether the technique is performed can be received via signaling through the method described in 'Center-based atlas search area setting method'. Alternatively, without signaling, the density of the point cloud can be determined in the decoder (see 'Center-based atlas search area setting method'), and if the density is greater than or equal to a specific density threshold d1, the center-based atlas search area setting technique can be performed.

[0375] The intra-atlas search unit can perform nearest neighbor search by setting only the nine minicubes along the central plane as the atlas search area through the planar atlas search area setting method (refer to 'Planar Atlas Search Area Setting Method'). Information regarding whether the technique has been performed can be detected by receiving a signal through the method described in 'Planar Atlas Search Area Setting Method'. Alternatively, without signaling, the decoder determines the angle with the Z-axis through the normal vector of the current point (refer to 'Planar Atlas Search Area Setting Method'), and if that angle is a specific angle threshold If the following applies, a planar atlas search area setting technique can be performed.

[0376] The intra-atlas search unit can perform nearest neighbor search by setting the atlas search area to include only the nine minicubes, including the central minicube, parallel to the X-axis (or Y-axis), through the method for setting the vertical atlas search area (refer to 'Method for Setting the Vertical Atlas Search Area'). Information regarding whether the technique is to be performed can be detected via a signal through the method described in 'Method for Setting the Vertical Atlas Search Area'; if performed, the status regarding the X-axis or Y-axis is also signaled, allowing the technique to be performed based on that axis. Alternatively, without signaling, the decoder determines the angle with respect to the X-axis (or Y-axis) using the normal vector of the current point (refer to 'Method for Setting the Vertical Atlas Search Area'), and if that angle is a specific angle threshold If it is less than that, a vertical atlas search area setting technique can be performed.

[0377] The intra-atlas search unit can perform nearest neighbor search by setting only the minicubes of a pre-selected index as the atlas search area through the index-based atlas search area setting method (refer to 'Index-based Atlas Search Area Setting Method'). Information regarding whether the technique is executed can be detected by receiving a signal through the method described in 'Index-based Atlas Search Area Setting Method', and when executed, information regarding the selected index is also signaled, allowing only the minicubes of that index to be searched.

[0378] The intra-atlas search unit can perform nearest neighbor search based on an expanded atlas search area through the method for setting an expanded atlas search area (see 'Method for setting an expanded atlas search area'). Information regarding whether the technique is performed can be signaled through the method described in 'Method for setting an expanded atlas search area', and if performed, information regarding the expansion step can be signaled together to expand the atlas search area by that step. Alternatively, without signaling, the density of the point cloud in the decoder can be determined (see 'Method for setting an expanded atlas search area'), and if the density is less than or equal to a specific density threshold d2, the technique for setting an expanded atlas search area can be performed.

[0379] The inter-search unit of the nearest neighbor search unit can perform the nearest neighbor search by setting the search area in various ways when performing the inter-search.

[0380] The inter-search unit can perform nearest neighbor search by setting only one central minicube as the search area through the center-based search area setting method (see 'Center-based Atlas Search Area Setting Method'). Information regarding whether the technique is performed can be received via signaling through the method described in 'Center-based Atlas Search Area Setting Method'. Alternatively, without signaling, the decoder can determine the density of the point cloud (see 'Center-based Atlas Search Area Setting Method'), and if the density is greater than or equal to a specific density threshold d1, the center-based search area setting technique can be performed.

[0381] The inter-search unit can perform nearest neighbor search by setting only the 9 minicubes along the central plane as the search area through the planar search area setting method (refer to 'Planar Atlas Search Area Setting Method'). Information regarding whether the technique has been performed can be detected by receiving a signal through the method described in 'Planar Atlas Search Area Setting Method'. Alternatively, without signaling, the decoder determines the angle with the Z-axis through the normal vector of the current point (refer to 'Planar Atlas Search Area Setting Method'), and if that angle is a specific angle threshold If the condition is lower, a planar search area setting technique can be performed.

[0382] The search unit can perform nearest neighbor search by setting the search area to include only the nine minicubes, including the central minicube, parallel to the X-axis (or Y-axis) through a vertical search area setting method (refer to 'Method for Setting Vertical Atlas Search Area'). Information regarding whether the technique is being performed can be detected via a signal through the method described in 'Method for Setting Vertical Atlas Search Area'; if performed, the status regarding the X-axis or Y-axis is also signaled, allowing the technique to be performed based on that axis. Alternatively, without signaling, the decoder determines the angle with the X-axis (or Y-axis) through the normal vector of the current point (refer to 'Method for Setting Vertical Atlas Search Area'), and if that angle is a specific angle threshold If the value is lower, a vertical search area setting technique can be performed.

[0383] The inter-search unit can perform nearest neighbor search by setting only the minicubes of a pre-selected index as the search area through the index-based search area setting method (see 'Index-based Atlas Search Area Setting Method'). Information regarding whether the technique is performed can be signaled through the method described in 'Index-based Atlas Search Area Setting Method', and when performed, information regarding the selected index is also signaled to search only the minicubes of that index.

[0384] The inter-search unit can perform nearest neighbor search based on the expanded search area through the method for setting an expanded search area (see 'Method for setting an expanded atlas search area'). Information regarding whether the technique is performed can be signaled through the method described in 'Method for setting an expanded atlas search area', and if performed, information regarding the expansion step can be signaled together to expand the search area by that step. Alternatively, without signaling, the density of the point cloud in the decoder can be determined (see 'Method for setting an expanded atlas search area'), and if the density is less than or equal to a specific density threshold d2, the technique for setting an expanded search area can be performed.

[0385] FIGS. 26a and FIGS. 26b illustrate the generation of a predictor based on search area-related information according to embodiments.

[0386] When a decoder, which is a decoding method and apparatus according to embodiments (a receiving device (10004) in FIG. 1, a receiving (20002) to a rendering (20004) in FIG. 2, a decoder in FIG. 7 and FIG. 9, each device in FIG. 10, decoding in FIG. 11 to FIG. 19, receiving device in FIG. 22 to FIG. 23, decoding in FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition in FIG. 28 to FIG. 30, decoding in FIG. 31 to FIG. 32, decoding in FIG. 33, parameter acquisition in FIG. 34, method in FIG. 36, etc.), receives information related to setting an atlas search area and performs the decoding, the detailed flowchart is as shown in FIG. 26a and FIG. 26b.

[0387] FIG. 26 is a flowchart illustrating an example of a procedure for determining a search reference point and a search area in a nearest neighbor search. When point information of a reference frame is input, a reference point for inter-search can be selected depending on whether the same index reference point is used (same_index_reference_point?), for example, a reference point can be selected based on the same index, or a reference point can be selected based on the current coordinates as the nearest reference point (e.g., a reference point that reduces index movement).

[0388] When subsampled index point information is input, it can be determined whether to perform an intra-atlas search range modification (modify_intra_atlas_search_range_enabled==0?), and if no modification is performed to the intra-atlas search range, the search range can be set based on a predetermined method according to subsequent procedures.

[0389] It can be determined whether to perform an inter-atlas search range change (modify_inter_atlas_search_range_enabled==0?), and if no change to the inter-atlas search range is performed, nearest neighbor search can be performed without changing the search range.

[0390] If a change in the search range is performed, if the enlarged search range (enlarged_atlas_search_range?) is selected, (3+2N)^3 minicubes can be set as the search range (N is the signaled expansion step); if the planar search range (planar_atlas_search_range?) is selected, a planar search range can be set; and if the vertical search range (vertical_atlas_search_range?) is selected, a vertical search range can be set based on the signaled axis (X or Y). Additionally, if the center-based search range (centerbase_atlas_search_range?) is selected, a center-based search range can be set, and if the index-based search range (indexbase_atlas_search_range?) is selected, minicubes at the signaled index can be added to the search range.

[0391] Based on the search area set as above, nearest neighbor point search can be performed, and a predictor can be generated using the result.

[0392] FIGS. 27a and FIGS. 27b illustrate the generation of a predictor based on search area-related information according to embodiments.

[0393] When a decoder, which is a decoding method and apparatus according to embodiments (a receiving device (10004) in FIG. 1, a receiving (20002) to a rendering (20004) in FIG. 2, a decoder in FIG. 7 and FIG. 9, each device in FIG. 10, decoding in FIG. 11 to FIG. 19, receiving devices in FIG. 22 to FIG. 23, decoding in FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition in FIG. 28 to FIG. 30, decoding in FIG. 31 to FIG. 32, decoding in FIG. 33, parameter acquisition in FIG. 34, method in FIG. 36, etc.), determines whether to perform an atlas search area change, the detailed flowchart is as shown in FIG. 27a and FIG. 27b.

[0394] FIG. 27 is a flowchart illustrating an example of a procedure for determining the selection of an inter-search reference point and the setting of a search area in the nearest neighbor point search. As illustrated, when point information of a reference frame is input, the reference point for inter-search can be selected based on whether the same index reference point is used (same_index_reference_point?), for example, a reference point with the same index as the current point can be selected as the reference point, or the inter-search reference point can be selected based on the current coordinates.

[0395] When subsampled index point information is input, it can be determined whether to perform an intra-search range change (modify_intra_atlas_search_range_enabled==0?) and whether to perform an inter-search range change (modify_inter_atlas_search_range_enabled==0?). If an inter-search range change is performed, and an index-based search range (indexbase_atlas_search_range?) is selected, the minicube of the signaled index can be added to the search range. If an index-based search range is not selected, whether the density (d) of the point cloud is less than or equal to a threshold (d2) Depending on ), an extended search area setting can be performed to set 125 minicubes as the search area.

[0396] If the extended search area setting is not applied, a planar search area setting may be performed based on whether the angle with the Z-axis using the normal vector of the current point is less than or equal to a threshold (c2). Additionally, a vertical search area setting based on the X-axis or Y-axis may be performed based on signaled axis information based on whether the angle with the X-axis or Y-axis using the normal vector of the current point is less than or equal to a threshold (c1). Furthermore, even if the above conditions are not satisfied, a center-based search area setting may be performed based on whether the density (d) of the point cloud is greater than a threshold (d1) (d > d1).

[0397] Based on the search area and reference point determined as above, the nearest neighbor point search can be performed, and a prediction system can be generated using the results.

[0398] FIG. 28 shows a bitstream including geometry data, attribute data, and parameter information according to embodiments.

[0399] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, encoder of FIG. 8, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can generate a bitstream including an encoded geometry bitstream, an encoded attribute bitstream, and related parameter information and / or signaling information as in FIG. 28.

[0400] Decoding method and apparatus according to embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7, decoder of FIG. 9, each device of FIG. 10, decoding of FIG. 11 to 19, receiving device of FIG. 22 to 23, decoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to 30, decoding of FIG. 31 to 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.), as in FIG. 28, parameter information and / or signaling information can be acquired from a bitstream, and geometry data within the bitstream and attribute data can be decoded based on the acquired information.

[0401] Relevant information may be signaled to apply the embodiments. The signaling information according to the embodiments may be used at a transmitting end or a receiving end, etc.

[0402] The configuration of the encoded point cloud may be as follows. A point cloud data encoder that performs geometry encoding and / or attribute encoding processes may generate an encoded point cloud (or a bitstream containing a point cloud). Additionally, signaling information regarding the point cloud data may be generated and processed by a metadata processing unit of a point cloud data transmission device, and said signaling information may be included in the point cloud and transmitted.

[0403] The abbreviations used in this disclosure have the following meanings, and each abbreviation may be referred to by other terms within the scope of equivalent meaning. SPS (Sequence Parameter Set) means sequence parameter set, GPS (Geometry Parameter Set) means geometry parameter set, APS (Attribute Parameter Set) means attribute parameter set, and TPS (Tile Parameter Set) means tile parameter set. Additionally, Geom (Geometry bitstream) is a geometry bitstream and may include a geometry slice header and geometry slice data, and may have a configuration such as "geometry slice header + [geometry PU header + geometry PU data] | geometry slice data". Attr (Attribute bitstream) is an attribute bitstream and may include an attribute data unit header and attribute data unit data, and may have a configuration such as "attribute data unit header + [attribute PU header + attribute PU data] | attribute data unit data".

[0404] FIG. 28 illustrates an example configuration of an encoded point cloud (or a bitstream containing a point cloud). As illustrated, the bitstream may include a sequence parameter set (SPS), a geometry parameter set (GPS), one or more attribute parameter sets (APS_0, APS_1), and a tile parameter set (TPS), and subsequently, a plurality of slices (slice 0 to slice n) may be arranged sequentially. Each slice may include a geometry bitstream (Geom) and one or more attribute bitstreams (Attr_0, Attr_1).

[0405] In the embodiments, the TPS may include tile-bounding information, for example, tile_bounding_box_xyz0 and tile_bounding_box_whd may be included for Tile (0) to Tile (n). Additionally, the geometry bitstream (Geom) of each slice may include a geometry slice header (geom_slice_header) and geometry slice data (geom_slice_data), and the geometry slice header may include, for example, geom_geom_parameter_set_id, geom_tile_id, geom_slice_id, geomBoxOrigin, geom_box_log2_scale, geom_max_node_size_log2 and geom_num_points. According to this configuration, the receiver can restore the geometry and attributes of the point cloud using SPS / GPS / APS / TPS and slice-unit Geom / Attr information.

[0406] FIG. 29 shows an attribute parameter set (APS) within a bitstream according to embodiments.

[0407] Figure 29 shows the APS syntax within the bitstream of Figure 28.

[0408] Parameter information regarding the nearest neighbor search area setting technique for high-speed LOD (Level of Detail) generation can be signaled as syntax added to the Attribute Parameter Set (APS). For example, the signaling information of FIG. 29 can be efficiently signaled to support a high-speed nearest neighbor search method for LOD generation by being combined with the ASP. In addition, the name of the signaling information can be understood within the scope of the meaning and function of the signaling information, and can be referred to by another name within the scope of equivalent meaning.

[0409] The syntactic structure of an Attribute Parameter Set (APS) may include identifier information for APS identification and application relationships, and extension parameter information for setting prediction and nearest neighbor search areas, which can be signaled.

[0410] The attribute parameter set ID (`aps_attr_parameter_set_id`) provides an identifier for the APS so that it can be referenced by other syntax elements, and the value of `aps_attr_parameter_set_id` can be in the range of 0 to 15 (inclusive).

[0411] The sequence parameter set ID (`aps_seq_parameter_set_id`) specifies the value of `sps_seq_parameter_set_id` for the active SPS, and the value of `aps_seq_parameter_set_id` may be in the range of 0 to 15 (inclusive).

[0412] The extension presence (`aps_extension_present`) specifies whether `aps_extension_data` syntax elements exist within the APS syntax structure, and in bitstreams suitable for this version of the document, `aps_extension_present` must be 0, and a value where `aps_extension_present` is 1 may be reserved for future use by ISO / IEC.

[0413] Distribution-based prediction enable (`prediction_with_distribution_enabled`) specifies whether prediction coefficients are derived based on the spatial distribution of predictors when it is 1, and whether they are not when it is 0, and can be inferred as 0 if `prediction_with_distribution_enabled` does not exist.

[0414]

[0415] To support the nearest neighbor search area setting technique for high-speed LOD generation, the following parameters may be included and signaled in the APS.

[0416] Same index reference point enable (`same_index_reference_point_enabled`) specifies the method for selecting reference points in a reference frame (see 'Method for selecting reference points for atlas search in a reference frame'), and 0 indicates proximity-based reference point selection, and 1 indicates same index-based reference point selection.

[0417] The intra-atlas search range change enable (`modify_intra_atlas_search_range_enabled`) can be signaled with a flag to determine whether to perform an intra-atlas search range change (see 'Atlas Search Range Configuration Method').

[0418] The Inter-Atlas Search Range Modification Enable (`modify_inter_atlas_search_range_enabled`) can be signaled with a flag to determine whether to perform an Inter-Atlas Search Range modification (see 'Atlas Search Range Configuration Methods').

[0419] The point cloud density first threshold (`point_cloud_density_threshold1`) can be signaled as a point cloud density threshold for whether to perform center-based atlas search area setting (see 'Center-based atlas search area setting method').

[0420] The point cloud density second threshold (`point_cloud_density_threshold2`) can be signaled as a point cloud density threshold for determining whether to perform extended atlas search area setting (see 'Method for setting extended atlas search area').

[0421] The normal angle threshold (`normal_angle_threshold`) can be signaled as an angle threshold with respect to the axis for determining whether to perform planar and vertical atlas search area settings (see 'Planar Atlas Search Area Setting Method' and 'Vertical Atlas Search Area Setting Method').

[0422] Center-based atlas search range enable (centerbase_atlas_search_range_enabled): A flag to determine whether to perform center-based atlas search range configuration (see 'How to configure center-based atlas search range').

[0423] planar_atlas_search_range_enabled: A flag to determine whether to perform planar atlas search range configuration (see 'How to configure planar atlas search range').

[0424] vertical_atlas_search_range_enabled): A flag for determining whether to perform vertical atlas search range configuration (see 'How to configure vertical atlas search range').

[0425] Index-based atlas search range enable (indexbase_atlas_search_range_enabled): A flag for whether to perform index-based atlas search range configuration (see 'How to configure index-based atlas search range').

[0426] Enlarged_atlas_search_range_enabled: A flag to determine whether to configure the enlarged atlas search range (see 'How to configure the enlarged atlas search range').

[0427] FIG. 29 illustrates an example of the syntax structure of an Attribute Parameter Set (APS), showing a configuration that includes additional syntax to support a nearest neighbor search area setting technique for high-speed Level of Detail (LOD) generation. `aps_attr_parameter_set_id` is an identifier for identifying the APS, `aps_seq_parameter_set_id` indicates a value associated with the identifier of the active SPS, and `aps_extension_present` may indicate whether the `aps_extension_data` syntax element exists within the APS syntax structure.

[0428] FIG. 29 illustrates that `prediction_with_distribution_enabled` and `same_index_reference_point_enabled` may be included when conditions such as `attr_coding_type` and `pred_set_size_minus1` are satisfied, and indicates that signaling information for high-speed nearest neighbor search can be added to and combined with the APS. For example, `modify_intra_atlas_search_range_enabled`, `modify_inter_atlas_search_range_enabled`, `point_cloud_density_threshold1`, `point_cloud_density_threshold2`, `normal_angle_threshold`, `centerbase_atlas_search_range_enabled`, `planar_atlas_search_range_enabled`, `vertical_atlas_search_range_enabled`, `indexbase_atlas_search_range_enabled`, and `enlarged_atlas_search_range_enabled` may be included in the APS and signaled. In this case, each syntax element may be encoded with a descriptor such as `u(1)` or `ue(v)` as illustrated, and `byte_alignment()` may be applied at the end of the APS.

[0429] Accordingly, by combining the highlighted signaling information according to the syntax structure of Fig. 29, the relevant parameters can be efficiently signaled to support a high-speed nearest neighbor search method for LOD generation, and the names of the signaling information can be understood within the scope of their meaning and function.

[0430] FIG. 30 shows an attribute data unit header within a bitstream according to embodiments.

[0431] FIG. 30 illustrates the header of an attribute data unit (or slice) within the bitstream of FIG. 28.

[0432] Parameter information regarding the nearest neighbor search area setting technique for high-speed LOD (Level of Detail) generation can be signaled as syntax added to the Attribute Data Unit Header. For example, signaling information corresponding to rows highlighted in gray in the drawing can be configured to be included in the Attribute Data Unit Header, and by combining the signaling information of the highlighted rows, it can be efficiently signaled to support the high-speed nearest neighbor search method for LOD generation. Additionally, the name of the signaling information can be understood within the scope of the meaning and function of the signaling information, and may be referred to by other names within the scope of equivalent meaning.

[0433] The Attribute Data Unit Header may be signaled with additional parameter information regarding the nearest neighbor search area setting technique for high-speed Level of Detail (LOD) generation, along with identification and frame association information of the active Attribute Parameter Set (APS).

[0434] The attribute parameter set ID (`adu_attr_parameter_set_id`) can specify the active APS identified by `aps_attr_parameter_set_id`.

[0435] The time ID (`adu_temporal_id`) can specify the temporal ID of the frame associated with the attribute data unit.

[0436] The axis type (`axis_type`) is a syntax element for specifying the reference axis when performing a vertical atlas search area setting method (see 'Vertical Atlas Search Area Setting Method'), where 0 indicates the X-axis and 1 indicates the Y-axis.

[0437] The expansion factor (`enlarge_factor`) is a syntactic element used to specify the expansion step when performing the method for setting an expanded atlas search area (see 'Method for setting an expanded atlas search area'), and can be configured so that (3+2enlarge_factor)^3 minicubes are set as the atlas search area centered around the central minicube.

[0438] The number of indices in the index array (`num_of_indexes_in_index_array`) can represent the number of elements included in the minicube index array used in the index-based atlas search area setting method (see 'Index-based Atlas Search Area Setting Method'),

[0439] The minicube index array (`minicube_indexes_array[]`) can represent an array of minicube indices used in the index-based atlas search area setting method (see 'Index-based atlas search area setting method').

[0440] FIG. 30 illustrates an example of the syntax structure of an attribute data unit header, showing a configuration in which parameters related to setting the nearest neighbor search area for high-speed LOD generation are conditionally signaled in the header of the attribute data unit. `adu_attr_parameter_set_id` can be encoded as `u(4)` as a syntax element for specifying the active APS as `aps_attr_parameter_set_id`, and `adu_temporal_id` can be encoded as `u(3)` as a syntax element for specifying the temporal ID of the frame associated with the attribute data unit. Additionally, if `vertical_atlas_search_range_enabled` is true, `axis_type` can be included as `u(1)` to specify the axis, if `enlarged_atlas_search_range_enabled` is true, `enlarge_factor` can be included as `ue(v)`, if `indexbase_atlas_search_range_enabled` is true, `num_of_indexes_in_index_array` is included as `ue(v)`, and depending on `num_of_indexes_in_index_array`, `minicube_indexes_array[i]` can be repeatedly included as `ue(v)`. `byte_alignment()` can be applied at the end of the header.

[0441] Example 2

[0442] Modifications and combinations are possible between the embodiments. Additionally, terms used in this document may be understood based on their intended meanings within the scope of their widespread use in the field.

[0443] The improved nearest neighbor search scheme can be performed and applied in both the PCC encoder and the PCC decoder.

[0444] The nearest neighbor search method within the attribute encoding / decoding step of the encoding / decoding step according to the embodiments may include an axis-based atlas search step and / or a step for setting an atlas search area in the axis-based atlas search.

[0445] FIG. 31(a) shows an axis-based search according to embodiments, and FIG. 31(b) shows a multi-axis-based search according to embodiments.

[0446] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, encoder of FIG. 8, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by predicting attribute data by searching an axis-based atlas as in FIG. 31.

[0447] The decoding method and apparatus according to the embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform decoding of attribute data by predicting attribute data by searching an axis-based atlas as in FIG. 31.

[0448] In an axis-based atlas search method, the atlas search can be implemented by setting an area containing a point (gray area in FIG. 11) based on the point being searched for the nearest neighbor point, as shown in the example in FIG. 11, and setting an area of ​​the same size as said area in all directions to perform the search. If the area of ​​the size to which the point belongs is referred to as a mini-cube, a total of 27 mini-cubes are searched, including the mini-cube to which the point belongs, and such an area for 27 mini-cubes can be referred to as the atlas search range.

[0449] In one embodiment, when the size of the minicube is set to 2x2x2, there may be up to 8 points within one minicube, and the size of the atlas search area may be 6x6x6. Additionally, when the size of the minicube is 2Nx2Nx2N, the size of the atlas search area may be 6Nx6Nx6N.

[0450] However, in accordance with the characteristics of the point cloud, setting the atlas search area only for the region of interest can be efficient in terms of speed and accuracy. For example, if the method of performing nearest neighbor search by setting 27 minicubes as the atlas search area, as shown in FIG. 11, is called the default atlas search, then an axis-based atlas search method that searches only the region of interest perpendicular to a specific axis, as shown in FIG. 31(a), can be applied. In this case, the gray area in FIG. 31(a) represents a minicube where a reference point exists, and the illustrated minicubes can be the search targets.

[0451] In addition, as shown in FIG. 31(b), two or more axes may be selected to form an atlas search area together, and an atlas search may be performed based thereon. In this case, information about the axes can be signaled by the user from the encoder to the decoder.

[0452] FIG. 31(a) illustrates an example of an axis-based atlas search. As illustrated, the default atlas search can perform the search by setting 27 minicubes, which are extended in all directions centered around a minicube containing a target point that serves as the reference for the nearest neighbor search, as the atlas search area.

[0453] In contrast, axis-based atlas search can perform nearest neighbor search by restricting the search area based on specific axes that are predetermined or signaled. For example, X-axis atlas search can set the atlas search area to a set of minicubes extending in the X-axis direction, Y-axis atlas search to a set of minicubes extending in the Y-axis direction, and Z-axis atlas search to a set of minicubes extending in the Z-axis direction. By configuring the search area along the axes in this way, neighbor search along the axes can be performed efficiently while reducing the number of minicubes to be searched compared to forward search.

[0454] FIG. 31(b) illustrates an example of a multi-axis-based atlas search. As illustrated, the multi-axis-based atlas search may be configured such that two or more axes are selected to form an atlas search area in which regions of interest corresponding to the selected axes are combined, and to perform nearest neighbor search within the atlas search area.

[0455] XY-axis atlas search can establish an atlas search area by combining regions of interest corresponding to the X-axis and Y-axis, and accordingly, search can be performed centering on minicubes that are valid for search in the X-axis and Y-axis directions while including minicubes (gray areas) where reference points exist. Additionally, XYZ-axis atlas search can establish an atlas search area by combining regions of interest corresponding to the X-axis, Y-axis, and Z-axis. In this case, the combination of selected axes can be defined in various forms, and information regarding the axis combination can be signaled from the encoder to the decoder.

[0456] FIG. 32 shows an example of setting a search area when searching the z-axis according to embodiments.

[0457] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, FIG. 8, encoder, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by predicting attribute data by setting a search area during z-axis search as in FIG. 32.

[0458] The decoding method and apparatus according to the embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform decoding of attribute data by predicting attribute data by setting a search area during z-axis search as in FIG. 32.

[0459] The point cloud data encoding / decoding device (or method) according to the embodiments may include and perform an axis-based atlas search method (see 'Axis-based Atlas Search Method'), a center-based atlas search area setting method (see 'Center-based Atlas Search Area Setting Method'), an index-based atlas search area setting method ('Index-based Atlas Search Area Setting Method'), and an extended atlas search area setting method ('Extended Atlas Search Area Setting Method'). The operating principle of the basic technology and the signaling for execution are identical to the atlas search area setting method in basic atlas search (see 'Atlas Search Area Setting Method'), and examples for each method when Z-axis atlas search is performed are as shown in FIG. 32. The gray area in FIG. 32 is a minicube containing a reference point, and all minicubes present in the figure can be search targets.

[0460] When performing a center-based atlas search area setting method, the search area is set to include only the center minicube, just like in basic atlas search. When performing an index-based atlas search area setting method, the LuT can be used in the same way as in basic atlas search, and it may be effective to have a separate LuT for each axis-based atlas search. When performing an extended atlas search, if the extension step is denoted as N, it may be effective to search a total of (3+2N)^2 minicubes perpendicular to the selected axis based on the center minicube.

[0461] FIG. 32 illustrates examples of methods for setting an atlas search range when performing a Z-axis atlas search. The example on the left in FIG. 32 shows an example where only the center minicube (indicated in gray) containing the reference point is set as the search range according to a center-based atlas search, while the example in the center shows an example where the surrounding minicubes designated by the index are set as the search range including the center minicube according to an index-based atlas search. Additionally, the example on the right in FIG. 32 shows an example where multiple minicubes are extended in a direction perpendicular to the selected axis, including the center minicube, and set as the search range according to an extended atlas search. In FIG. 32, the gray area represents the minicube containing the reference point, and each minicube depicted in the figure can be a search target.

[0462] Referring to FIG. 25, the intra-atlas search unit can perform axis-based atlas search (see 4.2.1) and change the atlas search area (see 'Method for setting the atlas search area in axis-based atlas search'). In the intra-entire search unit, if three neighbor points are not found in the intra-atlas search unit, neighbor point search can be performed across the entire point cloud range.

[0463] In the inter-atlas search section, axis-based atlas search can be performed (see 'Axis-based atlas search method'), and the inter-atlas search area can be changed (see 'Method for setting the atlas search area in axis-based atlas search'). In the inter-total search section, neighbor points are searched according to Morton code order rather than 3D space (see 'Intra-atlas search method'), and the search range can be a preset range or the number of points, and the nearest neighbor point search can also be performed for the entire point cloud.

[0464] In the <Intra-search Unit>, the intra-atlas search unit can perform axis-based atlas search (see 'Axis-based Atlas Search Method'), and as described in the 'Axis-based Atlas Search Method', it can perform atlas search only on regions perpendicular to a specific axis or multiple axes using signaled axis information. Additionally, it can perform a method for setting the atlas search area (see 'Method for Setting the Atlas Search Area in Axis-based Atlas Search').

[0465] In the <Inter-atlas Search Unit>, the Inter-atlas Search Unit can perform axis-based atlas search (see 'Axis-based Atlas Search Method'), and as described in 'Axis-based Atlas Search Method', it can perform atlas search only on areas perpendicular to a specific axis or multiple axes using signaled axis information. Additionally, it can perform an atlas search area setting method (see 'Atlas Search Area Setting Method in Axis-based Atlas Search').

[0466] FIG. 33 illustrates an inter-atlas search according to embodiments.

[0467] The encoding method and apparatus according to the embodiments (transmitting device (10000) of FIG. 1, acquisition (20000) to transmission (20002) of FIG. 2, each device of FIG. 3, encoder of FIG. 8, each device of FIG. 10, encoding of FIG. 11 to 19, transmitting device of FIG. 20 to 21, encoding of FIG. 24 to 27a and FIG. 27b, bitstream and parameter generation of FIG. 28 to 30, encoding of FIG. 31 to 32, encoding of FIG. 33, parameter generation of FIG. 34, method of FIG. 35, etc.) can perform encoding of attribute data by predicting attribute data through inter-atlas search as in FIG. 33.

[0468] The decoding method and apparatus according to the embodiments (receiving device (10004) of FIG. 1, receiving (20002) to rendering (20004) of FIG. 2, decoder of FIG. 7 and FIG. 9, each device of FIG. 10, decoding of FIG. 11 to FIG. 19, receiving device of FIG. 22 to FIG. 23, decoding of FIG. 24 to FIG. 27a and FIG. 27b, bitstream and parameter acquisition of FIG. 28 to FIG. 30, decoding of FIG. 31 to FIG. 32, decoding of FIG. 33, parameter acquisition of FIG. 34, method of FIG. 36, etc.) can perform decoding of attribute data by predicting attribute data through inter-atlas search as in FIG. 33.

[0469] FIG. 33 illustrates a flow for determining whether to apply an axis-based atlas search and the applicable axis (or multiple axes) during the process of performing an atlas search based on point information. As shown in FIG. 33, after receiving point information, the device (or encoder / decoder) may select and perform at least one of an X-axis atlas search, a Y-axis atlas search, or a Z-axis atlas search based on axis information indicating the axis-based atlas search (e.g., whether `atlas_search_x_axis`, `atlas_search_y_axis`, or `atlas_search_z_axis` is present). If multiple axes are indicated, the process may branch to perform an XY-axis atlas search, an XZ-axis atlas search, a YZ-axis atlas search, or an XYZ-axis atlas search. Additionally, if the axis-based atlas search is not indicated or is not applied, the process may branch to a basic atlas search. Subsequently, a prediction value may be generated based on the selected search result.

[0470] FIG. 34 shows an attribute parameter set (APS) within a bitstream according to embodiments.

[0471] FIG. 34 is an additional example of FIG. 29 APS.

[0472] Bitstream (or parameter set) syntax according to the embodiments may include identifier information for identifying and applying an Adaptation Parameter Set (APS). For example, an attribute parameter set ID (`aps_attr_parameter_set_id`) provides an identifier of an APS that is referenced by other syntax elements, and the value of `aps_attr_parameter_set_id` may be in the range of 0 to 15 (inclusive). Additionally, `aps_seq_parameter_set_id` specifies the value of `sps_seq_parameter_set_id` of an active SPS, and the value of `aps_seq_parameter_set_id` may be in the range of 0 to 15 (inclusive).

[0473] The extension presence (`aps_extension_present`) specifies whether `aps_extension_data` syntax elements exist within the APS syntax structure, and in bitstreams conforming to this version of this document, `aps_extension_present` must be 0, and a value where `aps_extension_present` is 1 may be reserved by ISO / IEC for future use.

[0474] The APS syntax according to the embodiments may further include control information related to the calculation of prediction coefficients. For example, distribution-based prediction (`prediction_with_distribution_enabled`) may specify whether prediction coefficients are derived based on the spatial distribution of predictors when it is 1, and specify that the prediction coefficients are not derived based on the spatial distribution of predictors when it is 0. If `prediction_with_distribution_enabled` is not present in the syntax, `prediction_with_distribution_enabled` may be inferred as 0.

[0475] Additionally, the APS syntax according to the embodiments may include flags indicating whether to perform an axis-based atlas search. For example, the X-axis atlas search flag `atlas_search_x_axis` may be a flag for whether to perform an X-axis atlas search (see 'Axis-based atlas search method'), `atlas_search_y_axis` may be a flag for whether to perform a Y-axis atlas search (see 'Axis-based atlas search method'), and `atlas_search_z_axis` may be a flag for whether to perform a Z-axis atlas search (see 'Axis-based atlas search method').

[0476] FIG. 35 illustrates a encoding method according to embodiments.

[0477] The method according to the embodiments may include the step of encoding geometry data of point cloud data (S3500); and / or the step of encoding attribute data of point cloud data (S3510); etc.

[0478] The step of encoding geometry data (S3500) includes geometry data encoding operations described in FIG. 1 transmitting device (10000), FIG. 2 acquisition (20000) to transmission (20002), FIG. 3, FIG. 8 encoder, FIG. 10 each device, FIG. 11 to FIG. 19 encoding, FIG. 20 to FIG. 21 transmitting device, FIG. 24 to FIG. 27a and FIG. 27b encoding, FIG. 28 to FIG. 30 bitstream and parameter generation, FIG. 31 to FIG. 32 encoding, FIG. 33 encoding, FIG. 34 parameter generation, etc.

[0479] The step of encoding attribute data (S3510) includes attribute data encoding operations described in FIG. 1 transmitting device (10000), FIG. 2 acquisition (20000) to transmission (20002), FIG. 3, FIG. 8 encoder, FIG. 10 each device, FIG. 11 to FIG. 19 encoding, FIG. 20 to FIG. 21 transmitting device, FIG. 24 to FIG. 27a and FIG. 27b encoding, FIG. 28 to FIG. 30 bitstream and parameter generation, FIG. 31 to FIG. 32 encoding, FIG. 33 encoding, FIG. 34 parameter generation, etc.

[0480] The step of encoding attribute data (S3510) may include: a step of generating a Level of Detail (LoD) by subsampling point cloud data; and a step of predicting points based on the LoD.

[0481] The step of predicting points includes: spatially dividing points regarding a reference frame for a frame containing point cloud data into regions; and based on the regions, the nearest neighbors for the points can be searched.

[0482] The method of FIG. 35 can be performed by a device. The device includes a memory; and at least one processor connected to the memory; and the at least one processor may be configured to: encode geometry data of point cloud data; and encode attribute data of point cloud data.

[0483] The embodiments further include a computer-readable storage medium for storing a bitstream generated by the method according to FIG. 35.

[0484] The embodiments further include a method comprising the steps of: acquiring a bitstream for point cloud data; generating the bitstream based on the steps of encoding geometry data of the point cloud data and encoding attribute data of the point cloud data; and transmitting data including the bitstream.

[0485] FIG. 36 illustrates a decoding method according to embodiments.

[0486] The method according to the embodiments may include the step of decoding geometry data of point cloud data within a bitstream (S3600); and / or the step of decoding attribute data of point cloud data (S3610); etc.

[0487] The step of decoding geometry data (S3600) includes geometry data decoding operations described in FIG. 1 transmitting device (10000), FIG. 2 acquisition (20000) to transmission (20002), FIG. 3, FIG. 8 encoder, FIG. 10 each device, FIG. 11 to 19 encoding, FIG. 20 to 21 transmitting device, FIG. 24 to 27a and FIG. 27b encoding, FIG. 28 to 30 bitstream and parameter generation, FIG. 31 to 32 encoding, FIG. 33 encoding, FIG. 34 parameter generation, etc.

[0488] The step of decoding attribute data (S3610) includes attribute data decoding operations described in FIG. 1 transmitting device (10000), FIG. 2 acquisition (20000) to transmission (20002), FIG. 3, FIG. 8 encoder, FIG. 10 each device, FIG. 11 to FIG. 19 encoding, FIG. 20 to FIG. 21 transmitting device, FIG. 24 to FIG. 27a and FIG. 27b encoding, FIG. 28 to FIG. 30 bitstream and parameter generation, FIG. 31 to FIG. 32 encoding, FIG. 33 encoding, FIG. 34 parameter generation, etc.

[0489] In relation to attribute information decoding, LoD, and predictor search in FIG. 23, the step of decoding attribute data (S3610) may include: a step of generating a Level of Detail (LoD) by subsampling point cloud data; and a step of predicting points based on the LoD.

[0490] In relation to FIG. 25, the nearest neighbor search unit and initial-inter-level predictor search, the step of predicting a point includes: spatially dividing points in a reference frame for a frame containing point cloud data into regions; and based on the regions, the nearest neighbors for the point are searched. The regions according to the embodiments may be cubic blocks having a constant size. The regions may be referred to as blocks according to the embodiments.

[0491] Regarding center-based atlas search area setting and initial inter-frame predictor search, points for a reference frame are spatially divided into regions of a specific size, and the nearest neighbors for a point are searched based on one of the regions.

[0492] The single area used to search for the nearest neighbor is located at the center of the space.

[0493] With respect to FIG. 29, APS, 'modify_inter_atlas_search_range_enabled', 'centerbase_atlas_search_range_enabled', etc., the bitstream includes a set of attribute parameters, and the set of attribute parameters may include at least one of: information related to a search area for decoding attribute data based on prediction, and information regarding an area including points of a reference frame used to predict attribute data.

[0494] In relation to the same index reference point in the reference frame of FIG. 14 and the nearest location embodiment of FIG. 15, the step of predicting a point may include at least one of: a step of searching for a point having the same index as the index of a point in the frame in the reference frame for a frame containing point cloud data; a step of searching for a point in the reference frame that is closest to the point in the frame; a step of searching for a point based on central horizontal regions containing points in the reference frame; a step of searching for a point based on information indicating the order of indices of points within the reference frame; or a step of searching for a point by expanding the number of regions containing points within the reference frame.

[0495] In relation to the axis-based atlas search of FIG. 31, the step of predicting points may include: searching for regions containing points within a reference frame based on at least one direction of one axis or two or more axes.

[0496] The method of FIG. 36 can be performed by a device. The device includes a memory; and at least one processor connected to the memory; and the at least one processor may be configured to: decode geometry data of point cloud data within a bitstream; and decode attribute data of point cloud data.

[0497] The method / device according to the embodiments provides the following technical effects.

[0498] The PCC encoding method, PCC decoding method, and signaling method of the above-described embodiments (e.g., Embodiment 1) can provide the following effects. Compressing the attributes of a point cloud containing a vast amount of information takes a significant amount of time, and in particular, since LOD generation performed during the attribute compression process involves nearest neighbor search, the encoding / decoding speed becomes very slow. In environments such as autonomous driving that require low-latency encoding / decoding, such low encoding / decoding speeds become a problem. The present embodiments support a method for managing the setting of nearest neighbor search areas for high-speed LOD generation that can be utilized in point cloud attribute compression. The present embodiments support a method for increasing the speed of LOD generation by reducing the time required for nearest neighbor search through a technique that adaptively changes the atlas search area according to the characteristics of the point cloud content. Furthermore, by combining the present embodiments with techniques that adjust the distribution of points more densely, such as a Morton code scaling-based atlas search method, a method is supported that enables accurate and fast neighbor search by maximizing efficiency. Accordingly, the transmitting method / device according to the embodiments can transmit data by compressing point cloud data at high speed, and by transmitting signaling information for this purpose, the receiving method / device according to the embodiments can also decode / restore point cloud data at high speed. The operation of the transmitting and receiving device according to the above-described embodiments can be explained in combination with the following point cloud compression processing process.

[0499] The PCC encoding method, PCC decoding method, and signaling method of the above-described embodiments (e.g., Embodiment 2) can provide the following effects. Compressing the attributes of a point cloud containing a vast amount of information takes a long time, and in particular, since LOD generation performed during the attribute compression process involves nearest neighbor search, the encoding / decoding speed becomes very slow. In environments such as autonomous driving that require low-latency encoding / decoding, such low encoding / decoding speeds become a problem. The present embodiments support a method for managing the setting of nearest neighbor search areas for high-speed LOD generation that can be utilized in point cloud attribute compression. The present embodiments support a method for increasing the speed of LOD generation by reducing the time required for nearest neighbor search through a technique that performs atlas search only based on specific axes according to the characteristics of the point cloud. Accordingly, the transmitting method / device according to the embodiments can compress point cloud data at high speed and transmit the data, and by transmitting signaling information for this purpose, the receiving method / device according to the embodiments can also decode / restore point cloud data at high speed. The operation of the transmitting and receiving device according to the above-described embodiments can be explained in combination with the following point cloud compression processing process.

[0500] The embodiments have been described in terms of methods and / or devices, and the description of the methods and the description of the devices may be applied complementarily.

[0501] Although the drawings have been described separately for the convenience of explanation, it is also possible to design a new embodiment by combining the embodiments described in each drawing. Furthermore, designing a computer-readable recording medium containing a program for executing the previously described embodiments, as required by a person skilled in the art, falls within the scope of the claims of the embodiments. The apparatus and method according to the embodiments are not limited to the configuration and method of the embodiments described above; rather, the embodiments may be configured by selectively combining all or part of each embodiment to allow for various modifications. Although preferred embodiments have been illustrated and described, the embodiments are not limited to the specific embodiments described above. It is not only possible for a person skilled in the art to make various modifications without departing from the essence of the embodiments claimed in the claims, but such modifications should not be understood individually from the technical concept or perspective of the embodiments.

[0502] Various components of the device of the embodiments may be implemented by hardware, software, firmware, or a combination thereof. Various components of the embodiments may be implemented as a single chip, for example, a single hardware circuit. Depending on the embodiments, the components according to the embodiments may each be implemented as separate chips. Depending on the embodiments, at least one of the components of the device according to the embodiments may be composed of one or more processors capable of executing one or more programs, and one or more programs may include instructions for performing or executing any one or more of the operations / methods according to the embodiments. Executable instructions for performing the methods / operations of the device according to the embodiments may be stored in non-transient CRMs or other computer program products configured to be executed by one or more processors, or may be stored in transient CRMs or other computer program products configured to be executed by one or more processors. Additionally, memory according to the embodiments may be used as a concept that includes not only volatile memory (e.g., RAM, etc.) but also non-volatile memory, flash memory, PROM, etc. In addition, it may also include implementation in the form of carrier waves, such as transmission over the Internet. Furthermore, processor-readable recording media are distributed across networked computer systems, allowing processor-readable code to be stored and executed in a distributed manner.

[0503] In this document, “ / ” and “,” are interpreted as “and / or.” For example, “A / B” is interpreted as “A and / or B,” and “A, B” is interpreted as “A and / or B.” Additionally, “A / B / C” means “at least one of A, B and / or C.” Also, “A, B, C” means “at least one of A, B and / or C.” Additionally, in this document, “or” is interpreted as “and / or.” For example, “A or B” may mean 1) “A” alone, 2) “B” alone, or 3) “A and B.” In other words, “or” in this document may mean “additionally or alternatively.”

[0504] Terms such as "first," "second," etc., may be used to describe various components of the embodiments. However, the interpretation of the various components according to the embodiments should not be limited by these terms. These terms are merely used to distinguish one component from another. For example, the first user input signal may be referred to as the second user input signal. Similarly, the second user input signal may be referred to as the first user input signal. The use of these terms should be interpreted as not departing from the scope of the various embodiments. Although the first user input signal and the second user input signal are both user input signals, they do not imply the same user input signals unless clearly indicated in the context.

[0505] The terms used to describe the embodiments are intended for the purpose of describing specific embodiments and are not intended to limit the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless explicitly indicated in the context. Expressions of and / or are used to mean including all possible combinations between the terms. Expressions of include describe the presence of features, numbers, steps, elements, and / or components and do not imply the exclusion of additional features, numbers, steps, elements, and / or components. Conditional expressions such as "if" or "when" used to describe the embodiments are not limited to being optional. It is intended to be interpreted as "when a specific condition is satisfied," "when a related action is performed in response to a specific condition," or "when a related definition is interpreted."

[0506] Additionally, operations according to the embodiments described herein may be performed by a transmitting and receiving device including memory and / or a processor, depending on the embodiments. The memory may store programs for processing / controlling operations according to the embodiments, and the processor may control various operations described in this document. The processor may be referred to as a controller, etc. Operations in the embodiments may be performed by firmware, software, and / or a combination thereof, and the firmware, software, and / or a combination thereof may be stored in the processor or in memory.

[0507] Meanwhile, the operation according to the embodiments described above may be performed by a transmitting device and / or a receiving device according to the embodiments. The transmitting and receiving device may include a transmitting and receiving unit for transmitting and receiving media data, a memory for storing instructions (program code, algorithm, flowchart and / or data) for a process according to the embodiments, and a processor for controlling the operations of the transmitting and receiving devices.

[0508] The processor may be referred to as a controller, etc., and may correspond, for example, to hardware, software, and / or a combination thereof. The operation according to the embodiments described above may be performed by the processor. Additionally, the processor may be implemented as an encoder / decoder, etc., for the operation of the embodiments described above.

[0509] As described above, the relevant details have been explained in the best mode for carrying out the embodiments.

[0510] As described above, the embodiments may be applied wholly or partially to point cloud data transmission and reception devices and systems.

[0511] Those skilled in the art may make various changes or modifications to the embodiments within the scope of the embodiments.

[0512] The embodiments may include modifications / variations, and such modifications / variations do not exceed the scope of the claims and their equivalents.

Claims

1. A step of decoding geometry data of point cloud data within a bitstream; and A step of decoding attribute data of the above point cloud data; comprising method.

2. In Paragraph 1, The step of decoding the above attribute data is: A step of generating Level of Detail (LoD) by subsampling the above point cloud data; and The step of predicting points based on the above LoD; is included, method.

3. In Paragraph 2, The step of predicting the above points is: The step of spatially dividing points regarding a reference frame for a frame containing the above point cloud data into regions; Based on the above area, the nearest neighbor for the above point is searched, method.

4. In Paragraph 3, The points for the above reference frame are spatially divided into regions of a specific size, and The nearest neighbor to the above point is searched based on one of the above regions, method.

5. In Paragraph 4, The one area used to search for the nearest neighbor is located in the center of space, method.

6. In Paragraph 1, The above bitstream includes a set of attribute parameters, and The above set of attribute parameters is: Information related to a search area for decoding the above attribute data based on prediction, or Information regarding a region including points of a reference frame used to predict the above attribute data, comprising at least one of method.

7. In Paragraph 3, The step of predicting the above points is: A step of searching for a point having the same index as the index of a point in a reference frame for a frame containing the point cloud data; or A step of searching for a point in the reference frame that is closest to a point in the frame; or A step of searching for points based on central horizontal regions including points of the above reference frame; or A step of searching for points based on information indicating the order of indices of points within the above reference frame; or A step of searching for points by expanding the number of regions containing points within the reference frame; comprising at least one of the above steps, method.

8. In Paragraph 3, The step of predicting the above points is: A step of searching regions including points within the reference frame based on at least one direction among one axis or two or more axes; comprising method.

9. Memory; and At least one processor connected to the memory; comprising, wherein the at least one processor: Decoding geometry data of point cloud data within a bitstream; and Decoding attribute data of the above point cloud data; configured to do so, device.

10. A step of encoding the geometry data of the point cloud data; and A step of encoding attribute data of the above point cloud data; comprising method.

11. In Paragraph 10, The step of encoding the above attribute data is: A step of generating Level of Detail (LoD) by subsampling the above point cloud data; and The step of predicting points based on the above LoD; is included, method.

12. In Paragraph 11, The step of predicting the above points is: The step of spatially dividing points regarding a reference frame for a frame containing the above point cloud data into regions; Based on the above area, the nearest neighbor for the above point is searched, method.

13. Memory; and At least one processor connected to the memory; comprising, wherein the at least one processor: Encoding the geometry data of the point cloud data; and Configured to encode the attribute data of the above point cloud data; device.

14. A computer-readable storage medium for storing a bitstream generated by the method according to paragraph 10.

15. Step for acquiring a bitstream for point cloud data, The bitstream is generated based on the step of encoding geometry data of the point cloud data; and the step of encoding attribute data of the point cloud data; and A method comprising the step of transmitting data including the bitstream above.

Images