Point cloud data transmission device, point cloud data transmission method, point cloud data receiving device, and point cloud data receiving method

Abstract

A point cloud data transmission method according to an embodiment includes a step of encoding geometry data of point cloud data, a step of encoding characteristic data of the point cloud data based on the geometry data, and a step of transmitting the encoded geometry data, the encoded characteristic data, and signaling data.

Description

[Technical Field]

[0001] The embodiment relates to a method and apparatus for processing point cloud content. [Background technology]

[0002] Point cloud content is content represented by a point cloud, which is a collection of points belonging to a coordinate system that represents three-dimensional space. Point cloud content can represent three-dimensional media and is used to provide various services such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), XR (Extended Reality), and autonomous driving services. However, representing point cloud content requires tens of thousands to hundreds of thousands of point data points. Therefore, there is a need for a method to efficiently process such a massive amount of point data.

[0003] In other words, there is a problem in that sending and receiving point cloud data requires a large amount of processing power. Therefore, encoding for compression and decoding for decompression are performed during the process of sending and receiving point cloud data, but because the size of point cloud data is large, the calculations are complex and take a long time. [Overview of the project] [Problems that the invention aims to solve]

[0004] The technical challenges presented in this embodiment are to provide a point cloud data transmission device, a transmission method, a point cloud data receiving device, and a receiving method for efficiently sending and receiving point clouds in order to solve the problems mentioned above.

[0005] The technical challenges presented in the embodiment are to provide a point cloud data transmission device, transmission method, point cloud data receiver, and reception method that address latency and the complexity of encoding / decoding.

[0006] The technical problem related to the embodiment is to provide a point cloud data transmission device, transmission method, point cloud data receiving device, and receiving method for efficiently managing buffers in a receiving device when entropy continuity is given between multiple slices.

[0007] However, the scope of the embodiments is not limited to the technical problems described above, but can be extended to other technical problems that a person skilled in the art may derive from the entirety of this specification. [Means for solving the problem]

[0008] To achieve the above objectives and other advantages, the present invention, the embodiment of the point cloud data transmission method includes the steps of encoding geometric data of point cloud data, encoding characteristic data of point cloud data based on the geometric data, and transmitting the encoded geometric data, encoded characteristic data and signaling data.

[0009] One embodiment involves segmenting the encoded geometric data into multiple slices, where the context of any of these slices is referenced by at least one other slice.

[0010] One embodiment of this example is that the signaling data includes slice-related information and buffer management-related information.

[0011] One embodiment of the buffer management information is that it includes information indicating whether the context of the current slice is referenced by at least one other slice, or, if the context of the current slice is referenced by at least one other slice, information identifying the number of times the context of the current slice has been referenced.

[0012] The point cloud data transmission device according to the embodiment includes a geometry encoder for encoding the geometry data of point cloud data, a characteristic encoder for encoding characteristic data of point cloud data based on the geometry data, and a transmission unit for transmitting the encoded geometry data, encoded characteristic data, and signaling data.

[0013] One embodiment involves segmenting the encoded geometric data into multiple slices, where the context of any of these slices is referenced by at least one other slice.

[0014] One embodiment of this example is that the signaling data includes slice-related information and buffer management-related information.

[0015] One embodiment of the buffer management information is that it includes information indicating whether the context of the current slice is referenced by at least one other slice, or, if the context of the current slice is referenced by at least one other slice, information identifying the number of times the context of the current slice has been referenced.

[0016] The method for receiving point cloud data according to the embodiment includes the steps of receiving geometry data, characteristic data and signaling data, decoding the geometry data based on the signaling data, decoding the characteristic data based on the signaling data and the decoded geometry data, and rendering the point cloud data decoded based on the signaling data.

[0017] The geometric data is included in a plurality of slices, and in one embodiment, the context of any of the plurality of slices is referenced by at least one other slice.

[0018] In one embodiment, the signaling data includes slice-related information and buffer management-related information.

[0019] In one embodiment, the buffer management-related information includes information indicating whether the context of the current slice is referenced by at least one other slice, or information for identifying the number of references to the context of the current slice if the context of the current slice is referenced by at least one other slice.

[0020] In one embodiment, the geometric decoding stage controls the buffer storage of the context of the current slice based on the signaling-related information and the buffer management-related information.

[0021] In one embodiment, if it is confirmed that the context of the current slice is not referenced by at least one other slice based on the signaling-related information and the buffer management-related information, the geometric decoding stage deletes the current slice context from the buffer.

[0022] In one embodiment, if it is confirmed that the context of the current slice is referenced by at least one other slice based on the signaling-related information and the buffer management-related information, after the reference by at least one other slice, the geometric decoding stage deletes the current slice context from the buffer.

Advantages of the Invention

[0023] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiment provide a high-quality point cloud service.

[0024] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiment can achieve various video codec schemes.

[0025] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiment provide general-purpose point cloud content such as autonomous driving services.

[0026] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiment provide improved parallel processing and scalability by performing spatially adaptive partitioning of point cloud data for independent encoding and decoding of point cloud data.

[0027] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiment improve the performance of encoding and decoding point clouds by dividing the point cloud data into tile and / or slice units, encoding and decoding them, and signaling the data necessary for that purpose.

[0028] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiment can transmit compressed data in sections according to certain criteria for point cloud data. Furthermore, when using layered coding, compressed data can be transmitted in sections according to the layer. Consequently, the storage and transmission efficiency of the transmission device is increased.

[0029] In the point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiment, when transmitting a bitstream divided into slice units, the receiver can selectively transmit the bitstream to the decoder according to the density of the point cloud data to be represented, depending on the performance of the decoder or the application field. In this case, since the selection is performed before decoding, the decoder's efficiency is increased, and it has the advantage of being able to support decoders of various performance levels.

[0030] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiment enable efficient buffer management of the receiver by signaling whether or not to use the coding information / node information of the current slice in the next slice, and managing the storage and deletion of the current slice's context in the buffer based on this. In other words, even when the point cloud data is transmitted in multiple slices, the transmission device can improve compression efficiency by allowing the use of consecutive coding information and / or adjacent node information. The reception device can also efficiently manage its buffer because it can recognize whether or not to use the coding information / node information of the current slice in the next slice. For example, the receiver can efficiently manage resources by deciding in advance whether or not to use the current slice's context in the next slice. [Brief explanation of the drawing]

[0031] The drawings are attached to further understand the embodiment and show the embodiment along with a description of the embodiment.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18(a)-18(c)

Figure 19

Figure 20

Figure 21(a)

Figure 21(b)

Figure 22(a)

Figure 22(b)

Figure 23

Figure 24

Figure 25(a)

Figure 25(b)

Figure 25(c)-25(e)

【Figure This figure shows an example of a neighbor structure according to the embodiment. ​This figure shows an example of a bitstream structure for point cloud data used for transmission and reception in the embodiment. ​ This figure shows one example of the syntactic structure of a sequence parameter set according to the examples. ​ This figure shows one example of the syntactic structure of a sequence parameter set according to the examples. ​ This figure shows an example of the syntactic structure of a geometry parameter set according to an example. ​ This figure shows another example of the syntactic structure of a geometry parameter set by the example. ​ This figure shows an example of the syntactic structure of a characteristic parameter set according to an example. ​ This figure shows an example of the syntactic structure of a characteristic parameter set according to an example. ​ This figure shows an example of the syntactic structure of a geometry slice bitstream according to an embodiment. ​ This figure shows an example of the syntactic structure of a geometry slice header according to an example. ​ This figure shows an example of the syntactic structure of a geometry data unit header according to an embodiment. ​ This figure shows an example of the syntactic structure of a geometry data unit according to an embodiment. ​ This figure shows an example of the syntactic structure of a characteristic slice bitstream according to the embodiment. ​ This figure shows an example of the syntactic structure of a characteristic slice header according to the example. ​ This figure shows another example of the syntactic structure of a characteristic data unit header according to the examples. ​ This figure shows another example of the syntactic structure of a characteristic data unit according to the examples. ​ This figure shows yet another example of a point cloud transmission device according to the embodiment. ​This is a configuration block diagram showing yet another example of a point cloud receiving device according to the embodiment. [Modes for carrying out the invention]

[0032] The embodiments described in this specification will be explained in detail below with reference to the attached drawings. Regardless of the reference numerals in the drawings, identical or similar components will be given the same reference numerals, and redundant descriptions will be omitted. The following embodiments are intended to embody the present invention and do not limit or restrict the scope of the rights of the present invention. Anything that can be easily inferred by experts in the art to which the present invention belongs from the detailed description and embodiments of the present invention shall be interpreted as falling within the scope of the rights of the present invention.

[0033] The detailed description in this specification should not be constrained in any way and should be considered illustrative. The scope of the invention should be determined based on a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

[0034] Desired embodiments will be described in detail with reference to the attached drawings. The following detailed description with reference to the attached drawings is intended to describe desired embodiments rather than to show only embodiments that can be embodied by the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments can be carried out even without such details. Most of the terms used in the embodiments are common terms widely used in the art, but some have been arbitrarily selected by the applicant, and their meanings will be explained in detail below as needed. Therefore, embodiments should be understood based on the intended meaning of the terms, not simply on their names or meanings. Furthermore, the following drawings and detailed description should not be interpreted as being limited to the embodiments specifically described, but should be interpreted as including equivalent or substitute embodiments to those described in the drawings and detailed description.

[0035] Figure 1 shows an example of a point cloud content provision system according to an embodiment.

[0036] The point cloud content provision system shown in Figure 1 includes a transmission device 10000 and a reception device 10004. The transmission device 10000 and the reception device 10004 are capable of wireless communication to send and receive point cloud data.

[0037] The transmitting device 10000 in the embodiment secures, processes, and transmits point cloud video (or point cloud content). In the embodiment, the transmitting device 10000 includes a fixed station, a base transceiver system (BTS), a network, AI (Artificial Intelligence) equipment and / or systems, robots, AR / VR / XR equipment and / or servers, etc. In the embodiment, the transmitting device 10000 also includes equipment that communicates with base stations and / or other wireless equipment using wireless connection technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)), robots, vehicles, AR / VR / XR equipment, mobile devices, home appliances, IoT (Internet of Things) equipment, AI equipment / servers, etc.

[0038] The transmission device 10000 according to the embodiment includes a Point Cloud Video Acquisition unit 10001, a Point Cloud Video Encoder 10002, and / or a Transmitter (or communication module) 10003.

[0039] The point cloud video acquisition unit 10001 according to the embodiment acquires point cloud video through processing steps such as capture, synthesis, or generation. Point cloud video is point cloud content represented by a point cloud, which is a collection of points located in three-dimensional space, and is called point cloud video data, etc. The point cloud video according to the embodiment includes one or more frames. One frame represents a still image / picture. Therefore, point cloud video includes point cloud video / frames / pictures and is referred to as point cloud video, frame, or picture.

[0040] The point cloud video encoder 10002 according to the embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 encodes the point cloud video data based on point cloud compression coding. The point cloud compression coding according to the embodiment includes G-PCC (Geometry-based Point Cloud Compression) coding and / or V-PCC (Video-based Point Cloud Compression) coding or next-generation coding. Note that the point cloud compression coding according to the embodiment is not limited to the embodiment described above. The point cloud video encoder 10002 can output a bitstream containing the encoded point cloud video data. The bitstream contains not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0041] In this embodiment, the transmitter 10003 transmits a bitstream containing encoded point cloud video data. The bitstream in this embodiment is encapsulated into a file or segment (e.g., a streaming segment) and transmitted over various networks such as broadcast networks and / or broadband networks. Although not shown, the transmitting device 10000 includes an encapsulation unit (or encapsulation module) that performs the encapsulation operation. In this embodiment, the encapsulation unit is also included in the transmitter 10003. In this embodiment, the file or segment is transmitted to the receiving device 10004 over the network or stored on a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitter 10003 in this embodiment can communicate wirelessly with the receiving device 10004 (or receiver 10005) over a network such as 4G, 5G, or 6G. The transmitter 10003 can also perform necessary data processing operations through a network system (e.g., a communication network system such as 4G, 5G, or 6G). Furthermore, the transmitting device 10000 can also transmit encapsulated data using an on-demand method.

[0042] The receiving device 10004 in the embodiment includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. In the embodiment, the receiving device 10004 includes devices, robots, vehicles, AR / VR / XR devices, mobile devices, home appliances, IoT (Internet of Things) devices, AI devices / servers, etc., that communicate with base stations and / or other wireless devices using wireless connectivity technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)).

[0043] The receiver 10005 in this embodiment receives a bitstream containing point cloud video data or a file / segment in which the bitstream is encapsulated from a network or storage medium. The receiver 10005 performs the necessary data processing operations according to the network system (e.g., a communication network system such as 4G, 5G, or 6G). The receiver 10005 in this embodiment decapsulates the received file / segment and outputs a bitstream. In this embodiment, the receiver 10005 also includes a decapsulation unit (or decapsulation module) for performing the decapsulation operation. The decapsulation unit is also embodied as a separate element (or component) from the receiver 10005.

[0044] The point cloud video decoder 10006 decodes a bitstream containing point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data using the encoded scheme (for example, the reverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression coding, which is the reverse process of point cloud compression. Point cloud decompression coding includes G-PCC coding.

[0045] Renderer 10007 renders the decoded point cloud video data. In one embodiment, renderer 10007 renders the decoded point cloud video data using a viewport or the like. Renderer 10007 renders not only the point cloud video data but also the audio data to output the point cloud content. In one embodiment, renderer 10007 includes a display for displaying the point cloud content. In another embodiment, the display is not included in renderer 10007 but is embodied by a separate device or component.

[0046] In the drawing, the dotted arrows indicate the transmission path of the feedback information obtained by the receiving device 10004. The feedback information is information that reflects the interaction with the user consuming the point cloud content and includes user information (e.g., head orientation information, viewport information, etc.). In particular, if the point cloud content is for a service that requires interaction with the user (e.g., an autonomous driving service), the feedback information can be transmitted to the content sender (e.g., the transmitting device 10000) and / or the service provider. In this embodiment, the feedback information can be used by the receiving device 10004 as well as the transmitting device 10000, and may not be provided.

[0047] In this embodiment, head orientation information refers to information about the user's head position, direction, angle, and movement. The receiving device 10004 in this embodiment calculates viewport information based on the head orientation information. Viewport information is information about the area of ​​the point cloud video that the user is viewing (i.e., the area the user is currently viewing). In other words, viewport information is information about the area that the user is currently viewing within the point cloud video. To put it another way, a viewport or viewport area means the area that the user is viewing in the point cloud video. The viewpoint is the point that the user is viewing in the point cloud video, and it means the center of the viewport area. In other words, the viewport is an area centered on the viewpoint, and the size and shape of this area are determined by the FOV (Field of View). Therefore, in addition to head orientation information, the receiving device 10004 can extract viewport information based on vertical or horizontal FOVs supported by the device. Furthermore, the receiving device 10004 can perform gaze analysis based on head orientation information and / or viewport information to confirm the user's point cloud video consumption method, the point cloud video area the user gazes at, and the gazing time. In the embodiment, the receiving device 10004 transmits feedback information, including the gaze analysis results, to the transmitting device 10000. According to the embodiment, devices such as VR / XR / AR / MR displays extract viewport areas based on the user's head position / direction, the vertical or horizontal FOV supported by the device, etc. According to the embodiment, head orientation information and viewport information are also called feedback information, signaling information, or metadata.

[0048] The feedback information in the embodiment is obtained during the rendering and / or display process. The feedback information in the embodiment is obtained by one or more sensors included in the receiving device 10004. In the embodiment, the feedback information is also obtained by the renderer 10007 or another external element (or device, component, etc.). The dotted line in Figure 1 shows the transmission process of the feedback information obtained by the renderer 10007. The feedback information is not only transmitted to the transmitting side but can also be consumed by the receiving side. That is, the point cloud content provisioning system can process (encode / decode / render) the point cloud data based on the feedback information. For example, the point cloud video decoder 10006 and the renderer 10007 can use the feedback information, i.e., head orientation information and / or viewport information, to preferentially decode and render only the point cloud video for the area currently being viewed by the user.

[0049] Furthermore, the receiving device 10004 can transmit feedback information to the transmitting device 10000. The transmitting device 10000 (or the point cloud video encoder 10002) performs encoding operations based on the feedback information. Therefore, the point cloud content provision system does not process (encode / decode) all point cloud data, but can efficiently process the necessary data (for example, point cloud data corresponding to the user's head position) based on the feedback information and provide point cloud content to the user.

[0050] In this embodiment, the transmitting device 10000 is referred to as an encoder, transmitting device, or transmitter, and the receiving device 10004 is referred to as a decoder, receiving device, or receiver.

[0051] The point cloud data processed (through a series of processes including acquisition / encoding / transmission / decoding / rendering) in the point cloud content delivery system shown in Figure 1 according to the embodiment is also called point cloud content data or point cloud video data. In the embodiment, point cloud content data can be used as a concept that includes metadata or signaling information related to the point cloud data.

[0052] The elements of the point cloud content delivery system shown in Figure 1 are embodied in hardware, software, processors, and / or combinations thereof.

[0053] Figure 2 is a block diagram showing the operation of point cloud content provision according to an example.

[0054] Figure 2 is a block diagram illustrating the operation of the point cloud content delivery system described in Figure 1. As mentioned above, the point cloud content delivery system processes point cloud data based on point cloud compression coding (e.g., G-PCC).

[0055] In the point cloud content provision system according to the embodiment (for example, a point cloud transmission device 10000 or a point cloud video acquisition unit 10001), a point cloud video is acquired (20000). The point cloud video is represented by a point cloud belonging to a coordinate system that represents three-dimensional space. The point cloud video according to the embodiment includes a Ply (Polygon File format or the Stanford Triangle format) file. If the point cloud video has one or more frames, the acquired point cloud video includes one or more Ply files. The Ply file includes point cloud data such as the geometry and / or attributes of the points. The geometry includes the position of the points. The position of each point is represented by parameters that indicate a three-dimensional coordinate system (for example, a coordinate system consisting of XYZ axes, etc.) (for example, the values ​​of the X, Y, and Z axes). The attributes include the attributes of the points (for example, the texture information, hue (YCbCr or RGB), reflectance (r), transparency, etc. of each point). A single point has one or more attributes (or attributes). For example, a single point may have one characteristic, hue, or two characteristics, hue and reflectance. In the embodiment, geometry is also called position, geometry information, geometry data, etc., and characteristics are also called characteristics, characteristic information, characteristic data, etc. Furthermore, the point cloud content provision system (e.g., point cloud transmission device 10000 or point cloud video acquisition unit 10001) can secure point cloud data from information related to the point cloud video acquisition process (e.g., depth information, hue information, etc.).

[0056] The point cloud content provision system according to the embodiment (e.g., a transmitter 10000 or a point cloud video encoder 10002) encodes point cloud data (20001). The point cloud content provision system encodes point cloud data based on point cloud compression coding. As described above, the point cloud data includes the geometry and characteristics of points. Therefore, the point cloud content provision system can output a geometry bitstream by performing geometry coding to encode the geometry. The point cloud content provision system can output a characteristic bitstream by performing characteristic coding to encode the characteristics. In the embodiment, the point cloud content provision system can perform characteristic coding based on geometry coding. The geometry bitstream and characteristic bitstream according to the embodiment are multiplexed and output as a single bitstream. The bitstream according to the embodiment further includes signaling information related to geometry coding and characteristic coding.

[0057] The point cloud content provision system according to the embodiment (e.g., transmitter 10000 or transmitter 10003) transmits encoded point cloud data (20002). As explained in Figure 1, the encoded point cloud data is represented by a geometry bitstream and a characteristic bitstream. The encoded point cloud data is also transmitted in bitstream form along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometry encoding and characteristic encoding). The point cloud content provision system also encapsulates the bitstream that transmits the encoded point cloud data and transmits it in file or segment form.

[0058] The point cloud content provision system according to the embodiment (e.g., receiving device 10004 or receiver 10005) receives a bitstream containing encoded point cloud data. The point cloud content provision system (e.g., receiving device 10004 or receiver 10005) can also demultiplex the bitstream.

[0059] The point cloud content delivery system (e.g., receiver 10004 or point cloud video decoder 10005) decodes the encoded point cloud data (e.g., geometry bitstream, characteristic bitstream) transmitted in a bitstream. The point cloud content delivery system (e.g., receiver 10004 or point cloud video decoder 10005) decodes the point cloud video data based on the signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content delivery system (e.g., receiver 10004 or point cloud video decoder 10005) decodes the geometry bitstream to restore the point locations (geometry). The point cloud content delivery system decodes the characteristic bitstream based on the restored geometry to restore the point characteristics. The point cloud content delivery system (e.g., receiver 10004 or point cloud video decoder 10005) restores the point cloud video based on the restored geometry locations and decoded characteristics.

[0060] The point cloud content provision system according to the embodiment (e.g., receiving device 10004 or renderer 10007) renders the decoded point cloud data (20004). The point cloud content provision system (e.g., receiving device 10004 or renderer 10007) renders the decoded geometry and characteristics during the decoding process using various rendering methods. The points of the point cloud content are rendered as fixed points with a certain thickness, cubes with a predetermined minimum size centered at the position of the fixed point, or circles centered at the position of the fixed point, etc. All or part of the rendered point cloud content is provided to the user by a display (e.g., VR / AR display, general display, etc.).

[0061] The point cloud content provision system according to the embodiment (for example, the receiving device 10004) can secure feedback information (20005). The point cloud content provision system encodes and / or decodes the point cloud data based on the feedback information. The feedback information and operation of the point cloud content provision system according to the embodiment are the same as the feedback information and operation described in Figure 1, so a detailed explanation is omitted.

[0062] Figure 3 shows an example of the point cloud video capture process according to an embodiment.

[0063] Figure 3 shows an example of the point cloud video capture process of the point cloud content provision system described in Figures 1 and 2.

[0064] Point cloud content includes point cloud videos (images and / or footage) that represent objects and / or environments located in various three-dimensional spaces (e.g., a three-dimensional space representing a real environment, a three-dimensional space representing a virtual environment, etc.). Accordingly, the point cloud content provision system according to the embodiment captures point cloud videos using one or more cameras (e.g., an infrared camera capable of securing depth information, an RGB camera capable of extracting hue information corresponding to the depth information), a projector (e.g., an infrared pattern projector for securing depth information), LiDAR, etc., in order to generate point cloud content. The point cloud content provision system according to the embodiment secures point cloud data by extracting the form of the geometry composed of points in three-dimensional space from the depth information and extracting the characteristics of each point from the hue information. The images and / or footage according to the embodiment are captured based on either an inward-facing or outward-facing method.

[0065] The left side of Figure 3 shows the inward-facing method. The inward-facing method is a method in which one or more cameras (or camera sensors) positioned around a central object capture the central object. The inward-facing method is used to generate point cloud content (for example, VR / AR content that provides the user with a 360° image of an object (e.g., a core object such as a character, athlete, item, or actor)) that provides the user with a 360° image of the core object.

[0066] The outward-facing method is shown on the right side of Figure 3. In the outward-facing method, one or more cameras (or camera sensors) positioned around a central object capture the environment of a central object that is not the central object itself. The outward-facing method is used to generate point cloud content (for example, content showing the external environment provided to a user of an autonomous vehicle) to provide the surrounding environment from the user's point of view.

[0067] As illustrated, point cloud content is generated based on the capture operation of one or more cameras. In this case, since the coordinate systems of each camera are different, the point cloud content provider performs calibration of one or more cameras to set the global coordinate system before the capture operation. The point cloud content provider also generates point cloud content by combining the images and / or videos captured using the capture method described above with arbitrary images and / or videos. Furthermore, when the point cloud content provider generates point cloud content that represents a virtual space, it does not perform the capture operation described in Figure 3. The point cloud content provider system according to the embodiment can also perform post-processing on the captured images and / or videos. That is, the point cloud content provider can remove unwanted areas (e.g., background) or recognize the space in which the captured images and / or videos are linked and fill in any spatial holes if present.

[0068] Furthermore, the point cloud content provision system can generate a single point cloud content by performing coordinate system transformations on the points in the point cloud video captured from each camera. The point cloud content provision system performs coordinate system transformations on the points based on the position coordinates of each camera. As a result, the point cloud content provision system can generate content that shows a single wide area, or point cloud content with a high density of points.

[0069] Figure 4 shows an example of a Point Cloud Video Encoder according to an embodiment.

[0070] Figure 4 shows an example of the point cloud video encoder 10002 in Figure 1. The point cloud video encoder reconstructs and encodes the point cloud data (e.g., point locations and / or characteristics) to adjust the quality of the point cloud content (e.g., lossless, lossy, near-lossless) depending on network conditions or applications. If the overall size of the point cloud content is large (e.g., 60Gbps for 30fps), the point cloud content provider system cannot stream the content in real time. Therefore, the point cloud content provider system can reconstruct the point cloud content based on the maximum target bitrate to provide it according to the network environment.

[0071] As shown in Figures 1 and 2, the point cloud video encoder can perform geometry coding and characteristic coding. Geometry coding is performed before characteristic coding.

[0072] The point cloud video encoder according to the embodiment includes a Transformation Coordinates unit 40000, a Quantization unit 40001, an Octtree Analysis unit 40002, a Surface Approximation Analysis unit 40003, an Arithmetic Encoder 40004, a Geometry Reconstruction unit 40005, a Color Transformation unit 40006, an Attribute Transformation unit 40007, a Region Adaptive Hierarchical Transform (RAHT) transformation unit 40008, an LOD Generation unit 40009, a Lifting Transformation unit 40010, a Coefficient Quantization unit 40011, and / or an Arithmetic Encoder 40012.

[0073] The coordinate system transformation unit 40000, quantization unit 40001, octree analysis unit 40002, surface approximation analysis unit 40003, computational encoder 40004, and geometry reconstruction unit 40005 can perform geometry coding. Geometry coding according to the embodiment includes octree geometry coding, direct coding, trisoup geometry coding, and entropy coding. Direct coding and trisoup geometry coding can be applied selectively or in combination. Note that geometry coding is not limited to the examples given above.

[0074] As illustrated, the coordinate system transformation unit 40000 in this embodiment receives a position and converts it into a coordinate system. For example, the position is converted into position information in a three-dimensional space (for example, a three-dimensional space represented by an XYZ coordinate system). The position information in the three-dimensional space in this embodiment is also called geometric information.

[0075] The quantization unit 40001 in the embodiment quantizes the geometry. For example, the quantization unit 40001 quantizes points based on the minimum position value of the total points (e.g., the minimum value on each axis for the X, Y, and Z axes). The quantization unit 40001 multiplies the difference between the minimum position value and the position value of each point by a predetermined quatization scale value, and then performs a quantization operation to find the closest integer value by rounding down or up. Therefore, one or more points can have the same quantized position (or position value). In order to reconstruct the quantized points, the quantization unit 40001 in the embodiment performs voxelization based on the quantized position. Just as the smallest unit containing 2D image / video information is a pixel, the points of the point cloud content (or 3D point cloud video) in the embodiment are contained in one or more voxels. A voxel is a combination of the words "volume" and "pixel," and refers to a three-dimensional cubic space that arises when a three-dimensional space is divided into units (unit = 1.0) based on axes that represent three-dimensional space (e.g., X-axis, Y-axis, Z-axis). The quantization unit 40001 can match groups of points in three-dimensional space using voxels. In the embodiment, a single voxel may contain only one point. In the embodiment, a single voxel may contain one or more points. Furthermore, in order to represent a single voxel with one point, the position of the center point (ceter) of the voxel can be set based on the positions of one or more points contained within that voxel. In this case, the characteristics of all positions contained within the voxel are combined and assigned to that voxel.

[0076] The octree analysis unit 40002 in the example performs octree geometry coding (or octree coding) to represent voxels as an octree structure. The octree structure represents points matched to voxels based on an octave structure.

[0077] The surface approximation analysis unit 40003 in the example analyzes and approximates an octree. The octree analysis and approximation in the example is a process of analyzing a region containing a large number of points in order to voxelize it, in order to efficiently provide an octree and voxelization.

[0078] The arithmetic encoder 40004 according to the embodiment entropy encodes an octree and / or an approximated octree. For example, the encoding scheme includes an arithmetic encoding method. A geometry bitstream is generated as a result of the encoding.

[0079] The color conversion unit 40006, the characteristic conversion unit 40007, the RAHT conversion unit 40008, the LOD generation unit 40009, the lift conversion unit 40010, the coefficient quantization unit 40011, and / or the arithmetic encoder 40012 perform characteristic coding. As described above, a single point has one or more characteristics. The characteristic coding according to the embodiment is applied equally to the characteristics of a single point. However, if a single characteristic (e.g., hue) includes one or more elements, independent characteristic coding is applied to each element. The characteristic coding according to the embodiment includes color conversion coding, characteristic conversion coding, RAHT (Region Adaptive Hierarchical Transform) coding, predictive transformation (Interpolaration-based hierarchical nearest-neighbour prediction-Prediction Transform) coding, and lift transformation (interpolaration-based hierarchical nearest-neighbour prediction with an update / lifting step (Lifting Transform)) coding. Depending on the point cloud content, the RAHT coding, predictive text coding, and lift coding described above may be used selectively, or a combination of one or more coding methods may be used. Furthermore, the characteristic coding in the examples is not limited to those described above.

[0080] The color conversion unit 40006 in the embodiment performs color conversion coding to convert the color values ​​(or textures) included in the characteristics. For example, the color conversion unit 40006 converts the format of the hue information (for example, from RGB to YCbCr). The operation of the color conversion unit 40006 in the embodiment is applied optionally depending on the color values ​​included in the characteristics.

[0081] The geometry reconstruction unit 40005 in the embodiment reconstructs (restores) the octree and / or approximated octree. The geometry reconstruction unit 40005 reconstructs the octree / voxels based on the results of analyzing the distribution of points. The reconstructed octree / voxels are also called the reconstructed geometry (or restored geometry).

[0082] The characteristic transformation unit 40007 in the embodiment performs characteristic transformations based on positions where geometry coding has not been performed and / or on reconstructed geometry. As described above, since characteristics are dependent on geometry, the characteristic transformation unit 40007 can transform characteristics based on reconstructed geometry information. For example, the characteristic transformation unit 40007 can transform the characteristics of a point at a given position based on the position value of the point contained in that voxel. As described above, if the position of the center point of a voxel is set based on the positions of one or more points contained in that voxel, the characteristic transformation unit 40007 transforms the characteristics of one or more points. If trisoup geometry coding has been performed, the characteristic transformation unit 40007 can transform characteristics based on trisoup geometry coding.

[0083] The characteristic conversion unit 40007 performs characteristic conversion by calculating the average value of the characteristics or characteristic values ​​(e.g., hue or reflectance of each point) of points adjacent to the center point (or position value) of each voxel within a specific position / radius. When calculating the average value, the characteristic conversion unit 40007 applies a weighted value based on the distance from the center point to each point. Therefore, each voxel has a position and a calculated characteristic (or characteristic value).

[0084] The characteristic transformation unit 40007 searches for adjacent points within a specific position / radius from the position of the center point of each voxel based on a KD tree or Moulton code. The KD tree is a binary search tree that supports a data structure that manages points on a location basis, enabling rapid Nearest Neighbor Search (NNS). The Moulton code represents the coordinate values ​​(e.g., (x,y,z)) indicating the 3D position of all points as bit values, and is generated by mixing the bits. For example, if the coordinate value indicating the position of a point is (5,9,1), the bit values ​​of the coordinate value are (0101, 1001, 0001). Mixing the bit values ​​in the order of z, y, x according to the bit index gives 010001000111. This value in decimal is 1095. That is, the Moulton code value of the point with coordinate values ​​(5,9,1) is 1095. The characteristic conversion unit 40007 aligns points based on Moulton code values ​​and performs shortest neighbor search (NNS) using a depth-first traversal process. After the characteristic conversion operation, if shortest neighbor search (NNS) is required in other conversion processes for characteristic coding, the KD tree or Moulton code is utilized.

[0085] As shown in the diagram, the converted characteristics are input to the RAHT conversion unit 40008 and / or the LOD generation unit 40009.

[0086] The RAHT conversion unit 40008 in this embodiment performs RAHT coding to predict characteristic information based on the reconstructed geometry information. For example, the RAHT conversion unit 40008 can predict the characteristic information of a node at a higher level of the octum tree based on the characteristic information associated with a node at a lower level of the octum tree.

[0087] The LOD generation unit 40009 in the embodiment generates LOD (Level of Detail). The LOD in the embodiment indicates the level of detail of the point cloud content; a smaller LOD value indicates lower detail of the point cloud content, while a larger LOD value indicates higher detail of the point cloud content. Points can be classified by LOD.

[0088] The lift conversion unit 40010 in this embodiment performs lift conversion coding, which converts the characteristics of the point cloud based on weighted values. As described above, the lift conversion coding is applied selectively.

[0089] The coefficient quantization unit 40011 in the embodiment quantizes the feature-coded feature based on the coefficient.

[0090] The arithmetic encoder 40012 according to the embodiment encodes the quantized characteristics based on arithmetic coding.

[0091] The elements of the point cloud video encoder in Figure 4, although not shown, are embodied in hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud provider. One or more processors can perform any one of the operations and / or functions of the elements of the point cloud video encoder in Figure 4 described above. One or more processors can also operate or execute a set of software programs and / or instructions for performing the operations and / or functions of the elements of the point cloud video encoder in Figure 4. One or more memories in the embodiment include high-speed random-access memory or non-volatile memory (e.g., one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0092] Figure 5 shows an example of a voxel according to an embodiment.

[0093] Figure 5 shows voxels located in a three-dimensional space represented by a coordinate system consisting of three axes: the X, Y, and Z axes. As shown in Figure 4, a point cloud video encoder (e.g., quantization unit 40001) performs voxelization. A voxel represents a three-dimensional cubic space that arises when three-dimensional space is divided into units (unit = 1.0) based on the axes that represent three-dimensional space (e.g., X, Y, Z axes). Figure 5 shows two poles (0,0,0) and (2 d , 2 d , 2 d This shows an example of a voxel generated by an octree structure that recursively subdivides a cubical axis-aligned bounding box defined by ). Each voxel contains at least one point. The spatial coordinates of a voxel can be estimated from its positional relationship to a group of voxels. As mentioned above, voxels have properties (such as hue or reflectivity) similar to pixels in a 2D image / video. A detailed explanation of voxels is provided in Figure 4 and will be omitted here.

[0094] Figure 6 shows an example of an octree and occupancy code according to an embodiment.

[0095] As shown in Figures 1 to 4, the point cloud content provision system (point cloud video encoder 10002) or the octree analysis unit 40002 of the point cloud video encoder performs octree geometry coding (or octree coding) on ​​an octree structure basis in order to efficiently manage the area and / or position of voxels.

[0096] The upper side of FIG. 6 shows an octree structure. The three-dimensional space of the point cloud content according to the embodiment is represented by the axes of the coordinate system (for example, the X-axis, Y-axis, and Z-axis). The octree structure is recursively subdivided from a bounding box (cubical axis-aligned bounding box) defined by two extreme points (0, 0, 0) and (2 d 、2 d 、2 d ). 2d is set to the value that constitutes the smallest bounding box surrounding all the points of the point cloud content (or point cloud video). The d value is determined by Equation 1 below. In Equation 1 below, (x int n 、y int n 、z int n ) indicates the position (or position value) of the quantized point.

[0097]

Equation

[0098] As shown in the upper center of FIG. 6, by the division, the entire three-dimensional space is divided into eight spaces. Each of the divided spaces is represented by a cube having six faces. As shown in the upper right of FIG. 6, the eight spaces are each further divided by the axes of the coordinate system (for example, the X-axis, Y-axis, and Z-axis). Therefore, each space is again divided into eight small spaces. The divided small spaces are also represented by cubes having six faces. Such a division method is applied until the leaf nodes of the octree become voxels.

[0099] The lower part of Figure 6 shows the occupancy code of an octree. The occupancy code of an octree is generated to indicate whether each of the eight partitioned spaces resulting from the division of a single space contains at least one point. Therefore, one occupancy code is represented by eight child nodes. Each child node indicates the occupancy of the partitioned space, and each child node has a 1-bit value. Thus, the occupancy code is represented by an 8-bit code. That is, if the space corresponding to a child node contains at least one point, the node has a value of 1. If the space corresponding to a node does not contain a point (empty), the node has a value of 0. The occupancy code shown in Figure 6 is 00100001, which indicates that the spaces corresponding to the third and eighth child nodes each contain at least one point. As shown in the diagram, the third and eighth child nodes each have eight child nodes, and each child node is represented by an 8-bit occupancy code. The diagram shows that the occupation code of the third child node is 10000111 and the occupation code of the eighth child node is 01001111. The point cloud video encoder according to the embodiment (e.g., arithmetic encoder 40004) can entropy encode the occupation codes. In addition, to improve compression efficiency, the point cloud video encoder can intra / intercode the occupation codes. The receiving device according to the embodiment (e.g., receiving device 10004 or point cloud video decoder 10006) reconstructs the octree based on the occupation codes.

[0100] The point cloud video encoder according to the embodiment (e.g., the octree analysis unit 40002) performs voxelization and octree coding to store the position of the points. However, since points in 3D space are not always uniformly distributed, there may be certain regions where there are few points. Therefore, performing voxelization on the entire 3D space is inefficient. For example, if there are almost no points in a particular region, there is no need to perform voxelization on that region.

[0101] Therefore, the point cloud video encoder according to the embodiment does not perform voxelization for the specific region (or nodes excluding leaf nodes of the octree) mentioned above, and performs direct coding, which directly codes the positions of points included in the specific region. The coordinates of the points directly coded in the embodiment are called Direct Coding Mode (DCM). The point cloud video encoder according to the embodiment can also perform trisoup geometry encoding, which reconstructs the positions of points within a specific region (or node) based on voxels, based on a surface model. Trisoup geometry encoding is a geometry encoding that represents an object as a series of triangle meshes. Therefore, the point cloud video decoder can generate a point cloud from a mesh surface. Direct coding and trisoup geometry encoding according to the embodiment are performed selectively. Direct coding and trisoup geometry encoding according to the embodiment can also be performed in combination with octree geometry coding (or octree coding).

[0102] For direct coding to be performed, the option to use direct mode for applying direct coding must be activated, and the node to which direct coding is applied must not be a leaf node, but rather a specific node containing points below a certain threshold. Furthermore, the total number of points to be directly coded must not exceed a predetermined threshold. When the above conditions are met, the point cloud video encoder according to the embodiment (e.g., computation encoder 40004) can entropy code the position (or position value) of the points.

[0103] The point cloud video encoder according to the embodiment (e.g., surface approximation analysis unit 40003) can define a specific level of the octree (the level being less than the depth d of the octree) and perform trisoup geometry coding from that level using a surface model to reconstruct the positions of points within the node region based on voxels (trisoup mode). The point cloud video encoder according to the embodiment can specify the level to which trisoup geometry coding is applied. For example, if the specified level is the same as the depth of the octree, the point cloud video encoder will not operate in trisoup mode. That is, the point cloud video encoder according to the embodiment can only operate in trisoup mode if the specified level is less than the depth value of the octree. The three-dimensional cubic region of the nodes at a specified level according to the embodiment is called a block. A block contains one or more voxels. A block or voxel can also correspond to a brick. Within each block, the geometry is represented as a surface. The surface according to the embodiment can intersect each edge of the block at most once.

[0104] Since a block has 12 edges, there are at least 12 intersections within a block. Each intersection is called a vertex. A vertex along an edge is detected if, among all the blocks sharing that edge, there is at least one occupied voxel adjacent to that edge. In this embodiment, an occupied voxel means a voxel containing a point. The position of a vertex detected along an edge is the average position along the edge of all voxels adjacent to that edge among all the blocks sharing that edge.

[0105] When a vertex is detected, the point cloud video encoder according to the embodiment can entropy encode the edge's starting point (x, y, z), the edge's direction vector (Δx, Δy, Δz), and the vertex's position value (relative position value within the edge). When trisoup geometry coding is applied, the point cloud video encoder according to the embodiment (e.g., geometry reconstruction unit 40005) can perform triangle reconstruction, up-sampling, and voxelization processes to generate the reconstructed geometry.

[0106] Vertices located at the edges of a block determine the surface that passes through the block. The surface in the embodiment is a non-planar polygon. In the process of triangle reconstruction, the surface represented by triangles is reconstructed based on the starting point of the edge, the direction vector of the edge, and the position value of the vertex. The process of triangle reconstruction is as shown in Equation 2 below. In Equation 2, (1) the centroid value of each vertex is calculated, (2) the value obtained by subtracting the centroid value from the value of each vertex is (3) squared, and all the resulting values ​​are added together to obtain a value.

[0107]

number

[0108] Next, the minimum value of the added elements is found, and a projection process is performed along an axis where the minimum value is located. For example, if the x-element is the minimum, each vertex is projected onto the x-axis with respect to the center of the block, and then projected onto the (y,z) plane. If the value obtained by projecting onto the (y,z) plane is (ai,bi), the θ value is found using atan2(bi,ai), and the vertices are aligned based on the θ value. Table 1 below shows the combinations of vertices for generating triangles depending on the number of vertices. The vertices are aligned sequentially from 1 to n. Table 1 below shows that for four vertices, two triangles are constructed by the combination of vertices. The first triangle is composed of the 1st, 2nd, and 3rd vertices from the aligned vertices, and the second triangle is composed of the 3rd, 4th, and 1st vertices from the aligned vertices.

[0109] Table 1.Triangles formed from vertices ordered 1,…,n

[0110] [Table 1]

[0111] The upsampling process is performed to voxelize by adding intermediate points along the edges of a triangle. Additional points are generated based on the upsampling factor and the block width. These additional points are called refined vertices. The point cloud video encoder in this example can voxelize the refined vertices. Furthermore, the point cloud video encoder can perform feature coding based on the voxelized positions (or position values).

[0112] Figure 7 shows an example of an adjacent node pattern according to the embodiment.

[0113] To increase the compression efficiency of point cloud video, the point cloud video encoder in this embodiment performs entropy coding based on context adaptive arithmetic coding.

[0114] As explained in Figures 1 to 6, the point cloud content delivery system or the point cloud video encoder 10002 in Figure 2 or the point cloud video encoder or arithmetic encoder 40004 in Figure 4 can immediately entropy encode the occupied code. Furthermore, the point cloud content delivery system or point cloud video encoder can perform entropy encoding (intra encoding) based on the occupied code of the current node and the occupied rate of neighboring nodes, or entropy encoding (inter encoding) based on the occupied code of a previous frame. In the embodiment, a frame refers to a set of point cloud videos generated at the same time. The compression efficiency of intra encoding / inter encoding in the embodiment varies depending on the number of neighboring nodes referenced. While it becomes more complex as the number of bits increases, the compression efficiency can be improved by tilting it to one side. For example, with a 3-bit context, there are 8 ways to encode (2 to the power of 3). The parts that are coded separately affect the complexity of the implementation. Therefore, it is necessary to match the appropriate level of compression efficiency with the complexity.

[0115] Figure 7 shows the process of determining the occupancy pattern based on the occupancy rate of neighboring nodes. The point cloud video encoder in this example determines the occupancy rate of neighboring nodes for each node in the octvine and obtains a neighbor pattern value. The neighbor pattern is used to infer the occupancy pattern of the node in question. The left side of Figure 7 shows the cube corresponding to the node (the cube located in the middle) and the six cubes (neighboring nodes) that share at least one face with that cube. The illustrated nodes are nodes of the same depth. The illustrated numbers indicate the weight values ​​(1, 2, 4, 8, 16, 32, etc.) associated with each of the six nodes. Each weight value is assigned sequentially according to the position of the neighboring node.

[0116] The right side of Figure 7 shows the neighboring node pattern value. The neighboring node pattern value is the sum of the weighted values ​​of the occupied neighboring nodes (neighboring nodes with points). Therefore, the neighboring node pattern value can range from 0 to 63. A neighboring node pattern value of 0 means that none of the neighboring nodes of that node have points (occupied nodes). A neighboring node pattern value of 63 means that all neighboring nodes are occupied nodes. As shown in the figure, neighboring nodes assigned weights of 1, 2, 4, and 8 are occupied nodes, so the neighboring node pattern value is 15, which is the sum of 1, 2, 4, and 8. The point cloud video encoder can perform coding based on the neighboring node pattern value (for example, if the neighboring node pattern value is 63, it will perform 64 codings). In the embodiment, the point cloud video encoder can reduce the complexity of coding by changing the neighboring node pattern value (for example, based on a table that changes 64 to 10 or 6).

[0117] Figure 8 shows an example of the point configuration for each LOD according to the embodiment.

[0118] As explained in Figures 1 through 7, before feature coding is performed, the coded geometry is reconstructed (restored). When direct coding is applied, the geometry reconstruction operation involves changing the placement of the directly coded points (for example, placing the directly coded points in front of the point cloud data). When trisoup geometry coding is applied, the geometry reconstruction process involves triangulation reconstruction, upsampling, and voxelization, and since features are dependent on geometry, feature coding is performed based on the reconstructed geometry.

[0119] The point cloud video encoder (e.g., LOD generation unit 40009) classifies points according to their LOD. The diagram shows the point cloud content corresponding to the LODs. In the diagram, the left side shows the original point cloud content. In the diagram, the second from the left shows the point distribution for the lowest LOD, and the far right shows the point distribution for the highest LOD. That is, the points for the lowest LOD have a sparse distribution, and the points for the highest LOD have a fine distribution. In other words, as the LOD increases along the direction of the arrow at the bottom of the diagram, the spacing (or distance) between points becomes shorter.

[0120] Figure 9 shows an example of the point configuration for each LOD according to the embodiment.

[0121] As explained in Figures 1 to 8, the point cloud content delivery system or point cloud video encoder (for example, the point cloud video encoder 10002 in Figure 2, or the point cloud video encoder or LOD generation unit 40009 in Figure 4) generates LODs. The LOD is generated by rearranging points in a set of refinement levels according to a set LOD distance value (or a set of Euclidean distances). The LOD generation process is performed not only by the point cloud video encoder but also by the point cloud video decoder.

[0122] The upper part of Figure 9 shows an example of points (P0-P9) in a point cloud content distributed in 3D space. The original order in Figure 9 shows the order of points P0-P9 before LOD generation. The LOD-based order in Figure 9 shows the order of points after LOD generation. Points are rearranged according to LOD. Also, higher LODs include points belonging to lower LODs. As shown in Figure 9, LOD0 includes P0, P5, P4, and P2. LOD1 includes the points of LOD0 and P1, P6, and P3. LOD2 includes the points of LOD0 and LOD1 and P9, P8, and P7.

[0123] As explained in Figure 4, the point cloud video encoder according to the embodiment can selectively or in combination perform LOD-based predictive transformation coding, lift transformation coding, and RAHT transformation coding.

[0124] The point cloud video encoder according to the embodiment performs LOD-based predictive transformation coding to generate predictors for points and set predictive characteristics (or predictive characteristic values) for each point. That is, N predictors are generated for N points. The predictors according to the embodiment can calculate a weighted value (=1 / distance) based on the LOD value of each point, index information for neighboring points within a distance set for each LOD, and the distance value to the neighboring point.

[0125] In the embodiment, the predicted characteristic (or characteristic value) is set as the average value obtained by multiplying the characteristics (or characteristic values, e.g., hue, reflectance, etc.) of adjacent points set in the predictor of each point by a weight (or weight value) calculated based on the distance to each adjacent point. The point cloud video encoder (e.g., coefficient quantization unit 40011) in the embodiment can quantize and inverse quantize the residual value (also called residual characteristic, residual characteristic value, characteristic prediction residual value, prediction error characteristic value, etc.) of the point obtained by subtracting the predicted characteristic (characteristic value) from the characteristic of the point (i.e., original characteristic value). The quantization process performed at the transmitter on the residual characteristic value is shown in Table 2. The inverse quantization process performed at the receiver on the residual characteristic value quantized as shown in Table 2 is shown in Table 3.

[0126] [Table 2]

[0127] [Table 3]

[0128] The point cloud video encoder according to the embodiment (e.g., computation encoder 40012) entropy codes the quantized and dequantized residual values ​​as described above, if there are adjacent points to the predictor of each point. 1) An array QW (QuantizationWeight) is generated to store the weight values ​​of each point. The initial value of all elements of QW is 1.0. The QW value of the predictor index of the adjacent node registered with the predictor is multiplied by the weight value of the predictor of the current point and added.

[0129] 2) Lift prediction process: To calculate the predicted attribute value, subtract the weighted value obtained by multiplying the attribute value of the point by a weighted value from the existing attribute value.

[0130] 3) Create temporary arrays named updateweight and update, and initialize these temporary arrays to 0.

[0131] 4) The weight calculated for all predictors is further multiplied by the weight stored in the QW corresponding to the predictor index, and the resulting weight is accumulated and summed in the update weight array as the index of the adjacent node. The update array is then accumulated and summed by multiplying the characteristic value of the index of the adjacent node by the calculated weight.

[0132] 5) Lift update process: For all predictors, divide the characteristic value of the update array by the weight value of the update weight array of the predictor index, and add the existing characteristic value back to the divided value.

[0133] 6) For all predictors, the predicted characteristic value is calculated by further multiplying the characteristic value updated in the lift update process by the weighted value updated in the lift prediction process (stored in QW). The point cloud video encoder according to the embodiment (e.g., coefficient quantization unit 40011) quantizes the predicted characteristic value. The point cloud video encoder (e.g., computation encoder 40012) entropy codes the quantized characteristic value.

[0134] The point cloud video encoder in the embodiment (e.g., RAHT transformation unit 40008) performs RAHT transformation coding, which uses characteristics associated with lower-level nodes of an octree to predict characteristics of higher-level nodes. RAHT transformation coding is an example of characteristic intracoding by an octree backward scan. The point cloud video encoder in the embodiment scans from voxels to the entire area, and each step repeats a merging process up to the root node, aligning the voxels with larger blocks. The merging process in the embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes, but on the node immediately above the empty node.

[0135]

number

[0136]

number

[0137] The gDC value, like the high-pass coefficient, is quantized and entropy-coded.

[0138] Figure 10 shows an example of a Point Cloud Video Decoder according to an embodiment.

[0139] The point cloud video decoder shown in Figure 10 is an example of the point cloud video decoder 10006 shown in Figure 1, and performs the same or similar operation as the point cloud video decoder 10006 described in Figure 1. As illustrated, the point cloud video decoder receives geometry bitstreams and attribute bitstreams contained in one or more bitstreams. The point cloud video decoder includes a geometry decoder and an attribute decoder. The geometry decoder performs geometry decoding on the geometry bitstream and outputs the decoded geometry. The attribute decoder performs attribute decoding on the attribute bitstream based on the decoded geometry and outputs the decoded attributes. The decoded geometry and decoded attributes are used to decode the point cloud content.

[0140] Figure 11 shows an example of a point cloud video decoder according to an embodiment.

[0141] The point cloud video decoder shown in Figure 11 is an example of the point cloud video decoder described in Figure 10, and performs the decoding operation, which is the reverse process of the encoding operation of the point cloud video encoder described in Figures 1 to 9.

[0142] As explained in Figures 1 and 10, the point cloud video decoder performs geometry decoding and feature decoding. Geometry decoding is performed before feature decoding.

[0143] The point cloud video decoder according to the embodiment includes an arithmetic decoder (11000), an octree synthesis unit (11001), a surface approximation synthesis unit (11002), a geometry reconstruction unit (11003), a coordinates inverse transformation unit (11004), an arithmetic decoder (11005), an inverse quantization unit (11006), an RAHT transformation unit (11007), an LOD generation unit (11008), an inverse lifting unit (11009), and / or a color inverse transformation unit (11010).

[0144] The computation decoder 11000, octree synthesis unit 11001, surface approximation synthesis unit 11002, geometry reconstruction unit 11003, and coordinate system inverse transformation unit 11004 perform geometry decoding. Geometry decoding according to the embodiment includes direct decoding and trisoup geometry decoding. Direct decoding and trisoup geometry decoding are applied selectively. Furthermore, geometry decoding is not limited to the above examples and is performed by the inverse process of geometry coding described in Figures 1 to 9.

[0145] The arithmetic decoder 11000 in this embodiment decodes the received geometry bitstream based on arithmetic coding. The operation of the arithmetic decoder 11000 corresponds to the inverse process of the arithmetic encoder 40004.

[0146] The octree synthesis unit 11001 in this embodiment obtains an octree by acquiring an octree code from the decoded geometry bitstream (or the decoded result, information about the allocated geometry). A detailed explanation of the octree code is shown in Figures 1 to 9.

[0147] In the embodiment, the surface approximation synthesis unit 11002 synthesizes surfaces based on the decoded geometry and / or the generated octree when trisoup geometry coding is applied.

[0148] The geometry reconstruction unit 11003 in the embodiment regenerates the geometry based on the surface and / or decoded geometry. As explained in Figures 1 to 9, direct coding and trisoup geometry coding are applied selectively. Therefore, the geometry reconstruction unit 11003 directly retrieves and adds the positional information of points to which direct coding has been applied. Also, when trisoup geometry coding is applied, the geometry reconstruction unit 11003 restores the geometry by performing the reconstruction operations of the geometry reconstruction unit 40005, such as triangle reconstruction, upsampling, and voxelization operations. The specific details are explained in Figure 6 and will be omitted here. The restored geometry includes a point cloud picture or frame that does not contain any features.

[0149] The coordinate system inverse transformation unit 11004 in this embodiment transforms the coordinate system based on the restored geometry to obtain the position of the point.

[0150] The computation decoder 11005, inverse quantization unit 11006, RAHT transform unit 11007, LOD generation unit 11008, inverse lift unit 11009, and / or inverse color transform unit 11010 perform characteristic decoding as described in Figure 10. Characteristic decoding according to this embodiment includes RAHT (Region Adaptive Hierarchical Transform) decoding, prediction transform (Interpolaration-based hierarchical nearest-neighbour prediction-Prediction Transform) decoding, and lift transform (interpolaration-based hierarchical nearest-neighbour prediction with an update / lifting step (Lifting Transform)) decoding. These three decoding methods may be used selectively, or a combination of one or more decoding methods may be used. Furthermore, characteristic decoding according to this embodiment is not limited to the examples described above.

[0151] The arithmetic decoder 11005 according to the embodiment decodes the characteristic bitstream into arithmetic coding.

[0152] The inverse quantization unit 11006 in this embodiment inverse quantizes the decoded characteristic bitstream or the information about the characteristics obtained as a result of decoding, and outputs the inversely quantized characteristics (or characteristic values). Inverse quantization is selectively applied based on the characteristic encoding of the point cloud video encoder.

[0153] In the embodiment, the RAHT conversion unit 11007, the LOD generation unit 11008, and / or the inverse lift unit 11009 process the reconstructed geometry and the inversely quantized properties. As described above, the RAHT conversion unit 11007, the LOD generation unit 11008, and / or the inverse lift unit 11009 selectively perform the corresponding decoding operation by encoding with a point cloud video encoder.

[0154] The color inverse conversion unit 11010 in this embodiment performs inverse conversion coding to inversely convert the color values ​​(or textures) contained in the decoded characteristics. The operation of the color inverse conversion unit 11010 is selectively performed based on the operation of the color conversion unit 40006 of the point cloud video encoder.

[0155] The elements of the point cloud video decoder in Figure 11, although not shown, are embodied in hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud provider. One or more processors perform any of the operation and / or function of the point cloud video decoder elements in Figure 11 as described above. One or more processors also operate or execute a set of software programs and / or instructions for performing the operation and / or function of the point cloud video decoder elements in Figure 11.

[0156] Figure 12 shows an example of a transmitting device according to an embodiment.

[0157] The transmitting device shown in Figure 12 is an example of the transmitting device 10000 in Figure 1 (or the point cloud video encoder in Figure 4). The transmitting device shown in Figure 12 performs any operation and method that is the same as or similar to the operation and encoding method of the point cloud video encoder described in Figures 1 to 9. The transmitting device according to the embodiment includes a data input unit 12000, a quantization processing unit 12001, a voxelization processing unit 12002, an octree occupy code generation unit 12003, a surface model processing unit 12004, an intra / intercoding processing unit 12005, an arithmetic coder 12006, a metadata processing unit 12007, a hue conversion processing unit 12008, a feature conversion processing unit (or attribute conversion processing unit) 12009, a prediction / lift / RAHT conversion processing unit 12010, an arithmetic coder 12011 and / or a transmitting processing unit 12012.

[0158] The data input unit 12000 in this embodiment receives or acquires point cloud data. The data input unit 12000 performs the same or similar operations and / or acquisition methods as the operation and / or acquisition method of the point cloud video acquisition unit 10001 (or the acquisition process 20000 shown in Figure 2).

[0159] The data input unit 12000, quantization processing unit 12001, voxelization processing unit 12002, octree occupy code generation unit 12003, surface model processing unit 12004, intra / intercoding processing unit 12005, and arithmetic coder 12006 perform geometry coding. The geometry coding according to the embodiment is the same as or similar to the geometry coding described in Figures 1 to 9, so a detailed explanation is omitted.

[0160] The quantization processing unit 12001 in this embodiment quantizes geometry (for example, the position value of a point or position value). The operation and / or quantization of the quantization processing unit 12001 is the same as or similar to the operation and / or quantization of the quantization unit 40001 shown in Figure 4. A detailed explanation is provided in Figures 1 to 9.

[0161] The voxelization processing unit 12002 in this embodiment voxels the position values ​​of the quantized points. The voxelization processing unit 120002 performs the same or similar operations and / or processes as the quantization unit 40001 and / or voxelization process shown in Figure 4. A detailed explanation is provided in Figures 1 to 9.

[0162] The octree-occupied code generation unit 12003 in this embodiment performs octree coding based on the octree structure of the voxelized point locations. The octree-occupied code generation unit 12003 generates occupied codes. The octree-occupied code generation unit 12003 performs the same or similar operations and / or methods as the point cloud encoder (or octree analysis unit 40002) described in Figures 4 and 6. A detailed explanation is provided in Figures 1 to 9.

[0163] The surface model processing unit 12004 in this embodiment performs trisoup geometry coding to reconstruct the positions of points within a specific region (or node) on a voxel basis based on a surface model. The surface model processing unit 12004 performs the same or similar operations and / or methods as the point cloud video encoder (e.g., surface approximation analysis unit 40003) shown in Figure 4. A detailed explanation is provided in Figures 1 to 9.

[0164] In this embodiment, the intra / intercoding processing unit 12005 performs intra / intercoding of point cloud data. The intra / intercoding processing unit 12005 performs coding that is the same as or similar to the intra / intercoding described in Figure 7. A detailed explanation is provided in Figure 7. In this embodiment, the intra / intercoding processing unit 12005 is included in the arithmetic coder 12006.

[0165] The arithmetic coder 12006 according to the embodiment entropy encodes the octree and / or approximated octree of point cloud data. For example, the encoding scheme includes an arithmetic encoding method. The arithmetic coder 12006 performs the same or similar operations and / or methods as the arithmetic encoder 40004.

[0166] The metadata processing unit 12007 in the embodiment processes metadata related to point cloud data, such as setting values, and provides it to necessary processing processes such as geometry coding and / or characteristic coding. The metadata processing unit 12007 in the embodiment also generates and / or processes signaling information related to geometry coding and / or characteristic coding. The signaling information in the embodiment is coded separately from geometry coding and / or characteristic coding. The signaling information in the embodiment may also be interleaved.

[0167] The hue conversion processing unit 12008, the characteristic conversion processing unit 12009, the prediction / lift / RAHT conversion processing unit 12010, and the arithmetic coder 12011 perform characteristic coding. The characteristic coding according to the embodiment is the same as or similar to the characteristic coding described in Figures 1 to 9, so a detailed explanation is omitted.

[0168] The hue conversion processing unit 12008 in this embodiment performs hue conversion coding to convert the hue values ​​included in the characteristics. The hue conversion processing unit 12008 performs hue conversion coding based on the reconstructed geometry. The reconstructed geometry is described in Figures 1 to 9. It also performs the same or similar operation and / or method as the color conversion unit 40006 described in Figure 4. A detailed explanation is omitted.

[0169] The characteristic transformation processing unit 12009 in this embodiment performs characteristic transformation based on positions where geometry coding has not been performed and / or the reconstructed geometry. The characteristic transformation processing unit 12009 performs the same or similar operations and / or methods as the characteristic transformation unit 40007 described in Figure 4. A detailed explanation is omitted. The prediction / lift / RAHT transformation processing unit 12010 in this embodiment codes the transformed characteristics using one or a combination of RAHT coding, prediction transformation coding, and lift transformation coding. The prediction / lift / RAHT transformation processing unit 12010 performs one or a similar operations as the RAHT transformation unit 40008, LOD generation unit 40009, and lift transformation unit 40010 described in Figure 4. Furthermore, the explanations regarding prediction transformation coding, lift transformation coding, and RAHT transformation coding are as described in Figures 1 to 9, so a detailed explanation is omitted.

[0170] The arithmetic coder 12011 according to the embodiment encodes the coded characteristics based on the arithmetic coding. The arithmetic coder 12011 performs the same or similar operations and / or methods as the arithmetic encoder 40012.

[0171] The transmission processing unit 12012 in the embodiment transmits each bitstream containing encoded geometry and / or encoded attributes and metadata information, or transmits the encoded geometry and / or encoded attributes and metadata information as a single bitstream. When the encoded geometry and / or encoded attributes and metadata information in the embodiment is composed of a single bitstream, the bitstream contains one or more sub-bitstreams. The bitstream in the embodiment contains signaling information including SPS (Sequence Parameter Set) for sequence-level signaling, GPS (Geometry Parameter Set) for geometry information coding signaling, APS (Attribute Parameter Set) for attribute information coding signaling, and TPS (Tile Parameter Set) for tile-level signaling, as well as slice data. The slice data contains information about one or more slices. One slice in the embodiment is one geometry bitstream (Geom0 0 ) and one or more attribute bitstreams (Attr0 0 , Attr1 0) is included. The TPS in the embodiment includes information about each tile for one or more tiles (e.g., coordinate values ​​and height / size information of the bounding box). The geometry bitstream includes a header and a payload. The header of the geometry bitstream in the embodiment includes identification information of the parameter set included in the GPS (geom_parameter_set_id), tile identifier (geom_tile_id), slice identifier (geom_slice_id), and information about the data included in the payload. As described above, the metadata processing unit 12007 in the embodiment can generate and / or process signaling information and transmit it to the transmission processing unit 12012. In the embodiment, the elements that perform geometry coding and the elements that perform characteristic coding can share data / information with each other, as shown by the dotted lines. The transmission processing unit 12012 in the embodiment performs the same or similar operation and / or transmission method as the operation and / or transmission method of the transmitter 10003. A detailed explanation is provided in Figures 1 and 2, so it is omitted here.

[0172] Figure 13 shows an example of a receiving device according to an embodiment.

[0173] The receiving device shown in Figure 13 is an example of the receiving device 10004 in Figure 1 (or the point cloud video decoder in Figures 10 and 11). The receiving device shown in Figure 13 performs any operation and decoding method that is the same as or similar to that of the point cloud video decoder described in Figures 1 to 11.

[0174] The receiving device according to the embodiment includes a receiving unit 13000, a receiving processing unit 13001, an arithmetic decoder 13002, an occupancy code-based octree reconstruction processing unit 13003, a surface model processing unit (triangle reconstruction, upsampling, voxelization) 13004, an inverse quantization processing unit 13005, a metadata analysis unit 13006, an arithmetic decoder 13007, an inverse quantization processing unit 13008, a prediction / lift / RAHT inverse conversion processing unit 13009, a hue inverse conversion processing unit 13010, and / or a renderer 13011. Each component of the decoding according to the embodiment performs the reverse process of the component of the encoding according to the embodiment.

[0175] The receiver 13000 in this embodiment receives point cloud data. The receiver 13000 performs the same or similar operation and / or reception method as the receiver 10005 in Figure 1. A detailed explanation is omitted.

[0176] The receiving processing unit 13001 in this embodiment obtains a geometry bitstream and / or characteristic bitstream from the received data. The receiving processing unit 13001 is included in the receiving unit 13000.

[0177] The arithmetic decoder 13002, the octree reconstruction processing unit 13003 of the occupied code base, the surface model processing unit 13004, and the inverse quantization processing unit 13005 perform geometry decoding. The geometry decoding according to the embodiment is the same as or similar to the geometry decoding described in Figures 1 to 10, so a detailed explanation is omitted.

[0178] The arithmetic decoder 13002 in this embodiment decodes the geometry bitstream based on arithmetic coding. The arithmetic decoder 13002 performs the same or similar operations and / or coding as the arithmetic decoder 11000.

[0179] The octree reconstruction processing unit 13003 of the occupied code base according to the embodiment obtains the occupied code from the decoded geometry bitstream (or the decoding result, information about the allocated geometry) and reconstructs the octree. The octree reconstruction processing unit 13003 of the occupied code base performs the same or similar operations and / or methods as the octree synthesis unit 11001 and / or the octree generation method. When trisoup geometry coding is applied, the surface model processing unit 13004 according to the embodiment performs trisoup geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on the surface model scheme. The surface model processing unit 13004 performs the same or similar operations as the surface approximation synthesis unit 11002 and / or the geometry reconstruction unit 11003.

[0180] The inverse quantization processing unit 13005 in this embodiment inverse quantizes the decoded geometry.

[0181] The metadata analysis 13006 in this embodiment analyzes metadata contained in the received point cloud data, such as setting values. The metadata analysis 13006 transmits the metadata to geometry decoding and / or characteristic decoding. A detailed explanation of the metadata is shown in Figure 12 and will therefore be omitted here.

[0182] The arithmetic decoder 13007, the inverse quantization processing unit 13008, the prediction / lift / RAHT inverse conversion processing unit 13009, and the hue inverse conversion processing unit 13010 perform characteristic decoding. Since the characteristic decoding is the same as or similar to the characteristic decoding explained in Figure 1 or Figure 10, a detailed explanation is omitted.

[0183] The arithmetic decoder 13007 in this embodiment decodes the characteristic bitstream into arithmetic coding. The arithmetic decoder 13007 decodes the characteristic bitstream based on the reconstructed geometry. The arithmetic decoder 13007 performs the same or similar operations and / or coding as the arithmetic decoder 11005.

[0184] The inverse quantization processing unit 13008 in this embodiment inverse quantizes the decoded characteristic bitstream. The inverse quantization processing unit 13008 performs the same or similar operations and / or methods as the inverse quantization unit 11006 and / or the inverse quantization method.

[0185] The prediction / lift / RAHT inverse conversion processing unit 13009 in the embodiment processes the reconstructed geometry and inversely quantized properties. The prediction / lift / RAHT inverse conversion processing unit 13009 performs one of the operations and / or decoding that are the same as or similar to the operations and / or decoding of the RAHT conversion unit 11007, the LOD generation unit 11008 and / or the inverse lift unit 11009. The hue inverse conversion processing unit 13010 in the embodiment performs inverse conversion coding to inversely convert the color values ​​(or textures) contained in the decoded properties. The hue inverse conversion processing unit 13010 performs the same as or similar operations and / or inverse conversion coding of the color inverse conversion unit 11010. The renderer 13011 in the embodiment renders the point cloud data.

[0186] Figure 14 shows an example of a structure that can be linked to a point cloud data transmission / reception method / device according to an embodiment.

[0187] The structure in Figure 14 shows a configuration in which one of the following is connected to the cloud network 17100: server 17600, robot 17100, autonomous vehicle 17200, XR device 17300, smartphone 17400, home appliance 17500, and / or HMD (Head-Mount Display) 17700. The robot 17100, autonomous vehicle 17200, XR device 17300, smartphone 17400, or home appliance 17500 are also referred to as devices. The XR device 17300 corresponds to or is linked to a point cloud data (PCC) device according to the embodiment.

[0188] Cloud network 17000 refers to a network that constitutes part of the cloud computing infrastructure or resides within the cloud computing infrastructure. Here, cloud network 17000 is configured using a 3G network, 4G or LTE network, or 5G network, etc.

[0189] Server 17600 is connected to any of the following via cloud network 17000: robot 17100, autonomous vehicle 17200, XR device 17300, smartphone 17400, home appliance 17500, and / or HMD 17700, and can assist in at least part of the processing of the connected devices 17100 to 17700.

[0190] HMD (Head-Mount Display) 17700 refers to one of the types in which the XR device and / or PCC device according to the embodiment are embodied. The HMD type device according to the embodiment includes a communications unit, a control unit, a memory unit, an I / O unit, a sensor unit, and a power supply unit, etc.

[0191] The following describes various embodiments of the devices 17100 to 17500 to which the above technology is applied. Here, the devices 17100 to 17500 shown in Figure 14 can be linked / coupled with the point cloud data transmission and reception device according to the embodiments described above.

[0192] ​

[0193] The XR / PCC device 17300 can be implemented using PCC and / or XR (AR+VR) technology and can be embodied in HMDs (Head-Mount Displays), HUDs (Head-Up Displays) installed in vehicles, TVs, mobile phones, smartphones, computers, wearable devices, home appliances, digital signage, vehicles, stationary robots, and mobile robots.

[0194] The XR / PCC device 17300 can obtain information about the surrounding space or real-world objects by analyzing 3D point cloud data or image data acquired by various sensors or external devices to generate positional and characteristic data for 3D points, and then render and output an XR object to be output. For example, the XR / PCC device 17300 can output an XR object that includes additional information about the recognized object, corresponding to the recognized object.

[0195] ​

[0196] The autonomous vehicle 17200 will be realized as a mobile robot, vehicle, or unmanned aerial vehicle by applying PCC technology and XR technology.

[0197] The autonomous vehicle 17200 to which XR / PCC technology is applied refers to an autonomous vehicle equipped with means for providing XR images, or an autonomous vehicle that is the target of control / interaction within the XR images. In particular, the autonomous vehicle 17200 that is the target of control / interaction within the XR images can be separated from the XR device 17300 and can be linked together.

[0198] An autonomous vehicle 17200 equipped with means for providing XR / PCC images obtains sensor information from sensors including cameras and outputs XR / PCC images generated based on the obtained sensor information. For example, by providing a HUD and outputting XR / PCC images, the autonomous vehicle 17200 can provide the occupant with XR / PCC objects corresponding to real-world objects or objects on the screen.

[0199] In this case, when the XR / PCC object is output to the HUD, at least a portion of the XR / PCC object is output so as to overlap with the actual object that the occupant is looking at. On the other hand, when the XR / PCC object is output to a display installed in the autonomous vehicle, at least a portion of the XR / PCC object is output so as to overlap with the object on the screen. For example, the autonomous vehicle 1220 can output XR / PCC objects that correspond to objects such as roads, other vehicles, traffic lights, traffic signs, motorcycles, pedestrians, and buildings.

[0200] The VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and / or PCC (Point Cloud Compression) technologies described in the examples are applicable to a variety of devices.

[0201] In other words, VR technology is a display technology that provides real-world objects and backgrounds using only computer graphics (CG). On the other hand, AR technology is a technology that displays virtual CG images alongside images of real things. MR technology is similar to AR technology in that it mixes virtual objects with the real world. However, in AR technology, the distinction between real objects and virtual objects consisting of CG images is clear, and virtual objects are used in a way that complements real objects. In contrast, MR technology is distinguished from AR technology in that virtual and real objects are considered to have similar characteristics. More specifically, for example, hologram services are an application of the MR technology described above.

[0202] However, recently, rather than clearly distinguishing between VR, AR, and MR technologies, they are also referred to as XR (extended reality) technology. Therefore, the embodiments of the present invention are applicable to any of VR, AR, MR, and XR technologies. Such technologies utilize PCC, V-PCC, and G-PCC technology-based encoding / decoding.

[0203] The PCC method / device according to the embodiment can be applied to vehicles that provide autonomous driving services.

[0204] Vehicles providing autonomous driving services are connected to PCC devices via wired and wireless communication.

[0205] The Point Cloud Compressed Data (PCC) transceiver according to the embodiment, when wirelessly connected to a vehicle, can receive and process AR / VR / PCC service-related content data that can be provided along with autonomous driving services and transmit it to the vehicle. Furthermore, when the Point Cloud Data Transceiver is installed in a vehicle, it can receive and process AR / VR / PCC service-related content data based on user input signals received by a user interface device and provide it to the user. The vehicle or user interface device according to the embodiment receives the user input signals. The user input signals according to the embodiment include signals instructing autonomous driving services.

[0206] The point cloud data transmission method / apparatus according to the embodiment is interpreted as referring to the transmission device 10000, point cloud video encoder 10002, transmitter 10003 in Figure 1, the acquisition-encoding-transmission 20000-20001-20002 in Figure 2, the point cloud video encoder in Figure 4, the transmission device in Figure 12, the device in Figure 14, and the transmission device in Figure 42, etc.

[0207] The point cloud data receiving method / apparatus according to the embodiment is interpreted to refer to applications such as the receiving device 10004, receiver 10005, point cloud video decoder 10006 in Figure 1, the transmission-decoding-rendering 20002-20003-20004 in Figure 2, the decoder in Figure 10, the point cloud video decoder in Figure 11, the receiving device in Figure 13, the device in Figure 14, and the receiving device in Figure 43.

[0208] Furthermore, the point cloud data transmission / reception method / apparatus according to the embodiment may be simply referred to as the method / apparatus according to the embodiment.

[0209] In the embodiment, the geometry data, geometry information, and location information that constitute the point cloud data can be interpreted as having the same meaning as one another. The characteristic data, characteristic information, and attribute information that constitute the point cloud data can also be interpreted as having the same meaning as one another.

[0210] The method / apparatus described in the embodiment processes point cloud data with scalable transmission in mind.

[0211] The method / apparatus described in the embodiment describes a method for efficiently supporting situations where selective decoding of a portion of the data is necessary during the transmission and reception of point cloud data, depending on the receiver's performance or transmission speed.

[0212] To this end, this specification divides geometry and attribute data into semantic units such as geometry octrees and LoDs (level of detail), making it possible to select necessary information or remove unnecessary information on a bitstream basis.

[0213] Furthermore, this specification allows the receiving device to perform selective decoding or parallel decoding by transmitting geometry and characteristic data in multiple slices.

[0214] According to the embodiments, this specification subdivides a geometry bitstream and / or characteristic bitstream, and / or a point cloud bitstream structure in which the geometry bitstream and characteristic bitstream are multiplexed, and transmits the subdivided bitstream in slice units, thereby enabling selective decoding or parallel decoding by the receiving device.

[0215] This specification, based on embodiments, describes techniques for constructing a data structure composed of point clouds. More specifically, it proposes a method to reduce performance degradation that may occur due to division (i.e., segmentation) in the slice configuration and to alleviate the burden on the receiver. Referring to the point cloud data transmission / reception device (or simply encoder / decoder) according to embodiments, the point cloud data consists of a collection of points, each point consisting of geometry information (or geometry, geometry data) and characteristic information (or characteristic, characteristic data). The geometry information is the three-dimensional position information (x, y, z) of each point. That is, the position of each point is expressed by parameters on a coordinate system that represents three-dimensional space (for example, parameters of the three axes that represent space, namely the X, Y, and Z axes (x, y, z)). The characteristic information refers to the hue (RGB, YUV, etc.), reflectance, normal vectors, transparency, etc. of that point. Point Cloud Compression (PCC) efficiently compresses the unevenly distributed characteristics in three-dimensional space by performing octvine-based compression and then compressing characteristic information based on that. The point cloud video encoder and point cloud video decoder shown in Figures 4 and 11 process the operation according to the embodiment using their respective components.

[0216] According to the embodiment, the transmitting device compresses the geometric information (e.g., location) and characteristic information (e.g., hue / brightness / reflectivity, etc.) of the point cloud data and transmits them to the receiving device. At this time, the point cloud data is constructed using an octree structure or LOD (Level of Detail) with layers depending on the degree of detail, and based on these, scalable point cloud data coding and representation are possible. At this time, depending on the performance of the receiving device or the transmission speed, only a portion of the point cloud data can be decoded or represented, but currently there is no way to remove unnecessary data in advance.

[0217] In other words, when only a portion of an expandable point cloud compressed bitstream needs to be transmitted (for example, when decrypting only some layers of an expandable decryption), it is not possible to select and send only the necessary portion. Therefore, as shown in Figure 15, the transmitting device must decode the data and then re-encode the necessary portion, or as shown in Figure 16, the entire data must be transmitted to the receiving device, decoded by the receiving device, and then the necessary data must be selectively applied.

[0218] However, in the case of Figure 15, delays may occur due to the time required for decoding and re-encoding, and in the case of Figure 16, bandwidth efficiency decreases by transmitting unnecessary data to the receiving device, and when using a fixed bandwidth, there is a disadvantage in that the data quality must be reduced before transmission.

[0219] Therefore, to solve this problem, the method / apparatus according to the embodiment provides slicing to process the point cloud by dividing it into regions.

[0220] In particular, the methods / apparatus in the embodiments can use a method of transmitting slices separately, but if slices are coded independently, the compression efficiency may decrease due to a reduction in entropy continuity, etc. Therefore, the methods / apparatus in the embodiments propose a method for using the entropy between separated slices continuously. In this case, by transmitting information about the slices to be used continuously in advance, the resources of the receiving device can be managed efficiently. In addition, it is necessary to use information from other slices to refer to neighbors at the boundaries of separated slices, so in this case, a processing method at the slice boundary is proposed to ensure independence between slices for parallel processing and to improve compression efficiency. The apparatus and method in the embodiments propose a method for efficiently managing buffers in the receiving device when entropy continuity is given between multiple slices.

[0221] Furthermore, the methods / apparatus described in the examples define a slice subdivision structure for point cloud data and signal scalable layer and slice structures for scalable transmission.

[0222] Furthermore, this specification allows the receiving device to perform selective decoding or parallel decoding by transmitting geometry and characteristic data in multiple slices.

[0223] According to the embodiment, the method / apparatus segments (or divides, separates, or splits) one slice into multiple slices, and transmits the point cloud bitstream divided by the multiple segmented slices, thereby enabling the receiving device to support selective decoding or parallel decoding.

[0224] In this specification, a brick is defined as a higher-level concept than a segmented slice. According to the examples, a brick can be considered as a sub-tree of the tree that constitutes the input data. Each brick is a unit that includes the sub-tree depth of nodes occupied at a certain tree depth in the tree structure that constitutes the input data.

[0225] Figure 17 shows an example of a tree structure according to an embodiment in which one or more bricks are formed.

[0226] For example, in Figure 17, four subtrees are constructed using different bricks, with each subtree considering the node occupied at a tree depth of 2 as the root. For instance, a tree structure based on point cloud data (e.g., geometry data) can be divided into four bricks, represented as 50001, 50003, 50005, and 50007. The number of bricks in Figure 17 is an example to aid understanding, and the number of bricks is variable. Point cloud data belonging to each brick is transmitted by each data unit. In this case, each brick can be constructed independently, and in this case, parallel processing is possible. Also, one brick consists of one or more slices.

[0227] Figures 18(a) to 18(c) illustrate the matching relationship between a tree layer and a slice for transmitting point cloud data belonging to a single brick according to an embodiment.

[0228] The transmission method / device / encoder according to the embodiment can construct slices in finer units when dividing a point cloud bitstream into a slice structure. A slice is a data unit for detailed data representation.

[0229] For example, one or more octree layers are matched to a single slice.

[0230] In the transmission method / apparatus according to the embodiment, for example, the encoder scans the nodes (points) contained in the octree in the direction of scan order 51000 to construct a bitstream based on slice 51001. A slice may contain nodes at one or more levels in the octree structure, only nodes at a specific level, or only some nodes at a specific level, or only some nodes at one or more levels.

[0231] Figure 18(a) shows an example of an octree structure consisting of seven slices, where slice 51002 consists of nodes from level 0 to level 4. Slice 51003 consists of some nodes from level 5, slice 51004 consists of some nodes from level 5, and slice 51005 consists of yet another set of nodes from level 5. In other words, in Figure 18(a), level 5 is divided into three slices. Similarly, in Figure 18(a), level 6 (i.e., the leaf level) is also divided into three slices. In other words, a slice can be made up of some nodes from a particular level.

[0232] Figure 18(b) shows an example of an ocvine tree structure consisting of four slices: one slice is made up of nodes from level 0 to level 3 and some nodes from level 4; another slice is made up of the remaining nodes from level 4 and some nodes from level 5; yet another slice is made up of the remaining nodes from level 5 and some nodes from level 6; and yet another slice is made up of the remaining nodes from level 6.

[0233] Figure 18(c) shows an example of an octree structure consisting of five slices, with one slice made up of nodes from level 0 to level 3, and four slices made up of nodes from level 4 to level 6. That is, one slice is made up of some nodes from level 4, some nodes from level 5, and some nodes from level 6.

[0234] In other words, as shown in Figures 18(b) and 18(c), when multiple octree layers are matched to a single slice, only some nodes from each layer may be included. When multiple slices constitute a single geometry / characteristic frame in this way, the information necessary to construct the layers can be transmitted to the receiving device via signaling information. For example, the signaling information may include layer information contained in each slice and node information contained in each layer. In Figures 18(a) to 18(c), hollow circles (e.g., 51007) indicate coding units, and solid circles (e.g., 51008) indicate the end point of the slice (or the start point of the previous slice).

[0235] The encoder and corresponding device according to the embodiment encode point cloud data and generate and transmit a bitstream that further includes the encoded data and signaling information (or parameter information) related to the point cloud data.

[0236] Furthermore, a bitstream is generated based on the bitstream structure according to the embodiment. Therefore, the receiving device, decoder, and corresponding devices according to the embodiment can receive and parse a bitstream that conforms to the selective decoding structure of some data, and decode only some of the point cloud data to provide it efficiently.

[0237] The following describes scalable transmission of point cloud data.

[0238] The point cloud data transmission method / device according to the embodiment transmits a bitstream containing point cloud data in an expandable manner, and the point cloud data reception method / device according to the embodiment receives and decodes the bitstream in an expandable manner.

[0239] When the structures described in Figures 18(a) to 18(c) are used for expandable transmission, signaling information can be transmitted to the receiver to select the slices required by the receiver. Expandable transmission means transmitting or decoding only a portion of the bitstream, rather than the entire bitstream. Therefore, the receiver can provide low-resolution point cloud data (or content).

[0240] In the embodiment, when applying extensible transmission to an octree-based geometry bitstream, it is necessary to construct point cloud data using only the information up to a specific octree layer for each octree layer bitstream from the root node to the leaf node.

[0241] Therefore, the target octree layer must not have any dependencies on the information of its lower octree layers. This is a constraint that applies to geometry / feature coding in general.

[0242] Furthermore, during extensible transmission, the extensible structure needs to be communicated to the receiving device for the transmitting / receiving device to select the extensible layers. When considering the octree structure in the embodiment, all octree layers can support extensible transmission, but it is also possible to enable extensible transmission only for specific octree layers and below. For example, if a slice contains part of an octree layer, the receiving device can determine whether the slice is necessary or unnecessary at the bitstream stage by informing the receiving device via signaling information which extensible layer the slice belongs to. In the example in Figure 18(a), 51002 does not support extensible transmission from level 0 (i.e., the root level) to level 4, but constitutes one extensible layer, and can be configured to perform a 1:1 matching with the extensible layer for the octree layers below it. Generally, extensibility can be supported for the parts corresponding to leaf nodes, but if multiple octree layers are included in one slice, as in Figure 18(c), it is possible to define that each layer constitutes one extensible layer.

[0243] In this case, expandable transmission and expandable decoding can be used separately depending on the purpose. According to the embodiment, expandable transmission is used to select information up to a specific layer without going through a decoder in the transmitting / receiving device. According to the embodiment, expandable decoding can be used to select a specific layer during coding. That is, expandable transmission, in a compressed state (i.e., at the bitstream stage), does not go through a decoder and helps to select the necessary information, making it possible for the transmitting or receiving device to identify a specific layer. On the other hand, expandable decoding can be used in cases such as expandable representation by assisting in encoding / decoding only up to the necessary parts in the encoding / decoding process.

[0244] In this case, the layer configuration for scalable transmission is different from the layer configuration for scalable decoding. For example, the three lowest octree layers including leaf nodes can form one layer from the perspective of scalable transmission, but if all layer information is included from the perspective of scalable decoding, scalable decoding is possible for each of the leaf node layer, leaf node layer - 1, and leaf node layer - 2.

[0245] Details of the slice structure for the layer configuration and the signaling method for scalable transmission described above will be described later.

[0246] As described above, in the method / apparatus according to the embodiment, the bitstream is segmented into specific units for efficient transmission and decoding of the bitstream.

[0247] The method / apparatus according to the embodiment selectively performs transmission and decoding in bitstream units for point cloud data composed of layers.

[0248] The units according to the embodiments are referred to as LOD, layer, slice, etc. LOD is the same term as the LOD in the feature data coding, but in other meanings, it also means the data unit for the layer structure of the bitstream. The LOD according to the embodiments is a concept corresponding to one depth or collecting two or more depths based on the layer structure of the point cloud data, such as an octree or various trees. Similarly, a layer is for generating a unit of the sub-bitstream, which is a concept corresponding to one depth or collecting two or more depths, corresponding to one LOD or two or more LODs. Also, a slice is a unit for constructing a unit of the sub-bitstream, corresponding to one depth, a part of one depth, or two or more depths. Also, a slice corresponds to one LOD, a part of one LOD, or two or more LODs. According to the embodiments, LOD, layer, and slice correspond to or include each other. Also, the units according to the embodiments include LOD, layer, slice, layer group, subgroup, etc., and can also be referred to as complementary to each other. According to the embodiments, in the octree structure, layer, depth, level, and depth level are used in the same meaning.

[0249] FIG. 19 is a diagram showing an example of the layer-based point cloud data configuration according to the embodiments. FIG. 19 is an example of an octree structure in which the depth level of the root node is set to 0 and the depth level of the leaf node is set to 7.

[0250] In the method / apparatus according to the embodiments, the layer-based point cloud data is configured as shown in FIG. 19, and the point cloud data is encoded and decoded.

[0251] The layering of point cloud data in the examples has various layer structures depending on the application field, such as SNR, spatial resolution, color, temporal frequency, and bit depth, and layers are formed in a direction that increases the data density based on an octree structure or LOD structure.

[0252] In other words, when generating LOD based on an octree structure, the LOD can be defined to increase in the direction of increasing detail, i.e., in the direction of increasing octree depth level. In this specification, layer is used synonymously with level, depth, and depth level.

[0253] For example, in Figure 19, in an ocvine tree structure having seven depth levels excluding the root node level (or root level), LOD0 consists of the root node level up to ocvine tree depth level 4, LOD1 consists of the root node level up to ocvine tree depth level 5, and LOD2 consists of the root node level up to ocvine tree depth level 7.

[0254] According to the embodiment, the bitstream obtained by point cloud compression can be transmitted by separating it into a geometry bitstream and a characteristic bitstream, as shown in Figure 20, depending on the type of data. In this case, each bitstream is transmitted as a slice.

[0255] Figure 20 shows an example in which the geometry bitstream and characteristic bitstream are included in their respective slices according to the embodiment. Specifically, referring to Figure 20, the geometry bitstream containing the geometry data is composed of slice 0, and the characteristic bitstream containing the characteristic data is composed of slice 1 and transmitted.

[0256] The method / apparatus according to the embodiment generates LODs based on the layering of an octvine structure as shown in Figure 19, and constructs a geometry bitstream and a characteristic bitstream as shown in Figure 20.

[0257] When transmitting geometry bitstreams and characteristic bitstreams as separate slices, regardless of layer information or LoD information, using only a portion of the layer or LoD requires a process of decoding the bitstream, selecting only the portion to be used and removing the unnecessary portion, and re-encoding based only on the necessary information.

[0258] This specification proposes a method for transmitting a bitstream by dividing it into multiple slices in order to avoid such unnecessary intermediate processes.

[0259] Figure 21(a) shows an example of dividing a geometry bitstream into multiple slices according to the embodiment, and Figure 21(b) shows an example of dividing a characteristic bitstream into multiple slices according to the embodiment.

[0260] In other words, when a bitstream is divided into multiple slices and transmitted to a receiving device, the geometry bitstream and characteristic bitstream can each be transmitted in multiple slices, as shown in Figures 21(a) and 21(b). Each slice consists of a slice header (or data unit header) and slice data (or data unit data). In this case, the slice header contains reference information related to the slice and / or a reference slice (e.g., the previous slice), and the slice data contains the actual bitstream.

[0261] In this case, each of the divided slices exists independently. That is, in the examples in Figures 21(a) and 21(b), slices 0, 1, and 2 are coded for the geometry slices (slice0, slice1, slice2) without any relationship to each other. In this case, since each slice can be coded independently, three geometry coders can operate simultaneously from the perspective of parallel processing, and the slices can be configured most efficiently based on execution time in live encoding / decoding application fields. For example, in predictive geometry coding where the relationship between layers is reduced, slices can be configured independently of each other as in Figure 21(a), and in this case, the number of independently decodeable slices is notified to the receiving device by signaling information, allowing the receiving device to operate in parallel. This can be applied similarly to or in the same way to characteristic slices (slice3, slice4, slice5).

[0262] Furthermore, in other methods, when a bitstream is divided into multiple slices for transmission, interrelationships between them can be considered, as shown in Figures 22(a) and 22(b). Figure 22(a) is a diagram showing another example of dividing a geometry bitstream into multiple slices according to the embodiment, and Figure 22(b) is a diagram showing another example of dividing a characteristic bitstream into multiple slices according to the embodiment.

[0263] For example, in octvine-based geometry coding, compression performance can be improved by sequentially and cumulatively using contextual information for previous nodes. Also, in neighbor search and intra prediction, the occupation information of the decoded neighbor (or peripheral) node is used first, but in this case, information from the immediately preceding slice can be used. Alternatively, information from the preceding slice can be used for parallel processing. In this case, dependencies between slices occur, as shown in Figures 22(a) and 22(b). At this time, information from the preceding slice can be transmitted to the receiving device via signaling information to identify it.

[0264] Figure 22(a) shows an example where slice 2 refers to slice 1, and slice 1 refers to slice 0. Figure 22(b) shows an example where slice 5 refers to slice 4, and slice 4 refers to slice 3.

[0265] In this case, various embodiments can be applied to the bitstream alignment method for transmitting geometry bitstreams and characteristic bitstreams using multiple slices.

[0266] Figure 23 shows an example of a method for aligning geometry bitstreams and characteristic bitstreams according to an embodiment.

[0267] The transmission method / device according to the embodiment transmits geometry data (or geometry bitstream, referred to as geometry information) and characteristic data (or characteristic bitstream, referred to as characteristic information) in series when transmitting a bitstream, as shown in Figure 23. In this case, depending on the type of data, all of the geometry data is sent first, followed by the transmission of the characteristic data. This has the advantage that the geometry data can be quickly reconstructed based on the information in the transmitted bitstream.

[0268] FIG. 23 is an example in which slices 0, 1, and 2 containing geometry data, and slices 3, 4, and 5 containing trait data are transmitted in this order. At this time, the position can be variously changed depending on the embodiment. Also, reference between geometry headers is possible, and reference between trait headers and geometry headers is also possible.

[0269] FIG. 24 is a diagram showing another example of a method for aligning a geometry bitstream and a trait bitstream according to an embodiment.

[0270] In the transmission method / apparatus according to the embodiment, when transmitting a bitstream, the geometry bitstream and the trait bitstream that constitute the same layer can be collected and transmitted as shown in FIG. 24. In this case, when using a compression technique that enables parallel decoding of geometry and traits, the decoding execution time can be shortened. At this time, the information to be processed first is arranged first (that is, a small LoD, and geometry precedes traits).

[0271] FIG. 24 is an example in which slices 0 containing geometry data, slice 3 containing trait data, slice 1 containing geometry data, slice 4 containing trait data, slice 2 containing geometry data, and slice 5 containing trait data are transmitted in this order. At this time, the position can be variously changed depending on the embodiment. Also, reference between geometry headers is possible, and reference between trait headers and geometry headers is also possible.

[0272] In the transmission / reception method / apparatus according to the embodiment, when transmitting / receiving a bitstream, the layer (or LOD) desired in the application field can be efficiently selected at the bitstream level. Among the bitstream alignment methods according to the embodiment, when collecting and transmitting geometry information first as shown in FIG. 23, there may be a vacant part in the middle after selecting a specific bitstream level. In this case, rearrangement of the bitstream is necessary.

[0273] On the other hand, when geometric data and characteristic data are collected and transmitted using layers, as shown in Figure 24, it is possible to selectively transmit the information necessary for the application field and / or selectively remove unnecessary information.

[0274] In the embodiment, when selecting a portion of the bitstream, for example, in Figure 24, the transmitting device may select and transmit geometry slice 0 and characteristic slice 3 of the same layer, as well as geometry slice 1 and characteristic slice 4 of another layer, while removing geometry slice 2 and characteristic slice 5 of another layer from the bitstream and not transmitting them. That is, in the case of symmetric geometry-characteristic selection, the geometry data and characteristic data of the same layer are either selected and transmitted simultaneously, or selected and removed simultaneously.

[0275] In the embodiment, when selecting a portion of the bitstream, for example, in Figure 24, the transmitting device may select and transmit geometry slice 0 and characteristic slice 3 of the same layer, as well as geometry slice 1 and characteristic slice 4 of another layer, and geometry slice 2 and characteristic slice 5 of yet another layer, up to geometry slice 2, and remove characteristic slice 5 from the bitstream and not transmit it. In other words, in the case of asymmetric geometry-characteristic selection, only one of the geometry data or characteristic data of the same layer is selected and transmitted or removed.

[0276] The aforementioned bitstream splitting and selection of bitstreams are intended to support the scalability of point cloud data.

[0277] Next, the operation of continuous slicing and the management of the context buffer will be described. In one embodiment, the context buffer is the geometry buffer and / or characteristic buffer of the receiver shown in Figure 43.

[0278] As mentioned above, in octvine-based geometry coding, compression performance can be improved by sequentially and cumulatively using contextual information about previously accessed nodes. Furthermore, in neighbor search and intra prediction, the occupation information of previously decoded neighboring (or peripheral) nodes is used; in this case, information from the immediately preceding slice can be used. Alternatively, for parallel processing, information from preceding slices can be used. In this case, dependencies between slices arise.

[0279] In the case of dependent slices, the coding efficiency degradation that occurs when slicing is divided can be resolved by using information from the previous slice for coding the next slice. In the case of geometry coding, information such as CABAC (context-based adaptive binary arithmetic coding) context, context map, dictionary LuT, and planar coding variables can be used sequentially. When using context sequentially in this way, context buffer management is necessary.

[0280] Figure 25(a) shows an example of inter-slice dependencies in a bitstream structure according to the embodiment. Figure 25(b) shows an example of the context generated in each slice when dependencies occur between slices, as in Figure 25(a). Figures 25(c) to 25(e) show examples of context buffer management when dependencies are generated between slices according to the embodiment.

[0281] In Figure 25(a), when a bitstream is divided into multiple slices for transmission, slices 2 and 3 use the information from slice 0, and slice 4 uses the information from slice 3. That is, slices 2 and 3 depend on slice 0, and slice 4 depends on slice 3. For example, since slice 2 is encoded based on the context information of slice 0 (e.g., geometry data or characteristic data), the encoding of slice 2 begins after the encoding of slice 0 is completed. Also in Figure 25(a), slice 0 is referenced by slices 2 and 3, so slice 0 is referenced by other slices twice. On the other hand, slice 1 is independent of the other slices. Therefore, the data in slice 1 (e.g., geometry data or characteristic data) is encoded independently without any relationship to other slices.

[0282] The embodiment in Figure 25(a) describes the case where the tree is divided into slice units, but this is only one example. That is, even for bitstreams divided into brick units (which include subtrees), tile units (divided into regions), or frame units, information such as continuation information, context usage information, and whether or not there are adjacent references to other bricks, tiles, or frames are used.

[0283] Furthermore, when the data belonging to each slice (e.g., geometry data or characteristic data) is encoded by an encoder, a context is generated for each slice. Here, the encoders include the point cloud video encoder 10002 in Figure 1, the encoder 20001 in Figure 2, the point cloud video encoder in Figure 4, the transmitter in Figure 12, and the encoder in Figure 42.

[0284] As shown in Figure 25(b), when the data from the first slice (slice0) is input to the encoder, context information, context A, is generated through encoding. When the data from the second slice (slice1) is encoded by the encoder, context B is generated, and when the data from the third slice (slice2) is encoded by the encoder, context A' is generated. Furthermore, when the data from the fourth slice (slice3) is encoded by the encoder, context A'' is generated, and when the data from the fifth slice (slice4) is encoded by the encoder, context C is generated. For example, context A' refers to context A, but it is an independent (or different) context from context A. In other words, context A' is an updated context of context A.

[0285] Furthermore, the receiving device stores the context of each slice in a context buffer.

[0286] In this case, as shown in Figure 25(c), if all contexts are stored in the context buffer without controlling the context buffer, storage space will run out, or the cost of securing larger storage space will increase.

[0287] Therefore, this specification proposes a method for efficiently managing context buffers. Figures 25(d) and 25(e) illustrate examples of how context buffers are managed by embodiments.

[0288] Referring to Figure 25(d), when the data from the first slice (slice0) is input to the transmitter's encoder, context information, context A, is generated through encoding. In this case, since context A is used in one or more subsequent slices (e.g., slice 2, slice 3, and other slices not shown), the receiver stores context A in its context buffer. In this case, the receiver can efficiently manage its buffer (or context buffer) by transmitting information about the slices that use the context of slice 0 (e.g., slices 2 and 3) to the receiver via signaling information (e.g., buffer management-related information). For example, the total number of subsequent slices that use context A (=N) (e.g., num_context_reuse_minus1) can be transmitted to the receiver via signaling information (e.g., buffer management-related information or a counter). For example, in Figure 25(a), N is 3. In this case, if N is greater than 0, after storing context A in the receiver's context buffer, context A can be removed from the context buffer after it has been referenced N times by other slices. In Figure 25(d), 'in' means storing the context in the context buffer, and 'out' means removing the context from the context buffer.

[0289] In the example, since slice 0 is used by slice 2, slice 3, and other slices not shown, context A, which is the context of slice 0, is not removed from the context buffer but is subsequently stored (context A(in)).

[0290] According to the embodiment, slice 1 is encoded (or decoded) independently of slice 0 and the next slice, so context B, which is the context of slice 2, can be removed immediately after processing in the context buffer (context B(in / out)). According to the embodiment, when processing slice 0, the execution time can be reduced by processing slice 1 in parallel at the same time.

[0291] According to the example, slice 2 references slice 0, but has no relationship with the next slice, so context A', which is the context of slice 2, can be removed immediately after processing in the context buffer (context A'(in / out)). Since slice 2 is encoded (or decoded) based on context A, which is the context (or context information) of slice 0, encoding (or decoding) can begin after encoding (or decoding) of slice 0 is completed. Also, in the case of slice 2, it is not referenced by the next slice, so it can be removed from the context buffer after encoding (or decoding) is completed.

[0292] In the example, since slice 3 is used in slice 4 and other slices not shown, context A'', which is the context of slice 3, is not removed from the context buffer but is subsequently stored (context A''(in)). In the case of slice 3, encoding (or decoding) is performed based on context A of slice 0, similar to slice 2. Therefore, by encoding (or decoding) slice 3 in parallel with slice 2, execution time can be reduced. However, in the case of the context generated in slice 3 (i.e., context A''), it is used for the encoding (or decoding) of the next slice, so it is not removed after being stored in the context buffer. Furthermore, context A'' stored in the context buffer is used for the encoding (or decoding) of slice 4.

[0293] According to the example, slice 4 refers to slice 3, but has no relationship with the next slice, so context C, which is the context of slice 4, can be processed within the context buffer and then immediately removed (context C(in / out)).

[0294] According to one embodiment, the duration for which the context generated after encoding a particular slice is stored in the context buffer is determined based on the number of times that slice is referenced in at least one subsequent slice.

[0295] When additional information about the context buffer exists, the context buffer can be efficiently managed by removing it from the context buffer if it is no longer needed, based on information about whether the context is being used again and how many times it has been used. In other words, the context buffer can be efficiently managed by immediately removing the context of a slice that has no relationship with the next slice after processing it within the context buffer. Furthermore, if parallel processing is possible based on dependency information, execution time can be reduced by executing concurrently.

[0296] Figure 25(e) shows an example of managing a context buffer when performing parallel processing on two or more slices.

[0297] For example, if slice 0 and slice 1 are processed in parallel, context A of slice 0 and context B of slice 1 are stored in the context buffer simultaneously. At this time, since slice 0 is used by slice 2, slice 3, and other slices not shown, context A is not removed from the context buffer and is stored thereafter (context a(in)). However, since slice 1 has no relationship with slice 0 or the next slice, context B is processed in the context buffer and immediately removed (context b(in / out)).

[0298] In another example, if slices 2 and 3 are processed in parallel, the context A' of slice 2 and the context A'' of slice 3 are stored in the context buffer simultaneously. In this case, since slice 2 is not used in the next slice, context A' is processed in the context buffer and immediately removed (context A'(in / out)). However, since slice 3 is used in slice 4, context A'' is not removed from the context buffer and is stored there as well (context a''(in)).

[0299] Next, we will explain the behavior of consecutive slices (neighbour continuation).

[0300] According to the example, when coding by dividing into slices, positionally adjacent nodes may exist in different slices.

[0301] Figures 26(a) to 26(d) illustrate examples of neighbor structures according to the embodiment.

[0302] For example, in the neighbor structure shown in Figures 26(a) to 26(d), if node 53000, located in the center of the neighbor structure, is on the boundary of a slice, nodes 1, 8, and 32 are in the same slice, while nodes 2, 4, and 16 may be in one or more different slices. In this case, considering the relationship with neighboring (or peripheral) nodes, neighboring nodes are used or not used depending on whether there is a dependency between slices. For example, when neighboring nodes are used, the dependency between slices is indicated, and the slice containing each neighboring node is also indicated. Figures 26(a) to 26(d) show examples of a central node with 6 dependent neighboring nodes, 18 dependent neighboring nodes, and 32 dependent neighboring nodes. Alternatively, if a dependency is not recognized, neighbor search is used by assuming that there are no such neighboring nodes, that all are occupied, or that they are not occupied (i.e., unoccupied). When such assumptions are used, relevant information can be transmitted to the receiving device via signaling information. At this time, the possibility of parallel processing is opened up in that each slice can be processed independently. Conversely, accurately using information about adjacent nodes requires processing results for adjacent slices. This relationship with adjacent nodes is used in processes such as determining the occupancy map for geometry coding, or in adjacent search for LoD generation and RAHT prediction.

[0303] Figures 26(a) through 26(d) show the case where the tree is divided into slice units, but continuation information, context usage information, and whether adjacent references exist are also used for bitstreams divided into brick units, tile units (divided into regions), or frame units, which are concepts that include subtrees.

[0304] Figure 27 shows an example of the bitstream structure of point cloud data for transmission and reception according to the embodiment. According to the embodiment, the bitstream output by any of the point cloud video encoders in Figures 1, 2, 4, 12, and 42 is in the form shown in Figure 27.

[0305] According to the embodiment, the bitstream of point cloud data provides tiles or slices so that the point cloud data can be processed by dividing it into regions. Each region of the bitstream according to the embodiment has a different degree of importance. Therefore, when the point cloud data is divided into tiles, different filters (encoding methods) and different filter units can be applied to each tile. Similarly, when the point cloud data is divided into slices, different filters and different filter units can be applied to each slice.

[0306] When the transmitting and receiving devices according to the embodiment divide and compress point cloud data into regions, they can transmit and receive bitstreams with a high-level syntactic structure for selective transmission of characteristic information within the divided regions.

[0307] The transmitting device according to the embodiment transmits point cloud data using a bitstream structure as shown in Figure 27, applying different encoding operations depending on importance to provide a solution for using a high-quality encoding method for critical areas. It also supports efficient encoding and transmission based on the characteristics of point cloud data and can provide characteristic values ​​according to user requirements.

[0308] In the receiving device according to the embodiment, point cloud data is received using a bitstream structure as shown in Figure 27. This allows the receiving device to provide different filtering (decoding) methods for each region (regions divided into tiles or slices) rather than using a complex decoding (filtering) method for the entire point cloud data, depending on the processing power of the receiving device. Therefore, better image quality and appropriate latency for the system can be guaranteed for areas important to the user.

[0309] In the embodiment, if the geometry bitstream, attribute bitstream, and / or signaling bitstream (or signaling information) consist of a single bitstream (or G-PCC bitstream) as shown in Figure 27, the bitstream includes one or more sub-bitstreams. The bitstream in the embodiment includes an SPS (Sequence Parameter Set) for sequence-level signaling, a GPS (Geometry Parameter Set) for signaling geometry information coding, one or more APSs (Attribute Parameter Set, APS0, APS1) for signaling attribute information coding, a tile catalog (or TPS) for tile-level signaling, and one or more slices (slice 0 to slice n). That is, the bitstream of point cloud data in the embodiment includes one or more tiles, and each tile is a group of slices containing one or more slices (slice 0 to slice n). The tile catalog (i.e., TPS) in the embodiment includes information about each tile (e.g., coordinate values ​​and height / size information of the tile bounding box) for one or more tiles. Each slice contains one geometry bitstream (Geom0) and / or one or more attribute bitstreams (Attr0, Attr1). For example, slice 0 contains one geometry bitstream (Geom0 0 ) and one or more attribute bitstreams (Attr0 0 ,Attr1 0 ) includes.

[0310] Each slice contains a geometry bitstream consisting of a geometry slice header (geom_slice_header) and geometry slice data (geom_slice_data). In this embodiment, the geometry bitstream within each slice is referred to as a geometry data unit, the geometry slice header is referred to as a geometry data unit header, and the geometry slice data is also referred to as geometry data unit data.

[0311] Each characteristic bitstream within each slice consists of a characteristic slice header (attr_slice_header) and characteristic slice data (attr_slice_data). In this embodiment, each characteristic bitstream within each slice is referred to as a characteristic data unit, the characteristic slice header is referred to as a characteristic data unit header, and the characteristic slice data is also referred to as characteristic data unit data.

[0312] According to the embodiment, the parameters required for encoding and / or decoding point cloud data are newly defined in the point cloud data parameter set (e.g., SPS, GPS, APS, and TPS (or tile catalog)) and / or the header of the corresponding slice. For example, when encoding and / or decoding geometry information, they can be added to the geometry parameter set (GPS), and when encoding and / or decoding tile-based data, they can be added to the tile (TPS) and / or slice header.

[0313] According to the embodiment, information related to slicing (or referred to as slice-related information) and / or information related to buffer management (or referred to as buffer management information) are signaled to one of the sequence parameter set, geometry parameter set, characteristic parameter set, tile parameter set, and SEI message. In addition, information related to slicing (or referred to as slice-related information) and / or information related to buffer management (or referred to as buffer management information) are signaled to one of the geometry slice header (or referred to as geometry data unit header) and characteristic slice header (or referred to as characteristic data unit header).

[0314] According to the examples, slice-related information and / or buffer management information can be defined in a corresponding location or another location depending on the application and system, thereby changing the scope and method of application. The term "field," used in the syntax of this specification as described later, has the same meaning as "parameter" or "syntactic element."

[0315] In other words, the signal (e.g., slice-related information and / or buffer management information) has different meanings depending on where it is transmitted. If defined in SPS, it is applied uniformly to the entire sequence; if defined in GPS, it is used for location reconstruction; if defined in APS, it is applied to characteristic reconstruction; and if defined in TPS, it is applied only to points within a tile. Furthermore, if transmitted at the slice level, it is applied only to that slice. On the other hand, if the fields (or syntactic elements) defined below are currently applied not only to one point cloud data stream but to multiple point cloud data streams, they can be transmitted by a higher-level parameter set or the like.

[0316] According to the embodiment, parameters (variously referred to as metadata, signaling information, etc.) are generated in the metadata processing unit (or metadata generator), signaling processing unit, or processor of the transmitting device, transmitted to the receiving device, and used in the decoding / reconstruction process. For example, parameters generated and transmitted by the transmitting device are obtained by the metadata parser of the receiving device.

[0317] In this embodiment, the relevant information is defined independently of the coding technique, but in other embodiments, it can be defined in conjunction with the coding method, and can be defined in a tile parameter set to support different scalability in different regions.

[0318] Alternatively, bitstreams can be selected at the system level by defining a NAL (Network Abstract Layer) unit and transmitting relevant information (e.g., slice-related information and / or buffer management information) that allows for layer selection, such as layer_id.

[0319] Figure 28 shows one example of the syntactic structure of a sequence parameter set (seq_parameter_set) (SPS) according to this specification. The SPS contains sequence information of a point cloud data bitstream.

[0320] The SPS in the example includes the fields main_profile_compatibility_flag, unique_point_positions_constraint_flag, level_idc, sps_seq_parameter_set_id, sps_bounding_box_present_flag, sps_source_scale_factor_numerator_minus1, sps_source_scale_factor_denominator_minus1, sps_num_attribute_sets, log2_max_frame_idx, axis_coding_order, sps_bypass_stream_enabled_flag, and sps_extension_flag.

[0321] The `main_profile_compatibility_flag` field indicates whether the bitstream conforms to the main profile. For example, a value of 1 in the `main_profile_compatibility_flag` field indicates that the bitstream conforms to the main profile. For example, a value of 0 in the `main_profile_compatibility_flag` field indicates that the bitstream conforms to a profile other than the main profile.

[0322] If the value of the unique_point_positions_constraint_flag field is 1, all output points have unique positions in each point cloud frame referenced by the current SPS. If the value of the unique_point_positions_constraint_flag field is 0, two or more output points have the same position in any point cloud frame referenced by the current SPS. For example, even if all points are unique in their respective slices, slices and other points within a frame can overlap. In this case, the value of the unique_point_positions_constraint_flag field is set to 0.

[0323] The level_idc field indicates the level the bitstream follows.

[0324] The sps_seq_parameter_set_id field provides an identifier for the SPS for reference by other syntax elements.

[0325] The sps_bounding_box_present_flag field indicates whether the bounding box exists in the SPS. For example, a value of 1 for the sps_bounding_box_present_flag field indicates that the bounding box exists in the SPS, while a value of 0 indicates that the size of the bounding box is undefined.

[0326] According to the embodiment, if the value of the sps_bounding_box_present_flag field is 1, the SPS further includes the sps_bounding_box_offset_x field, sps_bounding_box_offset_y field, sps_bounding_box_offset_z field, sps_bounding_box_offset_log2_scale field, sps_bounding_box_size_width field, sps_bounding_box_size_height field, and sps_bounding_box_size_depth field.

[0327] The sps_bounding_box_offset_x field indicates the x-offset of the source bounding box in Cartesian coordinates. If there is no x-offset for the source bounding box, the value of the sps_bounding_box_offset_x field is 0.

[0328] The sps_bounding_box_offset_y field indicates the y-offset of the source bounding box in the Cartesian coordinate system. If there is no y-offset for the source bounding box, the value of the sps_bounding_box_offset_y field is 0.

[0329] The sps_bounding_box_offset_z field indicates the z-offset of the source bounding box in the Cartesian coordinate system. If there is no z-offset for the source bounding box, the value of the sps_bounding_box_offset_z field is 0.

[0330] The sps_bounding_box_offset_log2_scale field indicates the scale factor for scaling the quantized x,y,z source bounding box offset.

[0331] The sps_bounding_box_size_width field indicates the width of the source bounding box in a Cartesian coordinate system. If there is no source bounding box width, the value of the sps_bounding_box_size_width field may be 1.

[0332] The sps_bounding_box_size_height field indicates the height of the source bounding box in the Cartesian coordinate system. If there is no source bounding box height, the value of the sps_bounding_box_size_height field may be 1.

[0333] The sps_bounding_box_size_depth field indicates the depth of the source bounding box in the Cartesian coordinate system. If there is no source bounding box depth, the value of the sps_bounding_box_size_depth field may be 1.

[0334] sps_source_scale_factor_numerator_minus1 plus1 indicates the numerator of the source point cloud scaling factor.

[0335] sps_source_scale_factor_denominator_minus1 plus1 indicates the denominator of the source point cloud scaling factor.

[0336] The sps_num_attribute_sets field indicates the number of coded attributes in the bitstream.

[0337] In one embodiment, the SPS includes a loop statement that is repeated as many times as the value of the sps_num_attribute_sets field. In this embodiment, i is initialized to 0 and increases by 1 each time the loop statement is executed, and the loop statement is repeated until the value of i reaches the value of the sps_num_attribute_sets field. This loop statement includes the attribute_dimension_minus1[i] field and the attribute_instance_id[i] field. attribute_dimension_minus1[i] plus 1 indicates the number of components of the i-th attribute.

[0338] The attribute_instance_id[i] field indicates the instance identifier of the i-th attribute.

[0339] According to the embodiment, if the value of the attribute_dimension_minus1[i] field is greater than 1, the loop statement further includes the attribute_secondary_bitdepth_minus1[i] field, the attribute_cicp_colour_primaries[i] field, the attribute_cicp_transfer_characteristics[i] field, the attribute_cicp_matrix_coeffs[i] field, and the attribute_cicp_video_full_range_flag[i] field.

[0340] attribute_secondary_bitdepth_minus1[i] plus 1 indicates the bit depth for the second component of the i-th attribute signal.

[0341] The attribute_cicp_colour_primaries[i] field shows the initial chromaticity coordinates of the i-th attribute's color attribute source.

[0342] The attribute_cicp_transfer_characteristics[i] field is a function of the source input linear optical intensity with a nominal real-valued range of 0 to 1, indicating the reference opto-electronic transfer characteristic function, or a function of the output linear optical intensity, indicating the inverse of the reference opto-electronic transfer characteristic function.

[0343] The attribute_cicp_matrix_coeffs[i] field describes the matrix coefficients used in deriving luma and chroma signals from the green, blue, and red (or Y, Z, and X primaries) of the i-th attribute.

[0344] The attribute_cicp_video_full_range_flag[i] field indicates the range of the black level and lumens and saturation signals derived from the E'Y, E'PB and E'PR or E'R, E'G and E'B actual-value component signals of the i-th attribute.

[0345] The `known_attribute_label_flag[i]` field indicates whether the `know_attribute_label[i]` or `attribute_label_four_bytes[i]` field is signaled for the i-th attribute. For example, a value of 0 for `known_attribute_label_flag[i]` indicates that the `known_attribute_label[i]` field is signaled for the i-th attribute, and a value of 1 for `known_attribute_label_flag[i]` indicates that the `attribute_label_four_bytes[i]` field is signaled for the i-th attribute.

[0346] The `known_attribute_label[i]` field indicates the type of the i-th attribute. For example, a value of 0 for `known_attribute_label[i]` indicates that the i-th attribute is color, a value of 1 indicates that the i-th attribute is reflectivity, and a value of 2 indicates that the i-th attribute is frame index. Additionally, a value of 4 for `known_attribute_label[i]` indicates that the i-th attribute is transparency, and a value of 5 indicates that the i-th attribute is normals.

[0347] The attribute_label_four_bytes[i] field indicates the known attribute type using a 4-byte code.

[0348] According to the example, a value of 0 in the attribute_label_four_bytes[i] field indicates that the i-th attribute is color, 1 indicates that the i-th attribute is reflectance, 2 indicates that the i-th attribute is frame index, 4 indicates that the i-th attribute is transparency, and 5 indicates that the i-th attribute is normals.

[0349] The log2_max_frame_idx field indicates the number of bits used to signal the variable in the frame_idx syntax.

[0350] The axis_coding_order field indicates the correspondence between the three positional components in the reconstructed point cloud RecPic[pointidx][axis] with X, Y, Z output axis labels and axis=0..2.

[0351] A value of 1 in the sps_bypass_stream_enabled_flag field indicates that bypass coding mode will be used when reading the bitstream. In other examples, a value of 0 in the sps_bypass_stream_enabled_flag field indicates that bypass coding mode will not be used when reading the bitstream.

[0352] The sps_extension_flag field indicates whether the sps_extension_data syntax structure exists in the corresponding SPS syntax structure. For example, a value of 1 in the sps_extension_present_flag field indicates that the sps_extension_data syntax structure exists in this SPS syntax structure, while a value of 0 indicates that it does not exist.

[0353] In the example, the SPS further includes the sps_extension_data_flag field if the value of the sps_extension_flag field is 1.

[0354] The sps_extension_data_flag field can have any value.

[0355] Figure 29 shows another example of the syntactic structure of a sequence parameter set (SPS) according to the example.

[0356] The SPS in Figure 29 further includes the sps_entropy_continuation_enabled_flag field. For example, a value of 1 in the sps_entropy_continuation_enabled_flag field indicates that a slice's initial entropy context state is determined by the last entropy context state of the preceding state. The ntext state may depend upon the final entropy context state of the preceding slice. A value of 0 for the sps_entropy_continuation_enabled_flag field specifies that the initial entropy context state of each slice is independent.

[0357] According to the example, the sps_entropy_continuation_enabled_flag field in Figure 29 is included at any position in the SPS in Figure 28.

[0358] Figure 30 shows an example of the syntactic structure of a geometry parameter set (GPS) according to this specification. The GPS in this example includes information on how to encode the geometry information of point cloud data contained in one or more slices.

[0359] The GPS in the example includes the fields gps_geom_parameter_set_id, gps_seq_parameter_set_id, gps_box_present_flag, unique_geometry_points_flag, geometry_planar_mode_flag, geometry_angular_mode_flag, neighborhood_context_restriction_flag, inferred_direct_coding_mode_enabled_flag, bitwise_occupancy_coding_flag, adjacent_child_contextualization_enabled_flag, log2_neighbour_avail_boundary, log2_intra_pred_max_node_size, log2_trisoup_node_size, geom_scaling_enabled_flag, gps_implicit_geom_partition_flag, and gps_extension_flag.

[0360] The gps_geom_parameter_set_id field provides the identifier of the GPS, which is referenced by other syntactic elements.

[0361] The gps_seq_parameter_set_id field indicates the value of the sps_seq_parameter_set_id field for the active SPS.

[0362] The gps_box_present_flag field indicates whether additional bounding box information is provided from the geometry slice header currently referencing the GPS. For example, a value of 1 for the gps_box_present_flag field indicates that additional bounding box information is provided within the geometry slice header currently referencing the GPS. Therefore, if the value of the gps_box_present_flag field is 1, the GPS will also include the gps_gsh_box_log2_scale_present_flag field.

[0363] The gps_gsh_box_log2_scale_present_flag field indicates whether the gps_gsh_box_log2_scale field is currently signaled to each geometry slice header referencing GPS. For example, a value of 1 for the gps_gsh_box_log2_scale_present_flag field indicates that the gps_gsh_box_log2_scale field is currently signaled to each geometry slice header referencing GPS. In another example, a value of 0 for the gps_gsh_box_log2_scale field indicates that the gps_gsh_box_log2_scale field is not currently signaled to each geometry slice header referencing GPS, and a common scale for all slices is currently signaled to the gps_gsh_box_log2_scale field of GPS.

[0364] If the value of the gps_gsh_box_log2_scale_present_flag field is 0, the GPS will also include the gps_gsh_box_log2_scale field.

[0365] The gps_gsh_box_log2_scale field indicates the common scale factor of the bounding box origin for all slices currently referencing GPS.

[0366] The `unique_geometry_points_flag` field indicates whether all output points have unique positions within a single slice in all slices that currently reference the GPS. For example, a value of 1 in the `unique_geometry_points_flag` field indicates that all output points have unique positions within a single slice in all slices that currently reference the GPS. A value of 0 in the `unique_geometry_points_flag` field indicates that two or more output points may have the same position within a single slice in all slices that currently reference the GPS.

[0367] The geometry_planar_mode_flag field indicates whether the planar coding mode is activated or not. For example, a value of 1 for the geometry_planar_mode_flag field indicates that the planar coding mode is activated, and a value of 0 indicates that the planar coding mode is not activated.

[0368] If the value of the geometry_planar_mode_flag field is 1, i.e., true, then GPS further includes the geom_planar_mode_th_idcm field, the geom_planar_mode_th[1] field, and the geom_planar_mode_th[2] field.

[0369] The geom_planar_mode_th_idcm field indicates the activation threshold for direct coding mode.

[0370] The geom_planar_mode_th[i] field specifies the threshold for activation of the planar coding mode, along with the i-th most probable direction for the planar coding mode, within the range of 0-2.

[0371] The geometry_angular_mode_flag field indicates whether or not the angular coding mode is active. For example, a value of 1 for the geometry_angular_mode_flag field indicates that the angular coding mode is active, and a value of 0 indicates that the angular coding mode is not active.

[0372] If the value of the geometry_angular_mode_flag field is 1, i.e., true, then GPS further includes the lidar_head_position[0] field, lidar_head_position[1] field, lidar_head_position[2] field, number_lasers field, planar_buffer_disabled field, implicit_qtbt_angular_max_node_min_dim_log2_to_split_z field and implicit_qtbt_angular_max_diff_to_split_z field.

[0373] The lidar_head_position[0], lidar_head_position[1], and lidar_head_position[2] fields indicate the (X,Y,Z) coordinates of the lidar head in a coordinate system with internal axes.

[0374] The `number_lasers` field indicates the number of lasers used for angular coding mode.

[0375] The GPS in this example includes a loop statement that is repeated a number of times equal to the value of the number_lasers field. In this example, i is initialized to 0 and increases by 1 each time the loop statement is executed, and the loop statement is repeated until the value of i reaches the value of the number_lasers field. This loop statement includes the laser_angle[i] field and the laser_correction[i] field.

[0376] The laser_angle[i] field represents the tangent of the elevation angle of the i-th laser relative to the horizontal plane defined by the 0th and 1st internal axes.

[0377] The laser_correction[i] field indicates the correction of the i-th laser position, related to the lidar_head_position[2] field, via the second internal axis.

[0378] A value of 1 in the `planar_buffer_disabled` field indicates that tracking the closest nodes using the buffer is not used in the process of coding the planar mode flag and plane position in planar mode. A value of 0 in the `planar_buffer_disabled` field indicates that tracking the closest nodes using the buffer is used.

[0379] The implicit_qtbt_angular_max_node_min_dim_log2_to_split_z field indicates the log2 value of the node size at which horizontal splitting of nodes is preferred over vertical splitting.

[0380] The implicit_qtbt_angular_max_diff_to_split_z field indicates the log2 value of the maximum vertical relative to the horizontal node size ratio allowed for the node.

[0381] A value of 0 for the `neighbour_context_restriction_flag` field indicates that the geometry node occupancy of the current node is coded with the contexts determined from neighboring nodes located inside the parent node of the current node. A value of 1 for the `neighbour_context_restriction_flag` field indicates that the geometry node occupancy of the current node is coded with the contexts determined from neighboring nodes located inside or outside the parent node of the current node.

[0382] The `inferred_direct_coding_mode_enabled_flag` field indicates whether the `direct_mode_flag` field exists in the corresponding geometry node syntax. For example, a value of 1 for the `inferred_direct_coding_mode_enabled_flag` field indicates that the `direct_mode_flag` field exists in the corresponding geometry node syntax. For example, a value of 0 for the `inferred_direct_coding_mode_enabled_flag` field indicates that the `direct_mode_flag` field does not exist in the corresponding geometry node syntax.

[0383] The `bitwise_occupancy_coding_flag` field indicates whether the geometry node occupancy is encoded using the bitwise contextualization of its syntax element occupancy map. For example, a value of 1 for the `bitwise_occupancy_coding_flag` field indicates that the geometry node occupancy is encoded using the bitwise contextualization of its syntax element `occupancy_map`. For example, a value of 0 for the `bitwise_occupancy_coding_flag` field indicates that the geometry node occupancy is encoded using its directory-encoded syntax element `occupancy_byte`.

[0384] The adjacent_child_contextualization_enabled_flag field indicates whether the adjacent children of neighbouring octree nodes are used for bitwise occupancy contextualization. For example, if the value of the adjacent_child_contextualization_enabled_flag field is 1, it indicates that the adjacent children of neighbouring octree nodes are used for bitwise occupancy contextualization. For example, if the value of the adjacent_child_contextualization_enabled_flag field is 0, it indicates that the children of neighbouring octree nodes are not used for bitwise occupancy contextualization.

[0385] The log2_neighbour_avail_boundary field indicates the value of the variable NeighbAvailBoundary used in the decoding process. For example, if the value of the neighbour_context_restriction_flag field is 1, NeighbAvailabilityMask is set to 1. For example, if the value of the neighbour_context_restriction_flag field is 0, NeighbAvailabilityMask is set to 1 << log2_neighbour_avail_boundary.

[0386] The log2_intra_pred_max_node_size field indicates the octree node size eligible for occupancy intra prediction.

[0387] The log2_trisoup_node_size field specifies the variable TrisoupNodeSize as the size of the triangle nodes.

[0388] The `geom_scaling_enabled_flag` field indicates whether the scaling process for the geometry position is applied during the geometry slice decoding process. For example, a value of 1 for the `geom_scaling_enabled_flag` field indicates that the scaling process for the geometry position is applied during the geometry slice decoding process. A value of 0 for the `geom_scaling_enabled_flag` field indicates that the geometry position does not require scaling.

[0389] The geom_base_qp field indicates the base value of the geometry position quantization parameter.

[0390] The gps_implicit_geom_partition_flag field indicates whether an implicit geometry partition is enabled for the sequence or slice. For example, a value of 1 for the gps_implicit_geom_partition_flag field indicates that the implicit geometry partition is enabled for the sequence or slice, while a value of 0 indicates that it is disabled. If the value of the gps_implicit_geom_partition_flag field is 1, the following two fields, namely the gps_max_num_implicit_qtbt_before_ot field and the gps_min_size_implicit_qtbt field, are signaled.

[0391] The gps_max_num_implicit_qtbt_before_ot field specifies the maximal number of implicit QT and BT partitions before OT partitions. The variable K is then initialized by the gps_max_num_implicit_qtbt_before_ot field as follows:

[0392] K=gps_max_num_implicit_qtbt_before_ot

[0393] The gps_min_size_implicit_qtbt field specifies the minimal size of implicit QT and BT partitions. The variable M is then initialized with the gps_min_size_implicit_qtbt field as follows:

[0394] M=gps_min_size_implicit_qtbt

[0395] The gps_extension_flag field indicates whether the gps_extension_data syntax structure exists in the corresponding GPS syntax structure. For example, a value of 1 in the gps_extension_flag field indicates that the gps_extension_data syntax structure exists in the corresponding GPS syntax. For example, a value of 0 in the gps_extension_flag field indicates that the gps_extension_data syntax structure does not exist in the corresponding GPS syntax.

[0396] In the example, if the value of the gps_extension_flag field is 1, the GPS further includes the gps_extension_data_flag field.

[0397] The gps_extension_data_flag field can have any value. Its existence and value do not affect the decoder conformance to profiles.

[0398] According to the example, the GPS further includes a geom_tree_type field. For example, a value of 0 in the geom_tree_type field indicates that the location information (or geometry) was coded using an octree, and a value of 1 indicates that the location information (or geometry) was coded using a predictive tree.

[0399] Figure 31 shows one example of the syntactic structure of a geometry parameter set (GPS) that includes slice-related information and / or buffer management-related information according to an embodiment.

[0400] According to the example, GPS includes the geom_slice_segmentation_enabled_flag field.

[0401] For example, a value of 1 in the geom_slice_segmentation_enabled_flag field indicates that the geometry bitstream will be divided and transmitted across multiple slices. In this case, additional conditions allow for parallel processing, scalable transmission, and spatial scalability. A value of 1 in the geom_slice_segmentation_enabled_flag field indicates that the geometry bitstream will be transmitted across each individual slice.

[0402] According to the embodiment, the slice-related information and / or buffer management-related information in Figure 31 are included at any position of the GPS in Figure 30.

[0403] Figure 32 shows one embodiment of the syntactic structure of an attribute parameter set (APS) according to this specification. The APS in this embodiment includes information on how to encode attribute information of point cloud data contained in one or more slices.

[0404] The APS in the example includes the fields aps_attr_parameter_set_id, aps_seq_parameter_set_id, attr_coding_type, aps_attr_Initial_qp, aps_attr_chroma_qp_offset, aps_slice_qp_delta_present_flag, and aps_extension_flag.

[0405] The `aps_attr_parameter_set_id` field indicates an identifier for the APS for reference by other syntactic elements.

[0406] The `aps_seq_parameter_set_id` field indicates the value of `sps_seq_parameter_set_id` for the active SPS.

[0407] The `attr_coding_type` field indicates the coding type for the trait.

[0408] According to the example, a value of 0 in the attr_coding_type field indicates predicting weight lifting, a value of 1 indicates RAHT, and a value of 2 indicates fixed weight lifting.

[0409] The `aps_attr_Initial_qp` field specifies the initial value of the variable SliceQp for each slice referring to the APS.

[0410] The aps_attr_chroma_qp_offset field indicates the offset to the initial quantization parameter signaled by the syntax aps_attr_initial_qp.

[0411] The `aps_slice_qp_delta_present_flag` field indicates whether the `ash_attr_qp_delta_luma` and `ash_attr_qp_delta_chroma` syntax elements are present in the corresponding trait slice header (ASH). For example, a value of 1 in the `aps_slice_qp_delta_present_flag` field indicates that the `ash_attr_qp_delta_luma` and `ash_attr_qp_delta_chroma` syntax elements are present in the corresponding trait slice header (ASH). For example, a value of 0 in the aps_slice_qp_delta_present_flag field indicates that the ash_attr_qp_delta_luma and ash_attr_qp_delta_chroma syntax elements are not present in the corresponding trait slice header (ASH).

[0412] The APS according to the embodiment further includes the lifting_num_pred_nearest_neighbours_minus1 field, the lifting_search_range_minus1 field, and the lifting_neighbour_bias[k] field, if the value of the attr_coding_type field is 0 or 2, i.e., the coding type is predictive weighted lifting or fixed weighted lifting.

[0413] The `lifting_num_pred_nearest_neighbours_minus1` field plus 1 indicates the maximum number of nearest neighbors used for prediction. According to the example, the value of `NumPredNearestNeighbours` is set to be the same as `lifting_num_pred_nearest_neighbours`.

[0414] The `lifting_search_range_minus1` field plus 1 specifies the search range used to determine the nearest neighbors to be used for prediction and to build distance-based levels of detail (LOD). The variable `LiftingSearchRange`, which explicitly specifies the search range, can be obtained by adding 1 to the value of the `lifting_search_range_minus1` field (`LiftingSearchRange = lifting_search_range_minus1 + 1`).

[0415] The `lifting_neighbour_bias[k]` field specifies a bias used to weight the k-th components in the calculation of the Euclidean distance between two points as part of the nearest neighbor derivation process.

[0416] According to the example, if the value of the attr_coding_type field is 2, that is, if the coding type indicates fixed-weighted lifting, the APS further includes the lifting_scalability_enabled_flag field.

[0417] The `lifting_scalability_enabled_flag` field indicates whether the attribute decoding process allows the pruned octree decoding result for the input geometry points. For example, a value of 1 for the `lifting_scalability_enabled_flag` field indicates that the attribute decoding process allows the pruned octree decoding result for the input geometry points. A value of 0 for the `lifting_scalability_enabled_flag` field indicates that the attribute decoding process requires the complete octree decoding result for the input geometry points.

[0418] According to the example, if the value of the lifting_scalability_enabled_flag field is false, the APS further includes the lifting_num_detail_levels_minus1 field.

[0419] The `lifting_num_detail_levels_minus1` field specifies the number of levels of detail for attribute coding. The variable `LevelDetailCount`, used to explicitly specify the number of LODs, can be obtained by adding 1 to the value of the `lifting_num_detail_levels_minus1` field (LevelDetailCount = lifting_num_detail_levels_minus1 + 1).

[0420] According to the example, if the value of the lifting_num_detail_levels_minus1 field is greater than 1, the APS further includes the lifting_lod_regular_sampling_enabled_flag field.

[0421] The `lifting_lod_regular_sampling_enabled_flag` field indicates whether the levels of detail (LOD) are formed using a regular sampling strategy. For example, a value of 1 for the `lifting_lod_regular_sampling_enabled_flag` field indicates that the LOD is formed using a regular sampling strategy. For example, a value of 0 for the `lifting_lod_regular_sampling_enabled_flag` field indicates that a distance-based sampling strategy is used instead.

[0422] In one embodiment, when the value of the lifting_scalability_enabled_flag field is false, the APS further includes a loop statement that is repeated a number of times equal to the value of the lifting_num_detail_levels_minus1 field. In this case, the index (idx) is initialized to 0 and increments by 1 each time the loop statement is executed, and the loop statement is repeated until the index (idx) is greater than the value of the lifting_num_detail_levels_minus1 field. This loop statement includes the lifting_sampling_period_minus2[idx] field if the value of the lifting_lod_regular_sampling_enabled_flag field is true (e.g., 1), and includes the lifting_sampling_distance_squared_scale_minus1[idx] field if it is false (e.g., 0). Furthermore, if the value of idx is not 0 (idx != 0), it further includes the lifting_sampling_distance_squared_offset[idx] field.

[0423] The `lifting_sampling_period_minus2[idx]` field `plus2` specifies the sampling period for the level of detail idx.

[0424] The `lifting_sampling_distance_squared_scale_minu1 [idx]` field plus1 specifies the scale factor for the derivation of the square of the sampling distance for the level of detail idx.

[0425] The `lifting_sampling_distance_squared_offset[idx]` field specifies the offset for deriving the square of the sampling distance for the level of detail (LOD) idx.

[0426] In the embodiment, if the value of the attr_coding_type field is 0, that is, the coding type is predictive weighted lifting, the APS further includes the lifting_adaptive_prediction_threshold field, the lifting_intra_lod_prediction_num_layers field, the lifting_max_num_direct_predictors field, and the inter_component_prediction_enabled_flag field.

[0427] The `lifting_adaptive_prediction_threshold` field specifies the threshold to enable adaptive prediction. In the example, the variable `AdaptivePredictionThreshold`, which explicitly specifies the threshold to switch the adaptive predictor selection mode, is set to the same value as the `lifting_adaptive_prediction_threshold` field (AdaptivePredictionThreshold = lifting_adaptive_prediction_threshold).

[0428] The `lifting_intra_lod_prediction_num_layers` field specifies the number of LOD layers where decoded points in the same LOD layer could be referred to generate a prediction value of the target point. For example, the value of the `lifting_intra_lod_prediction_num_layers` field equal to the value of `LevelDetailCount` indicates that the target point could refer to decoded points in the same LOD layer for all LOD layers. For example, a value of 0 in the `lifting_intra_lod_prediction_num_layers` field indicates that the target point could not refer to decoded points in the same LOD layer for any LOD layer. The `lifting_max_num_direct_predictors` field indicates the maximum number of predictors used for direct prediction. The value of the `lifting_max_num_direct_predictors` field ranges from 0 to `LevelDetailCount`.

[0429] The `inter_component_prediction_enabled_flag` field indicates whether the primary component of a multi-component attribute is used to predict the reconstructed value of non-primary components. For example, a value of 1 in the `inter_component_prediction_enabled_flag` field specifies that the primary component of a multi-component attribute is used to predict the reconstructed value of non-primary components. Conversely, a value of 0 in the `inter_component_prediction_enabled_flag` field specifies that all attribute components are reconstructed independently.

[0430] According to the example, if the value of the attr_coding_type field is 1, that is, the characteristic coding type is RAHT, then the APS further includes the raht_prediction_enabled_flag field.

[0431] The `raht_prediction_enabled_flag` field indicates whether conversion weight prediction from neighboring points is enabled during the RAHT decoding process. For example, a value of 1 for the `raht_prediction_enabled_flag` field indicates that conversion weight prediction from neighboring points is enabled during the RAHT decoding process, while a value of 0 indicates that it is disabled.

[0432] According to the example, if the value of the raht_prediction_enabled_flag field is true, the APS further includes the raht_prediction_threshold0 field and the raht_prediction_threshold1 field.

[0433] The raht_prediction_threshold0 field indicates the threshold for terminating the conversion weight prediction from adjacent points.

[0434] The raht_prediction_threshold1 field indicates the threshold for skipping the conversion weight prediction from adjacent points.

[0435] The `aps_extension_flag` field indicates whether the `aps_extension_data` syntax structure exists in the corresponding APS syntax structure. For example, a value of 1 in the `aps_extension_flag` field indicates that the `aps_extension_data` syntax structure exists in the corresponding APS syntax structure. For example, a value of 0 in the `aps_extension_flag` field indicates that the `aps_extension_data` syntax structure does not exist in the corresponding APS syntax structure.

[0436] In the example, if the value of the aps_extension_flag field is 1, the APS further includes the aps_extension_data_flag field.

[0437] The `aps_extension_data_flag` field can have any value. Its existence and value do not affect the decoder standard.

[0438] The APS in the example further includes information related to the characteristic compression of the LoD base.

[0439] Figure 33 shows one example of the syntactic structure of an attribute parameter set (APS) that includes slice-related information and / or buffer management-related information according to an example.

[0440] According to the example, the APS includes the attr_slice_segmentation_enabled_flag field.

[0441] For example, a value of 1 in the `attr_slice_segmentation_enabled_flag` field indicates that the characteristic bitstream is divided and transmitted across multiple slices. In this case, additional conditions allow for parallel processing, scalable transmission, and spatial scalability. A value of 1 in the `attr_slice_segmentation_enabled_flag` field indicates that the characteristic bitstream is transmitted across each individual slice.

[0442] According to the embodiment, information relating to the segmented (separated) slices in Figure 33 and / or information relating to direct coding is included at any position in the APS in Figure 32.

[0443] Figure 34 shows one embodiment of the syntactic structure of a geometry slice bitstream according to this specification.

[0444] In the embodiment, the geometry slice bitstream (geometry_slice_bitstream) includes a geometry slice header (geometry_slice_header) and geometry slice data (geometry_slice_data). According to the embodiment, the geometry slice bitstream is also referred to as the geometry data unit, the geometry slice header as the geometry data unit header, and the geometry slice data as the geometry data unit data.

[0445] Figure 35 shows one example of the syntactic structure of the geometry_slice_header according to this specification.

[0446] The bitstream transmitted by the transmitting device (or received by the receiving device) according to the embodiment includes one or more slices. Each slice includes a geometry slice and an attribute slice. The geometry slice includes a geometry slice header (GSH). The attribute slice includes an attribute slice header (ASH).

[0447] The geometry slice header (geometry_slice_header) in this example includes the fields gsh_geometry_parameter_set_id, gsh_tile_id, gsh_slice_id, frame_idx, gsh_num_points, and byte_alignment.

[0448] The geometry slice header in the embodiment includes the gsh_box_log2_scale field, the gsh_box_origin_x field, the gsh_box_origin_y field, and the gsh_box_origin_z field, provided that the gps_box_present_flag field in the geometry parameter set (GPS) is true (e.g., 1) and the gps_gsh_box_log2_scale_present_flag field is true (e.g., 1).

[0449] The gsh_geometry_parameter_set_id field specifies the value of the gps_geom_parameter_set_id of the active GPS.

[0450] The gsh_tile_id field indicates the identifier of the tile referenced by the corresponding geometry slice header (GSH).

[0451] The `gsh_slice_id` field identifies the slice header for reference by other syntax elements.

[0452] The `frame_idx` field represents the log2_max_frame_idx + 1 least significant bits of the notional frame number counter. Consecutive slices with differing frame index values ​​form parts of different output point cloud frames. Consecutive slices with identical frame index values ​​without an intervening frame boundary marker data unit form parts of the same output point cloud frame.

[0453] The gsh_num_points field indicates the maximum number of coded points in the given slice. According to the examples, it is a requirement of bitstream conformance that gsh_num_points is greater than or equal to the number of decoded points in the slice.

[0454] The gsh_box_log2_scale field indicates the bounding box origin scaling factor for the slice in question.

[0455] The gsh_box_origin_x field indicates the x value of the bounding box origin, scaled by the value of the gsh_box_log2_scale field.

[0456] The gsh_box_origin_y field shows the y value of the bounding box origin, scaled by the value of the gsh_box_log2_scale field.

[0457] The gsh_box_origin_z field shows the z value of the bounding box origin, scaled by the value of the gsh_box_log2_scale field.

[0458] At this point, the variables slice_origin_x, slice_origin_y, and slice_origin_z are derived (extracted) as follows:

[0459] If the value of the gps_gsh_box_log2_scale_present_flag field is 0, then originScale will be gsh_box_log2_scale.

[0460] Additionally, if the value of the gps_gsh_box_log2_scale_present_flag field is 1, originScale will be gps_gsh_box_log2_scale.

[0461] Additionally, if the value of the gps_box_present_flag field is 0, the values ​​of the variables slice_origin_x, slice_origin_y, and slice_origin_z will also be 0.

[0462] If the value of the gps_box_present_flag field is 1, the following formula is applied to the variables slice_origin_x, slice_origin_y, and slice_origin_z.

[0463] slice_origin_x=gsh_box_origin_x< <originScale

[0464] slice_origin_y=gsh_box_origin_y< <originScale

[0465] slice_origin_z=gsh_box_origin_z< <originScale

[0466] The geometry_slice_header in the embodiment further includes the gsh_log2_max_nodesize_x field, the gsh_log2_max_nodesize_y_minus_x field, and the gsh_log2_max_nodesize_z_minus_y field if the value of the gps_implicit_geom_partition_flag field is true (i.e., 0), and further includes the gsh_log2_max_nodesize field if the value of the gps_implicit_geom_partition_flag field is false (i.e., 1).

[0467] The gsh_log2_max_nodesize_x field specifies the bounding box size in the x dimension, i.e., MaxNodesizeXLog2 that is used in the decoding process, as follows:

[0468] MaxNodesizeXLog2=gsh_log2_max_nodesize_x

[0469] MaxNodeSizeX=1< <MaxNodesizeXLog2

[0470] The gsh_log2_max_nodesize_y_minus_x field specifies the bounding box size in the y dimension, i.e., MaxNodesizeYLog2 that is used in the decoding process, as follows:

[0471] MaxNodeSizeYLog2=gsh_log2_max_nodesize_y_minus_x+MaxNodesizeXLog2.

[0472] MaxNodeSizeY=1< <MaxNodeSizeYLog2.

[0473] The gsh_log2_max_nodesize_z_minus_y field specifies the bounding box size in the z dimension, i.e., MaxNodesizeZLog2 that is used in the decoding process, as follows:

[0474] MaxNodeSizeZLog2=gsh_log2_max_nodesize_z_minus_y+MaxNodeSizeYLog2

[0475] MaxNodeSizeZ=1< <MaxNodeSizeZLog2

[0476] The gsh_log2_max_nodesize field is calculated as follows, assuming the value of the gps_implicit_geom_partition_flag field is 1.

[0477] gsh_log2_max_nodesize=max{MaxNodesizeXLog2,MaxNodeSizeYLog2,MaxNodeSizeZLog2}

[0478] The gsh_log2_max_nodesize field indicates the size of the root geometry octvine node when the value of the gps_implicit_geom_partition_flag field is 0.

[0479] At this time, the variables MaxNodeSize and MaxGeometryOctreeDepth are calculated as follows.

[0480] MaxNodeSize=1< <gsh_log2_max_nodesize

[0481] MaxGeometryOctreeDepth=gsh_log2_max_nodesize?log2_trisoup_node_size

[0482] The geometry_slice_header in the example further includes the geom_slice_qp_offset field and the geom_octree_qp_offsets_enabled_flag field if the value of the geom_scaling_enabled_flag field is true.

[0483] The `geom_slice_qp_offset` field specifies an offset to the base geometry quantization parameter `geom_base_qp`.

[0484] The `geom_octree_qp_offsets_enabled_flag` field indicates whether the `geom_octree_qp_offsets_depth` field exists in the corresponding geometry slice header. For example, a value of 1 for the `geom_octree_qp_offsets_enabled_flag` field indicates that the `geom_octree_qp_offsets_depth` field exists in the corresponding geometry slice header, while a value of 0 indicates that it does not exist.

[0485] The `geom_octree_qp_offsets_depth` field indicates the depth of the geometry octree.

[0486] Figure 36 shows one example of the syntactic structure of a geometry data unit header (or geometry slice header) that includes slice-related information and / or buffer management-related information according to an embodiment.

[0487] The geometry data unit header in the example includes the slice_id field and the dependent_neighbour_enabled_flag field.

[0488] The slice_id field indicates an identifier for the relevant data unit (i.e., slice). That is, the slice_id field indicates a directive for distinguishing slices or data units and conveys a directive for data units (or slices) belonging to a slice layer. Alternatively, the slice_id field identifies the relevant slice header for reference by other syntactic elements.

[0489] For example, a value of 1 in the `dependent_neighbour_enabled_flag` field indicates that adjacent information from outside the slice will be used during the coding process. A value of 0 in the `dependent_neighbour_enabled_flag` field indicates that adjacent relationships will be estimated using only internal information, without using adjacent information from outside the slice.

[0490] The geometry data unit header in the embodiment further includes the gsh_entropy_continuation_flag field if the value of the sps_entropy_continuation_enabled_flag field is false (i.e., 0), and further includes the gsh_prev_slice_id field if the value of the gsh_entropy_continuation_flag field is true (i.e., 1).

[0491] The sps_entropy_continuation_enabled_flag field is included in the SPS, and a value of 0 for this field explicitly indicates that the initial entropy context state of each slice is independent.

[0492] For example, a value of 1 in the gsh_entropy_continuation_flag field indicates that the parsing state used in the entropy coding of the current geometry data unit is dependent on the final parsing state of the previous geometry data unit, and that the parsing state used in the entropy coding of the attribute data unit that refers to the current geometry data unit is dependent on the final parsing state of the previous attribute data unit.

[0493] For example, a value of 0 in the gsh_entropy_continuation_flag field indicates that the parsing state used in the entropy coding of the current geometry data unit and attribute data unit that refers to the current geometry data unit does not depend upon any previous data unit. In this embodiment, it is a requirement of bitsream conformance that the value of the gsh_entropy_continuation_flag field is 0 when the current geometry data unit is the first data unit in a point cloud frame.

[0494] The gsh_prev_slice_id field indicates the value of the gsh_slice_id field (or slice_id field) of the preceding geometry data unit in bitstream order.

[0495] The geometry data unit header in the embodiment further includes a context_reuse_flag field if the value of the geom_slice_segmentation_enabled_flag field is 1, and further includes a num_context_reuse_minus1 field if the value of the context_reuse_flag field is 1.

[0496] A value of 1 in the geom_slice_segmentation_enabled_flag field indicates that the geometry bitstream will be divided and transmitted across multiple slices.

[0497] For example, a value of 1 in the context_reuse_flag field indicates that the context of the current slice will be used for at least one subsequent slice. In the embodiment, if context buffer control is used in the receiving device, the context of the current slice can be stored in the context buffer for at least one subsequent slice. A value of 0 in the context_reuse_flag field indicates that the context of the current slice will not be used for the next slice.

[0498] Adding 1 to the num_context_reuse_minus1 field indicates the number of times the current slice's context will be used for the next slice.

[0499] For example, when using context buffer control in a receiving device, a counter (called a context reference counter) can be used to delete the context of the current slice in the context buffer after the num_context_reuse_minus1 field + 1 has been referenced.

[0500] In one embodiment, the geometry data unit header contains a loop statement that is repeated as many times as the value of the num_context_reuse_minus1 field. In this embodiment, i is initialized to 0 and increments by 1 each time the loop statement is executed, and the loop statement is repeated until the value of i reaches the value of the num_context_reuse_minus1 field. This loop statement includes the subsequent_slice_id field.

[0501] The `subsequent_slice_id` field indicates an identifier for the i-th next slice that currently uses the context. In other words, the `subsequent_slice_id` field is used to indicate the next slice that uses the context of the current slice.

[0502] For example, if a receiving device uses context buffer control and receives a slice identified by the subsequent_slice_id field (i.e., the next slice), it can be used to decrement a counter (or context reference counter) as follows, or to delete the current context in the context buffer after referencing num_context_reuse_minus1+1 times using the counter. That is, the counter value is decremented by 1 each time it is used for the next slice, and when the counter value reaches 0, the context is deleted from the context buffer.

[0503] NumContextReuse=num_context_reuse_minus1+1

[0504] If(subsequent_slice_id==slice_id)

[0505] NumContextReuse=NumContextReuse-1

[0506] The geometry data unit header in the embodiment further includes a num_neighbour_slice field if the value of the dependent_neighbour_enabled_flag field is 1, and further includes a neighborhood_occupancy_type field if the value of the dependent_neighbour_enabled_flag field is 0.

[0507] The `num_neighbour_slice` field indicates the number of slices containing a node when referencing node information outside of a slice.

[0508] The `neighbor_occupancy_type` field defines the assumptions made about a node when it does not refer to node information outside the slice. For example, a value of 0 for the `neighbor_occupancy_type` field indicates that there are no neighboring nodes; 1 indicates that all nodes are occupied; 2 indicates that the node is unoccupied; and 3 indicates that the occupancy information of nodes within a slice that is symmetrical with respect to the central node is used.

[0509] In one embodiment, the geometry data unit header contains a loop statement that is repeated as many times as the value of the num_neighbour_slice field. In this embodiment, i is initialized to 0 and increments by 1 each time the loop statement is executed, and the loop statement is repeated until the value of i reaches the value of the num_neighbour_slice field. This loop statement includes the neighbor_slice_id field.

[0510] The neighbor_slice_id field is an identifier used to identify the i-th slice containing a node when referencing node information outside of a slice.

[0511] According to the embodiment, the slice-related information and / or buffer management-related information in Figure 36 is included at any position in the geometry slice header (i.e., geometry data unit header) in Figure 35.

[0512] Figure 37 shows one embodiment of the syntactic structure of geometry_slice_data according to this specification. The geometry_slice_data according to this embodiment transmits the geometry bitstream belonging to the slice (or data unit) in question.

[0513] In one example, the geometry slice data (geometry_slice_data) in the embodiment includes a first iteration statement that is repeated as many times as the value of MaxGeometryOctreeDepth. In this embodiment, the depth is initialized to 0 and increases by 1 each time the iteration statement is executed, and the first iteration statement is repeated until the depth reaches the value of MaxGeometryOctreeDepth. The first iteration statement includes a second iteration statement that is repeated as many times as the value of NumNodesAtDepth. In this embodiment, nodeidx is initialized to 0 and increases by 1 each time the iteration statement is executed, and the second iteration statement is repeated until nodeidx reaches the value of NumNodesAtDepth. The second iteration statement includes xN=NodeX[depth][nodeidx], yN=NodeY[depth][nodeidx], zN=NodeZ[depth][nodeidx], and geometry_node(depth,nodeidx,xN,yN,zN). MaxGeometryOctreeDepth indicates the maximum geometry octree depth, and NumNodesAtDepth[depth] indicates the number of nodes decoded at that depth. The variables NodeX[depth][nodeidx], NodeY[depth][nodeidx], and NodeZ[depth][nodeidx] indicate the x,y,z coordinates of the idx-th nodes in decoded order at a given depth. geometry_node(depth,nodeidx,xN,yN,zN) sends the geometry bitstream of the corresponding node at the given depth.

[0514] In the example, the geometry slice data (geometry_slice_data) further includes geometry_trisoup_data if the value of the log2_trisoup_node_size field is greater than 0. That is, if the size of a triangle node is greater than 0, a geometry bitstream encoded with trisoup geometry is sent using geometry_trisoup_data.

[0515] Figure 38 shows one embodiment of the syntactic structure of a characteristic slice bitstream according to this specification.

[0516] In the example, the attribute slice bitstream (attribute_slice_bitstream) includes an attribute slice header (attribute_slice_header) and attribute slice data (attribute_slice_data). According to the example, the attribute slice bitstream is also called the attribute data unit, the attribute slice header is also called the attribute data unit header, and the attribute slice data is also called the attribute data unit data.

[0517] Figure 39 shows one example of the syntactic structure of the attribute_slice_header according to this specification.

[0518] The attribute slice header (attribute_slice_header) in the example includes the fields ash_attr_parameter_set_id, ash_attr_sps_attr_idx, ash_attr_geom_slice_id, ash_attr_layer_qp_delta_present_flag, and ash_attr_region_qp_delta_present_flag.

[0519] In the example, the attribute slice header (attribute_slice_header) further includes the ash_attr_qp_delta_luma field if the value of the aps_slice_qp_delta_present_flag field in the attribute parameter set (APS) is true (e.g., 1), and further includes the ash_attr_qp_delta_chroma field if the value of the attribute_dimension_minus1[ash_attr_sps_attr_idx] field is greater than 0.

[0520] The ash_attr_parameter_set_id field indicates the value of the aps_attr_parameter_set_id field of the currently active APS.

[0521] The ash_attr_sps_attr_idx field indicates the set of traits in the currently active SPS.

[0522] The ash_attr_geom_slice_id field currently indicates the value of the gsh_slice_id field in the geometry slice header.

[0523] The ash_attr_qp_delta_luma field represents the luma-delta quantization parameter (qp) derived from the initial slice qp in the active trait parameter set.

[0524] The ash_attr_qp_delta_chroma field indicates the chromium delta quantization parameter (qp) derived from the initial slice qp in the active trait parameter set.

[0525] At this point, the variables InitialSliceQpY and InitialSliceQpC are derived as follows.

[0526] InitialSliceQpY=aps_attrattr_Initial_qp+ash_attr_qp_delta_luma

[0527] InitialSliceQpC=aps_attrtr_Initial_qp+aps_attr_chroma_qp_offset+ash_attr_qp_delta_chroma

[0528] The `ash_attr_layer_qp_delta_present_flag` field indicates whether the `ash_attr_layer_qp_delta_luma` and `ash_attr_layer_qp_delta_chroma` fields exist in the corresponding trait slice header (ASH) for each layer. For example, a value of 1 for the `ash_attr_layer_qp_delta_present_flag` field indicates that the `ash_attr_layer_qp_delta_luma` and `ash_attr_layer_qp_delta_chroma` fields exist in the corresponding trait slice header, while a value of 0 indicates that they do not exist.

[0529] If the value of the ash_attr_layer_qp_delta_present_flag field is true, the trait slice header will also include the ash_attr_num_layer_qp_minus1 field.

[0530] The `ash_attr_num_layer_qp_minus1` field plus 1 indicates the number of layers for which the `ash_attr_qp_delta_luma` and `ash_attr_qp_delta_chroma` fields are signaled. If the `ash_attr_num_layer_qp` field is not signaled, its value will be 0. According to the example, NumLayerQp, which explicitly indicates the number of layers, is obtained by adding 0 to the value of the `ash_attr_num_layer_qp_minus1` field (NumLayerQp = ash_attr_num_layer_qp_minus1 + 1).

[0531] In one embodiment, the geometry slice header contains a loop of the value of NumLayerQp if the value of the ash_attr_layer_qp_delta_present_flag field is true. In this embodiment, i is initialized to 0 and increases by 1 each time the loop is executed, and the loop continues until i is equal to the value of NumLayerQp. This loop contains the ash_attr_layer_qp_delta_luma[i] field. Furthermore, if the value of the attribute_dimension_minus1[ash_attr_sps_attr_idx] field is greater than 0, the loop also contains the ash_attr_layer_qp_delta_chroma[i] field.

[0532] The ash_attr_layer_qp_delta_luma field shows the luma-delta quantization parameter (qp) from InitialSliceQpY for each layer.

[0533] The ash_attr_layer_qp_delta_chroma field shows the chromium delta quantization parameter (qp) from InitialSliceQpC for each layer.

[0534] The variables SliceQpY[i] and SliceQpC[i] with i=0…1 are derived as follows.

[0535] for (i=0;i <NumLayerQPNumQPLayer;i++){

[0536] SliceQpY[i]=InitialSliceQpY+ash_attr_layer_qp_delta_luma[i]

[0537] SliceQpC[i]=InitialSliceQpC+ash_attr_layer_qp_delta_chroma[i]

[0538] }

[0539] In the example, an attribute slice header (attribute_slice_header) with a value of 1 in the ash_attr_region_qp_delta_present_flag field indicates that ash_attr_region_qp_delta, region bounding box origin, and size are currently present in the attribute slice header. Conversely, if the value of the ash_attr_region_qp_delta_present_flag field is 0, it indicates that ash_attr_region_qp_delta, region bounding box origin, and size are not currently present in the attribute slice header.

[0540] That is, if the value of the ash_attr_layer_qp_delta_present_flag field is 1, the characteristic slice header further includes the ash_attr_qp_region_box_origin_x field, ash_attr_qp_region_box_origin_y field, ash_attr_qp_region_box_origin_z field, ash_attr_qp_region_box_width field, ash_attr_qp_region_box_height field, ash_attr_qp_region_box_depth field, and ash_attr_region_qp_delta field.

[0541] The `ash_attr_qp_region_box_origin_x` field indicates the x offset of the region bounding box relative to `slice_origin_x`.

[0542] The `ash_attr_qp_region_box_origin_y` field indicates the y offset of the region bounding box relative to `slice_origin_y`.

[0543] The `ash_attr_qp_region_box_origin_z` field indicates the z offset of the region bounding box relative to `slice_origin_z`.

[0544] The `ash_attr_qp_region_box_size_width` field specifies the width of the region bounding box.

[0545] The ash_attr_qp_region_box_size_height field specifies the height of the region bounding box.

[0546] The ash_attr_qp_region_box_size_depth field indicates the depth of the region bounding box.

[0547] The ash_attr_region_qp_delta field indicates the delta qp from SliceQpY[i] and SliceQpC[i] of the region specified by the ash_attr_qp_region_box field.

[0548] According to the example, the variable RegionboxDeltaQp, which explicitly specifies the region box delta quantization parameter, is set to the same value as the ash_attr_region_qp_delta field (RegionboxDeltaQp = ash_attr_region_qp_delta).

[0549] Figure 40 shows one example of the syntactic structure of a characteristic data unit header (or characteristic slice header) containing slice-related information and / or buffer management-related information according to an embodiment.

[0550] The characteristic data unit header in the example includes the slice_id field and the dependent_neighbour_enabled_flag field.

[0551] The slice_id field indicates an identifier for identifying the data unit (i.e., slice). That is, the slice_id field indicates a directive for distinguishing slices or data units, and can convey directives for data units (or slices) belonging to a slice layer. Alternatively, the slice_id field can identify the slice header for reference by other syntactic elements.

[0552] For example, a value of 1 in the `dependent_neighbour_enabled_flag` field indicates that adjacent information from outside the slice will be used during the coding process. A value of 0 in the `dependent_neighbour_enabled_flag` field indicates that adjacent relationships will be estimated using only internal information, without using adjacent information from outside the slice.

[0553] The characteristic data unit header in the embodiment further includes an ash_continuation_flag field if the value of the sps_entropy_continuation_enabled_flag field is false (i.e., 0), and an ash_prev_slice_id field if the value of the ash_entropy_continuation_flag field is true (i.e., 1).

[0554] The sps_entropy_continuation_enabled_flag field is included in the SPS, and a value of 0 in this field explicitly indicates that the initial entropy context states of each slice are independent.

[0555] For example, a value of 1 in the ash_entropy_continuation_flag field indicates that the parsing state used for the entropy coding of the current trait data unit will change depending on the last parsing state of the previous trait data unit. Conversely, a value of 0 in the ash_entropy_continuation_flag field indicates that the parsing state used for the entropy coding of the current trait data unit will not change depending on any previous data unit. In this example, if the current trait data unit is the first data unit in the point cloud frame, then a value of 0 in the ash_entropy_continuation_flag field is a requirement for bitstream compatibility.

[0556] The ash_prev_slice_id field indicates the value of the ash_slice_id field (or slice_id field) of the preceding (previous) attribute data unit in bitstream order.

[0557] The characteristic data unit header in the embodiment further includes a context_reuse_flag field if the value of the attr_slice_segmentation_enabled_flag field is 1, and further includes a num_context_reuse_minus1 field if the value of the context_reuse_flag field is 1.

[0558] A value of 1 in the `attr_slice_segmentation_enabled_flag` field indicates that the trait bitstream will be divided and propagated across multiple slices.

[0559] For example, a value of 1 in the context_reuse_flag field indicates that the context of the current slice will be used for at least one subsequent slice. In the embodiment, if context buffer control is used in the receiving device, the context of the current slice can be stored in the context buffer for at least one subsequent slice. A value of 0 in the context_reuse_flag field indicates that the context of the current slice will not be used for the next slice.

[0560] Adding 1 to the num_context_reuse_minus1 field indicates the number of times the current slice's context will be used for the next slice.

[0561] For example, when using context buffer control in a receiving device, a counter (called a context reference counter) can be used to delete the context of the current slice in the context buffer after the num_context_reuse_minus1 field + 1 has been referenced.

[0562] In one embodiment, the characteristic data unit header contains a loop statement that is repeated as many times as the value of the num_context_reuse_minus1 field. In this embodiment, i is initialized to 0 and increases by 1 each time the loop statement is executed, and the loop statement is repeated until the value of i reaches the value of the num_context_reuse_minus1 field. This loop statement includes the subsequent_slice_id field.

[0563] The `subsequent_slice_id` field indicates an identifier for the i-th next slice that currently uses the context. In other words, the `subsequent_slice_id` field is used to indicate the next slice that uses the context of the current slice.

[0564] For example, if the receiving device uses context buffer control and a slice identified by the subsequent_slice_id field (i.e., the next slice) is received, it can be used to decrement a counter (or context reference counter) or to delete the current context in the context buffer after the counter has referenced num_context_reuse_minus1+1 times, as follows:

[0565] NumContextReuse=num_context_reuse_minus1+1

[0566] If(subsequent_slice_id==slice_id)

[0567] NumContextReuse=NumContextReuse-1

[0568] In the embodiment, the characteristic data unit header further includes a num_neighbour_slice field if the value of the dependent_neighbour_enabled_flag field is 1, and further includes a neighbor_occupancy_type field if the value of the dependent_neighbour_enabled_flag field is 0.

[0569] The `num_neighbour_slice` field indicates the number of slices containing a node when referencing node information outside of a slice.

[0570] The `neighbor_occupancy_type` field defines the assumptions made about a node when it does not refer to node information outside the slice. For example, a value of 0 for the `neighbor_occupancy_type` field indicates that there are no neighboring nodes; 1 indicates that it is fully occupied; 2 indicates that it is unoccupied; and 3 indicates that it uses the occupation information of nodes within a symmetrical slice based on the central node.

[0571] In one embodiment, the characteristic data unit header contains a loop statement that is repeated as many times as the value of the num_neighbour_slice field. In this embodiment, i is initialized to 0 and increases by 1 each time the loop statement is executed, and the loop statement is repeated until the value of i reaches the value of the num_neighbour_slice field. This loop statement includes the neighbor_slice_id field.

[0572] The neighbor_slice_id field is an identifier used to identify the i-th slice containing a node when referencing node information outside of a slice.

[0573] According to the embodiment, the slice-related information and / or buffer management-related information in Figure 40 is included at any position in the characteristic slice header (i.e., characteristic data unit header) in Figure 39.

[0574] Figure 41 shows one embodiment of the syntactic structure of attribute slice data (attribute_slice_data) according to this specification. The attribute slice data (attribute_slice_data) according to the embodiment transmits the attribute bitstream belonging to the slice. The attribute slice data according to the embodiment includes attribute or attribute-related data in relation to part or all of the point cloud.

[0575] In the attribute slice data (attribute_slice_data) in Figure 41, dimension=attribute_dimension[ash_attr_sps_attr_idx] indicates the attribute dimension (attribute_dimension) of the attribute set identified by the ash_attr_sps_attr_idx field in the corresponding attribute slice header. The attribute dimension (attribute_dimension) means the number of components that make up the attribute. Attributes in the examples represent reflectance, hue, etc. Therefore, the number of components that an attribute has will differ. For example, an attribute corresponding to hue has three hue components (e.g., RGB). Therefore, an attribute corresponding to reflectance is a mono-dimensional attribute, and an attribute corresponding to hue is a three-dimensional attribute.

[0576] The characteristics described in the examples are characteristic-encoded on a dimension-by-dimension basis.

[0577] For example, the characteristic corresponding to reflectance and the characteristic corresponding to hue are characteristic-encoded, respectively. Furthermore, the characteristics according to the embodiment are characteristic-encoded together regardless of dimension. For example, the characteristic corresponding to reflectance and the characteristic corresponding to hue are characteristic-encoded together.

[0578] In Figure 41, zerorun indicates the number of zeros prior to the residual characteristic value.

[0579] In Figure 41, i represents the i-th point value of that characteristic, and in one example, the attr_coding_type field and lifting_adaptive_prediction_threshold field are signaled to APS.

[0580] Furthermore, the variable MaxNumPredictors in Figure 41 is a variable used in the point cloud data decoding process and is obtained as follows based on the lifting_adaptive_prediction_threshold field value signaled to APS.

[0581] MaxNumPredictors = lifting_max_num_direct_predicots field + 1

[0582] Here, the `lifting_max_num_direct_predictors` field indicates the maximum number of predictors used for direct prediction.

[0583] In the example, predIndex[i] specifies the predictor index (also called the prediction mode) for decoding the i-th point value of the attribute. The value of predIndex[i] ranges from 0 to the value of the lifting_max_num_direct_predictors field.

[0584] Figure 42 shows the structure of a point cloud data transmission device according to an embodiment.

[0585] The transmitting devices according to the embodiment correspond to the transmitting device 10000 in Figure 1, the point cloud video encoder 10002, the transmitter 10003, the acquire-encode-transmit 20000-20001-20002 in Figure 2, the encoder in Figure 4, the transmitting device in Figure 12, the device in Figure 14, and the encoder in Figure 20, etc. Each component in Figure 42 corresponds to hardware, software, a processor, and / or a combination thereof.

[0586] The operation of the encoder and transmitter in this embodiment is as follows.

[0587] When point cloud data is input to the transmitting device, the geometry encoder 60010 encodes the positional information (geometry data: eg, XYZ coordinates, phi-theta coordinates, etc.) from the point cloud data, and the attribute encoder 60020 encodes the attribute information (attribute data: eg, color, reflectance, intensity, grayscale, opacity, medium, material, glossiness, etc.) from the point cloud data.

[0588] The compressed (encoded) data is divided into units for transmission, but via the sub-bitstream generation unit 60040, the layering structure information allows for the selection of necessary information in each bitstream unit and subsequent packing.

[0589] In this embodiment, the geometry bitstream output by the geometry encoder 60010 and / or the characteristic bitstream output by the characteristic encoder 60020 are input to the sub-bitstream generation unit 60040. The sub-bitstream generation unit 60040 divides each bitstream into multiple sub-bitstreams and constructs them into multiple slices based on the slice-related information and / or buffer management-related information output by the metadata generation unit 60030. For detailed information regarding the division of each bitstream and the configuration of the slices, please refer to Figures 17 to 24 described above, and a detailed explanation is omitted here. In this embodiment, the slice-related information includes layering structure information, and the slice-related information is information indicating the bitstream configuration, alignment, selection, and slice configuration as described in Figures 17 to 24, and refers to the information shown in Figures 27 to 41, etc.

[0590] According to the embodiment, the metadata generation unit 60030 generates slice-related information and / or buffer management-related information, including layering structure information.

[0591] Slice-related information and / or buffer management-related information are included in one of the following messages and transmitted to the receiving device: SPS, GPS, TPS, APS, geometry slice header, characteristic slice header, and SEI message. Slice-related information and / or buffer management-related information includes one of the following fields: sps_entropy_continuation_enabled_flag field, geom_slice_segmentation_enabled_flag field, attr_slice_segmentation_enabled_flag field, slice_id field, gsh_entropy_continuation_flag field, gsh_prev_slice_id field, context_reuse_flag field, num_context_reuse_minus1 field, subsequent_slice_id field, dependent_neighbour_enabled_flag field, num_neighbour_slice field, neighborhood_slice_id field, neighborhood_occupancy_type field, ash_continuation_flag field, and ash_prev_slice_id field. For a description of each of the above fields, please refer to the explanations in Figures 27 to 41, and a detailed explanation is omitted here.

[0592] In other embodiments, the sub-bitstream generation unit 60040 divides each bitstream, generates slice-related information (or layering structure information) indicating the division process, and transmits it to the metadata generation unit 60030. The metadata generation unit 60030 receives information indicating the geometry encoding process and the characteristic encoding process from each encoder 60010, 60020 and generates it as metadata (parameters).

[0593] According to the embodiment, the sub-bitstream generation unit 60040 performs a slice segmentation process based on slice-related information and / or buffer-related information provided by the metadata generation unit 60030 in order to transmit multiple sub-bitstreams separated from the geometry bitstream as multiple slices. Furthermore, the sub-bitstream generation unit 60040 performs a slice segmentation process based on slice-related information and / or buffer-related information provided by the metadata generation unit 60030 in order to transmit multiple sub-bitstreams separated from the characteristic bitstream as multiple slices. That is, by transmitting the geometry bitstream (or geometry data) and characteristic bitstream (or characteristic data) as multiple slices, the receiving device can perform selective decoding or parallel decoding.

[0594] In this embodiment, multiple slices containing geometry data are sent first, followed by multiple slices containing characteristic data. For detailed information, please refer to the explanation in Figure 23; it is omitted here.

[0595] In other embodiments, layers may transmit slices of geometry data and slices of characteristic data. For details, please refer to the explanation in Figure 24; details are omitted here.

[0596] In some embodiments, multiple slices may be independent of each other or may have dependencies on each other. In some embodiments, in the case of octvine-based geometry coding, compression performance can be improved by sequentially and cumulatively using contextual information about previously accessed nodes. In the case of adjacent search and interleaving prediction, the occupation information of previously decoded adjacent (or peripheral) nodes is used, in which case the information of the immediately preceding slice can be used. Alternatively, information from a preceding slice can be used for parallel processing. In this case, dependencies between slices occur. In other embodiments, at least one slice may be independent without any relationship to other slices. A description of dependent slices and / or independent slices is provided in the explanations in Figures 20 to 25 and is omitted here.

[0597] In the embodiment, when the data belonging to each slice (e.g., geometry data or characteristic data) is encoded by the encoder, a context is generated for each slice. When contexts are used sequentially, context buffer management by the receiving device is required. In the embodiment, the details of context buffer management are explained in Figures 25 and 26, so they are omitted here. Furthermore, the signaling information for managing the receiving device's context buffer (e.g., slice-related information and / or buffer management-related information) is explained in Figures 27 to 41, and is omitted here.

[0598] The multiplexer 60060 multiplexes multiple segments output by the sub-bitstream generation unit 60040 and signaling information generated by the metadata generation unit 60050 and outputs it to the transmitter 60070. Multiplexing by the multiplexer 60050 is performed layer by layer. Refer to Figures 27 to 48 for slice-related information and / or buffer management-related information generated by the metadata generation unit 60030.

[0599] In this embodiment, transmitter 60060 transmits data (or a bitstream in slice units) multiplexed by multiplexer 60050. The bitstream in this embodiment is encapsulated in a file or segment (e.g., a streaming segment) and transmitted over various networks such as broadcast networks and / or broadband networks. Although not shown, transmitter 60060 includes an encapsulation unit (or encapsulation module) that performs the encapsulation operation.

[0600] Figure 43 shows the structure of a point cloud data receiving device according to an embodiment.

[0601] The receiving device according to the embodiment in Figure 43 corresponds to the receiving device 10004, receiver 10005, point cloud video decoder 10006 in Figure 1, the transmit-decode-render 20002-20003-20004 in Figure 2, the decoder in Figure 10, the decoder in Figure 11, the receiving device in Figure 13, and the device in Figure 14, etc. Each component in Figure 43 corresponds to hardware, software, a processor, and / or a combination thereof.

[0602] The operation of the decoder / receiver according to the embodiment is as follows:

[0603] When a bitstream is input to the receiver 65010 of the receiving device, the receiver 65010 outputs the bitstream to the demultiplexer 65020, which separates the bitstream containing geometry information and characteristic information from signaling information containing slice-related information and / or buffer management-related information.

[0604] The bitstream containing the segmented geometry information and characteristic information is output to the sub-bitstream classifier 65040, and the segmented signaling information is output to the metadata parser 65030.

[0605] The sub-bitstream classification unit 65040 processes the bitstream containing geometry information and characteristic information based on the information of each header of one or more slices and / or slice-related information and / or buffer management-related information. After processing, the bitstream (or sub-bitstream) containing geometry information is output to the geometry decoder 65060, and the bitstream (or sub-bitstream) containing characteristic information is output to the characteristic decoder 65080.

[0606] Alternatively, the receiver may select the layers it requires during this process. The classified bitstream can then be reconstructed into geometry data and characteristic data by the geometry decoder 65060 and characteristic decoder 65080, respectively, based on the characteristics of the data, and then converted to the final output format by the renderer 65090.

[0607] According to the embodiment, the sub-bitstream classification unit 65040 classifies / selects bitstreams based on metadata obtained by the metadata parser 65030 (e.g., slice-related information and / or buffer management-related information).

[0608] Slice-related information and / or buffer management-related information is received in one of the following: SPS, GPS, TPS, APS, geometry slice header, characteristic slice header, and SEI message. Slice-related information and / or buffer management-related information includes one of the following fields: sps_entropy_continuation_enabled_flag field, geom_slice_segmentation_enabled_flag field, attr_slice_segmentation_enabled_flag field, slice_id field, gsh_entropy_continuation_flag field, gsh_prev_slice_id field, context_reuse_flag field, num_context_reuse_minus1 field, subsequent_slice_id field, dependent_neighbour_enabled_flag field, num_neighbour_slice field, neighborhood_slice_id field, neighborhood_occupancy_type field, ash_continuation_flag field, and ash_prev_slice_id field. For explanations of each of the fields mentioned above, please refer to the explanations in Figures 27 to 41; detailed explanations are omitted here.

[0609] According to the embodiment, point cloud data consists of location (i.e., geometry) information and characteristic information such as hue / brightness / reflectivity for each point, and these are compressed and transmitted to the receiving device. In this case, the transmitting device uses a method of transmitting the point cloud data (or coded bitstream) in multiple slices so that the receiving device can decode or represent only a portion of the point cloud data depending on the performance or transmission speed of the receiving device. In this case, the multiple slices are signaled and dependencies between slices are defined so that the current slice uses information generated in the previous slice, but in this case, buffer management by the receiving device is required.

[0610] In one embodiment, the geometry decoder 60060 adjusts the geometry buffer 65050 based on slice-related information and / or buffer management-related information provided by the metadata parser 65030. In one embodiment, the geometry buffer 65050 is the context buffer described in Figures 25 and 26. For example, when using the geometry context of the current slice for the next slice, the geometry context of the current slice is stored in the geometry buffer 65050, and the stored information can be used until the context counter (e.g., see N or num_reuse_minus1) becomes 0 or when the matching next slice identifier (slice_id) is entered. Alternatively, as described in Figure 26, when using neighbor node information (e.g., dependent_neighbour_enabled_flag==1), the relevant node information can be stored in the geometry buffer 65050 and used when needed. A detailed explanation of the management (or control) of the geometry buffer 65050 is provided in Figures 25 and 41 and is omitted here.

[0611] In one embodiment, the characteristic decoder 60680 adjusts the characteristic buffer 65070 based on slice-related information and / or buffer management-related information provided by the metadata parser 65030. In one embodiment, the characteristic buffer 65070 is the context buffer described in Figures 25 and 26. The management (or control) of the characteristic buffer 65070 is described in Figures 25 and 41 and is omitted here.

[0612] As described above, to efficiently manage the receiver's buffer (or context buffer), signaling can be used to indicate whether the current slice's coding information / node information (e.g., slice-related information and / or buffer management-related information) will be used in the next slice. Based on this, the receiver can efficiently manage its buffer by storing and deleting the current slice's context. In other words, even when transmitting point cloud data divided into multiple slices, the transmitting device can improve compression efficiency by allowing the use of consecutive coding information and / or adjacent node information. Similarly, the receiving device can efficiently manage its buffer because it knows whether the current slice's coding information / node information (e.g., slice-related information and / or buffer management-related information) will be used in the next slice. For example, the receiving device can efficiently manage resources by predetermining whether the current slice's context will be used in the next slice.

[0613] Each of the parts, modules, or units described above are software, processor, and hardware parts that perform a sequence of execution processes stored in memory (or a storage unit). Each step described in the above embodiment is performed by the processor, software, and hardware parts. Each module / block / unit described in the above embodiment operates as a processor, software, and hardware. The method presented in the embodiment is executed as code. This code is written to a processor-readable storage medium and is therefore read by the processor provided by the device.

[0614] Furthermore, throughout the specification, when a part "includes" a certain component, unless otherwise specified, this means that it includes other components, not that it excludes other components. Also, terms such as "...part" in the specification mean a unit that processes at least one function or operation, which is embodied by hardware, software, or a combination of hardware and software.

[0615] For the sake of explanation, each figure has been described separately, but it is also possible to combine the embodiments described in each figure to design a new embodiment. Furthermore, designing a computer-readable recording medium on which a program for executing the previously described embodiments is recorded, as is the need of a typical engineer, also falls within the scope of the rights of the embodiments.

[0616] As stated above, the apparatus and methods of the embodiments are not limited to the configurations and methods of the embodiments described above, and the embodiments can be configured by selectively combining all or part of each embodiment in a variety of modifiable ways.

[0617] While preferred embodiments of the embodiments have been shown and described, the embodiments are not limited to the specific embodiments described above, and various modifications can be implemented by a person with ordinary skill in the art to which the invention belongs, without deviating from the gist of the embodiments claimed in the claims, and such modifications should not be understood individually from the technical idea or prospects of the embodiments.

[0618] The various components of the apparatus according to the embodiment consist of hardware, software, firmware, or a combination thereof. The various components of the embodiment are embodied in a single chip, for example, a single hardware circuit. In the embodiment, each component of the embodiment is embodied in an individual chip. In the embodiment, any of the components of the apparatus according to the embodiment consists of one or more processors capable of executing one or more programs, and one or more programs include instructions to cause or execute one or more of the operations / methods of the embodiment. The executable instructions for performing the methods / operations of the apparatus according to the embodiment may be stored in a non-temporary CRM or other computer program product configured to be executed by one or more processors, or in a temporary CRM or other computer program product configured to be executed by one or more processors. Furthermore, memory in the embodiment is used as a concept that includes not only volatile memory (e.g., RAM), but also non-volatile memory, flash memory, PROM, etc. Furthermore, it also includes being embodied in a carrier wave form, such as transmission over the Internet. Furthermore, the recording medium read by the processor can be distributed across a network of connected computer systems, and the code read by the processor can be stored and executed in a distributed manner.

[0619] In this specification, " / " and "," are interpreted as "and / or". For example, "A / B" is interpreted as "A and / or B", and "A, B" is interpreted as "A and / or B". Furthermore, "A / B / C" means "any of A, B and / or C". Also, "A, B, C" means "any of A, B and / or C". Furthermore, in this document, "or" is interpreted as "and / or". For example, "A or B" means 1) only "A", 2) only "B", or 3) "A and B". In other words, in this specification, "or" means "additionally or alternatively".

[0620] The various components of the embodiment consist of hardware, software, firmware, or a combination thereof. The various components of the embodiment are carried out on a single chip, such as a hardware circuit. In the embodiment, they are selectively carried out on individual chips. In the embodiment, any of the components of the embodiment are carried out within one or more processors that include instructions to perform the operation according to the embodiment.

[0621] Furthermore, the operations according to the embodiments described in this specification are performed by a transceiver including one or more memories and / or one or more processors, depending on the embodiment. The one or more memories store programs for processing / controlling the operations according to the embodiments, and the one or more processors control the various operations described in this specification. The one or more processors are also referred to as controllers. In the embodiments, the operations are performed by firmware, software, and / or a combination thereof, and the firmware, software, and / or a combination thereof are stored in the processor or in memory.

[0622] Terms such as "first," "second," etc., are used to describe the various components of the embodiment. However, the interpretation of the various components in the embodiment should not be limited by these terms. Such terms are merely used to distinguish one component from another. For example, the first user input signal may be referred to as the second user input signal. Similarly, the second user input signal may be referred to as the first user input signal. The use of such terms does not deviate from the scope of the various embodiments. While both the first user input signal and the second user input signal are user input signals, they do not mean the same user input signal unless explicitly stated in the context.

[0623] The terms used to describe the embodiments are used to describe specific embodiments and are not limiting to the embodiments. Unless otherwise explicitly stated in context, as used in the description of the embodiments and the claims, singular includes plural. “and / or” expressions are used to include all possible combinations between terms. “Includes” describes the presence of features, numbers, stages, elements, and / or components, and does not imply the absence of further features, numbers, stages, elements, and / or components. Conditional expressions such as “if ~” and “when ~” used to describe the embodiments are not interpreted as restrictive only when they are optional. They are intended to perform relevant actions or interpret relevant definitions in accordance with specific conditions when certain conditions are met. Furthermore, the actions of the embodiments described in this specification are performed by a transceiver including memory and / or a processor. The memory stores programs for processing / controlling the actions of the embodiments, and the processor controls the various actions described in this specification. The processor is also referred to as a controller. In the embodiments, actions are performed by firmware, software, and / or a combination thereof, which are stored in the processor or in memory. [Modes for carrying out the invention]

[0624] This document specifically describes the best mode for carrying out the invention. [Industrial applicability]

[0625] It will be apparent to those skilled in the art that these embodiments can be modified in various ways without deviating from the concept or scope of these embodiments. Accordingly, these embodiments include modifications and variations of embodiments provided within the scope of the appended claims and equivalent scope.

Claims

1. A point cloud data encoding method, The step of encoding the geometric data of the aforementioned point cloud data; A step of encoding characteristic data of the point cloud data based on the geometry data; and The step of transmitting the encoded geometry data, the encoded characteristic data, and the signaling data; The aforementioned geometric data is included in multiple slices, The signaling data includes slice-related information and buffer management-related information. The aforementioned buffer management-related information is, At a minimum, information indicating whether the current slice's context is referenced by at least one other slice, or, A point cloud data encoding method, including information for specifying the number of at least one other slices that reference the current slice.

2. The point cloud data encoding method according to claim 1, wherein the context of any of the plurality of slices is referenced by at least one other slice.

3. A point cloud data encoding device, A geometry encoder configured to encode the geometric data of the aforementioned point cloud data; A characteristic encoder configured to encode characteristic data of the point cloud data based on the geometry data; and A transmitting unit configured to transmit the encoded geometry data, the encoded characteristic data, and the signaling data; The aforementioned geometric data is included in multiple slices, The signaling data includes slice-related information and buffer management-related information. The aforementioned buffer management-related information is, At a minimum, information indicating whether the current slice's context is referenced by at least one other slice, or, A point cloud data encoding device, including information for specifying the number of at least one other slices that reference the current slice.

4. The point cloud data encoding device according to claim 3, wherein the context of any of the plurality of slices is referenced by at least one other slice.

5. A method for decrypting point cloud data, The stage of receiving geometry data, characteristic data, and signaling data; A step of decoding the geometry data based on the signaling data; and The step of decoding the characteristic data based on the signaling data and the decoded geometry data; The aforementioned geometric data is included in multiple slices, The signaling data includes slice-related information and buffer management-related information. The aforementioned buffer management-related information is, At a minimum, information indicating whether the current slice's context is referenced by at least one other slice, or, A point cloud data decoding method, including information for specifying the number of at least one other slices that reference the current slice.

6. The point cloud data decoding method according to claim 5, wherein the context of any of the plurality of slices is referenced by at least one other slice.

7. The step of decoding the aforementioned geometry data is: A point cloud data decoding method according to claim 5, comprising the step of controlling buffer storage of the context of the current slice based on the slice-related information and the buffer management-related information.

8. The step of decoding the aforementioned geometry data is: A point cloud data decoding method according to claim 7, wherein, based on the slice-related information and buffer management-related information, when it is confirmed that the context of the current slice is not referenced by at least one other slice, the context of the current slice is removed from the buffer.

9. The step of decoding the aforementioned geometry data is: A point cloud data decoding method according to claim 7, wherein, based on the slice-related information and buffer management-related information, it is confirmed that the context of the current slice is referenced by at least one other slice, and after the context of the current slice has been referenced by at least one other slice, the context of the current slice is removed from the buffer.

10. A point cloud data decoding device, A receiver configured to receive geometry data, characteristic data, and signaling data; A geometry decoder configured to decode the geometry data based on the signaling data; and A feature decoder configured to decode the feature data based on the signaling data and the decoded geometry data; The aforementioned geometric data is included in multiple slices, The signaling data includes slice-related information and buffer management-related information. The aforementioned buffer management-related information is, At a minimum, information indicating whether the current slice's context is referenced by at least one other slice, or, A point cloud data decoding device, including information for specifying the number of at least one other slices that reference the current slice.

11. The point cloud data decoding device according to claim 10, wherein the context of one of the plurality of slices is referenced by at least one other slice.

12. The point cloud data decoding device according to claim 10, wherein the geometry decoder controls buffer storage of the context of the current slice based on the slice-related information and the buffer management-related information.

13. The point cloud data decoding device according to claim 12, wherein, based on the slice-related information and the buffer management-related information, when it is determined that the context of the current slice is not referenced by at least one other slice, the geometry decoder removes the context of the current slice from the buffer.

14. The point cloud data decoding device according to claim 12, wherein, based on the slice-related information and the buffer management-related information, it is determined that the context of the current slice is referenced by at least one other slice, and after the context of the current slice has been referenced by at least one other slice, the geometry decoder removes the context of the current slice from the buffer.

Images