Point cloud data transmitting device, point cloud data transmitting method, point cloud data receiving device, and point cloud data receiving method

Abstract

To provide a point cloud data transmission and reception method for efficient transmission and reception of point cloud data.SOLUTION: A point cloud data transmission method includes encoding point cloud data, encapsulating the point cloud data, and transmitting the point cloud data. A point cloud data reception method includes receiving the point cloud data, decapsulating the point cloud data, and decoding the point cloud data.SELECTED DRAWING: Figure 74

Description

[Technical Field]

[0001] The example shows how to provide users with VR (Virtual Reality), AR (Augmented Reality), and Augmented Reality, Mixed Reality (MR) In order to provide various services such as real-world and autonomous driving services, point cloud (Point Cloud) Provides a method for providing content. [Background technology]

[0002] A point cloud is a collection of points in 3D space. The quantity of points in 3D space is However, there are many problems, such as difficulty in generating point cloud data.

[0003] The problem is that sending and receiving point cloud data requires a large amount of processing power. There is. Summary of the Invention [Problem to be solved by the invention]

[0004] The technical problem related to the embodiment is to solve the above problems by efficiently processing point clouds. Point cloud data transmission device, transmission method, and point cloud data The present invention aims to provide a data receiving device and a receiving method.

[0005] The technical problem associated with the embodiment is to reduce the latency and the complexity of encoding / decoding. Point cloud data transmission device, transmission method, and point cloud data reception method for solving the above problem The present invention aims to provide a receiving device and a receiving method.

[0006] However, the present invention is not limited to the above technical problems, and a person skilled in the art can solve the above problems based on the entire contents of this specification. The scope of the invention can be extended to other derived technical problems. [Means for solving the problem]

[0007] To achieve the above objects and other advantages, a point cloud data transmission method according to an embodiment is provided. The method includes encoding the point cloud data; encapsulating the point cloud data. encapsulating the point cloud data; and transmitting the point cloud data. To include (to comprise; to constitute; to construct; to set up; to encompass; to contain; to contain).

[0008] A point cloud data receiving device according to an embodiment receives point cloud data. Receiver: Decapsulating point cloud data an encapsulator; and a decoder for decoding the point cloud data. [Effects of the Invention]

[0009] Point cloud data transmission method, transmission device, and point cloud data reception device according to the embodiment The receiving method and receiving device can provide high-quality point cloud services.

[0010] Point cloud data transmission method, transmission device, and point cloud data reception device according to the embodiment The receiving method and receiving device can achieve various video codec formats.

[0011] Point cloud data transmission method, transmission device, and point cloud data reception device according to the embodiment The receiving method and receiving device provide general-purpose point cloud content such as for autonomous driving services. It can be provided. [Brief explanation of the drawings]

[0012] The drawings are attached for a better understanding of the embodiments and together with the description of the embodiments, illustrate the embodiments. vinegar.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47

Figure 48

Figure 49

Figure 50

Figure 51

Figure 52

Figure 53

Figure 54

Figure 55

Figure 56

Figure 57

Figure 58

Figure 59

Figure 60

Figure 61

Figure 62

Figure 63

Figure 64

Figure 65

Figure 66

Figure 67

Figure 68

Figure 69

Figure 70

Figure 71

Figure 72

Figure 73

Figure 74

Figure 75

[0013] Preferred embodiments will now be described in detail with reference to the accompanying drawings. The following detailed description, as referenced, illustrates only possible embodiments that may be implemented by the embodiments. The following detailed description is provided to illustrate the preferred embodiment. However, the embodiments may be practiced without such details. It will be apparent to those skilled in the art that this can be done.

[0014] Most of the terms used in the examples are common terms widely used in the relevant fields. However, some of them have been arbitrarily selected by the applicant, and their meanings are explained below as necessary. Therefore, the embodiment is not a simple name or meaning of a term, but an intended meaning of a term. It should be understood based on meaning.

[0015] FIG. 1 illustrates a transmission / reception system for providing point cloud content according to an embodiment. An example of the system structure is shown below.

[0016] In this document, we will introduce VR (Virtual Reality), AR (Augmented Reality) and Augmented Reality, Augmented Reality, MR (Mixed Reality, In order to provide various services such as point-classification, mixed reality, and autonomous driving services, The point cloud content according to the embodiment is , which represents data representing objects by points, point cloud, point cloud Also called point cloud data, point cloud video data, point cloud image data, etc. can be.

[0017] Point cloud data transmission device according to an embodiment ce)10000 is the Point Cloud Video Acquisition Unit (Point Cloud Vid eo Acquisition unit) 10001, point cloud video encoding Point Cloud Video Encoder 10002, File / Segment Encapsulation Unit 10003 and / or Transmitter (or The transmitter according to the embodiment includes a communication module 10004. The device acquires and processes the point cloud video (or point cloud content), In some embodiments, the transmitting device may be a fixed station. on), BTS (base transceiver system), network, AI (Artificial Intelligence) equipment and / or systems, bots, AR / VR / XR devices and / or servers. The communication device 10000 is a wireless communication technology (for example, 5G NR (New RAT), LTE ( Long Term Evolution)) to base stations and / or other wireless devices. devices that communicate with robots, vehicles, AR / VR / XR devices, mobile devices, home appliances, IoT ( Includes Internet of Things (IoT) devices, AI devices / servers, etc.

[0018] Point Cloud Video Acquisition Unit (Point Cloud Video Acquisition Unit) Acquisition unit) 10001 is a point cloud video capture The point cloud video is obtained by a camera, synthesis, or generation process.

[0019] Point Cloud Video Encoder according to an embodiment o Encoder 10002 encodes the point cloud video data. As usual, the Point Cloud Video Encoder 10002 is a point cloud encoder. It is called a point cloud data encoder, a point cloud data encoder, an encoder, etc. The point cloud compression coding (encoding) is not limited to the above-mentioned embodiment. The point cloud video encoder generates a bitstream containing encoded point cloud video data. The bitstream is the encoded point cloud video data. It contains not only the data but also the signaling information related to the encoding of the point cloud video data. nothing.

[0020] The encoder according to the embodiment is G-PCC (Geometry-based Point Coding). Cloud Compression coding method and / or V-PCC (Video- Based Point Cloud Compression) encoding method The encoder also supports point cloud (point cloud data or point (all of which are referred to herein) and / or signaling data relating to the point cloud. The specific operation of the encoding according to the embodiment will be described later.

[0021] On the other hand, the term V-PCC used in this specification refers to video-based point cloud compression ( Video-based point Cloud Compression(V-PC C), and the term V-PCC stands for Visual Volumetric Video-Based Coding. Visual Volumetric Video-based Coding(V 3C)) and can be referred to as complementary terms.

[0022] File / Segment Encapsulation Unit (File / Segment Encapsulation Unit) according to the embodiment The point cloud data is then stored in a file. The point cloud data according to the embodiment is encapsulated in the form of a set of rules and / or segments. The transmission method / apparatus transmits point cloud data in the form of files and / or segments. It can be trusted.

[0023] Transmitter (or Communication Device) according to an embodiment module))10004 converts the encoded point cloud video data into a bitstream. Depending on the embodiment, the file or segment may be transmitted in the form of a network stream. to a receiving device via a digital storage medium (e.g., USB, SD, CD) , DVD, Blu-ray, HDD, SSD, etc.) Device (or receiver) and 4G, 5G, 6G, etc. networks The transmitter can communicate with the network system (e.g., 4G, 5G) via wired / wireless communication. It can perform the necessary data processing operations according to the communication network system (such as 6G, The transmitter can also encapsulate the data on demand. It is also possible to transmit encoded data.

[0024] Point cloud data reception device according to an embodiment 10005 is a receiver (Receiver) 10006, a file / segment decapsulator 10007, Point Cloud Video Decoder eo Decoder 10008 and / or Renderer 100 09. In some embodiments, the receiving device is a wireless access technology (e.g., 5G NR (New R AT), LTE (Long Term Evolution)) to base stations and / or Or devices that communicate with other wireless devices, robots, vehicles, AR / VR / XR devices, mobile devices , home appliances, IoT (Internet of Things) devices, AI devices / servers, etc. Includes:

[0025] The receiver 10006 according to the embodiment receives point cloud video data. In some embodiments, the receiver 10006 receives a bitstream containing feedback data. Feedback information is sent as point cloud data. Send to device 10000.

[0026] File / Segment Decapsulation ation module)10007 is a file containing point cloud data and The decapsulating unit according to the embodiment decapsulates / decapsulates the encapsulation segment according to the embodiment. The reverse process of the cellification process is carried out.

[0027] Point Cloud Video Decoder er) 10007 decodes the received point cloud video data. The decoder reverses the encoding process according to the embodiment.

[0028] Renderer 10007 renders the decoded point cloud video data. In some embodiments, the renderer 10007 renders the data acquired at the receiving end. The feedback information is sent to the point cloud video decoder 10006. The point cloud video data transmitted to the receiver provides feedback information. If so, the feedback information received by the point cloud transmitting device is The data is provided to the video encoder.

[0029] In the drawing, the arrows indicated by dotted lines indicate feedback information acquired by the receiving device 10005. This shows the transmission path of feedback information. is information to reflect the interaction of users consuming point cloud content. User information (e.g., head orientation information, viewport information, etc.) ) especially for services where point cloud content requires user interaction (e.g. For example, for an autonomous driving service, the feedback information is The data is transmitted to the sender (e.g., the sending device 10000) and / or the service provider. In some examples, feedback information is transmitted not only to the transmitting device 10000 but also to the receiving device 1000. 5 may or may not be provided.

[0030] The head orientation information according to the embodiment includes the position, direction, angle, and movement of the user's head. The receiving device 10005 according to the embodiment is information about head orientation. The viewport information is calculated based on the viewport information. Information about the area of ​​the in-cloud video. is the point at which you are viewing the point cloud video, meaning the center of the viewport area. In other words, the viewport is an area centered on the viewpoint, and the size and shape of the area are determined by the FOV ( Therefore, the receiving device 10004 determines the field of view. In addition to door orientation information, the vertical or horizontal orientation that the device supports is also displayed. (horizontal) Viewport information can be extracted based on FOV etc. The receiving device 10005 also performs gaze analysis and the like. ,User's point cloud consumption method, point cloud video area that user gazes at, According to an embodiment, the receiving device 10005 includes the results of gaze analysis. The feedback information according to the embodiment is transmitted to the transmitting device 10000. The information is obtained during the rendering and / or display process. The clock information is obtained by one or more sensors included in the receiving device 10005. Also according to an embodiment, the feedback information may be provided by the renderer 10009 or another external element. The dotted lines in Figure 1 indicate The process of transmitting feedback information secured by the renderer 10009 is shown. The loud content provision system generates point cloud data based on feedback information. Therefore, the point cloud video data decoder 1000 processes (encodes / decodes) The receiving device 100 can perform a decoding operation based on the feedback information. 05 transmits feedback information to the transmitting device. The data encoder 10002 performs encoding operations based on the feedback information. Therefore, the point cloud content providing system stores all point cloud data. Without processing (encoding / decoding), necessary data (e.g., user) is generated based on feedback information. The system efficiently processes the point cloud data (corresponding to the user's head position) and provides the user with the point cloud data. It is possible to provide content cloud content.

[0031] In the embodiment, the transmitting device 10000 may be referred to as an encoder, a transmitting device, a transmitter, etc. The receiving device 10004 is called a decoder, a receiving device, a receiver, etc.

[0032] The point cloud content providing system of FIG. 1 according to the embodiment is processed (acquired / signed) Point cloud data (processed through a series of processes: encoding, transmitting, decoding, and rendering) The data is also called point cloud content data or point cloud video data. According to an embodiment, the point cloud content data relates to point cloud data. It can be used as a concept that includes associated metadata or signaling information.

[0033] The elements of the point cloud content provision system shown in Figure 1 are hardware, software, and The present invention may be embodied in software, a processor, and / or a combination thereof.

[0034] The example shows how to provide users with VR (Virtual Reality), AR (Augmented Reality), and Augmented Reality, Mixed Reality (MR) In order to provide various services such as point clouds (real-world) and autonomous driving services, Provide content.

[0035] To provide point cloud content services, we first The obtained point cloud video is sent through a series of processes to the receiving side. The data received is then processed back into the original point cloud video and rendered. This allows the user to view a point cloud video. It provides the necessary measures to carry out the process effectively.

[0036] The whole process of providing point cloud content services (point cloud The method for transmitting point cloud data and / or the method for receiving point cloud data are Encoding process, transmitting process, decoding process, rendering process and / or feed Includes back-processing.

[0037] According to an embodiment, providing point cloud content (or point cloud data) The process is called Point Cloud Compression. According to an embodiment, the point cloud compression process is Geometry-based Point Cloud Compression d Compression) process.

[0038] Point cloud data transmitting device and point cloud data receiving device according to the embodiment Each element may be hardware, software, a processor, and / or a combination thereof. means.

[0039] To provide point cloud content services, we first The obtained point cloud video is sent through a series of processes to the receiving side. The data received is then processed back into the original point cloud video and rendered. The present invention can provide a point cloud video to the user by using a series of these It provides the necessary measures to carry out the process effectively.

[0040] All processes for providing point cloud content services are acquisition processes. process, encoding process, transmission process, decoding process, rendering process and / or Includes feedback process.

[0041] The point cloud compression system includes a transmitting device and a receiving device. The program encodes the point cloud video and outputs a bitstream that can be saved to a file or is transmitted to a digital storage medium or network in the form of streaming (streaming segments) The digital storage media can be USB, SD, CD, DV This includes various storage media such as DVD, Blu-ray, HDD, SSD, etc.

[0042] The transmitting device is roughly composed of a point cloud video acquisition unit, a point cloud video encoder, and a The receiving device consists of a reader, a file / segment encapsulator, and a transmitter. , File / Segment Decapsulator, Point Cloud Video Decoder and Renderer The encoder is a point cloud video / image / picture / frame encoding device. Also called a device, the decoder is a point cloud video / image / picture / frame decoder. The transmitter may be included in a point cloud video encoder. The point cloud video decoder may include a renderer. The renderer and / or display may be a separate device or external component. The transmitting and receiving devices may be configured with It may further include other internal or external modules / units / components.

[0043] The operation of the receiving device according to the embodiment follows the reverse process of the operation of the transmitting device.

[0044] The point cloud video acquisition unit is responsible for capturing, synthesizing or The process of obtaining point cloud video is performed through a generation process. By specifying 3D position (x, y, z) / attributes (color, reflectivity, transparency) for multiple points, ) data, such as PLY (Polygon File format or th e Stanford Triangle format) files are generated. For videos with multiple frames, you can get more than one file. The point cloud is then processed using metadata associated with it (e.g., capture metadata about the

[0045] The point cloud data transmission device according to the embodiment encodes point cloud data. The point cloud data may also include an encoder for receiving the point cloud data, and a transmitter for transmitting the point cloud data. It may be transmitted in the form of a bitstream containing a content cloud.

[0046] A point cloud data receiving device according to an embodiment receives point cloud data. a receiving unit, a decoder for decoding the point cloud data, and a receiver for receiving the point cloud data. Contains the rendering renderer.

[0047] The method / apparatus according to the embodiment may include a point cloud data transmitting device and / or a point cloud data transmitting device. 1 shows a wood data receiving device.

[0048] FIG. 2 illustrates an example of capturing point cloud data according to an embodiment.

[0049] The point cloud data according to the embodiment can be obtained by a camera or the like. The approach can be, for example, inward-facing and / or outward-facing. There is an outward-facing method.

[0050] In the inward method according to the embodiment, one or more objects of point cloud data are is a capture method in which one or more cameras capture images from the outside to the inside of an object.

[0051] In the outward approach according to the embodiment, the object of the point cloud data is captured by one or more cameras. For example, in this embodiment, the camera captures the image from the inside to the outside of the object. There are four.

[0052] Point cloud data or point cloud content according to embodiments may be in various formats. It is a video or still image of an object / environment represented in a 3D space. Intcloud content is video / audio / images for objects. Includes images, etc.

[0053] Depth-capturing camera for point cloud content capture infrared pattern projector and infrared camera) and depth information It is composed of a combination of RGB cameras that can extract color information corresponding to the laser beam. A radar system that measures the time it takes for a reflector to return and determine the position coordinates of the reflector. Depth information can be extracted by LiDAR using a 3D camera. By extracting the geometry form composed of points in 3D space, RG From the B information, attributes that represent the color / reflection of each point can be extracted. The point cloud content includes the position (x, y, z) and color (YCbCr or The point cloud content is composed of RGB (red, green, blue) or reflectance (r) information. There are two methods: outward-facing, which captures the central object, and In VR / AR environments, The object (e.g., a character, a player, an object, an actor, or other core object) can be viewed in 360°. If you want to create point cloud content that can be freely viewed by users, The camera is configured inward. When configuring as a cloud content, the capture camera is configured as an outward facing camera. Since point cloud content is captured by multiple cameras, , the global coordinate system between the cameras A camera calibration process is required before capturing content to set the This can sometimes happen.

[0054] Point cloud content is a video of an object / environment shown in various forms in 3D space. Or it is still footage.

[0055] Other methods for obtaining point cloud content include: Any point cloud video can be synthesized based on the loud video. When trying to provide point cloud video for a virtual space generated by a virtual camera, Sometimes the camera does not capture the image at the time of the event. In this case, the relevant data is simply The generated process can replace the corresponding capture process.

[0056] The captured point cloud video can be later processed to improve the quality of the content. The video capture process requires the maximum possible processing within the capabilities of the camera equipment. The maximum / minimum depth values ​​can be adjusted, but even then, point data in the undesired area may still be included. This can be useful to remove unwanted areas (e.g., background) or to recognize connected spaces. Post-processing may be performed to fill spatial holes. The point cloud extracted from the cameras sharing the system was obtained through a calibration process. Based on the position coordinates of each camera, a conversion process is performed for each point to the global coordinate system. This allows for the integration of content into one broad range of point classes. It can also generate cloud content or generate point cloud content with high point density. You can also get the content.

[0057] The point cloud video encoder converts one or more input point cloud videos into A point cloud video consists of multiple frames. It may contain one frame, where one frame corresponds to a still image / picture. Point cloud video is a point cloud image / frame / picture / video / o audio / images, and point cloud video includes point cloud footage / framework. The point cloud video encoder can be used with video-based points. The point cloud video encoder performs point cloud compression (V-PCC) For the efficiency of compression and coding, prediction, transformation, quantization, entropy coding, etc. The coded data (coded video / image information) is divided into bits In the case of V-PCC procedure, the output is a point cloud video stream. The encoder converts the point cloud video into a geometry video, as described below, tribute video, occupancy map video, and additional information The geometry video is coded as The attribute video includes the attribute image; The occupancy map video contains an occupancy map image. Contains auxiliary patch information. Video / image includes texture video / image.

[0058] Encapsulation processing unit (file / segment encapsulation mod ule) 10003 is encoded point cloud video data and / or point Encapsulate cloud video related metadata in a file or other format. The cloud video related metadata is transmitted from the metadata processing unit. The processing unit may be included in the point cloud video encoder or may be a separate component. The encapsulation part may be configured as a packet / module. It may be encapsulated in a file format, such as a DASH segment. According to an embodiment, the encapsulation processor may process the point cloud video related Metadata may be included in the file format. The data can be stored in various levels of boxes in the ISOBMFF file format, for example. or contained in a separate track within the file. The processing unit encapsulates the point cloud video-related metadata in a file. The transmission processing unit receives the point cloud video encapsulated in a file format. The data may be processed for transmission. The transmission processing unit may be included in the transmission unit. The transmission processing unit may be configured as a separate component / module. Process point cloud video data according to the protocol. This includes processing for transmission via a network and processing for transmission via broadband. The transmission processing unit according to the embodiment not only processes point cloud video data but also metadata. The point cloud video related metadata is transmitted from the can.

[0059] The transmitter 10004 receives the coded video / image data in the form of a bitstream. Transferring information or data in file or streaming form to digital storage media or networks The digital storage media includes USB, SD, CD, , DVD, Blu-ray, HDD, SSD, etc. The sending unit sends the specified file format. It includes elements for generating media files by means of a matte, and The receiver extracts the bitstream and transmits it to a decoder. Send.

[0060] The receiver 10003 receives the point cloud video transmitted by the point cloud video transmitting device according to the present invention. Depending on the channel being transmitted, the receiver receives the broadcasting network. The point cloud video data may be received via The point cloud video data may be received via a digital storage medium. It may also receive audio video data.

[0061] The receiving processor processes the received point cloud video data according to the transmission protocol. The receiving processing section may be included in the receiving section or may be a separate component / module. The receiving side may be configured to correspond to the processing for transmission on the sending side. The receiving processing unit performs the reverse process of the sending processing unit. The cloud video data is transmitted to the decapsulation processing unit, and the acquired point cloud video related The metadata is transmitted to the metadata processing unit. The associated metadata may be in the form of a signaling table.

[0062] Decapsulation processing unit (file / segment decapsulation mo dule) 10007 is a point cloud visualization file format transmitted from the receiving processor. The decapsulation processing unit decapsulates the file using ISOBMFF or similar. decapsulates the file and converts it to a point cloud video bitstream or point cloud Obtain video-related metadata (metadata bitstream). The loud video bitstream is transmitted to the point cloud video decoder and acquired Point cloud video related metadata (metadata bitstream) is processed as metadata. The point cloud video bitstream is transmitted to the processing unit. The metadata processing unit may include a point cloud video decoder. It may be included in the decapsulation module or may consist of separate components / modules. The point cloud video related metadata acquired by the digitization processor is stored in the file format The decapsulator may be in the form of a box or track in the The metadata necessary for decapsulation is transmitted from the metadata processing unit. Point cloud The video related metadata is transmitted to the point cloud video decoder. The point cloud is then used in the point cloud video decoding procedure or transmitted to a renderer for point cloud video. Used in the rendering procedure.

[0063] The point cloud video decoder receives a bitstream and converts it into a point cloud. The video / picture is decoded by performing operations corresponding to those of the video encoder. The point cloud video decoder converts the point cloud video into a geometry decoder, as described below. Revideo, attribute video, occupancy map video It is also decoded into auxiliary information. A geometry video contains geometry images, and an attribute video contains attribute images. The quality image is included, and the occupancy map video is included. The additional information is the auxiliary patch information. Attribute video / image includes texture video / image.

[0064] Using the decoded geometry image, occupancy map, and additional patch information, the 3D geometry is The image is then reconstructed and may then go through a smoothing process. By using the texture image to give color values, it is possible to create a color point cloud video / picture. The renderer uses the restored geometry, color point cloud, Render the video / picture. The rendered video / picture is The user can choose to use a VR / AR display or a general display. View all or part of the area rendered by rays, etc.

[0065] Feedback process can be obtained during the rendering / display process A process that transmits various feedback information to the transmitter or to the decoder at the receiver. A feedback process may be used to improve the consumption of the point cloud video. According to an embodiment, the feed Head Orientation in the Back Process ) information, viewport information that shows the area the user is currently viewing, etc. According to the embodiment, the user can experience the virtual reality (VR / AR / MR / autonomous driving) environment. The system can interact with the object, in which case information related to that interaction is passed to the feedback process. Depending on the embodiment, the fee may be transmitted to the sender or service provider in the process. The feedback process may not be performed.

[0066] Head orientation information is information about the position, angle, and movement of the user's head. Based on this information, the information of the area the user is currently looking at in the point cloud video is Information, i.e., viewport information, is calculated.

[0067] The viewport information is the area the user is currently looking at in the point cloud video. This allows for gaze analysis, which determines what the user is thinking. How to consume the point cloud video, and which regions of the point cloud video to Gaze analysis is performed on the receiving side and fed back to the sending side. The device, such as a VR / AR / MR display, transmits the The user's head position / orientation, vertical or horizontal, supported by the device. Extract the viewport region based on the local FOV etc.

[0068] According to the embodiment, the above-mentioned feedback information is not only transmitted to the sending side, but also to the receiving side. That is, the above feedback information may be used to decode and recode the received signal. For example, head orientation information and / or uses viewport information to generate a point cloud video for the area the user is currently looking at. Only the audio is given priority for decoding and rendering.

[0069] Here, the viewport or viewport area is the area where the user points. The viewpoint is the area the user is looking at in the in-cloud video. The point you are looking at in the in-cloud video, which means the center of the viewport area. That is, the viewport is an area centered on the viewpoint, and the size, shape, etc. of that area is determined by the FOV (Field Of View).

[0070] This specification, as mentioned above, relates to point cloud video compression. The methods / embodiments disclosed herein are based on MPEG (Moving Picture Experts Group) s Group) PCC(point Cloud compression or Point Cloud coding) Standard or next generation video / image coding Applicable to standards.

[0071] In this specification, a picture / frame generally refers to one image at a particular time. It means units.

[0072] Pixels or pels make up a picture (or image). The term "sample" corresponds to a pixel. A sample generally refers to a pixel or pixel value, and brightness (l It may be possible to show only the pixel / pixel value of the saturation (ma) component, and the pixel value of the chroma (chroma) component. It may show only pixel / pixel values, or pixel / pixel values ​​of the depth component. Alternatively, only the xcel value may be shown.

[0073] A unit is a basic unit of image processing. A unit is a specific area of ​​a picture. The unit may include at least one of: a region and information about that region. It is often used interchangeably with terms such as block, area, or module. In the general case, an MxN block is a sample (or sample) consisting of M columns and N rows. array) or a set of transform coefficients (or array) Ray).

[0074] FIG. 3 shows an example of a point cloud, geometry, and texture image according to an embodiment. show.

[0075] The point cloud according to the embodiment is input to the V-PCC encoding process shown in FIG. 4, which will be described later. Then, a geometry image and a texture image are generated. Cloud is used interchangeably with point cloud data.

[0076] In Figure 3, the left image is a point cloud, and the point cloud is displayed in 3D space. This shows the point cloud where the object is located and represents it using a bounding box or similar. The middle image in Figure 3 shows the geometry image, and the right image shows the texture image (non- Pad).

[0077] Video-based Point Cloud Compression d Compression, V-PCC) is HEVC (Efficiency Vi video coding), VVC(Versatile Video Coding) 3D point cloud based on any 2D video codec The V-PCC compression process involves compressing the following data and and information is generated.

[0078] Occupancy map: Divides the points in a point cloud into patches. When mapping data to a 2D plane, the value is returned as 0 or 1 to indicate whether data exists at the corresponding position on the 2D plane. This shows a binary map where the value of 1 indicates The occupancy map) shows a 2D array corresponding to the atlas, and the values ​​of the occupancy map are Indicates whether each sample position in corresponds to a 3D point. denotes the object that contains information about the 2D patches for each point cloud frame. For example, the atlas may contain the 2D placement and size of patches, the corresponding 3D area within a 3D point, position, projection plan, LOD (Level of Detail) parameters Examples include ta.

[0079] An atlas is a representation of the 3D space in which volumetric data is rendered. The 2D bounding box located in the rectangular frame corresponding to the 3D bounding box of It is a collection of documents and related information.

[0080] Atlas bitstream is a One or more atlas frames and associated data that make up the atlas This is the bitstream for

[0081] Atlas frames are programs that consist of multiple patches. A 2D rectangular array of injected atlas samples. do.

[0082] Atlas sample is a collection of data related to atlas. The position of the rectangular frame onto which the patch is projected. tion).

[0083] The atlas frame is divided into tiles. A tile is a unit that divides a 2D frame. That is, a tile is a point cloud called an atlas. It is a unit for dividing the signaling information of loud data.

[0084] Patch: A set of points that make up a point cloud and belong to the same patch. The points are adjacent to each other in 3D space and are mapped onto six planes in the mapping process to the 2D image. indicates that the bounding box planes of the two images are mapped in the same direction.

[0085] A patch is a unit that divides a tile. A patch is a unit that divides a tile. This is signaling information.

[0086] The receiving device according to the embodiment can receive the same presentation data based on the atlas (tiles, patches). Attribute video data, which is actual video data with motion time, geometry video data , the exclusive video data can be restored.

[0087] Geometry image: The position information (geometry) of each point in the point cloud This shows an image in the form of a depth map expressed in units of patches. A geometry image is a single patch. The geometry is made up of pixel values ​​from the point cloud frame. indicates a set of

[0088] Texture image: each point in the point cloud The texture image is a multi-channel image that expresses the color information of the image in units of patches. It consists of pixel values ​​for each color (e.g., three channels R, G, B). According to an embodiment, the textures and / or characteristics are of the same object and / or containment relationship. It is interpreted as a relation.

[0089] Auxiliary patch info: Points from individual patches The additional patch information indicates the metadata required to reconstruct the patch cloud. Contains information about position, size, etc. in 2D / 3D space.

[0090] According to an embodiment, point cloud data, e.g., a V-PCC component, This includes the data, occupancy map, geometry, characteristics, etc.

[0091] An atlas represents a collection of 2D bounding boxes. For example, a patch projected onto a rectangular frame. It also supports 3D bounding boxes in 3D space, allowing subsetting of point clouds. Indicates a cot.

[0092] An attribute is a scalar associated with each point in the point cloud. It indicates a scalar or vector, e.g., color ), reflectance, surface normal, time stamps, material ID etc.

[0093] The point cloud data in this embodiment is V-PCC (Video-based Point Cloud Computing). This shows PCC data using the Intel Cloud Compression (PCC) method. Cloud data contains multiple components, such as occupancy maps, patches, and geometry. Includes birds and / or textures.

[0094] FIG. 4 illustrates an example of a V-PCC encoding process according to an embodiment.

[0095] Figure 4 shows the occupancy map, geometry image, and texture image, texture image, additional Generate and compress auxiliary patch information 1 shows the V-PCC encoding process for compression. The V-PCC encoding process of FIG. 4 is the same as that of the point cloud video encoder 1000 of FIG. 2. Each component in Figure 4 is a software, hardware, and processor. and / or a combination thereof.

[0096] The patch generation unit 14000 or patch generator Point cloud frame (a bitstream format containing point cloud data) The patch generation unit 40000 receives the patch from the point cloud data. Also, patch information including information regarding the generation of the patch is generated.

[0097] Patch packing 40001 or patch packer is a patch packing Patch the in-cloud data. For example, one or more patches It also generates an occupancy map that contains information about patch packing.

[0098] Geometry image generation 40002 or Geometry Image Generator, point cloud data, patches, and / or Or generate a geometry image based on the packed patches. Geometry Image refers to data including geometry related to point cloud data.

[0099] Texture image generation 40003 or texture image generator, point cloud data, patches, and and / or generate texture images based on packed patches. The reconstructed geometry image is smoothed based on the patch information ( (number) is smoothed and the resulting smoothed geometry is further based on the texture Generate the image.

[0100] Smoothing 40004 or smoothing part is included in the image data. Mitigate or eliminate errors, e.g., reconstructed geodata Based on the patch information of the metrology image, the part that induces the data error is softened. Generate smoothed geometry by filtering the

[0101] auxiliary patch info compress ion) 40005 or the additional patch information compression unit compresses the patch generated by the patch generation process. It also compresses auxiliary patch information related to patch information. The additional patch information is transmitted to the multiplexer, and the geometry image generator 40002 also generates the additional patch. Use the information.

[0102] Image padding 40006, 40007 or image The padding section pads the geometry image and the texture image respectively. , pad data is padded into the geometry image and texture image.

[0103] Group dilation 40008 or group dilation part is Similar to the image pad, it adds data to the texture image. Inserted into the kisser image.

[0104] Video compression: 40009, 40010, 400 11 or video compression part, padded geometry image, padded texture Compresses the image and / or occupancy map, respectively. Compression is performed on geometry information, textures, It encodes information such as user information and proprietary information.

[0105] Entropy compression 40012 or Entropy The entropy compressor compresses (e.g., encodes) the occupancy map based on an entropy scheme. .

[0106] According to an embodiment, when point cloud data is lossless and and / or lossy, entropy pressure on the occupied map frame compression and / or video compression is performed.

[0107] The multiplexer 40013 multiplexes the compressed geometry image ,Compressed texture image,Compressed occupancy map are multiplexed into the bitstream do.

[0108] The detailed operation of each process in FIG. 4 according to the embodiment will be described below.

[0109] Patch generation: 40000

[0110] The process of patch generation involves mapping the point cloud to a 2D image. In order to achieve this, the process of dividing the point cloud into patches, which are the units for mapping, is The process of patch generation is as follows: The three steps are: computation, segmentation, and patch division. It is divided into groups.

[0111] The process of calculating the normalized value will be specifically described with reference to FIG.

[0112] FIG. 5 shows a tangent plane of a surface according to an embodiment. ) and normal vector.

[0113] The surface of Figure 5 is generated by the patch generation process 40000 of the V-PCC encoding process in Figure 4. In this context, it is used as follows:

[0114] Normal calculation related to patch generation

[0115] Each point (e.g., point) that makes up the point cloud has a unique direction. This is expressed as a 3D vector called a normal. Using the neighbors of a point, we create a surface of the point cloud as shown in Figure 5. The tangent plane and normal vector of each point The search range in the process of finding neighboring points is calculated. ge) is defined by the user.

[0116] Tangen plane: A line that passes through a point on a surface and is tangent to a curve on the surface. shows the plane that completely contains

[0117] FIG. 6 shows a bounding box of a point cloud according to an embodiment. Here is an example of a box.

[0118] A method / apparatus according to an embodiment, for example, a patch generator for generating patches from point cloud data A bounding box is used in the process of generating the

[0119] In this embodiment, the bounding box is a rectangle that represents point cloud data in 3D space. This is a unit box that is divided based on a facet.

[0120] The bounding box is the area of ​​the target point cloud data. In the process of projecting the body onto the plane of each hexahedron based on the hexahedron in 3D space, The bounding box is used in the point cloud video acquisition unit 1000 in FIG. 0, generated and processed by the point cloud video encoder 10002. Based on the bounding box, the patch generation 400 of the V-PCC encoding process in FIG. 00, Patch Packing 40001, Geometry Image Generation 40002, Texture Image generation 40003 is performed.

[0121] Segmentation in relation to patch generation

[0122] Segmentation is performed by the initial segmentation. The two processes are: segmentation and refinement. It consists of:

[0123] The point cloud video encoder 10002 according to the embodiment converts points into bounding Specifically, each point in the point cloud is projected onto one side of the drawing box. 6, the projected point is placed on one of the faces of the six bounding boxes surrounding the point cloud. The initial segmentation is performed by The process of determining one of the planes of the projected bounding box. . [Table 1]

[0124] The determined plane is expressed as one of the index values ​​(cluster) between 0 and 5. er index).

[0125] Refine segmentation is the initial segmentation described above. The point cloud is determined by the process of spatial segmentation. The process of improving the projection plane of each point by considering the projection planes of neighboring points. In this process, the projection plane is The normal of each point and the normal of each plane of the bounding box considered to determine The projection plane of the current point, along with the score normal that forms the similarity to the value Score smoothing indicates the degree of agreement between adjacent points and the projection plane are considered simultaneously.

[0126] Score smooth is a function that gives weights to score normal. The weights are user-defined. The repetition count is user-defined.

[0127] Segment patches in relation to patch generation

[0128] Patch division is the process of dividing each point in the point cloud obtained by the initial / refinement division process described above. Based on the projection plane information, the entire point cloud is divided into a set of adjacent points. This is the process of dividing a file into patches. Patch division consists of the following steps:

[0129] 1) Calculate the neighboring points of each point in the point cloud using a KD tree or similar method. The maximum number of neighbors is user defined.

[0130] 2) When the adjacent point is projected on the same plane as the current point (same cluster in). If the current point has a cluster index value, the current point and its neighbors are multiplied by 1. Extract into one patch.

[0131] 3) Calculate the geometry value of the extracted patch. The detailed process will be described later.

[0132] 4) Repeat steps 2) to 4) until there are no more points left unextracted.

[0133] Through the process of patch division, the size of each patch and the occupancy map of each patch are calculated. The triimage, texture image, etc. are determined.

[0134] FIG. 7 shows an individual patch of an occupancy map according to an embodiment. An example of positioning of the tch) is shown below.

[0135] The point cloud encoder 10002 according to the embodiment performs patch packing and occupancy mapping. You can generate a map.

[0136] Patch packing & Occupancy map generation pancy map generation)40001

[0137] This process involves dividing the previously segmented patches into individual patches to map them into a single 2D image. The process of determining the location of different patches in a 2D image. A 2D image is a map of the location of a data point. The occupancy map is a binary map that indicates the value of 0 or 1. It consists of blocks, and the resolution is determined according to the size of the blocks. If the block size is 1*1, it has a resolution in pixels. The occupancy packing block size is up to the user. Therefore, it is determined.

[0138] The process of determining the location of an individual patch in the occupancy map is as follows.

[0139] 1) Set all values ​​in the global occupancy map to 0.

[0140] 2) The horizontal coordinates that lie on the occupancy map plane are (0, occupancySizeU-p occupancySizeV-patc h.sizeV0) to position the patch at a point (u, v) in the range

[0141] 3) The horizontal coordinate of the patch plane is (0, patch.sizeU0), and the vertical coordinate is The point (x, y) in the range of (0, patch.sizeV0) is the current point. and set it.

[0142] 4) For point (x, y), the (x, y) coordinate value in the patch occupancy map is 1 (pat The data exists at the corresponding point in the area), and the (u+x, v+y) coordinate value of the entire occupancy map is 1 (If the occupancy map is filled by a previous patch), change the (x,y) position in raster order If so, repeat steps 3) and 4). If not, perform step 6).

[0143] 5) Change the (u, v) position in the raster order and repeat steps 3) to 5).

[0144] 6) (u, v) is determined as the position of the corresponding patch, and the occupancy map data of the patch is converted to the overall occupancy data. Assign (copy) it to the corresponding part of the map.

[0145] 7) Repeat steps 2) to 7) for the next patch.

[0146] OccupancySizeU: indicates the width of the occupancy map. The unit is occupancy packing size block (occupancy packing block ock size).

[0147] OccupancySizeV: Height of the occupancy map and the unit is the occupied packing block size.

[0148] Patch size U0 (patch.sizeU0): The width of the occupancy map, in occupancy units. The packing block size.

[0149] Patch size V0 (patch.sizeV0): indicates the height of the occupancy map, and is measured in occupancy units. Yes, there is a packing block size.

[0150] For example, as shown in Figure 7, the patch size in the box corresponding to the occupied packing size block is There exists a box corresponding to a patch with the size, and a point in the box (x, y) is at position You may do so.

[0151] FIG. 8 shows the normal, tangent, and An example of a bitangent axis relationship is shown below.

[0152] The point cloud video encoder 10002 according to the embodiment generates a geometry image A geometry image is a collection of geometric information from a point cloud. The process of generating a geometry image is the patch data in Figure 8. It uses three axes (normal, tangent, and bitangent).

[0153] Geometry image generation 40002

[0154] In this process, the depth values ​​that make up the geometry image of the individual patches are determined by the patch packing process described above. Generates an image of the entire geometry based on the positions of the selected patches. The process for determining the depth values ​​that make up the tri-image is as follows.

[0155] 1) Calculate the parameters related to the position and size of each patch. The parameters are as follows: It includes information such as:

[0156] An index indicating the normal axis: normal is the parameter used in the patch generation process described above. The tangent axis is the horizontal axis of the patch image ( The bitangent axis is the axis that coincides with the normal axis and is perpendicular to the patch image. The three axes are shown in the diagram below, with the axis coinciding with the vertical (v) axis of the page.

[0157] FIG. 9 shows an example of a minimum mode and a maximum mode configuration of the projection mode according to the embodiment. Shows.

[0158] The point cloud video encoder 10002 according to the embodiment converts the geometry image into To generate the image, we perform patch-based projection and then use the projection according to the example. The modes are minimum and maximum.

[0159] 3D spatial coordinates of a patch: calculated by the minimum size bounding box that encloses the patch. For example, the 3D space coordinate of the patch is given by the minimum value of the tangent direction of the patch (patc h 3D shift tangent axis), the bi-tangent direction of the patch patch 3D shift bitangent axis, patch node patch 3D shift normal axis, etc. can be.

[0160] 2D size of the patch: the horizontal and vertical size when the patch is packed in the 2D image. The horizontal size (patch 2D size u) is the size of the bounding box. The difference between the maximum and minimum values ​​of the tangent direction of the pixel, and the vertical size (patch 2 D size v) are the maximum and minimum values ​​of the bitangent direction of the bounding box This is the difference between

[0161] 2) Determine the projection mode of the patch. The projection mode is divided into minimum mode (min mode) and maximum mode (max mode). The geometry information of a patch is given by its depth value, When projecting each point of the patch in the normal direction of the patch, the maximum depth value is The image consists of two layers: an image composed of the minimum value and an image composed of the minimum value. A message is generated.

[0162] To generate two layer images d0 and d1, in the minimum mode, as shown in Figure 9, The minimum depth is set to d0, and the surface thickness is calculated from the minimum depth. The maximum depth within d1 is configured as d1.

[0163] For example, if a point cloud is located in 2D as shown in the figure, it contains multiple points. There may be multiple patches. As shown in the figure, points with the same shading represent the same patch. The process of projecting a patch of points indicated by blank spaces is shown. vinegar.

[0164] When projecting the point indicated by the blank to the left / right side, the depth is calculated based on the left side. The depth of the point to the right increases by one, 0, 1, 2, ..6, 7, 8, 9, etc. Write down the numbers for calculation.

[0165] The projection mode is user-defined. Therefore, the same method may be applied to all point clouds, frame by frame or patch by patch. Different methods may be applied to the projection model for each frame or patch. When a code is applied, it can improve compression efficiency or minimize the missing point. A projection mode that allows for miniaturization is adaptively selected.

[0166] 3) Calculate the depth values ​​of the individual points.

[0167] In minimum mode, the minimum value of the normal axis of each point is set to the minimum value of the normal direction of the patch (p 3D shift normal axis) calculated in process 1) patch 3D shift normal axis The d0 image is constructed at depth0, which is the value obtained by subtracting s. If there is any other depth value between 0 and the surface thickness, set this value to depth1. If it does not exist, assign the value of depth0 to depth1 as well. Depth Configure the d1 image with a value of 1.

[0168] For example, the minimum value is calculated for the depth determination of point d0 (4 2 4 4 0 6 0 0 9 9 0 8 0). Also, by determining the depth of the point d1, If the larger value of two or more points is calculated, or if there is only one point, The value is calculated as (4 4 4 4 6 6 6 8 9 9 8 8 9). In the process where the points of the patch are encoded and reconstructed, Some points will be lost (e.g., 8 points are lost in the figure).

[0169] In Max mode, the maximum value of the normal axis of each point is set to the nodal point of the patch. Normal direction minimum (patch 3D shift normal axis) to 1) The minimum normal direction value of the patch calculated in the process (patch 3D shift The d0 image is constructed with depth0, which is the value obtained by subtracting the normal axis. If there is another depth value within the range of depth0 and the surface thickness at one position, this value If it does not exist, the value of depth0 is also assigned to depth1. The Depth1 value is used to construct the d1 image.

[0170] For example, the maximum value is calculated for the depth determination of the point d0 (4 4 4 4 6 6 6 8 9 9 8 8 9). Also, by determining the depth of the point d1, If the smaller of two or more points is calculated, or if there is only one point, The value is calculated as (4 2 4 4 5 6 0 6 9 9 0 8 0). In the process where the points of the patch are encoded and reconstructed, Some points will be lost (for example, 6 points will be lost in the figure).

[0171] The geometry images of the individual patches generated from the above process are referred to as the patch The position information of the individual patches generated through the packing process is used to calculate the overall geometry. By placing it in Reimage, you can generate an image of the entire geometry.

[0172] The d1 layer of the generated overall geometry image is coded using various methods. The first method is to encode the depth value of the previously generated d1 image as it is (abso The second is the previously generated d How to encode the difference between the depth values ​​of the d1 image and the depth values ​​of the d0 image tial encoding method).

[0173] This coding method using the depth values ​​of two layers, d0 and d1, is If there is another point in between, there will be a failure in the process of encoding the geometry information of that point. For lossless coding, Enhanced- Delta-Depth (EDD) code may also be used.

[0174] The EDD code will be specifically described with reference to FIG.

[0175] FIG. 10 shows an example of an EDD code according to an embodiment.

[0176] Point Cloud Video Encoder 10002 and / or part / all of V-PCC encoding Some processes (e.g., video compression 40009) point to the EOD code. The geometry information of the object can be encoded.

[0177] As shown in the figure, the EDD code defines the positions of all points within the surface thickness range, including d1, as follows: For example, for the points in the second column from the left in the figure, There are points in the first and fourth positions above D0, and the second and third positions are empty. Therefore, it is represented by the EDD code 0b1001 (=9). When the data is encoded and transmitted, the geometry information of all points can be recovered without loss at the receiving end. can be done.

[0178] For example, if a point exists on the reference point, it is 1, and if the point does not exist, it is 0. The code is represented based on four bits.

[0179] Smoothing40004

[0180] Smoothing reduces discontinuities that may occur at pad boundaries due to image quality degradation resulting from the compression process. This is the process of removing the image quality, which is performed in the point cloud video encoder or the smoother.

[0181] 1) Reconstruct the point cloud from the geometry image. This process can be considered the reverse of the geometry image generation process described above. For example, ,The reverse process of encoding is regeneration.

[0182] 2) Each point that makes up the point cloud regenerated using a KD tree, etc. Calculate adjacent points.

[0183] 3) For each point, determine whether the point is on the patch boundary surface. , the neighbors with a different projection plane (cluster index) from the current point If a point exists, it can be determined that the point is located on the boundary surface of the patch.

[0184] 4) If a patch boundary exists, the point is taken as the center of gravity of the neighboring points (the average x, y, z of the neighboring points). Moves the object to a new location (coordinates), i.e. changes the geometry value. If it does not exist, the previous Preserve geometry values.

[0185] FIG. 11 shows a recoloring process using color values ​​of neighboring points according to an embodiment. Here is an example of a ring.

[0186] Point Cloud Video Encoder or Texture Image Generator 400 according to an embodiment 03 can generate texture images based on color reproduction.

[0187] Texture image generation 4 0003

[0188] The process of generating texture images is the same as the process of generating geometry images described above. Similarly, generate texture images for individual patches and place them at defined positions. The process involves generating the entire texture image by In the process of generating texture images for the swatch, depth is used for geometry generation. Instead of the value, the color value of the point that composes the point cloud corresponding to that position (for example, R , G, B) is generated.

[0189] In the process of calculating the color values ​​of each point that makes up the point cloud, The smoothed point cloud is the original geometry. Some points may have moved in the point cloud, so A color matching process is required to find the color that matches the selected position. As an example, the new color value is calculated by dividing the color value of the nearest neighbor by the color value of the nearest neighbor, as shown in the figure. It can be calculated taking into account the color values ​​of the contact points.

[0190] For example, referring to the figure, the restoration is the characteristic of the closest original point to the point. Based on the average of the information and / or the attribute information of the closest original location to the point Calculate the appropriate color value for the changed position.

[0191] The texture image is also a geometry image generated by two layers, d0 / d1. Like a page, it is generated in two layers, t0 / t1.

[0192] Auxiliary patch info compress ion)40005

[0193] The point cloud video encoder or additional patch information compressor according to the embodiment It is possible to compress the point cloud's point information (additional information about the point cloud).

[0194] The additional patch information compression unit performs the above-mentioned patch generation, patch packing, and geometry generation. The additional patch information generated by the process is compressed. The additional patch information includes the following parameters: Parameters included:

[0195] An index that identifies the projection plane (normal). cluster index

[0196] 3D spatial position of the patch: minimum value of the tangent direction of the patch (patch 3D shi ft tangent axis), patch bitangent direction minimum value (patch 3D shift bitangent axis), patch normal direction minimum ( patch 3D shift normal axis)

[0197] 2D spatial position and size of the patch: horizontal size (patch 2D size u) , vertical size (patch 2D size v), horizontal minimum value (patch 2D shift u), vertical minimum value (patch 2D shift u)

[0198] Mapping information for each block and patch: candidate index index) (The patch is positioned in order based on the 2D spatial position and size information of the patch mentioned above. If you do this, multiple patches may be mapped to one block. The patches to be mapped form a candidate list, and the data of the patch in this list is local patch index) (An index indicating one of the total patches present in the frame). Table 1 is a block using a candidate list and a local patch index. Here is pseudo code showing the lock and patch matching process: do.

[0199] The maximum number of candidate lists is user defined.

[0200] Table 1-1 Pseudo code for block and patch mapping

[0201] for(i=0;i <BlockCount;i++){

[0202] if(candidatePatches[i].size()==1){

[0203] blockToPatch[i]=candidatePatches[i][0]

[0204] } else {

[0205] candidate_index

[0206] if(candidate_index==max_candidate_count ) {

[0207] blockToPatch[i]=local_patch_index

[0208] } else {

[0209] blockToPatch[i]=candidatePatches[i][can didate_index]

[0210] }

[0211] }

[0212] }

[0213] FIG. 12 illustrates a push-pull background filling according to an embodiment. ll background filling).

[0214] Image padding and group expansion up dilation)40006, 40007, 40008

[0215] The image padder according to the embodiment is based on a push-pull background filling method. Based on this, the space outside the patch area can be filled with meaningless additional data.

[0216] Image padding is a method to improve compression efficiency by adding space outside the patch area. Image padding is the process of filling in the missing data. The pixel values ​​of the corresponding column or row are copied to fill the empty space. Or, as shown, you can gradually reduce the resolution of the unpadded image and then increase it again. In the process of creating a push button, pixel values ​​from a lower resolution image are used to fill empty spaces. Push-pull background filling A lling method may be used.

[0217] Group dilation is the two-phase dilation of d0 / d1 and t0 / t1. A method for filling empty spaces in a layered geometry / texture image. The free space values ​​of the two layers calculated by the image padding described above are used to calculate the free space of the two layers. This is a process of filling with the average value of the values ​​for the same position on the ear.

[0218] FIG. 13 shows the possible traversals for a 4*4 size block according to the embodiment. An example of traversal order is shown below.

[0219] Occupancy map compression 40012 , 40011

[0220] The occupancy map compression according to the embodiment compresses the occupancy map described above. Specifically, lossy ( Video compression for lossy and lossless compression Entropy compression for lossless compression There are two methods: compression and compression. Video compression will be discussed later.

[0221] The entropy compression process is as follows:

[0222] 1) For each block that makes up the occupancy map, if all blocks are filled, If so, code a 1 and repeat the process for the next block; otherwise, code a 0. Encode and then carry out steps 2) to 5).

[0223] 2) Run-length coding is performed on the filled pixels of the block. Determine the best traversal order for the th coding. The figure shows four possible traversal orders for a 4*4 size block. is shown as an example.

[0224] FIG. 14 shows an example of a best traversal order according to an embodiment.

[0225] As described above, the entropy compression unit according to the embodiment uses the traversal order The block is coded based on the scheme.

[0226] For example, among the possible traversal orders, the best one with the smallest number of runs is t Select a traversal order and encode its index. In this case, the third traversal order in FIG. 13 is selected. The number of runs can be minimized to 2, which is called the best traversal order. Select as (persal order).

[0227] At this time, the number of runs is coded. In the example of Figure 14, there are two runs. , 2 is encoded.

[0228] 4) Encode the occupancy of the first run. In the example of Figure 14, Since the first run corresponds to an unfilled pixel, 0 is coded.

[0229] 5) Encode the length of each run (as many as there are runs). In the example in Figure 14, the lengths of the first and second runs, 6 and 10, are coded sequentially. To become.

[0230] Video compression: 40009, 40010, 400 11

[0231] The video compression unit according to the embodiment uses a 2D video codec (vid) such as HEVC or VVC. eo codec) and the geometry image generated by the process described above, Encode a sequence of texture images, occupancy map images, etc.

[0232] FIG. 15 shows a 2D video / image encoder according to an embodiment. An example of a GE Encoder is shown below.

[0233] The figure shows the video compression unit 400 described above. 09, 40010, 40011 are applied to the embodiment, and the encoding of the video / image signal 15 shows a schematic block diagram of a 2D video / image encoder 15000 in which The video / image encoder 15000 is the point cloud video encoder described above. Each component in Figure 15 is included in 10002 or consists of internal / external components. The elements correspond to software, hardware, processors and / or combinations thereof. do.

[0234] Here, the input image is the above-mentioned geometry image, texture image (characteristics image), image), occupancy map image, etc. Output of Point Cloud Video Encoder The bitstream (i.e. point cloud video / image bitstream) is Input image (geometry image, texture image, occupancy map image) Contains the output bitstream for a video stream (such as a DVD image).

[0235] The inter prediction unit 15090 and the intra prediction unit 15100 are collectively referred to as a prediction unit. That is, the prediction unit includes an inter prediction unit 15090 and an intra prediction unit 15100. 15030, quantization unit 15040, inverse quantization unit 15050, and inverse transformation unit 15060. The residual processing unit further includes a subtraction unit 15020. The above-mentioned image division unit 15010, subtraction unit 15020, conversion unit 15030, and quantization unit 15040, inverse quantization unit 15050, inverse transform unit 15060, addition unit 155, filter a prediction unit 15070, an inter prediction unit 15090, an intra prediction unit 15100, and an entropy prediction unit 15091. The P encoder 15110 may be implemented as a single hardware component (e.g., For example, an encoder or a processor). The memory 15080 is configured with a DPB (d It consists of a digital storage medium, including a digitally encoded picture buffer. .

[0236] The video division unit 15010 divides the input video (or picture) input to the encoding device 15000. Dividing a frame into one or more processing units As an example, the processing unit may be a coding unit (C In this case, the coding unit is also called a coding tree unit (co coding tree unit (CTU) or largest coding unit (larges LCU) to QTBT (Quad-tree binar It is split recursively by a y-tree structure. For example, One coding unit, Quad-tree structure and / or binary-tree It is divided into multiple coding units of deeper depth based on the e structure. In this case, for example, Quad-tree is applied first, and then binary-t tree may be applied, or binary-tree may be applied first. The coding procedure according to the present invention is based on the final coding unit that is not further divided. In this case, the optimum coding efficiency may be determined based on the characteristics of the video. The larger coding unit may be used as the final coding unit, or may be used as needed. The coding unit then recursively expands to a lower depth coding unit. The optimal size coding unit is then used for the final coding. Here, the coding procedure includes prediction, conversion, and In another example, the processing unit may be a prediction unit (PU). Diction Unit (TU) or Transform Unit (TU) In this case, each of the prediction unit and the transform unit may further include The prediction unit is divided or partitioned from the final coding unit. A transform unit is a unit of sample prediction, and a transform unit is a unit from which transform coefficients are derived and / or from which transform coefficients are derived. It is a unit that derives a residual signal from the signal.

[0237] A unit may be a block or an area or a module, as the case may be. In general, an MxN block consists of M columns and N rows. indicates a set of samples or transform coefficients A sample generally represents a pixel or the value of a pixel, and is the value of a pixel in the luminance (luma) component. It may show only the pixel / pixel value of the chroma component. A sample represents one picture (or video) pixel. It is used as a term corresponding to l) or pel.

[0238] The encoding device 15000 receives an input video signal (original block, original sample ray) output from the inter prediction unit 15090 or the intra prediction unit 15100 The prediction signal (predicted block, prediction sample array) is subtracted to obtain the residual signal (residual signal). dual signal, residual block, residual sample array) and generate the residual The signal is transmitted to a transform unit 15030. In this case, as shown in the figure, the encoding device 15000 In the input video signal (original block, original sample array), the predicted signal The unit that subtracts (prediction block, prediction sample array) is called a subtraction unit 15020. The prediction unit predicts the block to be processed (hereinafter referred to as the current block), predicted block containing predicted samples for the block The prediction unit applies intra prediction to the current block or CU unit, or The prediction unit determines whether to apply center prediction. Then, various information related to prediction such as prediction mode information is generated and sent to the entropy coding unit 15. The prediction information is encoded by the entropy encoding unit 15110. It is output in the form of a bitstream.

[0239] The intra prediction unit 15100 of the prediction unit predicts the current block by referring to samples in the current picture. The referenced samples are the neighbors (n) of the current block depending on the prediction mode. In intra prediction, the prediction mode is either located next to the pixel or located far away. The modes include multiple non-directional modes and multiple directional modes. The non-directional modes are, for example, DC The directional mode includes precision of the predicted direction. Depending on the degree, it may include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is just an example, and depending on the setting, a higher or lower directional prediction mode may be used. The intra prediction unit 15100 uses the prediction mode applied to the neighboring block to perform the following: A prediction mode to be applied to the current block may be determined.

[0240] The inter-prediction unit 15090 of the prediction unit performs a motion vector detection on the reference picture. The predicted value for the current block is calculated based on the reference block (reference sample array) In this case, to reduce the amount of motion information transmitted in inter prediction mode, To achieve this, the motion information is calculated based on the correlation between the motion information of the neighboring blocks and the current block. The motion information is a motion vector and a reference picture. The motion information also includes inter prediction direction (L0 prediction, L1 prediction, B In the case of inter prediction, the neighboring blocks are in the current picture. The spatial neighboring blocks and the reference blocks Temporal neighboring blocks in the reference picture g block). It includes the reference picture containing the reference block and the temporally adjacent blocks. The reference pictures containing the blocks may be the same or different. collocated reference block, collocated The reference picture containing the temporally adjacent blocks is called a reference CU (colCU), and Also called collocated picture (colPic) For example, the inter-prediction unit 15090 generates a candidate list of motion information based on neighboring blocks. and deriving a motion vector and / or a reference picture index for the current block. Based on the various prediction modes, the system generates information indicating which candidate is used for the prediction. For example, in the case of skip mode and merge mode, inter prediction is performed. The unit uses the motion information of the neighboring block as the motion information of the current block. In this case, unlike the merge mode, the residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the neighboring blocks The motion vector of the block is calculated by the motion vector predictor. or) as the motion vector difference The motion vector of the current block is indicated by signaling the motion vector (inc).

[0241] A prediction signal generated by the inter prediction unit 15090 or the intra prediction unit 15100 is used to generate the reconstructed signal or to generate the residual signal.

[0242] The transform unit 15030 applies a transform method to the residual signal to obtain transform coefficients (transform c For example, the transform method is DCT (Discrete Coefficient Transform). Cosine Transform), DST(Discrete Sine Tra nsform), KLT (Karhunen-Loeve Transform), GB T (Graph-Based Transform) or CNT (Condition Ally Non-linear Transform). Here, GBT is a method for expressing the relationship between pixels as a graph. CNT means the transformation of all previously restored pixels. A prediction signal is generated using the sly reconstructed pixel. The transformation process is based on square pixels of the same size. It may be applied to blocks of cells, or to blocks of variable size that are not square. good.

[0243] The quantization unit 15040 quantizes the transform coefficients and transmits them to the entropy coding unit 15110. The entropy coding unit 15110 then converts the quantized signal (information about the quantized transform coefficients) into The information about the quantized transform coefficients is coded and output to the bitstream. The quantization unit 15040 calculates the coefficients based on the scan order. The block-type quantized transform coefficients are rearranged into a one-dimensional vector form, and the one-dimensional vector-type quantized transform coefficients are It is also possible to generate information about the quantized transform coefficients based on the quantized transform coefficients. The Golomb coding unit 15110 may be, for example, an exponential Golomb mb), CAVLC(context-adaptive variable length th coding), CABAC(context-adaptive binary Various encoding methods are used, such as arithmetic coding. The tropy coding unit 15110 processes the quantized transform coefficients as well as the other components required for video / image reconstruction. Share important information (e.g., values ​​of syntax elements) The coded information (e.g. coded video / image information) NAL (Network Abstraction Layer) bitstream format The bitstream is transmitted or stored in units of r) units. The network may be a broadcast network or may be stored on a digital storage medium. and / or communication networks, etc., and digital storage media include USB, SD, CD, DVD, Blu-ray This includes various storage media such as RAID, HDD, and SSD. A transmitting unit (not shown) that transmits the signal output from the The transmitter may be configured as an internal / external element of the encryption device 15000, or the transmitter may be configured as an entropy - It may be included in the encoding unit 15110.

[0244] The quantized transform coefficients output from the quantization unit 15040 are used to generate a prediction signal. For example, the quantized transform coefficients are subjected to the inverse quantization unit 15040 and the inverse transform unit 1506. By applying inverse quantization and inverse transformation by 0, the residual signal (residual block or residual sample) is obtained. The adder 15200 restores the restored residual signal to the inter predictor 15090. Alternatively, by adding it to the predicted signal output from the intra prediction unit 15100, Generates the reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). If there is no residual for the current block, as in the case of skip mode, If not, the predicted block is used as the reconstruction block. The generated reconstructed signal is called the reconstructed block generator. It may be used for intra prediction of the image block, and may be filtered as described below. may be used for inter prediction of the next picture.

[0245] The filtering unit 15070 applies filtering to the restored signal to obtain a subjective / objective For example, the filtering unit 15070 may filter the restored picture. Modified restored pictures using various filtering methods The corrected restored picture is stored in memory 15080, specifically memory 15080. The DPB is stored in various filtering methods, such as deblocking filters. ring, sample adaptive offset , adaptive loop filter, bidirectional filter Filtering unit 15070 provides various information about filtering, such as the various filtering methods described below. The information about filtering is generated and transmitted to the entropy coding unit 15110. The data is coded by the entropy coding unit 15110 and output in the form of a bit stream.

[0246] The modified restored picture sent to memory 15080 is The image is used as a reference picture in the image prediction unit 15090. When inter-prediction is applied, the prediction mix in the encoding device 15000 and the decoding device This can avoid mismatching and improve coding efficiency.

[0247] The DPB in the memory 15080 sends the corrected reconstructed picture to the inter prediction unit 15090. The memory 15080 stores the current picture for use as a reference picture in the The motion information of the block from which the motion information in the block is derived (or coded) and / or the block already decoded. Stores motion information of blocks in the original picture. The stored motion information is used for spatially adjacent blocks. Inter prediction to utilize motion information of a block or temporally adjacent blocks The memory 15080 stores the reconstructed blocks in the current picture. The reconstructed samples of the block are stored and transmitted to the intra prediction unit 15100.

[0248] At least one of the above-described prediction, transformation, and quantization steps may be omitted. For example, a block where PCM (pulse code modulation) is applied For blocks, the prediction, transformation, and quantization steps are omitted, and the original sample values ​​are used as they are. It may be encoded and output to a bitstream.

[0249] FIG. 16 illustrates a V-PCC decoding process according to an embodiment. ) is shown below.

[0250] The V-PCC decoding process or V-PCC decoder is the V-PCC encoding process of FIG. The components in Figure 16 are software, hardware, and The software supports hardware, processors, and / or combinations thereof.

[0251] The demultiplexer 16000 demultiplexes the compressed bitstream. Demultiplexing the image to generate compressed texture images, compressed geometry images, It outputs a compressed occupancy map image and compressed additional patch information.

[0252] Video decompression 16001, 16002 or The video recovery unit recovers the compressed texture image and the compressed geometry image. Restore each one.

[0253] Occupancy map decompression 160 The occupancy map restorer 03 restores the compressed occupancy map image.

[0254] Auxiliary patch information restore The additional patch information restoration unit 16004 or the additional patch information restoration unit 16005 restores the compressed additional patch information. Restore the switch information.

[0255] Geometry reconstruction 16005 or The geometry reconstruction unit generates the reconstructed geometry image, the reconstructed occupancy map, and / Or restore (reconstruct) geometry information based on the restored additional patch information. For example, the encoding process reconstructs the changed geometry.

[0256] Smoothing 16006 or the smoothing part applies smoothing to the reconstructed geometry. Smoothing is applied to the image, e.g., smoothing filtering is applied.

[0257] Texture reconstruction 16007 or The texture reconstruction unit generates the restored texture image and / or the smoothed texture image. Reconstruct texture from geometry.

[0258] Color smoothing 16008 or color smoothing unit Smoothing color values ​​from the reconstructed texture. For example, smoothing filtering Applies.

[0259] As a result, reconstructed point cloud data is generated.

[0260] The figure shows the compressed occupancy map, geometry image, texture image, and additional patch. Illustrates the decoding process of V-PCC to decode the information and reconstruct the point cloud. The operation of each process according to the embodiment is as follows.

[0261] Video decompression 16001, 16002

[0262] This is the reverse process of the video compression mentioned above, and is used to encode 2D video such as HEVC and VVC. Using the above process, the geometry image, texture image, The process of decoding the compressed bitstream of an occupancy map image.

[0263] FIG. 17 shows a 2D video / image decoder according to an embodiment. e Decoder) is shown below.

[0264] The 2D video / image decoder is the reverse process of the 2D video / image encoder in Figure 15. This becomes:

[0265] The 2D video / image decoder of FIG. 17 is an implementation of the video restoration or video restoration unit of FIG. In an embodiment, a 2D video / image signal is decoded. A schematic block diagram of the 2D video / image decoder 17000 is shown. 00 may be included in the point cloud video decoder of Figure 1 or may be an internal or external component. Each component in Figure 17 is composed of software, hardware, processors, and and / or a combination thereof.

[0266] Here, the input bitstream contains the above-mentioned geometry image, texture image, It includes bitstreams for image (quality image), occupancy map image, etc. The image (or output image, decoded image) is the geometry image, texture image, etc. The image (quality image) shows the restored image for the occupancy map image.

[0267] Referring to the figure, an inter prediction unit 17070 and an intra prediction unit 17080 are combined. That is, the prediction unit includes an inter prediction unit 180 and an intra prediction unit 185. The inverse quantization unit 17020 and the inverse transform unit 17030 are collectively called a residual processing unit. The processing unit includes an inverse quantization unit 17020 and an inverse transform unit 17030. 010, an inverse quantization unit 17020, an inverse transformation unit 17030, an addition unit 17040, a filtering unit The prediction unit 17050, the inter prediction unit 17070, and the intra prediction unit 17080 are the same as those in the embodiment. Thus, it is possible to generate a The memory 17060 is configured as a DPB (decoded picture block). It may include a softener, or may be constituted by a digital storage medium.

[0268] When a bitstream containing video / image information is input, the decoding device 17000 performs the following steps. The video / image information is processed in the encoding device of 0.2-1. For example, the decoding device 17000 uses the processing unit applied in the encoding device to Therefore, the decoding processing unit is, for example, a coding unit. , whether the coding unit is a coding tree unit or a maximum coding unit The data is divided into a quad-tree structure and / or a binary-tree structure. The restored video signal decoded and output by the decoder 17000 is played back by a playback device. will be done.

[0269] The decoding device 17000 receives the signal output from the encoding device in the form of a bit stream. The received signal is decoded by the entropy decoder 17010. The decoder 17010 parses the bitstream and restores the video (or picture) ) to derive information (e.g., video / image information) necessary for the entropy decoding. 17010 is based on coding methods such as Exponential Golomb coding, CAVLC or CABAC. The information in the bitstream is decoded based on the above and the values ​​of the syntax elements required for video restoration, It outputs the quantized values ​​of the transform coefficients related to the covariance. More specifically, the CABAC entropy The decoding method includes receiving bins corresponding to each syntax element in the bitstream, and decoding the structure to be decoded. The sentence element information and the decoding information of the adjacent and target blocks or the symbols decoded in the previous step The context model is determined using the information from the rule / bin. The probability of occurrence of a bin is predicted according to the decoding) to generate symbols corresponding to the values ​​of each syntax element. After determining the context model, the CABAC entropy decoding method Update the context model using the decoded symbol / bin information for the context model The information related to prediction among the information decoded by the entropy decoding unit 17010 is The entropy is provided to the inter-prediction unit 17070 and the intra-prediction unit 17080. The residual values ​​obtained by entropy decoding in the decoder 17010, i.e., the quantized transform coefficients and The dequantization unit 17020 receives the dequantization signal and related parameter information. The information regarding filtering among the information decoded by the unit 17010 is sent to the filtering unit 17020. On the other hand, a receiving unit (not shown) receives the signal output from the encoding device. The receiving unit may be further configured as an internal / external element of the decoding device 17000. It may be a component of the entropy decoding unit 17010.

[0270] The inverse quantization unit 17020 quantizes the quantized transform coefficients and outputs the transform coefficients. The quantization unit 17020 rearranges the quantized transform coefficients into a two-dimensional block. In this case, the inverse quantization unit 1702 performs reordering based on the coefficient scanning order performed by the encoding device. 0 is the quantized transform using the quantization parameter (e.g., quantization step size information) Dequantize the coefficients to get the transform coefficients. ) to get the

[0271] The inverse transform unit 17030 inversely transforms the transform coefficients to generate a residual signal (residual block, residual sample array).

[0272] The prediction unit performs prediction for the current block and includes predicted samples for the current block. Generates a predicted block. The prediction part uses entropy. The current block is input based on the prediction information output from the block decoder 17010. Determine whether intra prediction or inter prediction is applied, and / Determine the inter prediction mode.

[0273] The intra prediction unit 265 predicts the current block by referring to samples in the current picture. The referenced samples are neighbors of the current block depending on the prediction mode. In intra prediction, the prediction mode may be located adjacent to the pixel or may be located apart. The intra prediction unit 265 predicts adjacent frames in a non-directional mode and a directional mode. The prediction mode applied to the block is used to determine the prediction mode applied to the current block. do.

[0274] The inter prediction unit 17070 predicts a reference picture identified by a motion vector on the reference picture. Based on the reference block (reference sample array), a predicted block for the current block is calculated. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, Then, the motion information is calculated based on the correlation between the motion information of the adjacent block and the current block. The motion information is a motion vector and a reference picture image. The motion information also includes the inter prediction method (L0 prediction, L1 prediction, Bi prediction). In the case of inter prediction, neighboring blocks contain information about the empty blocks in the current picture. Spatial neighboring blocks and reference pixels Temporal neighboring blocks in the chart For example, the inter prediction unit 17070 may include motion information based on neighboring blocks. and constructing a candidate information list, and calculating the motion vector and the motion vector of the current block based on the received candidate selection information. and / or reference picture indexes based on different prediction modes. A prediction is made, and the prediction information indicates the mode of inter prediction for the current block. Contains information about:

[0275] The adder 17040 outputs the residual signal obtained by the inverse transformer 17030 to the inter predictor 1707 0 or the predicted signal output from the intra prediction unit 17080 (predicted block, prediction signal) The reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) is obtained by adding array) for the current block, as if skip mode was applied. If there is no corresponding residual, the predicted block is used as the reconstruction block.

[0276] The adder 17040 is called a restoration unit or restoration block generator. It may be used for intra prediction of the next block to be processed in the picture, as described below. It can also be used for inter prediction of the next picture after filtering, like this: good.

[0277] The filtering unit 17050 applies filtering to the restored signal to obtain a subjective / objective For example, the filtering unit 17050 applies various filters to the restored picture. applying a filtering method to generate a modified reconstructed picture; The corrected restored picture is sent to memory 17060, specifically to the DPB of memory 17060. Various filtering methods include, for example, deblocking filtering, sampling Sample adaptive offset, adaptive loop adaptive loop filter, bidirectional filter lateral filter) etc.

[0278] The (corrected) restored picture stored in the DPB in memory 17060 is inter-predicted. The memory 17060 stores the current picture. The motion information and / or the already existing motion information of the blocks from which the motion information in the The motion information of the blocks in the reconstructed picture is stored in the In order to utilize the motion information of the image-based neighboring blocks as the motion information of the image-based neighboring blocks or the motion information of the temporally neighboring blocks, The memory 17060 stores the reconstructed data in the current picture. The reconstructed samples of the block are stored and transmitted to the intra prediction unit 17080.

[0279] In this specification, the filtering unit 160 and the inter-prediction unit 1 The embodiment described in the intra-prediction unit 185 and the filtering unit 180 of the decoding device 17000 The same or different prediction units are also used for the prediction unit 17050, the inter prediction unit 17070, and the intra prediction unit 17080. The corresponding methods can be applied.

[0280] On the other hand, even if at least one of the above-mentioned prediction, inverse transform, and inverse quantization steps is omitted, For example, PCM (pulse code modulation) is applied to For locking, the prediction, inverse transform, and inverse quantization steps are omitted and the decoded sample values ​​are used. is used as a sample of the restored video.

[0281] Occupancy map decompression 160 03

[0282] This is the reverse process of the occupancy map compression described above, and restores the compressed occupancy map bitstream. This is the process of decoding the data and restoring the occupancy map.

[0283] Auxiliary patch info decompre ssion)16004

[0284] This is the reverse process of the additional patch information compression described above, and the compressed additional patch information bit stream is This is the process of decoding the program to recover the additional patch information.

[0285] Geometry reconstruction 16005

[0286] This is the reverse process of the geometry image generation described above. First, the restored occupancy map and the additional The patch information includes the 2D position / size information of the patch and the mapping between the block and the patch. The patch is extracted from the geometry image using the pattern information. Using the 3D position information of the patch included in the geometry image and the additional patch information, Reconstruct the cloud in 3D space. It corresponds to any point (u, v) that exists within one patch. The geometry value is called g(u, v), and the normal axis of the patch's position in 3D space, When the coordinate values ​​of the tangent axis and bitangent axis are (δ0, s0, r0), The normal axis, tangent axis, and b of the position in 3D space that is mapped to the point (u, v) The coordinate values ​​of the itangent axis, δ(u, v), s(u, v), and r(u, v), are as follows: It is shown as follows.

[0287] d(u,v)=d0+g(u,v)

[0288] s(u,v)=s0+u

[0289] r(u,v)=r0+v

[0290] Smoothing16006

[0291] This is similar to the smoothing in the encoding process described above, and is a result of the image quality degradation that occurs during the compression process. This is a process to remove discontinuities that may arise from patch interfaces due to degradation.

[0292] Texture reconstruction 16007

[0293] A color point cloud is created by assigning a color value to each point that makes up the smoothed point cloud. This is the process of restoring the diorama. Using the mapping information of the metrology image and point cloud, the geometry is calculated in 2D space. The color values ​​corresponding to the texture image pixels at the same position as the image are set to the same value in 3D space. This is done by assigning a point in the point cloud that corresponds to a location.

[0294] Color smoothing 16008

[0295] Similar to the geometry smoothing process described above, the image quality degradation that occurs from the compression process is a process for removing discontinuities in color values ​​that may arise from patch boundaries by Color smoothing is performed as follows:

[0296] 1) Each component of the color point cloud is reconstructed using a KD tree or similar method. Calculate the neighboring points of a point. The neighboring point information calculated in the geometry smoothing process described above is used It may be used as is.

[0297] 2) For each point, determine whether the point is located on the patch boundary surface. The boundary surface information calculated in the geometry smoothing process may be used as is.

[0298] 3) Whether to smooth the distribution of color values ​​for neighboring points on the boundary surface For example, if the entropy of the brightness value is below the threshold local entry) (when there are many similar brightness values), The smoothing method is to calculate the color of the point by the average value of the adjacent points. -There is a way to replace the value.

[0299] FIG. 18 shows an example of the flow of operations of the transmitting device according to the embodiment.

[0300] The transmitting device according to the embodiment is the transmitting device of FIG. 1, the encoding process of FIG. 4, the 2D video encoding process of FIG. It may correspond to an audio / image encoder or perform some or all of these operations. Each component of the device may be software, hardware, a processor, and / or a combination thereof. Corresponds to matching.

[0301] The operation of the transmitting end for compressing and transmitting point cloud data using V-PCC is shown in Fig. This seems to be shown.

[0302] The point cloud data transmission device according to the embodiment is referred to as a transmission device or the like.

[0303] The patch generation unit 18000 receives point cloud data and generates a point cloud Patches for 2D image mapping of point clouds patch information and / or additional patch information are generated as a result of the patch generation; The generated patch information and / or additional patch information are used as a geometry image (geomet ry image generation, texture image generation, Used in the geometry reconstruction process for smoothing or smoothing.

[0304] The patch packing unit 18001 packs the patches generated by the patch generation unit into a 2D image. The patch packing process is performed to map the An occupancy map is generated, and the occupancy map is used to generate the geometry image. It is used in the geometry reconstruction process for image generation, texture image generation, and smoothing.

[0305] The geometry image generator 18002 generates a geometry image using the additional patch information and the occupancy map. A tri-image is generated, and the generated geometry image is encoded into one bit It is encoded into a stream.

[0306] The encoding preprocessing unit 18003 includes an image processing unit. Geometry regenerated by decoding an image or coded geometry bitstream The images are used for 3D geometry decoding, followed by a smoothing process.

[0307] The texture image generator 18004 generates smoothed 3D geometry, point clouds, and Generate texture images using loud data, additional patch information, and occupancy maps The generated texture images are then encoded into a single video bitstream.

[0308] The metadata encoding unit 18005 encodes the additional patch information into one metadata bitstream. Encode it into a

[0309] The video encoding unit 18006 encodes the occupancy map into a video bitstream. do.

[0310] The multiplexing unit 18007 multiplexes the generated geometry, texture images, and occupancy maps. Video bitstream and additional patch information metadata bitstream are combined into one bitstream. Multiplex the data into a single stream.

[0311] The transmitting unit 18008 transmits the bit stream to the receiving end. Video bitstream and additional patch information metadata for image, texture image, and occupancy map A data bitstream is a file generated from one or more tracks of data or segments. The packet is encapsulated and sent from the sender to the receiver.

[0312] FIG. 19 shows an example of the flow of operations of the receiving device according to the embodiment.

[0313] The receiving device according to the embodiment includes the receiving device of FIG. 1, the decoding process of FIG. 16, and the 2D video decoding process of FIG. Each receiver corresponds to an audio / image encoder or performs part or all of the operations of the audio / image encoder. Components may be software, hardware, processors and / or combinations thereof. handle.

[0314] Operational process of the receiver for receiving and restoring point cloud data using V-PCC The operation of the V-PCC receiving end is the reverse of the operation of the V-PCC transmitting end in Figure 18. That's about it.

[0315] The point cloud data receiving device according to the embodiment is referred to as a receiving device or the like.

[0316] The received point cloud bitstream is stored in a file / segmenter after encapsulation. Compressed video bitstreams of geometry images, texture images, and occupancy maps The demultiplexer 19000 demultiplexes the bit stream and the additional patch information metadata bit stream. The video decoder 19001 and the metadata decoder 19002 demultiplex the The geometry recovery unit decodes the video bitstream and metadata bitstream. Using the geometry image decoded by 19003, the occupancy map, and additional patch information, The 3D geometry is then restored, and a smoothing process is then performed by the smoothing unit 19004. Applying color values ​​to smoothed 3D geometry using texture images The color point cloud video / picture is then processed by the texture restoration unit 19005. Then, a color smoothing process is performed to improve the objective and subjective visual quality. Further processing is carried out, and the modified point cloud derived from this The video / picture is shown to the user after the rendering process. The lubrication process may be omitted in some cases.

[0317] FIG. 20 illustrates a V-PCC-based point cloud data storage and streaming system according to an embodiment. 1 shows an example of an architecture for grouping.

[0318] Part or all of the system of FIG. 20 may be implemented using the transmitter / receiver of FIG. 1, the encoding process of FIG. 4, and the 16, the 2D video / image encoder of FIG. 16, the transmitting device of FIG. 18, and / or Or includes part or all of the receiving device of Figure 19. Each component of the drawing is software, hardware, etc. Supports hardware, processors and combinations thereof.

[0319] 20 to 22 show a structure in which a system is further connected to a transceiver according to an embodiment. The transmitting / receiving device and the system according to the embodiment are all collectively referred to as the transmitting / receiving device according to the embodiment. do.

[0320] The apparatus according to the embodiment shown in FIGS. 20 to 22 is a transmitting apparatus corresponding to FIG. 18 etc. Data format for transmitting a bitstream containing the encoded point cloud data. Generate a container to fit the mat.

[0321] The V-PCC system according to the embodiment generates a container containing point cloud data. , and further adds additional data to the container that is necessary for efficient transmission and reception.

[0322] The receiving device according to the embodiment receives the container based on the system shown in FIGS. 20 to 22. The receiving device corresponding to FIG. 19 receives the parsed data and parses it. Decode and recover point cloud data from the bit stream.

[0323] Figure 1. Point clouds compressed based on video-based point cloud compression (V-PCC). The overall architecture for storing or streaming loud data is shown. The process of storing and streaming cloud data includes the acquisition process, encoding process, process, transmission process, decoding process, rendering process and / or feedback process Includes the process.

[0324] The embodiment illustrates how to efficiently provide point cloud media / content / data. suggest.

[0325] Point cloud acquisition unit 20000 is a point cloud media / content / data To efficiently provide the data, first, we acquire a point cloud video. For example, These cameras allow for the capture, synthesis or generation of point clouds. This acquisition process allows for the creation of 3D point cloud data for each point. Position (expressed as x, y, z position values, hereafter referred to as geometry), characteristics of each point ( You can get point cloud videos that include color, reflectance, transparency, etc. The captured point cloud video contains this, e.g., PLY (Polygon Filter) le format or the Stanford Triangle format t) file etc. Point cloud data with multiple frames can be generated. In the case of data, one or more files can be retrieved. Generates cloud-related metadata (e.g., metadata related to the capture, etc.) It can be achieved.

[0326] The captured point cloud video can be later processed to improve the quality of the content. During the video capture process, the camera equipment may need to process the You may adjust the maximum and minimum depth values ​​within the desired range, but if the adjustments still result in points in the undesired area, Since the image may contain unwanted data, it is possible to remove unwanted areas (e.g., background) or Post-processing may be performed to recognize connected spaces and fill spatial holes. In addition, point clouds extracted from cameras sharing a spatial coordinate system can be calibrated. Based on the acquired position coordinates of each camera, the coordinates of each point in the global coordinate system are calculated. They may be merged into one content through a conversion process, which allows for the density of points to be High-quality point cloud video can be obtained.

[0327] Point cloud pre-processing unit)20001 converts a point cloud video into one or more pictures / frames Here, a picture / frame generally represents one image at a particular time. The points that make up a point cloud video are divided into one or more patches (points). A set of points that make up a patch cloud is a set of points that belong to the same patch in 3D space. They are adjacent to each other and form a six-sided bounding box in the mapping process to the 2D image. When dividing the object into points (a set of points on the object plane that are mapped in the same direction) and mapping them to a 2D plane, , a binary map ( It can generate an occupancy map picture / frame, which is a binary map In addition, the position information (geometry) of each point that makes up the point cloud video is stored in a patch-by-patch manner. Geometry picture / frame, which is a picture / frame in depth map format, expressed as a It is possible to generate a point cloud video by representing the color information of each point in the video patch. You can generate a texture picture / frame that is a picture / frame that In this process, the number of iterations required to reconstruct a point cloud from individual patches is It can generate metadata that describes the 2D / 3D space of each patch. It includes information about the patch, such as its position, size, etc. (called additional information or additional patch information) These pictures / frames are generated in time sequence and form a video stream or A metadata stream can be configured.

[0328] Point Cloud Video Encoder 20002 is related to point cloud video It can be encoded into one or more video streams. A video can consist of multiple frames. In this specification, points Point cloud video includes point cloud footage / frames / pictures, Point cloud video may be mixed with point cloud footage / frames / pictures The point cloud video encoder uses video-based point cloud compression (V-PC) C) The point cloud video encoder performs the compression and coding procedures. To achieve this, a series of steps such as prediction, transformation, quantization, and entropy coding are performed. The coded data (coded video / image information) is stored in a bitstream. Based on the V-PCC procedure, the point cloud video encoder is As described below, we convert point cloud video into geometry video and attribute e) Video, occupancy map video, and metadata, e.g., Geometry video can be coded by dividing it into geometry images. attribute video may include attribute images; The occupancy map video may include an occupancy map image. The patch data may contain information about the patch. , may contain texture video / images.

[0329] Point Cloud Image Encoder 20003 is a point cloud video related Point Cloud Image Encoding2 0003 performs the Video-Based Point Cloud Compression (V-PCC) procedure. Cloud Image Encoder predicts, transforms, and scales for compression and coding efficiency. A series of steps such as encoding, entropy coding, etc. can be performed. The image is output in bitstream format. Cloud Image Encoder 20003 converts point cloud images into Geometry image, attribute image, occupancy ) map image and metadata, e.g., information about the patch, can be encoded separately. This can be done.

[0330] Point cloud video encoder and / or point cloud image encoder according to embodiments The encoder generates a PCC bitstream (G-PCC and / or V-PCC) according to the embodiment. bitstream).

[0331] Depending on the embodiment, a video encoder 20002, an image encoder 20003, a video The image decoder 20006, as mentioned above, is a single encoder / decoder. This may be done by the da, or by another route as shown in the drawing.

[0332] file / segment encapsulation unit 20004 relates to encoded point cloud data and / or point clouds The metadata is encapsulated in the form of a file or a segment for streaming. Here, the metadata related to the point cloud is transferred from the metadata processing unit etc. The metadata processing unit may be a point cloud video / image encoder. It may be contained within a single component or may consist of a separate component / module. The management department then stores the video / image / metadata in a file format such as ISOBMFF. The encapsulation processing unit encapsulates the data into a packet or processes it into a form such as a DASH segment. According to an embodiment, metadata associated with the point cloud is stored in a file format. Point cloud metadata can be included in e.g. ISOBMFF files. The boxes are contained in various levels of the file format, or According to an embodiment, the encapsulation unit Cloud-related metadata itself can be encapsulated in the file.

[0333] The encapsulation and encapsulation unit according to the embodiment is a G-PCC / V-PCC bitstream. The data is split into one or more tracks within a file, and the signaling information for that is stored in the Also, the G-PCC / V-PCC bitstream contains The atlas streams are stored in tracks within the file, along with the associated signaling information. Furthermore, the SE present on the G-PCC / V-PCC bitstream may be stored. I store messages in tracks within a file, along with associated signaling information. good.

[0334] The transmission processing unit receives the encapsulated point cloud data according to the file format. The transmission processing unit may be included in the transmission unit or may be a separate computer. The transmission processing unit may be configured as a component / module. Therefore, the point cloud data can be processed. The processing may include processing for transmission over a broadband network, and processing for transmission over a broadband network. According to the embodiment, the transmission processing unit processes not only the point cloud data but also the metadata. The point cloud related metadata is transmitted from the good.

[0335] The transmitter transmits the point cloud bitstream or a file containing the bitstream. and transmitting the message / segment to the receiving section of the receiving device via a digital storage medium or a network. For transmission, processing may be performed according to any transmission protocol. The data is transmitted via broadcast networks and / or broadband. It is transmitted to the receiver on demand. Includes a variety of USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The sender is a component for generating a media file in a predetermined file format. and may include elements for transmission over a broadcast / communication network. The bitstream is extracted and sent to the decoder.

[0336] The receiving unit receives the point cloud data transmitted by the point cloud data transmitting device according to this specification. Depending on the channel being transmitted, the receiver receives point data via the broadcast network. The point cloud data may be received via broadband. Alternatively, the point cloud video data may be received by a digital storage medium. The receiving unit may decode the received data and display it in response to the user's viewport, etc. may be rendered accordingly.

[0337] The receiving processor processes the received point cloud video data according to the transmission protocol. The reception processing unit may be included in the reception unit or may be a separate component / module. In response to the processing for transmission on the sending side, the receiving processing The receiving processing unit performs the reverse process of the transmission processing unit. The point cloud is then sent to the decapsulator, and the metadata related to the acquired point cloud is The data is transmitted to the data processing unit.

[0338] file / segment decapsulation unit )20005 is a data processing system that processes point cloud data in file format sent from the receiving processor. The decapsulation part decapsulates the file using ISOBMFF etc. and the point cloud bitstream or point cloud related metadata (or other The acquired point cloud image (metadata bitstream) can be obtained. The point cloud stream is a point cloud video decoder and a point cloud image decoder. The point cloud-related metadata (or metadata bitstream) is transmitted to the The point cloud bitstream is transmitted to the metadata processing unit. The metadata processing unit may include the pointer data (metadata bitstream). It may be included in a cloud video decoder or may consist of a separate component / module. The point cloud related metadata obtained by the decapsulation unit may be stored in the file. The decapsulator may be in the form of a box or track within the format. In such cases, the metadata necessary for decapsulation may be transmitted from the metadata processing unit. Point cloud related metadata is stored in the point cloud video decoder and / or point may be transmitted to a point cloud image decoder for use in point cloud decoding; Or it may be transmitted to a renderer and used for point cloud rendering.

[0339] The point cloud video decoder 20006 receives the bitstream and Decode the video / image by performing the reverse process corresponding to the operation of the cloud video encoder In this case, the point cloud video decoder 20006 generates the point cloud video as described below. Loud video, geometry video, attribute video, occupancy pancy) map video and auxiliary patch information Geometry video can be decoded into geometry information. Attribute videos may contain attribute images. Frequently, the occupancy map video may include an occupancy map image. The additional information is auxiliary patch information. n). Attribute video / image may include texture video / image. good.

[0340] 3D using decoded geometry video / image, occupancy map and additional patch information The geometry is restored and then smoothed. The smoothed 3D geometry is then Color point classification by assigning color values ​​using textured video / images The renderer uses the restored geometry, color points, and Render the cloud video / picture. The rendered video / picture The user can view the image on a VR / AR display or a general display. You can view all or part of the rendered results on a display or other device. Cut.

[0341] The Sensing / Tracking section 20007 Obtain orientation information and / or user viewport information from the user or receiver. The orientation information is transmitted to the receiving unit and / or transmitting unit. It shows information about the position, angle, movement, etc. of the device the user is looking at. Based on this information, the system calculates the area the user is currently viewing in 3D space. The information to be displayed, i.e., viewport information, is calculated.

[0342] Viewport information is the information that a user currently sees in 3D space through a device or HMD. The display or other device may be used to provide orientation information. Information, vertical or horizontal FO supported by the device You can extract the viewport area based on the orientation or V. The orientation or location information is extracted or calculated at the receiving end. The viewport information may be transmitted to the sender via a feedback channel.

[0343] The receiving unit receives the orientation information and and / or viewport information indicating the area the user is currently looking at to identify a specific area, i.e. Only the media data in the area indicated by the orientation information and / or viewport information is displayed. The transmitting unit also transfers the data to the sensing / tracking unit. Orientation information and / or viewport information obtained by The media data of the area, i.e., the area indicated by the orientation information and / or viewport information It is possible to efficiently encode only the data or generate and transmit files.

[0344] The renderer renders the decoded point cloud data in 3D space. The rendered video / image is displayed through the display unit. The image is rendered via a VR / AR display or a general display. You can view all or part of the results.

[0345] The feedback process is as follows: Various feedback information can be sent to the transmitter or to the decoder at the receiver. The feedback process may include: According to an embodiment, the present invention provides interactivity by: In the feedback process, head orientation ation information, and a viewport that shows the area the user is currently looking at. According to the embodiment, the user can transmit information such as VR / AR / MR / independent It is possible to interact with the objects embodied in the driving environment, but in this case, the interaction information about the service provider and the sender in a feedback process. According to an embodiment, the feedback process may be omitted.

[0346] According to an embodiment, the above-mentioned feedback information is not only transmitted to the sender side, but also: It can also be consumed at the receiving end, i.e., the above-mentioned feedback information is used to It may also perform encapsulation, decoding, rendering processes, etc. location information and / or viewport information to determine the area the user is currently looking at. The point cloud data may be decapsulated, decoded and rendered as a priority.

[0347] The point cloud data transmission method according to the embodiment includes encoding the point cloud data. encapsulating the point cloud data; and transmitting;

[0348] A point cloud data receiving method according to an embodiment receives point cloud data. a step of decapsulating the point cloud data; and a step of extracting the point cloud data. decrypting the data.

[0349] FIG. 21 shows an example of the configuration of a point cloud data storage and transmission device according to an embodiment. vinegar.

[0350] FIG. 21 illustrates a point cloud system according to an embodiment, where part / all of the system The transmitter / receiver device of FIG. 1, the encoding process of FIG. 4, the 2D video / image encoder of FIG. 15, 16, a part or all of the transmitting device in FIG. 18 and / or the receiving device in FIG. 19, etc. It may also be included in or correspond to part / all of the system in Figure 20. can be done.

[0351] The point cloud data transmitting device according to the embodiment is configured as shown in the drawing. Each component of the system is a module / unit / component / hardware / software. / processor, etc.

[0352] Point cloud geometry, characteristics, additional data (or information), and mesh data Each of these consists of an independent stream, or each is stored in a different file. The data may be stored in different tracks, or may be included in different segments.

[0353] Point Cloud Acquisition Unit t) 21000 acquires a point cloud, e.g., via one or more cameras. Point cloud data can be generated by a variety of processes, including point cloud capture, synthesis, or generation. This acquisition process allows us to obtain the 3D position (x, y, z position) of each point. The geometry is represented by a number of points, such as a color, reflectance, transparency, etc. Point cloud data including brightness etc. is obtained, and this includes, for example, PLY(Po lygon File format or the Stanford Triang It can be generated as a multi-frame file. For in-cloud data, one or more files can be retrieved. In this case, point cloud related metadata (e.g., related to the capture) It is possible to generate data such as

[0354] The patch generation unit 21001 is The patch generator 21001 generates patches from point cloud data. Generate a data or point cloud video with one or more pictures / frames. In other words, a picture / frame is a unit that shows one image at a specific time. The points that make up the point cloud video are divided into one or more patches (a collection of points that make up the point cloud). In this case, points that belong to the same patch are adjacent to each other in the 3D space and are in the 2D image. In the mapping process, the six bounding box planes are mapped in the same direction. When the data is divided into points (a set of points to be mapped) and mapped onto a 2D plane, the data is stored at that position on the 2D plane. A binary map that indicates whether something is present or not with a value of 0 or 1. Occupancy map pictures / frames can be generated. Represents the geometry of each point in a point cloud video in patch units. Generates a geometry picture / frame that is a picture / frame in depth map format. A picture / frame that represents the color information of each point that makes up the point cloud video in units of patches. In this process, a texture picture / frame can be generated. Generate the metadata needed to reconstruct a point cloud from individual patches This metadata contains information about each patch, such as its location in 2D / 3D space, its size, etc. Such pictures / frames may contain information about the time sequence. It can be generated to constitute a video stream or a metadata stream.

[0355] Patches may also be used for 2D image mapping, e.g., points Cloud data is projected onto each face of the cube. After patch generation, the generated patch Based on the geometry image, one or more attribute images, an occupancy map, and Data and / or mesh data, etc. can be generated.

[0356] The preprocessing unit or controller generates a geometry image (G geometry image generation, attribute image generation Butte Image Generation, Occupancy Map Generation map generation, Auxiliary Data generation Generation) and / or Mesh Data Generation ration) is carried out.

[0357] Geometry Image Generation unit)21002 generates a geometry image based on the results of patch generation The geometry represents a point in 3D space. The 2D image of the patch is based on the patch. Occupancy maps, additional data (patch data) and / or metadata containing information related to packing Generate a geometry image using the image data. This relates to information such as the depth (e.g., closeness, distance) of the patch generated after the patch is generated.

[0358] Attribute image generation unit It 21003 generates a feature image. For example, a feature is a texture. The texture is a color value corresponding to each point. According to an embodiment, multiple (N) characteristics (color, reflectance, etc.) including texture can be used. Generates an image. Multiple attributes include material, reflectivity, etc. In addition, according to the embodiment, the characteristic is that the color changes depending on the visual and light even with the same texture. The information may further include:

[0359] Occupancy map generation unit 21004 generates an occupancy map from the patch. The occupancy map is a representation of the geometry or characteristics of the patch. Contains information indicating the presence or absence of data in a pixel of a quality image, etc.

[0360] Auxiliary Data Generation unit ) 21005 generates additional data containing information about the patch. , which indicates metadata about a patch of a point cloud object. For example, It is possible to display information such as line (normal) vectors. For example, the additional data includes the information necessary to reconstruct a point cloud from the patches ( For example, information about the patch's position in 2D / 3D space, size, etc., projection plane (normal) identity, patch mapping information, etc.

[0361] Mesh Data Generation unit 21 006 generates mesh data from the patch. The mesh is a set of connections between adjacent points. For example, triangle data may be shown. For example, a mesh according to an embodiment The data represents the connectivity information between each point.

[0362] The point cloud preprocessor or controller performs patch generation, geometry image generation, and feature Metadata related to image generation, occupancy map generation, additional data generation, and mesh data generation Generates metadata.

[0363] The point cloud transmitting device performs video encoding in response to the result generated by the pre-processing unit. and / or image encoding. The point cloud transmitting device transmits the point cloud video. In addition to the image data, point cloud image data is generated. Point cloud data can be video data only, image data only, and / or video It may contain both audio data and image data.

[0364] The video encoding unit 21007 performs geometry video compression, characteristic video compression, occupancy map video compression, and the like. The video coding unit performs video compression, additional data compression and / or mesh data compression. A video stream containing each of the encoded video data is generated.

[0365] Specifically, geometry video compression encodes point cloud geometry video data. Feature video compression encodes the feature video data of the point cloud. Data compression encodes additional data associated with the point cloud video data. Mesh data compression is used to compress point cloud video data. Each operation of the point cloud video encoding unit is performed in parallel. will be held.

[0366] The image encoding unit 21008 performs geometry image compression, characteristic image compression, occupancy mapping, and the like. Image coding performs snap image compression, additional data compression and / or mesh data compression. The encoding unit generates an image containing each of the encoded image data.

[0367] Specifically, geometry image compression is used to compress point cloud geometry image data. Feature image compression encodes the feature image data of the point cloud. Additional data compression encodes additional data associated with the point cloud image data. Mesh data compression is a point class. Encode mesh data associated with point cloud image data. The operations of the encoding unit are performed in parallel.

[0368] The video encoder and / or image encoder receives the metadata from the pre-processor. The video encoder and / or the image encoder perform their respective encoding processes based on the metadata. Do the following.

[0369] File / Segment Encapsulation tion)21009 is used to store video streams and / or images in files and / or The file / segment encapsulator encapsulates the video track data. The encapsulation includes metadata track encapsulation and / or image encapsulation.

[0370] A video track encapsulation is a single stream of one or more video streams. The track on the can be encapsulated.

[0371] The metadata track encapsulation contains metadata associated with the video stream and / or image. The metadata is encapsulated in one or more tracks. Contains data related to the content of the audio data, e.g., initial viewing orientation. Orientation Metadata (Initial Viewing Orientation Me According to an embodiment, the metadata is encapsulated in a metadata track. They may be encapsulated together in a video track or an image track.

[0372] Image encapsulation is the process of storing one or more images in one or more tracks or Encapsulate it into an item.

[0373] For example, according to an embodiment, four video streams and two images are included in the encapsulation. If input, 4 video streams and 2 images are encapsulated into one file. Cellify.

[0374] Point cloud video encoder and / or point cloud image encoder according to embodiments The encoder generates a G-PCC / V-PCC bitstream according to the embodiment.

[0375] The file / segment encapsulator receives the metadata from the preprocessor. The / segment encapsulator performs encapsulation based on metadata.

[0376] The files and / or segments created by file / segment encapsulation are , transmitted by a point cloud transmitting device or transmitting unit. For example, DASH-based Segments can be delivered based on the protocol.

[0377] The encapsulation and encapsulation unit according to the embodiment converts the V-PCC bitstream into a file. The signaling information is also encapsulated in one or more tracks. In addition, the atlas data included in the V-PCC bitstream can be Streams can be stored in tracks within a file, along with associated signaling information. Furthermore, SEI messages present on the V-PCC bitstream can be stored in a file. The signaling information can be stored in a track within the ROM and associated signaling information can be stored.

[0378] Delivery is a point cloud bitstream or its bitstream. Receive files / segments containing streams via digital storage media or network The data is transmitted to the receiving section of the device, and then processed according to the desired transmission protocol. The processed data is transmitted via a broadcast network and / or broadband. This data may be transmitted to the recipient on an on-demand basis. Digital storage media include USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. There are various storage media. The transmission unit transfers media files in a predetermined file format. It includes elements for generating the signal and for transmitting it over a broadcast / communication network. The transmitting unit receives the orientation information and / or the viewport information from the receiving unit. The transmission unit transmits the acquired orientation information and / or viewport information (or the user Selected information) is transferred to the preprocessing section, video coding section, image coding section, file / segment Orientation information is transmitted to the encapsulator and / or point cloud encoder. and / or based on the viewport information, the point cloud encoder encodes all the point clouds Encode loud data or orientation information and / or viewport information Encodes the point cloud data indicated by the orientation information and / or view Based on the port information, the file / segment encapsulator will encapsulates the data or indicates orientation and / or viewport information It can encapsulate point cloud data, including orientation information and and / or based on the viewport information, the transmitter transmits all point cloud data. Or the orientation information and / or viewport information indicates the point cloud data. Transmit data.

[0379] For example, the preprocessing unit may perform the above-described operations on all point cloud data. Often, orientation information and / or viewport information indicates point cloud data. The video encoder and / or image encoder may perform operations on all The operations described above may be performed on the point cloud data, and the orientation information And / or perform the above-mentioned operations on the point cloud data indicated by the viewport information. The file / segment encapsulator is responsible for all point cloud data. The above-described operations may be performed using the orientation information and / or viewport information. The transmitting unit may perform the above-described operations on the point cloud data indicated by all The operations described above may be performed on the point cloud data, and the orientation information And / or perform the above-mentioned operations on the point cloud data indicated by the viewport information. That's fine.

[0380] FIG. 22 illustrates an example of the configuration of a point cloud data receiving device according to an embodiment.

[0381] FIG. 22 illustrates a point cloud system according to an embodiment, where part / all of the system The transmitter / receiver device of FIG. 1, the encoding process of FIG. 4, the 2D video / image encoder of FIG. 15, 16, the transmitting device of FIG. 18, and / or the receiving device of FIG. 19, etc. 20 and 21. corresponds to.

[0382] The receiver consists of modules, units, components, hardware, and software. It may be a hardware / processor etc. Delivery Client ent) is a point cloud transmitted by the point cloud data transmitting device according to the embodiment. Data, point cloud bitstream or file containing that bitstream / Depending on the channel being transmitted, the receiving device receives the points via the broadcast network. receive point cloud data or transmit point cloud data via broadband Alternatively, the point cloud data may be received by a digital storage medium. The receiving device decodes the received data and renders it according to the user's viewport, etc. The receiving processing unit may include a process for processing the received point cloud data. The reception processing unit may be included in the receiving unit or may be a separate unit. It may be composed of the following components / modules. Correspondingly, the reception processing unit performs the reverse process of the transmission processing unit described above. The point cloud data is sent to the file / segment decapsulator and the acquired points are The client cloud related metadata may be transmitted to a metadata processing unit.

[0383] The sensing / tracking section is The sensing / tracking unit acquires the motion information and / or viewport information. The client, file, and / or server communicate the acquired orientation and / or viewport information. The data is then transmitted to the point cloud decapsulator / segment decapsulator and point cloud decoder.

[0384] The communication client may determine the orientation and / or viewport information based on the orientation and / or viewport information. , receive all point cloud data, or orientation information and / or Receives point cloud data indicated by viewport information. File / Segment Deck The encapsulation unit encapsulates all the images based on the orientation information and / or viewport information. Decapsulate point cloud data or orientation information and / or Decapsulates the point cloud data indicated by the viewport information. Point Cloud The decoder (video decoder and / or image decoder) receives the orientation information and / or Based on the viewport information, it decodes all point cloud data or Decode point cloud data indicated by orientation information and / or viewport information The point cloud processing unit processes all point cloud data or Or point cloud data indicated by orientation information and / or viewport information Process.

[0385] File / Segment decapsulation ation unit)22000 is a video track decapsulation ack Decapsulation), Metadata Track Decapsulation ata Track Decapsulation) and / or Image Decapsulation ( The decapsulation processing unit performs the image decapsulation. Decapsulates point cloud data in a file format transmitted from The decapsulation processing unit decapsulates the file or segment by ISOBMFF etc. and Point cloud bitstream and point cloud related metadata (or separate metadata) The acquired point cloud bit stream is The point cloud related metadata (or metadata) is transmitted to the point cloud decoder. The point cloud bitstream is transmitted to the metadata processing unit. The stream may include metadata (metadata bitstream). may be included in the point cloud video decoder or may be a separate component / module The point cloud related metadata acquired by the decapsulation processing unit may be configured as may be in the form of boxes or tracks within a file format. The segment decapsulator, if necessary, obtains the necessary data for decapsulation from the metadata processor. Metadata may be transmitted. Point cloud related metadata may be transmitted The point cloud may be decoded by the decoder. The file may be transmitted to the rendering component and used for point cloud rendering. The / Segment Decapsulator generates metadata associated with the point cloud data. It is possible.

[0386] Video Track Decapsulation n) decapsulates the video tracks contained in the files and / or segments. Includes geometry video, attribute video, occupancy map, additional data and / or mesh data. Decapsulates the video stream containing the

[0387] Metadata Track Decapsulation ation) is the metadata and / or additional data associated with point cloud data. Decapsulates bitstreams containing

[0388] Image Decapsulation is a method for decapsulating geometry images. Images including image, feature image, occupancy map, additional data and / or mesh data Decapsulate the message.

[0389] The decapsulation or decapsulation unit according to the embodiment is a G-PCC / V-PCC bitstream. Parsing (decomposing) the stream into parts based on one or more tracks in the file The signaling information for this is also decapsulated. -The atlas stream included on the PCC bitstream is used as a track in the file. It decapsulates the packet based on the packet's address and parses the associated signaling information. The SEI messages present on the PCC / V-PCC bitstream are stored in the file. The signaling information can be decapsulated based on the packet and related signaling information can be obtained. .

[0390] The video decoding unit 22001 is a Video restoration, characteristic video restoration, occupancy map restoration, additional data restoration and / or mesh data restoration The video decoding unit performs data reconstruction. Depending on the process of adding, the geometry video, the attribute video, the additional data and / or Decode the mesh data.

[0391] The image decoding unit (Image Decoding) 22002 receives the geometry image restoration, feature image restoration, occupancy map restoration, additional data restoration and / or mesh data restoration The image decoding unit performs image encoding of the point cloud transmitting device according to the embodiment. Geometry images, characteristic images, additional data and / or decodes the mesh data.

[0392] As described above, the video decoding unit and the image decoding unit according to the embodiment are configured to decode one video / image. This may be processed by the decoder or may be done in a separate path as shown.

[0393] The video decoder and / or the image decoder decodes the video data and / or the image data. Generate associated metadata.

[0394] Point cloud video encoder and / or point cloud image encoder according to embodiments The encoder decodes the G-PCC / V-PCC bitstream according to the embodiment.

[0395] Point Cloud Processing unit)22003 is Geometry Reconstruction tion) and / or Attribute Reconstruction )

[0396] The geometry reconstruction is performed from the decoded video data and / or the decoded image data. Based on the occupancy map, additional data and / or mesh data, the geometry video and and / or restore the geometry image.

[0397] The feature reconstruction is performed by extracting the occupancy matrix from the decoded feature video and / or the decoded feature image. Based on the mesh data, additional data and / or mesh data, In some embodiments, the attribute is a texture. , characteristic means plural characteristic information. When there are plural characteristics, the point classification according to the embodiment The cloud processing section performs multiple feature reconstructions.

[0398] The point cloud processing unit may include a video decoder, an image decoder and / or a file decoder. Receives metadata from the file / segment decapsulator and points based on the metadata. It can handle the cloud.

[0399] Point Cloud Rendering nit) renders the reconstructed point cloud. The decoding unit may include a video decoder, an image decoder and / or a file / segment decapsulator. Receives metadata from the rendering unit and renders the point cloud based on the metadata. do.

[0400] Display displays the rendered results on a real display device. do.

[0401] As shown in FIGS. 15 to 19, the method / apparatus according to the embodiment uses point cloud data. After encoding / decoding the data, the bitstream containing the point cloud data is saved to a file. and / or encapsulating and / or deencapsulating into segment format.

[0402] For example, the point cloud data transmission device according to the embodiment may transmit point cloud data Encapsulates based on a file, where the file contains information about the point cloud. A V-PCC track containing parameters, a geometry track containing geometry, and a The attribute track includes a characteristic track containing the attribute information and an occupancy track containing the occupancy map.

[0403] Furthermore, the point cloud data receiving device according to the embodiment receives the point cloud data file. The file contains parameters about the point cloud. V-PCC track containing meters, geometry track containing geometry, characteristic track containing characteristics It includes an occupancy track that includes a quality track and an occupancy map.

[0404] The above-mentioned operation is performed by the file / segment encapsulation unit in FIG. 20 and the file / segment encapsulation unit in FIG. 21. The file / segment encapsulation unit in Figure 22 and the file / segment decapsulation unit in Figure 23 are used. can be.

[0405] FIG. 23 shows a structure that can be linked with a method / apparatus for transmitting and receiving point cloud data according to an embodiment. An example is shown below.

[0406] In the example structure, a server 2360, a robot 2310, an autonomous vehicle 2320 , XR device 2330, smartphone 2340, home appliance 2350 and / or HMD 2370 One of the robots 231 connects to the cloud network 2310. 0, autonomous vehicles 2320, XR devices 2330, smartphones 2340 or home appliances 235 0 and the like are referred to as devices. The XR device 1730 is a device that processes point cloud compression data according to an embodiment. It may correspond to or interface with a PCC device.

[0407] Cloud Network 2300 is part of the cloud computing infrastructure It may refer to a network that exists within a cloud computing infrastructure. Here, the cloud network 2300 may be a 3G network, a 4G network, or an LTE (Long-Term Evolution) network. Long Term Evolution (5G) networks or 5G networks It may be configured as follows.

[0408] The AI ​​server 2360 is responsible for the robot 2310, the autonomous vehicle 2320, and the XR device 233. 2300, a smartphone 2340, a home appliance 2350, and / or an HMD 2370 The connected device 2310 is connected to one or more other devices via a cloud network 2300. Assists in at least part of the processing of ~2370.

[0409] The HMD (Head-Mount Display) 23700 is an XR device according to the embodiment. 1 shows one type of device and / or PCC device that can be implemented. The device consists of a communication unit, a control unit, a memory unit, and an I / O It includes a unit, a sensor unit, and a power supply unit.

[0410] Various embodiments of the devices 2310 to 2350 to which the above technology is applied will be described below. Here, the devices 2310 to 2350 shown in FIG. 23 are the points according to the above-mentioned embodiment. It can be linked / combined with cloud data transmission / reception devices.

[0411] <PCC + XR>

[0412] The XR / PCC device 2330 is applied with PCC and / or XR (AR+VR) technology, HMD (Head-Mounted Display), HUD (Head Mounted Display) -Up Display), TV, mobile phone, smartphone, computer, wear Wireless devices, home appliances, digital signs, vehicles, fixed robots, mobile robots, etc. This is embodied in

[0413] The XR / PCC device 2330 uses 3D points acquired by various sensors or from external devices. Analyzes cloud data or image data to obtain position data and XR client that obtains and outputs information about the surrounding space or real objects by generating characteristic data For example, the XR / PCC device 2330 can render and output the Outputting an XR object containing additional information about the recognized object corresponding to the recognized object. This can be done.

[0414] <PCC + XR + Mobile Phone>

[0415] The XR / PCC device 2330 is a device that applies PCC technology to a mobile phone 2340. It will be revealed.

[0416] Mobile phone 2340 decodes point cloud content based on PCC technology. , display.

[0417] <PCC + Autonomous Driving + XR>

[0418] The autonomous vehicle 2320 is a mobile robot, vehicle, and is based on PCC and XR technologies. , and is realized in unmanned aerial vehicles.

[0419] The autonomous vehicle 2320 to which XR / PCC technology is applied is equipped with a means for providing XR images. This refers to autonomous vehicles that are controlled and interacted with within XR images. In particular, the autonomous vehicle 2320 that is the target of control / interaction in the XR image may be It is separate from the XR device 2330 and can be linked to each other.

[0420] The autonomous vehicle 2320 equipped with a means for providing XR / PCC video includes a sensor including a camera. Sensor information is acquired from the camera, and XR / PCC video is generated based on the acquired sensor information. For example, an autonomous vehicle can output XR / PCC images using a HUD. , providing passengers with XR / PCC objects that correspond to real-world objects or objects on the screen.

[0421] In this case, if the XR / PCC object is output to the HUD, At least a part of the image is output so that it overlaps with the actual object that the passenger is looking at. On the other hand, XR / PCC objects are output to a display installed in an autonomous vehicle. When the XR / PCC object is overlapped with an object on the screen, at least a part of the XR / PCC object overlaps with the object on the screen. For example, an autonomous vehicle may detect the road, other vehicles, traffic lights, and other traffic Outputs XR / PCC objects corresponding to objects such as signs, motorcycles, pedestrians, buildings, etc. It is possible.

[0422] VR (Virtual Reality) technology and AR (Augmented Reality) technology d Reality) technology, MR (Mixed Reality) technology and / or PCC (Point Cloud Compression) technology can be applied to various devices It is Noh.

[0423] In other words, VR technology is a display technology that provides real objects and backgrounds only as CG images. On the other hand, AR technology is a technology that displays virtual CG images on top of real images. MR technology is similar to the AR technology mentioned above in that it mixes virtual objects into the real world. However, with AR technology, the distinction between real objects and virtual objects made up of CG images is clear. While virtual objects are used to complement real objects, MR technology uses virtual objects to It is different from AR technology in that the body and real objects are considered to have the same character. Specifically, for example, the application of the above-mentioned MR technology is a hologram service.

[0424] However, rather than clearly distinguishing between VR, AR, and MR technologies, the term XR (extended reality) is now used. Therefore, the embodiment of the present invention is applicable to VR, AR, and MR. , XR technology. Such technology is PCC, V-PCC, G-PCC Technology-based encoding / decoding is applied.

[0425] The PCC method / apparatus according to the embodiment can be applied to vehicles that provide autonomous driving services.

[0426] The autonomous vehicle that provides the autonomous driving service is connected to the PCC device via wired or wireless communication. .

[0427] The compressed point cloud data (PCC) transmitting and receiving device according to the embodiment is a device for transmitting and receiving data in a manner that is compatible with an autonomous vehicle. When wirelessly connected, AR / VR / PCC can be provided along with autonomous driving services. It can receive / process service-related content data and transmit it to the autonomous vehicle. In addition, if the point cloud data transmission and reception device is installed on an autonomous vehicle, The cloud transmission / reception device receives a user input signal from the user interface device. Receives and processes AR / VR / PCC service related content data and provides it to users A vehicle or user interface device according to an embodiment can receive a user input signal. The user input signal according to the embodiment can be a signal instructing an autonomous driving service. It may include.

[0428] The transmitting device according to the embodiment is a device for transmitting point cloud data, The receiving device is a device that receives point cloud data.

[0429] The method / apparatus according to the embodiment includes a point cloud data transmission / reception method / apparatus according to the embodiment, Point cloud encoder and decoder included in the transmitter and receiver, point cloud data A device, processor and / or a method for generating and parsing data for transmitting and receiving data Or the corresponding methods thereof.

[0430] The point cloud data transmission device according to the embodiment includes a point cloud data encoder and a transmitter for transmitting point cloud data. Point cloud data transmission The device can also be configured with formats for efficient transmission of point cloud data. It includes a point cloud data encapsulator that compresses point cloud data. The point cloud data including the coder and the encapsulator for transmission The above components are referred to simply as the method / apparatus according to the embodiment in this specification. do.

[0431] The point cloud data receiving device according to the embodiment includes a point cloud data decoder and and a receiver for receiving point cloud data. The device also provides a data structure formatted for efficient reception of point cloud data. It includes a decapsulator that parses point cloud data from Decoder to recover cloud data and decapsulate for receiving and parsing The cell generation unit and other components are called the point cloud data system. This is simply referred to herein as the method / apparatus according to the embodiment.

[0432] The V-PCC (Video-based point cloud compression) is V3C (Visual Volumetric Vid The term V-PCC in the example is the same as V3 It is often used interchangeably with C.

[0433] The method / apparatus according to the embodiment may include a file format for dynamic point cloud objects. and provides a signaling scheme for it (File encapsulation) on of Dynamic point cloud object).

[0434] FIG. 24 illustrates the structure of an encapsulated V-PCC data container according to an embodiment.

[0435] FIG. 25 illustrates an isolated V-PCC data container structure according to an embodiment.

[0436] The point cloud video encoder 10002 of the transmitting device 10000 of FIG. 1, FIG. 4 and The encoder of FIG. 15, the transmitter of FIG. 18, and the video / image encoder 2000 of FIG. 29 2, 20003, the processors of FIG. 21, the encoders 21000 to 21008, and FIG. The XR device 2330 or the like may include a bitstream containing point cloud data according to an embodiment. Create a stream.

[0437] The file / segment encapsulation unit 10003 in FIG. 1 and the file / segment in FIG. 20 encapsulation unit 20004, the file / segment encapsulation unit 21009 of FIG. 21, and The XR device in Figure 23 initializes the bitstream to the file structure in Figures 24 and 25. (formatting).

[0438] Similarly, the file / segment decapsulation unit 10007 of the receiving device 10005 in FIG. , the file / segment decapsulation units 20005, 21009 in Figs. 20 to 23, 22000 and the XR device 2330 in FIG. 23 receive and decapsulate the file. The bitstream is parsed using the point cloud video data shown in Figure 1. 16 and 17, the receiver of FIG. 19, and the decoders of FIGS. 20 to 23. 23 Video / Image Decoders 20006, 21007, 21008, 22001, 2 2002, and decoded by the XR device 2330 in FIG. 23 to generate point cloud data will be restored.

[0439] Figures 24 and 25 show point cloud data in ISOBMFF file format. This shows the data container structure.

[0440] Figures 24 and 25 show the container that transmits the point cloud based on multi-track. It is a structure.

[0441] The method / apparatus according to the embodiment may include point cloud data and point Send and receive additional data related to in-cloud data in a container file.

[0442] The first track 24000 is a characteristic track, and is similar to that shown in FIGS. The attribute data 24040 is encoded as follows:

[0443] The second track 24010 is an occupied track, and is shown in FIGS. The data includes geometry data 24050 encoded as follows:

[0444] The third track 24020 is a geometry track, and is used for the following purposes: How encoded occupancy data 24060 is included.

[0445] The fourth track 24030 is the v-pcc (v3c) track, which is a point cloud data The atlas bitstream 27070 includes data related to the atlas data.

[0446] Each track consists of a sample entry and a sample. A sample is a frame. To decode the Nth frame, the Nth frame is decoded by A corresponding sample or sample entry is required. A sample entry is a sample Contains information describing the

[0447] FIG. 25 is a detailed structural diagram of FIG.

[0448] v3c track 25000 corresponds to the fourth track 24030. v3c track 25 The data contained in 000 has a data container format called a box. c Track 25000 is related to V3C Component Tracks 25010 and 25030 Contains reference information for

[0449] The receiving method / apparatus according to the embodiment receives content including point cloud data as shown in FIG. Receive the file and parse the V3C track. Decode and recover occupancy, geometry, and attribute data based on included reference information do.

[0450] The occupied track 25010 corresponds to the second track 24010 and contains occupied data. The metrology track 25020 corresponds to the third track 24020 and contains geometry data. The attribute track 25030 corresponds to the first track 24000 and contains attribute data.

[0451] The following explains the syntax of the data structure contained in the files in Figures 24 and 25. We will explain in detail.

[0452] FIG. 26 shows the structure of a bitstream containing point cloud data according to an embodiment.

[0453] FIG. 26 shows the points according to the embodiment that are encoded or decoded as described in FIG. 24 and FIG. It is a bitstream structure containing the content cloud data.

[0454] The method / apparatus according to the embodiment generates a bitstream for a dynamic point cloud object. and propose a file format for the bitstream and signaling for it. Provides guidance solutions.

[0455] The method / apparatus according to the embodiment converts a V-PCC (=V3C) bitstream into a file Point cloud containers that can be efficiently stored in racks and provide signaling for them. The present invention is directed to a transmitter, receiver and / or processor for providing content services.

[0456] The method / apparatus according to the embodiment may include a V-PCC bitstream containing point cloud data. The data format for storing the program is provided. receives point cloud data and provides an efficient data access to the point cloud data. Therefore, it is possible to efficiently store and signal the approach point cloud data. Based on the file storage technique containing the point cloud data, the transmitter and / or receiver It is possible to provide content services.

[0457] The method / apparatus according to the embodiment is for generating point cloud bitstream (V-PCC bitstream) Streams) are stored efficiently within the tracks of a file. Nulling information is generated and stored in the file. File-implemented storage techniques to support efficient access to the data stream. and then (or further modify / combine) the V-PCC bitstream into one or more files. A technique for splitting and storing data across multiple tracks is provided.

[0458] The definitions of terms used in this specification are as follows:

[0459] VPS: V-PCC parameter set. AD: Atlas data. OVD: Occupancy video data. GVD: Geometry video data. AVD: Attribute video data deo data). ACL: Atlas Coding Layer AAPS: Atlas Adaptation Parameter Set ASPS: Atlas Sequence Parameter Set. Atlas sequence parameter set. A syntax structure containing a syntax element may consist of zero or more whole coded This applies to atlas sequences (CASs) and is the syntax within each tile group header. It is determined by the content of the ASPS syntax element referenced by the element.

[0460] AFPS: Atlas frame parameter set A syntax structure containing a syntax element can consist of zero or more whole This applies to coded atlas frames and is specified in the tile group header syntax. The value is determined by the content of the .

[0461] SEI: Supplemental enhancement information ormation)

[0462] Atlas: A collection of 2D bounding boxes. For example, a rectangle Supports 3D bounding boxes in 3D space projected onto the frame The atlas is a point cloud representation of a patch projected onto a rectangular frame. The subset is shown.

[0463] Atlas sub-bitstream: Atlas Sub-bitstream extracted from the V-PCC bitstream containing the NAL bitstream part It is a fast stream.

[0464] V-PCC content: Based on V-PCC (V3C) It is a point cloud that is encoded using

[0465] V-PCC track: The end of the V-PCC bitstream. It is a volumetric visual track that carries a 3D bitstream.

[0466] V-PCC component track : V-PCC bitstream occupancy map, geometry, and characteristics of component video A video track carrying 2D video encoded data for a bit stream. .

[0467] Dynamic point cloud object partial access An embodiment for supporting ss will be described. The embodiment is at the file system level (file At the system level, each spatial region Atlas style group (atlas) related to some data of V-PCC objects included in The example also includes information about each atla tile group (atl as tile group) contains labels and / or patches Includes an expanded signaling scheme for (catch) information.

[0468] Referring to FIG. 26, points included in data transmitted and received by the method / apparatus according to the embodiment 1 shows the structure of a cloud bitstream.

[0469] The point cloud data compression and restoration technique according to the embodiment is 1 illustrates volumetric encoding and decoding of data information.

[0470] Coded point cloud sequence Point cloud bitstream (V-PC) including d sequence,CPCS Also known as V3C bitstream or V3C bitstream, 26000 It consists of a pull stream V-PCC unit 26010. Sample stream VP The CC unit 26010 stores V-PCC parameter set (VPS) data 26020. Atlas bitstream 26030, 2D video coding occupancy map bitstream 2 6040, 2D video coding geometry bitstream 26050, and zero and one The above 2D video coding characteristic bitstream 26060 is transmitted.

[0471] Point cloud bitstream 24000 is sample stream VPCC header 2 Including 6070.

[0472] SSVH Unit Size Precision (ssvh_unit_size_precision_ bytes_minus1): Adding 1 to this value will increase the total sample stream VP SSVU VPCC unit size in CC unit (ssvu_vpcc_unit_ Indicates the precision in bytes of the ssvh_unit_size_preci sion_bytes_minus1 has a range of 0 to 7.

[0473] The syntax 24080 of the sample stream V-PCC unit 24010 is as follows: Each sample stream V-PCC unit has VPS, AD, OVD, GV Each sample stream V contains one of the types of V-PCC units, D and AVD. The content of the PCC unit is contained within the sample stream V-PCC unit. It is associated with the same access unit as the V-PCC unit.

[0474] SSVU VPCC unit size (ssvu_vpcc_unit_size): Indicates the size in bytes of the subsequent V-PCC unit. The number of bits used to indicate ssvu_vpcc_unit_size is (ssv h_unit_size_precision_bytes_minus1+1)*8 be.

[0475] The method / apparatus according to the embodiment may be implemented using the bitstream of FIG. 26 containing encoded point cloud data. 24 and 25 are received by the encapsulation units 20004, 21009, etc. Generate a file like 25.

[0476] The method / apparatus according to the embodiment receives the files shown in FIGS. 24 and 25 and decapsulates them. The point cloud data is decoded by a decoding unit 22000 or the like.

[0477] VPS26020 and / or AD26030 are in the 4th track (V3C track) 240 It is encapsulated in 30.

[0478] The OVD 26040 is encapsulated in the second track (occupied track) 24010 .

[0479] GVD26050 is encapsulated in the third track (geometry track) 24020 do.

[0480] The AVD 26060 is encapsulated in the first track (attribute track) 24000.

[0481] FIG. 27 shows the structure of a sample stream V-PCC unit according to an embodiment.

[0482] The bitstream 27000 in FIG. 27 corresponds to the bitstream 26000 in FIG. do.

[0483] Included in bitstream 27000 regarding point cloud data according to an embodiment The sample stream V-PCC unit is V-PCC unit size 27010 and V -Includes PCC unit 27020.

[0484] The definitions of each abbreviation are as follows: VPS (V-PCC parameter set ), AD(atlas data), OVD(occupancy video dat) a), GVD (geometry video data), AVD (attribute e video data)

[0485] Each V-PCC unit 27020 has a V-PCC unit header 27030 and a V- Contains the PCC unit payload 27040. The V-PCC unit header 27030 is V -Describes the PCC unit type. Attribute Video Data V-PCC Unit Header Types and their indexes, multiple instances of the same attribute type supported, etc. explain.

[0486] Occupancy, Geometry, and Attributes Video Data Unit Payload 27050, 27060, 2 7070 corresponds to a video data unit, e.g., occupied video data, geometry Video data, characteristic video data 27050, 27060, 27070 are HEVC N AL unit. Such video data is decoded by a video decoder according to the embodiment. do.

[0487] FIG. 28 shows a V-PCC unit and a V-PCC unit header according to an embodiment.

[0488] FIG. 28 shows the V-PCC unit 27020 and V-PCC unit header described in FIG. The syntax of Da27030 is shown below.

[0489] A V-PCC bitstream according to an embodiment includes a series of V-PCC sequences.

[0490] V-PCC unit with the same vuh_unit_type value as VPCC_VPS The V-PCC unit type is expected to be the first V-PCC unit type in the V-PCC sequence. All other V-PCC unit types are allowed within their coding order without additional restrictions. According to this unit type, V conveys occupancy video, attribute video or geometry video. - The V-PCC unit payload of a PCC unit may consist of one or more NAL units. A V-PCC bitstream contains a ies of V-PCC sequences. A vpcc unit type with a value of vuh_unit_type equal to VPCC_VPS is expected to be the first VP CC unit type in a V-PCC sequence. her V-PCC unit types follow this unit ty pe without any additional restrictions i n their coding order. A V-PCC unit paylo ad of V-PCC unit carrying Occupancy vide o, attribute video, or geometry video is composed of one or more NAL units).

[0491] The VPCC unit includes a header and a payload.

[0492] The VPCC unit header contains the following information based on the VUH unit type:

[0493] VUH unit type indicates the type of V-PCC unit 27020 as follows:

[0494] [Table 2]

[0495] VUH unit type (vuh_unit_type) is occupied video data (VPCC _AVD), Geometry Video Data (VPCC_GVD), Occupancy Video Data (VPC When VUH VPCC parameters are displayed, Data set ID (vuh_vpcc_parameter_set_id) and VUH address The truss ID (vuh_atlas_id) is conveyed in the unit header. V-PCC The parameter set ID and atlas ID associated with the unit can be transmitted.

[0496] If the unit type is Atlas Video Data, the unit header contains the attribute index attribute partition index (vuh_attribute_index) uh_attribute_partition_index), map index ( vuh_map_index), additional video flag (vuh_auxiliary_vi deo_flag).

[0497] If the unit type is geometry video data, the map index (vuh_m ap_index), additional video flag (vuh_auxiliary_video_f lag).

[0498] If the unit type is proprietary video data or atlas data, the unit header is Contains additional reserved bits.

[0499] VUH VPCC parameter set ID (vuh_vpcc_parameter_s et_id): vps_vpcc_parameter for active V-PCC VPS Indicates the value of the VPCC parameter in the header of the current V-PCC unit. The VPS parameter set ID can be identified by the parameter set ID, and the V-PCC unit and V -The relationship between PCC parameter sets can be signaled.

[0500] VUH Atlas ID (vuh_atlas_id): The ID currently corresponding to the V-PCC unit. Indicates the index of the atlas currently being used. This allows you to grasp the atlas index and know the atlas corresponding to the V-PCC unit. It can be done.

[0501] VUH attribute index (vuh_attribute_index): Attribute video data Indicates the index of the characteristic data transmitted from the data unit.

[0502] VUH attribute partition index (vuh_attribute_partit ion_index): Attribute dimension group conveyed from the attribute video data unit Indicates the loop index.

[0503] VUH map index (vuh_map_index): If this value exists, Indicates the map index of the current geometry or attribute stream.

[0504] VUH auxiliary video flag (vuh_auxiliary_video_flag): If the value of is 1, the associated geometry or attribute video data unit is only RAW and / or indicates that it is an EOM coded point video. If this value is 0, The associated geometry or characteristic video data units are RAW and / or EOM coded. indicates that the `.

[0505] FIG. 29 illustrates the payload of a V-PCC unit according to an embodiment.

[0506] Figure 27 shows the syntax of the payload 25040 of the V-PCC unit.

[0507] V-PCC unit type (vuh_unit_type) is V-PCC parameter set The payload of the V-PCC unit is a parameter set. Contains the parameter (vpcc_parameter_set()).

[0508] V-PCC unit type (vuh_unit_type) is V-PCC atlas data If the V-PCC unit is an Atlas sub-bit, the payload of the V-PCC unit is an Atlas sub-bit. Contains the stream (atlas_sub_bitstream( )).

[0509] V-PCC unit type (vuh_unit_type) is V-PCC exclusive video data Video data (VPCC_OVD), Geometry video data (VPCC_GVD) or Attribute video data If the V-PCC unit is video data (VPCC_AVD), the payload of the V-PCC unit is video bits. This includes the sub-bitstream (video_sub_bitstream()).

[0510] FIG. 30 shows a parameter set (V-PCC parameter set) according to an embodiment. Shows.

[0511] FIG. 30 shows a unit 27 of a bitstream according to an embodiment of the present invention, as shown in FIGS. 27 to 29. If the payload 27040 of the 020 contains a parameter set, Shows tax.

[0512] Profile Tier Level (profile_tier_level()): Bits Indicates limitations on the capabilities required to decode the bitstream. Profiles, tiers, and levels indicate interoperability between individual decoder implementations. Used to indicate points (specifies restrictions o n the bitstreams and hence limits on the capabilities needed to decode the bitst reams. Profiles, tiers, and levels may a lso be used to indicate interoperability points between individual decoder implementation mentations).

[0513] Parameter set ID (vps_vpcc_parameter_set_id): Other Provides an identifier for the V-PCC VPS for reference by syntax elements in provides an identifier for the V-PCC VPS for reference by other syntax elements) .

[0514] Atlas Count (vps_atlas_count_minus1): Set this value to 1. When added, it indicates the total number of atlases currently supported in the bitstream (plus 1 indicates the total number of supported atlases in the current bitstream).

[0515] Depending on the number of atlases, the following parameters are further included in the parameter set:

[0516] Frame width (vps_frame_width[j]): The width of the atom with index J. Indicates the V-PCC frame width in integer luma samples for the class. The nominal widths associated with all V-PCC components for atlases with AX J are (indicates the V-PCC frame width in ter ms of integer luma samples for the atlas with index j. This frame width is the n ominal width that is associated with all V-PCC components for the atlas with ind ex j).

[0517] Frame height (vps_frame_height[j]): has index J Indicates the V-PCC frame height in integer luma samples for the atlas. The length is related to all V-PCC components for the atlas with index J. indicates the nominal height of the V-PCC frame. t in terms of integer luma samples for t he atlas with index j. This frame height is the nominal height that is associated d with all V-PCC components for the atla s with index j).

[0518] MAP Count (vps_map_count_minus1[j]): Set this value to 1. In addition, the index J is used to encode geometry and attribute data. plus 1 indicates the number of maps f maps used for encoding the geometry an d attribute data for the atlas with inde xj).

[0519] MAP count (vps_map_count_minus1[j]) is greater than 0 and the following parameters are further included in the parameter set:

[0520] Depending on the value of the MAP count (vps_map_count_minus1[j]), The following parameters are further included in the parameter set:

[0521] Multi-map stream existence flag (vps_multiple_map_strea ms_present_flag[j]): If this value is 0, then for index J All geometry or attribute maps of a single geometry or attribute video stream are If this value is 1, it indicates that the atlas with index J exists in All geometry or feature maps for the indicates that all (equal to 0) geometry or attribute maps for the atlas with index j are placed in a single geo metric or attribute video stream, respect ively.vps_multiple_map_streams_present_ flag[j] equal to 1 indicates that all ge ometry or attribute maps for the atlas w ith index j are placed in separate video streams).

[0522] Multi-map stream existence flag (vps_multiple_map_strea ms_present_flag[j]) indicates 1, vps_map_absolu te_coding_enabled_flag[j][i] is added to the parameter set otherwise, vps_map_absolute_coding_ena bled_flag[j][i] has a value of 1.

[0523] Map absolute coding enable flag (vps_map_absolute_codin g_enabled_flag[j][i]): If this value is 1, then index J is What kind of map is the geometry map with index I for the atlas? If this value is 1, the index J is valid. Before coding, a geometry map with index I for the atlas to be coded is It indicates that the map is predicted to be first before the others (equal to 1 indicates that the geometry map wi th index i for the atlas with index j is coded without any form of map prediction n. vps_map_absolute_coding_enabled_flag[ j][i]equal to 0 indicates that the geome try map with index i for the atlas with index j is first predicted from another, earlier coded map, prior to coding).

[0524] Map absolute coding enable flag (vps_map_absolute_codin g_enabled_flag[j][0]) is 1, the j with index 0 Indicates that the geometry map is coded without map prediction.

[0525] Map absolute coding enable flag (vps_map_absolute_codin g_enabled_flag[j][i]) is 0 and I is greater than 0, then vps _map_predictor_index_diff[j][i] is the parameter set otherwise, vps_map_predictor_index _diff[j][i] will be 0.

[0526] Map predictor index difference (vps_map_predictor_index_d iff[j][i]): This value is the map absolute coding valid flag (vps_map_ absolute_coding_enabled_flag[j][i]) is 0 and a geometry map with index I for an atlas with index J is used to compute the predictors of predictor of the geometry map with inde xi for the atlas with index j when vps_ map_absolute_coding_enabled_flag[j][i] i s equal to 0).

[0527] Additional video present flag (vps_auxiliary_video_present_ flag[j]): If this value is 1, the index data is Additional information for the atlas with J is provided as individual video streams called additional video streams. If this value is 0, the address with index J is stored in the Indicates that additional information for the truss is not stored in a separate video stream (equal to 1 indicates that auxiliary information on for the atlas with index j, ie RAW or EOM patch data, may be stored in a se parate video stream, referred to as the a uxiliary video stream. vps_auxiliary_vid eo_present_flag[j] equal to 0 indicates that auxiliary information for the atlas with index j is not be stored in a sepa rate video stream).

[0528] Occupancy_information(): Information related to the occupancy video. Includes occupancy video related information information).

[0529] Geometry information (geometry_information()): Geometry video Includes information related to geometry, video rela ted information).

[0530] attribute_information(): Attributes related to the video Includes attribute video related information).

[0531] Extension Present Flag (vps_extension_present_flag): Syntax The vps element extension length (vps_extension_length) is set as a parameter (vpcc_parameter_set) indicates that it exists in the syntax structure. If the value of vps_extension_leng is 0, the syntax element extension length (vps_extension_leng th) does not exist (equal to 1 specifies that the syntax element vps_extension_length is present in vpcc_parameter_set syntax structure. vps_extension_present_flag e qual to 0 specifies that syntax element vps_extension_length is not present).

[0532] Extension Length (vps_extension_length_minus1): This value plus 1 Adding this syntax element will add extension data (vps_extension on_data_byte) indicates the number of elements (plus 1 specifies th e number of vps_extension_data_byte elem ents that follow this syntax element).

[0533] The extension length (vps_extension_length_minus1) determines the Extension data is further included in the parameter set.

[0534] Extension Data (vps_extension_data_byte): included by the extension It may have any value.

[0535] FIG. 31 shows a tile according to an embodiment.

[0536] 31 shows the encoder 10002 of FIG. 1, the encoder of FIG. 4, the encoder of FIG. 15, and the encoder of FIG. 1 8, and an address including tiles encoded by the system of FIG. 20, FIG. 21, etc. 1, the decoders of FIGS. 16 and 17, and FIG. 19. and an atlas frame including tiles decoded by a receiving device such as the system of FIG. Shows.

[0537] An atlas frame is divided into one or more tile rows and one or more tile columns. A tile is a rectangular area in the atlas frame. A tile group is a group of tiles in the atlas frame. A tile and a tile group are not distinguished from each other. A group can correspond to one tile. Only rectangular tile groups are supported. In this mode, a tile group (or tile) assembles a rectangular region of the atlas frame. Figure 31 shows a tile or tile group of an atlas frame according to an embodiment. The loop division is shown. The atlas frame consists of 24 tiles (6 tile rows and 4 tile columns). A tile group (row) and nine rectangular tile groups are shown in Figure 31. There is no distinction between groups and tiles, and tile group is used as the term corresponding to tiles.

[0538] That is, in some embodiments, tile group 31000 corresponds to tile 31010, and tile Also, the tile 31010 corresponds to the tile partition, and the tile The names of signaling information also reflect this mutually complementary correspondence. The name can be changed depending on the department.

[0539] FIG. 32 shows the structure of an atlas bitstream according to an embodiment.

[0540] FIG. 32 shows the payload 27 of the unit 27020 of the bitstream 27000 of FIG. 040 shows an example of conveying an atlas sub-bitstream 32000.

[0541] V-PCC unit pair of V-PCC units carrying atlas sub-bitstreams The image contains one or more sample stream NAL units 32010.

[0542] The atlas sub-bitstream 32000 according to the embodiment is a sample stream NAL It includes a header 32020 and a sample stream NAL unit 32010.

[0543] The sample stream NAL header 32020 contains the unit size precision bytes (ssnh _unit_size_precision_bytes_minus1). Adding 1 to the value will increase the NAL unit size in all sample stream NAL units. (ssnu_nal_unit_size) Indicates the precision of the element in bytes. unit_size_precision_bytes_minus1 ranges from 0 to 7 has (plus 1 specifies the precision, in b ytes, of the ssnu_nal_unit_size element in all sample stream NAL units. ssnh_uni t_size_precision_bytes_minus1 is in the range of 0 to 7).

[0544] Sample stream NAL unit 32010 is the NAL unit size (ssnu_ nal_unit_size).

[0545] The NAL unit size (ssnu_nal_unit_size) is the size of the subsequence Indicates the size of the NAL unit in bytes. The number of bits used to represent the n_bytes_minus1+1)*8 (specifies the size e, in bytes, of the subsequent NAL_unit. The number of bits used to represent ss nu_nal_unit_size is equal to (ssnh_unit_ size_precision_bytes_minus1+1)*8).

[0546] FIG. 33 shows a NAL unit according to an embodiment.

[0547] FIG. 33 shows the syntax of the NAL unit in FIG.

[0548] A NAL unit is a NAL unit header (nal_unit_header( )), Includes the number of bytes (NumBytesInRbsp).

[0549] Number of bytes (NumBytesInRbsp): corresponds to the payload of the NAL unit It is a byte and is initially set to 0.

[0550] The NAL unit header contains the NAL Forbidden Zero bit (nal_forbidden n_zero_bit), NAL unit type (nal_unit_type), NA NAL Layer ID (nal_layer_id), NAL Temporary ID (nal_tempora l_id_plus1) etc.

[0551] NAL Forbidden Zero Bit (nal_forbidden_zero_bit): This field is used for error detection in NAL units and is always 0.

[0552] NAL unit type (nal_unit_type): As shown in Figure 32, Indicates the type of RBSP data structure contained within the block.

[0553] NAL layer ID (nal_layer_id): The layer to which the ACL NAL unit belongs. Indicates the layer identifier or the layer identifier to which the NON-ACL NAL unit applies ( specifies the identifier of the layer to which an ACL NAL unit belongs or the id entifier of a layer to which a non-ACL N AL unit applies).

[0554] NAL Temporal ID (nal_temporal_id_plus1): Subtract 1 from this value indicates a temporary identifier of the NAL unit (minus 1 specifies a temporal identifier for the NAL unit).

[0555] FIG. 34 shows NAL unit types according to an embodiment.

[0556] Figure 34 shows the NAL unit number of the sample stream NAL unit 32010 in Figure 32. Indicates the type of NAL unit included in the NAL unit header.

[0557] NAL Trail (NAL_TRAIL): Non-TSA, Non-STSA Trail The coded tile-groups of a large atlas frame are included in the NAL unit. The RBSP syntax structure of a NAL unit is the atl tile group layer (atl as_tile_group_layer_rbsp()) or Atlas Tile Layer (atlas_tile_layer_rbsp()). NAL unit type A tile group corresponds to a tile, depending on the embodiment.

[0558] NAL TSA: The coded tile group of the TSA atlas frame is NA. The RBSP syntax structure of the NAL unit is Loop layer (atlas_tile_group_layer_rbsp( )) or Atlas tile layer (atlas_tile_layer_rbsp()). The type class of the NAL unit is ACL.

[0559] NAL_STSA: Coded tile group of STSA atlas frame The RBSP syntax structure of the NAL unit is Tile Group Layer (atlas_tile_group_layer_rbsp( )) or atlas tile layer (atlas_tile_layer_rbsp()) The type class of the NAL unit is ACL.

[0560] NAL_RADL: RADL atlas frame coded tile group The RBSP syntax structure of the NAL unit is Tile Group Layer (atlas_tile_group_layer_rbsp( )) or atlas tile layer (atlas_tile_layer_rbsp()) The type class of the NAL unit is ACL.

[0561] NAL_RASL: The coded tile group of the RASL atlas frame The RBSP syntax structure of the NAL unit is Tile Group Layer (atlas_tile_group_layer_rbsp( )) or atlas tile layer (atlas_tile_layer_rbsp()) The type class of the NAL unit is ACL.

[0562] NAL_SKIP: Skip the coded tile group of the atlas frame The RBSP syntax structure of the NAL unit is Tile Group Layer (atlas_tile_group_layer_rbsp( )) or atlas tile layer (atlas_tile_layer_rbsp()) The type class of the NAL unit is ACL.

[0563] NAL_RSV_ACL_6 to NAL_RSV_ACL_9: Reserved IRAP ACL NAL unit type is included in the NAL unit. NAL unit type The class is an ACL.

[0564] NAL_BLA_W_LP, NAL_BLA_W_RADL, NAL_BLA_N_L P: The coded tile group of the BLA atlas frame is included in the NAL unit. The RBSP syntax structure of a NAL unit is atlas_tile_group_layer_rbsp( )) or Atlas Tile Layer (atlas_tile_layer_rbsp()). NAL unit The type class of is ACL.

[0565] NAL_GBLA_W_LP, NAL_GBLA_W_RADL, NAL_GBLA_ N_LP: The coded tile group of a GBLA atlas frame is a NAL unit. The RBSP syntax structure of a NAL unit is contained in the atlas style group. Layer(atlas_tile_group_layer_rbsp( )) or Atlas Style layer (atlas_tile_layer_rbsp()). NAL The type class of the unit is ACL.

[0566] NAL_IDR_W_RADL, NAL_IDR_N_LP: IDR atlas frame The coded tile-groups are contained in a NAL unit. The BSP syntax structure is the atlas_tile_gr oup_layer_rbsp( )) or atlas_til e_layer_rbsp()). The type class of the NAL unit is ACL. do.

[0567] NAL_GIDR_W_RADL, NAL_GIDR_N_LP: GIDR atlas A coded tile-group of frames is contained in a NAL unit. The RBSP syntax structure of the atlas_tile group layer is _group_layer_rbsp( )) or atlas style layer (atlas_ tile_layer_rbsp( )). The NAL unit type class is AC It's L.

[0568] NAL_CRA: The coded tile group of the CRA atlas frame is NA The RBSP syntax structure of the NAL unit is Loop layer (atlas_tile_group_layer_rbsp( )) or Atlas tile layer (atlas_tile_layer_rbsp()). The type class of the NAL unit is ACL.

[0569] NAL_GCRA: The coded tile group of a GCRA atlas frame The RBSP syntax structure of the NAL unit is Tile Group Layer (atlas_tile_group_layer_rbsp( )) or atlas tile layer (atlas_tile_layer_rbsp()) The type class of the NAL unit is ACL.

[0570] NAL_IRAP_ACL_22, NAL_IRAP_ACL_23: Reserved IR AP ACL NAL unit type is included in the NAL unit. The type class is ACL.

[0571] NAL_RSV_ACL_24 to NAL_RSV_ACL_31: Reserved The n-IRAP ACL NAL unit type is included in the NAL unit. The type class of the bit is ACL.

[0572] NAL_ASPS: Atlas sequence parameter set is contained in the NAL unit The RBSP syntax structure of the NAL unit is the Atlas Sequence Parameter Set ( atlas_sequence_parameter_set_rbsp( )) The type class of the NAL unit is non-ACL.

[0573] NAL_AFPS: The atlas frame parameter set is contained in the NAL unit. The RBSP syntax structure of a NAL unit is the Atlas Frame Parameter Set (at las_frame_parameter_set_rbsp( )). NA The type class of the L unit is non-ACL.

[0574] NAL_AUD: Access unit delimiter The RBSP syntax structure of a NAL unit is Access Unit Delimiter (access_unit_delimiter_rbsp ( )). The type class of the NAL unit is non-ACL.

[0575] NAL_VPCC_AUD: V-PCC access unit delimiter is a NAL unit The RBSP syntax structure of a NAL unit is contained in the access unit delimiter. (access_unit_delimiter_rbsp()). The type class of the bit is non-ACL.

[0576] NAL_EOS: The NAL unit type is end of sequence. The RBSP syntax structure of the unit is the end of sequence (end_of_seq _rbsp( )). The type class of the NAL unit is non-ACL.

[0577] NAL_EOB: The NAL unit type is end of bitstream. The RBSP syntax structure of the AL unit is the End of Atlas sub-bitstream. (end_of_atlas_sub_bitstream_rbsp()) The type class of the NAL unit is non-ACL.

[0578] NAL_FD Filler: NAL unit type is filler data (filler r_data_rbsp()). The NAL unit type class is non-AC It's L.

[0579] NAL_PREFIX_NSEI, NAL_SUFFIX_NSEI: NAL unit Type Non-essential supplemental RBS of NAL unit The syntax structure of the NAL unit is SEI(sei_rbsp()). The class is non-ACL.

[0580] NAL_PREFIX_ESEI, NAL_SUFFIX_ESEI:NAL unit Type: Essential supplemental enhancement information ncement information). RBSP syntax of NAL units The class structure is SEI(sei_rbsp()). The type class of the NAL unit is It is non-ACL.

[0581] NAL_AAPS: The NAL unit type is the Atlas Adaptation Parameter Set (Atla s adaptation parameter set). The BSP syntax structure is the atlas adaptation parameter set (atlas_adaptat ion_parameter_set_rbsp( )). NAL unit type The class is non-ACL.

[0582] NAL_RSV_NACL_44 or NAL_RSV_NACL_47: NAL unit NAL unit type is a reserved non-ACL NAL unit type. The type class is non-ACL.

[0583] NAL_UNSPEC_48 or NAL_UNSPEC_63: NAL unit type The type of the NAL unit is an unspecified non-ACL NAL unit type. The last is non-ACL.

[0584] FIG. 35 shows an atlas sequence parameter set according to an embodiment. nce parameter set).

[0585] Figure 35 shows the NAL unit type when the NAL unit type is an atlas sequence parameter. The syntax of the RBSP data structure included in the unit is shown below.

[0586] Each sample stream NAL unit contains an atlas parameter set, e.g., AS PS, AAPS, AFPS, one or more Atlas Style Group Information and SEI Contains any of the following.

[0587] ASPS is referenced as a syntax element within each tile group (tile) header. Zero or more total entries determined by the contents of syntax elements in the ASPS Syntax elements that apply to body-coded atlas sequences (CASs) include.

[0588] ASPS Atlas Sequence Parameter Set ID (asps_atlas_sequ ence_parameter_set_id): Used for reference by other syntax elements. This provides an identifier for the atlas sequence parameter set for the

[0589] ASPS frame width (asps_frame_width): The width of the sample corresponds to the luma samples of a component of the atlas frame in terms of an integer number of samples. indicates the atlas frame width in terms of integer number of samples, wher ea sample corresponds to a luma sample of a video component).

[0590] ASPS frame height (asps_frame_height): The height at which the sample is The atlas frame is in terms of an integer number of samples, corresponding to the luma samples of the component. indicates the atlas frame height in terms of integer number of samples, where a sample corresponds to a luma sam ple of a video component).

[0591] ASPS log patch packing block size (asps_log2_patch_p acking_block_size): for the horizontal and vertical position of the patch within the atlas. Indicates the value of the variable PatchPackingBlockSize to be used (specif ies the value of the variable PatchPacki ngBlockSize, that is used for the horizo tal and vertical placement of the patch es within the atlas).

[0592] ASPS log max atlas frame order count lsb(asps_log2 _max_atlas_frame_order_cnt_lsb_minus4): The variable MaxAtl used in the decoding process for truss frame order counting specifies the value of asFrmOrderCntLsb ue of the variable MaxAtlasFrmOrderCntLs b that is used in the decoding process f or the atlas frame order count).

[0593] ASPS Max Decode Atlas Frame Buffering (asps_max_dec_ atlas_frame_buffering_minus1): Adding 1 to this value , the maximum value of the decoded atlas frame for the atlas frame buffer storage is Indicates the requested size (plus 1 specifies the maximum required size of the decoded atlas fram e buffer for the CAS in units of atlas f rame storage buffers).

[0594] ASPS Long-Term Reference Atlas Frame Flag (asps_long_term_ref_ atlas_frames_flag): If this value is 0, the coding in CAS The long-term reference atlas frame used for inter-prediction of the atlas frame is If this value is 1, the long-term reference atlas frame is one or more in the CAS. Indicates that inter prediction is available for one or more coded atlas frames. (equal to 0 specifies that no long term reference atlas frame is used for inter prediction of any coded atlas frame in t he CAS. asps_long_term_ref_atlas_frames_ flag equal to 1 specifies that long term reference atlas frames may be used for inter prediction of one or more coded at las frames in the CAS).

[0595] Number of ASPS reference atlas frame lists (asps_num_ref_atlas_f rame_lists_in_asps): included in the atlas sequence parameter set Reference List Structure (ref_list_struct(rlsIdx)) Syntax Specifies the number of the ref_ list_struct(rlsIdx) syntax structures in cluded in the atlas sequence parameter s et).

[0596] asps_num_ref_atlas_frame_lists_in_asps If greater than 0, atgh_ref_atlas_frame_list_sps_fl ag contains the atlas tile group (tile) header.

[0597] asps_num_ref_atlas_frame_lists_in_asps If it is greater than 1, ATGH_ref_atlas_frame_list_idx is Included in the tile group (tile) header.

[0598] atgh_additiona for each NumLtrAtlasFrmEntries value l_afoc_lsb_present_flag[j] is the atlas style group ( Included in the file header.

[0599] Number of ASPS reference atlas frame lists (asps_num_ref_atlas_f rame_lists_in_asps) only reference list structure (ref_list_st ruct(i)) is included in the atlas sequence parameter set.

[0600] ASPS Eight Orientation Flag (asps_use_eight_orie ntations_flag): If this value is 0, the frame with index I The patch orientation index (pdu_ orientation_index[i][j]) is in the range 0 to 1 (inclusive) If this value is 1, then the index J in the frame with index I is The patch orientation index (pdu_orienta tion_index[i][j]) is in the range of 0 to 7 (inclusive) (equ al to 0 specifies that the patch orienta tion index for a patch with index j in a frame with index i, pdu_orientation_ind ex[i][j], is in the range of 0 to 1, inc lusive. asps_use_eight_orientations_flag equal to 1 specifies that the patch ori entation index for a patch with index j in a frame with index i, pdu_orientation _index[i][j], is in the range of 0 to 7, inclusive).

[0601] ASPS extended projection enabled flag (asps_extended_proje ction_enabled_flag): If this value is 0, patch projection is disabled. No information is currently signaled for the atlas style group or atlas style. If this value is 1, it indicates that the current Indicates that the patch projection information is signaled (equal to 0 specifies that the patch projection in formation is not signaled for the curre nt atlas tile group. asps_extended_proje ction_enabled_flag equal to 1 specifies that the patch projection information is signalled for the current atlas tile gr oup).

[0602] If the type (atgh_type) is skip tile (SKIP_TILE_GRP), If not present, the following elements are included in the atlas tile group (or tile) header:

[0603] ASPS vertical axis limit quantization enable flag (asps_normal_axis_limit ts_quantization_enabled_flag): If this value is 1, The quantization parameter is a patch data unit, a combined patch data unit, or an inter-pattern data unit. It is used to quantize the vertical axis related component of the data unit and is signaled. If this value is 0, it indicates that the patch data unit, consolidated patch data unit, or Indicates that no quantization is applied on the vertical axis related components of the interpatch data unit ( equal to 1 specifies that quantization p parameters shall be signaled and used fo r quantizing the normal axis related ele ments of a patch data unit, a merge patch h data unit, or an inter patch data unit .If asps_normal_axis_limits_quantizatio n_enabled_flag is equal to 0, then no qu antization is applied on any normal axis related elements of a patch data unit, a merge patch data unit, or an inter pat ch data unit).

[0604] asps_normal_axis_limits_quantization_en If abled_flag is 1, then atgh_pos_min_z_quantize r is included in the atlas tile group (or tile) header.

[0605] ASPS vertical axis max delta value valid flag (asps_normal_axis_m ax_delta_value_enabled_flag): If this value is 1, the The geometry information of the patch with index I in the frame with index J is The maximum nominal shift value of the vertical axis that can exist is The data units are indicated in the bitstream for inter-patch or inter-patch data units. If this value is 0, the frame with index I in the frame with index J is The maximum nominal shift value of the vertical axis that can exist in the geometry information of a patch is Bits for unit, integrated patch data unit or inter-patch data unit Indicates that it is not specified in the stream (equal to 1 specifies that the maximum nominal shift value of the normal axis that may be present in t he geometry information of a patch with index i in a frame with index j will be indicated in the bitstream for each patch h data unit, a merge patch data unit, or an inter patch data unit. If asps_norma l_axis_max_delta_value_enabled_flag is e qual to 0 theN the maximum nominal Shift value of the normal axis that may be pr esent in the geometry information of ap atch with index i in a frame with index j shall not be indicated in the bitst ream for each patch data unit, a merge p atch data unit, or an inter patch data u nit).

[0606] asps_normal_axis_max_delta_value_enable If d_flag is 1, then atgh_pos_delta_max_z_quanti zer is included in the atlas tile group (or tile) header.

[0607] ASPS remove duplicate points enabled flag (asps_remove_duplicate _point_enabled_flag): If this value is 1, duplicate points are Find points from the index map that have the same 2D and 3D geometric coordinates as other points. where the overlapping point is not reconstructed for the current atlas. A value of 0 indicates that all points are reconstructed (equal to 1 indicates that duplicated points are not econstructed for the current atlas, whe re a duplicated point is a point with th e same 2D and 3D geometry coordinates as another point from a Lower index map. sps_remove_duplicate_point_enabled_flag equal to 0 indicates that all points are reconstructed).

[0608] ASPS Max Decode Atlas Frame Buffering (asps_max_dec_ atlas_frame_buffering_minus1): Adding 1 to this value , the decoded atlas for CAS in the atlas frame buffer storage unit Specifies the maximum requested size of the frame buffer (plus 1 specifies the maximum required size of the decoded atlas frame buffer for the CAS in units of atlas frame storage buffers).

[0609] ASPS pixel deinterleaving flag (asps_pixel_deinterleaving rleaving_flag) If this value is 1, the decoded gi for the current atlas Geometry and quality video are spatially interleaved pixels from the two maps If this value is 0, it indicates that the decoded geometry corresponding to the current atlas is included. and attribute indicates that the video contains pixels from only a single map (equal to 1 indicates that the decoded geometry a nd attribute videos for the current atla s contain spatially interleaved pixels rom two maps. asps_pixel_deinterleaving_ flag equal to 0 indicates that the decod ed geometry and attribute videos corresp onding to the current atlas contain pixels lS from only a single map).

[0610] ASPS patch priority order flag (asps_patch_precedenc e_order_flag): If this value is 1, the patch priority order for the current atlas If this value is 0, it indicates that the order (priority) of the atlas is the same as the decoding order. indicates that the patch priority is the reverse of the decoding order (equal to 1 indicates ates that patch precedence for the curre nt atlas is the same as the decoding ord er. asps_patch_precedence_order_flag equ al to 0 indicates that patch precedence for the current atlas is the reverse of the decoding order).

[0611] ASPS patch size quantizer presence flag (asps_patch_size_qua ntizer_present_flag): If this value is 1, patch size quantization The parameter must be in an Atlas Style group header or an Atlas Style header. A value of 0 indicates that there is no patch size quantization parameter (e qual to 1 indicates that the patch size quantization parameters are present in a n atlas tile group header. If asps_patch _size_quantizer_present_flag is equal to 0, then the patch size quantization par ameters are not present).

[0612] asps_patch_size_quantizer_present_flag If it is 1, then atgh_patch_size_x_info_quantizer and atgh_patch_size_y_info_quantizer is atlas style Included in the group (or tile) header.

[0613] ASPS EOM Patch Enabled Flag (asps_eom_patch_enabled _flag): If this value is 1, the decoded occupancy map video for the current atlas contains information related to whether intermediate depth positions between two depth maps are occupied or not. If this value is 0, the decoded occupancy map video is indicates that intermediate depth positions of do not contain information related to whether they are occupied or not (e qual to 1 indicates that the decoded occ upancy map video for the current atlas c maintains information related to whether i mediate depth positions between two depth maps are occupied. ASPS_eom_patch_ enabled_flag equal to 0 indicates that t he decoded occupancy map video does not contain information related to whether I mediate depth positions between two depth maps are occupied).

[0614] ASPS Raw Patch Enabled Flag (asps_raw_patch_enabled_fl ag): If this value is 1, the decoded geometry and attribute video for the current atlas is If this value is 0, it indicates that the data contains information related to a RAW coded point. If present, the decoded geometry and characteristic video will be related to the RAW coded point. indicates that the data does not contain any related information (equal to 1 indicates that t the decoded geometry and attribute vid eos for the current atlas contains info mation related to RAW coded points. asps _raw_patch_enabled_flag equal to 0 indic ates that the decoded geometry and attri bute videos do not contain information r (elated to RAW Coded points).

[0615] ASPS EOM Patch Enabled Flag (asps_eom_patch_enabled _flag) or ASPS raw patch enable flag (asps_raw_patch_ena bled_flag) is 1, the APSP additional video enabled flag (asps_aux iliary_video_enabled_flag) is an atlas sequence parameter. Included in the dataset syntax.

[0616] APSP auxiliary video enable flag (asps_auxiliary_video_ena bled_flag): If this value is 1, the associated Indicates that the information is located in the additional video sub-bitstream. If this value is 0, Information related to RAW and EOM patch types is only available in the first video sub-bitstream. (equal to 1 indicates that information associated with RAW and EOM patch types could be placed in auxiliary video sub-bitstreams. asps_auxiliary_vi deo_enabled_flag equal to 0 indicates th at information associated with RAW and E OM patch types can only be placed in Pri mary video sub-bitstreams).

[0617] ASPS point local restore enabled flag (asps_point_local_re construction_enabled_flag): If this value is 1, that local reconstruction mode information is currently present in the bitstream for the atlas If this value is 0, the information related to the point-local reconstruction mode is currently indicates that it is not present in the bitstream for this class (equal to 1 in dicates that point local reconstruction mode information may be present in the b itstream for the current atlas. asps_poi nt_local_reconstruction_enabled_flag equ al to 0 indicates that no information re lated to the point local reconstruction mode is present in the bitstream for the current atlas).

[0618] ASPS point local restore enabled flag (asps_point_local_re If the construction_enabled_flag is 1, ASPS points Point Local Reconstruction Information (asps_point_local_reconstructi on_information) is conveyed to the atlas sequence parameter set.

[0619] ASPS map count (asps_map_count_minus1): This value Adding 1 will add the current atlas used to encode geometry and attribute data. plus 1 indicates the number of maps f maps that may be used for encoding the geometry and attribute data for the cur rent atlas).

[0620] ASPS pixel deinterleaving enable flag (asps_pixel_dein terrleaving_enabled_flag): If this value is 1, the current Decoded geometry and characteristics for the video are spatially interleaved. If this value is 0, it indicates that the decoded geometry for the current atlas is included. Indicates that the bird and attribute video contains pixels from only a single map (equal to 1 indicates that the decoded geometry and attribute videos for the current at las contains spatially interleaved pixels .asps_pixel_deinterleaving_flag equal t o 0 indicates that the decoded geometry and attribute videos corresponding to th e current atlas contains pixelS from only a single map).

[0621] ASPS pixel deinterleaving map flag (asps_pixel_dei interleaving_map_flag[i]): If this value is 1, the current The decoded geometry and feature video corresponding to the map with index i in the matrix is It is shown that the two maps contain corresponding spatially interleaved pixels. If this value is 0, the decoded map corresponding to the map with index i in the current atlas is geometry and characteristics to indicate that the video contains pixels corresponding to a single map (equ al to 1 indicates that decoded geometry and attribute videos correpsonding to ma p with index i in the current atlas cont ain spatially interleaved pixelS corresp onding to two maps. asps_pixel_deinterle aving_map_flag[i] equal to 0 indicates t hat decoded geometry and attribute video s corresponding to map index i in the cu rrent atlas contains pixelS corresponding to a single map).

[0622] ASPS pixel deinterleaving map flag (asps_pixel_dei If interleaving_map_flag[i] is 1, the ASPS map The value of the ASPS_map_count_minus1 Deinterleaving map flag (asps_pixel_deinterleavi ng_map_flag[i]) is propagated to the atlas sequence parameter set.

[0623] ASPS EOM Patch Enabled Flag (asps_eom_patch_enabled _flag) and ASPS map count (asps_map_count_minus 1) is 1, the ASPS EOM fixed bit count (asps_eom_fix_ bit_count_minus1) is propagated to the atlas sequence parameter set do.

[0624] ASPS EOM Fixed Bit Count (asps_eom_fix_bit_count t_minus1): Adding 1 to this value increases the size in bits of the EOM codeword. (plus 1 indicates the size in bits of the EOM codeword).

[0625] ASPS pixel deinterleaving enable flag (asps_pixel_dein terleaving_enabled_flag) or ASPS point local restore Enable flag(asps_point_local_reconstruction_en If enabled_flag is 1, the ASPS surface thickness (asps_surface_ thickness_minus1) is propagated to the atlas sequence parameter set do.

[0626] ASPS Surface Thickness (asps_surface_thickness_minus1) :Adding 1 to this value will increase the asps_pixel_deinterleaving_en abled_flag or asps_point_local_reconstruct If ion_enabled_flag is 1, explicitly coded depth values ​​and The maximum absolute difference between the interpolated depth value and the maximum absolute difference between an explicitly coded depth value and interpo lated depth value when asps_pixel_deinte rleaving_enabled_flag or asps_point_loca l_reconstruction_enabled_flag is equal t o 1).

[0627] ASPS vui parameters present flag (asps_vui_parameters_ present_flag): If this value is 1, vui_parameters( ) syntax structure is present. If this value is 0, the vui_parame ters() indicates that no syntax structure exists (equal to 1 sp ecifies that the vui_parameters( ) synta x structure is present. asps_vui_paramet ers_present_flag equal to 0 specifies th at the vui_parameters( ) syntax structur e is not present).

[0628] ASPS extension flag (asps_extension_flag): This value is 0 and the asps_extension_data_flag syntax element is ASPS Indicates that it is not present in the RBSP syntax structure (equal to 0 spe cifies that no asps_extension_data_flag syntax elements are present in the ASPS RBSP syntax structure).

[0629] ASPS Extension Data Flag (asps_extension_data_flag): Indicates that the data for the extension is contained within the ASPS RBSP syntax structure.

[0630] Trailing bits (rbsp_trailing_bits): Indicates the end of the RBSP data. After adding a stop bit of 1 to indicate It is used to fill the remaining bits with 0 for byte alignment.

[0631] FIG. 36 shows an atlas frame parameter set according to an embodiment. parameter set).

[0632] Figure 36 shows the NAL unit type when the NAL unit type is NAL_AFPS, as in Figure 34. Atlas frame parameter set included in L unit (Atlas frame parameter parameter set).

[0633] Atlas frame parameter set set, AFPS) is a set of zero or more fully coded atlas frames. It contains a syntax structure that contains syntax elements that apply to.

[0634] AFPS atlas frame parameter set ID (afps_atlas_frame _parameter_set_id): An attribute for reference by other syntax elements. Identifies the atlas frame parameter set. It provides an identifier that can be referenced by syntax elements.

[0635] AFPS atlas sequence parameter set ID (afps_atlas_sequ ence_parameter_set_id): Active Atlas sequence parameter specifies the value of asps_a tlas_sequence_parameter_set_id for the a ctive atlas sequence parameter set).

[0636] Atlas Frame Tile Information (atlas_frame_tile_informat ion(): This will be explained with reference to FIG.

[0637] AFPS output flag existence flag (afps_output_flag_pre sent_flag): If this value is 1, the Atlas Style Group Header Frame Adapter Output flag (atgh_frame_output_flag) or atlas type header frame output flag (ath_frame_output_flag) Indicates that the syntax element is present in an associated tile group header or tile header. If this value is 0, the atlas style group header frame output flag (at gh_frame_output_flag) or Atlas Style Header Frame Out The put flag (ath_frame_output_flag) syntax element is relevant. Indicates that it is not present in the tile group header or tile header (equal to 1 indicates that the atgh_frame_output_fl ag syntax element is present in the asso ciated tile group headers. afps_output_f lag_present_flag equal to 0 indicates th at the atgh_frame_output_flag syntax ele ment is not present in the associated ti le group headers).

[0638] AFPS reference index count (afps_num_ref_idx_default_ active_minus1): Adding 1 to this value will increase the atgh_num_ref_i Tile group or tile with dx_active_override_flag set to 0 The inferred value of the variable NumRefIdxActive for (plus 1 spec provides the inferred value of the variable NumRefIdxActive for the tile group with atgh_num_ref_idx_active_override_flag e qual to 0).

[0639] AFPS Additional Variables (afps_additional_lt_afoc_lsb_le n): Variable MaxLtAt used within the decoding process for atlas frame reference. Specifies the value of lasFrmOrderCntLsb ue of the variable MaxLtAtlasFrmOrderCnt Lsb that is used in the decoding process for reference atlas frame).

[0640] AFPS 3D Position X Bit Count (afps_3d_pos_x_bit_ count_minus1): Adding 1 to this value will increase the afps_atlas_fram Atlas style group or atlas named e_parameter_set_id A fixed length table of pdu_3d_pos_x[j] for the patch with index J in the file. indicates the number of bits in the current f bits in the fixed-length representation n of pdu_3d_pos_x[j] of patch with index j in an atlas tile group that refers to afps_atlas_frame_parameter_set_id).

[0641] AFPS 3D Position Y Bit Count (afps_3d_pos_y_bit_ count_minus1): Adding 1 to this value will increase the afps_atlas_fram Atlas style group or atlas named e_parameter_set_id A fixed length table of pdu_3d_pos_y[j] for the patch with index J in the file. indicates the number of bits in the current f bits in the fixed-length representation n of pdu_3d_pos_y[j] of patch with index j in an atlas tile group that refers to afps_atlas_frame_parameter_set_id).

[0642] AFPS LOD mode enabled flag (afps_lod_mode_enabled_ flag): A value of 1 indicates that LOD parameters may be present in the patch. A value of 0 indicates that the LOD parameter is not present in the patch (equa l to 1 indicates that the LOD parameters may be present in a patch. afps_lod_mod e_enabled_flag equal to 0 indicates that the LOD parameters are not present na patch).

[0643] AFPS override EOM flag (afps_override_eom_for _depth_flag): If this value is 1, afps_eom_number_o f_patch_bit_count_minus1 and afps_eom_max_b The it_count_minus1 value is explicitly indicated in the bitstream. If this value is 0, afps_eom_number_of_patch_bit _count_minus1 and afps_eom_max_bit_count_mi indicates that the value of nus1 is implicitly derived (equal to 1 indicates es that the values ​​of afps_eom_number_of _patch_bit_count_minus1 and afps_eom_max _bit_count_minus1 is explicitly present in the bitstream. afps_override_eom_for_ depth_flag equal to 0 indicates that the values ​​of afps_eom_number_of_patch_bit_ count_minus1 and afps_eom_max_bit_count_ minus1 are implicitly derived).

[0644] AFPS EOM patch bit count (afps_eom_number_of_ patch_bit_count_minus1): atlas frame parameter set The number of geometry patches associated with the EOM characteristic patch in the atlas frame associated with indicates the number of bits used to represent the umber of bits used to represent the numbness er of geometry patches associated in an EOM attribute patch in an atlas frame th at is associated with this atlas frame p parameter set).

[0645] AFPS EOM Max Bit Count (afps_eom_max_bit_count unt_minus1): The atlas frame associated with this atlas frame parameter set. Indicates the number of EOM points per geometry patch associated with the EOM quality patch in the frame. (plus 1 specifies the number of bits used for amber of bits used to represent the numbness r of EOM points per geometry patch assoc iated with an EOM attribute patch in an atlas frame that is associated with this atlas frame parameter set).

[0646] AFPS RAW 3D Position Bit Count Explicit Mode Flag (afps_ra w_3d_pos_bit_count_explicit_mode_flag): If the value of is 1, rpdu_3d_pos_x, rpdu_3d_pos_y and rp The number of bits in the fixed length representation of du_3d_pos_z is afps_atlas_fram Atra style group header or attribute called e_parameter_set_id atgh_raw_3d_pos_axis_bit_count in the LaTeX header _minus1 or ath_raw_3d_pos_axis_bit_count_m indicates that it is coded specifically by inus1. If this value is 0, then atgh _raw_3d_pos_axis_bit_count_minus1 or ath_r The value of aw_3d_pos_axis_bit_count_minus1 is implicitly derived. (equal to 1 indicates that the n umber of bits in the fixed-length repres entation of rpdu_3d_pos_x, rpdu_3d_pos_y , and rpdu_3d_pos_z is explicitly coded by atgh_raw_3d_pos_axis_bit_count_minus1 in the atlas tile group header that ref ers to afps_atlas_frame_parameter_set_id .afps_raw_3d_pos_bit_count_explicit_mod e_flag equal to 0 indicates the value of atgh_raw_3d_pos_axis_bit_count_minus1 i s implicitly derived).

[0647] afps_raw_3d_pos_bit_count_explicit_mode If _flag is 1, then atgh_raw_3d_pos_axis_bit_cou nt_minus1 is included in the atlas tile group (or tile) header.

[0648] AFPS extension flag (afps_extension_flag): This value is 0 and afps_extension_data in the AFPS RBSP syntax structure _flag indicates that the syntax element is not present (equal to 0 spec provides that no afps_extension_data_flag s yntax elements are present in the AFPS R BSP syntax structure).

[0649] AFPS Extension Data Flag (afps_extension_data_flag): Includes extended related data.

[0650] FIG. 37 shows the atlas frame tile information (atlas_frame_tile) according to the embodiment. e_information).

[0651] FIG. 37 shows the syntax of the atlas frame tile information included in FIG.

[0652] Single tile flag (afti_single_ti) in AFTI atlas frame le_in_atlas_frame_flag): If this value is 1, it refers to AFPS. A value of 0 indicates that there is only one tile in each atlas frame that the atlas references. then there is one or more tiles in each atlas frame that references AFPS. (equal to 1 specifies that there is only one tile in each atlas frame refer ring to the AFPS. afti_single_tile_in_at las_frame_flag equal to 0 specifies that there is more than one tile in each atl as frame referring to the AFPS).

[0653] AFTI uniform tile spacing flag (afti_uniform_tile_spacin g_flag): If this value is 1, the tile column and row bounds are aligned relative to the atlas frame. uniformly distributed across afti_tile_cols_width_minus1 and afti_tile_cols_width_minus1 ti_tile_rows_height_minus1, syntax elements respectively A value of 0 indicates that tile column and row boundaries are signaled using Uniformly distributed or not distributed across atlas frames, afti_num_t ile_columns_minus1, afti_num_tile_rows_mi nus1, a list of syntax element pairs afti _tile_column_width_minus1[i], afti_tile_r Use syntax elements such as ow_height_minus1[i] to signal Indicates that the

[0654] AFTI tile columns width (afti_tile_cols_width_minus1) :Adding 1 to this value will increase the afti_uniform_tile_spacing_fl If ag is 1, the rightmost tile column of the atlas frame within 64 sample units is excluded. plus 1 specifies the width of the tile column the tile columns excluding the right-mos t tile column of the atlas frame in unit s of 64 samples when afti_uniform_tile_s pacing_flag is equal to 1).

[0655] AFTI Tile Rows Height (afti_tile_rows_height_minus 1): Adding 1 to this value will increase the afti_uniform_tile_spacing_ If flag is 1, the bottom tile row of the atlas frame within 64 sample units is excluded. Specifies the height of the tile row (plus 1 specifies the height of the row) f the tile rows excluding the bottom til e row of the atlas frame in units of 64 samples when afti_uniform_tile_spacing_f lag is equal to 1).

[0656] Number of AFTI tile columns (afti_num_tile_columns_minus1 ): Adding 1 to this value will increase the afti_uniform_tile_spacing_f If lag is 0, it indicates the number of tile columns to divide the atlas frame into (plus 1 specifies the number of tile columns pa rtitioning the atlas frame when afti_uni form_tile_spacing_flag is equal to 0).

[0657] Number of AFTI tile rows (afti_num_tile_rows_minus1): Adding 1 to the value of afti_uniform_tile_spacing_flag If is 0, it indicates the number of tile rows to divide the atlas frame into (Plus 1 sp ecifies the number of tile rows partitio ning the atlas frame when pti_uniform_ti le_spacing_flag is equal to 0).

[0658] AFTI Tile Column Width (afti_tile_column_width_minus 1[i]): Adding 1 to this value indicates the width of the I-th tile column in 64 sample units. (plus 1 specifies the width of the i-th tile column in units of 64 samples).

[0659] AFTI Tile Row Height (afti_tile_row_height_minus1 [i]): Adding 1 to this value indicates the height of the I-th tile row in 64 sample units. (plus 1 specifies the height of the i-th tile row in units of 64 samples).

[0660] Single tile flag (afti_single_ti) for each AFTI tile group tile_per_tile_group_flag): A value of 1 indicates AFPS. This indicates that each tile group (or tile) contains one tile. and the tile group (or tile) that indicates this AFPS contains more than one tile. (equal to 1 specifies that each tile group that refers to this AFPS includes one tile. afti_single_tile_per_tile_grou p_flag equal to 0 specifies that a tile group that refers to this AFPS may include de more than one tile).

[0661] Single tile flag (afti_single_ti) for each AFTI tile group le_per_tile_group_flag) is 0, afti_num_t ile_groups_in_atlas_frame_minus1 is the atlas frame afti_num_tile_groups_in_at las_frame_minus1, afti_top_left_tile_ idx[i] and afti_bottom_right_tile_idx_delta [i] is conveyed in the atlas frame tile information.

[0662] Number of tile groups (or tiles) in the AFTI atlas frame (afti_num_ tile_groups_in_atlas_frame_minus1):AFPS Indicates the number of tile groups (or tiles) in each atlas frame shown (plus 1 specifies the number of tile groups in each atlas frame referring to the AFPS) .

[0663] AFTI top left tile index (afti_top_left_tile_idx [i]): The tile located in the upper left corner of the I-th tile group (or tile). specifies the tile index of the tile located at the top-left corner of the i-th tile group).

[0664] AFTI bottom right tile index delta (afti_bottom_right_t ile_idx_delta[i]):afti_top_left_tile_idx [i] and the tile located in the bottom right corner of the I-th tile group (or tile) Specifies the difference between tile indices. ce between the tile index of the tile lo cated at the bottom-right corner of the i-th tile group and afti_top_left_tile_i dx[i]).

[0665] AFTI signaled tile group ID flag (afti_signalle d_tile_group_id_flag): If this value is 1, each tile group The tile group ID or tile ID for the group or each tile is signaled. (equal to 1 specifies that the tile group ID for each tile group is signal ed).

[0666] AFTI signaled tile group ID flag (afti_signalle d_tile_group_id_flag) is 1, afti_signall ed_tile_group_id_length_minus1 and afti_til The e_group_id[i] is conveyed in the atlas frame tile information.

[0667] AFTI signaled tile group ID length (afti_signalled _tile_group_id_length_minus1): Add 1 to this value , the syntax element afti_tile_group_id[i] is used to indicate If present, the syntax element tile group header or tile There can be a 'atgh_address' in the header (plus 1 specifies the number of bits used to represent the syntax element afti_tile_group_id[i] wh en present, and the syntax element atgh_ address in tile group headers).

[0668] AFTI tile group ID (afti_tile_group_id[i]): I The syntax element afti_tile indicates the ID of the first tile group (or tile). The length of _group_id[i] is afti_signalled_tile_group p_id_length_minus1 plus 1 bit (specifies the tile group ID of the i-th tile grou p. The length of the afti_tile_group_id[ i] syntax element is afti_signalled_tile _group_id_length_minus1 +1 bits).

[0669] FIG. 38 shows the atlas adaptation parameter set (atlas_adaptati on_parameter_set_rbsp()).

[0670] Figure 38 shows the NAL unit transmitted when the NAL unit type is NAL_AAPS. The syntax of the atlas adaptation parameter set is shown below.

[0671] The AAPS RBSP contains the code for one or more coded atlas frames. Parameters indicated by the coded tile group (or tile) NAL unit At most one AAPS RBSP may be active at any given time during the decoding process. Activation of a particular AAPS RBSP is considered to be a previously active AAPS. This results in the deactivation of the RBSP (An AAPS RBSP includes parameters that can be referred to by th e coded tile group NAL units of one or m ore coded atlas frames. At most one AAPS RBSP is considered active at any given moment during the operation of the decod ing process, and the activation of any p articular AAPS RBSP results in the deact ivation of the previously-active AAPS RB SP).

[0672] AAPS atlas adaptation parameter set ID (aaps_atlas_adaptat ion_parameter_set_id): for reference by other syntax elements Identifies the atlas adaptation parameter set s adaptation parameter set for reference by other syntax elements).

[0673] AAPS Atlas Sequence Parameter Set ID (aaps_atlas_sequ ence_parameter_set_id): Active Atlas sequence parameter asps_atlas_sequence_parameter_s for datasets specifies the value of asps_a tlas_sequence_parameter_set_id for the a ctive atlas sequence parameter set).

[0674] AAPS camera parameter existence flag (aaps_camera_parameter s_present_flag): If this value is 1, the camera parameters are currently If this value is 0, it indicates that the current adaptation parameter is Indicates that there are no camera parameters for the dataset (equal to 1 specifies that camera parameters are pre sent in the current atlas adaptation par ameter set. aaps_camera_parameters_prese nt_flag equal to 0 specifies that camera parameters for the current adaptation parameter set are not present).

[0675] AAPS extension flag (aaps_extension_flag): This value is 0 and the aaps_extension_data_flag syntax element is AAPS Indicates that it is not present in the RBSP syntax structure (equal to 0 spec provides that no aaps_extension_data_flag s yntax elements are present in the AAPS R BSP syntax structure).

[0676] AAPS Extension Data Flag (aaps_extension_data_flag): Contains data about the extension.

[0677] FIG. 39 shows the atlas camera parameters (atlas_camera_parameter) according to the embodiment. indicates the number of ameters.

[0678] FIG. 39 shows a specific syntax of the atlas camera parameters in FIG.

[0679] ACP camera model (acp_camera_model): Current adaptive parameter set indicates the camera model for the point cloud frame associated with the s the camera model for point cloud frame s that are associated with the current a daptation parameter set).

[0680] For example, if acp_camera_model is 0, the camera model is UNSPE Indicates that it is CIFIED.

[0681] If acp_camera_model is 1, the camera model is an orthographic camera model. (Orthographic camera model).

[0682] If acp_camera_model is 2-255, the camera model is reserved. .

[0683] When the ACP camera model (acp_camera_model) is 1, the scale, The following elements related to offset and rotation are included in the atlas camera parameters: do.

[0684] ACP scale enabled flag (acp_scale_enabled_flag): This A value of 1 indicates that there is a scale parameter for the current camera model. A value of 0 indicates that there is no scale parameter for the current camera model. indicates that scale parameter eters for the current camera model are p resent. acp_scale_enabled_flag equal to 0 indicates that scale parameters for th e current camera model are not present).

[0685] ACP scale enabled flag (acp_scale_enabled_flag) is 1 If present, ACP scale on axis (acp_scale_on_axis[d] ) is included in the atlas camera parameters.

[0686] ACP Offset Enabled Flag (acp_offset_enabled_flag): A value of 1 indicates that there are offset parameters for the current camera model. If this value is 0, there are currently no offset parameters for the camera model. (equal to 1 indicates that offset parameters for the current camera model are present. acp_offset_enabled_flag equ al to 0 indicates that offset parameters for the current camera model are not pr esent).

[0687] ACP offset enabled flag (acp_offset_enabled_flag) If 1, the ACP offset axis (acp_offset_on_axis[ d]) elements are included in the atlas camera parameters.

[0688] ACP rotation enabled flag (acp_rotation_enabled_fl ag): If this value is 1, there are currently no rotation parameters for the camera model. If this value is 0, the rotation parameters for the current camera model are indicates that the meter is not present (equal to 1 indicates that rotation parameters for the current cam era model are present. acp_rotation_enab led_flag equal to 0 indicates that rotat ion parameters for the current camera mo del are not present).

[0689] ACP scale on axis (acp_scale_on_axis[d]): The current camera model This indicates the scale value for the d axis for the model. The d value ranges from 0 to 2 (inclusive). , y, z axes correspond to values ​​0, 1, 2 (specifies the value ue of the scale, Scale[d], along the da xis for the current camera model. lue of d is in the range of 0 to 2, incl useful, with the values ​​of 0, 1, and 2 co rresponding to the X, Y, and Z axis, res expressly).

[0690] ACP offset axis (acp_offset_on_axis[d]): 0 to 2 (inclusive) ) offset along the d axis for the current camera model. The set[d] value is 0, 1, or 2, which correspond to the x, y, and z axes, respectively. (indicates the value of the offset, Off set[d], along the d axis for the current camera model where d is in the range of 0 to 2, inclusive. The values ​​of d equa l to 0, 1, and 2 correspond to the X, Y, and Z axis, respectively).

[0691] ACP rotation enabled flag (acp_rotation_enabled_fl ag) is 1, the following rotation values ​​are included in the atlas camera parameters:

[0692] ACP rotation qx (acp_rotation_qx): Uses quaternion representation indicates the x component qX for the rotation of the current camera model (speci fies the x component, qX, for the rotati on of the current camera model using the quaternion representation).

[0693] ACP rotation qy (acp_rotation_qy): Uses quaternion representation indicates the y-component qY for the rotation of the current camera model (speci fies the y component, qY, for the rotati on of the current camera model using the quaternion representation).

[0694] ACP rotation qz (acp_rotation_qz): Uses quaternion representation indicates the z-component qZ for the rotation of the current camera model (speci fies the z component, qZ, for the rotati on of the current camera model using the quaternion representation).

[0695] Figure 40 shows the atlas_tile_group layer (atlas_tile_group) in an example. up_layer), Atlas tile group header (atlas_tile_group) p_header).

[0696] Figure 40 shows the attributes transmitted by NAL units according to NAL unit types, as in Figure 34. Syntax of Atlas Style Group Layer and included in Atlas Style Group Layer The syntax for an atlas style group header is as follows:

[0697] Depending on the embodiment, a tile group corresponds to a tile. Similarly, the term 'atgh' can also be interpreted as 'ath'. It is explained.

[0698] Atlas Style Group Layer or Atlas Style Layer is an Atlas Style Group Contains a header or atlas style header.

[0699] If the type (atgh_type) is skip tile (SKIP_TILE_GRP), If not, the atlas tile group (or tile) data is is contained in a tile) layer.

[0700] ATGH Atlas Frame Parameter Set ID (atgh_atlas_frame _parameter_set_id): The current atlas tile group (or tile) afps_atlas_f for the active atlas frame parameter set Specifies the value of the rame_parameter_set_id. value of afps_atlas_frame_parameter_set _id for the active atlas frame parameter set for the current atlas tile group).

[0701] ATGH atlas adaptation parameter set ID (atgh_atlas_adaptat ion_parameter_set_id): Current Atlas Style Group (or Style aaps_atlas_ for active atlas adaptive parameter sets for Indicates the value of adaptation_parameter_set_id (specified es the value of aaps_atlas_adaptation_pa rameter_set_id for the active atlas adap tation parameter set for the current atl as tile group).

[0702] ATGH address (atgh_address): The tag of the tile group (or tile). Indicates the tile group (or tile) address. If not present, atgh_addres The value of s is inferred to be 0. The tile group (or tile) address is the address of the tile group (or is the tile group (or tile) ID of the tile). afti_signalled_tile_group_id_length_mi nus1+1 bits. afti_signalled_tile_group_i If d_flag is 0, then atgh_address must be 0 or afti_num. _tile_groups_in_atlas_frame_minus1 (inclusive) On the other hand, the value of atgh_address is 0 to 2 (afti_signall ed_tile_group_id_length_minus1+1)-1 (inclusive) Specifies the tile group address. of the tile group. When not present, the value of atgh_address is inferred to be equal to 0. The tile group address is t he tile group ID of the tile group. length of atgh_address is afti_signalled _tile_group_id_length_minus1 + 1 bits. I f afti_signalled_tile_group_id_flag is e qual to 0, the value of atgh_address is in the range of 0 to afti_num_tile_group S_in_atlas_frame_minus1, inclusive. rwise, the value of atgh_address is in t he range of 0 to 2(afti_signalled_tile_g roup_id_length_minus1 +1 )-1, inclusive) .

[0703] ATGH type (atgh_type): Current atlas tile group (or tile) Indicates the coding type.

[0704] If the type value is 0, the atlas style group or atlas style type is P_ TILE_GRP (Inter atlas tile group, Inter atlas A tile group (or tile).

[0705] If the type value is 1, the type of the atlas style group or atlas style is I_ TILE_GRP (Intra atlas tile group, Intra atlas A tile group (or tile).

[0706] If the type value is 2, the atlas style group or atlas style type is SK IP_TILE_GRP(SKIP atlas tile group, skip atlas La Style Group (or Tile).

[0707] If the type value is 3, the atlas style group or atlas style type is reserved. It has a value.

[0708] ATGH Atlas Output Flag (atgh_atlas_output_fla g): This value affects the decoded atlas output and the removal processor (a ffectS the decoded atlas output and remo val processes).

[0709] ATGH Atlas Frame Order Count (atgh_atlas_frm_ord er_cnt_lsb): The current atlas type group or atlas style Atlas Frame Order Count modulo MaxAtlasFrmOrderCntLsb specifies the atlas frame order count t modulo MaxAtlasFrmOrderCntLsb for the current atlas tile group).

[0710] If afps_output_flag_present_flag is 1, Style group (tile) header is atgh_atlas_output_flag and and atgh_atlas_frm_order_cnt_lsb.

[0711] ATGH Reference Atlas Frame List SPS Flag (atgh_ref_atlas_ frame_list_sps_flag): If this value is 1, the current atlas style Group reference atlas frame list ref_list_s in the active ASPS It is derived based on one of the truct(rlsIdx) syntax structures. If this value is 0 If the current Atlas Style Group or Atlas Style Tile Group Header or ref_list_struct(rlsIdx) syntax included directly in the tile header Based on the frame structure, the reference atlas frame list of the current atlas style list is derived. (equal to 1 specifies that the ref erence atlas frame list of the current a tlas tile group is derived based on one of the ref_list_struct(rlsIdx) syntax st ructures in the active asps. atgh_ref_at las_frame_list_sps_flag equal to 0 speci fies that the reference atlas frame list of the current atlas tile list is deriv ed based on the ref_list_struct(rlsIdx) syntax structure that is directly included ed in the tile group header of the curre nt atlas tile group).

[0712] atgh_ref_atlas_frame_list_sps_flag is 0 and ref_list_struct(asps_num_ref_atlas_fra me_lists_in_asps) is included in the Atlas Style Group (tile) header can be.

[0713] ATGH Reference Atlas Frame List Index (atgh_ref_atlas_ frame_list_idx): The current atlas tile group (or tile) ref_list_struct(rlsId x) ref_list_struct included in the active ASPS of the syntax structure (rlsIdx) indicates the index into the list of syntax structures (specif ies the index, into the list of the ref_ list_struct(rlsIdx) syntax structures in cluded in the active ASPS, of the ref_li st_struct(rlsIdx) syntax structure that is used for derivation of the reference atlas frame list for the current atlas t ile group).

[0714] ATGH additional afoc lsb existence flag (atgh_additional_afo c_lsb_present_flag[j]): If this value is 1, then atgh_ad ditional_afoc_lsb_val[j] is currently in the atlas style group or Indicates that it exists for atlas styles. If this value is 0, then atgh_addi tional_afoc_lsb_val[j] does not exist (equal to 1 specifies that atgh_additional_afoc_lsb_ val[j] is present for the current atlas tile group. atgh_additional_afoc_lsb_pre sent_flag[j] equal to 0 specifies that a tgh_additional_afoc_lsb_val[j] is not pr esent).

[0715] atgh_additional_afoc_lsb_present_flag[j ] is 1, then atgh_additional_afoc_lsb_val[j] is Included in the atlas tile group (or tile) header.

[0716] ATGH additional afoc lsb value (atgh_additional_afoc_ls b_val[j]): FullAtlas for the current Atlas tile group or tile Indicates the value of sFrmOrderCntLsbLt[RlsIdx][j] (specif ies the value of FullAtlasFrmOrderCntLsb Lt[RlsIdx][j] for the current atlas tile group).

[0717] ATGH position minimum Z quantizer (atgh_pos_min_z_quantiz er): The quantizer applied to the pdu_3d_pos_min_z[p] value of patch p is If atgh_pos_min_z_quantizer is not present, this value is is inferred to 0 (specifies the quantizer that is to be applied to the pdu_3d_pos_min_z[p] value of the patch p. If atgh_pos_min_z _quantizer is not present, its value is inferred to be equal to 0).

[0718] ATGH Position Delta Max Z Quantizer (atgh_pos_delta_ma x_z_quantizer): pdu_3d_pos of patch with index p Indicates the quantizer applied to the value of _delta_max_z[p]. If delta_max_z_quantizer is not present, this value is inferred to be 0. (specifies the quantizer that is to be applied to the pdu_3d_pos_delta_max_z[p] value of the patch with index p. If atg h_pos_delta_max_z_quantizer is not prese nt, its value is inferred to be equal to 0).

[0719] ATGH patch size X information quantizer (atgh_patch_size_x_inf o_quantizer): pdu_2d_size_ of patch with index p x_minus1[p], mpdu_2d_delta_size_x[p], ipdu _2d_delta_size_x[p], rpdu_2d_size_x_minus P applied to the variables 1[p] and epdu_2d_size_x_minus1[p] patchSizeXQuantizer Indicates the value of the quantizer. If ize_x_info_quantizer is not present, this value is Inferred to g2_patch_packing_block_size (specif ies the value of the quantizer PatchSize XQuantizer that is to be applied to the variables pdu_2d_size_x_minus1[p], mpdu_ 2d_delta_size_x[p], Ipdu_2d_delta_size_x [p], rpdu_2d_size_x_minus1[p], and epdu_ 2d_size_x_minus1[p] of a patch with inde x p. If atgh_patch_size_x_info_quantizer is not present, its value is inferred t o be equal to asps_log2_patch_packing_bl ock_size).

[0720] ATGH patch size Y information quantizer (atgh_patch_size_y_inf o_quantizer): pdu_2d_size_ of patch with index P y_minus1[p], mpdu_2d_delta_size_y[p], ipdu _2d_delta_size_y[p], rpdu_2d_size_y_minus P applied to the variables 1[p] and epdu_2d_size_y_minus1[p] patchSizeYQuantizer Indicates the value of the quantizer. If ize_y_info_quantizer is not present, this value is Inferred to g2_patch_packing_block_size (specif ies the value of the quantizer PatchSize YQuantizer that is to be applied to the variables pdu_2d_size_y_minus1[p], mpdu_ 2d_delta_size_y[p], Ipdu_2d_delta_size_y [p], rpdu_2d_size_y_minus1[p], and epdu_ 2d_size_y_minus1[p] of a patch with inde x p. If atgh_patch_size_y_info_quantizer is not present, its value shall be infe rred to be equal to asps_log2_patch_pack ing_block_size).

[0721] ATGH Raw 3D Position Axis Bit Count (atgh_raw_3d_pos_ax is_bit_count_minus1): Adding 1 to this value will result in pos_x, rp The number of bits in the fixed-length representation of du_3d_pos_y and rpdu_3d_pos_z is (plus 1 specifies the number of bits i n the fixed-length representation of rpd u_3d_pos_x, rpdu_3d_pos_y, and rpdu_3d_p os_z).

[0722] Type (atgh_type) is P_TILE_GRP and num_ref_e If ntries[rlsIdx] is greater than 1, then atgh_num_ref_idx_ active_override_flag is set to Atlas style group (or tile) Also included in the header. If ide_flag is 1, then atgh_num_ref_idx_active_m inus1 is included in the atlas tile group (or tile) header.

[0723] ATGH Number Reference Index Active Override Flag (atgh_nu m_ref_idx_active_override_flag): This value is 1 and the syntax element atgh_num_ref_idx_active_minus1 currently exists for the atlas style group or atlas style. If is 0, the syntax element atgh_num_ref_idx_active_m Indicates that inus1 does not exist. atgh_num_ref_idx_active If _override_flag is not present, this value is inferred to be 0 (equal to to 1 specifies that the syntax element a tgh_num_ref_idx_active_minus1 is present for the current atlas tile group. atgh_ num_ref_idx_active_override_flag equal t o 0 specifies that the syntax element at gh_num_ref_idx_active_minus1 is not pres ent. If atgh_num_ref_idx_active_override _flag is not present, its value shall be inferred to be equal to 0).

[0724] ATGH Number Reference Index Active (atgh_num_ref_idx_ active_minus1): for decoding the current atlas tile group or tile Indicates the maximum reference index for the reference atlas frame list used for Nu If the value of mRefIdxActive is 0, the reference index of the reference atlas frame list is The index is not currently used to decode an atlas style group or atlas style. specifies the maximum reference index for reference the atlas frame list that may be used to decode the current a tlas tile group. When the value of NumRe fIdxActive is equal to 0, no reference i index for the reference atlas frame list may be used to decode the current atlas tile group).

[0725] byte_alignment: a stop to indicate the end of the data After adding a 1 (stop bit), the remaining bits are set to 0 for byte alignment. It is used to satisfy the requirements.

[0726] FIG. 41 shows a reference list structure (ref_list_struct) according to an embodiment.

[0727] Figure 41 shows the atlas parameter set of Figure 35, the atlas style group or This is the syntax for the reference list structure contained in the header of a trace tile.

[0728] Number of reference entries (num_ref_entries[rlsIdx]):ref_l ist_struct(rlsIdx) indicates the number of entries in the syntax structure (sp ecifies the number of entries in the ref _list_struct(rlsIdx) syntax structure).

[0729] The elements below the value of num_ref_entries[rlsIdx] constitute the reference list. Included in the structure.

[0730] asps_long_term_ref_atlas_frames_flag is 1 If present, the reference atlas frame flag (st_ref_atlas_frame_fla g[rlsIdx][i]) is included in the reference list structure.

[0731] Reference atlas frame flag (st_ref_atlas_frame_flag[r lsIdx][i]): If this value is 1, ref_list_struct(rls idx) The i-th entry in the syntax structure is the short-term reference atlas frame entry st_ref_atlas_frame_flag[rlsIdx If ][i] is 0, the i-th element in the ref_list_struct(rlsIdx) structure Indicates that this entry is a long-term reference atlas frame entry. In this case, the value of st_ref_atlas_frame_flag[rlsIdx][i] is is inferred to 1 (equal to 1 specifies that the i- th entry in the ref_list_struct(rlsIdx) syntax structure is a Short term referen ce atlas frame entry. st_ref_atlas_frame _flag[rlsIdx][i] equal to 0 specifies th at the i-th entry in the ref_list_struct (rlsIdx) syntax structure is a long term reference atlas frame entry. When not p resent, the value of st_ref_atlas_frame_ flag[rlsIdx][i] is inferred to be equal to 1).

[0732] st_ref_atlas_frame_flag[rlsIdx][i] is 1 and abs delta afoc(abs_delta_afoc_st[rlsIdx][i ]) is included within the reference list structure.

[0733] abs delta afoc(abs_delta_afoc_st[rlsIdx][i] ): the i-th entry is the first in ref_list_struct(rlsIdx) If the short-term reference atlas frame entry is currently an atlas style group (or tile) is referred to by the atlas frame order count value and the i-th entry It indicates the absolute value of the difference between atlas frames. The i-th entry is the short-term reference atlas frame. system and the first short-term reference atom in ref_list_struct(rlsIdx). If it is not a first frame entry, the atlas frame named by the i-th entry Atlas frame order count value and ref_list_struct(r lsIdx) named by the previous short-term reference atlas frame entry in the syntax structure. indicates the absolute value of the difference between the atlas frame order count values ​​of the atlas frames being when the i-th entry is the first Short t erm reference atlas frame entry in ref_l ist_struct(rlsIdx) syntax structure, spe cifies the absolute difference between t he atlas frame order count values ​​of the current atlas tile group and the atlas frame Referred to by the i-th entry, or, when the i-th entry is a Short term ref erence atlas frame entry but not the fir st short term reference atlas frame entr y in the ref_list_struct(rlsIdx) syntax structure, specifies the absolute difference ence between the atlas frame order count values ​​of the atlas frames referred to by the i-th entry and by the previous Sh ort term reference atlas frame entry in the ref_list_struct(rlsIdx) syntax struc ture).

[0734] abs_delta_afoc_st[rlsIdx][i] has a value greater than 0 Then, the entry sign flag (strpf_entry_sign_flag[rls Idx][i]) is included in the reference list structure.

[0735] Entry sign flag (strpf_entry_sign_flag[rlsId x][i]): If this value is 1, the syntax structure ref_list_struct The i-th entry in (rlsIdx) has a value of 0 or greater. strpf_entry_sign_flag[rlsIdx][i] is 0. If there is, it will be the i-th element in the syntax structure ref_list_struct(rlsIdx). indicates that the entry in strpf_e has a value less than 0. If it does not exist, The value of ntry_sign_flag[rlsIdx][i] is inferred to be 1 (e qual to 1 specifies that i-th entry in t he syntax structure ref_list_struct(rlsI dx) has a value greater than or equal to 0. strpf_entry_sign_flag[rlsIdx][i] equ al to 0 specifies that the i-th entry in the syntax structure ref_list_struct(rl sIdx) has a value less than 0. When not present, the value of strpf_entry_sign_f lag[rlsIdx][i] is inferred to be equal t o 1).

[0736] afoc lsb value(afoc_lsb_lt[rlsIdx][i]):ref_l By the I-th entry in the ist_struct(rlsIdx) syntax structure Atlas frame order count modulo MaxAtlas Indicates the value of sFrmOrderCntLsb. afoc_lsb_lt[rlsIdx] [i] The length of the syntax element is asps_log2_max_atlas_frame _order_cnt_lsb_minus4+4 bits (specifies the value of the atlas frame order count modulo MaxAtlasFrmOrderCntLsb of the at las frame referred to by the i-th entry in the ref_list_struct(rlsIdx) syntax st ructure. The length of the afoc_lsb_lt[r lsIdx][i] syntax element is ASPS_log2_MA X_atlas_frame_order_cnt_lsb_minus4 + 4 b its).

[0737] Figure 42 shows the atlas_tile_group data (atlas_tile_group) according to the embodiment. up_data_unit).

[0738] Figure 42 is included in the Atlas Style Group Layer (or Atlas Style Layer) of Figure 40. The syntax of the atlas style group data is shown below. The tile data corresponds to the atlas tile data, and the term tile group is also referred to as tile. It is possible.

[0739] p starts from 0 and increases by one, and the atlas-related element with index p is Included in tile group (or tile) data.

[0740] Patch mode (atgdu_patch_mode[p]): Currently atgdu style group indicates the patch mode for the patch with index p in the group (or tile). If the type is skip tile (SKIP_TILE_GRP), The tile group (or tile) information is the first reference. The same address (at) as the current tile group (or tile) corresponding to the atlas frame gh_address) indicates the patch mode for the tch with index p in the current atlas ti le Group. a tile group with atgh_type = SKIP_TILE_GRP implies that the entire ti le group information is copied directly from the tile group with the same atgh_a address as that of the current tile group that corresponds to the first reference atlas frame).

[0741] atgdu_patch_mode[p] is not I_END, but atgdu_pat If ch_mode[p] is not P_END, then patch_i for each index p information_data and atgdu_patch_mode[p] are Included in style group data (or atlas style data).

[0742] I_TILE_GRP type The patch mode types for atlas tile groups are: It can be expressed as follows:

[0743] If the patch mode (atgdu_patch_mode) value is 0, the identifier (Id entifier) ​​is I_INTRA and non-predicted patch mode (Non-predicted Indicates that the device is in the currently selected Patch mode.

[0744] If the patch mode (atgdu_patch_mode) has a value of 1, the identifier (Id entifier) ​​is I_RAW, and the RAW point patch mode (RAW poi Indicates that the device is in patch mode.

[0745] If the patch mode (atgdu_patch_mode) value is 2, the identifier (Id entifier) ​​is I_EOM, and EOM point patch mode (EOM poi Indicates that the device is in patch mode.

[0746] Patch mode (atgdu_patch_mode) values ​​between 3 and 13 are recognized as The identifier is I_RESERVED, and the reserved mode is indicates that the device is in the Enabled mode.

[0747] If the patch mode (atgdu_patch_mode) value is 14, the identifier (I The patch termination mode is I_END. Indicates that the country is in national mode.

[0748] P_TILE_GRP type Patch model for atlas tile group (or tile) The code type can be expressed as follows:

[0749] If patch mode (atgdu_patch_mode) is 0, the identifier (Iden tifier) ​​is P_SKIP, and the patch skip mode (Patch Skip Indicates that the mode is

[0750] If patch mode (atgdu_patch_mode) is 1, the identifier (Iden tifier) ​​is P_MERGE, which is the patch merge mode (Patch Merge Indicates that the mode is

[0751] If patch mode (atgdu_patch_mode) is 2, the identifier (Iden tifier) ​​is P_INTER, and the inter prediction patch mode (Inter pr Indicates that the device is in the currently selected Patch mode.

[0752] If patch mode (atgdu_patch_mode) is 3, the identifier (Iden tifier) ​​is P_INTRA, and non-predictive patch mode (Non-predict Indicates that the device is in patch mode.

[0753] If patch mode (atgdu_patch_mode) is 4, the identifier (Iden tifier) ​​is P_RAW, and RAW point patch mode (RAW point Indicates that the device is in Patch mode.

[0754] If patch mode (atgdu_patch_mode) is 5, the identifier (Iden tifier) ​​is P_EOM, EOM point patch mode (EOM point Indicates that the device is in Patch mode.

[0755] If patch mode (atgdu_patch_mode) is between 6 and 13, the identifier (Identifier) ​​is P_RESERVED and the reserved mode (Reserve d mode).

[0756] If patch mode (atgdu_patch_mode) is between 6 and 13, the identifier (Identifier) ​​is P_RESERVED and the reserved mode (Reserve d mode).

[0757] If patch mode (atgdu_patch_mode) is 14, the identifier (Ide The patch termination mode is P_END. Indicates that the device is in sync mode.

[0758] SKIP_TILE_GRP type: Parameter for atlas tile group (or tile) The touch mode type can be expressed as follows:

[0759] When patch mode (atgdu_patch_mode) is set, the identifier is P_SKI P, indicating patch skip mode .

[0760] AtgduTotalNumberOfPatches is the number of patches, and the final p-value It is set.

[0761] FIG. 43 shows the patch information data (patch_information_da) according to the embodiment. ta).

[0762] Figure 43 shows the parameters included in the atlas tile group (or tile) data unit of Figure 42. The syntax of the touch information data is shown below.

[0763] Type (atgh_type) is Skip Atlas Style Group (or Skip At If the skip patch data unit ( patchIdx)(skip_patch_data_unit(patchIdx )) is included as patch information data.

[0764] Type is Interatlas Style Group (or Interatlas Style) (P_TI LE_GR) and the patch mode (patchMode) is patch skip mode (P _SKIP), skip patch data unit (patch index) p_patch_data_unit(patchIdx) is included as patch information data. If the patch mode is patch merge mode (P_MERGE), the integrated patch data Data unit (patch index) (merge_patch_data_unit( patchIdx)) is included as patch information data. Patch mode is non-predictive patch In mode (P_INTRA), the patch data unit (patch index) patch_data_unit(patchIdx)) is this syntax structure (hereinafter, The patch mode is included in the inter-prediction patch mode (P _INTER), the inter patch data unit (patch index) (in ter_patch_data_unit(patchIdx)) is this syntax structure When the patch mode is RAW point patch mode (P_RAW), Patch Data Unit (Patch Index) (raw_patch_data_uni t(patchIdx) is included in this syntax structure. When in patch mode (P_EOM), the EOM patch data unit (patch index patchIdx)(eom_patch_data_unit(patchIdx)) Included in the tax structure.

[0765] The type is Intra-Tile Group (I_TILE_GR) and the patch model If the mode is non-predictive patch mode (I_INTRA), the patch data unit (patch index)(patch_data_unit(patchIdx)) is this syntax The patch mode is RAW point patch mode (I_RAW). The raw patch data unit (patch index (raw_patch_data a_unit(patchIdx)) is included in this syntax structure. Patch mode is in EOM point patch mode (I_EOM), the EOM patch data unit ( patch index)(eom_patch_data_unit(patchIdx) ) is included in this syntax structure.

[0766] FIG. 44 shows a patch data unit (patch_data_unit) according to an embodiment. vinegar.

[0767] FIG. 44 shows the syntax of the patch data unit contained in FIG.

[0768] PDU 2D position x (pdu_2d_pos_x[p]): current atlas style The upper left corner of the patch bounding box for patch p in the group (or tile) The atlas tile group indicates the x-coordinate (or left offset) of the tile group. The atlas tile has a tile index. The patch packing block size ze) (specifies the x-coordinate ( or left offset) of the top-left corner o f the patch bounding box for patch p in the current atlas tile group, tileGroupI dx, expressed as a multiple of PatchPack ingBlockSize).

[0769] PDU 2D position y (pdu_2d_pos_y[p]): current atlas style The upper left corner of the patch bounding box for patch p in the group (or tile) The atlas tile group indicates the y-coordinate (or top offset) of the tile group. The atlas tile has a tile index. The patch packing block size ze) (specifies the y-coordinate ( or top offset)of the top-left corner of the patch bounding box for patch p in th e current atlas tile group, tileGroupIdx , expressed as a multiple of PatchPackin gBlockSize).

[0770] PDU 2D size x (pdu_2d_size_x_minus1[p]): this value Adding 1 to the current atlas tile group, tileGroupIdx or current atlas indicates the quantized width value of the patch with index p in tiledx. (plus 1 specifies the quantized width v alue of the patch with index p in the cu rrent atlas tile group, tileGroupIdx).

[0771] PDU 2D size y (pdu_2d_size_y_minus1[p]): This value Adding 1 to the current atlas tile group, tileGroupIdx or current atlas The quantized height value of the patch with index p in tiledx. (plus 1 specifies the quantized height value of the patch with index p in the current atlas tile group, tileGroupIdx).

[0772] PDU 3D position x (pdu_3d_pos_x[p]): current position along the tangent axis Atlas style group or current atlas style in patch with index p Specifies the shift to be applied to the patch points ift to be applied to the reconstructed p atch points in patch with index p of the current atlas tile group along the tang ent axis).

[0773] PDU 3D position y (pdu_3d_pos_y[p]): Position along the minor tangent axis. In the atlas style group or in the patch with the current atlas style index p Specifies the shift to be applied to the restored patch points. shift to be applied to the reconstructed patch points in patch with index p of th e current atlas tile group along the bit angent axis).

[0774] PDU 3D position z(pdu_3d_pos_min_z[p]): Along the vertical axis In the patch with the current atlas style group or atlas style index p, Specifies the shift to be applied to the restored patch points of shift to be applied to the reconstructed patch points in patch with index p of t he current atlas tile group along the no rmal axis).

[0775] asps_normal_axis_max_delta_value_enable If d_flag is 1, pdu_3d_pos_delta_max_z[patc hIdx] is included in the patch data unit.

[0776] PDU 3D Position Delta Max z (pdu_3d_pos_delta_max x_z[p]): If this value is present, the current atlas style group or After being transformed into a nominal representation in a patch with atlas style index p, is the nominal maximum shift expected to exist within a given bit depth patch geometry sample. (If present, specifies the nominal max imum value of the shift expected to be p resent in the reconstructed bitdepth pat ch geometry samples, after conversion to their nominal representation, in patch with index p of the current atlas tile g along the normal axis).

[0777] PDU Projection ID (pdu_projection_id[p]): Current address For the patch with index p in the tra-tile group or the current tra-tile Normal index and projection mode value for the projection plane specifies the values ​​of the projecti on mode and of the index of the normal t o the projection plane for the patch wit h index p of the current atlas tile grou p).

[0778] PDU Orientation Index (pdu_orientation_inde x[p]): currently has index p of the atlas style group or atlas style indicates the patch orientation index for the patch (indicates t he patch orientation index for the patch with index p of the current atlas tile group).

[0779] If afps_lod_mode_enabled_flag is 1, pdu_lo d_enabled_flag[patchIndex] is included in the patch data unit. pdu_lod_enabled_flag[patchIndex] is greater than 0. pdu_lod_scale_x_minus1[patchIndex] and pdu_lod_scale_y[patchIndex] is included in the patch data unit. It can be enjoyed.

[0780] PDU LOD enabled flag (pdu_lod_enabled_flag[p]): A value of 1 indicates that LOD parameters are currently present for patch p. A value of 0 indicates that there are no LOD parameters for the current patch (equ al to 1 specifies that the LOD parameter s are present for the current patch p. I f pdu_lod_enabled_flag[p] is equal to 0, no LOD parameters are present for the c Current patch).

[0781] PDU LOD scale X (pdu_lod_scale_s_minus1[p]) :The current atlas style group before adding the patch coordinate Patch3DPosX[p] or applied to the local X coordinate of the point in the patch with index p in the atlas style Specifies the LOD scaling factor. e LOD scaling factor to be applied to th e local x coordinate of a point in a pat ch with index p of the current atlas til e group, prior to its addition to the pa tch coordinate Patch3DPosX[p]).

[0782] PDU LOD scale Y (pdu_lod_scale_y[p]): Patch coordinate P Before adding to the catch3DPosY[p], the current atlas style group or atlas style LOD multiplier applied to the local Y coordinate of the point in the patch with index p in the file. specifies the LOD scaling factor to be applied to the local y coordinate of a point in a patch with index p of the current atlas tile group, prior to its a ddition to the patch coordinate Patch3DP osY[p]).

[0783] asps_point_local_reconstruction_enabled If _flag is 1, point_local_reconstruction_d ata(patchIdx) is included in the patch data unit.

[0784] Point local restore data (patch index)(point_local_re construction_data(patchIdx)): lossy compression at the decoder end It contains information to restore missing points due to

[0785] FIG. 45 shows the rotation and offset for patch orientation according to an embodiment. Indicates the

[0786] Figure 45 shows the rotation matrix for the orientation index of Figure 44. Indicates the number and offset.

[0787] The method / apparatus according to the embodiment performs an orientation operation on point cloud data. The identifier, rotation, and offset for this are used as shown in Figure 45.

[0788] FIG. 46 shows the scene object information (scene_object_information) according to an embodiment. ion).

[0789] Figure 46 shows the sample stream N included in the bit stream 32000 as shown in Figure 32. The syntax of the SEI message for the AL unit is shown below.

[0790] The object in this embodiment is a point cloud object. It is a concept that includes objects.

[0791] The method / apparatus according to the embodiment may generate one or more scene objects based on scene object information according to the embodiment. A 3D spatial region containing a scene object Provides partial access to the unit.

[0792] According to an embodiment, the SEI message may be related to decoding, reconstruction, display, or other purposes. Contains information about the process.

[0793] The SEI messages according to the embodiment are classified as essential and non-essential. Contains two types:

[0794] Non-essential SEI messages are not required for the decoding process. No action is required to process this information for matching.

[0795] Scene Object Information According to an Embodiment , Object Label Information, Patch Information (Patch Information), and volumetric rectangle information (Volum Volumetric Rectangle Information Volumetric Annotation SEI Message is a non-mandatory SEI message.

[0796] Also, in some embodiments, the above information is conveyed in a mandatory SEI message.

[0797] Mandatory SEI messages are mandatory parts of the V-PCC bitstream and are Mandatory SEI messages are categorized into two types:

[0798] Mandatory SEI message of type A: Such an SEI is used to verify bitstream encoding. It contains information necessary to implement the DMA and information for the output timing decoder code. The V-PCC decoder by example does not discard relevant type A mandatory SEI messages. The V-PCC decoder according to the embodiment converts the information into bitstream codes and output tags. Consider for timing decoder code.

[0799] Mandatory SEI message of type B: V-PCC decoder according to a specific reconstruction profile does not discard the relevant Type B mandatory SEI messages, and We will use that information for the purposes for which it is applicable.

[0800] Figure 46 shows the Volumetric Annotation SEI message. ation SEI Message).

[0801] The V-PCC bitstream according to the embodiment is a partial access ) related volumetric annotation SEI messages are defined as shown in Figure 44.

[0802] Cancellation flag (soi_cancel_flag): If this value is 1, the scene The Scene Object Information SEI message retrieves the previous Scene Object Information SEI message in the order of output. (equal to 1 indicates that the scene object information SEI message canc els the persistence of any previous scene e object information SEI message in outp ut order).

[0803] Number of updated objects (soi_num_object_updates): Currently updated by SEI indicates the number of objects to be updated cts that are to be updated by the curren t SEI).

[0804] If soi_num_object_updates is greater than 0, the following elements are Included in the object information.

[0805] Simple Objects Flag(soi_simple_objects_flag):This value If 1, additional information about updated or newly introduced objects is signaled. If this value is 0, no information is provided about the updated or newly introduced objects. indicates that additional information is signaled (equal to 1 indicates s that no additional information for an updated or newly introduced object will be signalled. soi_simple_objects_flag eq ual to 0 indicates that additional information mation for an updated or newly introduced d object may be signaled).

[0806] If soi_simple_objects_flag is 0, the following elements are added to the scene: Included in object information.

[0807] If soi_simple_objects_flag is not 0, soi_obje ct_label_present_flag=0, soi_priority_pre sent_flag=0, soi_object_hidden_present_fl ag=0, soi_object_dependency_present_flag= 0, soi_visibility_cones_present_flag=0, soi i_3d_bounding_box_present_flag=0, soi_col lision_shape_present_flag=0, soi_point_st yle_present_flag=0, soi_material_id_prese nt_flag=0, soi_extension_present_flag= The value is set to .

[0808] Object label present flag(soi_object_label_present_fla g): If this value is 1, the object label information is currently in the scene object information SEI message. If this value is 0, it means that there is no object label information (e qual to 1 indicates that object label in formation is present in the current scene e object information SEI message. soi_ob ject_label_present_flag equal to 0 indic ates that object label information is no t present).

[0809] Priority present flag (soi_priority_present_flag): This If the value is 1, it means that priority information is currently present in the scene object information SEI message. A value of 0 indicates that there is no priority information (equal to 1 indicates that priority information i s present in the current scene object in formation SEI message. soi_priority_pres ent_flag equal to 0 indicates that prior ity information is not present).

[0810] Object Hidden Present Flag(soi_object_hidden_present_fla g): If this value is 1, there is currently hidden object information in the scene object information SEI message. If this value is 0, it means that there is no hidden object information (equ al to 1 indicates that hidden object inf ormation is present in the current scene object information SEI message. soi_obj ect_hidden_present_flag equal to 0 indic ates that hidden object information is n ot present).

[0811] Object dependency present flag (soi_object_dependency_present _flag): If this value is 1, the object dependent is currently in the scene object information SEI message. Indicates that information exists. If this value is 0, it indicates that object-dependent information does not exist. (equal to 1 indicates that object depend ency information is present in the curre nt scene object information SEI message. soi_object_dependency_present_flag equa l to 0 indicates that object dependency information is not present).

[0812] Observable cones present flag (soi_visibility_cones_present t_flag): If this value is 1, the observable cone information is currently stored in the scene object information SEI menu. A value of 0 indicates that no observable cone information is present in the message. (equal to 1 indicates that visibility ty cones information is present in the c current scene object information SEI mess age. soi_visibility_cones_present_flag e qual to 0 indicates that visibility cone s information is not present).

[0813] 3D bounding box existence flag (soi_3d_bounding_box_ present_flag): If this value is 1, the 3D bounding box information is Indicates that it is currently present in the Scene Object Information SEI message. If this value is 0, then 3 Indicates that bounding box information is not available (equal to 1 ind icates that 3D bounding box information is present in the current scene object i nformation SEI message. soi_3d_bounding_ box_present_flag equal to 0 indicates th at 3D bounding box information is not pr esent).

[0814] Collision shape present flag (soi_collision_shape_present_f lag): If this value is 1, collision information is currently present in the Scene Object Information SEI message. If this value is 0, it indicates that collision pattern information does not exist (equ al to 1 indicates that collision informa tion is present in the current scene obj ect information SEI message. soi_collisi on_shape_present_flag equal to 0 indicat es that collision shape information is n ot present).

[0815] Point style present flag (soi_point_style_present_f lag): If this value is 1, the point style information is currently If this value is 0, no point style information is present. (equal to 1 indicates that point s style information is present in the curre nt scene object information SEI message. soi_point_style_present_flag equal to 0 indicates that point style information is not present).

[0816] Material ID present flag (soi_material_id_present_flag) : If this value is 1, the substance ID information is currently present in the Scene Object Information SEI message. A value of 0 indicates that no substance ID information is available (equal to to 1 indicates that material ID informa tion is present in the current scene obj ect information SEI message. soi_materia l_id_present_flag equal to 0 indicates t hat material ID information is not prese nt).

[0817] Extension present flag (soi_extension_present_flag): This value If is 1, it indicates that additional extension information is currently present in the Scene Object Information SEI message. A value of 0 indicates that no additional extension information is present (equal to 1 indicates that additional extension in formation shall be present in the curren t scene object information SEI message. soi_extension_present_flag equal to 0 in dicates that additional extension information ation is not present. of bitstream conformance to this versi on of the specification that soi_extensi on_present_flag shall be equal to 0).

[0818] If soi_3d_bounding_box_present_flag is 1, The following elements are included in the scene object information:

[0819] 3D Bounding Box Scale (soi_3d_bounding_box_s cale_log2): 3D bounding box parameters described for the object indicates the scale to be applied to applied to the 3D bounding box parameter s that may be specified for an object).

[0820] 3D Bounding Box Precision (soi_3d_bounding_box_pre cision_minus8): Adding 8 to this value increases the 3D balance described for the object. Indicates the accuracy of the bounding box parameters (plus 8 indicates he precision of the 3D bounding box para meters that may be specified for an object ct).

[0821] Maximum object index (soi_log2_max_object_idx_up dated): Signals the object index value in the current scene object information SEI message. specifies the number of bits used to ring er of bits used to signal the value of a n object index in the current scene obje ct information SEI message).

[0822] soi_object_dependency_present_flag is 1 The following elements are included in the scene object information:

[0823] Maximum Object Dependent Index (soi_log2_max_object_depend dency_idx): Dependent object index in the current scene object information SEI message specifies the number of bits used to signal the value of e number of bits used to signal the value e of a dependency object index in the cu rrent scene object information SEI messa ge).

[0824] The following elements are included in the scene object information: soi_num_object_updates Included in.

[0825] Object index (soi_object_idx[i]): the i-th object to be updated Used to indicate the object index of the object. The number of bits is soi_log2_max_object_idx_updated. If soi_object_idx[i] does not exist in the bitstream, The value is inferred to 0 (indicates the object index of the i-th object to be updated. r of bits used to represent soi_object_i dx[i] is equal to soi_log2_max_object_id x_updated. When soi_object_idx[i] is not present in the bitstream, its value sha ll be inferred to be equal to 0).

[0826] Object cancellation flag(soi_object_cancel_flag[i]):This If the value is 1, the object with index I is canceled and the variable ObjectTrack ed[i] indicates 0. Object label, 3D bounding box parameters , priority information, hidden flag, dependency information, observable cone, collision pattern, point style and All relevant parameters, including the name and substance ID, are reset to their default values. If soi_object_cancel_flag is 0, soi_object The object with the same index as _idx[i] is updated to the information following this element. The variable ObjectTracked[i] is set to 1 (equal to l to 1 indicates that the object with in dex equal to i shall be canceled and tha t the variable ObjectTracked[i] shall be set to 0. Furthermore, all of its assoc iated parameters, including the object l abel, 3D bounding box parameters, priori ty information, hidden flag, dependency information, visibility cones, collision shapes, point style, and material id, w ill be reset to their default values. sh all also be set equal to 0. soi_object_c ancel_flag equal to 0 indicates that the object with index equal to soi_object_i dx[i] shall be updated with information that follows this element and that the v ariable ObjectTracked[i] shal be set to 1).

[0827] soi_object_cancel_flag[k] is not 1, soi_obj If ect_label_present_flag is 1, the following elements are scene objects: Included in the information.

[0828] Object label update flag(soi_object_label_update_flag [i]): If this value is 1, the object label update information is applied to the object with index i. If this value is 0, it means that there is no object label update information. indicates (equal to 1 indicates that object labe l update information is present for an o bject with object index i. soi_object_la bel_update_flag[i] equal to 0 indicates that object label update information is not present).

[0829] If soi_object_label_update_flag[k] is 1, The following elements are included in the scene object information:

[0830] Object label index (soi_object_label_idx[i]): indicates the label index of the object with index i abel index of an object with index i).

[0831] If soi_priority_present_flag is 1, the following elements are It is included in the object information.

[0832] Priority update flag (soi_priority_update_flag[i]): If this value is 1, priority update information exists for the object with object index i. If this value is 0, it indicates that there is no object priority information (eq ual to 1 indicates that priority update information is present for an object wit h object index i. soi_priority_update_fl ag[i] equal to 0 indicates that object p riority information is not present).

[0833] If soi_priority_update_flag[k] is 1, the following elements is included in the scene object information.

[0834] Priority value (soi_priority_value[i]): with index i Indicates the priority of the object. priority of an object with index i. lower the priority value, the higher the priority).

[0835] If soi_object_hidden_present_flag is 1, These elements are included in the scene object information.

[0836] Object hidden flag(soi_object_hidden_flag[i]):This value If this value is 1, it indicates that the object with index i is hidden. If this value is 0, Indicates that an object with index i exists (equal to 1 indic ates that the object with index i shall be hidden. soi_object_hidden_flag[i] equ al to 0 indicates that the object with i index i shall become present).

[0837] soi_object_dependency_present_flag is 1 The following elements are included in the scene object information:

[0838] Object dependency update flag (soi_object_dependency_update_ flag[i]): If this value is 1, the object dependent update information has object index i. If this value is 0, there is no object-dependent update information. (equal to 1 indicates that object dependency update information is present for an object with object index i. soi_ object_dependency_update_flag[i] equal t o 0 indicates that object dependency upd information is not present).

[0839] soi_object_dependency_update_flag[k] is 1 If present, the following elements are included in the scene object information:

[0840] Number of object dependencies (soi_object_num_dependencies[i]): indicates the number of object dependents with index i r of dependencies of object with index i ).

[0841] soi_object_num_dependencies[k] elements with values ​​less than or equal to is included in the scene object information.

[0842] Object dependency index (soi_object_dependency_idx[i] [j]): the index of the jth object that has a dependency with the object with object index i indicates the index of the j-th bject that has a dependency with the obj ect with object index i).

[0843] soi_visibility_cones_present_flag is 1 The following elements are included in the scene object information:

[0844] Observable cones update flag (soi_visibility_cones_update _flag[i]): If this value is 1, the observable cone update information is If this value is 0, the observable cone update information indicates that vi does not exist (equal to 1 indicates that vi sibility cones update information is pre sent for an object with object index i. soi_visibility_cones_update_flag[i] equa l to 0 indicates that visibility cones u update information is not present).

[0845] soi_visibility_cones_update_flag[k] is 1 Then, the following elements are included in the scene object information:

[0846] Direction X (soi_direction_x[i]): Object with object index i indicates the normalized X component of the direction vector of the observability cone of The value is inferred to be equal to 1.0 (indicates the normalized x-component value of the direction vect or for the visibility cone of an object with object index i. The value of soi_di reaction_x[i], when not present, is assume d to be equal to 1.0).

[0847] Direction Y (soi_direction_y[i]): Object with object index i Indicates the normalized Y component of the direction vector of the observable cone of the object. If not present, indicates the normalize d y-component value of the direction vec tor for the visibility cone of an object with object index i. The value of soi_d irection_y[i], when not present, is assum ed to be equal to 1.0).

[0848] Direction Z (soi_direction_z[i]): Object with object index i Indicates the normalized Z component of the direction vector of the observability cone of the object. If not present, indicates the normalize d z-component value of the direction vec tor for the visibility cone of an object with object index i. The value of soi_d irection_z[i], when not present, is assum ed to be equal to 1.0).

[0849] Angle (soi_angle[i]): the angle of the observable cone by the process direction vector If not present, the value of soi_angle[i] is inferred to be equal to 180. (indicates the angle of the visibility c one along the direction vector in degree s. The value of soi_angle[i], when not p resent, is assumed to be equal to 180°).

[0850] If soi_3d_bounding_box_present_flag is 1, The following elements are included in the scene object information:

[0851] 3D bounding box update flag (soi_3d_bounding_box_ update_flag[i]): If this value is 1, the object with object index i If this value is 0, the 3D bounding box information is indicates that 3D bounding box information is present for an object with object index i. soi_3d_b sounding_box_update_flag[i] equal to 0 in dicates that 3D bounding box information is not present).

[0852] 3D bounding box X(soi_3d_bounding_box_x[i] ): The X coordinate value of the origin position of the 3D bounding box of the object with index i The default value of soi_3d_bounding_box_x[i] is equal to 0. (indicates the x coordinate value of the origin position of the 3D bounding box of an object with index i. The default v alue of soi_3d_bounding_box_x[i] is equa l to 0).

[0853] 3D bounding box Y(soi_3d_bounding_box_y[i] ): The Y coordinate value of the origin position of the 3D bounding box of the object with index i The default value of soi_3d_bounding_box_y[i] is equal to 0. (indicates the y coordinate value of the origin position of the 3D bounding box of an object with index i. The default v alue of soi_3d_bounding_box_y[i] is equa l to 0).

[0854] 3D bounding box Z(soi_3d_bounding_box_z[i] ): The Z coordinate value of the origin position of the 3D bounding box of the object with index i The default value for this value is equal to 0 (indicates the z coor dinate value of the origin position of t he 3D bounding box of an object with ind ex i. The default value of soi_3d_boundi ng_box_z[i] is equal to 0).

[0855] 3D bounding box delta X (soi_3d_bounding_box_d elta_x[i]): The bounding box on the X axis of the object with index i The default value for this value is 0 (indicates the size of the size of the bounding box on the x axis of an object with index i. The default val ue of soi_3d_bounding_box_delta_x[i] is equal to 0).

[0856] 3D bounding box delta Y (soi_3d_bounding_box_d elta_y[i]): The bounding box on the Y axis of the object with index i The default value for this value is 0 (indicates the size of the size of the bounding box on the y axis of an object with index i. The default val ue of soi_3d_bounding_box_delta_y[i] is equal to 0).

[0857] 3D Bounding Box Delta Z (soi_3d_bounding_box_d elta_z[i]): The bounding box on the Z axis of the object with index i The default value for this value is 0 (indicates the size of the size of the bounding box on the z axis of an object with index i. The default val ue of soi_3d_bounding_box_delta_z[i] is equal to 0).

[0858] If soi_collision_shape_present_flag is 1, The following elements are included in the scene object information:

[0859] Collision shape update flag (soi_collision_shape_update_fl ag[i]): If this value is 1, the collision pattern update information is sent to the object with index i. If this value is 0, it means that there is no collision pattern update information. indicates that collision s hape update information is present for a n object with object index i. soi_collis ion_shape_update_flag[i] equal to 0 indi cates that collision shape update inform ation is not present).

[0860] soi_collision_shape_update_flag[k] is 1 Then, the following elements are included in the scene object information:

[0861] Collision pattern ID (soi_collision_shape_id[i]): Index indicates the collision pattern ID of the object with the given point i on shape id of an object with index i).

[0862] If soi_point_style_present_flag is 1, the following elements The elements are included in the scene object information.

[0863] Point style update flag (soi_point_style_update_fl ag[i]): If this value is 1, the point style update information with index i If this value is 0, the point style update information is indicates that it does not exist (equal to 1 indicates that nt style update information is present f or an object with object index i. soi_po int_style_update_flag[i] equal to 0 indi categories that point style update information n is not present).

[0864] If soi_point_style_update_flag[k] is 1, The following elements are included in the scene object information:

[0865] Point shape ID (soi_point_shape_id[i]): index i The default value of this value is 0 (indicating cates the point shape id of an object wi th index i. The default value of soi_poi nt_shape_id[i] is equal to 0).

[0866] Point size (soi_point_size[i]): the customer with index i Indicates the body point size. The default value for this value is equal to 1 (indicates the point size of an object with index i .The default value of soi_point_size[i] is equal to 1).

[0867] If soi_material_id_present_flag is 1, the following elements are The elements are included in the scene object information.

[0868] Material ID update flag (soi_material_id_update_flag[i ]): indicates that substance ID update information exists for the object with object index i A value of 0 indicates that there is no point style update information (equal to to 1 indicates that material ID update information is present for an object wit h object index i. soi_point_style_update _flag[i] equal to 0 indicates that point style update information is not present ).

[0869] If soi_material_id_update_flag[k] is 1, These elements are included in the scene object information.

[0870] Material ID (soi_material_id[i]): The ID of the object with index i Indicates the substance ID. The default value for this value is equal to 0 (indicates the m aterial ID of an object with index i. Th e default value of soi_material_id[i] is equal to 0).

[0871] FIG. 47 shows the object label information according to the embodiment. ion).

[0872] Figure 47 shows the sample stream N included in the bit stream 32000 as shown in Figure 32. The syntax of the object label information for the SEI message of the AL unit is shown below.

[0873] Cancellation flag (oli_cancel_flag): If this value is 1, the object label The object label information SEI message is output in the order of the remaining object label information SEI messages. Cancel the existence of the object (equal to 1 indicates that the object ect label information SEI message cancel s the persistence of any previous object label information SEI message in output order).

[0874] If oli_cancel_flag is not 1, the following elements are included in the object label information: can be.

[0875] Label Language Present Flag (oli_label_language_present_f lag): If this value is 1, the label language information is included in the object label information SEI message. If this value is 0, it means that label language information is not present (e qual to 1 indicates that label language information is present in the object lab el information SEI message. oli_label_la nguage_present_flag equal to 0 indicates that label language information is not present).

[0876] If oli_label_language_present_flag is 1, the following The following elements are included in the object label information:

[0877] Bit equal to zero (oli_bit_equal_to_zero): This value is equal to 0 It's nice.

[0878] Label Language (oli_label_language): null-terminated, e.g. 0x00 Contains a terminal byte followed by a language tag as described in IETF RFC 5646. The length of the _label_language syntax element includes the null termination byte. Not equal to or less than 255 times (contains a language tag as specified by IETF RFC 5646 followed by a null termination byte equal to 0x00 .The length of the oli_label_language s yntax element shall be less than or equal l to 255 bytes, not including the null t ermination byte).

[0879] Number of label updates (oli_num_label_updates): Currently updated by SEI indicates the number of labels to be updated els that are to be updated by the curren t SEI).

[0880] Depending on the oli_num_label_updates value, the following elements are added to the object label information: Included.

[0881] Label index (oli_label_idx[i]): the ith label to be updated indicates the label index of the i-th label to be updated).

[0882] Label Cancellation Flag (oli_label_cancel_flag): This value is 1 Then, the label with index like oli_label_idx[i] is taken. If this value is 0, then oli Label information with indexes such as _label_idx[i] follows this element (equal to 1 indicates that the e label with index equal to oli_label_id x[i] shall be canceled and set equal to an empty string. oli_label_cancel_flag e qual to 0 indicates that the label with index equal to oli_label_idx[i] shall be updated with information that follows t h...

Claims

1. A point cloud data transmission method, comprising: encoding geometry data of the point cloud data; encoding attribute data of the point cloud data; encoding an occupancy map of the point cloud data; encapsulating the encoded occupancy map, the encoded geometry data, and the encoded attribute data in a file; and transmitting said file; the file includes a first component track including the occupancy map, a second component track including the geometry data, and a third component track including the attribute data; the file further includes a track and a sample entry for the track containing spatial domain information for a spatial domain partial approximation to the point cloud data; The spatial domain information includes an identifier of a spatial domain for the point cloud data and position information of the spatial domain; The transmission method, wherein the spatial domain information further includes information indicating a number of atlas tiles associated with the spatial domain of the point cloud data, and tile identifier information for identifying the atlas tiles.

2. The transmission method of claim 1 , wherein the spatial domain information comprises static or dynamic information depending on time.

3. 1. An apparatus for receiving and encoding point cloud data, comprising: A processor; and a memory storing one or more instructions for executing said processor and operating said device; The instructions include: receiving a file including a first component track including an occupancy map of the point cloud data, a second component track including geometry data of the point cloud data, and a third component track including attribute data of the point cloud data; decapsulating the file; and encoding said geometry data, said characteristic data, and said occupancy map; the file further includes a track and a sample entry for the track containing spatial domain information for a spatial domain partial approximation to the point cloud data; The spatial domain information includes an identifier of a spatial domain for the point cloud data and position information of the spatial domain; The apparatus, wherein the spatial region information includes information indicating a number of atlas tiles associated with the spatial region of the point cloud data, and tile identifier information for identifying the atlas tiles.

4. The apparatus of claim 3 , wherein the spatial domain information comprises static or dynamic information over time.

5. An apparatus for encoding and transmitting point cloud data, comprising: A processor; and a memory storing one or more instructions for executing said processor and operating said device; The instructions include: encoding geometry data of the point cloud data; encoding attribute data of the point cloud data; encoding an occupancy map of the point cloud data; encapsulating the encoded occupancy map, the encoded geometry data, and the encoded attribute data in a file; and transmitting said file; the file includes a first component track including the occupancy map, a second component track including the geometry data, and a third component track including the attribute data; the file further includes a track and a sample entry for the track containing spatial domain information for a spatial domain partial approximation to the point cloud data; The spatial domain information includes an identifier of a spatial domain for the point cloud data and position information of the spatial domain; The apparatus, wherein the spatial region information further includes information indicating a number of atlas tiles associated with the spatial region of the point cloud data, and tile identifier information for identifying the atlas tiles.

6. The apparatus of claim 5 , wherein the spatial domain information comprises static or dynamic information over time.

7. A method for receiving point cloud data, comprising: receiving a file including a first component track including an occupancy map of the point cloud data, a second component track including geometry data of the point cloud data, and a third component track including attribute data of the point cloud data; decapsulating the file; and decoding said geometry data, said attribute data, and said occupancy map; the file further includes a track and a sample entry for the track containing spatial domain information for a spatial domain partial approximation to the point cloud data; The spatial domain information includes an identifier of a spatial domain for the point cloud data and position information of the spatial domain; The receiving method, wherein the spatial domain information further includes information indicating a number of atlas tiles associated with the spatial domain of the point cloud data, and tile identifier information for identifying the atlas tiles.

8. The receiving method according to claim 7 , wherein the spatial domain information comprises static or dynamic information depending on time.

Images