Mesh data transmitting apparatus, mesh data transmitting method, mesh data receiving apparatus, and mesh data receiving method

Abstract

The mesh data decoding method according to embodiments can include a step of receiving a base mesh bitstream, a displacement vector bitstream, a texture map bitstream, and signaling information; a base mesh processing step of recovering a base mesh from the base mesh bitstream; a displacement information processing step of recovering displacement information from the displacement vector bitstream; a recovery step of recovering a mesh based on the base mesh and the displacement information; and a texture map processing step of recovering a texture map from the texture map bitstream.

Description

Technical Field

[0001] The implementation provides a method for delivering 3D content to provide users with various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and autonomous driving services. Background Technology

[0002] Point cloud data or mesh data in 3D content is a collection of points in 3D space. However, due to the large number of points in 3D space, it is difficult to create point cloud data or mesh data.

[0003] In other words, high throughput is required to send and receive 3D data with a considerable number of points (e.g., point cloud or grid data). Summary of the Invention

[0004] Technical issues

[0005] The purpose of this disclosure is to provide an apparatus and method for efficiently sending and receiving grid data to solve the aforementioned problems.

[0006] Another object of this disclosure is to provide an apparatus and method for addressing latency and encoding / decoding complexity in grid data.

[0007] Another object of this disclosure is to provide an apparatus and method for efficiently performing the encoding and decoding of displacement vectors.

[0008] The implementation methods are not limited to the above objectives, and the scope of the implementation methods can be extended to other objectives that can be inferred by those skilled in the art based on the entire contents of this disclosure.

[0009] Technical solution

[0010] To achieve these and other advantages, and in accordance with the purposes of this disclosure, as implemented and broadly described herein, a method for decoding mesh data may include: receiving a base mesh bitstream, a displacement vector bitstream, a texture map bitstream, and signaling information; a base mesh processing operation to reconstruct the base mesh from the base mesh bitstream; a displacement information processing operation to reconstruct displacement information from the displacement vector bitstream; a reconstruction operation to reconstruct the mesh based on the base mesh and the displacement information; and a texture map processing operation to reconstruct the texture map from the texture map bitstream.

[0011] According to an implementation, the texture map processing operation may include: decoding the texture map bitstream based on signaling information and reconstructing the texture map; checking whether at least one texture map is skipped from the texture map bitstream based on signaling information; and generating at least one skipped texture map based on signaling information and at least one reference frame based on the detection that at least one skipped texture map exists.

[0012] According to an implementation, the skip unit of the at least one texture map can be at least one of a frame, patch, submesh, tile, or slice.

[0013] According to an implementation, the signaling information may include information for identifying whether to skip the at least one texture map and reference texture map information associated with the at least one reference frame.

[0014] According to an embodiment, an apparatus for decoding mesh data may include: a receiver configured to receive a base mesh bitstream, a displacement vector bitstream, a texture map bitstream, and signaling information; a base mesh processor configured to reconstruct a base mesh from the base mesh bitstream; a displacement information processor configured to reconstruct displacement information from the displacement vector bitstream; a reconstructor configured to reconstruct the mesh based on the base mesh and the displacement information; and a texture map processor configured to reconstruct a texture map from the texture map bitstream.

[0015] According to the implementation, the texture map processor can be configured to decode and reconstruct the texture map based on signaling information, check whether at least one texture map is skipped from the texture map bitstream based on the signaling information, and generate at least one skipped texture map based on the signaling information and at least one reference frame based on the detection of at least one skipped texture map.

[0016] According to an implementation, the skip unit of the at least one texture map can be at least one of a frame, patch, submesh, tile, or slice.

[0017] According to an implementation, the signaling information may include information for identifying whether to skip the at least one texture map and reference texture map information associated with the at least one reference frame.

[0018] According to an implementation, a method for encoding grid data may include: encoding the original grid; and transmitting a bit stream containing the encoded grid and signaling information.

[0019] According to the implementation, the encoding may include: a base grid processing operation that generates a base grid bitstream by encoding a base grid generated by simplifying the original grid; a displacement information processing operation that generates a displacement vector bitstream by encoding displacement information generated based on the base grid; a grid reconstruction operation that reconstructs the grid based on the encoded base grid and the encoded displacement information; and a texture map processing operation that determines whether at least one texture map among the texture maps generated based on the original grid and the reconstructed grid is skipped and generates a texture map bitstream by encoding the texture map that is not skipped.

[0020] According to an implementation, the texture map processing operation may include: comparing the similarity between the current texture map and a reference texture map before encoding the generated texture map, and determining whether the current texture map is skipped.

[0021] According to the implementation, the texture map processing operation may further include: skipping the encoding and transmission of the current texture map based on the difference between the peak signal-to-noise ratio (PSNR) calculated based on the current texture map and the PSNR calculated based on the reference texture map being less than a preset threshold.

[0022] According to an implementation, the skip unit of the at least one texture map can be at least one of a frame, patch, submesh, tile, or slice.

[0023] According to an implementation, the signaling information may include information for identifying whether the at least one texture map is skipped, and reference texture map information associated with at least one reference frame used to generate the skipped texture map.

[0024] According to an implementation, the reference texture map information may include information for identifying the orientation of the at least one reference frame.

[0025] According to the implementation method, a computer program stored in a computer-readable recording medium can be used in conjunction with a computer as hardware to execute the above-described method.

[0026] Beneficial effects

[0027] The implementation may provide a grid data transmission method, a grid data transmission device, a grid data reception method, and a grid data reception device capable of providing high-quality 3D services.

[0028] The implementation methods can provide grid data transmission methods, grid data transmission devices, grid data reception methods, and grid data reception devices capable of implementing various video codec schemes.

[0029] Implementations may provide a grid data transmission method, a grid data transmission device, a grid data reception method, and a grid data reception device capable of providing general 3D content such as autonomous driving services.

[0030] The implementation method can provide a mesh data transmission method, mesh data transmission device, mesh data reception method, and mesh data reception device as follows: when the similarity between the reference texture map of the dynamic mesh and the current texture map is high, the encoding and transmission of the current texture map are skipped, thereby reducing the number of bits of the transmitted texture map, increasing the transmission speed, and reducing the complexity of the encoder and decoder. Attached Figure Description

[0031] The accompanying drawings are included to provide a further understanding of this disclosure and are incorporated in and constitute a part of this application. The drawings illustrate embodiments of the disclosure and, together with the description, serve to illustrate the principles of the disclosure. For a better understanding of the various embodiments described below, reference should be made to the following description of embodiments in conjunction with the accompanying drawings. The same reference numerals will be used throughout the drawings to denote the same or similar parts. In the drawings: Figure 1 A system for providing dynamic mesh content is shown according to an embodiment; Figure 2 A V-MESH compression method according to an embodiment is shown; Figure 3 Preprocessing in V-MESH compression according to an embodiment is shown; Figure 4 The method for subdividing the edge midpoint according to an embodiment is shown; Figure 5 The displacement generation process according to the embodiment is shown; Figure 6 The intra-frame coding process of V-MESH data according to an implementation method is shown; Figure 7 The inter-frame coding process for V-MESH data according to an implementation method is illustrated; Figure 8 The process of displacement enhancement according to the embodiment is shown; Figure 9 This illustrates the process of packing transform coefficients into a 2D image according to an embodiment; Figure 10 The attribute transfer process in the V-MESH compression method according to an embodiment is illustrated; Figure 11 The intra-frame decoding process of V-MESH data according to an embodiment is shown; Figure 12 The inter-frame decoding process of V-MESH data according to an embodiment is illustrated; Figure 13 A transmitting device according to an embodiment is shown; Figure 14 A receiving device according to an embodiment is shown; Figure 15 Another example of a transmitting device according to an embodiment is shown; Figure 16 This is a flowchart illustrating an example of a method for determining whether to skip texture map encoding according to an implementation method; Figure 17 An example of a method for determining whether to skip a texture map using the maximum texture map distance parameter according to an implementation is shown; Figure 18An example of a texture map skipping state and reference frame index signaled on a per-submesh basis is shown according to an embodiment; Figure 19 This is a diagram illustrating an example of the relationship between frames of referenced texture maps when texture map encoding is skipped on a per-submesh basis, according to an embodiment. Figure 20 This is a diagram illustrating another example of a receiving device according to an embodiment; Figure 21 This is an example detailed block diagram of a texture map decoder according to an implementation method; Figure 22 An example is shown where the texture map skip flag is deduced to be 0 according to the implementation method; Figure 23 This is a flowchart illustrating an example of deriving a texture map skip flag from a texture map skip flag determiner according to an embodiment; Figure 24 An example is shown of a texture map skipping state and reference frame index notified by a signal on a sub-mesh basis, according to an embodiment. Figure 25 This is a diagram illustrating an example of the frame relationship of the texture maps referenced when skipping texture map encoding on a sub-mesh basis according to an embodiment; Figure 26 An example of the syntax structure of the Atlas Frame Parameter Set (AFPS) according to an implementation is shown; Figure 27 This is a table showing examples of texture map skip states and reference directions for afps_texture_skip_refDirection_idx according to an implementation; Figure 28 An example of the syntax structure of the atlas block header according to an implementation method is shown; Figure 29 An example of the syntax structure of atlas tile data units according to an implementation is shown; Figure 30 This is a table illustrating an example of determining the patch pattern based on an identifier when the encoding type of the current atlas tile is I_TILE, according to an implementation method; Figure 31 This is a table illustrating an example of determining the patch pattern based on an identifier when the encoding type of the current atlas tile is P_TILE, according to an implementation method; Figure 32 An example of the syntax structure of the subgrid header according to an implementation is shown; Figure 33 Another example of the syntax structure of the subgrid header according to the implementation is shown; Figure 34Another example of the syntax structure of the Atlas Sequence Parameter Set (ASPS) according to an implementation is shown; Figure 35 Another example of the syntax structure of AFPS according to the implementation is shown; Figure 36 This is a table showing the definitions of afve_texturemap_skip_flag and afve_ref_direction_idx according to the implementation method; Figure 37 This is a diagram illustrating an example of texture map skipping according to an implementation method; Figure 38 (a) and Figure 38 (b) shows an example of determining the texture map skip state and reference information index on a frame-by-frame basis according to an implementation method; Figure 39 An example of the syntax structure of the atlas tile layer according to an implementation method is shown; Figure 40 An example of the syntax structure of the atlas block header according to an implementation method is shown; Figure 41 An example of the syntax structure of patch information data according to an embodiment is shown; Figure 42 An example of the syntax structure of a patch data unit according to an embodiment is shown; Figure 43 An example of the syntax structure of inter-patch data units according to an embodiment is shown; Figure 44 An example of the semantics of the NAL unit header according to an implementation is shown; Figure 45 An example of the syntax structure of atlas_frame_rbsp() according to the implementation is shown; Figure 46 This is a flowchart illustrating an example of a transmission method according to an embodiment; and Figure 47 This is a flowchart illustrating an example of a receiving method according to an implementation. Detailed Implementation

[0032] Preferred embodiments of the present disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. The detailed description given below with reference to the drawings is intended to illustrate exemplary embodiments of the present disclosure, and not to show only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0033] Although most of the terms used in this disclosure are chosen from general terms widely used in the art, some terms were arbitrarily chosen by the applicant, and their meanings are detailed in the following description as needed. Therefore, this disclosure should be understood based on the intended meaning of the terms rather than their simple names or meanings.

[0034] With recent advancements in 3D data modeling and rendering technologies, active research has been conducted across various fields to generate and process 3D data, including virtual reality (VR), augmented reality (AR), autonomous driving, computer-aided design (CAD) / computer-aided manufacturing (CAM), and geographic information systems (GIS). 3D data can be represented as point clouds or meshes depending on the representation format. A mesh consists of geographic information indicating the coordinates of individual vertices or points, connectivity information indicating connections between vertices, a texture map representing color information about the mesh surface as 2D image data, and texture coordinates indicating the mapping information between the mesh surface and the texture map. In this disclosure, a mesh is defined as a dynamic mesh when at least one element constituting the mesh changes over time, and as a static mesh when it does not change.

[0035] Compared to 2D image data, dynamic mesh data involves a significantly larger amount of element data to represent the mesh. As a result, techniques have been developed for efficiently compressing large amounts of mesh data for storage and transmission.

[0036] Figure 1 A system for providing dynamic mesh content is shown according to an embodiment.

[0037] Figure 1 The system includes a transmitting device 100 and a receiving device 110. The transmitting device 100 may include a grid video acquisition unit (or part thereof) 101, a grid video encoder 102, a file / fragment encapsulator 103, and a transmitter 104. The receiving device 110 may include a receiver 111, a file / fragment decapsulator 112, a grid video decoder 113, and a renderer 114. Figure 1 The various components may correspond to hardware, software, processors, and / or combinations thereof. In the following description, the grid data transmitting device according to the embodiments may be interpreted as a 3D data transmitting device or transmitting device 100, or as a grid video encoder (hereinafter referred to as encoder) 102. The grid data receiving device according to the embodiments may be interpreted as a 3D data receiving device or receiving device 110, or as a grid video decoder (hereinafter referred to as decoder) 113.

[0038] Figure 1 The system can perform video-based dynamic grid compression and decompression.

[0039] With advancements in 3D capture, modeling, and rendering, users are now allowed to access various forms of 3D content (e.g., AR, XR, metaverse, and holograms) across multiple platforms and devices. 3D content is becoming increasingly sophisticated and realistic in its object representation to provide users with immersive experiences. However, the generation and use of 3D models require a significant amount of data. Among various types of 3D content, 3D meshes are widely used for efficient data utilization and realistic object representation. Implementation methods involve a series of processing steps within a system that utilizes mesh content.

[0040] First, methods for compressing dynamic mesh data begin with the Video-Based Point Cloud Compression (V-PCC) standard for point cloud data. Point cloud data is data containing color information in the coordinates (X, Y, Z) of vertices (or points). In this disclosure, vertex coordinates (i.e., position information) are referred to as geometric information, and color information about vertices is referred to as attribute information. Geometric information and attribute information together are referred to as vertex information or point cloud data. Mesh data refers to vertex information that includes connection information between vertices. Content can be initially created in the form of mesh data. Alternatively, connection information can be added to point cloud data, and the point cloud data can be transformed into mesh data.

[0041] Currently, the MPEG standards group defines two data types for dynamic mesh data: Category 1 mesh data with texture maps as color information and Category 2 mesh data with vertex colors as color information.

[0042] The grid coding standard for Category 1 data is currently underway, with the standardization of Category 2 data expected to follow. Figure 1 As shown, the overall process for providing grid content services may include acquisition, encoding, transmission, decoding, rendering, and / or feedback processes.

[0043] To provide mesh content services, 3D data acquired through multiple cameras or specialized cameras can be processed into a mesh data type through a series of steps to generate mesh video. The generated mesh video can then be sent via a series of operations, and the receiving side can process the received data back into the mesh video for rendering. Through this process, mesh video can be provided to users, allowing them to interactively utilize the mesh content according to their intentions.

[0044] like Figure 1 As shown, the grid compression system may include a transmitting device 100 and a receiving device 110. The transmitting device 100 may encode the grid video to output a bitstream, which may be transmitted to the receiving device 110 in the form of a file or a stream (stream segment) via a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD.

[0045] In transmitting device 100, the encoder may be referred to as a grid video / image / picture / frame encoding device. In receiving device 110, the decoder may be referred to as a grid video / image / picture / frame decoding device. The transmitter may be included in the grid video encoder, and the receiver may be included in the grid video decoder. Renderer 114 may include a display, and the renderer and / or display may be configured as separate devices or external components. Transmitting device 100 and receiving device 110 may also include separate internal or external modules / units / components for the feedback process.

[0046] Mesh data uses multiple polygons to represent the surface of an object. Each polygon is defined by vertices in 3D space and connection information indicating how the vertices are connected. Additionally, vertex attributes such as color and normals can be included in the data. Mapping information that allows the mesh surface to be mapped onto a 2D plane can also be included in the mesh attributes. Mapping is typically described using a set of parametric coordinates (called UV coordinates or texture coordinates) associated with the mesh vertices. The mesh contains a 2D attribute map, which can be used to store high-resolution attribute information such as texture, normals, and displacement. Here, displacement can be used interchangeably with displacement information or displacement vectors.

[0047] The mesh video acquisition unit 101 may include processing 3D object data acquired by a camera or the like into a mesh data type with the aforementioned attributes through a series of operations, and generating a video composed of the mesh data. In the mesh video, the attributes of the mesh (e.g., vertices, polygons, connections between vertices, color, and normals) may change over time. Mesh videos with time-varying attributes and connection information are called dynamic mesh videos.

[0048] The mesh video encoder 102 can encode input mesh video into one or more video streams. The video may contain multiple frames, each corresponding to a still image / picture. In this disclosure, mesh video may include mesh images / frames / pictures. The term "mesh video" is used interchangeably with mesh images / frames / pictures. The mesh video encoder 102 can perform a video-based dynamic mesh (V-Mesh) compression process. For compression and encoding efficiency, the mesh video encoder 102 can perform a series of processes such as prediction, transform, quantization, and entropy coding. The encoded data (encoded video / image information) can be output as a bitstream.

[0049] The file / fragment encapsulation module 103 can encapsulate encoded grid video data and / or grid video-related metadata in the form of files, etc. The grid video-related metadata can be received from a metadata processor. The metadata processing unit can be included in the grid video encoder 102, or it can be configured as a separate component / module. The file / fragment encapsulation module 103 can encapsulate the data into a file format such as ISOBMFF, or process it into a form such as DASH fragments. Depending on the implementation, the file / fragment encapsulator 103 can include grid video-related metadata in a file format. For example, grid video metadata can be included in various frames in an ISOBMFF file format, or as data on a separate track within a file. In some implementations, the file / fragment encapsulator 103 can encapsulate grid video-related metadata into a file.

[0050] The transmission processor can apply processing to the encapsulated mesh video data based on a file format for transmission. The transmission processor can be included in transmitter 104 or implemented as a separate component / module. The transmission processor can process the mesh video data according to any transmission protocol. Processing for transmission can include transmission processing via broadcast networks and transmission processing via broadband. In some embodiments, the transmission processor can receive mesh video-related metadata and mesh video data from a metadata processor and process them for transmission.

[0051] Transmitter 104 can transmit encoded video / image information or data output as a bitstream to receiver 111 of receiving device 110 via digital storage medium or network in the form of a file or stream. Digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. Transmitter 104 may include elements for generating media files according to a predetermined file format and may include elements for transmission via broadcast / communication network. Receiver 111 can extract the bitstream and transmit it to a decoding device.

[0052] Receiver 111 can receive grid video data transmitted by the grid data transmitting device. Depending on the channel used for transmission, receiver 111 can receive grid video data via a broadcast network or a broadband network, or via a digital storage medium.

[0053] The receiving processor can perform processing on the received mesh video data according to the transmission protocol. The receiving processor can be included in the receiver 111 or configured as a separate component / module. To correspond to the processing performed on the transmitting side for transmission, the receiving processor can perform the reverse process of the operations of the transmitting processor described above. The receiving processor can transmit the acquired mesh video data to the file / fragment decapsulator 112 and transmit the acquired mesh video-related metadata to the metadata parser. The mesh video-related metadata acquired by the receiving processor can be in the form of a signaling table.

[0054] File / fragment decapsulator 112 decapsulates mesh video data received from a receiving processor in file format. File / fragment decapsulator 112 can decapsulate files such as ISOBMFF to obtain a mesh video bitstream or mesh video-related metadata (metadata bitstream). The obtained mesh video bitstream can be transmitted to mesh video decoder 113, and the obtained mesh video-related metadata (metadata bitstream) can be transmitted to a metadata processor. The mesh video bitstream may include metadata (metadata bitstream). The metadata processor may be included in mesh video decoder 113 or configured as a separate component / module. The mesh video-related metadata obtained by file / fragment decapsulator 112 may be in the form of frames or tracks in a file format. When needed, file / fragment decapsulator 112 may receive metadata required for decapsulation from the metadata processor. The mesh video-related metadata may be transmitted to mesh video decoder 113 for use during mesh video decoding, or transmitted to renderer 114 for use during mesh video rendering.

[0055] The mesh video decoder 113 can receive the input bitstream and perform the inverse operation corresponding to the operation of the mesh video encoder 102 to decode the video / image. The decoded mesh video / image can be displayed on the display of the renderer 114. Users can view all or part of the rendering results through VR / AR displays, general displays, etc.

[0056] The feedback process may include sending various types of feedback information available during rendering / display operations to the decoder on the sending or receiving side. The feedback process can provide interactivity when consuming mesh video. In some implementations, the feedback process may include sending head orientation information, viewport information indicating the area the user is currently viewing, etc. In some implementations, the user may interact with objects implemented in a VR / AR / MR / autonomous driving environment. In this case, information related to the interaction may be transmitted to the sending side or service provider during the feedback process. In some implementations, the feedback process may be skipped.

[0057] Head orientation information refers to information about the user's head position, angle, movement, etc. Based on this information, information about the area the user is currently viewing within the grid video (i.e., viewport information) can be calculated.

[0058] Viewport information can be about the area a user is currently viewing in a mesh video. This information can be used to perform gaze analysis to determine how the user consumes the mesh video, how long the user views a specific area of ​​the mesh video, and so on. Gaze analysis can be performed on the receiving side, and the results can be transmitted to the sending side via a feedback channel. Devices such as VR / AR / MR displays can extract the viewport area based on the user's head position / orientation, the vertical or horizontal FOV supported by the device, etc.

[0059] In some implementations, the aforementioned feedback information may not only be transmitted to the transmitter but also consumed at the receiving end. In other words, operations such as decoding and rendering can be performed at the receiving end based on the aforementioned feedback information. For example, based on head orientation information and / or viewport information, only the grid video of the area currently being viewed by the user can be preferentially decoded and rendered.

[0060] This disclosure relates to embodiments of dynamic mesh video compression as described above. The methods / implementations disclosed herein can be applied to the Moving Picture Experts Group (MPEG) video-based dynamic mesh compression (V-Mesh) standard or any next-generation video / image coding standard. Dynamic mesh video compression is a method for processing time-varying mesh connectivity information and attributes. It can perform lossy and lossless compression for a variety of applications such as real-time communication, storage, free-viewpoint video, and AR / VR.

[0061] The dynamic mesh video compression method described below is based on the MPEG V-mesh method.

[0062] In this disclosure, a picture / frame can generally refer to a unit representing an image at a specific time.

[0063] A pixel or cell can refer to the smallest unit that makes up an image (or video). Additionally, the term "sample" can be used as the corresponding term for a pixel. A sample typically indicates a pixel or pixel value. It may indicate only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component, or only the pixel / pixel value of the depth component.

[0064] A unit can represent a basic unit of image processing. A unit may include at least one of a specific region of the image and information related to that region. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region." Typically, an M×N block may include a set (or array) of samples (or a sample array) consisting of M columns and N rows, or transform coefficients.

[0065] As mentioned above, Figure 1 The encoding process is executed as follows.

[0066] In other words, the video-based dynamic mesh compression (V-Mesh) method provides a way to compress dynamic mesh video data based on 2D video codecs (e.g., High Efficiency Video Surface (HEVC) and Multi-Function Video Coding (VVC)). During V-Mesh compression, the following data is received as input and compressed.

[0067] Input mesh: Includes the 3D coordinates of the vertices constituting the mesh, normal information for each vertex, mapping information for mapping the mesh surface to a 2D plane, and connections between the vertices constituting the surface. The mesh surface can be represented by triangles or other polygons, and the connection information between the vertices constituting the surface is stored according to a predetermined shape. The input mesh can be stored in OBJ file format.

[0068] Attribute graphs (texture graphs are used interchangeably below) contain information about the mesh's properties (color, normals, displacement, etc.) and are stored as mappings of the mesh surface to a 2D image. The mapping indicating which part of the mesh (surface or vertex) corresponds to a data point in the attribute graph is based on the mapping information contained in the input mesh. Because attribute graphs contain data about individual frames of a mesh video, they can also be called attribute graph videos. In V-Mesh compression methods, attribute graphs primarily contain color information about the mesh and are stored in image file formats (PNG, BMP, etc.).

[0069] Material library file: Contains information on the material properties used in the mesh, especially information linking the input mesh to the corresponding property map. It is stored in the wavefront material template library (MTL) file format.

[0070] In the V-Mesh compression method, the following data and information can be generated through the compression process.

[0071] Base Mesh: Objects in the input mesh are represented by the minimum vertices determined according to user criteria, which are simplified through a preprocessing process.

[0072] Displacement: Used to represent the displacement information of the input mesh as closely as possible to the base mesh, expressed in 3D coordinates.

[0073] Atlas information: Metadata required to reconstruct the mesh using base mesh, displacement, and property map information. It can be used to generate and utilize sub-cells (sub-mesh, patches, etc.) of the mesh.

[0074] Reference Figures 2 to 7 Describe the method for encoding mesh position information (or vertex position information), refer to Figures 6 to 10Methods for reconstructing grid location information to encode attribute information (attribute graphs) are described.

[0075] Figure 2 A V-MESH compression method according to an embodiment is shown.

[0076] Figure 2 Show Figure 1 The encoding process, which may include a preprocessing process and an encoding process. For example... Figure 2 As shown, Figure 1 The mesh video encoder 102 may include a preprocessor 200 and an encoder 201. Additionally, Figure 1 The transmitting device can be broadly referred to as an encoder. Figure 1 The mesh video encoder 102 can be referred to as an encoder. For example... Figure 2 As shown, the V-Mesh compression method may include preprocessing 200 and encoding 201. Figure 2 The preprocessor 200 can be located in Figure 2 The front end of encoder 201. Figure 2 The preprocessor 200 and encoder 201 can be referred to as a single encoder.

[0077] The preprocessor 200 can receive static or dynamic meshes (M(i)) and / or property maps (A(i)). The preprocessor 200 can generate a base mesh m(i) and / or displacement d(i) through preprocessing. The preprocessor 200 can receive feedback information from the encoder 201 and can generate the base mesh and / or displacement based on the feedback information.

[0078] Encoder 201 can receive a base grid m(i), a displacement d(i), a static or dynamic grid M(i), and / or a property map A(i). In this disclosure, at least one of the base grid m(i), the displacement d(i), the static or dynamic grid M(i), and / or the property map A(i) may be referred to herein as grid-related data. Encoder 201 can encode the grid-related data to generate a compressed bitstream.

[0079] Figure 3 Preprocessing in V-MESH compression according to an embodiment is shown.

[0080] Figure 3 Show Figure 2 The configuration and operation of the preprocessor. Figure 3 In the input mesh, the input mesh may include a static or dynamic mesh M(i) and / or a property graph A(i). The input mesh may also include the 3D coordinates of the vertices that make up the mesh, normal information about each vertex, mapping information for mapping the mesh surface to a 2D plane, and connection information between the vertices that make up the surface.

[0081] Figure 3 The process of performing preprocessing on the input mesh is illustrated. Preprocessing 200 may include four operations: 1) GoF group generation, 2) mesh simplification, 3) UV parameterization, and 4) fitting a subdivision surface (300). Depending on the implementation, GoF generation may be referred to as a GoF generation process or a GoF generator, mesh simplification may be referred to as a mesh simplification process or a mesh simplification section, UV parameterization may be referred to as a UV parameterization process or a UV parameterization section, and fitting a subdivision surface may be referred to as a fitting subdivision surface process or a fitting subdivision surface section. Preprocessor 200 may generate a displacement and / or base mesh from the received input mesh and transmit it to encoder 201. Preprocessor 200 may transmit GoF information related to GoF generation to encoder 201.

[0082] The following describes Figure 3 Each operation.

[0083] GoF generation: The process of generating a reference structure for mesh data. A previous frame can be set as the reference frame when the mesh of the previous frame and the current mesh have the same number of vertices, the same number of texture coordinates, the same vertex connectivity, and the same texture coordinate connectivity. In other words, if the current input mesh and the reference input mesh differ only in vertex coordinate values, encoder 201 can perform inter-frame coding. Otherwise, it performs intra-frame coding for each frame.

[0084] Mesh simplification: The process of simplifying an input mesh to create a simplified mesh (called the base mesh). Vertices to be removed from the original mesh can be selected based on user-defined criteria, and then the selected vertices and the triangles connected to them can be removed.

[0085] During mesh simplification, the input can be a voxelized input mesh, a target triangle ratio (TTR), and a minimum triangle component count (CCCount), and the output can be a simplified mesh. Connecting triangle components smaller than a set minimum triangle component count (CCCount) can be removed during this process.

[0086] UV parameterization: The process of mapping a 3D surface to the texture domain of a simplified mesh. Parameterization can be performed using the UVAtlas tool. This process generates mapping information indicating where each vertex of the simplified mesh can be mapped onto a 2D image. This mapping information is represented as texture coordinates and stored, and the final base mesh is generated through this process.

[0087] Fitting the subdivision surface (300): This is the process of subdividing the simplified mesh (i.e., the simplified mesh with texture coordinates). The displacement and base mesh generated by this process are output to encoder 201. User-defined methods (e.g., edge midpoint methods) can be applied as subdivision methods. The fitting process is performed so that the input mesh and the subdivision mesh become similar to each other. The mesh for which the fitting process is performed will be referred to as the fitted subdivision mesh in this paper.

[0088] Figure 4 The method for subdividing the edge midpoint according to an implementation is shown.

[0089] Figure 4 Show reference Figure 3 This describes a method for subdividing the edges of a surface using midpoint subdivision. (Refer to...) Figure 4 The original mesh, containing four vertices, is subdivided to create submesh. Submesh can be created by creating new vertices in the middle of edges between vertices. A fitting process is then performed to make the input mesh and the submesh similar to each other, resulting in a fitted subdivided mesh.

[0090] Once the fitted subdivision mesh is generated, displacements are calculated based on this result and the previously compressed and decoded base mesh (hereinafter referred to as the reconstructed base mesh). In other words, the reconstructed base mesh is subdivided in the same way as the fitted subdivision surface. The positional difference between this result and the individual vertices in the fitted subdivision mesh is the displacement of each vertex. Since displacement represents the positional difference in 3D space, it is represented as a value in the (x, y, z) space of the Cartesian coordinate system. Depending on the user input parameters, the (x, y, z) coordinate values ​​can be converted to (normal, tangent, bitangent) coordinate values ​​in the local coordinate system.

[0091] Figure 5 The displacement generation process according to the embodiment is shown. Figure 5 The displacement generation process can be executed by the preprocessor 200 or by the encoder 201.

[0092] Figure 5 Detailed explanation for reference Figure 4 The description describes how to calculate the displacement of the fitted subdivision surface 300.

[0093] The encoder and / or preprocessor according to the implementation may include 1) a subdivision unit, 2) a local coordinate system calculator, and 3) a displacement vector calculator. The subdivision unit performs subdivision on the reconstructed base mesh to generate a subdivided reconstructed base mesh. Here, the reconstruction of the base mesh may be performed by the preprocessor 200 or by the encoder 201. The local coordinate system calculator receives the fitted subdivision mesh and the subdivided reconstructed base mesh, and can transform the coordinate system associated with the mesh to a local coordinate system based on the received mesh. The local coordinate system calculation may be optional. The displacement calculator calculates the positional difference between the fitted subdivision mesh and the subdivided reconstructed base mesh. For example, it can generate the positional difference between vertices in two input meshes. The positional difference between vertices is a displacement.

[0094] The grid data transmission method and apparatus according to the embodiments can encode grid data as follows. Grid data is a term that includes point cloud data. Point cloud data according to the embodiments (which may be simply referred to as point cloud) can refer to data including vertex coordinates (also called geometric information) and color information (also called attribute information). Additionally, geometric images, attribute images, occupancy maps, and auxiliary information (also called patch information) generated and packaged based on vertex coordinates and color information through patching can also be referred to as point cloud data. Therefore, point cloud data including connectivity information can be referred to as grid data. The terms point cloud and grid data are used interchangeably herein.

[0095] According to the implementation method, the V-Mesh compression (reconstruction) method may include intra-frame coding ( Figure 6 ) and inter-frame coding ( Figure 7 ).

[0096] Based on the results generated by GoF, intra-frame coding or inter-frame coding is performed. In intra-frame coding, the data to be compressed can be the base grid, translation, attribute map, etc. In inter-frame coding, the data to be compressed can be translation, attribute map, and the motion field between the reference base grid and the current base grid.

[0097] Figure 6 The intra-frame coding process in the V-MESH compression method according to an embodiment is shown. Figure 6 The components of the intra-frame coding process correspond to hardware, software, processors, and / or combinations thereof.

[0098] Figure 6 The encoding process is described in detail. Figure 1 The encoding of the mesh video encoder 102. That is, it represents when Figure 1 The encoding is the configuration of the mesh video encoder 102 during intra-frame encoding. Figure 6 The encoder may include a preprocessor 200 and / or an encoder 201. Figure 6 The preprocessor 200 and encoder 201 can correspond to Figure 3The preprocessor 200 and encoder 201.

[0099] The preprocessor 200 can receive the input mesh and perform the preprocessing described above. A base mesh and / or a fitted subdivision mesh can be generated through preprocessing.

[0100] The encoder 201's quantizer 411 quantizes the base mesh and / or the fitted subdivision mesh. The static mesh encoder 412 encodes the static mesh (i.e., the quantized base mesh) and generates a bitstream containing the encoded base mesh (i.e., a compressed base mesh bitstream). The static mesh decoder 413 decodes the encoded static mesh (i.e., the encoded base mesh). The inverse quantizer 414 inverse-quantizes the quantized static mesh (i.e., the base mesh) and outputs a reconstructed (restored) base mesh. The displacement calculator 415 generates displacements based on the reconstructed static mesh (i.e., the base mesh) and the fitted subdivision mesh. According to an implementation, the displacement calculator 415 subdivides the reconstructed base mesh and then calculates the displacement, i.e., the positional difference between the individual vertices of the subdivision base mesh and the fitted subdivision mesh. In other words, when the fitted subdivision mesh is similar to the original mesh, the displacement is a displacement vector as the positional difference between the vertices in the two meshes. The forward linear lifter 416 performs a lift transformation on the input displacement to generate lift coefficients (also called transform coefficients). The quantizer 417 quantizes the lift coefficients. Image packer 418 can pack images based on quantization boost factors. Video encoder 419 can encode the packed images. That is, the quantization boost factors are packed into frames as 2D images by image packer 418, compressed by video encoder 419, and output as a shifted bitstream (i.e., compressed shifted bitstream).

[0101] Video decoder 420 decodes the compressed displacement bitstream. Image unpacker 421 unpacks the decoded displacement frames to output quantization boost coefficients. Inverse quantizer 422 inverse quantizes the quantization boost coefficients. Inverse linear boost unit 423 applies inverse boost to the inverse quantized boost coefficients to generate reconstructed displacements. Mesh reconstructor 424 recovers the reconstructed and deformed mesh based on the reconstructed displacements output from inverse linear boost unit 423 and the reconstructed base mesh (also called subdivision reconstructed base mesh) output from inverse quantizer 414. The reconstructed and deformed mesh is referred to herein as the reconstructed deformed mesh.

[0102] Attribute transfer 425 receives an input mesh and / or an input attribute map, and regenerates the attribute map based on the reconstructed deformed mesh. An attribute map refers to a texture map corresponding to attribute information within the mesh data components. In this disclosure, the terms attribute map and texture map are used interchangeably. Push-pull fill unit 426 fills the attribute map with data using a push-pull method. Color space converter 427 converts the color space of the attribute map. For example, the attribute map can be converted from RGB color space to YUV color space. Video encoder 428 encodes the attribute map to output a compressed attribute bitstream.

[0103] Multiplexer 430 can multiplex compressed base grid bitstream, compressed displacement bitstream, and compressed attribute bitstream to generate compressed bitstream.

[0104] exist Figure 6 In the preprocessor 200, the displacement calculator 415 may be included. Additionally, at least one of the following may be included in the preprocessor 200: a quantizer 411, a static mesh encoder 412, a static mesh decoder 413, or an inverse quantizer 414.

[0105] like Figure 6 The intra-frame coding method described herein includes basic mesh coding (also known as static mesh coding). That is, when intra-frame coding is performed on the current input mesh frame, the basic mesh generated during preprocessing in preprocessor 200 can be quantized by quantizer 411 and then encoded by static mesh encoder 412 using static mesh compression techniques. For example, in the V-Mesh compression method, Draco techniques are applied to encode the basic mesh, and vertex position information, mapping information (texture coordinates), vertex connectivity information, etc., associated with the basic mesh are compressed.

[0106] Figure 6 The encoder in the code compresses the underlying grid, displacement, and attributes in the frame to generate a bitstream, while Figure 7 The encoder in the code compresses the motion, displacement, and attributes between the current frame and the reference frame to generate a bitstream.

[0107] Figure 7 The inter-frame coding process in the V-MESH compression method according to an embodiment is shown. Figure 7 The components of the inter-frame coding process correspond to hardware, software, processors, and / or combinations thereof.

[0108] Figure 7 The encoding process is described in detail. Figure 1 The encoding. That is, it represents when Figure 1 The encoding is the encoder configuration during inter-frame encoding. Figure 7 The encoder may include a preprocessor 200 and / or an encoder 201. Figure 7The preprocessor 200 and encoder 201 can correspond to Figure 3 The preprocessor 200 and encoder 201.

[0109] For Figure 6 The encoding operation corresponding to Figure 7 The component for encoding operations, see reference. Figure 6 The description. That is, Figure 7 The operations of quantizer 511, displacement calculator 515, wavelet transform 516, quantizer 517, image packer 518, video encoder 519, video decoder 520, image unpacker 521, inverse quantizer 522 and inverse wavelet transform 523, mesh reconstructor 524, attribute transfer 525, push-pull fill 526, color space converter 527, video encoder 528 and multiplexer 530 are the same as described above. Figure 6 The quantizer 411, static mesh encoder 412, static mesh decoder 413, inverse quantizer 414, displacement calculator 415, forward linear boosting unit 416, quantizer 417, image packer 418, video encoder 419, video decoder 420, image unpacker 421, inverse quantizer 422, inverse linear boosting unit 423, mesh reconstructor 424, attribute transfer 425, push-pull fill 426, color space converter 427, video encoder 428, and multiplexer 430 operate in the same or similar manner, therefore regarding Figure 7 To avoid redundancy, further details will not be provided.

[0110] exist Figure 7 In the inter-frame coding, the motion encoder 512 obtains and encodes the motion vector between the reconstructed quantized reference base grid and the quantized current base grid, and outputs a compressed motion bitstream. The motion encoder 512 can be referred to as a motion vector encoder. The base grid reconstructor 513 reconstructs the base grid based on the reconstructed quantized reference base grid and the encoded motion vector. The reconstructed base grid is dequantized by the inverse quantizer 514 and output to the displacement calculator 515.

[0111] exist Figure 7 In the preprocessor 200, the displacement calculator 515 may be included. Additionally, at least one of the following may be included in the preprocessor 200: a quantizer 511, a motion encoder 512, a base mesh reconstructor 513, or an inverse quantizer 514.

[0112] For reference Figure 7The described inter-frame coding method may include motion field coding (also known as motion vector coding). Inter-frame coding can be performed when the reference mesh and the current input mesh have a one-to-one vertex correspondence and differ only in vertex position information. When performing inter-frame coding, the base mesh may not be compressed. Instead, the difference between the vertices of the reference base mesh and the current base mesh, i.e., the motion field (or motion vector), can be computed and encoded. The reference base mesh is the result of quantized decoding of the base mesh data and is determined by the reference frame index determined in GoF generation. The motion field can be encoded as is. Alternatively, the predicted motion field can be computed by averaging the motion fields of the reconstructed vertices connected to the current vertex, and the residual motion field can be encoded as the difference between the value of the predicted motion field and the value of the motion field of the current vertex. The value of the residual motion field can be encoded using entropy coding. Apart from motion field coding in inter-frame coding, the process of encoding displacement and attribute maps is structurally the same as intra-frame coding methods except for base mesh coding.

[0113] Figure 8 The displacement lifting transformation process according to the embodiment is shown.

[0114] Figure 9 This illustrates the process of packing transformation coefficients (also known as boosting coefficients) into a 2D image according to an embodiment.

[0115] Figure 8 and Figure 9 Show respectively Figure 6 and Figure 7 The process of transforming the displacement and packing transformation coefficients during the encoding process.

[0116] The encoding method according to the implementation method includes shift encoding.

[0117] After base mesh encoding and / or motion field encoding, a reconstructed base mesh can be generated through reconstruction and inverse quantization, and displacement can be calculated between the subdivision result of the reconstructed base mesh and the fitted subdivision mesh generated by fitting the subdivision surface (see [link to documentation]). Figure 6 415 or Figure 7 (See 515). Data transformation processes (e.g., wavelet transform) can be applied to displacement information for efficient encoding (see [reference]). Figure 6 416 or Figure 7 (516 in the middle).

[0118] Figure 8 Shown by Figure 6 Forward linear boosting unit 416 or Figure 7The wavelet transform 516 uses a lifting transform to transform the displacement information. For example, a lifting transform based on a linear wavelet can be performed. The transform coefficients generated by the transform process are quantized by a quantizer 417 (or 517) and then packed into a 2D image by an image packer 418 (or 518), such as... Figure 9 As shown. Transform coefficients can be organized into blocks, with each block consisting of 256 (=16×16) units. Each block can be packed in z-scan order. The number of rows in a block is fixed at 16, but the number of columns can be determined by the number of vertices in the subdivided base grid. Within a block, transform coefficients can be sorted and packed according to Morton codes. For the packed images, a displacement video can be generated per GoF. The displacement video can be encoded by a video encoder 419 (or 519) using a conventional video compression codec.

[0119] Reference Figure 8 The base mesh (original) may include the vertices and edges of LoD0. A first subdivided mesh, generated by splitting (or subdividing) the base mesh, includes vertices generated by further splitting (or subdividing) the edges of the base mesh. The first subdivided mesh contains the vertices of LoD0 and LoD1. LoD1 includes subdivided vertices and vertices from the base mesh (LoD0). The first subdivided mesh can be split (or subdivided) to generate a second subdivided mesh. The second subdivided mesh contains LoD2. LoD2 includes the vertices of the base mesh (LoD0), LoD1 contains vertices further split (or subdivided) from LoD0, and LoD2 contains vertices further split (or subdivided) from LoD1. LoD is an indicator of how detailed the mesh data content is. As the index of the level increases, the distance between vertices decreases, and the level of detail increases. In other words, as the value of LoD decreases, the detail of the mesh data content decreases. As the value of LoD decreases, the detail of the mesh data content increases. LoD N contains the vertices contained in LoD N-1. When further subdividing the mesh (or vertices) through subdivision, the mesh can be encoded based on prediction and / or update methods, taking into account previous vertices v1 and v2 and subdivided vertices v. Instead of encoding the current LoD N information as is, a residual relative to the previous LoD N-1 can be generated. Therefore, the residual can be used to encode the mesh to reduce the size of the bitstream. The prediction process refers to the operation of predicting the current vertex v from previous vertices v1 and v2. Since neighboring subdivided meshes have similar data, this property can be utilized for efficient encoding. The current vertex position information is predicted from the residual of the previous vertex position information, and the previous vertex position information is updated by the residual. In this disclosure, vertices and points are used interchangeably. LoD can be defined in the subdivision of the base mesh. According to the implementation, the subdivision of the base mesh can be performed by preprocessor 200 or by a separate component / module.

[0120] Reference Figure 9The vertices have transform coefficients (also called lift coefficients) generated by the lift transform. The transform coefficients of the vertices related to the lift transform can be packed into the image by the image packer 418 (or 518) and then encoded by the video encoder 419 (or 519).

[0121] Figure 10 The attribute transfer process in the V-MESH compression method according to an embodiment is shown.

[0122] According to the implementation method, Figure 10 Show Figure 6 , Figure 7 Detailed operation of attribute transfer 425 (or 525) in the encoding of etc.

[0123] The encoding according to the implementation includes attribute graph encoding. According to the implementation, the attribute graph encoding can be derived from... Figure 6 Video encoder 428 or Figure 7 The video encoder 528 is executed.

[0124] According to the embodiments, in this disclosure, the encoder compresses information about the input grid through basic grid coding (i.e., intra-frame coding), motion field coding (i.e., inter-frame coding), and displacement coding. The compressed input grid during the coding process is reconstructed through basic grid decoding (intra-frame), motion field decoding (inter-frame), and displacement video decoding. As a result of the reconstruction, the reconstructed deformed grid (hereinafter referred to as the reconstructed deformed grid) is used to compress the input attribute map, such as... Figure 6 and Figure 7 As shown. The reconstructed deformable mesh has positional information about vertices, texture coordinates, and corresponding connectivity, but lacks color information corresponding to the texture coordinates. Therefore, as... Figure 10 As shown, in the V-Mesh compression method, a new attribute map with color information corresponding to the texture coordinates of the reconstructed deformed mesh is regenerated through an attribute transfer process of attribute transfer 425 (or 525).

[0125] According to the implementation, attribute transfer 425 (or 525) first checks whether the corresponding vertex of each point P(u, v) in the 2D texture domain is within the texture triangle of the reconstructed deformable mesh. When the corresponding vertex is within the texture triangle T, attribute transfer calculates the barycentric coordinates (α, β, γ) of P(u, v) based on triangle T. Then, it calculates the 3D coordinates M(x, y, z) of P(u, v) based on the 3D vertex positions of triangle T and (α, β, γ). It searches the input mesh domain for the vertex coordinates M'(x', y', z') corresponding to the position closest to the calculated M(x, y, z) and the triangle T' containing that vertex. Then, it calculates the barycentric coordinates (α', β', γ') of M'(x', y', z') within triangle T'. Texture coordinates (u', v') are calculated based on the texture coordinates corresponding to the three vertices of triangle T' and (α', β', γ'), and the corresponding color information is searched in the input attribute map. The color information found in this way is then assigned to the (u, v) pixel position in the new input attribute map. If P(u, v) does not belong to any triangle, a filling algorithm (e.g., a push-pull algorithm with 426 (or 526) push-pull fill) is used to fill the pixel at that position in the new input attribute map with the color value.

[0126] The new property graph generated by property transfer 425 (or 525) is bundled into GoF to construct a property graph video, which is compressed using a video codec of video encoder 428 (or 528).

[0127] Available from Figure 10 The reference relationships between the input mesh, the input property map, the reconstructed deformable mesh, and the reconstructed property map can be observed.

[0128] Figure 1 The decoding process can be executed. Figure 1 The encoding process is the reverse process. Specifically, the decoding process is performed as disclosed below.

[0129] Figure 11 The intra-frame decoding process of V-Mesh technology according to an implementation method is shown.

[0130] Figure 11 Show Figure 1 The configuration and operation of the grid video decoder 113 of the receiving device. Additionally, Figure 11 It shows that it can be executed Figure 6 The grid data is reconstructed by reversing the intra-frame coding process. Figure 11 The components of the intra-frame decoding process correspond to hardware, software, and / or combinations thereof.

[0131] First, the bitstream (i.e., the compressed bitstream) received and input to the demultiplexer 611 of the intra-frame decoder 610 can be separated into a grid substream, a displacement substream, an attribute map substream, and a substream containing patch information about the grid (e.g., V-PCC / V3C). The term V-PCC (video-based point cloud compression) as used in this disclosure may have the same meaning as V3C (visual volumetric video encoding). The two terms are used interchangeably. Therefore, in this disclosure, the term V-PCC can be interpreted as V3C.

[0132] According to the implementation, the mesh substream can be input to and decoded by the static mesh decoder 612, the displacement substream can be input to and decoded by the video decoder 613, and the attribute graph substream can be input to and decoded by the video decoder 617.

[0133] According to the implementation, the mesh subflow can be decoded by decoder 612 of a static mesh codec (e.g., GoogleDraco) used in the encoding to reconstruct connection information, vertex geometry information, vertex texture coordinates, etc., related to the decoding result of the reconstructed quantized base mesh (e.g., reconstructed base mesh).

[0134] According to the implementation, the displacement substream can be decoded into displacement video by decoder 613 of the video compression codec used in the encoding. Then, image unpacking is performed by image unpacker 614, inverse quantization is performed by inverse quantizer 615, and inverse transform is performed by inverse linear lift unit 616 to reconstruct displacement information about each vertex (i.e., reconstruct displacement).

[0135] According to an implementation, the base mesh reconstructed by the static mesh decoder 612 is dequantized by the inverse quantizer 620 and output to the mesh reconstructor 630. The mesh reconstructor 630 reconstructs the reconstructed deformed mesh (i.e., the decoded mesh) based on the reconstructed displacement output from the inverse linear lifting unit 616 and the reconstructed base mesh output from the inverse quantizer 620. In other words, the inverse-quantized reconstructed base mesh is combined with the reconstructed displacement information to generate the final decoded mesh. In this disclosure, the final decoded mesh is referred to as the reconstructed deformed mesh.

[0136] According to the implementation, the attribute map substream is decoded by decoder 617 corresponding to the video compression codec used in the encoding, and then the final attribute map is reconstructed by color converter 640 through color format conversion, color space conversion, etc. (i.e., decoding the attribute map).

[0137] According to the implementation method, the reconstructed decoded grid and decoded attribute map can be used as the final grid data available to the user on the receiving side.

[0138] Reference Figure 11The received compressed bitstream includes patch information, grid substream, displacement substream, and attribute graph substream. The term "substream" is interpreted as a portion of the bitstream included within the main bitstream. The bitstream contains patch information (data), grid information (data), displacement information (data), and attribute graph information (data).

[0139] As mentioned above, Figure 11 The decoder performs intra-frame decoding as follows: Static mesh decoder 612 decodes the mesh substream to generate a reconstructed quantized base mesh, and inverse quantizer 620 applies the quantization parameters of the quantizer in reverse order to generate the reconstructed base mesh. Video decoder 613 decodes the displacement substream, image unpacker 614 unpacks the image of the decoded displacement video, and inverse quantizer 615 inverse quantizes the quantized image. Inverse linear lift unit 616 applies a lift transform in the inverse process of the encoder to generate the reconstructed displacement. Mesh reconstructor 630 generates a reconstructed deformable mesh based on the reconstructed base mesh and the reconstructed displacement. Video decoder 617 decodes the attribute map substream, and color transformer 64 transforms the color format and / or space of the decoded attribute map to generate the decoded attribute map.

[0140] Figure 12 This illustrates the inter-frame decoding process of V-Mesh technology.

[0141] Figure 12 Show Figure 1 The configuration and operation of the receiving device's mesh video decoder 113. Figure 12 In the middle, it can be executed Figure 7 The grid data is reconstructed by reversing the inter-frame coding process. Figure 12 The components of the intra-frame decoding process correspond to hardware, software, and / or combinations thereof.

[0142] First, the bitstream received and input to the demultiplexer 711 of the intra-frame decoder 710 can be separated into a motion substream (also known as a motion vector substream), a displacement substream, an attribute graph substream, and a substream containing patch information about the grid (e.g., V3C / V-PCC).

[0143] According to the implementation, the motion substream can be input to and decoded by the motion decoder 712, the displacement substream can be input to and decoded by the video decoder 713, and the attribute graph substream can be input to and decoded by the video decoder 717.

[0144] According to the implementation, the motion substream is reconstructed by motion decoder 712 through entropy decoding and inverse prediction decoding to reconstruct motion information (also known as motion vector information). Base mesh reconstructor 718 combines the reconstructed motion information with a pre-reconstructed and stored reference base mesh to generate a reconstructed quantized base mesh for the current frame. Inverse quantizer 720 applies inverse quantization to the reconstructed quantized base mesh to generate a reconstructed base mesh. Video decoder 713 decodes the displacement substream, image unpacker 714 unpacks the image of the decoded displacement video, and inverse quantizer 715 performs inverse quantization on the quantized image. Inverse linear lift unit 716 applies a lift transform in the inverse process of the encoder to generate reconstructed displacement. Mesh reconstructor 730 generates a reconstructed deformable mesh (i.e., the final decoded mesh) based on the reconstructed base mesh and the reconstructed displacement.

[0145] According to the implementation, the video decoder 717 decodes the attribute map substream in the same manner as intra-frame decoding, and the color converter 740 transforms the color format and / or space of the decoded attribute map to generate a decoded attribute map. The decoded grid and decoded attribute map can be used as final grid data available to the user at the receiving end.

[0146] Reference Figure 12 The bitstream contains motion information (also known as motion vectors), displacements, and attribute maps. Because inter-frame decoding is performed, Figure 12 The process also includes decoding inter-frame motion information. A reconstructed base grid is generated by decoding the motion information and generating a reconstructed quantization base grid based on a reference base grid. For Figure 12 Zhongyu Figure 11 For the same operation, refer to [the relevant documentation]. Figure 11 The description.

[0147] Figure 13 A grid data transmission device according to an embodiment is shown.

[0148] Figure 13 Corresponding to Figure 1 The transmitting device 100 or the grid video encoder 102, Figure 2 , Figure 6 or Figure 7 The encoder (preprocessor and encoder) and / or the corresponding transmission encoding device. Figure 13 The individual components correspond to hardware, software, processors, and / or combinations thereof.

[0149] The process of compressing and transmitting dynamic mesh data using V-Mesh compression technology at the sending end can be described as follows: Figure 13 The configuration shown. Figure 13 The transmitting device can perform intra-frame coding (also known as intra-frame coding or intra-picture coding) and / or inter-frame coding (also known as inter-frame coding or inter-picture coding).

[0150] The preprocessor 811 receives the original mesh and generates a simplified mesh (or base mesh) and a fitted subdivision (or subdivision) mesh. Simplification can be performed based on the target number of vertices or polygons constituting the mesh. Parameterization can be performed on the simplified mesh to generate texture coordinates and texture connectivity information for each vertex. For example, parameterization is the process of mapping a 3D surface onto the texture domain of the simplified mesh. When parameterization is performed using the UVAtlas tool, mapping information indicating where each vertex of the simplified mesh can be mapped to on a 2D image is generated. The mapping information is represented as texture coordinates and stored, and the final base mesh is generated through this process. Mesh information can be quantized from floating-point to fixed-point form. The result is the base mesh, which can be output to the motion vector encoder 813 or the static mesh encoder 814 via the switching unit 812. The preprocessor 811 can perform mesh subdivision on the base mesh to generate additional vertices. Depending on the subdivision method, vertex connectivity information including additional vertices, texture coordinates, and connectivity information about the texture coordinates can be generated. The preprocessor 811 can generate a fitted subdivision mesh by adjusting the vertex positions so that the subdivision mesh becomes similar to the original mesh.

[0151] According to the implementation, when inter-frame coding is performed on a grid frame, the base grid is output to the motion vector encoder 813 through the switching unit 812. When intra-frame coding is performed on a grid frame, the base grid is output to the static grid encoder 814 through the switching unit 812. The motion vector encoder 813 may be referred to as a motion encoder.

[0152] For example, when performing intra-frame coding on a mesh frame, the base mesh can be compressed using a static mesh encoder 814. In this case, connection information, vertex geometry information, vertex texture information, normal information, etc., related to the base mesh can be encoded. The base mesh bitstream generated by encoding is sent to multiplexer 823.

[0153] As another example, when performing inter-frame coding on a grid frame, the motion vector encoder 813 can receive a base grid and a reference reconstructed base grid (or a reconstructed quantized reference base grid) as input, calculate the motion vector between the two grids, and encode its values. Furthermore, the motion vector encoder 813 can use previously encoded / decoded motion vectors as predictors to perform prediction based on connectivity information and encode the residual motion vector obtained by subtracting the predicted motion vector from the current motion vector. The bitstream of motion vectors generated by encoding is sent to the multiplexer 823.

[0154] The base mesh reconstructor 815 can receive a base mesh encoded by a static mesh encoder 814 or motion vectors encoded by a motion vector encoder 813, and generate a reconstructed base mesh. For example, the base mesh reconstructor 815 can perform static mesh decoding on the base mesh encoded by the static mesh encoder 814 to reconstruct the base mesh. In this case, quantization can be applied before static mesh decoding, and inverse quantization can be applied after static mesh decoding. In another example, the base mesh reconstructor 815 can reconstruct the base mesh based on a reconstructed quantized reference base mesh and motion vectors encoded by the motion vector encoder 813. The reconstructed base mesh is output to a displacement calculator (or displacement vector calculator) 816 and a mesh reconstructor 820.

[0155] The displacement calculator 816 can perform mesh subdivision on the reconstructed base mesh. The displacement calculator 816 can calculate displacement vectors, which are the differences in vertex positions between the subdivided reconstructed base mesh and the fitted subdivision (or subdivision) mesh generated by the preprocessor 811. In this case, as many displacement vectors as vertices in the subdivided mesh can be calculated. The displacement calculator 816 can transform the displacement vectors calculated in the 3D Cartesian coordinate system to the local coordinate system based on the normal vectors of each vertex.

[0156] The displacement vector video generator 817 may include a linear lifting unit, a quantizer, and an image packer. Specifically, in the displacement vector video generator 817, the linear lifting unit can transform the displacement vector for efficient encoding. Depending on the implementation, the transformation may be a lifting transform, wavelet transform, etc. Additionally, the quantizer can perform quantization on the transformed displacement vector values ​​(i.e., transform coefficients). In this case, different quantization parameters can be applied to the axes of the transform coefficients respectively. The quantization parameters can be derived through an agreement between the encoder and decoder. After transformation and quantization, the displacement vector information can be packed into a 2D image by the image packer. The displacement vector video generator 817 can generate displacement vector videos by grouping the packed 2D images of each frame. Displacement vector videos can be generated for each frame group (GoF) of the input grid.

[0157] The displacement vector video encoder 818 can encode the generated displacement vector video using a video compression codec. The generated displacement vector video bitstream is sent to the multiplexer 823.

[0158] The displacement vector reconstructor 819 may include a video decoder, an image unpacker, an inverse quantizer, and an inverse linear lift unit. Specifically, in the displacement vector reconstructor 819, the video decoder decodes the encoded displacement vector, the image unpacker performs image unpacking, the inverse quantizer performs inverse quantization, and the inverse linear lift unit performs inverse transform to reconstruct the displacement vector. The reconstructed displacement vector is output to the mesh reconstructor 820. The mesh reconstructor 820 reconstructs a deformable mesh based on the base mesh reconstructed by the base mesh reconstructor 815 and the displacement vector reconstructed by the displacement vector reconstructor 819. The reconstructed mesh (also called the reconstructed deformable mesh) has reconstructed vertices, vertex connectivity information, texture coordinates, and texture coordinate connectivity information.

[0159] The texture map video generator 821 can regenerate a texture map based on the texture map (or attribute map) of the original mesh and the reconstructed deformed mesh output from the mesh reconstructor 820. According to one embodiment, the texture map video generator 821 can assign per-vertex color information from the texture map of the original mesh to the texture coordinates of the reconstructed deformed mesh. According to another embodiment, the texture map video generator 821 can generate a texture map video by grouping the frame-level regenerated texture maps into GoF (GoF) frames.

[0160] The generated texture map video can be encoded by the texture map video encoder 822 using a video compression codec. The resulting texture map video bitstream is then sent to the multiplexer 823.

[0161] Multiplexer 823 multiplexes the motion vector bitstream (e.g., in the case of inter-frame coding), the base mesh bitstream (e.g., in the case of intra-frame coding), the displacement vector bitstream, and the texture map bitstream into a single bitstream. This single bitstream can be transmitted to the receiving side via transmitter 824. Alternatively, for the motion vector bitstream, the base mesh bitstream, the displacement vector bitstream, and the texture map bitstream, a file with one or more track data can be generated, or the bitstream can be encapsulated into fragments and transmitted to the receiving side via transmitter 824.

[0162] Reference Figure 13The transmitter (encoder) can encode the mesh in an intra-frame or inter-frame manner. For intra-frame coding, the transmitting device can generate a base mesh, displacement vectors (or displacements), and a texture map (or attribute map). For inter-frame coding, the transmitting device can generate motion vectors (or motions), displacement vectors (or displacements), and a texture map (or attribute map). A texture map obtained from the data input unit is generated and encoded based on the reconstructed mesh. Displacements are generated and encoded based on the vertex position differences between the base mesh and the segmented (or subdivided) mesh. More specifically, the displacement is the position difference between the fitted subdivided mesh and the subdivided reconstructed base mesh, i.e., the vertex position difference between the two meshes. The base mesh is generated by simplifying the original mesh via preprocessing and encoding the simplified mesh. For motion, motion vectors are generated for the mesh of the current frame based on a reference base mesh from the previous frame.

[0163] Figure 14 A grid data receiving device according to an embodiment is shown.

[0164] Figure 14 Corresponding to Figure 1 The receiving device 110 or the mesh video decoder 113, Figure 11 or Figure 12 The decoder and / or corresponding receiving decoding device. Figure 14 The individual components correspond to hardware, software, processors, and / or combinations thereof. Figure 14 The receiving (decoding) operation can follow Figure 13 The reverse process of the corresponding sending (encoding) operation.

[0165] The bitstream of mesh data received by receiver 910 undergoes file / fragment decapsulation, and is then demultiplexed by demultiplexer 911 into compressed motion vector bitstream (e.g., inter-frame decoding) or basic mesh bitstream (e.g., intra-frame decoding), displacement vector bitstream, and texture map bitstream. For example, when the current mesh is inter-frame encoded, the motion vector bitstream is received, demultiplexed, and then output to motion vector decoder 913 via switch unit 912. In another example, when the current mesh is intra-frame encoded, the basic mesh bitstream is received, demultiplexed, and then output to static mesh decoder 914 via switch unit 912. Here, motion vector decoder 913 may be referred to as a motion decoder.

[0166] According to one embodiment, when inter-frame coding is applied to the current grid based on frame header information, the motion vector decoder 913 can decode the motion vector bitstream. According to another embodiment, the motion vector decoder 913 can use previously decoded motion vectors as predictors and add them to the residual motion vectors decoded from the bitstream to reconstruct the final motion vectors.

[0167] According to the implementation method, when intra-frame coding is applied to the current mesh based on the frame header information, the static mesh decoder 914 can decode the base mesh bitstream to reconstruct connection information, vertex geometry information, texture coordinates, normal information, etc. related to the base mesh.

[0168] According to the implementation, the base mesh reconstructor 915 can reconstruct the current base mesh based on the decoded motion vectors or the decoded base mesh. For example, when inter-frame coding is applied to the current mesh, the base mesh reconstructor 915 can add the decoded motion vectors to the reference base mesh and perform inverse quantization to generate the reconstructed base mesh. In another example, when intra-frame coding is applied to the current mesh, the base mesh reconstructor 915 can perform inverse quantization on the base mesh decoded by the static mesh decoder 914 to generate the reconstructed base mesh.

[0169] According to the implementation, the displacement vector video decoder 917 can use a video codec to decode the displacement vector bitstream into a video bitstream.

[0170] According to the implementation, the displacement vector reconstructor 918 extracts displacement vector transform coefficients from the decoded displacement vector video and applies inverse quantization and inverse transform to the extracted displacement vector transform coefficients to reconstruct the displacement vector. For this purpose, the displacement vector reconstructor 918 may include an image unpacker, an inverse quantizer, and an inverse linear lift section. If the reconstructed displacement vector is a value in a local coordinate system, an inverse transform to a Cartesian coordinate system can be performed.

[0171] The mesh reconstructor 916 can subdivide the base mesh to generate additional vertices. Subdivision generates vertex connectivity information, including additional vertices, texture coordinates, and connectivity information about the texture coordinates. In this case, the mesh reconstructor 916 can combine the subdivided base mesh with the reconstructed displacement vectors to generate the final reconstructed mesh (also known as the reconstructed deformable mesh).

[0172] According to the implementation, the texture map video decoder 919 can use a video codec to decode the texture map bitstream into a video bitstream to reconstruct the texture map. The reconstructed texture map has color information about each vertex in the reconstructed mesh, and the texture coordinates of each vertex can be used to obtain the vertex's color value from the texture map.

[0173] According to the implementation, the mesh reconstructed from the mesh reconstructor 916 and the texture map reconstructed from the texture map video decoder 919 are presented to the user through the rendering process in the mesh data renderer 920.

[0174] Reference Figure 14The receiving device (decoder) can decode the mesh in an intra-frame or inter-frame manner. According to intra-frame decoding, the receiving device can receive the base mesh, displacement vectors (or displacements), and texture maps (or attribute maps), and render the mesh data based on the reconstructed mesh and reconstructed texture map. According to inter-frame decoding, the receiving device can receive motion vectors (or motions), displacement vectors (or displacements), and texture maps (or attribute maps), and render the mesh data based on the reconstructed mesh and reconstructed texture map.

[0175] The mesh data transmitting apparatus and method according to the embodiments can preprocess mesh data, encode the preprocessed mesh data, and transmit a bit stream containing the encoded mesh data. The point mesh data receiving apparatus and method according to the embodiments can receive a bit stream containing mesh data and decode the mesh data. The mesh data transmitting / receiving method / apparatus according to the embodiments may be referred to as the method / apparatus according to the embodiments. The mesh data transmitting / receiving method / apparatus may also be referred to as a 3D data transmitting / receiving method / apparatus or a point cloud data transmitting / receiving method / apparatus.

[0176] As described above, in the V-Mesh method, the displacement information generated during the encoding process is converted into video and then compressed using an existing 2D video codec. Additionally, the texture map (equivalent to a property map) of the input mesh data is processed into video and compressed using an existing 2D video codec. In this case, the V-DMC encoder / decoder can perform inter-frame prediction and / or intra-frame prediction during encoding and / or decoding. Here, inter-frame prediction is referred to as inter-frame prediction, while intra-frame prediction is referred to as intra-frame prediction. When inter-frame prediction is performed by the V-DMC encoder / decoder, it is currently performed only for the base mesh, while inter-frame prediction is skipped for displacement information (also called displacement) and texture maps. For example, in the case of inter-frame prediction for the base mesh, prediction is performed using the base mesh and motion vectors of a reference frame. Conversely, for displacement information and texture maps, encoding / decoding is performed using intra-frame prediction (i.e., intra-frame coding).

[0177] However, applying inter-frame prediction only to the base mesh during inter-frame prediction can be inefficient. Therefore, to achieve more efficient compression, and especially to improve the efficiency of inter-frame prediction, this disclosure proposes a method to skip encoding the current texture map when the similarity between the reference texture map and the current texture map is high. Furthermore, when skipping texture map encoding only on a frame-by-frame basis, there may be cases where most regions are similar but some regions are different, preventing the skipping of texture map encoding. To achieve efficient compression even in such cases by skipping texture map encoding, this disclosure proposes a method of skipping texture map encoding at the unit level, such as sub-mesh, patch, and tile.

[0178] As described above, when the similarity between the reference texture map of the dynamic mesh and the current texture map is high, this disclosure allows skipping the encoding of the current texture map. In this case, the unit forming the basis for skipping texture map encoding can be a frame, submesh, patch, tile, etc. In other words, in this disclosure, by skipping a considerable proportion of the texture map in the V-DMC bitstream data, the number of bits of the texture map to be transmitted and the encoding / decoding complexity can be reduced. In other words, according to this disclosure, by improving the compression efficiency of the texture map, which accounts for the largest proportion of the main components of the V-DMC data (i.e., the base mesh, displacement vectors, and texture map), the overall compression efficiency of the V-DMC codec of this disclosure can be improved.

[0179] Furthermore, to apply texture map encoding skipping methods in V-DMC codecs, this disclosure proposes an advanced syntax signaling method based on atlas streams. Specifically, this disclosure proposes a method for applying texture map skipping methods based on frames, subgrids, patches, or tiles in V-DMC codecs, as well as a signaling method, syntax, and semantics based on atlas streams. According to the implementation, a flag indicating whether a texture map skipping method is used and a flag indicating the presence of skipped texture maps within the sequence can be signaled using the Atlas Sequence Parameter Set (ASPS). Additionally, in this disclosure, a flag indicating whether texture map skipping is applied on a frame basis and reference information for deriving skipped texture maps can be signaled at atlas stream-based syntax transmission locations (such as atlas frame parameter sets, tile headers, patch data units, and NAL unit headers).

[0180] Figure 15 A transmitting device according to an embodiment is shown. Figure 15 The transmitting device can be referred to as a mesh data transmitting device, encoder, encoder of transmitting device, V-Mesh encoder or dynamic mesh encoder.

[0181] Figure 15 Corresponding to Figure 1The transmitting device 100 or the grid video encoder 102, Figure 2 , Figure 6 or Figure 7 The encoder (preprocessor and encoder). Figure 13 The transmitting device and / or the corresponding transmitting encoding device. Figure 15 Each component corresponds to hardware, software, processor, and / or a combination thereof. Figure 15 In this context, the order in which blocks are executed can be changed. Some blocks can be omitted, and some blocks can be newly added.

[0182] In this disclosure, it is possible to... Figure 15 The procedure shown is for compressing and transmitting dynamic mesh data using V-Mesh compression technology. Figure 15 The transmitting device can support intra-frame encoding (or intra-screen encoding) and / or inter-frame encoding (or inter-screen encoding) processes.

[0183] exist Figure 15 In this context, the mesh simplifier 11011 uses a mesh simplification algorithm to simplify the input original mesh to generate a base mesh (also known as a simplified base mesh or a simplified mesh). Mesh simplification can be performed based on the number of target vertices or the number of target polygons constituting the mesh. For example, decimation can be used as a mesh simplification algorithm to simplify the original mesh. Specifically, simplification can be a process of selecting vertices to be removed from the original mesh based on a reference point, and then removing the selected vertices and the triangles connected to the selected vertices.

[0184] In other words, the mesh reducer 11011 can reduce an input mesh to a target number of vertices or a target number of faces. This reduction can be performed using various methods, such as triangle folding and edge folding.

[0185] According to the implementation, the base mesh simplified by the mesh simplifier 11011 is provided to the mesh parameterizer 11012 and the mesh subdivision 11018.

[0186] Mesh parameterizer 11012 performs an operation to map a 3D surface to a texture domain for a simplified mesh. That is, mesh parameterizer 11012 generates texture coordinate information and texture connectivity information associated with the input mesh. In one embodiment, mesh parameterizer 11012 can use a UV atlas tool to perform parameterization. In this operation, mapping information indicating the location to which each vertex of the simplified mesh in the 2D image can be mapped is generated. The mapping information is represented and stored as texture coordinates, and this operation generates the final base mesh. In other words, mesh parameterizer 11012 performs parameterization to generate texture coordinates (UV coordinates) and texture connectivity information for each vertex of the input mesh (i.e., the simplified mesh or the simplified base mesh).

[0187] The final base mesh (also known as the base mesh with texture map) generated by parameterizer 11012 is input to mesh quantizer 11013 for quantization.

[0188] According to an implementation, the mesh quantizer 11013 can quantize mesh information in floating-point form (e.g., geometric information (x, y, z) and / or texture coordinates (u, v), normal information (nx, ny, nz) etc.) into fixed-point form. That is, the mesh quantizer 11013 can quantize the vertex coordinates and texture coordinates of the base mesh. In some implementations, the quantization of certain components can be skipped.

[0189] Mesh subdivision 11018 subdivides the base mesh obtained through simplification by mesh simplifier 11011. Mesh subdivision 11018 can perform mesh subdivision on the base mesh to generate additional vertices. Depending on the subdivision method, vertex connection information including additional vertices, texture coordinates, and connection information about the texture coordinates can be generated. In this case, the geometric connection information, texture coordinate connection information, and texture coordinates can be generated through implicit derivation dependent on the subdivision method. Depending on the implementation, mesh subdivision 11018 can use methods such as edge midpoints, loops, or Catmul & Clark to perform subdivision.

[0190] Furthermore, the mesh subdivision unit 11018 can perform mesh subdivision n times according to user parameters or an agreement between the encoder and decoder. According to the implementation, when the vertices of the base mesh are defined as R0, the newly generated vertices by performing one subdivision are defined as R1, ..., and the vertices generated by performing n subdivisions are defined as R... n LOD n It can be defined as follows.

[0191]

[0192] According to an implementation, the mesh fitter 11019 can generate a fitted subdivided mesh by adjusting vertex positions to perform fitting, making the mesh subdivided by the mesh subdivision unit 11018 similar to the original mesh. That is, the mesh fitter 11019 performs vertex position adjustments to ensure that the subdivided mesh becomes similar to the original mesh. According to an implementation, the mesh simplifier 11011, mesh parameterizer 11012, mesh subdivision unit 11018, and mesh fitter 11019 can be omitted. When the corresponding operations are skipped, the original mesh can be provided as input to the mesh quantizer 11013.

[0193] In this case, the coordinate information associated with the original mesh can be provided as input to the displacement vector calculator 11020. According to the implementation, the displacement vector encoding process (i.e., displacement vector calculator 11020, displacement vector coordinate transformer 11021 and displacement vector encoder 11022) can be omitted.

[0194] In this disclosure, the combination of mesh simplifier 11011, mesh parameterizer 11012, mesh subdivision 11018, and mesh fitter 11019 can be referred to as a preprocessor. According to embodiments, the preprocessor may further include a displacement vector calculator 11020.

[0195] According to the implementation, the base grid quantized by the grid quantizer 11013 can be output to the motion vector encoder 11015 or the static grid encoder 11016 via the switching section 11014.

[0196] According to the implementation, when inter-frame encoding is performed on a grid frame, the base grid is output to the motion vector encoder 11015 via the switching section 11014. When intra-frame encoding is performed on a grid frame, the base grid is output to the static grid encoder 11016 via the switching section 11014. The motion vector encoder 11015 may be referred to as a motion encoder.

[0197] In one example, when intra-frame encoding is performed on a mesh frame, the base mesh can be compressed by a static mesh encoder 11016. In this case, connection information, vertex geometry information, vertex texture information, normal information, etc., related to the base mesh can be encoded. That is, vertex coordinates, vertex connection information, texture coordinates, texture connection information, etc., related to the mesh can be encoded by the static mesh encoder 11016. The base mesh bitstream generated by encoding is sent to a multiplexer (not shown).

[0198] In another example, when performing inter-frame coding on a grid frame, the motion vector encoder 11015 can receive a base grid and a reference reconstructed base grid (or a reconstructed quantized reference base grid) as input, calculate the motion vector between the two grids, and encode their values. Furthermore, the motion vector encoder 11015 can use previously encoded / decoded motion vectors as predictors to perform prediction based on connectivity information and perform entropy coding on the motion vector difference (also known as the residual motion vector) obtained by subtracting the predicted motion vector from the current motion vector. The resulting motion vector bitstream is sent as the base grid bitstream to a multiplexer (not shown). That is, in the case of intra-frame coding, the static grid bitstream is input to the multiplexer as the base grid bitstream.

[0199] exist Figure 15 In this configuration, the base mesh decoder (also called the base mesh reconstructor) 11017 can receive a base mesh encoded by the static mesh encoder 11016 or motion vectors encoded by the motion vector encoder 11015, and generate a reconstructed base mesh. The base mesh decoder 11017 reconstructs the base mesh based on the encoding type of the current mesh (inter-frame coding or intra-frame coding). For example, the base mesh decoder 11017 can reconstruct the base mesh by performing static mesh decoding on the base mesh encoded by the static mesh encoder 11016. In this case, quantization can be applied before static mesh decoding and inverse quantization can be applied after static mesh decoding. In other words, when intra-frame coding is performed, inverse quantization can be performed on the base mesh quantized by the mesh quantizer 11013 to reconstruct the current base mesh. As another example, the base mesh decoder 11017 can reconstruct the base mesh based on a reconstructed quantized reference base mesh and motion vectors encoded by the motion vector encoder 11015. In other words, when performing inter-frame coding, motion vector decoding methods can be used to decode motion vectors, and then the decoded motion vectors can be applied (i.e., added) to a reference base mesh to generate the current base mesh. Without quantizing the motion vectors, the motion vector reconstruction process can be skipped, and the current base mesh can be reconstructed using the motion vectors computed by the motion vector encoder 11015. The reconstructed base mesh is output to the displacement vector calculator 11020 and the mesh inverse quantizer 11024.

[0200] According to the implementation, the displacement vector calculator 11020 can perform mesh subdivision on the reconstructed base mesh. Furthermore, the displacement vector calculator 11020 can calculate a displacement vector that is the value of the vertex position difference between the subdivided reconstructed base mesh and the fitted subdivision (or subdivided) mesh generated by the mesh fitter 11019. In this case, the displacement vector can be calculated as a multiple of the number of vertices in the subdivided mesh. In other words, the displacement vector calculator 11020 can calculate a displacement vector corresponding to the number of vertices in the subdivided mesh.

[0201] According to an implementation, the displacement vector coordinate transformer 11021 can directly output the vertex displacement vector calculated in a 3D Cartesian coordinate system (i.e., (x, y, z) space) (also called a canonical coordinate system), or it can transform it to a local coordinate system (i.e., normal, tangential, and bitangential coordinates) based on the normal vector of each vertex. In this case, the normal vector can be calculated for each subdivided vertex based on geometric and connectivity information associated with neighboring vertices. According to an implementation, only the normal component among the normal, tangential, and bitangential components of the local coordinate system can be encoded. According to the agreement between the encoder and decoder, when applying coordinate transformation, only the normal component can always be encoded, or the encoder can determine and signal a 1-bit flag. In this case, the normal vector can be calculated for each subdivided vertex based on geometric and / or connectivity information associated with neighboring vertices.

[0202] Furthermore, whether the displacement vector coordinate transformer 11021 performs a displacement vector coordinate transformation can be determined by an agreement between the encoder (i.e., the transmitting device) and the decoder (i.e., the receiving device), or by signaling a coordinate transformation flag (e.g., asps_vmc_ext_displacement_coordinate_system) on a per-sequence, per-frame-group (GOF), per-frame, or per-sub-grid basis. As an example, the displacement vector coordinate transformation flag asps_vmc_ext_displacement_coordinate_system, which serves as information for identifying whether a displacement vector coordinate transformation has been performed, can be signaled in signaling information (e.g., the Graph Sequence Parameter Set (ASPS)) and sent to the receiving device. For instance, when the value of the syntax element (also called the field) asps_vmc_ext_displacement_coordinate_system is 0, it can indicate the use of a canonical coordinate system. When the value of the element is 1, it can indicate that a transformation to a local coordinate system has been performed.

[0203] According to an implementation, the displacement vector encoder 11022 encodes a displacement vector in Cartesian coordinates or a displacement vector transformed to a local coordinate system by the displacement vector coordinate transformer 11021 into a displacement vector bitstream (also referred to as a displacement vector video bitstream). In one implementation, the displacement vector encoder 11022 can use a 2D video encoder such as H.264, HEVC, or VVC to encode a displacement vector in Cartesian coordinates or a displacement vector transformed to a local coordinate system into a displacement vector bitstream.

[0204] In other words, the displacement vector encoder 11022 can encode displacement vectors or displacement vector coefficients (also known as displacement vector transform coefficients). In this disclosure, the displacement vector encoder 11022 can perform encoding using a video codec-based encoder, a zero-run encoder, an arithmetic encoder, or the like. For example, when the encoding method is video codec-based, the displacement vector encoder 11022 can pack displacement vectors or displacement vector coefficients into frames for encoding. That is, the displacement vector coefficients can be packed into a 2D image and then encoded using a 2D video codec (i.e., a video compression codec), or zero-run coding or arithmetic coding can be used to generate a displacement vector video bitstream.

[0205] According to one embodiment, the displacement vector video bitstream encoded and generated by the displacement vector encoder 11022 is sent to a multiplexer (not shown). According to another embodiment, regarding the method for selecting encoding by the displacement vector encoder 11022, a displacement vector encoder agreed upon between the encoder (i.e., the transmitting side) and the decoder (i.e., the receiving side) can be used. Alternatively, the encoder on the transmitting side can analyze the characteristics of the displacement vector and send the type of the selected displacement vector encoder to the decoder on the receiving side.

[0206] According to the implementation, the displacement vector reconstruction unit 11023 can reconstruct the displacement vector by performing the inverse process of displacement vector encoding on the displacement vector or displacement vector coefficients encoded by the displacement vector encoder 11022. That is, depending on the encoding method, for example, when using video codec-based encoding, the displacement vector reconstructor 11023 can perform inverse packing (also known as unpacking) of the displacement vector. More specifically, the displacement vector reconstructor 11023 can decode the bitstream using a 2D video decoder and perform inverse packing, where 2D images / videos are packed into the bitstream and encoded by a 2D video encoder. Then, inverse quantization and inverse transform can be performed on the unpacked quantized transform coefficients to compute the reconstructed displacement vector. That is, the displacement vector reconstructor 11023 can additionally perform inverse quantization and inverse transform depending on whether quantization and transform have been performed during displacement vector encoding.

[0207] According to the implementation, the mesh inverse quantizer 11024 can inverse quantize the vertex coordinates or texture coordinates of the reconstructed base mesh through the inverse quantization process. When the mesh quantizer 11013 skips quantization, the mesh inverse quantizer 11024 can also skip inverse quantization.

[0208] According to an implementation, the mesh reconstruction unit 11025 can reconstruct a mesh based on the reconstructed displacement vector output from the displacement vector reconstructor 11023 and the reconstructed base mesh (or the dequantized reconstructed base mesh) output from the mesh inverse quantizer 11024. More specifically, the mesh reconstructor 11025 can subdivide the reconstructed base mesh output from the mesh inverse quantizer 11024 and add it to the reconstructed displacement vector output from the displacement vector reconstructor 11023 to generate a reconstructed deformable mesh. According to an implementation, the mesh reconstructed by the mesh reconstructor 11025 (also referred to as the reconstructed mesh or the reconstructed deformable mesh) may include reconstructed vertices, connections between vertices, texture coordinates, and connections between texture coordinates. The reconstructed mesh (also referred to as the reconstructed deformable mesh) generated by the mesh reconstructor 11025 is provided to the texture map generator 11026.

[0209] According to the implementation, the texture map generator 11026 can regenerate the texture map of the current mesh based on the texture map (also called the attribute map) of the original mesh and the reconstructed mesh from the mesh reconstructor 11025. That is, the texture map generator 11026 can generate the texture map of the reconstructed mesh based on the texture map of the original mesh and the relationship between the original mesh and the reconstructed mesh. In other words, the texture map generator 11026 generates the texture map of the reconstructed mesh based on the texture coordinates and connection information associated with the reconstructed mesh and the relationship between the original mesh and its texture map.

[0210] According to one implementation, texture map generator 11026 can assign per-vertex color information contained in the texture map of the original mesh to the texture coordinates of the reconstructed base mesh (or the reconstructed deformable mesh). According to another implementation, texture map generator 11026 can group the regenerated texture maps on a per-GoF basis to generate texture maps (also known as texture map videos).

[0211] According to an implementation, the texture map skipping determiner 11027 determines whether to skip encoding the current texture map. In one embodiment of this disclosure, when the similarity between the reference texture map and the current texture map is high, encoding of the current texture map can be skipped. Here, skipping encoding of the current texture map includes skipping the encoding of the current texture map and skipping the transmission of the current texture map to the receiving device. The method for determining whether to skip encoding of the current texture map will be described in detail later.

[0212] Texture map skip determiner 11027 determines which texture maps will not be skipped and can be encoded by texture map encoder 11028. For example, texture map encoder 11028 can encode texture maps using an encoder based on a 2D video codec, a zero-run encoder, or an entropy-based arithmetic encoder. Furthermore, texture map encoder 11028 can also perform color space transformation on the texture maps. The texture map substream (or texture map video bitstream) generated by texture map encoding is sent to a multiplexer (not shown).

[0213] According to the implementation, the texture map encoder 11028 may include a video encoder (e.g., VVC, HEVC, etc.) and an entropy-based encoder. Regarding the method of selecting the texture map encoder 11028, a texture map encoder agreed upon between the encoder (i.e., the transmitting side) and the decoder (i.e., the receiving side) can be used. Alternatively, information regarding the type of texture map encoder selected by the encoder on the transmitting side can be sent to the decoder on the receiving side.

[0214] According to one implementation, a multiplexer (not shown) can multiplex the input base mesh bitstream, displacement vector bitstream, and texture map bitstream into a single bitstream for transmission to a receiving device. Alternatively, the base mesh bitstream, displacement vector bitstream, and texture map bitstream can be encapsulated into a file / fragment to be transmitted to the receiving device.

[0215] According to the implementation, the bitstream multiplexed by the multiplexer can be transmitted over a network or stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD.

[0216] Next, the method by which the texture map skip determiner 11027 determines whether the texture map encoding is skipped and the method of notifying relevant information by signaling will be described.

[0217] According to the implementation, the texture map skip determiner 11027 can determine whether to skip encoding of the texture map generated by the texture map generator 11026 through the following process.

[0218] According to the implementation, whether the texture map skip determiner 11027 should skip the encoding / decoding of the texture map can be determined by explicitly signaling the texture map skip flag (texture_skip_flag), or implicitly deduced based on conditions defined in the same way by the encoder / decoder.

[0219] Additionally, in this disclosure, the flag `asve_textureSkip_enable_flag`, indicating whether texture map skipping technology (i.e., the method) is used in the V-DMC codec, can be signaled via the Atlas Sequence Parameter Set (ASPS), thereby sending information to the receiving device regarding whether the texture map skipping method is applied to the corresponding content or sequence. In this disclosure, the signaling notification can be performed by the texture map skipping determiner 11027 or by a separate block or module such as a signaling processor (not shown).

[0220] For example, when the value of `asve_textureSkip_enable_flag` is 1, parameters related to skipping texture maps can be resolved. In this case, if a skipped texture map exists in the corresponding sequence, then `asve_textureSkip_present_flag` can be signaled as 1. Otherwise, the flag can be signaled as 0.

[0221] According to the implementation, the criteria used by the texture map skip determiner 11027 to determine whether a texture map is skipped may include the similarity between the current texture map and the reference texture map, the estimated number of generated bits, and the distance between the current texture map and the reference texture map.

[0222] Furthermore, the unit for skipping texture information (i.e., texture map) can be a frame, patch, submesh, tile, or slice. That is, depending on the skipping unit (such as frame, patch, slice, submesh, or tile) of the texture map determined by the texture map skip determiner 11027, a texture map skip flag (texture_skip_flag) indicating whether to skip the texture map can be determined.

[0223] Figure 16 This is a flowchart illustrating an example of a method for determining whether to skip texture map encoding according to an implementation method. That is, Figure 16 An example of a method for determining whether texture map encoding is skipped by texture map skip determiner 11027 is shown. In this method, some operations can be skipped or some operations can be added.

[0224] According to the implementation, when it is determined in operation 12011 that curUnitIdx is less than lastUnitSize, the process proceeds to operation 12012 to determine whether to skip texture map encoding. Otherwise, the process of determining whether to skip texture map encoding terminates. That is, the comparison between curUnitIdx and lastUnitSize in operation 12011 aims to determine whether to skip texture map encoding on a frame group (GoF) basis. For example, when the GoF includes 30 frames, curUnitIdx can be an index increasing from 0, 1, 2, ..., to 29, and lastUnitSize can indicate the number of frames in the GoF (i.e., 30). In other words, this is an example of determining whether to skip texture map encoding on a frame-by-frame basis within the GoF.

[0225] When it is determined in operation 12011 that curUnitIdx is less than lastUnitSize, it is checked whether the underlying mesh type is I (operation 12012). When the underlying mesh type is I, the process for determining whether to skip the texture map for the current frame is not performed (i.e., texture map encoding is not skipped). Otherwise (e.g., when the underlying mesh type is P or skip), the process proceeds to operation 12013 to determine whether to skip texture map encoding for the current frame. Operation 12013 may be referred to as texture map reference information and skip determiner. Although this disclosure describes not skipping texture map encoding when the underlying mesh type is I, this is only one implementation. That is, texture map encoding can be skipped even when the underlying mesh type is I.

[0226] According to the implementation, in this disclosure, the texture skip flag can only be sent when the type of the current mesh or the base mesh is for inter-frame prediction. When the type of the current mesh is for intra-frame prediction, the transmission of the texture skip flag can be skipped, and the texture skip flag can be implicitly deduced to be 0.

[0227] According to the implementation, when the current mesh type is a mesh type used for inter-frame prediction, the texture map reference information and texture map skip information can be determined by the texture map reference information and the skip determiner (i.e., operation 12013).

[0228] According to the implementation method, when the current grid meets the conditions for flag transmission, it can be determined whether to skip texture map encoding by comparing various metrics based on the current texture map and the reference texture map.

[0229] According to the implementation method, when the difference between the PSNR calculated based on the current texture map and the PSNR calculated based on the reference texture map is less than the threshold TH PSNRAt this time, the texture map reference information and the skip determiner (i.e., operation 12013) can determine that the texture map skip flag is 1 (i.e., texture_skip_flag=1), and skip the encoding of the current texture map. In other words, the difference between the PSNR of the current texture map and the PSNR of the reference texture map is less than TH. PSNR This indicates a high degree of similarity between the current texture map and the reference texture map. This means that the current texture map is similar enough to the reference texture map that no encoding of the current texture map is necessary.

[0230] Here, PSNR can include point cloud-based brightness and / or chromaticity PSNR between the original mesh and the mesh constructed from current or reference texture maps and reconstructed geometric information.

[0231] Additionally, the texture map reference information and the skip determiner (i.e., operation 12013) can determine whether to skip texture map encoding by measuring the weighted PSNR by assigning weights to units or specific regions, such as subgrids or tiles.

[0232] Next, we will describe the skipped units in texture map encoding.

[0233] According to the implementation, texture map skipping information can be signaled and determined on a per-frame, per-patch, per-submesh, per-tile, or per-slice basis, or it can be implicitly derived by the encoder / decoder based on the same rules.

[0234] According to the implementation, a texture skip flag, determined on a per-frame, per-patch, per-submesh, per-tile, or per-slice basis, can be sent to the receiving device for all encoding / decoding types, or it can be sent to the receiving device only for specific types.

[0235] For example, when determining whether to skip texture map encoding on a per-frame basis, a frame-level texture map skip flag (afps_vdmc_texture_skip_flag) can be sent. That is, the frame-level texture map skip flag (afps_vdmc_texture_skip_flag) can be a flag indicating whether to skip texture maps on a per-frame basis.

[0236] For example, when determining whether to skip texture map encoding on a per-patch basis, a patch-level texture map skip flag (atdu_vdmc_texture_skip_flag) can be sent. That is, the patch-level texture map skip flag (atdu_vdmc_texture_skip_flag) can be a flag indicating whether to skip texture maps on a per-patch basis.

[0237] For example, when determining whether to skip texture map encoding on a per-submesh basis, a submesh-level texture map skip flag (smh_vdmc_texture_skip_flag) can be sent. That is, the submesh-level texture map skip flag (smh_vdmc_texture_skip_flag) can be a flag indicating whether to skip texture maps on a per-submesh basis.

[0238] For example, when determining whether to skip texture map encoding on a per-tile basis, a tile-level texture map skip flag (ath_vdmc_texture_skip_flag) can be sent. That is, the tile-level texture map skip flag (ath_vdmc_texture_skip_flag) can be a flag indicating whether to skip texture maps on a per-tile basis.

[0239] According to the implementation, the texture map reference information and the skip determiner (i.e., operation 12013) can determine information about the reference texture map used to generate the skipped texture map as follows.

[0240] According to embodiments, information about a reference texture map may be an index of the reference texture map, a list of reference texture maps, or reference rule information. In this disclosure, information about the referenced texture map may be used interchangeably with reference texture map information.

[0241] According to the implementation, reference texture map information can be determined by explicitly signaling the same reference texture map information to the encoder / decoder, and / or whether to skip a map can be implicitly deduced based on conditions defined by the encoder / decoder in the same way.

[0242] When reference texture map information (also known as reference information) is explicitly communicated via signals, markers indicating the orientation information of the reference texture (e.g., past or future) can be identified.

[0243] According to the implementation, after selecting the orientation of the reference texture map, information about the orientation of the reference texture map can be sent to the decoder (i.e., the receiving device). Alternatively, the same orientation can be implicitly derived by the encoder / decoder.

[0244] In this case, the direction of the texture map to be referenced can be towards a previous texture map relative to the current texture map index (when the index is less than the current texture map index), or towards a subsequent texture map on the timeline (when the index is greater than the current texture map index).

[0245] According to the implementation, when the frame sequence count (POC) gap between a reference texture map in a previous texture map direction and the current texture map is greater than a certain threshold, it can be preferentially determined whether to refer to a reference texture map in a subsequent texture map direction, and then the reference texture direction information can be signaled. Alternatively, when the POC gap is greater than a certain threshold, it can be determined whether to refer to a future texture map, and the reference texture direction information can be implicitly derived according to the same rules between the encoder and decoder.

[0246] According to the implementation, when the unit, such as a frame, submesh, tile, slice, or patch, is of type P, a previous texture map can be used as a reference texture map. When it is of type B, either a previous texture map or a future texture map can be selected.

[0247] According to the implementation, after performing a comparison using texture maps in two directions, the direction of the texture map to be referenced can be selected, and information about the direction can be determined by explicit signaling or implicitly inferred by the encoder / decoder based on the same conditions.

[0248] According to the implementation, when the information about the reference texture map is the index of the reference texture map, the similarity between the current texture map and the reference texture map can be measured to determine the texture map with high similarity as the reference texture map, or the texture map order can be considered to determine the index of the reference texture map.

[0249] According to the implementation, when the information about the reference texture map is reference rule information, the texture map located at a constant interval in the forward coding order in the texture map can be used as the reference texture map, and the information about the specific interval can be determined by explicit signaling, or the reference information can be implicitly derived by the encoder / decoder using the same interval.

[0250] According to one implementation, the encoder / decoder can determine whether to skip a texture based on the same maximum texture map distance parameter (max_skip_texturemap_distance), and this maximum texture map distance parameter (max_skip_texturemap_distance) can be sent to the decoder (i.e., the receiving device). Alternatively, when the encoder / decoder uses the same maximum texture map distance parameter, the decoder can implicitly infer whether to skip a texture map.

[0251] In this disclosure, the maximum texture map distance parameter (max_skip_texturemap_distance) can be a parameter used to deduce whether to skip a texture map.

[0252] According to the implementation, when the encoding of the Nth texture map is skipped and all texture maps within the maximum distance from the skipped texture map on the time axis are skipped consecutively, the texture skip flag (texture_skip_flag) for the (N+max_skip_texturemap_distance)th texture map can be determined to be 0.

[0253] Figure 17 An example is shown that, according to an implementation, the maximum texture map distance parameter is used to determine whether to skip a texture map. Figure 17 An implementation of determining the texture skip flag is shown when the value of the maximum texture map distance parameter (max_skip_texturemap_distance) is 3.

[0254] Reference Figure 17 As an example, when skipping the first, second, and third texture maps in the encoding order, the fourth texture map, which is located on the time axis after the maximum texture map distance parameter interval from the first skipped texture map, may not be skipped, and the texture skip flag (texture_skip_flag) may be set to 0. In this case, the texture skip flag (texture_skip_flag) does not need to be sent separately.

[0255] For example, when determining whether to skip a texture map on a per-submesh basis, texture maps of submesh that meet the conditions for skipping texture maps can be skipped on a frame-by-frame basis. When the determination is performed per GOP (or GOF) (i.e., two or more frames), texture maps of submesh that meet the texture map skipping conditions can be skipped if the total number of submesh in each frame within the group is the same and the texture maps of the submesh have the same height and the same width.

[0256] Furthermore, according to the implementation method, as a method for skipping sub-grid texture maps, the skipped texture map areas can be filled with specific values.

[0257] In addition, when skipping texture maps on a per-submesh basis, a flag indicating that texture maps should be skipped on a per-submesh basis can be signaled (smh_texture_skip_flag), or it can be deduced by the encoder / decoder according to the same rules.

[0258] When the flag indicating whether to skip texture maps on a per-frame basis (afps_vdmc_texture_skip_flag) is 0, the decision to skip texture maps on a per-submesh basis can be determined by afps_vdmc_sm_skip_enable_flag.

[0259] For example, when afps_vdmc_sm_skip_enable_flag is 0, the sub-mesh-level texture skip flag (smh_texture_skip_flag) for the corresponding frame can be deduced to be 0 and can be left unsigned.

[0260] For example, when afps_vdmc_sm_skip_enable_flag is 1, the index information indicating the direction of the reference texture (e.g., refDirection_idx) and the index information indicating the index of the reference texture (e.g., ref_texture_idx) can be signaled.

[0261] When skipping texture map encoding on a per-submesh basis, the texture map referenced by the skipped texture map of the submesh within the same frame can reference the submesh texture map of a different frame index.

[0262] Figure 18 An example is shown of a texture map skipping state and reference frame index signaled on a per-submesh basis, according to an implementation method. Specifically, Figure 18 An example is shown where the referenced texture map has different frame indices based on the sub-mesh. More specifically, Figure 18 An example is shown where each frame comprises five sub-meshes, and for a texture map with frame index 2, texture maps with sub-mesh indices 0 and 4 are skipped respectively (i.e., the smh_texture_skip_flag for sub-mesh indices 0 and 4 is 1). In this case, a texture map referenced by a sub-mesh with index 0 can have frame index 0, and a texture map referenced by a sub-mesh with index 4 can have frame index 1, allowing texture maps from different frames to be referenced.

[0263] Figure 19 This is a diagram illustrating an example of the relationship between frames referencing a texture map when texture map encoding is skipped on a per-submesh basis, according to an implementation method. Figure 19 yes Figure 18 A visual example. Therefore, as... Figure 18 As shown, the encoding of the texture map for the sub-mesh corresponding to sub-mesh indices 0 and 4 included in the sub-mesh in the frame with index 2 is skipped.

[0264] In other words, among the five sub-mesh included in the frame with frame index 2, the texture map referenced by the texture map corresponding to sub-mesh index 0 (texture 0) is the texture map of sub-mesh index 0 in the frame with frame index 0 (texture 0), and the texture map referenced by the texture map corresponding to sub-mesh index 4 (texture 4) is the texture map of sub-mesh index 4 in the frame with frame index 1 (texture 4). In other words, once the reference frame is determined, the texture map of the sub-mesh at the same position within the reference frame is referenced.

[0265] As described above, when the texture map skip determiner 11027 determines whether texture map encoding and its signaling should be performed in the texture map generated by the texture map generator 11026, the texture map encoder 11028 skips the encoding of texture maps where texture_skip_flag is 1. That is, it skips the encoding and transmission of texture maps where texture_skip_flag is 1.

[0266] Then, the texture map encoder 11028 can encode the texture map with texture_skip_flag set to 0 using a 2D video encoder or the like.

[0267] According to the implementation, when the color space of the texture map to be encoded is RGB 4:4:4, encoding can be performed after converting the color space to a color space such as YUV 4:2:0, YUV 4:4:4, YUV 4:0:0 or YCgCo.

[0268] Figure 20 A receiving device according to an embodiment is shown. In this disclosure, Figure 20 The receiving device can be referred to as a mesh data receiving device, decoder, receiver decoder, V-Mesh decoder, or dynamic mesh decoder.

[0269] Figure 20 Corresponding to Figure 1 The receiving device 110 or the mesh video decoder 113, Figure 11 or Figure 12 decoder Figure 14 The receiving device and / or the corresponding receiving and decoding device. Figure 20 Each component corresponds to hardware, software, processor, and / or a combination thereof. Figure 20 The receive (decode) operation can follow Figure 15 The reverse process of the corresponding sending (encoding) operation. In Figure 20 In this context, you can change the execution order of blocks, omit some blocks, and add new blocks.

[0270] Figure 20It can mainly include a basic mesh decoder, a displacement information decoder, and a texture map decoder. According to an embodiment, the basic mesh decoder may include a switching unit 15011, a motion vector decoder 15012, a static mesh decoder 15013, a basic mesh reconstructor 15014, a mesh subdivision unit 15015, and a mesh reconstructor 15016. According to an embodiment, the displacement information decoder may include a displacement vector decoder 15017, a displacement vector inverse quantizer 15018, a displacement vector inverse transformer 15019, and a displacement vector coordinate inverse transformer 15020.

[0271] According to the implementation, the bitstream of mesh data received by the receiver (not shown) can undergo file / fragment decapsulation and can then be demultiplexed by the demultiplexer (not shown) into a base mesh bitstream, a displacement vector bitstream, and a texture map bitstream. Where inter-frame encoding has already been applied to the current mesh, the base mesh bitstream can be a motion vector bitstream.

[0272] According to the implementation, the basic grid bitstream can be provided to the motion vector decoder 15012 or the static grid decoder 15013 via the switching section 15011.

[0273] For example, if inter-frame encoding has already been applied to the current grid, the base grid bitstream (i.e., motion vector bitstream) is received, demultiplexed, and then output to the motion vector decoder 15012 via switching section 15011. In another example, if intra-frame encoding has already been applied to the current grid, the base grid bitstream is received, demultiplexed, and then output to the static grid decoder 15013 via switching section 15011. Here, the motion vector decoder 15012 can be referred to as a motion decoder.

[0274] According to the implementation, the motion vector decoder 15012 can decode the motion vector bitstream on a per vertex or per subgroup basis.

[0275] According to an implementation, the motion vector decoder 15012 can reconstruct the final motion vector by adding the decoded motion vector difference (i.e., residual motion vector) from the bitstream to the previously decoded motion vector used as a predictor. That is, the motion vector decoder 15012 can decode the motion vector difference (or residual motion vector) from the motion vector bitstream at the vertex or subgroup (or subblock) level, perform prediction based on connection information using the previously decoded motion vector as a predictor, and add the residual motion vector to the predicted information to decode the motion vector.

[0276] According to the implementation method, the static mesh decoder 15013 can decode the base mesh bitstream to recover connection information, vertex geometry information, texture coordinates (i.e., attribute geometry information), normal information, etc. associated with the base mesh.

[0277] According to the implementation, the base mesh reconstructor 15014 can reconstruct the current base mesh based on the decoded motion vectors or the decoded base mesh. For example, if inter-frame coding has already been applied to the current mesh, the base mesh reconstructor 15014 can add the decoded (or reconstructed) motion vectors to a reference base mesh and perform inverse quantization to generate the reconstructed base mesh (i.e., the current base mesh). In another example, if intra-frame coding has already been applied to the current mesh, the base mesh reconstructor 15014 can perform inverse quantization on the decoded (or reconstructed) base mesh output by the static mesh decoder 15013 to generate the reconstructed base mesh (i.e., the current base mesh).

[0278] According to the implementation, the mesh subdivision unit 15015 can subdivide a base mesh to generate additional vertices. Depending on the subdivision method, geometric connectivity information, texture coordinate connectivity information, and texture coordinates can be implicitly derived to generate vertices.

[0279] According to the implementation, the mesh subdivision 15015 can use methods such as edge midpoint, loop, or Catmul & Clark to perform subdivision.

[0280] According to the implementation, the mesh subdivision unit 15015 can perform mesh subdivision n times according to user parameters or an agreement between the encoder and decoder. According to the implementation, when the vertices of the base mesh are defined as R0, the newly generated vertices by performing one subdivision are defined as R1, ..., and the vertices generated by performing n subdivisions are defined as R... n LOD n It can be defined as follows.

[0281]

[0282] According to the embodiments, the shift vector decoder 15017 can perform video codec-based decoding on the demultiplexed shift vector bitstream that is a video bitstream, or it can perform zero-run decoding or arithmetic decoding. In this disclosure, the shift vector decoder can be used interchangeably with a shift vector transform decoder having the same meaning.

[0283] According to the implementation, the displacement vector decoder 15017 can decode the displacement vector by performing the inverse process of the displacement vector encoding method on the transmitting side to reconstruct the displacement vector.

[0284] According to an embodiment, the displacement vector coefficient inverse quantizer 15018 may inverse-quantize the displacement vector coefficients reconstructed by the displacement vector decoder 15017. According to an embodiment, different quantization parameters may be used for each axis to quantize the displacement vector transform coefficients. In this case, the displacement vector coefficient inverse quantizer 15018 may derive quantization parameters or scaling parameters based on an encoder / decoder convention to determine a quantization ratio for each LoD level.

[0285] According to an embodiment, the displacement vector inverse transform unit 15019 may perform an inverse transform corresponding to the transform performed by the encoder of the transmitting device on the inverse-quantized displacement vector coefficients, and output a displacement vector. According to an embodiment, an inverse lifting transform, an inverse wavelet transform, etc. may be performed.

[0286] When performing an inverse lifting transform, the vertex R at the k-th subdivision level may be predicted as follows k . R t 's subdivision vertex displacement vector may be used as a predictor (where t < k or t <= k) to perform displacement vector prediction at the k-th subdivision level.

[0287] According to an embodiment, when performing displacement vector prediction, an average or distance-based weighted average prediction may be performed using n closest vertices among vertices having a lower subdivision level than the current vertex based on connection information.

[0288] According to an embodiment, prediction may be performed based on the displacement vectors of n vertices used to generate the current vertex in a mesh subdivision operation.

[0289] In addition, when performing an inverse lifting transform, an analytical residual signal may be used to perform a process of updating the displacement vector of the vertex used by the encoder for prediction.

[0290] According to an embodiment, the displacement vector coordinate inverse transform unit 15020 may analyze a coordinate system transform flag (asps_vmc_ext_displacement_coordinate_system or applyLocalCoord) included in signaling information on a per-sequence, per-GOF, per-frame, or per-submesh basis. When the value of the flag is 1, the inverse transform unit may inverse-transform the inverse-quantized (or inverse-transformed) reconstructed displacement vector from a local coordinate system (n, t, b) to a standard coordinate system (x, y, z).

[0291] Furthermore, a normal vector of each vertex may be calculated based on the reconstructed vertex position information related to the reconstructed base mesh. For vertices additionally generated by a subdivision operation, a normal vector of the newly generated vertex may be assigned by performing interpolation based on the normal vectors of the vertices of the reconstructed base mesh that have been calculated.

[0292] In this case, interpolation can be performed by calculating the average or a distance-based weighted sum of the normal information associated with the base mesh used for subdivision. Alternatively, for subdivision vertices on the same plane, the normal information associated with the base mesh can be used.

[0293] Then, the tangential and bitangential vectors orthogonal to the normal vector can be calculated based on the calculated normal vector at each vertex, and the inverse transformation of the displacement vector coordinate system can be performed using Equation 1 below. In Equation 1 below, dispn[0], dispn[1], and dispn[2] represent the results of the normal, tangential, and bitangential components obtained by performing the inverse transformation and inverse quantization. In this case, based on the property that the normal, tangential, and bitangential components are orthogonal to each other, the displacement vector of the bitangential component can be calculated by the cross product of the final displacement vectors of the normal and tangential components. Alternatively, the displacement vector of the bitangential component can be derived and calculated using methods such as linear regression or multiple regression.

[0294] [Equation 1]

[0295] According to the implementation method, the inverse coordinate system transformation can always be performed without sending a flag.

[0296] The output of the displacement vector coordinate inverse transformer 15020 is provided to the mesh reconstructor 15016.

[0297] That is, the encoder of the transmitting device can transform the vertex displacement vector calculated in the (x, y, z) space based on the normal vector of each vertex into a (normal, tangential, bitangential) coordinate system (also known as a local coordinate system). In this case, the normal vector can be calculated for each subdivided vertex based on the geometric and connectivity information associated with neighboring vertices.

[0298] According to the implementation, the mesh reconstructor 15016 reconstructs the mesh based on the mesh subdivided by the mesh subdivision unit 15015 and the reconstructed displacement vector output from the displacement vector coordinate inverse transformer 15020.

[0299] According to one embodiment, the received and demultiplexed texture map bitstream is input to the texture map decoder 15021. According to another embodiment, the texture map decoder 15021 can use a video codec (e.g., a 2D scalable decoder) to decode the texture map. That is, the texture map decoder 15021 can reconstruct the texture map by applying 2D scalable decoding to the texture map. Additionally, the texture map decoder 15021 can generate a texture map skipped on the transmitting side based on the received reference texture map information included in the signaling information.

[0300] Figure 21 This is an example detailed block diagram of a texture map decoder according to an implementation method.

[0301] exist Figure 21 In the texture map decoder 15021, there may be a texture map skip flag determiner 16011, a reference texture map information determiner 16012, a texture map decoder 16013, and a texture map deducer 16014. Figure 21 Each component in the document corresponds to hardware, software, a processor, and / or a combination thereof. Figure 21 In this context, you can change the execution order of blocks, omit some blocks, and add new blocks.

[0302] According to the implementation, the texture map decoder 15021 can parse the asve_textureSkip_enable_flag that is signaled and received in the Atlas Sequence Parameter Set (ASPS). When the value of the flag is 1, it can be determined that a texture map skipping method (or technique) is applied to the corresponding content or sequence, and parameters related to texture map skipping can be parsed.

[0303] According to the implementation, the texture map decoder 15021 can parse the asve_textureSkip_present_flag sent and received in the ASPS to determine whether there are skipped texture maps in the current sequence. When the value of asve_textureSkip_present_flag is 1, it can be determined that there are skipped texture maps and a reconstruction process needs to be performed.

[0304] exist Figure 21 In the text, the texture skip flag determiner 16011 can determine the 1-bit flag texture_skip_flag by parsing. The 1-bit flag texture_skip_flag indicates whether the texture (also known as the texture map) of each unit determined by the encoder is skipped.

[0305] According to the implementation, texture map decoding and texture map skipping can be performed on a per-frame, per-patch, per-submesh, per-tile, or per-slice basis.

[0306] exist Figure 21 In this process, the reference texture map information determiner 16012 determines reference texture map information for texture maps that are determined to be skipped by the encoder of the transmitting device (i.e., texture maps whose encoding and transmission are skipped). In this case, the reference structure can be determined explicitly or implicitly.

[0307] When the reference structure is explicitly determined through parsing, the reference information can be determined by a 1-bit flag indicating the direction (past or future) of the reference texture and a flag or index indicating the difference in picture order count (POC) between the current texture map and the reference texture map.

[0308] According to the implementation, the flag or index indicating the direction of the reference texture can be parsed from the signaling information as information indicating whether the reference texture map is a past texture map or a future texture map relative to the current texture map, and / or information indicating whether texture maps in both directions are used can be parsed from the signaling information.

[0309] According to the implementation method, when the current mesh is a P-type mesh to which unidirectional prediction is performed, the flag indicating the direction of the reference texture can be omitted.

[0310] According to the implementation, when texture map skipping is performed by always referencing the texture map with the smallest POC difference from the current texture map according to the agreement between the encoder and decoder, the flag or index indicating the POC difference can be omitted.

[0311] According to the implementation method, when the POC gap between the reference texture map in the previous texture map direction and the current texture map is greater than a certain threshold, the reference texture direction information can be implicitly deduced as the future direction according to the same rules between the encoder and the decoder.

[0312] According to the implementation, when the current grid is a Type I grid that is a grid that does not reference a different frame in time, texture skipping is not performed. Therefore, the texture skip flag determiner 16011, the reference texture information determiner 16012, and the texture deducer 16014 can be omitted, and the texture skip flag (texture_skip_flag) can be deduced to 0.

[0313] According to an implementation, the texture map decoder 16013 can reconstruct the texture map by decoding the received texture map bitstream into a video bitstream using a video codec (e.g., a 2D scalable decoder). In this case, since the texture maps skipped on the transmitting side are not included in the received texture map bitstream, they are not reconstructed by the texture map decoder 16013.

[0314] The unreconstructed texture map (i.e., the texture map that was not transmitted because the encoding on the transmitting side was skipped) of the texture map decoder 16013 is generated by the texture map derivator 16014 based on the reference texture map and / or signaling information.

[0315] That is, for a texture map with texture_skip_flag set to 1, the operation of generating the current texture map based on the reference texture map can be performed by the texture map deducer 16014.

[0316] In this scenario, when referencing a single texture map, the reference texture map can be used as the current texture map based on reference information (also known as reference texture map information, or information about the reference texture map). That is, the single reference texture map is derived as the current texture map that is skipped by the texture map encoder of the transmitting device.

[0317] When deriving the current texture map by referencing multiple texture maps, the current texture map can be derived by averaging the multiple texture maps or by performing a distance-based (POC) weighted summation of the multiple texture maps.

[0318] In this disclosure, the flags for units skipped according to the texture map may include afps_vdmc_texture_skip_flag, atdu_texture_skip_flag, smh_texture_skip_flag, and ath_texture_skip_flag.

[0319] afps_vdmc_texture_skip_flag can be a flag indicating whether to skip texture maps on a per-frame basis (i.e., whether to skip texture map encoding).

[0320] atdu_texture_skip_flag can be a flag indicating whether to skip the texture map on a per-patch basis.

[0321] smh_texture_skip_flag can be a flag indicating whether to skip the texture map on a per-submesh basis.

[0322] ath_texture_skip_flag can be a flag indicating whether to skip texture maps on a per-tile basis.

[0323] Included Figure 21 The texture map skip flag determiner 16011, reference texture map information determiner 16012, and texture map deducer 16014 in the texture map decoder 15021 will be described in detail below.

[0324] Texture map skip flag determiner 16011

[0325] The encoder / decoder can use the same maximum texture map distance parameter (max_skip_texturemap_distance) to infer whether to skip a texture map.

[0326] The maximum texture map distance parameter (max_skip_texturemap_distance) is used to determine whether to skip a texture map. By omitting the texture map skip flag (texture_skip_flag), bits to be signaled can be saved.

[0327] According to the implementation, when skipping the Nth texture map and skipping all texture maps on the timeline that are within the maximum inter-frame distance relative to the skipped texture map, the texture skip flag (texture_skip_flag) for the (N+max_skip_texturemap_distance)th texture map can be set to 0.

[0328] Figure 22 An example is shown where the texture map skip flag is deduced to be 0 according to the implementation method. That is, Figure 22 An implementation is shown in which the texture map skip flag is deduced to be 0 based on the maximum texture map distance parameter. For example, when the maximum texture map distance parameter (max_skip_texturemap_distance) is set to 2, and the first and second texture maps in the encoding order are skipped, the flag texture_skip_flag for the third texture map can be deduced to be 0 through the maximum texture map distance parameter (max_skip_texturemap_distance).

[0329] Figure 23 This is a flowchart illustrating an example of deriving a texture map skip flag from a texture map skip flag determiner according to an embodiment. That is, Figure 23 An implementation of implicitly inferring whether to skip a texture map using the maximum texture map distance parameter (max_skip_texturemap_distance) is shown, and specific operations (or steps) can be added or skipped.

[0330] First, initialize the number of consecutively skipped texture maps, curSkipCount, to 0 (operation 17011).

[0331] Then, it is checked whether curUintIdx is less than lastUnitSize (operation 17012). When it is determined that curUnitIdx is less than lastUnitSize, the process proceeds to operation 17013 to determine the texture map skip flag. Otherwise, the process of determining the texture map skip flag terminates. That is, the comparison between curUnitIdx and lastUnitSize in operation 17012 aims to determine the texture map skip flag on a per GoF basis. For example, when a GoF consists of 30 frames, curUnitIdx can be an index increasing from 0, 1, 2, ..., up to 29, and lastUnitSize can indicate the number of frames in the GoF (i.e., 30).

[0332] If it is determined in operation 17012 that curUnitIdx is less than lastUnitSize, check if the underlying mesh type is I (operation 17013). If the underlying mesh type is I, the procedure for determining the texture map skip flag is not executed. Otherwise (e.g., when the underlying mesh type is P or skip), the procedure proceeds to operation 17014 to determine the texture map skip flag.

[0333] In operation 17014, check whether curSkipCount is greater than or equal to the maximum texture map distance parameter (max_skip_texturemap_distance).

[0334] When curSkipCount is greater than or equal to the maximum texture map distance parameter (max_skip_texturemap_distance), the texture map skip flag for the corresponding texture map is deduced to be 0 (operation 17015).

[0335] When curSkipCount is less than the maximum texture map distance parameter (max_skip_texturemap_distance), texture_skip_flag can be obtained through the texture map skip flag parser (not shown) or the texture map skip flag deducer (not shown) (operation 17016).

[0336] Then, check if texture_skip_flag is 1 (operation 17017). When texture_skip_flag is 1, curSkipCount can be incremented by 1 (operation 17019). When texture_skip_flag is 0, curSkipCount can be initialized to 0 (operation 17018).

[0337] Texture map deriver 16014

[0338] According to the implementation, when determining whether to skip a texture map on a per-submesh basis, the texture map derivator 16014 can derive the skipped texture map for the submesh on a per-frame basis. When skipping is performed in units of every two or more frames (e.g., GOP), the skipped texture map for the submesh can be derived if the total number of submesh per frame is the same within the group and the texture maps of the submesh have the same height and the same width.

[0339] In this case, the texture map of the sub-mesh skipped on the sending side can be derived from the texture map information associated with the sub-mesh that is referenced based on the reference information (i.e., reference texture map information) determined by the reference texture map information determiner 16012.

[0340] According to the implementation, when reference direction information is used to indicate the past or future, the most recent past or future texture map relative to the current texture map can be used as a reference texture map to deduce that the current texture map has the same value as the reference texture map.

[0341] According to the implementation, when the reference direction information indicates bidirectionality, the most recent past and future texture maps relative to the current texture map can be used as reference texture maps to derive the average of the two texture maps as the current texture map. Alternatively, the current texture map can be derived by performing a weighted average based on the POC difference between the past and future texture maps relative to the current texture map.

[0342] According to the implementation method, when using reference texture map index information, the texture map corresponding to the reference texture map index can be used as a reference texture map to deduce that the current texture map has the same value as the reference texture map.

[0343] Furthermore, when skipping the texture map on a per-submesh basis on the transmitting side, the texture map derivator 16014 can receive a flag (smh_texture_skip_flag) indicating that the parsing of the texture map is skipped on a per-submesh basis, or the parsing flag can be derived according to the same rules between the encoder and decoder.

[0344] When the flag indicating whether to skip the texture map on a per-frame basis (afps_vdmc_texture_skip_flag) is 0, the texture map deducer 16014 can receive afps_vdmc_sm_skip_enable_flag by parsing to determine whether to parse the skip flag on a per-submesh basis.

[0345] In this case, when afps_vdmc_sm_skip_enable_flag is 0, the sub-mesh-level texture skip flag (smh_texture_skip_flag) for the corresponding frame can be deduced to be 0 and can be left unparsed.

[0346] Furthermore, the index of the referenced subgrid can be deduced to be the same as the index of the skipped subgrid, or it can be deduced in the same way as the reference structure of the base grid, or it can be resolved.

[0347] When smh_texture_skip_flag is 1, the index refDirection_idx indicating the direction of the reference texture map can be resolved, and / or the index ref_texture_idx indicating the reference texture map can be resolved.

[0348] According to the implementation method, when skipping a texture map on a per-sub-mesh basis, the texture map referenced by the skipped texture map of the sub-mesh within the same frame can reference the texture map of the sub-mesh at a different frame index.

[0349] Figure 24 An example is shown of a texture map skipping state and reference frame index signaled on a per-submesh basis, according to an implementation method. Specifically, Figure 24 An example is shown where the referenced texture map has different frame indices based on the sub-mesh. More specifically, Figure 24 An example is shown where each frame comprises five sub-meshes, and for a texture map with frame index 2, texture maps with sub-mesh indices 0 and 4 are skipped respectively (i.e., the smh_texture_skip_flag for sub-mesh indices 0 and 4 is 1). In this case, a texture map referenced by a sub-mesh with index 0 can have frame index 0, and a texture map referenced by a sub-mesh with index 4 can have frame index 1, allowing texture maps from different frames to be referenced.

[0350] Figure 25 This is a diagram illustrating an example of the relationship between frames of referenced texture maps when texture map encoding is skipped on a per-submesh basis, according to an embodiment. Figure 25 yes Figure 24 A visual example. Therefore, as... Figure 24 As shown, the encoding of the texture map for the submesh corresponding to submesh indices 0 and 4 in the submesh included in the frame with index 2 is skipped.

[0351] In other words, among the five sub-mesh included in the frame with frame index 2, the texture map referenced by the texture map corresponding to sub-mesh index 0 (texture 0) is the texture map of sub-mesh index 0 in the frame with frame index 0 (texture 0), and the texture map referenced by the texture map corresponding to sub-mesh index 4 (texture 4) is the texture map of sub-mesh index 4 in the frame with frame index 1 (texture 4). In other words, once the reference frame is determined, the texture map of the sub-mesh at the same position within the reference frame is referenced.

[0352] According to an implementation, signaling information can be generated by a metadata processor (which may also be referred to as a metadata generator) (not shown) in a transmitting device (or the encoder of the transmitting device), and can be provided to the corresponding block in the transmitting device and / or to the receiving device (or the decoder of the receiving device). The metadata parser (not shown) of the receiving device can parse the received signaling information and provide it to the corresponding block. According to an implementation, each block of the receiving device can perform each operation based on the signaling information.

[0353] As mentioned above, the skip unit for texture information (i.e., texture map) can be a frame, patch, submesh, tile, or slice. That is, depending on the skip unit of the texture map (e.g., frame, patch, slice, submesh, or tile), the location of the texture skip flag (texture_skip_flag) that signals whether to skip the texture map can be determined.

[0354] Figure 26 An example of the syntax structure of the Atlas Frame Parameter Set (AFPS) in the signaling information of a bitstream according to an implementation method is shown. Specifically, Figure 26 An example of the extended RBSP syntax and semantics of the atlas frame parameter set vdmc is shown. That is, when the skip unit of the texture map is a frame, Figure 26 The afps_vdmc_extension() function can be extended from AFPS to include parameters (i.e., information) related to texture map skipping.

[0355] In this case, the texture map decoder 1521 of the receiving device can receive texture map skip states and reference information parsed on a per-frame basis, or it can receive a single parsed index.

[0356] As an example, when parsing a single index, you can parse afps_texture_skip_refDirection_idx.

[0357] As another example, when resolving state and reference information, the `afps_vdmc_texture_skip_flag` indicating whether to skip texture maps can be resolved on a per-frame basis. Then, when the flag is 0, the `afps_vdmc_sm_skip_enable_flag` indicating whether texture map skipping is enabled can be resolved on a per-sub-mesh basis within the frame. Furthermore, when `afps_vdmc_texture_skip_flag` is 1, the `refDirection_idx` indicating the direction of the reference texture map can be resolved, and / or the `ref_texture_idx` indicating the index of the reference texture map can be resolved.

[0358] More specifically, in Figure 26 In this context, afps_texture_skip_refDirection_idx is an index indicating the texture map skip state and the reference texture map direction information (i.e., on a per-frame basis).

[0359] Figure 27 This is a table illustrating the implementation of the texture map skip state and reference direction for afps_texture_skip_refDirection_idx according to the implementation method. That is, Figure 27 An implementation can be shown for signaling the codeword and meaning of the skip state and reference direction information of the texture map using a 2-bit index (texture_skip_refDirection_idx).

[0360] As an example, when the value of afps_texture_skip_refDirection_idx is 00, the texture map is not skipped and there is no reference direction.

[0361] As another example, when the value of afps_texture_skip_refDirection_idx is 01, it indicates that the texture map can be skipped and that a reference can be made to a past frame relative to the current frame.

[0362] As another example, when the value of afps_texture_skip_refDirection_idx is 10, it indicates that the texture map can be skipped and that future frames relative to the current frame can be referenced.

[0363] As another example, when the value of afps_texture_skip_refDirection_idx is 11, it indicates that the texture map can be skipped and the frame in both directions can be referenced relative to the current frame.

[0364] afps_vdmc_texture_skip_flag is a flag that indicates whether to skip texture maps on a per-frame basis.

[0365] afps_vdmc_sm_skip_enable_flag is a flag that indicates whether the resolution of the texture map skip flag is enabled on a per-submesh basis.

[0366] refDirection_idx is an index that indicates the direction information of the reference texture map.

[0367] According to the implementation, when the reference texture map direction information indicates whether the reference texture map is a past texture map or a future texture map relative to the current texture map, refDirection_idx can be u(1). Alternatively, when the information indicates a bidirectional reference between the past texture map and the future texture map, refDirection_idx can be u(v). Here, u(1) represents a value represented by 1 bit, while u(v) indicates a value represented by v bits, where v can be any number greater than 1.

[0368] ref_texture_idx indicates the reference texture frame index.

[0369] `max_skip_texturemap_distance` indicates the maximum distance between the current texture map and the skipped texture map when determining whether to skip a texture map.

[0370] Figure 28 An example of the syntax structure of an atlas tile header according to an implementation is shown. That is, when the unit skipped by the texture map is a tile, Figure 28 The atla_tile_header() function can include parameters (i.e., information) related to skipping texture maps.

[0371] In this case, the texture map decoder 1521 of the receiving device can receive texture map skip states and reference information parsed on a per-tile basis, or it can receive a single parsing index.

[0372] As an example, when resolving a single index, you can resolve ath_texture_skip_refDirection_idx.

[0373] As another example, when resolving state and reference information, the ath_vdmc_texture_skip_flag flag indicating whether to skip the texture map can be resolved on a per-tile basis. When the flag is 1, the refDirection_idx indicating the direction of the reference texture map can be resolved, and / or the ref_texture_idx indicating the index of the reference texture map can be resolved.

[0374] More specifically, in Figure 28 In this context, `ath_type` indicates the encoding type of the current atlas tile. For example, among the values ​​of `ath_type`, 0 can indicate `P_TILE`; 1 can indicate `I_TILE`; and 2 can indicate `SKIP_TILE`.

[0375] `ath_texture_skip_refDirection_idx` is an index indicating both the texture map skip state and the reference texture map direction information (i.e., on a per-image basis). For details regarding the texture map skip state and reference texture map direction information, refer to... Figure 27 The description of that will not be repeated below to avoid redundancy.

[0376] ath_texture_skip_flag is a flag that indicates whether to skip texture maps on a per-tile basis.

[0377] refDirection_idx is an index that indicates the direction information of the reference texture map.

[0378] ref_texture_idx is the reference texture frame index.

[0379] Figure 29 An example of the syntax structure of atlas tile data units according to an implementation is shown. Specifically, when the unit skipped in the texture map is a patch, Figure 29 The atlas_tile_data_unit(tileID) can include parameters (i.e., information) related to texture map skipping.

[0380] In this case, the texture map decoder 1521 of the receiving device can receive texture map skip states and reference information parsed on a per-patch basis, or it can receive a single parsed index.

[0381] As an example, when resolving a single index, you can resolve ath_texture_skip_refDirection_idx.

[0382] As another example, when resolving the state and reference information, the atdu_vdmc_texture_skip_flag indicating whether to skip the texture map can be resolved on a per-patch basis. When the flag is 1, the refDirection_idx indicating the direction of the reference texture map can be resolved, and / or the ref_texture_idx indicating the index of the reference texture map can be resolved.

[0383] More specifically, in Figure 29 In this context, atdu_patch_mode[tileID][p] indicates the patch mode of the patch with index p in the current tileID.

[0384] Figure 30 This is a table illustrating an example of determining the patch mode based on an identifier when the encoding type of the current atlas tile is I_TILE, according to an implementation method. For example, among the values ​​of atdu_patch_mode[tileID][p], 0 can indicate I_INTRA mode (i.e., non-predictive patch mode); 1 can indicate I_RAW mode (i.e., RAW point patch mode); and 2 can indicate I_EOM mode (i.e., EOM point patch mode).

[0385] Figure 31 This is a table illustrating an example of determining the patch mode based on an identifier when the encoding type of the current atlas tile is P_TILE, according to an implementation method. For example, among the values ​​of atdu_patch_mode[tileID][p], 0 can indicate P_SKIP mode (i.e., patch skip mode), 1 can indicate P_MERGE mode (i.e., patch merge mode), and 2 can indicate P_INTER mode (i.e., inter-frame prediction patch mode).

[0386] Additionally, when the encoding type of the current atlas tile is not I_TILE, the atlas tile data unit can contain information related to texture skipping (e.g., atdu_texture_skip_flag, refDirection_idx, and ref_texture_idx).

[0387] atdu_texture_skip_flag is a flag that indicates whether to skip the texture map on a per-block basis.

[0388] refDirection_idx is an index that indicates the direction information of the reference texture map.

[0389] ref_texture_idx is the reference texture frame index.

[0390] Figure 32An example of the syntax structure of the base mesh submesh header according to an implementation is shown. Specifically, when the unit skipped by the texture map is a submesh, Figure 32 The submesh_header() function can include parameters (i.e., information) related to skipping texture maps.

[0391] In this case, the texture map decoder 1521 of the receiving device can receive texture map skip states and reference information parsed on a per-submesh basis, or it can receive a single parsed index.

[0392] As an example, when resolving a single index, you can resolve smh_texture_skip_refDirection_idx.

[0393] As another example, when resolving state and reference information, the smh_vdmc_texture_skip_flag indicating whether to skip the texture map can be resolved on a per-submesh basis. When the flag is 1, the refDirection_idx indicating the direction of the reference texture map can be resolved, and / or the ref_texture_idx indicating the index of the reference texture map can be resolved.

[0394] More specifically, in Figure 32 In the provided text, `smh_texture_skip_refDirection_idx` is an index indicating the texture map skip state and the reference texture map direction information (i.e., on a per-submesh basis). For details regarding the texture map skip state and reference texture map direction information, refer to... Figure 27 The description of that will not be repeated below to avoid redundancy.

[0395] smh_texture_skip_flag is a flag that indicates whether to skip the texture map on a per-submesh basis.

[0396] refDirection_idx is an index that indicates the direction information of the reference texture map.

[0397] ref_texture_idx is the reference texture frame index.

[0398] Figure 33 Another example of the syntax structure of the base mesh submesh header according to an implementation is shown. Specifically, when the unit skipped by the texture map is a submesh, Figure 33 The submesh_header() function can include parameters (i.e., information) related to skipping texture maps.

[0399] exist Figure 33In this context, afps_vdmc_texture_skip_flag is a flag indicating whether to skip texture maps on a per-frame basis.

[0400] afps_vdmc_sm_skip_enable_flag is a flag that indicates whether the resolution of the texture map skip flag is enabled on a per-submesh basis.

[0401] sm_index is the subgrid index.

[0402] `smh_texture_skip_refDirection_idx` is an index indicating both the texture map skip state and the reference texture map direction information (i.e., on a per-submesh basis). For details regarding the texture map skip state and reference texture map direction information... Figure 27 The description of that will not be repeated below to avoid redundancy.

[0403] smh_texture_skip_flag is a flag that indicates whether to skip the texture map on a per-submesh basis.

[0404] refDirection_idx is an index that indicates the direction information of the reference texture map.

[0405] ref_texture_idx is the reference texture frame index.

[0406] Next, as Figures 26 to 33 As shown, a description of the operation of the transmitting and receiving devices is provided when information about texture map skipping is signaled on a per-frame, per-tile, per-patch, or per-submesh basis. Here, the transmitting device can be... Figure 12 The transmitting device 100 or the grid video encoder 102, Figure 2 , Figure 6 or Figure 7 The encoder (preprocessor and encoder). Figure 13 The transmitting device or Figure 15 Any of the transmitting devices. In this disclosure, Figure 15 The transmitting device is described as an example. Furthermore, the receiving device can be... Figure 1 The receiving device 110 or the mesh video decoder 113, Figure 11 or Figure 12 decoder Figure 14 Receiving device or Figure 20 Any of the receiving devices. In this disclosure, Figure 20 The receiving device is described as an example.

[0407] Specifically, in the transmitting device, a raw mesh to be transmitted is generated as a base mesh by a mesh simplifier 11011 and a mesh parameterizer 11012. The generated base mesh is quantized by a mesh quantizer 11013. In the case of inter-frame transmission, the motion vectors from the base mesh reconstructed from the previous reference are calculated and encoded by a motion vector encoder 11015. In the case of intra-frame transmission, the base mesh is transmitted as a base mesh bitstream by a static mesh encoder 11016. Additionally, the mesh simplified by the mesh simplifier is subdivided and fitted by a mesh subdivision 11018 and a mesh fitter 11019. A displacement vector calculator 11020 calculates the displacement vector between the subdivided and fitted mesh data and the reconstructed mesh data from the previously encoded base mesh. To efficiently encode the calculated displacement vectors, a displacement vector coordinate transformer 11021 can transform the displacement vector coordinate system to a local coordinate system. Then, the displacement vectors are transformed into displacement vector coefficients by the displacement vector transformer in the displacement vector encoder 11022 and quantized. A 2D video encoder, such as H.264, HEVC, or VVC, is used to encode 2D images packed by a displacement vector coefficient packer to generate a displacement vector bitstream.

[0408] Texture map generator 11026 generates a new texture map that has color information corresponding to the texture coordinates of the reconstructed mesh. The texture map is encoded by a 2D video encoder and sent as a texture bitstream.

[0409] In this disclosure, during the operation of generating a texture map and encoding it into a texture map bitstream, the texture map skip determiner 11027 can determine whether to skip encoding the texture map, and only encodes texture maps other than those determined to be skipped. Here, the texture maps determined to be skipped are neither encoded nor transmitted.

[0410] In this disclosure, after generating a texture map based on the reconstructed mesh data, the texture map skip determiner 11027 can determine whether to encode / decode the texture map. At this point, whether to encode / decode the texture map can be determined by explicitly signaling the texture map skip flag (texture_skip_flag), or whether to skip encoding can be implicitly deduced based on conditions defined in the same way by the encoder / decoder. Additionally, when explicitly signaling reference texture map information (i.e., reference information), the refDirection_idx indicating reference texture direction information (e.g., past or future) can be determined and signaled, and / or the ref_texture_idx indicating the reference texture map index can be signaled. Furthermore, the encoder / decoder can determine whether to skip the texture map based on the same maximum texture map distance parameter (max_skip_texturemap_distance), and the maximum texture map distance parameter can be sent to the decoder. Alternatively, whether to skip the texture map can be deduced at the decoder (i.e., the texture map decoder) based on the same maximum texture map distance parameter shared between the encoder and decoder.

[0411] According to the implementation, the criteria used to determine whether to skip a texture map may include the similarity between the current texture map and the reference texture map, the estimated number of generated bits, and the distance between the current texture map and the reference texture map. Furthermore, the unit for skipping texture information (i.e., texture map) can be a frame, sub-mesh, patch, tile, or slice. Depending on the unit of texture map skipping, the texture map skip flag can be signaled as afps_vdmc_texture_skip_flag (on a frame basis), smh_texture_skip_flag (on a sub-mesh basis), atdu_texture_skip_flag (on a patch basis), or ath_texture_skip_flag (on a tile basis).

[0412] As described above, in the texture maps generated by the texture map generator 11026, the encoding of texture maps whose texture_skip_flag is determined to be 1 by the texture map skip determiner 11027 can be skipped. Conversely, the texture map encoder 11028 can encode texture maps whose texture_skip_flag is 0 using a 2D video encoder or the like.

[0413] Then, the base mesh bitstream, displacement vector bitstream, and texture bitstream generated in the transmitting device through the entire process described above are multiplexed into a single bitstream by a multiplexer and sent to the receiving device by a transmitter.

[0414] The receiving device receives the bit stream sent by the transmitting device and decodes the basic mesh bit stream, displacement vector bit stream and texture map bit stream respectively through a demultiplexer.

[0415] First, in the case of inter-frame decoding, the base grid bitstream is decoded by motion vector decoder 15012, or in the case of intra-frame decoding, the base grid bitstream is decoded by static grid decoder 15013. The decoded base grid undergoes grid subdivision through base grid reconstructor 15014 and grid subdivision 15015.

[0416] The displacement vector decoder 15017 decodes the displacement vector bitstream in the reverse order of encoding and recovers the displacement vector coefficients. The recovered displacement vector coefficients are then inversely quantized and transformed by the displacement vector inverse quantizer 15018 and the displacement vector inverse transformer 15019. Finally, the displacement vector coordinate inverse transformer 15020 inversely transforms the coordinate system of the inversely transformed displacement vector from the local coordinate system to the Cartesian coordinate system to reconstruct the mesh geometry information together with the base mesh data.

[0417] The mesh reconstructor 15016 can reconstruct the final geometric information by adding the reconstructed displacement vector to the vertices generated by the subdivision operation of the mesh subdivisionor 15015 to calculate the vertex geometry information associated with the reconstructed mesh.

[0418] The texture map decoder 15021 reconstructs the texture map from the texture map bitstream. The operation of the texture map decoder 15021 according to this disclosure will be described in more detail below.

[0419] Specifically, when the texture map bitstream is input to the texture map decoder 15021, the texture skip flag determiner 16011 can first determine whether to skip a texture map based on the parsed 1-bit flag `texture_skip_flag`, which indicates whether the texture map for each unit is skipped as determined by the transmitting device. Here, depending on the texture map skip unit, the texture map skip flag can be parsed as `afps_vdmc_texture_skip_flag` (on a frame basis), `smh_texture_skip_flag` (on a sub-mesh basis), `atdu_texture_skip_flag` (on a patch basis), or `ath_texture_skip_flag` (on a tile basis).

[0420] According to the implementation, when the texture map skipping unit is a frame, the texture map skipping state and reference information can each be parsed on a per-frame basis, or they can be parsed as a single index. When they are parsed as a single index, `afps_texture_skip_refDirection_idx` can be parsed. When the state and reference information are parsed separately, `afps_vdmc_texture_skip_flag`, indicating whether the texture map is skipped on a per-frame basis, can be parsed. Then, when the flag is 0, the `afps_vdmc_sm_skip_enable_flag`, indicating whether texture map skipping is enabled, can be parsed on a per-submesh basis within the frame. When `afps_vdmc_texture_skip_flag` is 1, the `refDirection_idx`, indicating the direction of the reference texture map, and / or the `ref_texture_idx`, indicating the index of the reference texture map, can be parsed.

[0421] According to the implementation, when the texture map skip unit is a tile, the texture map skip status and reference information can be resolved on a per-tile basis, or they can be resolved to a single index. When they are resolved to a single index, `ath_texture_skip_refDirection_idx` can be resolved. When the status and reference information are resolved individually, `ath_texture_skip_flag` indicating whether the texture map is skipped on a per-tile basis can be resolved. Then, when the flag is 1, `refDirection_idx` indicating the direction of the reference texture map can be resolved, and / or `ref_texture_idx` indicating the index of the reference texture map can be resolved.

[0422] According to the implementation, when the texture map skip unit is a patch, the texture map skip status and reference information can each be parsed on a per-patch basis, or they can be parsed as a single index. When the status and reference information are parsed, the atdu_texture_skip_flag indicating whether the texture map is skipped on a per-patch basis can be parsed. Then, when the flag is 1, the refDirecion_idx indicating the direction of the reference texture map can be parsed, and / or the ref_texture_idx indicating the index of the reference texture map can be parsed.

[0423] According to the implementation, when the texture map skip unit is a submesh, the texture map skip state and reference information can be resolved on a per-submesh basis, or they can be resolved as a single index. When they are resolved as a single index, `smh_texture_skip_refDirection_idx` can be resolved. When the state and reference information are resolved individually, `smh_texture_skip_flag` indicating whether the texture map is skipped on a per-submesh basis can be resolved. Then, when the flag is 1, `refDirection_idx` indicating the direction of the reference texture map can be resolved, and / or `ref_texture_idx` indicating the index of the reference texture map can be resolved. Alternatively, when the state and reference information are resolved individually, `afps_vdmc_texture_skip_flag` can be resolved from the Atlas Frame Parameter Set (AFPS). Then, when the flag is 0, `afps_vdmc_sm_skip_enable_flag` indicating whether texture map skipping is enabled can be resolved within the frame on a per-submesh basis. When afps_vdmc_sm_skip_enable_flag is 1, the smh_texture_skip_flag indicating whether to skip texture maps for each sub-mesh within a frame can be resolved. When smh_texture_skip_flag is 1, the refDirection_idx indicating the direction of the reference texture map and / or the ref_texture_idx indicating the index of the reference texture map can be resolved.

[0424] Additionally, the encoder / decoder can infer whether to skip a texture map based on the same maximum texture map distance parameter (max_skip_texturemap_distance). The maximum texture map distance parameter (max_skip_texturemap_distance) can be used to infer whether to skip a texture map. By omitting the texture map skip flag, bits to be signaled can be saved. When the number of consecutively skipped texture maps, curSkipCount, is greater than or equal to the maximum texture map distance parameter (max_skip_texturemap_distance), the texture map skip flag for the corresponding texture map can be inferred to be 0. When curSkipCount is less than max_skip_texturemap_distance, texture_skip_flag can be obtained through the texture skip flag parser or texture skip flag deducer. When texture_skip_flag is 1, curSkipCount can be incremented by 1. When texture_skip_flag is 0, curSkipCount can be initialized to 0.

[0425] Once the texture map has been reconstructed through the above process, it is used together with the previously reconstructed geometric information to generate the final reconstructed mesh.

[0426] As described above, according to this disclosure, when the similarity between a reference texture map and a current texture map is high, the encoding of the current texture map can be skipped. In this case, the unit for skipping texture map encoding can be a frame, sub-mesh, patch, tile, or slice. That is, texture map encoding can be skipped by determining the similarity at the skipped unit. Typically, among the main components of mesh data (i.e., the base mesh, displacement vector, and texture map), the texture map occupies the majority of the bit capacity, usually approximately 60% to 80%, depending on the mesh content and transmission conditions. This disclosure achieves significant bit savings and advantages in transmission speed by skipping the encoding and transmission of a portion of the texture map that occupies the majority of the bits.

[0427] Figure 34 Another example of the syntax structure of the Atlas Sequence Parameter Set (ASPS) in the signaling information of a bitstream according to an implementation is shown. Specifically, Figure 34 An example is shown of the extended RBSP syntax and semantics of the atlas sequence parameter set vdmc. Specifically, Figure 34 The asps_vdmc_extension() function can include flags indicating whether a texture map skipping method is used on a per-sequence basis and whether skipped frames exist within that sequence. In other words, in this disclosure, the flags indicating whether a texture map skipping method is used on a per-sequence basis and whether skipped frames exist within that sequence can be signaled via the atlas sequence parameter set vdmc extension and sent to the receiving device.

[0428] exist Figure 34 In this context, `asve_textureSkip_enable_flag` is a flag used to determine whether the texture map skipping method is enabled on a per-sequence basis. When `asve_textureSkip_enable_flag` is 1, the texture map skipping method can be executed. When `asve_textureSkip_enable_flag` is 0, it indicates that the texture map skipping method is not used.

[0429] `asve_textureSkip_present_flag` is a flag indicating whether skipped frames exist in the sequence. A value of 1 indicates the presence of skipped frames in the sequence. A value of 0 indicates the absence of skipped frames in the sequence.

[0430] According to embodiments of this disclosure, a flag indicating that a texture map should be skipped on a per-frame basis (texturemap_skip_flag) and information regarding the reference direction (ref_direction_idx) of the skipped texture map can be signaled via one of the AFPS, tile header, patch unit header, and NAL unit header, and these are sent to the receiving device. Here, textureSkip_present_flag can have the same characteristics as... Figures 26 to 33 The same meaning as texture_skip_flag in the text, and ref_direction_idx can have the same meaning as Figures 26 to 33 The same meaning as refDirection_idx in the text.

[0431] Figure 35 Another example of the syntax structure of AFPS in signaling information in a bitstream according to an implementation is shown. Specifically, Figure 35 An example of the extended RBSP syntax and semantics of the atlas frame parameter set vdmc is shown. That is, Figure 35 The `afps_vdmc_extension()` function can include a flag `texturemap_skip_flag` indicating that texture maps are skipped on a per-frame basis, and information about the reference direction (`ref_direction_idx`) of the skipped texture maps. In other words, information indicating that texture maps for the V-DMC extension of the reference atlas frame parameter set are skipped and information about the reference direction of the skipped texture maps can be signaled.

[0432] exist Figure 35 In this context, `afve_texturemap_skip_flag` is a flag indicating whether the texture map for the current frame is skipped. When `afve_texturemap_skip_flag` is 1, it indicates that the frame with the texture map skipped is skipped. When `afve_texturemap_skip_flag` is 0, it indicates that the frame is not a frame with the texture map skipped.

[0433] afve_ref_direction_idx indicates the reference direction information associated with the skipped texture map.

[0434] Figure 36 This is a table showing the definitions of afve_texturemap_skip_flag and afve_ref_direction_idx according to the implementation.

[0435] For example, an afve_ref_direction_idx value of 0 indicates that only past frames are referenced. An afve_ref_direction_idx value of 1 indicates that only future frames are referenced. An afve_ref_direction_idx value of 2 indicates that frames in both directions are referenced.

[0436] When the value of afve_texturemap_skip_flag is 0, the value of afve_ref_direction_idx may not exist or may be filled with a predefined value.

[0437] Figure 37 This is a diagram illustrating an example of texture map skipping according to an implementation. Specifically, the diagram shows an example of syntax information when relevant information is signaled via AFPS in the case of skipping texture map encoding.

[0438] Reference Figure 37 Frames 0, 2, 3, and 5 correspond to non-skipped frames, while frames 1 and 4 correspond to skipped frames. In this case, Figure 37 It is shown that when a frame with frame index 1 is generated, the texture map decoder of the receiving device refers to the past frame (frame 0), and when a frame with frame index 4 is generated, it refers to the future frame (frame 5).

[0439] Figure 38 (a) and Figure 38 (b) illustrates an example of determining texture map skip states and reference information indices on a frame-by-frame basis according to an implementation method. Specifically, Figure 38 (a) is a table showing an example of texture map skip states and reference information determined on a per-frame basis, and Figure 38 (b) shows an example of syntax information based on the atlas frame parameter set identifier (atlas_frame_parameter_set_id).

[0440] Reference Figure 38 (a) serves as an example. When skipping frames 1 and 4 (frameIdx is 1 and 4), the afve_texturemap_skip_flag for the frames can be set to 1. In this case, when referring to past frames for frame 1, afve_ref_direction_idx can be set to 0. When referring to future frames for frame 4, afve_ref_direction_idx can be set to 1.

[0441] For frames 0, 2, 3, and 5, which are non-skipped frames, afve_texturemap_skip_flag can be set to 0.

[0442] Figure 38 (b) shows an example of added AFPS based on a combination of skipped state and reference information selected from the texture map.

[0443] For example, a texture map that is not skipped (afve_texturemap_skip_flag=0) can be mapped to AFPS ID 0. A texture map that is skipped (afve_texturemap_skip_flag=1) with a reference index of 0 can be mapped to AFPS ID 1. A texture map that is skipped (afve_texturemap_skip_flag=1) with a reference index of 1 can be mapped to AFPS ID 2.

[0444] like Figure 37 , Figure 38 (a) and Figure 38 As shown in (b), the syntax information varies depending on whether texture map encoding is skipped and the texture map reference orientation. Whenever the syntax information changes, the value of `afps_atlas_frame_parameter_set_id` can be incremented by 1, and the new syntax information can be signaled for differentiation. For example, in the `afps_frame_tile_information` syntax, if a matching set does not exist in the previously generated AFPSs, a new AFPS can be added.

[0445] According to the implementation, information regarding whether to skip texture map encoding and reference information for frame derivation can be signaled and sent on a per-tile basis via the atlas icon header.

[0446] Figure 39 An example of the syntax structure of an atlas tile layer according to an implementation is shown. According to an implementation, the atlas tile layer may include an atlas tile header (atlas_tile_header()) and atlas tile data units (atlas_tile_data_unit()).

[0447] Figure 40 An example of the syntax structure of the atlas tile header (atlas_tile_header()) according to the implementation is shown.

[0448] exist Figure 40 In this context, `ath_texture_skip_flag` is a flag indicating whether a texture map is skipped. For example, `ath_texture_skip_flag` equal to 1 indicates that the texture map is skipped, and `ath_texture_skip_flag` equal to 0 indicates that the texture map is not skipped.

[0449] `ath_ref_direction_idx` is an index indicating the reference direction when deriving skipped texture maps. For example, `ath_ref_direction_idx` equal to 0 indicates referencing a texture map in a past direction. `ath_ref_direction_idx` equal to 1 indicates referencing a texture map in a future direction. `ath_ref_direction_idx` equal to 2 indicates referencing texture maps in both directions.

[0450] According to the implementation, the texture map skipping status and reference information used for frame derivation can be signaled and sent on a per-patch basis via patch data units and / or inter-patch data units.

[0451] Figure 41 An example of the syntax structure of patch information data (patch_information_data(tileID, patchIdx, patchMode)) according to an implementation is shown.

[0452] When ath_type indicates P_TILE and patchMode indicates P_INTRA Figure 41 The patch information data (patch_information_data(tileID, patchIdx, patchMode)) can include patch data units (patch_data_unit(tileID, patchIdx)). When ath_type indicates P_TILE and patchMode indicates P_INTER, the patch information data can include inter_patch_data_unit(tileID, patchIdx). Furthermore, when ath_type indicates I_TILE and patchMode indicates I_INTRA, the patch information data can include patch data units (patch_data_unit(tileID, patchIdx)).

[0453] Figure 42 An example of the syntax structure of a patch data unit according to an implementation is shown. That is, when the tile type is P_TILE and the patch mode is P_INTRA, or when the tile type is I_TILE and the patch mode is I_INTRA, as shown... Figure 42 The patch mode data unit syntax shown is sent to the receiving device. Figure 42In the context of texture graph decoders, when asve_textureSkip_enable_flag and asve_textureSkip_present_flag are equal to 1 and the tile type is not I_TILE, pdu_texture_skip_flag can be parsed.

[0454] exist Figure 42 In this context, `pdu_texture_skip_flag` is a flag indicating whether a texture map is skipped. For example, `pdu_texture_skip_flag` equal to 1 indicates that the texture map is skipped, and `pdu_texture_skip_flag` equal to 0 indicates that the texture map is not skipped.

[0455] `pdu_ref_direction_idx` is an index indicating the reference direction when deriving skipped texture maps. For example, `pdu_ref_direction_idx` equal to 0 indicates referencing a texture map in a past direction. `pdu_ref_direction_idx` equal to 1 indicates referencing a texture map in a future direction. `pdu_ref_direction_idx` equal to 2 indicates referencing texture maps in both directions.

[0456] Figure 43 An example of the syntax structure of inter-patch data units according to an implementation is shown. That is, when the tile type is P_TILE and the patch mode is I_INTER, as shown... Figure 43 The inter-pattern data unit syntax shown is sent to the receiving device. When asve_textureSkip_enable_flag and asve_textureSkip_present_flag are equal to 1, ipdu_texture_skip_flag can be parsed for the texture map decoder.

[0457] exist Figure 43 In this context, `ipdu_texture_skip_flag` is a flag indicating whether a texture map is skipped. For example, an `ipdu_texture_skip_flag` of 1 indicates that the texture map is skipped, and an `ipdu_texture_skip_flag` of 0 indicates that the texture map is not skipped.

[0458] `ipdu_ref_direction_idx` is an index indicating the reference direction when deriving skipped texture maps. For example, `ipdu_ref_direction_idx` equal to 0 indicates referencing a texture map in a past direction. `ipdu_ref_direction_idx` equal to 1 indicates referencing a texture map in a future direction. `ipdu_ref_direction_idx` equal to 2 indicates referencing texture maps in both directions.

[0459] According to the implementation, atlas_frame_rbsp(), which serves as a structure for sending frame-level information, can be added to the existing NAL unit parsing structure, and the texture map skip flag and reference information index for deriving skipped frames can be sent to the receiving device.

[0460] Figure 44 An example of the semantics of the NAL unit header according to an implementation is shown. That is, Figure 44 An example is shown where atlas_frame_rbsp() is included in the NAL cell header.

[0461] Figure 45 An example of the syntax structure of atlas_frame_rbsp() according to an implementation is shown. According to the implementation, af_texture_skip_flag can be parsed for the texture map decoder when both asve_textureSkip_enable_flag and asve_textureSkip_present_flag are equal to 1.

[0462] exist Figure 45 In this context, `af_texture_skip_flag` is a flag indicating whether a texture map is skipped. For example, `af_texture_skip_flag` equal to 1 indicates that the texture map is skipped, and `af_texture_skip_flag` equal to 0 indicates that the texture map is not skipped.

[0463] `af_ref_direction_idx` is an index indicating the reference direction when deriving skipped texture maps. For example, `af_ref_direction_idx` equal to 0 indicates referencing the texture map in the past direction. `af_ref_direction_idx` equal to 1 indicates referencing the texture map in the future direction. `af_ref_direction_idx` equal to 2 indicates referencing texture maps in both directions.

[0464] in this case, Figures 34 to 45 The textureSkip_present_flag in the text can have the same characteristics as... Figures 26 to 33The same meaning as `texture_skip_flag` in [the original text]. Additionally, Figures 34 to 45 The ref_direction_idx in the context can have the same characteristics as... Figures 26 to 33 The same meaning as refDirction_idx in the example.

[0465] The following text will describe when... Figures 34 to 45 As shown, the operation of the transmitting and receiving devices is indicated by signals in at least one of ASPS, AFPS, atlas tile header, patch data unit, inter-patch data unit, or atlas_frame_rbsp when the texture map skip flag and reference information index used to derive the skipped frame are communicated. Here, the transmitting device can be... Figure 12 The transmitting device 100 or the grid video encoder 102, Figure 2 , Figure 6 or Figure 7 The encoder (preprocessor and encoder). Figure 13 The transmitting device or Figure 15 Any of the transmitting devices. In this disclosure, Figure 15 The transmitting device is described as an example. Furthermore, the receiving device can be... Figure 1 The receiving device 110 or the mesh video decoder 113, Figure 11 or Figure 12 decoder Figure 14 Receiving device or Figure 20 Any of the receiving devices. In this disclosure, Figure 20 The receiving device is described as an example.

[0466] Specifically, in the transmitting device, a raw mesh to be transmitted is generated as a base mesh by a mesh simplifier 11011 and a mesh parameterizer 11012. The generated base mesh is quantized by a mesh quantizer 11013. In the case of inter-frame transmission, the motion vectors from the base mesh reconstructed from the previous reference are calculated and encoded by a motion vector encoder 11015. In the case of intra-frame transmission, the base mesh is transmitted as a base mesh bitstream by a static mesh encoder 11016. Additionally, the mesh simplified by the mesh simplifier is subdivided and fitted by a mesh subdivision 11018 and a mesh fitter 11019. A displacement vector calculator 11020 calculates the displacement vector between the subdivided and fitted mesh data and the reconstructed mesh data from the previously encoded base mesh. To efficiently encode the calculated displacement vectors, a displacement vector coordinate transformer 11021 can transform the displacement vector coordinate system to a local coordinate system. Then, the displacement vectors are transformed into displacement vector coefficients and quantized by a displacement vector transformer in a displacement vector encoder 11022. A 2D video encoder, such as H.264, HEVC, or VVC, is used to encode 2D images packed by a displacement vector coefficient packer to generate a displacement vector bitstream.

[0467] Texture map generator 11026 generates a new texture map with color information corresponding to the texture coordinates of the reconstructed mesh.

[0468] Subsequently, it can be determined whether to apply texture map skipping technology based on the current content or sequence. Based on this determination, it is determined whether to operate the texture map skipping determiner 11027. When texture map skipping technology is applied, the value of asve_textureSkip_enable_flag can be signaled as 1 via the ASPS V-DMC extension to send information about whether texture map skipping technology is applied and texture map-related parameters to the receiving device.

[0469] Texture map skip determiner 11027 determines whether to skip the encoding of the texture map for the current frame. Texture map skipping can be determined when the difference between the PSNR calculated based on the metric between the original mesh and the current mesh and the PSNR calculated based on the metric between the original mesh and the reference mesh is less than or equal to a predetermined threshold. When a skipped texture map exists in the current sequence, the value of `asve_textureSkip_present_flag` can be signaled as 1 to deliver information to the receiving device indicating that a skipped texture exists in the current sequence and that the skipped texture map needs to be reconstructed. Additionally, when a skip for texture map encoding for the current frame is determined, a texture map encoding skip flag and reference direction information can be signaled. In this disclosure, four methods for signaling syntax (i.e., information) related to frame-level texture map encoding skipping are proposed below.

[0470] As a first method, when it is determined that texture map encoding is skipped, notification can be performed via a signal using the V-DMC extension of AFPS, such as... Figure 34 As shown. A signal (afve_texturemap_skip_flag) can be used to indicate whether the texture map of the current frame has been skipped. In this case, a fve_texturemap_skip_flag equal to 1 indicates that the texture map of the current frame has been skipped, and a fve_texturemap_skip_flag equal to 0 indicates that the texture map has not been skipped. Additionally, when skipping a texture map, a signal (afve_ref_direction_idx) can be used to indicate the reference direction. In this case, a fve_ref_direction_idx equal to 0 indicates a reference to a past frame, a fve_ref_direction_idx equal to 1 indicates a reference to a future frame, and a fve_ref_direction_idx equal to 2 indicates a reference to frames in both directions. When the texture map has not been skipped, the reference direction information (afve_ref_direction_idx) may not exist, or it may be assigned or filled with a predetermined value (e.g., afve_ref_direction_idx = 3). Additionally, when the values ​​or information of afve_texturemap_skip_flag and afve_ref_direction_idx within the AFPS V-DMC extension are different, afps_atlas_frame_parameter_set_id can be assigned different values ​​such as 0, 1, 2, etc., and notified by a signal.

[0471] As a second method, such as Figure 40As shown, the `atlas_tile_header` can signal the skip parameters for the texture map and send them to the receiving device. When the tile type is not type I and the texture map is skipped, the `ath_texture_skip_flag` can be used to signal whether the texture map of the current frame is skipped. In this case, `ath_texture_skip_flag` equal to 1 indicates that the texture map of the current frame is skipped, and `ath_texture_skip_flag` equal to 0 indicates that the texture map is not skipped. Additionally, when skipping a texture map, the reference direction (`ath_ref_direction_idx`) can be signaled. In this case, `ath_ref_direction_idx` equal to 0 indicates a reference to a past frame, `ath_ref_direction_idx` equal to 1 indicates a reference to a future frame, and `ath_ref_direction_idx` equal to 2 indicates a reference to frames in both directions. When the texture map is not skipped, the reference direction information (`ath_ref_direction_idx`) may not exist, or it may be assigned or filled with a predetermined value (e.g., `ath_ref_direction_idx=3`).

[0472] As a third method, parameters related to texture skipping can be signaled and sent via patch_data_unit. When the patch type is P_TILE and the patch mode is P_INTRA, or when the patch type is I_TILE and the patch mode is I_INTRA, information indicating whether the texture of the current frame is skipped (pdu_texture_skip_flag) and reference direction information (pdu_ref_direction_idx) can be sent to the receiving device via the patch data unit(), such as... Figure 42 As shown. When the tile type is P_TILE and the patch mode is I_INTER, the information indicating whether the texture map of the current frame is skipped (ipdu_texture_skip_flag) and the reference direction information (ipdu_ref_direction_idx) can be sent through the inter-patch data unit (), such as Figure 43 As shown.

[0473] As a fourth method, it can be achieved through methods such as... Figure 44 and Figure 45 The nal_unit_header shown uses a signal to notify and send a texture map to skip related parameters. In this disclosure, it is possible to... Figure 44The structure atlas_frame_rbsp(), used for sending frame-level information, is newly added to the existing nal unit parsing structure. Through this structure, it is possible to... Figure 45 The diagram shows the use of signals to notify the receiving device of the texture skip flag (af_texture_skip_flag) and reference direction information (af_ref_direction_flag). Similarly, af_texture_skip_flag equal to 1 indicates that the texture of the current frame is skipped, and af_texture_skip_flag equal to 0 indicates that the texture is not skipped. Additionally, af_ref_direction_flag equal to 0 indicates a reference to a past frame, af_ref_direction_flag equal to 1 indicates a reference to a future frame, and af_ref_direction_flag equal to 2 indicates a reference to frames in both directions. When the texture is not skipped, the reference direction information (af_ref_direction_flag) may be absent, or it may be assigned or filled with a predetermined value (e.g., af_ref_direction_flag = 3).

[0474] As described above, in the texture maps generated by the texture map generator 11026, the encoding and transmission of texture maps for which the texture map skip determiner 11027 sets texture_skip_flag to 1 can be skipped. Conversely, the texture map encoder 11028 can encode texture maps for which texture_skip_flag is 0 using a 2D video encoder or the like.

[0475] Then, the base mesh sub-bit stream, displacement vector sub-bit stream, and texture map sub-bit stream generated in the transmitting device through the entire process described above are multiplexed into a single bit stream by a multiplexer and sent to the receiving device by a transmitter.

[0476] The receiving device receives the bit stream sent by the transmitting device and decodes the basic mesh bit stream, displacement vector bit stream and texture map bit stream respectively through a demultiplexer.

[0477] First, in the case of inter-frame decoding, the base grid bitstream is decoded by motion vector decoder 15012, or in the case of intra-frame decoding, the base grid bitstream is decoded by static grid decoder 15013. The decoded base grid undergoes grid subdivision through base grid reconstructor 15014 and grid subdivision 15015.

[0478] The displacement vector decoder 15017 decodes the displacement vector bitstream in the reverse order of encoding and recovers the displacement vector coefficients. The recovered displacement vector coefficients are then inversely quantized and transformed by the displacement vector inverse quantizer 15018 and the displacement vector inverse transformer 15019. Finally, the displacement vector coordinate inverse transformer 15020 inversely transforms the coordinate system of the inversely transformed displacement vector from the local coordinate system to the Cartesian coordinate system to reconstruct the mesh geometry information together with the base mesh data.

[0479] The mesh reconstructor 15016 can reconstruct the final geometric information by adding the reconstructed displacement vector to the vertices generated by the subdivision operation of the mesh subdivisionor 15015 to calculate the vertex geometry information associated with the reconstructed mesh.

[0480] The texture map decoder 15021 reconstructs the texture map from the texture map bitstream. The operation of the texture map decoder 15021 according to this disclosure will be described in more detail below.

[0481] First, the texture map decoder 15021 can parse the `asve_textureSkip_enable_flag` from the ASPS to determine whether to apply the texture map skipping technique to the current content or sequence. When the value of `asve_textureSkip_enable_flag` is 1 (indicating the application of the texture map skipping technique), a process for reconstructing the skipped texture maps is required. For this purpose, the texture map decoder 15021 can receive parameters related to texture map skipping. Additionally, when `asve_texture_present_flag` is parsed and it equals 1, it can be determined that some texture maps are skipped within the received texture map bitstream on a per-sequence basis, and a process for reconstructing the skipped texture maps can be performed. In this disclosure, the parameters required for reconstruction can be parsed using the following method to reconstruct the skipped texture maps on a per-frame basis.

[0482] For a common approach, when `asve_textureSkip_enable_flag` in ASPS can be parsed and equals 1, it can be determined that the texture map encoding skipping technique is applied to the current content or sequence. Then, when `asve_texture_present_flag` is parsed and equals 1, it can be determined that a skipped texture map exists within the sequence. Subsequently, texture map reconstruction can be performed by parsing the relevant parameters (or related information for reconstructing the skipped texture map) used for reconstructing the skipped texture map according to each signaling method.

[0483] First, when through such Figure 35When the AFPS sends information for reconstructing skipped texture maps, it parses the `afve_texturemap_skip_flag` in the V-DMC extension of the atlas frame parameter set. Then, when the parsed `afve_texturemap_skip_flag` equals 1, it can be determined that the texture map for the current frame has been skipped. `afve_texturemap_skip_flag` equals 0 indicates that the texture map for the current frame has not been skipped. When `afve_texturemap_skip_flag` equals 1, `avfe_ref_direction_idx` can be parsed to determine the reference direction for reconstructing the skipped texture map. `avfe_ref_direction_idx` equals 0 indicates that reconstruction is performed by referencing past frames. `avfe_ref_direction_idx` equals 1 indicates that reconstruction is performed by referencing future frames. `avfe_ref_direction_idx` equals 2 indicates that reconstruction is performed by referencing frames in both directions. When afve_texturemap_skip_flag equals 0, avfe_ref_direction_idx may not exist, or it may be assigned a specific value or a value agreed upon with the encoder. Furthermore, in the case of afps_vdmc_extension, different set IDs can be assigned to signal syntax information when the configuration or value of syntax elements (also known as fields) differ.

[0484] Second, when the relevant information used to reconstruct the skipped texture map is passed through Figure 40When the atlas tile header is sent, if the tile type is not I_TILE, the ath_texture_skip_flag can be parsed from atlas_tile_header(). Then, when the parsed flag is equal to 1, it can be determined that the texture map for the current frame will be skipped. ath_texture_skip_flag equal to 0 indicates that the texture map for the current frame has not been skipped. When ath_texturemap_skip_flag equals 1, ath_ref_direction_idx can be parsed to determine the reference direction used to reconstruct the skipped texture map. ath_ref_direction_idx equal to 0 indicates that reconstruction is performed by referencing past frames. ath_ref_direction_idx equal to 1 indicates that reconstruction is performed by referencing future frames. ath_ref_direction_idx equal to 2 indicates that reconstruction is performed by referencing frames in both directions. When ath_texturemap_skip_flag equals 0, ath_ref_direction_idx may not exist, or a specific value or a value agreed upon with the encoder may be assigned to ath_ref_direction_idx.

[0485] Third, when relevant information for reconstructing skipped texture maps is sent via the patch data unit, this information can be based on, for example... Figure 41 The parsing changes depending on the atlas and patch types shown in `patch_information_data()`. When the patch type is P and the patch mode is P_INTRA, or when the atlas type is I and the patch mode is I_INTRA, the parsing... Figure 42The `pdu_texture_skip_flag` in `patch_data_unit()` is used to determine the skipped texture map in the current frame. When this flag is 1, it indicates that the texture map in the current frame is skipped. A `pdu_texture_skip_flag` of 0 indicates that the texture map in the current frame is not skipped. When `pdu_texture_skip_flag` is 1, `pdu_ref_direction_idx` can be parsed to determine the reference direction used to reconstruct the skipped texture map. `pdu_ref_direction_idx` equal to 0 indicates that reconstruction is performed by referencing a past frame. `pdu_ref_direction_idx` equal to 1 indicates that reconstruction is performed by referencing a future frame. `pdu_ref_direction_idx` equal to 2 indicates that reconstruction is performed by referencing frames in both directions. When `pdu_texture_skip_flag` is 0, `pdu_ref_direction_idx` may not exist, or it may be assigned a specific value or a value agreed upon with the encoder. When the tile type is P and the patch mode is P_INTER, it can be parsed... Figure 43 The `ipdu_texture_skip_flag` in `inter_patch_data_unit()`, as shown, determines whether a texture map is skipped, and `ipdu_ref_direction_idx` can be parsed to determine the reference direction of the texture map. The meanings of the individual syntax elements are the same as those of `pdu`.

[0486] Fourth, when sending information related to reconstructing skipped texture maps via the NAL unit header, parse the newly added information such as... Figure 44The existing NAL unit parsing structure is shown in `atlas_frame_rbsp()`. When `af_texture_skip_flag` equals 1, it can be determined that the texture map of the current frame is skipped. `af_texture_skip_flag` equals 0 indicates that the texture map of the current frame is not skipped. When `af_texture_skip_flag` equals 1, `af_ref_direction_idx` can be parsed to determine the reference direction used to reconstruct the skipped texture map. `af_ref_direction_idx` equals 0 indicates that reconstruction is performed by referencing past frames. `af_ref_direction_idx` equals 1 indicates that reconstruction is performed by referencing future frames. `af_ref_direction_idx` equals 2 indicates that reconstruction is performed by referencing frames in both directions. When `af_texture_skip_flag` equals 0, `af_ref_direction_idx` may not exist, or a specific value or a value agreed upon with the encoder may be assigned to `af_ref_direction_idx`.

[0487] As described above, the texture map deducer 16014 of the texture map decoder 15021 can use the methods described above (i.e., based on texture_skip_flag and ref_direction_idx) to identify frames that skip texture maps, obtain information about reference frames, and reconstruct the skipped texture map by copying the skipped texture map from the reference frame. In the case of bidirectional frame reference, the reconstruction can be performed using the average of the texture maps of past and future frames.

[0488] When the texture map is reconstructed through the above process, the final reconstructed mesh is generated together with the previously reconstructed final geometry information.

[0489] As described above, in this disclosure, when the similarity between a reference texture map and the current texture map is high, the encoding of the current texture map is skipped. Furthermore, in order to allow the texture map decoder of a receiving device that receives the texture map bitstream of the skipped texture map (i.e., the texture map whose encoding and transmission are skipped) to reconstruct the skipped texture map, the transmitting device notifies the receiving device by signaling and sends information related to the skipped texture map. For example, to achieve efficient syntax transmission and operation of the V-DMC codec based on the existing V-DMC bitstream structure, a flag indicating whether the texture map skipping technique is enabled and a flag indicating the presence of skipped texture maps within the sequence are notified by signaling and sent via ASPS. Additionally, based on the atlas stream, frame-level texture map skipping flags and reference information for deriving the skipped texture map are notified by signaling at the corresponding syntax transmission locations (including the atlas frame parameter set, tile header, patch data unit, and NAL unit header). Therefore, when the texture map skipping technique is applied to the V-DMC codec and a signaling method for information related to the skipped texture map is used, the required bits can be significantly reduced, and a faster transmission speed can be achieved.

[0490] Figure 46 This is a flowchart illustrating an example of a transmission method according to an embodiment. The transmission method according to the embodiment may include: encoding grid data (21011); and transmitting a bit stream containing the encoded grid data (21012).

[0491] According to the implementation, the encoding of the mesh data (21011) includes encoding that skips the current texture map when the similarity between the reference texture map and the current texture map is high. At this point, details of the method for determining the similarity between the reference texture map and the current texture map to skip the texture map, and corresponding signaling references to information related to the skipped texture map, are provided. Figures 15 to 19 and Figures 26 to 45 The description of that will not be repeated below to avoid redundancy.

[0492] Figure 47 This is a flowchart illustrating an example of a receiving method according to an embodiment. The receiving method according to the embodiment may include: receiving a bit stream containing grid data (22011); and decoding the grid data contained in the bit stream (22012).

[0493] In this disclosure, decoding the mesh data contained in the bitstream (22012) includes reconstructing the texture map by decoding the demultiplexed texture map bitstream based on signaling information. In this case, some texture maps are skipped within the received texture map bitstream, and the decoding of the mesh data (22012) includes reconstructing the skipped texture maps based on signaling information. For details regarding the method of reconstructing the skipped texture maps and the signaling information associated with the skipped texture maps, please refer to [reference needed].Figures 20 to 45 The description of that will not be repeated below to avoid redundancy.

[0494] The aforementioned parts, modules, or units can be software, processors, or hardware components that execute sequential processes stored in memory (or storage units). The steps described in the above embodiments can be executed by a processor, software, or hardware component. The modules / blocks / units described in the above embodiments can operate as processors, software, or hardware. Furthermore, the methods presented in the embodiments can be executed as code. This code can be written to a processor-readable storage medium and thus read by a processor provided by the device.

[0495] In this specification, when a part "comprises" or "includes" an element, unless otherwise stated, it means that the part also includes or contains another element. Furthermore, the term "...module (or unit)" disclosed in this specification means a unit for performing at least one function or operation, and may be implemented by hardware, software, or a combination of hardware and software.

[0496] Although embodiments have been described with reference to the accompanying drawings for simplicity, new embodiments can be designed by incorporating the embodiments shown in the drawings. If a person skilled in the art designs a computer-readable recording medium containing a program for performing the embodiments mentioned above, it will fall within the scope of the appended claims and their equivalents.

[0497] The apparatus and methods are not limited to the configurations and methods of the above embodiments. The above embodiments can be configured by selectively combining them completely or partially to enable various modifications.

[0498] Although preferred embodiments of the embodiments have been shown and described, the embodiments are not limited to the specific embodiments described above. Various modifications may be made by those skilled in the art without departing from the spirit of the embodiments claimed in the claims, and these modifications should not be understood in isolation from the technical concept or vision of the embodiments.

[0499] Various elements of the device according to the embodiments can be implemented by hardware, software, firmware, or a combination thereof. Various elements in the embodiments can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiments, the components according to the embodiments can be implemented as separate chips. According to the embodiments, at least one or more components of the device according to the embodiments can include one or more processors capable of executing one or more programs. The one or more programs can execute any one or more of the operations / methods according to the embodiments, or include instructions for performing them. Executable instructions for performing the methods / operations of the device according to the embodiments can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transient CRM or other computer program product configured to be executed by one or more processors. Additionally, the memory according to the embodiments can be used as a concept encompassing not only volatile memory (e.g., RAM) but also non-volatile memory, flash memory, and PROM. Furthermore, it can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). Additionally, the processor-readable recording medium can be distributed across computer systems connected via a network, such that processor-readable code can be stored and executed in a distributed manner.

[0500] In this document, the terms “ / ” and “、” should be interpreted as indicating “and / or”. For example, the expression “A / B” can mean “A and / or B”. Additionally, “A, B” can mean “A and / or B”. Furthermore, “A / B / C” can mean “at least one of A, B, and / or C”. “A / B / C” can mean “at least one of A, B, and / or C”. Additionally, in this document, the term “or” should be interpreted as indicating “and / or”. For example, the expression “A or B” can include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term “or” in this document should be interpreted as indicating “additionally or alternatively”.

[0501] The various elements of the embodiments can be implemented by hardware, software, firmware, or a combination thereof. The various elements of the embodiments can be executed by a single chip (e.g., a single hardware circuit). According to the embodiments, the elements can be selectively executed by separate chips. According to the embodiments, at least one element of the embodiments can be executed in one or more processors including instructions for performing operations according to the embodiments.

[0502] Operations according to the embodiments described herein can be performed by a transmitting / receiving device comprising one or more memories and / or one or more processors according to the embodiments. One or more memories may store programs for processing / controlling operations according to the embodiments, and one or more processors may control the various operations described herein. One or more processors may be referred to as controllers, etc. In the embodiments, operations may be performed by firmware, software, and / or combinations thereof. Firmware, software, and / or combinations thereof may be stored in a processor or memory.

[0503] Terms such as first and second may be used to describe various elements of the embodiments. However, the various components according to the embodiments should not be limited by the terms above. These terms are used only to distinguish one element from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should be interpreted as not departing from the scope of the various embodiments. Both the first user input signal and the second user input signal are user input signals, but do not refer to the same user input signal unless the context clearly specifies otherwise. The terms used to describe embodiments are used only for describing particular embodiments and are not intended to limit the embodiments. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” include plural indicators unless the context clearly specifies otherwise. The expression “and / or” is used to include all possible combinations of items. Terms such as “comprising” or “having” are intended to indicate the presence of figures, numbers, steps, elements, and / or components and should be understood not to exclude the possibility of the additional presence of figures, numbers, steps, elements, and / or components.

[0504] As used herein, conditional expressions such as “if” and “when” are not limited to optional cases and are intended to be interpreted as performing a related operation or interpreting a related definition when a specific condition is met. Implementations may include variations / modifications within the scope of the claims and their equivalents. It will be apparent to those skilled in the art that various modifications and variations may be made to this disclosure without departing from the spirit and scope thereof. Therefore, this disclosure is intended to cover modifications and variations thereof, provided they fall within the scope of the appended claims and their equivalents.

[0505] This publicly disclosed model

[0506] As described above, the relevant content has been described in the best mode of implementation.

[0507] Industrial applicability

[0508] As described above, the embodiments can be applied in whole or in part to 3D data transmitting / receiving apparatuses and systems. It will be apparent to those skilled in the art that various changes or modifications can be made to the embodiments within their scope. Therefore, the embodiments are intended to cover modifications and variations, provided they fall within the scope of the appended claims and their equivalents.

Claims

1. A method for decoding grid data, the method comprising: Receives basic grid bitstream, displacement vector bitstream, texture map bitstream, and signaling information; Basic grid processing operations for reconstructing the basic grid from the basic grid bitstream; Displacement information processing operation to reconstruct displacement information from the displacement vector bitstream; The reconstruction operation based on the base mesh and the displacement information; and Texture map processing operations that reconstruct the texture map from the texture map bitstream.

2. The method according to claim 1, wherein, The texture map processing operations include: The texture map bitstream is decoded and the texture map is reconstructed based on the signaling information; Based on the signaling information, check whether at least one texture map is skipped from the texture map bitstream; and Based on the detection of at least one skipped texture map, the at least one skipped texture map is generated based on the signaling information and at least one reference frame.

3. The method according to claim 2, wherein, The skip unit of the at least one texture map is at least one of a frame, patch, submesh, tile, or slice.

4. The method according to claim 2, wherein, The signaling information includes: Information used to identify whether the at least one texture map is skipped; and Reference texture map information associated with the at least one reference frame.

5. An apparatus for decoding grid data, the apparatus comprising: A receiver configured to receive a base grid bitstream, a displacement vector bitstream, a texture map bitstream, and signaling information; A base grid processor configured to reconstruct a base grid from the base grid bitstream; A displacement information processor configured to reconstruct displacement information from the displacement vector bitstream; A reconstructor configured to reconstruct the mesh based on the base mesh and the displacement information; as well as A texture graph processor configured to reconstruct a texture graph from the texture graph bitstream.

6. The device according to claim 5, wherein, The texture map processor is configured to: The texture map bitstream is decoded and the texture map is reconstructed based on the signaling information; Based on the signaling information, check whether at least one texture map is skipped from the texture map bitstream; as well as Based on the detection of at least one skipped texture map, the at least one skipped texture map is generated based on the signaling information and at least one reference frame.

7. The device according to claim 6, wherein, The skip unit of the at least one texture map is at least one of a frame, patch, submesh, tile, or slice.

8. The device according to claim 6, wherein, The signaling information includes: Information used to identify whether the at least one texture map is skipped; and Reference texture map information associated with the at least one reference frame.

9. A method for encoding grid data, the method comprising: Encode the original mesh; as well as Send a bit stream containing encoded grid and signaling information.

10. The method according to claim 9, wherein, The encoding includes: The basic grid processing operation generates the basic grid bitstream by encoding the basic grid generated from the simplified original grid; Displacement information processing operation that generates a displacement vector bit stream by encoding displacement information generated based on the basic grid; Mesh reconstruction operations based on encoded base mesh and encoded displacement information; and A texture map processing operation that determines whether at least one texture map in the texture maps generated based on the original mesh and the reconstructed mesh is skipped and generates a texture map bitstream by encoding the texture maps that are not skipped.

11. The method according to claim 10, wherein, The texture map processing operations include: Before encoding the generated texture map, the similarity between the current texture map and the reference texture map is compared, and it is determined whether the current texture map is skipped.

12. The method according to claim 11, wherein, The texture map processing operation also includes: If the difference between the peak signal-to-noise ratio (PSNR) calculated based on the current texture map and the PSNR calculated based on the reference texture map is less than a preset threshold, the encoding and transmission of the current texture map are skipped.

13. The method according to claim 10, wherein, The skip unit of the at least one texture map is at least one of a frame, patch, submesh, tile, or slice.

14. The method according to claim 13, wherein, The signaling information includes: Information used to identify whether the at least one texture map is skipped; and Reference texture map information associated with at least one reference frame used to generate the skipped texture map.

15. The method according to claim 14, wherein, The reference texture map information includes information for identifying the orientation of the at least one reference frame.

Images