3D data decoding device and 3D data encoding device

Abstract

To encode and decode 3D data with high quality by improving the coding efficiency of the base mesh in encoding and decoding of the 3D data.SOLUTION: The 3D data decoding device for decoding mesh data or point cloud data includes: an atlas information decoding unit that decodes atlas information from encoded data in which the mesh data or the point cloud data are encoded; a base mesh decoding unit that decodes a base mesh from the encoded data and the decoded atlas information; a mesh displacement decoding unit that decodes mesh displacement from the encoded data and the decoded atlas information; and a mesh reconstruction unit that decodes a mesh from the decoded base mesh and the mesh displacement. In the mesh displacement decoding part, the atlas information that is decoded by the atlas information decoding unit is used to decode the mesh displacement from the encoded data.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a 3D data encoding device and a 3D data decoding device. [Background technology]

[0002] To efficiently transmit or record 3D data, the 3D data is converted into 2D images and video There are 3D data encoding devices that encode data using an image encoding method to generate encoded data, and 3D data decoding devices that decode the encoded data into a 2D image and reconstruct 3D data.

[0003] Specific examples of 3D data encoding methods include MPEG-I's ISO / IEC 23090-5 V3C (Volumetric Video-based Coding) and V-PCC (Video-based Point Cloud Compression). V3C encodes and decodes point clouds that consist of point positions and attribute information. Furthermore, ISO / IEC 23090-12 (MPEG Immersive Video, MIV) and ISO / IEC 23090-29 (Video-based Dynamic Mesh Coding, V-DMC), which are currently being standardized, are enabling the encoding of multi-viewpoint video, It is also used for encoding and decoding mesh images. A soft document has been disclosed.

[0004] In these 3D data coding methods, the geometry and attributes that make up the 3D data are encoded and decoded as images using video coding methods such as H.265 / HEVC (High Efficiency Video Coding) and H.266 / VVC (Versatile Video Coding).

[0005] In the case of a point cloud, the geometry image is the depth to the projection plane, and the attribute image is the image of the attributes projected onto the projection plane.

[0006] 3D data (mesh) as in Non-Patent Document 1 consists of a base mesh, mesh displacement, and texture mapping image. The encoding of the base mesh is done using vertex coding such as Draco. The mesh displacement is encoded as a two-dimensional mesh. In addition to encoding the displacement image using a video codec, there is also a method of encoding it directly using arithmetic coding. The texture mapping image is encoded as an attribute image using a video codec. The video codec can be the above-mentioned HEVC or VVC. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] WD 6.0 of V-DMC (MDS23617_WG07_N00822_clean), ISO / IEC JTC 1 / SC 29 / WG 7 N0822, January 2024 Summary of the Invention [Problem to be solved by the invention]

[0008] In the 3D data encoding method described in Non-Patent Document 1, the mesh displacements that make up the 3D data (mesh) can be encoded and decoded on a frame-by-frame basis using a video encoding method, but there is a problem in that the mesh displacements cannot be encoded and decoded for each sub-mesh, which is the reconstruction unit of the 3D data (mesh).

[0009] The present invention aims to achieve highly efficient encoding and decoding of 3D data by encoding and decoding mesh displacement for each sub-mesh when encoding and decoding 3D data using a video encoding method. [Means for solving the problem]

[0010] In order to solve the above problem, a 3D data decoding device according to one embodiment of the present invention is a 3D data decoding device that decodes mesh data or point cloud data, and includes an atlas information decoding unit that decodes atlas information from encoded data in which the mesh data or point cloud data is encoded, a base mesh decoding unit that decodes a base mesh from the encoded data and the decoded atlas information, a mesh displacement decoding unit that decodes mesh displacement from the encoded data and the decoded atlas information, and a mesh reconstruction unit that decodes a mesh from the decoded base mesh and mesh displacement, and is characterized in that the mesh displacement decoding unit decodes the mesh displacement from the encoded data using the atlas information decoded by the atlas information decoding unit.

[0011] the mesh displacement decoding unit includes a video decoding unit that decodes a mesh displacement image decoded from a geometry video stream whose Unit Type of the encoded data is V3C_GVD, and a displacement undapping unit that derives mesh displacements dispQuantCoeffArray for each vertex v, component d, and submeshIdx from a mesh displacement image dispQuantCoeffFrame having positions x, y and components d and submeshIdx; The displacement unmapping unit is characterized in that it derives the mesh displacement by deriving the coordinates of the geometry image from the height of each component of the mesh displacement or the width of each component of the mesh displacement, a variable d with a value −1 indicating the number of dimensions of the geometry (DisplacementDim), and a variable submeshIdx with a value −1 indicating the number of submesh divisions (NumSubMeshes) of the mesh displacement.

[0012] In order to solve the above problem, a 3D data encoding device according to one embodiment of the present invention is a 3D data encoding device that encodes mesh data or point cloud data, and is equipped with an atlas information encoding unit that encodes atlas information, a base mesh encoding unit that encodes a base mesh using the atlas information, and a mesh displacement encoding unit that encodes mesh displacement using the atlas information, and is characterized in that the mesh displacement encoding unit encodes the mesh displacement using the atlas information encoded by the atlas information encoding unit.

[0013] The mesh displacement encoding unit is composed of a displacement mapping unit that derives a mesh displacement image dispQuantCoeffFrame with positions x, y and components d and submeshIdx from a mesh displacement dispQuantCoeffArray with vertex v, components d and submeshIdx, and a geographic unit whose Unit Type is V3C_GVD. The image encoding unit encodes the mesh displacement having a vertex v, a component d, and a submeshIdx into a geometry video stream, and the displacement mapping unit maps the mesh displacement having a vertex v, a component d, and a submeshIdx to coordinates x, y of the geometry image using the height of each component of the mesh displacement or the width of each component of the mesh displacement. [Effects of the Invention]

[0014] According to one aspect of the present invention, the encoding efficiency of mesh displacement can be improved, and 3D data can be encoded and decoded with high quality. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a schematic diagram showing the configuration of a 3D data transmission system according to the present embodiment. [Figure 2] FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. [Figure 3] FIG. 2 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31. [Figure 4]FIG. 2 is a functional block diagram showing the configuration of an atlas information decoding unit 302. [Figure 5] FIG. 2 is a functional block diagram showing the configuration of a base mesh decoding unit 303. [Figure 6] FIG. 10 is a functional block diagram showing the configuration of a mesh displacement decoding unit 305. [Figure 7] FIG. 2 is a functional block diagram showing the configuration of a mesh reconstruction unit 307. [Figure 8] 10 is an example of a syntax for transmitting coordinate transformation parameters of mesh data at the sequence level (ASPS). [Figure 9] 10 is an example of a syntax for transmitting coordinate transformation parameters at the picture / frame level (AFPS). [Figure 10] This is an example of the syntax for transmitting frame tile information at the picture / frame level (AFPS). [Figure 11] 10 is an example of a syntax for transmitting coordinate transformation parameters of mesh data at the picture / frame level (AFPS). [Figure 12] This is an example of the syntax for transmitting mesh and submesh information at the picture / frame level (AFPS). [Figure 13] 10 is an example of a syntax for transmitting patch information at the picture / frame level (AFPS). [Figure 14] FIG. 10 is a diagram for explaining the operation of the mesh reconstruction unit 307. [Figure 15] 1 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11. FIG. [Figure 16] FIG. 2 is a functional block diagram showing the configuration of an atlas information encoding unit 101. [Figure 17] FIG. 2 is a functional block diagram showing the configuration of a base mesh encoding unit 103. [Figure 18] FIG. 2 is a functional block diagram showing the configuration of a mesh displacement encoding unit 107. [Figure 19] FIG. 2 is a functional block diagram showing the configuration of a mesh separation unit 115. [Figure 20] 10 is a diagram for explaining the operation of the mesh separating unit 115. FIG. [Figure 21] This is an example of the syntax for transmitting mesh and submesh information at the picture / frame level (AFPS). [Figure 22] 10 is an example of a syntax for transmitting coordinate transformation parameters of mesh data at the picture / frame level (AFPS). [Figure 23] 10 is an example of a mesh displacement image divided according to the number of dimensions of the geometry and the number of sub-mesh divisions. [Figure 24] 10 is an example of a mesh displacement image divided according to the number of dimensions of the geometry and the number of sub-mesh divisions. [Figure 25] 10 is an example of a mesh displacement image divided according to the number of dimensions of the geometry and the number of sub-mesh divisions. [Figure 26] 10 is an example of a mesh displacement image divided according to the number of dimensions of the geometry and the number of sub-mesh divisions. [Figure 27] 10 is an example of a mesh displacement image divided according to the number of dimensions of the geometry and the number of sub-mesh divisions. [Figure 28] 10 is an example of a mesh displacement image divided according to the number of dimensions of the geometry and the number of sub-mesh divisions. [Figure 29] 10 is an example of a syntax for transmitting coordinate transformation parameters at the sequence level (ASPS). DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0017] FIG. 1 is a schematic diagram showing the configuration of a 3D data transmission system 1 according to this embodiment.

[0018] The 3D data transmission system 1 is a system that transmits an encoded stream obtained by encoding 3D data to be encoded, decodes the transmitted encoded stream, and displays the 3D data. The 3D data transmission system 1 includes a 3D data encoding device 11, a network 21, a 3D data decoding device 31, and a 3D data display device 41.

[0019] The 3D data T is input to the 3D data encoding device 11.

[0020] The network 21 transmits the coded stream Te generated by the 3D data coding device 11 to the 3D data decoding device 31. The network 21 may be the Internet, a wide area network (WAN), a local area network (LAN), or The network 21 is not necessarily limited to a two-way communication network, but may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may also be a network that transmits DVD (Digital Versatile Disc: registered trademark), BD (Blu-ray Disc: registered trademark), etc. It may be replaced by a storage medium on which the coded stream Te, such as a standard (or a DVD-ROM drive), is recorded.

[0021] The 3D data decoding device 31 decodes each of the coded streams Te transmitted by the network 21, and generates one or more decoded 3D data Td.

[0022] The 3D data display device 41 displays all or part of one or more pieces of decoded 3D data Td generated by the 3D data decoding device 31. The 3D data display device 41 is equipped with a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. The display may be in the form of a stationary display, a mobile display, an HMD, or the like. When the device has a high processing power, it displays a high quality image, and when the device has a lower processing power, it displays an image that does not require a high processing power or display power.

[0023] <operator> The operators used in this specification are listed below.

[0024] >> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR , |= is the OR assignment operator, and || indicates logical sum.

[0025] x?y:z is a ternary operator that takes y if x is true (non-zero) and z if x is false (zero). y..z denotes the set of integers from y to z.

[0026] <Structure of the coded stream Te> Before proceeding to a detailed description of the 3D data encoding device 11 and the 3D data decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the 3D data encoding device 11 and decoded by the 3D data decoding device 31 will be described.

[0027] FIG. 2 shows the hierarchical structure of data in the coded stream Te. The frame Te has either a V3C sample stream or a V3C unit stream data structure. A V3C sample stream includes a sample stream header and a V3C unit. A V3C unit stream includes a V3C unit.

[0028] A V3C unit includes a V3C unit header and a V3C unit payload. The V3C unit header is a Unit Type, which is an ID that indicates the type of V3C unit, and can be V3C_VPS, V3C_AD, V3C_AVD, V3C_GVD, V3C_OVD, etc. It takes the value indicated by the label.

[0029] If the Unit Type is V3C_VPS (Video Parameter Set), the V3C unit contains a V3C parameter set.

[0030] If the Unit Type is V3C_AD (Atlas Data), the V3C unit includes a VPS ID, an atlasID, a sample stream nal header, and multiple NAL units. The atlasID is an ID (Identification). Yes, takes an integer value of 0 or greater.

[0031] A NAL unit includes a NALUnitType, a layerID, a TemporalID, and an RBSP (Raw byte sequence payload).

[0032] NAL units are identified by NALUnitType, and are classified as ASPS (Atlas Sequence Parameter Set), AAPS (Atlas Adaptation Parameter Set), ATL (Atlas Tile layer), SEI (Supplemental Enh Including the Service Information.

[0033] The ATL includes an ATL header and an ATL data unit, and the ATL data unit includes information such as the patch position and size, such as patch information data.

[0034] The SEI includes payloadType indicating the type of SEI, payloadSize indicating the size (number of bytes) of the SEI, and sei_payload of the SEI data.

[0035] If the Unit Type is V3C_AVD (Attribute Video Data), The unit includes the VPS ID, atlasID, attribute image ID attrIdx, partition ID partIdx, map ID mapIdx, flag auxFlag indicating whether it is auxiliary data or not, and video stream. The video stream is data encoded using HEVC, VVC, etc. Attribute data corresponds to a texture image in V-DMC.

[0036] If NalUnitType is V3C_GVD (Geometry Video Data), the V3C unit contains VPS ID, atlasID, mapIdx, auxFlag, and video stream. Geometry data corresponds to mesh displacement in V-DMC.

[0037] If the Unit Type is V3C_OVD (Occupancy Video Data), the V3C unit includes a VPS ID, an atlas ID, and a video stream.

[0038] If the Unit Type is V3C_MD (Mesh data), the V3C unit contains the VPS ID, atlas ID, and mesh_payload. In V-DMC, it corresponds to the base mesh.

[0039] (Configuration of 3D data decoding device according to the first embodiment) 3 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31 according to the first embodiment. The 3D data decoding device 31 includes a demultiplexing unit 301, a submesh information decoding unit 3024, an atlas information decoding unit 3026, a demultiplexing unit 3028, a demultiplexing unit 3029, a demultiplexing unit 3030, a demultiplexing unit 3031, a demultiplexing unit 3032, a demultiplexing unit 3033, a demultiplexing unit 3034, a demultiplexing unit 3035, a demultiplexing unit 3036, a demultiplexing unit 3037, a demultiplexing unit 3038, a demultiplexing unit 3039 ... The 3D data decoding device 31 comprises an information decoding unit 302, a base mesh decoding unit 303, a mesh displacement decoding unit 305, a mesh reconstruction unit 307, an attribute decoding unit 306, and a color space conversion unit 308. The 3D data decoding device 31 inputs coded 3D data and outputs atlas information, meshes, and attribute images.

[0040] The demultiplexing unit 301 receives encoded data multiplexed in a byte stream format, ISOBMFF (ISO Base Media File Format), or the like, and demultiplexes it to generate an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a base mesh encoded stream (V3C_MD mesh_payload), a mesh displacement encoded stream (V3C_GVD video stream), an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a mesh displacement encoded stream (V3C_MD video stream), an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a mesh displacement encoded stream (V3C_MD video stream), an atlas information encoded stream (V3C_MD ... Outputs a live video stream (V3C_AVD video stream).

[0041] The atlas information decoding unit 302 receives the atlas information coded stream output from the demultiplexing unit 301 and decodes the atlas information.

[0042] 3 decodes coordinate system transformation information displacementCoordinateSystem (mdu_displacement_coordinate_system) indicating the coordinate system from the encoded data. Note that a separate gating flag may be provided, and each piece of coordinate system transformation information may be decoded only when the gating flag is 1. The gating flag may be, for example, mdu_displacement_coordinate_system_enable_flag.

[0043] The base mesh decoding unit 303 decodes the base mesh coded stream coded by vertex coding (3D data compression coding method, for example, Draco), and outputs a base mesh. The base mesh will be described later.

[0044] The mesh displacement decoding unit 305 decodes a geometry video stream (mesh displacement coded stream) coded using VVC, HEVC, or the like, and outputs mesh displacement. The type of codec (video codec) used for coding is determined by decoding the V3C parameter set of the coded data. It can also be indicated by the Four CC code (four-character code, 4CC code) indicated by gi_geometry_codec_id[atlasID] in the V3C parameter set. gi_geometry_codec_id[atlasID] indicates the decoding of the geometry video stream in the atlas ID. The set indicating the correspondence between the codec ID (ccm_codec_id) and its 4CC code (ccm_codec_4cc[ccm_codec_id]) may be transmitted in a separate codec mapping SEI (component_codec_mapping SEI). The codec may decode mesh displacement in segments (slices) that are further divided into frames. HEVC and VVC can divide frames into slices. Rice is coded in units of Coded Tree Units (CTUs), not slices. Subpictures or tile divisions may be used as segments. These subpictures, tiles, and slices can be decoded independently, making it possible to decode only a portion of a frame without decoding the entire frame. When subpictures or tiles are used, slices are replaced with subpictures or tiles.

[0045] The mesh reconstruction unit 307 receives the base mesh and the mesh displacement and reconstructs the mesh in 3D space. Reconstruct the cache.

[0046] The attribute decoding unit 306 decodes an attribute video stream coded by VVC, HEVC, or the like, and outputs an attribute image. The attribute image is a texture image expanded on the UV axis (a texture mapping image converted by the UV atlas method) in YCbCr frame. The type of codec used for encoding is indicated by ptl_profile_codec_group_idc, which is obtained by decoding the V3C parameter set of the encoded data. It may also be indicated by the Four CC code indicated by ai_geometry_codec_id[atlasID] in the V3C parameter set. ai_geometry_codec_id[atlasID] indicates the index corresponding to the codec ID of the decoder used to decode the attribute video stream in the atlas ID.

[0047] A color space conversion unit 308 converts the attribute image from the YCbCr format to the RGB format. Color space conversion is performed. Note that the attribute video stream is encoded as RGB format. It is also possible to decode the image data and omit the color space conversion.

[0048] (Decoding atlas information) 4 is a functional block diagram showing the configuration of the atlas information decoding unit 302. The atlas information decoding unit 302 includes a parameter decoding unit 3021, a tile information decoding unit 3022, an extended information decoding unit 3023, a submemory decoding unit 3024, and a tile information decoding unit 3025. The image processing unit 3020 is configured of a patch information decoding unit 3024 and a patch information decoding unit 3025.

[0049] (Decoding and derivation of control parameters) The parameter decoding unit 3021 decodes control parameters from the atlas information coded stream. The control parameters include ASPS (Atlas Sequence Parameter Set), which is a sequence level parameter set, and AFPS (Atlas Sequence Parameter Set), which is a picture / frame level parameter set. Frame Parameter Set).

[0050] Figure 29 shows the sequence-level parameter set, ASPS (Atlas Sequence Parameter Set). This is an example of the syntax for the Atlas Information Set. ASPS is one of the NAL units of atlas information and contains syntax elements that apply to the atlas information coded stream. The semantics of each field are as follows:

[0051] asps_geometry_3d_bit_depth_minus1: Add 1 to asps_geometry_3d_bit_depth_minus1 The returned value indicates the bit depth of the geometric coordinates of the reconstructed volume content.

[0052] asps_geometry_2d_bit_depth_minus1: Add 1 to asps_geometry_2d_bit_depth_minus1 The value indicates the bit depth of the geometry when projected onto a 2D image.

[0053] Figure 8 shows an example of the syntax of ASVE (ASPS Vdmc Extension), which is a sequence-level mesh data extension control parameter set. The semantics of each field are as follows: That's right.

[0054] asve_subdivision_iteration_count: Indicates the number of mesh division iterations.

[0055] asve_1d_displacement_flag: A flag indicating whether the mesh displacement is one-dimensional. If the value is true, it indicates that the mesh displacement is one-dimensional. If the value is false, it indicates that the mesh displacement is three-dimensional.

[0056] Figure 9 shows an example of the syntax of AFPS (Atlas Frame Parameter Set), a picture / frame-level parameter set. AFPS is one of the NAL units of atlas information and contains syntax elements that apply to the atlas information coded stream. The semantics of each field are as follows: AFPS contains atlas_frame_tile_information() and atlas_frame_mesh_information().

[0057] afps_atlas_frame_parameter_set_id: The atlas referenced by other syntax elements. Identifies the frame parameter set AFPS.

[0058] afps_atlas_sequence_parameter_set_id: Specifies the value of asps_atlas_sequence_parameter_set_id of the active atlas sequence parameter set ASPS.

[0059] (Decoding and deriving tile-level control parameters) The tile-level control parameters decoded from the coded data by the tile information decoding unit 3022 will be described.

[0060] Figure 10 shows an example of the syntax of tile information in AFPS, a picture / frame-level parameter set. The semantics of each field are as follows:

[0061] afti_single_tile_in_atlas_frame_flag: Flag indicating whether there is exactly one tile in each atlas frame that references the atlas frame parameter set AFPS. A value of true indicates that there is exactly one tile in each atlas frame that references AFPS. Value If false, each atlas frame that references AFPS has multiple tiles (greater than 1). Indicates existence.

[0062] afti_single_partition_per_tile_flag: See Atlas Frame Parameter Set AFPS A flag indicating whether each tile referencing the AFPS contains only one tile partition. A value of true indicates that each tile referencing the AFPS contains only one tile partition; a value of false indicates that each tile referencing the AFPS contains multiple tile partitions ( If not present, the value of afti_single_partition_per_tile_flag is inferred to be equal to 1.

[0063] afti_num_tiles_in_atlas_frame_minus1: Specifies the number of tiles in each atlas frame that references the atlas frame parameter set AFPS. The value of s1 must be in the range of 0 to NumPartitionsInAtlasFrame-1. If not present and afti_single_partition_per_tile_flag is equal to 1, the value of afti_num_tiles_in_atlas_frame_minus1 is inferred to be equal to NumPartitionsInAtlasFrame-1.

[0064] afti_signalled_tile_id_flag: Indicates whether the tile ID of each tile is signaled. If the flag is set to 1, the tile ID of each tile is signaled. If the flag is set to 0, the tile ID is not signaled.

[0065] afti_signalled_tile_id_length_minus1:afti_signalled_tile_id_length_minus1+1 is the length of the syntax element afti_tile_id[i] (if present) and the syntax Specifies the number of bits used to represent the afti_signalled_tile_id element ath_id. The value of afti_signalled_tile_id_length_minus1 must be in the range 0 to 15.

[0066] afti_tile_id[i]: Specifies the tile ID of the ith tile. If not present, the value of afti_tile_id[i] is inferred to be equal to i for each i in the range 0 to afti_num_tiles_in_atlas_frame_minus1. afti_tile_id[i] is inferred to be equal to afti_tile_id[j] for all i != j. The inequality is a requirement for bitstream conformance.

[0067] When decoding and encoding afti_single_tile_in_atlas_frame_flag and afti_single_partition_per_tile_flag, the tile information decoding unit 3022 decodes a syntax element afti_num_tiles_in_atlas_frame_minus2 indicating the number of tiles minus 2 (the value obtained by subtracting 2 from the number of tiles). Alternatively, if the value of afti_single_tile_in_atlas_frame_flag is false or Only if the value of afti_single_partition_per_tile_flag is false, decode and encode the syntax element afti_num_tiles_in_atlas_frame_minus2, which indicates the number of tiles referenced minus 2. Semantics may use the following examples:

[0068] afti_num_tiles_in_atlas_frame_minus2: Specifies the number of tiles in each atlas frame that references the atlas frame parameter set AFPS. The value of afti_num_tiles_in_atlas_frame_minus1 must be in the range from 0 to NumPartitionsInAtlasFrame-2. If not present and afti_single_partition_per_tile_flag is equal to 1, the value of afti_num_tiles_in_atlas_frame_minus2 is inferred to be equal to NumPartitionsInAtlasFrame-2.

[0069] In this configuration, the case where one tile is referenced is indicated by afti_single_tile_in_atlas_frame_flag Therefore, the syntax element indicating the number of tiles minus 2 is decoded and coded. By encoding, the overhead of the amount of coding can be reduced.

[0070] (Decoding and deriving extended control parameters) The extended control parameters decoded from the coded data by the extended information decoding unit 3023 will be described.

[0071] FIG. 11 shows an example of the syntax of mesh information in AFPS, which is a picture / frame level parameter set.

[0072] afve_overriden_flag: A flag indicating whether or not to update the coordinate system of the mesh displacement. If this flag is true, the coordinate system of the mesh displacement is updated based on the value of mdu_displacement_coordinate_system described below. If this flag is false, the coordinate of the mesh displacement is updated. The system is not updated.

[0073] afve_subdivision_iteration_count: Indicates the number of mesh division iterations.

[0074] (Decoding and derivation of mesh-level control parameters) The mesh-level control parameters decoded from the coded data by the sub-mesh information decoding unit 3024 will be described.

[0075] Figure 12 shows an example of the syntax structure of atlas_frame_mesh_information(), which transmits submesh information using AFPS. In the example syntax structure of Figure 12, the number of submesh IDs is encoded and decoded regardless of the number of submeshes referenced. atlas_frame_mesh_information() may contain any of the following syntax elements. The semantics of each field are as follows:

[0076] afmi_use_single_mesh_flag: In each atlas frame that references AFPS, a mesh A flag indicating whether the patch references a single submesh or not (greater than 1). A value of true indicates that there is only one referenced submesh. A value of false indicates that there are multiple referenced submeshes (greater than 1).

[0077] afmi_num_submeshes_minus2: The number of submeshes referenced by a mesh patch. If the value of afmi_use_single_mesh_flag is true, the sub-mesh The number is 1. If the value of afmi_use_single_mesh_flag is false, the number of submeshes is afmi_num_submeshes_minus2+2.

[0078] afmi_signalled_submesh_id_flag: Flag indicating whether the submesh ID referenced by the mesh patch is signaled. If the value is true, it indicates that the submesh ID is signaled. If the value is false, it indicates that the submesh ID is not signaled. show.

[0079] afmi_signalled_submesh_id_length_minus1: The value obtained by adding 1 to afmi_signalled_submesh_id_length_minus1 is the value of the syntax element mdu_submesh_id[ tileID ][ patchIdx ] and the syntax element afmi_submesh_id[ i Specifies the number of bits used to represent ] in the current atlas tile whose tile ID is equal to tileID. The value of afmi_signalled_submesh_id_length_minus1 must be in the range 0 to 15. If not present, its value is inferred to be equal to Ceil( Log2( NumSubMeshes ) ) - 1.

[0080] afmi_submesh_id[i]: a parameter that indicates the submesh ID of the i-th submesh If the value of afmi_signalled_submesh_id_flag is false, i.e., afmi_submesh_id[i] is If not present, the value of afmi_submesh_id[i] is inferred to be equal to i, for each i in the range 0 to NumSubMeshes-1. It is a bitstream conformance requirement that afmi_submesh_id[i] is not equal to afmi_submesh_id[j] for all i != j. The variable FirstSubmeshID is derived as follows:

[0081] FirstSubmeshID = afmi_submesh_id[0] for ( i = 1; i < NumSubMeshes; i++ ) FirstSubmeshID = Min( FirstSubmeshID, afmi_submesh_id[ i ] ) The atlas information decoding unit 302 (submesh information decoding unit 3024) decodes the AFPS code of the atlas information. Decode frame mesh information from the encoded data. For example, afmi_use_single_mesh_flag , afmi_num_submeshes_minus2, afmi_signalled_submesh_id_flag, afmi_signalled_submesh_id_length_minus1, and afmi_submesh_id. The atlas information encoding unit 101 (submesh information encoding unit 1012) encodes frame mesh information into AFPS encoded data of the atlas information.

[0082] (Operation of submesh information decoding unit) When afmi_signalled_submesh_id_flag is true, the submesh information decoding unit 3024 decodes afmi_submesh_id[i] in the range of i = 0.. NumSubMeshes-1 for the number of submeshes NumSubMeshes, and derives the arrays SubMeshIDToIndex and SubMeshIndextoID for i = 0.. NumSubMeshes-1 as follows:

[0083] SubMeshIDToIndex[ afmi_submesh_id[ i ] ] = i SubMeshIndextoID[ i ] = afmi_submesh_id[ i ] In the syntax configuration of Figure 12, afmi_signalled_submesh_id_flag, if (afmi_signalled_submesh_id_flag), and the brackets {} following it may be configured to exist only in the case of if (!afmi_use_single_mesh_flag). Alternatively, the above sections may be configured to exist when the number of submeshes, NumSubmeshes, is greater than 1.

[0084] In this case, if afmi_use_single_mesh_flag is false (or the number of submeshes NumSubmeshes is greater than 1) and afmi_signalled_submesh_id_flag is true, the submesh information decoding unit 3024 decodes afmi_signalled_submesh_id_length_minus1 and afmi_submesh_id[ i ] in the range of i = 0..NumSubMeshes-1 for the number of submeshes minus 1 (NumSubMeshes-1). Also, if afmi_use_single_mesh_flag is true (or if the sub-mesh If the number of submeshes is 1, then SubMeshIDToIndex[0] = 0, SubMeshIndexToID[0] = 0, The ID may always be set to 0.

[0085] In addition, in the syntax configuration of Figure 12, if the value of afmi_use_single_mesh_flag is true, In this case, the value of afmi_signalled_submesh_id_flag may always be set to false, and the ID may always be set to 0, i.e., SubMeshIDToIndex[0] = 0 and SubMeshIndexToID[0] = 0.

[0086] The submesh information decoding unit 3024 does not decode afmi_submesh_id[i] when afmi_signalled_submesh_id_flag is true, and derives the array as follows for i = 0..NumSubMeshes-1: do.

[0087] SubMeshIDToIndex[ i ] = i SubMeshIndextoID[ i ] = i Figure 21 shows an example of the syntax for mesh and submesh information.

[0088] As shown in the syntax configuration of FIG. 21, the submesh information decoding unit 3024 decodes the flag afmi_in_tile_flag[afti_tile_id[i]][afmi_submesh_id[j]] indicating whether or not submesh information having a certain submesh ID is referenced in tile information having a certain tile ID, and if there is a submesh ID of a certain j referenced in tile information of a certain i, the flag afmi_in_tile_flag[afti_tile_id[i]][afmi_submesh_id[j]] indicates whether or not submesh information having a certain submesh ID is referenced in tile information of a certain i. A value indicating that afti_tile_id[ i ] ][ afmi_submesh_id[ j ] ] is referenced (e.g., 1) If there is a submesh ID j that is not referenced in the tile information for a given i, afmi_in_tile_flag[ afti_tile_id[ i ] ][ afmi_submesh_id[ j ] ] is decoded as a value indicating that it is not referenced (for example, 0). There may be multiple submesh IDs referenced in the tile information for a given i. The following examples may be used for semantics:

[0089] afmi_in_tile_flag[ afti_tile_id[ i ] ][ afmi_submesh_id[ j ] ]: Indicates whether or not submesh information with a submesh ID is referenced in tile information with a certain tile ID. If the flag is set to 1, the tile information with a certain tile ID (afti_tile_id[ i ]) The submesh information with the real submesh ID j (afmi_submesh_id[ j ]) is referenced. If the flag is set to 0, the tile with tile ID i (afti_tile_id[ i ]) is The file information contains the submesh information with the submesh ID j (afmi_submesh_id[ j ]). Indicates that the information is not referenced.

[0090] In the syntax configuration of Figure 21, the number of tiles, NumTiles, is derived by decoding afti_num_tiles_in_atlas_frame_minus2 and calculating NumTiles = afti_num_tiles_in_atlas_frame_minus2 + 2. Alternatively, afti_num_tiles_in_atlas_frame_minus1 may be decoded and NumTiles = afti_num_tiles_in_atlas_frame_minus1+1 may be derived.

[0091] In another example, as shown in the following syntax configuration, the flag afmi_in_tile_flag[afmi_submesh_id[i]][afti_tile_id[j]] indicating whether or not tile information having a certain tile ID is referenced by submesh information having a certain submesh ID is decoded, and the submesh information of a certain i is If there is a tile ID of a certain j that is referenced in the mesh information, afmi_in_tile_flag[ afti_submesh_id[ i ] ][ afmi_tile_id[ j ] ] may be decoded as a value indicating that it is referenced (for example, 1), and if there is a tile ID of a certain j that is not referenced in the submesh information of a certain i, afmi_in_tile_flag[ afmi_submesh_id[ i ] ][ afti_tile_id[ j ] ] may be decoded as a value indicating that it is not referenced (for example, 0). There may be multiple tile IDs referenced in the submesh information of a certain i.

[0092] atlas_frame_mesh_information() { ... for( i = 0; i < NumSubMeshes; i++ ) { for( j = 0; j < NumTiles; j++ ) { afmi_in_tile_flag[ afmi_submesh_id[ i ] ][ afti_tile_id[ j ] ] } } ... } Here, the following shows an example of the values ​​of afmi_in_tile_flag[ afmi_submesh_id[ i ] ][ afti_tile_id[ j ] ] when numTiles = 2, numSubMeshes = 3, afmi_signalled_submesh_id_flag = 0, afti_tile_id[0] is referenced by afmi_submehs_id[1] and afmi_submesh_id[2], and afti_tile_id[1] is referenced by afmi_submesh_id[0].

[0093] afmi_in_tile_flag[0][0] = 0 afmi_in_tile_flag[0][1] = 1 afmi_in_tile_flag[1][0] = 1 afmi_in_tile_flag[1][1] = 0 afmi_in_tile_flag[2][0] = 1 afmi_in_tile_flag[2][1] = 0 In the syntax configuration of Figure 21, the syntax element afmi_in_tile_flag may be coded and decoded using indexes such as i and j as follows, rather than using the identification numbers of afti_tile_id[i] and afmi_submesh_id[j] or afmi_submesh_id[i] and afti_tile_id[j].

[0094] atlas_frame_mesh_information() { ... for( i = 0; i < NumSubMeshes; i++ ) { for( j = 0; j < NumTiles; j++ ) { afmi_in_tile_flag[ i ][ j ] } } ... } In the syntax configuration of Figure 21, the syntax element afmi_in_tile_flag[ afti_tile_id[ i ] ][ afmi_submesh_id [ j ] ] or afmi_in_tile_flag[ afmi_submesh_id[ i ] ][ afti_tile_id[ j ] ] is not encoded and decoded as a two-dimensional array, but is instead encoded and decoded as a one-dimensional array as follows: It may be encoded and decoded as a sequence.

[0095] atlas_frame_mesh_information() { ... for( i = 0; i < NumSubMeshes; i++ ) { for( j = 0; j < NumTiles; j++ ) { index = (NumSubMeshes-1) * i + j afmi_in_tile_flag[ index ] } } ...} In this case, the submesh ID and tile ID are derived from the indexes i and j as follows:

[0096] Submesh ID = SubMeshIndextoID[ i ] Tile ID = TileIndexToID[ j ] Here, the following shows an example of the value of afmi_in_tile_flag[index] when numTiles = 2, numSubMeshes = 3, afmi_signalled_submesh_id_flag = 0, afti_tile_id[0] is referenced by afmi_submehs_id[1] and afmi_submesh_id[2], and afti_tile_id[1] is referenced by afmi_submesh_id[0].

[0097] afmi_in_tile_flag[0] = 0 afmi_in_tile_flag[1] = 1 afmi_in_tile_flag[2] = 1 afmi_in_tile_flag[3] = 0 afmi_in_tile_flag[4] = 1 afmi_in_tile_flag[5] = 0 (Decrypting patch information) Here, the patch level control parameters decoded from the coded data by the patch information decoding unit 3025 will be described.

[0098] Figure 13 shows an example of the syntax of patch information in AFPS, a picture / frame level parameter set. The semantics of each field are as follows:

[0099] mdu_submesh_id[ tileID ][ patchIdx ]: indicates the associated submesh ID specified in the current mesh patch with index patchIdx, in the current atlas tile whose tile ID is equal to tileID. The value of mdu_submesh_id[ tileID ][ patchIdx ] must be one of afmi_submesh_id[ i ].

[0100] mdu_vertex_count_minus1[ tileID ][ patchIdx ]: Current vertex whose tile ID is equal to tileID In atlas style, specifies the number of vertices associated with the current mesh patch with index patchIdx.

[0101] mdu_subdivision_iteration_count[ tileID ][ patchIdx ]: Index patchIdx mdu_subdivision_iteration_count[ tileID ][ patchId ] indicates the number of iterations used for subdivision in the current atlas tile with tile ID equal to tileID. x ] is not present or if mdu_subdivision_method[ tileID ][ patchIdx ] is equal to 0, its value is inferred to be equal to 0, otherwise its value is inferred to be equal to afve_subdivision_iteration_count.

[0102] mdu_displacement_coordinate_system[ tileID ][patchIdx ]: tile ID equals tileID Specifies the identifier of the coordinate system of the subpart of the mesh associated with the mesh patch with index patchIdx, within the new current atlas style.

[0103] (Decoding the base mesh) 5 is a functional block diagram showing the configuration of the base mesh decoding unit 303. The base mesh decoding unit 303 includes a mesh decoding unit 3031, a motion information decoding unit 3032, a mesh motion compensation unit 3033, a reference The base mesh decoding unit 303 is configured with a reference mesh memory 3034, a switch 3035, a switch 3036, and a skip decoding unit 3037. The base mesh decoding unit 303 performs a base mesh decoding (not shown) before outputting the base mesh. The switches 3035 and 3036 may be configured to include a mesh dequantization unit. If the base mesh to be decoded has been coded (intra-coded) without reference to other base meshes (for example, base meshes that have already been coded and decoded), the switches 3035 and 3036 are connected to the mesh decoding unit 3031. If the base mesh to be decoded has been coded (inter-coded) with reference to other base meshes, the switches are connected to the side that performs motion compensation. When motion compensation is performed, the target vertex coordinates are derived by referencing already decoded vertex coordinates and motion information. If the base mesh to be decoded has been skipped and another base mesh has been coded (skip coded), the switches are connected to the skip decoding unit 3037.

[0104] Each base mesh is made up of one or more sub-meshes. If a mesh exists, the tile header in the atlas data sub-bitstream needs the ID to find the sub-mesh corresponding to the tile. A mesh is a subset of a mesh defined by specifying a part of the original model, and is a mesh created by dividing a mesh into multiple parts. By dividing the mesh into subsets, specific areas of the mesh can be defined individually. Each sub-mesh has its own vertex coordinates, normal vectors, texture coordinates, etc., and can be manipulated and edited individually. The mesh of a frame is called a mesh frame.

[0105] The mesh decoding unit 3031 decodes the intra-coded base mesh coded stream and outputs the base mesh (base mesh vertex positions, base mesh vertex position vectors). As the coding method, Draco, Edge Breaker, etc. are used.

[0106] The motion information decoding unit 3032 decodes the inter-coded base mesh coded stream and outputs motion information (mesh motion information, mesh motion vectors) for each vertex of a reference mesh (described later). Entropy coding such as arithmetic coding is used as the coding method.

[0107] The mesh motion compensation unit 3033 performs motion compensation on each vertex of the reference mesh input from the reference mesh memory 3034 based on the motion information, and outputs a motion-compensated mesh.

[0108] The reference mesh memory 3034 is a memory that holds the decoded mesh for reference in subsequent decoding processes.

[0109] (Mesh displacement decoding) FIG. 6 is a functional block diagram showing the configuration of the mesh displacement decoding unit 305. The mesh displacement decoding unit 305 is composed of a displacement unmapping unit 3052 (image unpacking unit, displacement decoding unit), an inverse quantization unit 3053, an inverse transformation unit 3054, and a coordinate system transformation unit 3055. As shown in the figure, the mesh displacement decoding unit 305 further The mesh displacement decoding unit 305 may include the decoding unit 3051, or the mesh displacement decoding unit 305 may include the video decoding unit 3051. Alternatively, the mesh displacement decoding unit 305 may be configured to use the video decoding device 31 to decode the displaced image (displacement sequence). Also, the mesh displacement decoding unit 305 does not include the inverse quantization unit 3053, and image quality control is performed only by the video decoding device 31. The following configuration may also be used.

[0110] The atlas information decoding unit 302 decodes the coordinate system transformation information displacementCoordinateSystem (mdu_displacement_coordinate_system) indicating the coordinate system from the coded data. The submesh division information of the displacement (displacement submesh division parameter, displacement segment parameter, displacement submesh division flag, displacement segment flag) may be decoded. The submesh division information is the displacementSubmeshFlag that indicates whether to divide into segments. Furthermore, the sub-mesh division information may include a component height origHeight. Furthermore, the sub-mesh division information may include a component width origWidth. In addition, the sub-mesh division information is a syntax that indicates the top position of each LOD of mesh displacement. Contains the mesh element dispPos[lodIdx] or the number of mesh displacements per LOD dispCount[lodIdx] The sub-mesh division information may also include an index dispCountIdx[lodIdx] indicating the number of mesh displacements. The sub-mesh division information may also include the size of the block for sub-mesh alignment, ctuSize, or an index ctuSizeIdx indicating ctuSize. The component height is a parameter indicating the height of the image corresponding to each component (e.g., n, t, b) of the mesh displacement 3D vector.

[0111] Alternatively, a gating flag may be provided separately, and each piece of coordinate system transformation information may be decoded only when the gating flag is 1. The gating flag may be, for example, afve_overriden_flag. Also, a gating flag may be provided in the sub-mesh division information, and the sub-mesh division information may be decoded only when the gating flag is 1. The gating flag may be, for example, For example, afve_displacement_submesh_alignment_flag.

[0112] (Coordinate system) The following two types of coordinate systems are used for mesh displacement (3D vector). Cartesian coordinate system (canonical): A rectangular coordinate system commonly defined throughout the entire 3D space. (X, Y, Z) coordinate system. A rectangular coordinate system whose direction does not change at the same time (within the same frame, within the same tile). Local coordinate system: A Cartesian coordinate system defined per region or per vertex in 3D space. A Cartesian coordinate system whose direction can change at the same time (within the same frame, within the same tile). A coordinate system with normal (D), tangent (U), and bi-tangent (V) axes. In other words, a certain vertex (a certain vertex) The first axis (D) is indicated by the normal vector n_vec on the surface containing the normal vector n_vec, and the second axis (U) and third axis (V) are indicated by two tangent vectors t_vec and b_vec that are perpendicular to the normal vector n_vec. n_vec, t_vec, b_vec are three-dimensional vectors. (D, U, V) coordinates The reference system may also be called the (n, t, b) coordinate system.

[0113] (Operation of mesh displacement decoding unit) The video decoder 3051 receives a geometry video stream (V3C_GVD) encoded by VVC, HEVC, or the like. The decoded image (mesh) is generated by decoding the (quantized) mesh displacement as pixel values. The output is a mesh displacement image, a mesh displacement array, and a color component of the geometry in DecGeoChromaFormat. The image may be in YCbCr4:2:0 format. The mesh displacement image may also be a transformed mesh displacement image, or a residual of the mesh displacement image.

[0114] The displacement unmapping unit 3052 generates mesh displacement from the mesh displacement image. Specifically, it converts the mesh displacement dispQuantCoeffArray[v][d] of one-dimensional signals in component d units from the two-dimensional mesh displacement image dispQuantCoeffFrame[x][y][d] according to the correspondence relationship of the coordinate positions. Note that dispQuantCoeffFrame is derived from the geometry video stream (video The image array DecGeoFrames[mapIdx][frameIdx] or GeoFramesNF[mapIdx][compTimeIdx] may be decoded by a codec from the stream. Here, the correspondence of coordinate positions may be a Z-order scan in block units. NF is a nominal format ) and is the image after adjusting the image size, color sampling, etc. frameIdx and compTimeIdx are composition time indexes.

[0115] The displacement unmapper 3052 determines whether or not to use the one-dimensional displacement decoded from the encoded data. DisplacementDim is derived according to the value of the flag asve_1d_displacement_flag.

[0116] DisplacementDim = (asve_1d_displacement_flag) ? 1 : 3 Here, asve_1d_displacement_flag=1 means that only one dimension of the three-dimensional displacement is transmitted. Indicates that the normal component, or x-component (first component) of the displacement is present in the (compressed) geometry image. If the 1D flag is true, the displacement unmapper 3052 The other two components are assumed to be 0. asve_1d_displacement_flag=0 indicates that all three components of the displacement are present in the (compressed) geometry image.

[0117] The displacement unmapper 3052 generates coded data of the NAL units of the atlas, e.g., ASPS You can also decode the ctuSizeIdx (videoBlockSizeIdx) value and find the value 16< <ctuSizeIdxもしくは32<<ctuSizeIdx、64<<ctuSizeIdxから、ctuSize may be derived.

[0118] The displacement unmapper 3052 may use, for example, 64 if gi_geometry_codec_id[DecAtlasID] is HEVC, or 128 if it is VVC, as follows:

[0119] ctuSize = ptl_profile_codec_group_idc == 3 (VVC) ? 128 : 64 Here, the value of ptl_profile_codec_group_idc is 0: AVC Progressive High, 1: HEVC Main 10 , 2:HEVC Main 444, 3:VVC Main 10.

[0120] ctuSize = gi_geometry_codec_id[DecAtlasID] 4CC code indicates HEVC ? 64 : 128 Here, the 4CC code strings indicating HEVC and VVC are "hev1" and "vvi1", respectively.

[0121] Alternatively, the displacement unmapper 3052 may be configured to set the maximum CTU size of HEVC to 64 and the maximum CTU size of VVC to 64. The larger of the maximum size values ​​of 128 may be fixed to 128.

[0122] In one configuration, the displacement unmapper 3052 receives a three-dimensional array dispQuantCoeffFrame of size asps_frame_width × asps_frame_height × DisplacementDim, variables patch2dSizeX, patch2dSizeY, patch2dPosX, patch2dPosY, bitDepth, subdivisionIterationCount, vertex Using the number of vertices verCoordCount and the array levelOfDetailVertexCounts of size subdivisionIterationCount + 1 as input, a two-dimensional array dispQuantCoeffArray of size verCoordCount × DisplacementDim is derived, which indicates the quantized displacement wavelet coefficients. The two-dimensional array dispQuantCoeffArray is initialized to 0. Also, a one-dimensional array of size subdivisionIterationCount + 1 is All elements of the column, vStart, vEnd, startBlock are initialized as 0 and the variable blockCount is set as 0.

[0123] where patch2dSizeX specifies the width of the patch's bounding box, patch2dSizeY specifies the height of the patch's bounding box, and patch2dPosX specifies the width of the patch's bounding box. Specify the x coordinate of the upper left corner of the bounding box of the patch, and patch2dPosY specifies the y coordinate of the upper left corner of the bounding box of the patch. Also, for example, from ASPS, set bitDepth=asps_geometry_3d_bit_depth_minus1+1, subdivisionIterationCount=asve_subdivision_iteration_count. The parameters may be derived: blockSize may be ctuSize.

[0124] Each variable may be derived as follows:

[0125] patchWidthInBlocks = (patch2dSizeX + blockSize - 1) / blockSize pixelsPerBlock = blockSize * blockSize shift = (1 << bitDepth) >> 1 if ( subdivisionIterationCount = 0 ){ vEnd

[0000] = verCoordCount blockCount = ( verCoordCount + blockSize - 1 ) / blockSize } else { for( i = 0; i < subdivisionIterationCount + 1; i++ ){ vStart[ i ] = i == 0? 0 : levelOfDetailVertexCounts[ i - 1 ] vEnd[ i ] = levelOfDetailVertexCounts[ i ] blockCountLevel [ i ] = (vEnd[ i ] - vStart[ i ] + blockSize - 1) / blockSize startBlock[ i ] = i == 0 ? 0 : (startBlock[ i - 1 ] + blockCount [ i ]) blockCount += blockCountLevel[ i ] } } Here, the variable dispPackingOrder is set to the syntax element asve_packing_method decoded from the coded data, and the variable videoChromaFormat is set to the variable DecGeoChromaFormat of the decoded geometry video component. dispQuantCoeffArray is set as follows: It may be derived.

[0126] heightInBlocks = (blockCount + widthInBlocks - 1) / patchWidthInBlocks origHeight = heightInBlocks * blockSize totalBlocksInPatch = ( patch2dSizeX * origHeight ) / pixelsPerBlock for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for( v = vStart[ lodIdx ]; v < vEnd[ lodIdx ]; v++ ) { blockIndex = (v - vStart[ lodIdx ]) / pixelsPerBlock + startBlock[ lodIdx ] indexWithinBlock = ( v - vStart[ lodIdx ] ) % pixelsPerBlock if( dispPackingOrder ){ blockIndex = totalBlocksInVideoFrame - 1 - blockIndex indexWithinBlock = pixelsPerBlock - 1 - indexWithinBlock } x0 = ( blockIndex % widthInBlocks ) * blockSize y0 = ( blockIndex / widthInBlocks ) * blockSize ( x, y ) = computeMorton2D( indexWithinBlock ) x1 = x0 + x + patch2dPosX y1 = y0 + y + patch2dPosX for( d = 0; d < DisplacementDim; d++ ) { if ( videoChromaFormat == 4:2:0 || videoChromaFormat == 4:2:2 || videoChromaFormat == 4:0:0 ) { if( dispPackingOrder ) dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ x1 ][ d * origHeight + y1 ]

[0000] - shift } else { dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ x1 ][ y1 ][ d ] - shift } } } } where asve_packing_method=0 means that the displacement component samples are packed in ascending order. asve_packing_method=1 indicates that the displacement component samples are packed in descending order. computeMorton2D is a function for implementing Z-order scanning and is defined below.

[0127] x = extracOddBits(x) { x = x & 0x55555555 x = (x | (x >> 1)) & 0x33333333 x = (x | (x >> 2)) & 0x0F0F0F0F x = (x | (x >> 4)) & 0x00FF00FF x = (x | (x >> 8)) & 0x0000FFFF } (x, y) = computeMorton2D(i) { x = extracOddBits(i>>1) y = extracOddBits(i) } In addition, in the above pseudocode, instead of a double-loop structure with an outer loop for the level of detail and an inner loop for the dimension unit, we decode the vertically arranged displacement images using a double-loop structure with an outer loop for the dimension unit and an inner loop for the level of detail, and derive dispQuantCoeffArray as follows: That's fine.

[0128] heightInBlocks = (blockCount + widthInBlocks - 1) / patchWidthInBlocks origHeight = heightInBlocks * blockSize totalBlocksInPatch = ( patch2dSizeX * origHeight ) / pixelsPerBlock for( d = 0; d < DisplacementDim; d++ ) { for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for( v = vStart[ lodIdx ]; v < vEnd[ lodIdx ]; v++ ) { blockIndex = (v-vStart[ lodIdx ]) / pixelsPerBlock + startBlock[ lodIdx ] indexWithinBlock = ( v - vStart[ lodIdx ] ) % pixelsPerBlock if( dispPackingOrder ){ blockIndex = totalBlocksInVideoFrame - 1 - blockIndex indexWithinBlock = pixelsPerBlock - 1 - indexWithinBlock } x0 = ( blockIndex % widthInBlocks ) * blockSize y0 = ( blockIndex / widthInBlocks ) * blockSize ( x, y ) = computeMorton2D( indexWithinBlock ) x1 = x0 + x + patch2dPosX y1 = y0 + y + patch2dPosX if ( videoChromaFormat == 4:2:0 || videoChromaFormat == 4:2:2 || videoChromaFormat == 4:0:0 ) { if( dispPackingOrder ) dispQuantCoeffArray[ d ][ v ] = dispQuantCoeffFrame

[0000] [ x1 ][ d * origHeight + y1 ] - shift } else { dispQuantCoeffArray[ d ][ v ] = dispQuantCoeffFrame[ d ][ x1 ][ y1 ] - shift } } } } (Decoding and derivation for each sub-mesh) Alternatively, the displacement unmapper 3052 may decode and derive the mesh displacement for each sub-mesh.

[0129] Figure 22 shows the mesh displacement division parameter afve_displacement_component_height_submesh[i] for each submesh and the mesh displacement division flag afve_displacement_submesh_alignment_flag This is an example of the syntax for transmitting at the picture / frame level (AFPS) depending on the value of afve_displacement_submesh_alignment_flag. afve_displacement_component_height_submesh[i] indicates whether to divide the mesh displacement into sub-meshes or not, and afve_displacement_component_height_submesh[i] indicates whether to divide the mesh displacement into sub-meshes or not. indicates the height of each component of the sub-mesh corresponding to the index i of the mesh displacement obtained. vinegar.

[0130] Here, verCoordCount[ submeshIdx ] represents the number of vertices in a certain submeshIdx, and each variable The number may be derived as follows:

[0131] for( submeshIdx = 0; submeshIdx < NumSubMeshes; submeshIdx++ ){ if ( subdivisionIterationCount = 0 ){ vEnd[ submeshIdx ]

[0000] = verCoordCount[ submeshIdx ] blockCount[ submeshIdx ] = ( verCoordCount[ submeshIdx ] + blockSize - 1 ) / blockSize } else { for( i = 0; i < subdivisionIterationCount + 1; i++ ){ vStart[ submeshIdx ][ i ] = i==0 ? 0 : levelOfDetailVertexCounts[ submeshIdx ][ i - 1 ] vEnd[ submeshIdx ][ i ] = levelOfDetailVertexCounts[ submeshIdx ][ i ] blockCountLevel[ submeshIdx ][ i ] = ( vEnd[ submeshIdx ][ i ] - vStart[ submeshIdx ][ i ] + blockSize - 1 ) / blockSize startBlock[ submeshIdx ][ i ] = i==0 ? 0 : ( startBlock[ submeshIdx ][ i - 1 ] + blockCount [ i ] ) blockCount[ submeshIdx ] += blockCountLevel[ submeshIdx ][ i ] } } } Figures 23, 24, 25, 26, 27, and 28 are examples of mesh displacement images divided by the number of geometry dimensions and the number of sub-mesh divisions. Mesh displacement images can be partitioned within the screen using sub-meshes and dimensions as units, and the mesh displacement can be coded and decoded as an image in partition units. Codecs such as HEVC and VVC divide the screen Since it has the function of decoding in units of divided tiles and subpictures, it becomes possible to decode mesh displacement images of specific sub-meshes and dimensions.

[0132] The displacement unmapper 3052 receives the three-dimensional array dispQuantCoeffFrame of size asps_frame_width × asps_frame_height × DisplacementDim as input and performs the following processing to generate a quantized displacement wavelet coefficient array of size verCoordCount[submeshIdx] × Displacement For example, as shown in Figure 23, a two-dimensional array dispQuantCoeffArray of entDim can be derived. The displacement images arranged in the direction may be decoded. The following is a specific example in which NumSubMeshes=3, DisplacementDim=3, videoChromaFormat=4:2:0, and dispPackingOrder=1 are set when deriving dispQuantCoeffArray using the following processing procedure.

[0133] heightOffset = 0 for( submeshIdx = 0; submeshIdx < NumSubMeshes; submeshIdx++ ) { origHeight = afve_displacement_component_height_submesh[ submeshIdx ] for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++) { blockIndex = ( v - vStart[ submeshIdx ][ lodIdx ] ) / pixelsPerBlock + startBlock[ submeshIdx ][ lodIdx ] indexWithinBlock = ( v - vStart[submeshIdx][lodIdx] ) % pixelsPerBlock if( dispPackingOrder ){ blockIndex = totalBlocksInVideoFrame - 1 - blockIndex indexWithinBlock = pixelsPerBlock - 1 - indexWithinBlock } x0 = ( blockIndex % widthInBlocks ) * blockSize y0 = ( blockIndex / widthInBlocks ) * blockSize ( x, y ) = computeMorton2D( indexWithinBlock ) x1 = x0 + x + patch2dPosX y1 = y0 + y + patch2dPosX for( d = 0; d < DisplacementDim; d++ ) { if ( videoChromaFormat == 4:2:0 || videoChromaFormat == 4:2:2 || videoChromaFormat == 4:0:0 ) { if( dispPackingOrder ) dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[x1][d*origHeight+heightOffset+y1][0] - shift } else { dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[x1][heightOffset / DisplacementDim+y1][d]-shift }} } } heightOffset += DisplacementDim * origHeight } In addition, the displacement unmapping unit 3052 may take as input a four-dimensional array dispQuantCoeffFrame of size NumSubMeshes × asps_frame_width × asps_frame_height × DisplacementDim and derive a two-dimensional array dispQuantCoeffArray of size verCoordCount[submeshIdx] × DisplacementDim that indicates the quantized displacement wavelet coefficients by the following processing.

[0134] for( submeshIdx = 0; submeshIdx < NumSubMeshes; submeshIdx++ ) { origHeight = afve_displacement_component_height_submesh[ submeshIdx ] for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++) { ... for( d = 0; d < DisplacementDim; d++ ) { if ( videoChromaFormat == 4:2:0 || videoChromaFormat == 4:2:2 || videoChromaFormat == 4:0:0 ) { if( dispPackingOrder ) dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[submeshIdx][x1][d*origHeight+y1][0] - shift } else { dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ submeshIdx ][ x1 ][ y1 ][ d ] - shift } } } } } In another configuration, the displacement unmapper 3052 may use the three-dimensional array dispQuantCoeffFrame of size asps_frame_width × asps_frame_height × DisplacementDim as input to derive the two-dimensional array dispQuantCoeffArray of size verCoordCount[submeshIdx] × DisplacementDim, which indicates the quantized displacement wavelet coefficients, by the following processing procedure (dimension first) instead of the above example (submesh first). For example, as shown in Figure 24, a double loop structure is used, with the dimension unit (d) as the outer loop and the submesh unit (subMeshIdx) as the inner loop, to derive the two-dimensional array dispQuantCoeffArray of size verCoordCount[submeshIdx] × DisplacementDim, which indicates the quantized displacement wavelet coefficients. The dispQuantCoeffArray This is a specific example where NumSubMeshes=3, DisplacementDim=3, videoChromaFormat=4:2:0, and dispPackingOrder=1 are used when deriving.

[0135] heightOffset = 0 for( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) heightOffset += afve_displacement_component_height_submesh[ submeshIdx ] for( d = 0; d < DisplacementDim; d++ ) { origHeight = 0 for( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) { for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++){ ... if ( videoChromaFormat == 4:2:0 || videoChromaFormat == 4:2:2 || videoChromaFormat == 4:0:0 ) { if( dispPackingOrder ) dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[x1][d*heightOffset+origHeight+y1][0]-shift } else { dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ x1 ][ origHeight + y1 ][ d ] - shift } } } origHeight += afve_displacement_component_height_submesh[ submeshIdx ] } } In addition, the displacement unmapping unit 3052 may use a four-dimensional array dispQuantCoeffFrame of size DisplacementDim × NumSubMeshes × asps_frame_width asps_frame_height as input, and derive a two-dimensional array dispQuantCoeffArray of size verCoordCount[submeshIdx] × DisplacementDim that indicates the quantized displacement wavelet coefficients by the following processing.

[0136] for( d = 0; d < DisplacementDim; d++ ) { for( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) { origHeight = afve_displacement_component_height_submesh[ submeshIdx ] for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++){ ... if ( videoChromaFormat == 4:2:0 || videoChromaFormat == 4:2:2 || videoChromaFormat == 4:0:0 ) { if( dispPackingOrder ) dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[0][submeshIdx][x1][d*origHeight+y1] - shift } else { dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ d ][ submeshIdx ][ x1 ][ y1 ]- shift } } } } } In another configuration, for example, as shown in Figure 25, a double loop structure is used, with the submesh unit (subMeshIdx) as the outer loop and the dimension unit (d) as the inner loop, and the submesh is arranged vertically and the dimension is arranged horizontally. Here, the displacement unmapper 3052 generates a three-dimensional image of size asps_frame_width × asps_frame_height × DisplacementDim by the following process. Takes the array dispQuantCoeffFrame as input, indicating the quantized displacement wavelet coefficients. A two-dimensional array dispQuantCoeffArray of size verCoordCount[submeshIdx]×DisplacementDim may be derived.

[0137] heightOffset = 0 origWidth = widthInBlocks * blockSize / DisplacementDim for( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) { for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++) { ... for( d = 0; d < DisplacementDim; d++ ) { if ( videoChromaFormat == 4:2:0 || videoChromaFormat == 4:2:2 || videoChromaFormat == 4:0:0 ) { if( dispPackingOrder ) dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[d*origWidth+x1][heightOffset+y1][0] - shift } else { dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ x1 ][ heightOffset + y1 ][ d ] - shift } } } } heightOffset += afve_displacement_component_height_submesh[ submeshIdx ] } In another configuration, in the syntax example of Figure 22, the mesh displacement divided into sub-meshes is Instead of decoding and encoding afve_displacement_component_height_submesh[i], which indicates the height of each component of the submesh corresponding to the index i of the mesh displacement divided into sub-meshes, Alternatively, dispQuantCoeffArray may be derived by decoding and encoding afve_displacement_component_width_submesh[i], which indicates the width of the submesh.

[0138] afps_vdmc_extension( ) { ... if( afve_displacement_submesh_alignment_flag ) { for( i=0; i <NumSubMeshes; i++ ) afve_displacement_component_width_submesh[ i ] } ... } In this case, for example, as shown in Figure 26, a double loop structure is created with the dimension unit (d) as the outer loop and the submesh unit (subMeshIdx) as the inner loop, with the dimension in the vertical direction and the submesh in the horizontal direction. The displacement unmapper 3052 may use the three-dimensional array dispQuantCoeffFrame of size asps_frame_width × asps_frame_height × DisplacementDim as input to derive the two-dimensional array dispQuantCoeffArray of size verCoordCount[submeshIdx] × DisplacementDim that indicates the quantized displacement wavelet coefficients, by the following process.

[0139] heightInBlocks = (blockCount + widthInBlocks - 1) / patchWidthInBlocks origHeight = heightInBlocks * blockSize / DisplacementDim for( d = 0; d < DisplacementDim; d++ ) { origWidth = 0 for( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) { for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++){ if ( videoChromaFormat == 4:2:0 || videoChromaFormat == 4:2:2 || videoChromaFormat == 4:0:0 ) { if( dispPackingOrder ) dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame [submeshIdx*origWidth+x1][d*origHeight+y1][0] - shift } else { dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[submeshIdx*origWidth+x1][ y1 ][ d ] - shift } } } } origWidth += afve_displacement_component_width_submesh[ submeshIdx ] } In another configuration, the index i of the mesh displacement divided into sub-meshes is Instead of decoding and encoding afve_displacement_component_height_submesh[i], which indicates the height of each component of the corresponding submesh, Only the maximum value of the height of each component for each sub-mesh is decoded and encoded. For example, as shown in Figure 27, a double loop is used, with the sub-mesh unit (subMeshIdx) as the outer loop and the dimension unit (d) as the inner loop. The displacement image is decoded by sub-mesh in the vertical direction and dimension in the horizontal direction. In this case, the excess part (the shaded area in the figure) is padded with a value such as 0. Here, afve_displacement_component_height_submesh_max is the height of each component of the mesh displacement divided into sub-meshes. Indicates the maximum height.

[0140] afps_vdmc_extension( ) { ... if( afve_displacement_submesh_alignment_flag ) afve_displacement_component_height_submesh_max ... } In another configuration, the index i of the mesh displacement divided into sub-meshes is Instead of decoding and encoding afve_displacement_component_width_submesh[i], which indicates the width of each component in the corresponding submesh, only the maximum value of the width of each component in the submesh corresponding to the index i of the mesh displacement divided into submeshes is decoded and encoded. For example, as shown in Figure 28, the dimension unit (d) is the outer loop, and the submesh unit (subMeshIdx) is the inner loop. A two-dimensional array dispQuantCoeffArray may be derived by decoding a displacement image with a vertical dimension and a horizontal submesh in a double-loop structure. In this case, the excess part (the shaded area in the figure) is padded with a value such as 0. Here, afve_displacement_component_width_submesh_max indicates the maximum width of each component for each submesh of the mesh displacement divided into submeshes.

[0141] afps_vdmc_extension( ) { ... if( afve_displacement_submesh_alignment_flag ) afve_displacement_component_width_submesh_max ... } In another configuration, the displacement unmapper 3052 always sets the value of afve_displacement_submesh_alignment_flag to false if mesh displacements are always decoded and derived for each submesh. Or, without decoding or encoding afve_displacement_submesh_alignment_flag, each component of the submesh corresponding to the index i of the mesh displacement that is always divided into submeshes Alternatively, dispQuantCoeffArray may be derived by decoding and encoding afve_displacement_component_height_submesh[i], which indicates the height of the component.

[0142] In another configuration, the displacement unmapper 3052 always sets the value of afve_displacement_submesh_alignment_flag to false if mesh displacements are always decoded and derived for each submesh. Or, without decoding or encoding afve_displacement_submesh_alignment_flag, each component of the submesh corresponding to the index i of the mesh displacement that is always divided into submeshes Alternatively, dispQuantCoeffArray may be derived by decoding and encoding afve_displacement_component_width_submesh[i], which indicates the width of the component.

[0143] In another configuration, the displacement unmapping unit 3052 does not decode and encode afve_displacement_component_height_submesh[i], which indicates the height of each component of the submesh corresponding to the index i of the mesh displacement divided into sub-meshes, but instead decodes and encodes afve_displacement_component_height_submesh[i], which indicates the height of each component of the submesh corresponding to the index i of the mesh displacement divided into sub-meshes, as shown below. may be derived.

[0144] The displacement unmapper 3052 uses origHeight or origW derived from afve_displacement_component_height_submesh[i] and afve_displacement_component_width_submesh[i]. A conformance constraint may be added that requires the idth, or both, to be an integer multiple of ctuSize.

[0145] heightInBlocks = (blockCount[submeshIdx]+widthInBlocks-1) / patchWidthInBlocks origHeight = heightInBlocks * blockSize In another configuration, the displacement unmapper 3052 does not decode or encode afve_displacement_component_width_submesh[i], which indicates the width of each component of the submesh corresponding to the index i of the mesh displacement divided into submeshes, but instead performs the submesh division as shown below. The width of each component of the sub-mesh corresponding to the index i of the mesh displacement is derived. You can put it out.

[0146] widthInBlocks = (blockCount[submeshIdx]+heightInBlocks-1) / patchHeightInBlocks origWidth = widthInBlocks * blockSize The inverse quantization unit 3053 performs inverse quantization based on the quantization scale value iscale to derive the mesh displacement dispCoeffArray after transformation (e.g., wavelet transformation). dispCoeffArray may be in a Cartesian coordinate system or a local coordinate system. iscale is a value derived from the quantization parameter of each component of the mesh displacement image.

[0147] Vcount0 = 0 for( i = 0; i < subdivisionIterationCount; i++ ) { vcount1 = levelOfDetailCounts[ i ] for( v = vcount0; v < vcount1; v++ ) { for( d = 0; d < DisplacementDim; d++ ) { dispCoeffArray[v][d] = dispQuantCoeffArray[ v ][ d ] * iscale[ i ][ d ] } } vcount0 = vcount1 } Here, iscale is derived as follows:

[0148] lodQuantizationFlag = vqp_lod_quantization_flag[ QpIndex ] directQuantizationEnableFlag = vqp_direct_quantization_enabled_flag[QpIndex] for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for( dimIdx = 0; dimIdx < DisplacementDim; dimIdx ++ ) { iscale[ lodIdx ][ dimIdx ] = InverseScale[ QpIdx ][ lodIdx ][ dimIdx ] } } } else { for( dimIdx = 0; dimIdx < DisplacementDim; dimIdx++ ) { iscale

[0000] [ dimIdx ] = InverseScale[ QpIdx ]

[0000] [ dimIdx ] levelOfDetailInverseScale[ dimIdx ] = 1 << vqp_log2_lod_inverse_scale[ QpIdx ][ dimIdx ] } for( lodIdx = 1; lodIdx < lodCount; lodIdx++ ) { for( dimIdx = 0; dimIdx < DisplacementDim; dimIdx++ ) { iscale[ lodIdx ][ dimIdx ] = iscale[ lodIdx - 1 ][ dimIdx ] * levelOfDetailInverseScale[ dimIdx ] } } } The inverse transform unit 3054 performs an inverse transform g (for example, an inverse wavelet transform) to obtain the mesh displacement d as Derive.

[0149] d[d][v] = g(dispCoeffArray[v][d]) The coordinate system conversion unit 3055 converts the mesh displacement (coordinate system of the mesh displacement) into a Cartesian coordinate system based on the value of the coordinate system conversion information displacementCoordinateSystem. Specifically, when displacementCoordinateSystem = 1, the displacement in the local coordinate system is converted into a displacement in the Cartesian coordinate system. Here, d is a three-dimensional vector indicating the mesh displacement before the coordinate system conversion. disp is a three-dimensional vector indicating the mesh displacement after the coordinate system conversion, and is a Cartesian coordinate system. n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target region or target vertex.

[0150] if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } The derivation method shown above for vector multiplication can be expressed individually as scalars as follows:

[0151] if (displacementCoordinateSystem == 0) { for (i = 0; i < 3; i++) { disp[i] = d[i] } } else if (displacementCoordinateSystem == 1){ for (i = 0; i < 3; i++) { disp[i] = d[0] * n_vec[i] + d[1] * t_vec[i] + d[2] * b_vec[i] } } Note that disp=d is used to assign the same variable name before and after the transformation, and the value of d is updated by the coordinate transformation. The configuration may be such that:

[0152] Alternatively, the following configuration may be used.

[0153] if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 2){ disp = d[0] * n_vec2 + d[1] * t_vec2 + d[2] * b_vec2 } Here, n_vec2, t_vec2, and b_vec2 are three-dimensional vectors (in the Cartesian coordinate system) corresponding to the axes of the local coordinate system of the adjacent region.

[0154] The following configuration may also be used.

[0155] if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } Here, n_vec3, t_vec3, b_vec3 are the axes of the local coordinate system of the target area with reduced fluctuation. The corresponding 3D vector (in Cartesian coordinates). For example, the previous coordinates A vector of the coordinate system to be used for decoding is derived from the coordinate system and the current coordinate system.

[0156] n_vec3 = (w*n_vec3 + (WT - w)*n_vec)>>wShift t_vec3 = (w*t_vec3 + (WT - w)*t_vec)>>wShift b_vec3 = (w*b_vec3 + (WT - w)*b_vec)>>wShift Here, each variable can also have values such as wShift = 2, 3, 4, WT = 1<<wShift, and w = 1..WT - 1 For example, when w = 3 and wShift = 3, the following can also be used.

[0157] n_vec3 = (3*n_vec3 + 5*n_vec)>>3 t_vec3 = (3*t_vec3 + 5*t_vec)>>3 b_vec3 = (3*b_vec3 + 5*b_vec)>>3 Also, a configuration may be used where it can be selected according to the value of the coordinate system conversion information displacementCoordinateSystem decoded from the encoded data as follows.

[0158] if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 6){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } (Mesh Reconstruction) Figure 7 is a functional block diagram showing the configuration of the mesh reconstruction unit 307. The mesh reconstruction unit 307 is composed of a mesh division unit 3071 and a mesh deformation unit 3072.

[0159] The mesh dividing unit 3071 divides the base mesh output from the base mesh decoding unit 303. Divide the mesh into smaller pieces and generate a divided mesh.

[0160] FIG. 14(a) shows a part (triangle) of the base mesh, and the triangle is composed of vertices v1, v2, and v3. v1, v2, and v3 are three-dimensional vectors. The mesh division unit 3071 generates the divided mesh by adding new vertices v12, v13, and v23 to the middle of each side of the triangle. and output (Fig. 14(b)).

[0161] v12 = (v1 + v2) / 2 v13 = (v1 + v3) / 2 v23 = (v2 + v3) / 2 The following is also possible:

[0162] v12 = (v1 + v2 + 1) >> 1 v13 = (v1 + v3 + 1) >> 1 v23 = (v2 + v3 + 1) >> 1 The mesh deformation unit 3072 receives the division mesh and the mesh displacement, and outputs the mesh displacement d12, A deformed mesh is generated and output by adding d13 and d23 (FIG. 14(c)). The mesh displacement is the output of the mesh displacement decoding unit 305 (coordinate system conversion unit 3055). d12, d13, and d23 are mesh displacements corresponding to the vertices v12, v13, and v23 added by the mesh division unit 3071.

[0163] v12' = v12 + d12 v13' = v13 + d13 v23' = v23 + d23 It should be noted that d12 = disp[0][], d23 = disp[1][], and d23 = disp[3][] may also be used.

[0164] (Configuration of 3D data encoding device according to the first embodiment) FIG. 15 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11 according to the first embodiment. The 3D data encoding device 11 includes an atlas information encoding unit 101, a base mesh encoding unit 103, a base mesh decoding unit 104, a mesh displacement updating unit 106, a mesh displacement encoding unit 107, a mesh The 3D data encoding device 11 is composed of a mesh displacement decoding unit 108, a mesh reconstruction unit 109, an attribute update unit 110, a padding unit 111, a color space conversion unit 112, an attribute encoding unit 113, a multiplexing unit 114, and a mesh separation unit 115. The 3D data encoding device 11 inputs atlas information, a base mesh, a mesh displacement, a mesh, and an attribute image as 3D data, and outputs encoded data.

[0165] The atlas information encoding unit 101 encodes the atlas information and outputs the atlas information encoded stream. Output the program.

[0166] The base mesh encoding unit 103 encodes the base mesh and generates a base mesh encoding sequence. The stream is output using an encoding method such as Draco.

[0167] The base mesh decoding unit 104 is similar to the base mesh decoding unit 303, and therefore a description thereof will be omitted.

[0168] The mesh displacement update unit 106 updates the (original) base mesh and the decoded base mesh. Based on the mesh, adjust the mesh displacement and output the updated mesh displacement.

[0169] The mesh displacement encoding unit 107 encodes the updated mesh displacement and generates a mesh displacement code The encoded stream is output using VVC, HEVC, or other encoding methods.

[0170] The mesh displacement decoding unit 108 is similar to the mesh displacement decoding unit 305, and therefore a description thereof will be omitted.

[0171] The mesh reconstruction unit 109 is similar to the mesh reconstruction unit 307, and therefore a description thereof will be omitted.

[0172] The attribute update unit 110 inputs the (original) mesh, the reconstructed mesh output from the mesh reconstruction unit 109 (mesh deformation unit 3072), and the attribute image, updates the attribute image to match the position (coordinates) of the reconstructed mesh, and outputs the updated attribute image.

[0173] The padding unit 111 receives the attribute image and pads the pixel values ​​in the empty area. Perform the processing.

[0174] The color space conversion unit 112 performs color space conversion from the RGB format to the YCbCr format.

[0175] The attribute encoding unit 113 encodes the YCbCr formatted image data output from the color space conversion unit 112. The attribute image is encoded and an attribute video stream is output. The encoding method is VVC or HEVC.

[0176] The multiplexing unit 114 multiplexes the atlas information coded stream, the base mesh coded stream, The mesh displacement coding stream and the attribute video stream are multiplexed and output as coded data. Multiplexing methods include byte stream format and ISOBMFF. is used.

[0177] (Mesh separation unit operation) The mesh separation unit 115 generates a base mesh and a mesh displacement from the mesh.

[0178] 19 is a functional block diagram showing the configuration of the mesh separation unit 115. The mesh separation unit 115 is made up of a mesh thinning unit 1151, a mesh division unit 1152, and a mesh displacement derivation unit 1153.

[0179] The mesh thinning unit 1151 generates a base mesh by thinning out some of the vertices from the mesh.

[0180] Figure 20(a) shows a part of a mesh, which has vertices v1, v2, v3, v4, v5, 20(b)).

[0181] The mesh dividing unit 1152 divides the base mesh to generate divided meshes, similar to the mesh dividing unit 3071 (FIG. 20(c)).

[0182] v4' = (v1 + v2) / 2 v5' = (v1 + v3) / 2 v6' = (v2 + v3) / 2 The mesh displacement calculation part calculates the displacement for vertices v4', v5', and v6' based on the mesh and the divided mesh. The displacements d4, d5, and d6 of the vertices v4, v5, and v6 are calculated as mesh displacements and output (Fig. 20(d)). .

[0183] d4 = v4 - v4' d5 = v5 - v5' d6 = v6 - v6' (Base mesh encoding) FIG. 17 is a functional block diagram showing the configuration of the base mesh encoding unit 103. The mesh encoding unit 103 is composed of a mesh encoding unit 1031, a mesh decoding unit 1032, a motion information encoding unit 1033, a motion information decoding unit 1034, a mesh motion compensation unit 1035, a reference mesh memory 1036, a switch 1037, and a switch 1038. The base mesh encoding unit 103 may also include a base mesh quantization unit (not shown) after the base mesh is input. When encoding a base mesh without referring to other base meshes (e.g., an already encoded base mesh) (intra-coding), the switches 1037 and 1038 are connected to the side that does not perform motion compensation. When encoding a base mesh with reference to other base meshes (inter-coding), the switches 1037 and 1038 are connected to the side that performs motion compensation.

[0184] The mesh encoding unit 1031 has an intra-encoding function, performs intra-encoding on the base mesh, and outputs a base mesh encoded stream. There are.

[0185] The mesh decoding unit 1032 is similar to the mesh decoding unit 3031, and therefore a description thereof will be omitted.

[0186] The motion information encoding unit 1033 has an inter-encoding function, performs inter-encoding on the base mesh, and outputs a base mesh encoded stream. The encoding method used is entropy encoding such as arithmetic encoding.

[0187] The motion information decoding unit 1034 is similar to the motion information decoding unit 3032, and therefore a description thereof will be omitted.

[0188] The mesh motion compensation unit 1035 is similar to the mesh motion compensation unit 3033, and therefore a description thereof will be omitted.

[0189] The reference mesh memory 1036 is similar to the reference mesh memory 3034, and therefore a description thereof will be omitted.

[0190] (Encoding of atlas information) 16 is a functional block diagram showing the configuration of the atlas information encoding unit 101. The encoding unit 101 includes a patch information encoding unit 1011, a submesh information encoding unit 1012, and an extension information encoding unit 1013. The image encoding unit 1014 is made up of a coding unit 1013, a tile information coding unit 1014, and a parameter coding unit 1015.

[0191] The submesh information encoding unit 1012 encodes the number of submeshes and submesh IDs referenced at the picture / frame level.

[0192] The submesh information encoding unit 1012 may encode the number of submeshes and submesh IDs referenced at the tile level of the picture / frame, or if the referenced submesh includes tiles, may encode the number of referenced tiles and tile IDs so that they correspond to the submesh IDs.

[0193] (Mesh displacement encoding) 18 is a functional block diagram showing the configuration of the mesh displacement encoding unit 107. The encoding unit 107 includes a coordinate system conversion unit 1071, a conversion unit 1072, a quantization unit 1073, and a displacement mapping unit 1074. As shown in the figure, the mesh displacement coding unit 107 may further include a moving image coding unit 1075. Alternatively, the moving image coding unit 1075 may include The mesh displacement encoding unit 107 does not include the mesh displacement encoding unit 107, and the encoding of the displaced image is performed by an external image encoding device. It may be configured to use

[0194] The coordinate system conversion unit 1071 converts the coordinate system of the mesh displacement from a Cartesian coordinate system to a coordinate system that encodes the displacement (for example, a local coordinate system) based on the value of the coordinate conversion information displacementCoordinateSystem. Here, disp is a three-dimensional vector indicating the mesh displacement before the coordinate system conversion, d is a three-dimensional vector indicating the mesh displacement after the coordinate system conversion, and n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) indicating each axis of the local coordinate system.

[0195] if (displacementCoordinateSystem == 0) { d = disp } else if (displacementCoordinateSystem == 1){ d = (disp * n_vec, disp * t_vec, disp * b_vec) } The mesh displacement coding unit 107 encodes the value of the displacementCoordinateSystem into the picture / frame. It may be updated at the system level.

[0196] When encoding the displacementCoordinateSystem at the sequence level, the sequence of the configuration in Figure 8 The asve_displacement_coordinate_system uses the Cartesian coordinate system. Set to 0 if the coordinate system is in the local coordinate system, and 1 if the coordinate system is in the local coordinate system.

[0197] When changing the displacementCoordinateSystem at the picture / frame level, use the syntax in the configuration in Figure 11. Set afve_overriden_flag to 1 if you want to update the coordinate system, or 0 if you do not want to update the coordinate system. Set afve_displacement_coordinate_system to 0 if you want to use a Cartesian coordinate system, or 1 if you want to use a local coordinate system.

[0198] The transformation unit 1072 performs a transformation f (for example, a wavelet transformation) to derive the transformed mesh displacement Tdisp for pos=0..NumDisp-1, where NumDisp is the mesh vertex displacement. Number of points.

[0199] dispCoeffArray[v][d] = f(d[d][v]) The quantization unit 1073 performs quantization based on the quantization scale value "scale" derived from the quantization parameter of each component of the mesh displacement, and derives the mesh displacement dispQuantCoeffArray after quantization.

[0200] Vcount0 = 0 for( i = 0; i < subdivisionIterationCount; i++ ) { vcount1 = levelOfDetailCounts[ i ] for( v = vcount0; v < vcount1; v++ ) { for( d = 0; d < DisplacementDim; d++ ) { dispQuantCoeffArray[v][d] = dispCoeffArray[ v ][ d ] / iscale[ i ][ d ] } } vcount0 = vcount1 } Alternatively, you can approximate the scale value as a power of 2 and derive dispQuantCoeffArray using the following formula: .

[0201] Vcount0 = 0 for( i = 0; i < subdivisionIterationCount; i++ ) { scale[i] = 1 << scale2[i] vcount1 = levelOfDetailCounts[ i ] for( v = vcount0; v < vcount1; v++ ) { for( d = 0; d < DisplacementDim; d++ ) { dispQuantCoeffArray[v][d] = dispCoeffArray[v][d] >> scale2[i][d] } } vcount0 = vcount1 } The displacement mapping unit 1074 generates an image dispQuantCoeffFrame from the quantized mesh displacements dispQuantCoeffArray based on the value of the displacement mapping parameter displacementChromaLocationType.

[0202] The displacement mapper 1074 calculates the first component of the (quantized) mesh displacement array as follows: The min dispQuantCoeffArray[v][0] may be mapped to the luminance (Y) image component. For image width W and height H (y=0..H-1, x=0..W-1), apply the following:

[0203] H = origHeight shift = (1 << bitDepth) >> 1 dispQuantCoeffFrame[x][ y][0] = dispQuantCoeffArray[v][0] + shift dispQuantCoeffFrame[x][ H+y][0] = dispQuantCoeffArray[v][1] + shift dispQuantCoeffFrame[x][2*H+y][0] = dispQuantCoeffArray[v][2] + shift v++ dispQuantCoeffFrame[x / 2][ y / 2][1] = shift dispQuantCoeffFrame[x / 2][H / 2+y / 2][1] = shift dispQuantCoeffFrame[x / 2][ H+y / 2][1] = shift dispQuantCoeffFrame[x / 2][ y / 2][2] = shift dispQuantCoeffFrame[x / 2][H / 2+y / 2][2] = shift dispQuantCoeffFrame[x / 2][ H+y / 2][2] = shift Alternatively, you can do the following:

[0204] for (d = 0; d < 3; d++) { dispQuantCoeffFrame [x][d*H+y][0] = dispQuantCoeffArray[v][d] + shift if (d != 0) { dispQuantCoeffFrame[x / 2][d*H / 2+yc][d] = shift } } v++ Alternatively, you can do the following:

[0205] heightOffset = 0 for( submeshIdx = 0; submeshIdx < NumSubMeshes; submeshIdx++ ) { afve_displacement_component_height_submesh[submeshIdx] = origHeight[ submeshIdx ] if( submeshIdx > 0 ) heightOffset += afve_displacement_component_height_submesh[submeshIdx-1] H = afve_displacement_component_height_submesh[ submeshIdx ] for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for( v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++ ){ for( d = 0; d < DisplacementDim; d++ ) { dispQuantCoeffFrame[x][ d * H + heightOffset + y ][0] = dispQuantCoeffArray[ v ][ d ] + shift } } } heightOffset += DisplacementDim * origHeigh } Alternatively, the following may be done.

[0206] for( submeshIdx = 0; submeshIdx < NumSubMeshes; submeshIdx++ ) { afve_displacement_component_height_submesh[submeshIdx] = origHeight[ submeshIdx ] H = afve_displacement_component_height_submesh[ submeshIdx ] for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for( v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++ ){ for( d = 0; d < DisplacementDim; d++ ) { dispQuantCoeffFrame[ submeshIdx ][ x ][ d * H + y ]

[0000] = dispQuantCoeffArray[ v ][ d ] + shift } } } } Alternatively, you may do the following.

[0207] heightOffset = 0 for ( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) { afve_displacement_component_height_submesh[ submeshIdx ] = origHeight[ submeshIdx ] heightOffset += afve_displacement_component_height_submesh[ submeshIdx ] } for( d = 0; d < DisplacementDim; d++ ) { H = 0 for ( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) { for ( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++){ dispQuantCoeffFrame[ x ][ d * heightOffset + H + y ]

[0000] = dispQuantCoeffArray[ v ][ d ] + shift } } } H += afve_displacement_component_height_submesh[ submeshIdx ] } Alternatively, you can do the following:

[0208] for( d = 0; d < DisplacementDim; d++ ) { for( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) { H = afve_displacement_component_height_submesh[ submeshIdx ] for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++){ afve_displacement_component_height_submesh[ submeshIdx ] = origHeight[ submeshIdx ] H = afve_displacement_component_height_submesh[ submeshIdx ] dispQuantCoeffFrame

[0000] [ submeshIdx ][ x ][ d * H + y ] = dispQuantCoeffArray[ v ][ d ] + shift } } } } Alternatively, you can do the following:

[0209] heightOffset = 0 W = width / DisplacementDim H = 0 for ( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) { afve_displacement_component_height_submesh[ submeshIdx ] = origHeight[ submeshIdx ] for ( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++){ for( d = 0; d < DisplacementDim; d++ ) { dispQuantCoeffFrame[ d * W + x ][ H + y ][0] = dispQuantCoeffArray[ v ][ d ] + shift } } } H = afve_displacement_component_height_submesh[ submeshIdx ] } Alternatively, you may do the following.

[0210] H = height / DisplacementDim for( d = 0; d < DisplacementDim; d++ ) { W = 0 for ( submeshIdx = 0; submeshIdx < NumSubmesh; submeshIdx++ ) { afve_displacement_component_width_submesh[ submeshIdx ] = origWidth[ submeshIdx ] W = afve_displacement_component_width_submesh[ submeshIdx ] for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for(v = vStart[submeshIdx][lodIdx]; v < vEnd[submeshIdx][lodIdx]; v++){ dispQuantCoeffFrame[ submeshIdx * W + x ][ d * H + y ][0] = dispQuantCoeffArray[ v ][ d ] - shift } } } W += afve_displacement_component_width_submesh[ submeshIdx ] } Alternatively, the mesh displacement image dispQuantCoeffFrame may be coded using only the maximum value of origHeight[submeshIdx].

[0211] Alternatively, the mesh displacement image dispQuantCoeffFrame may be coded using only the maximum value of origWidth[submeshIdx].

[0212] The processing may be switched depending on DecGeoChromaFormat. That is, when DecGeoChromaFormat=1 (4:2:0), the above processing is performed, and when DecGeoChromaFormat=3 (4:4:4), the following processing is performed.

[0213] dispQuantCoeffFrame[x][y][d] = dispQuantCoeffArray[v][0] dispQuantCoeffFrame[x][y][d] = dispQuantCoeffArray[v][1] dispQuantCoeffFrame[x][y][d] = dispQuantCoeffArray[v][2] v++ The mesh displacement encoding unit 107 may update the values ​​of origHeight and origWidth at the picture / frame level.

[0214] The video encoding unit 1075 encodes the image in YCbCr4:2:0 format including the (quantized) mesh displacement image, and outputs a mesh displacement encoded stream. The encoding method used is VVC, HEVC, or the like.

[0215] The video encoding unit 1075 may divide the mesh displaced image into slices for each origHeight and encode the slices. Alternatively, the origHeight may be aligned to a predetermined size according to the CTU size. .

[0216] The video encoding unit 1075 converts a first component (e.g., D) of the mesh displacement into a first slide. slice, the second component (e.g., U) to the second slice, the third component (e.g., V ) may be assigned to the third slice and coded (displacementSliceType=1).

[0217] Alternatively, the video encoding unit 1075 may encode the mesh displacement by allocating the first component to the first slice and the second and third components to the second slice (displacementSliceType=2).

[0218] As described above, by assigning different slices to each component of the mesh displacement, the decoding device can decode only some of the components, simplifying the process. In addition, since the decoding device can decode slices including the second and third components of the mesh displacement as needed, a scalability function can be realized. Furthermore, even if an error is mixed into the encoded data, the decoding device can decode only the slices (components) without errors, thereby enhancing error resistance.

[0219] One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention.

[0220] [Application example] The above-described 3D data encoding device 11 and 3D data decoding device 31 can be mounted on various devices that transmit, receive, record, and play back 3D data. The 3D data may be natural 3D data captured by a camera or the like, or artificial 3D data (including CG and GUI) generated by a computer or the like.

[0221] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. In other words, embodiments obtained by combining technical means modified appropriately within the scope of the claims are also included in the technical scope of the present invention. [Industrial Applicability]

[0222] The embodiments of the present invention can be suitably applied to a 3D data decoding device that decodes coded data in which 3D data has been coded, and a 3D data coding device that generates coded data in which 3D data has been coded, and can also be suitably applied to the data structure of coded data that is generated by the 3D data coding device and referenced by the 3D data decoding device. [Explanation of symbols]

[0223] 11 3D data encoding device 101 Atlas Information Encoding Unit 1011 Patch information encoding unit 1012 Submesh information encoding unit 1013 Extended information encoding unit 1014 Tile information encoding unit 1015 Parameter Encoding Unit 103 Base mesh coding unit 1031 Mesh coding unit 1032 Mesh Decoding Unit 1033 Motion information encoding unit 1034 Motion information decoding unit 1035 Mesh motion compensation unit 1036 reference mesh memory 1037 Switch 1038 Switch 1039 Skip Coding 104 Base mesh decoding unit 106 Mesh displacement update section 107 Mesh displacement coding unit 1071 Coordinate system conversion unit 1072 Conversion Unit 1073 Quantization section 1074 Binarization section 1075 Arithmetic encoder 1076 Context Selection 1077 Context Initialization Unit 108 Mesh displacement decoding unit 109 Mesh reconstruction unit 110 Attribute Update Section 111 Padding section 112 Color space conversion unit 113 Attribute Encoding Unit 114 Multiplexer 115 mesh separation section 1151 Mesh thinning section 1152 Mesh division section 1153 Mesh displacement derivation part 21 Network 31 3D data decoding device 301 Demultiplexer 302 Atlas Information Decoding Unit 3021 Parameter Decoding Unit 3022 Tile information decoding unit 3023 Extended Information Decoding Unit 3024 Submesh information decoding unit 3025 Patch Information Decoding Unit 303 Base mesh decoding unit 3031 Mesh Decoding Unit 3032 Motion information decoding unit 3033 Mesh Motion Compensation Unit 3034 Reference Mesh Memory 3035 Switch 3036 Switch 3037 Skip Decoding Unit 305 Mesh displacement decoding unit 3051 Arithmetic Decoding Unit 3052 Multi-level conversion section 3053 Inverse quantization section 3054 Inverse Conversion Unit 3055 Coordinate system conversion unit 3056 Context Selection Section 3057 Context Initialization Unit 307 Mesh Reconstruction Unit 306 Attribute Decoding Unit 3071 Mesh division section 3072 Mesh deformation part 308 Color Space Conversion Unit 41 3D data display device

Claims

1. A 3D data decoding device that decodes mesh data or point cloud data comprises an atlas information decoding unit that decodes atlas information from encoded data in which the mesh data or point cloud data is encoded, a base mesh decoding unit that decodes a base mesh from the encoded data and the decoded atlas information, a mesh displacement decoding unit that decodes mesh displacement from the encoded data and the decoded atlas information, and a mesh reconstruction unit that decodes a mesh from the decoded base mesh and mesh displacement, wherein the mesh displacement decoding unit decodes the mesh displacement from the encoded data using the atlas information decoded by the atlas information decoding unit.

2. the mesh displacement decoding unit includes a video decoding unit that decodes a mesh displacement image decoded from a geometry video stream whose Unit Type of the encoded data is V3C_GVD, and a displacement undapping unit that derives mesh displacements dispQuantCoeffArray for each vertex v, component d, and submeshIdx from a mesh displacement image dispQuantCoeffFrame having positions x, y and components d and submeshIdx; The 3D data decoding device described in claim 1, characterized in that the displacement unmapping unit derives the mesh displacement by deriving the coordinates of the geometry image from the height of each component of the mesh displacement or the width of each component of the mesh displacement, a variable d with a value -1 indicating the number of dimensions of the geometry, DisplacementDim, and a variable submeshIdx with a value -1 indicating the number of submesh divisions, NumSubMeshes, of the mesh displacement.

3. The 3D data decoding device of claim 2, characterized in that the atlas information includes a flag indicating whether or not the mesh displacement is to be divided into sub-meshes, and the atlas information decoding unit decodes a value indicating that the mesh displacement is to be divided into sub-meshes if the mesh displacement is to be divided into sub-meshes, and decodes a value indicating that the mesh displacement is not to be divided into sub-meshes if the mesh displacement is not to be divided into sub-meshes.

4. The 3D data decoding device according to claim 2 or 3, characterized in that the atlas information includes parameters indicating the height of each component for each submesh of the mesh displacement divided into sub-meshes, the atlas information decoding unit decodes the parameters indicating the height of each component for each submesh of the submesh-divided mesh displacement depending on the value of a flag indicating whether the mesh displacement is divided into sub-meshes (always if there is no value), and the displacement umapping unit derives coordinates of a geometry image of the mesh displacement from the height of each component for each submesh of the decoded sub-mesh-divided mesh displacement.

5. The 3D data decoding device according to claim 2 or 3, characterized in that the atlas information includes parameters indicating the width of each component for each submesh of the mesh displacement divided into sub-meshes, the atlas information decoding unit decodes the parameters indicating the width of each component for each submesh of the submesh-divided mesh displacement depending on the value of a flag indicating whether the mesh displacement is to be divided into sub-meshes (always if there is no value), and the displacement umapping unit derives coordinates of a geometry image of the mesh displacement from the width of each component for each submesh of the decoded sub-mesh-divided mesh displacement.

6. A 3D data encoding device that encodes mesh data or point cloud data comprises an atlas information encoding unit that encodes atlas information, a base mesh encoding unit that encodes a base mesh using the atlas information, and a mesh displacement encoding unit that encodes mesh displacement using the atlas information, wherein the mesh displacement encoding unit encodes the mesh displacement using the atlas information encoded by the atlas information encoding unit.

7. The mesh displacement encoding unit is composed of a displacement mapping unit that derives a mesh displacement image dispQuantCoeffFrame with positions x, y and components d and submeshIdx from a mesh displacement dispQuantCoeffArray with vertex v, components d and submeshIdx, and a geographic unit whose Unit Type is V3C_GVD.

7. The 3D data encoding device of claim 6, further comprising an image encoding unit that encodes a mesh displacement having a vertex v, a component d, and a submeshIdx into a geometry video stream, wherein the displacement mapping unit maps the mesh displacement having a vertex v, a component d, and a submeshIdx to coordinates x, y of the geometry image using the height of each component of the mesh displacement or the width of each component of the mesh displacement.

8. The 3D data encoding device of claim 7, characterized in that the atlas information includes a flag indicating whether the mesh displacement is to be divided into sub-meshes, and the atlas information encoding unit encodes a value indicating that the mesh displacement is to be divided into sub-meshes if the mesh displacement is to be divided into sub-meshes, and encodes a value indicating that the mesh displacement is not to be divided into sub-meshes if the mesh displacement is not to be divided into sub-meshes.

9. The 3D data encoding device described in claim 7 or claim 8, characterized in that the atlas information includes parameters indicating the height of each component for each sub-mesh of the mesh displacement divided into sub-meshes, and the atlas information encoding unit encodes the parameters indicating the height of each component for each sub-mesh of the mesh displacement divided into sub-meshes depending on the value of a flag indicating whether the mesh displacement is to be divided into sub-meshes (always if there is no value).

10. The 3D data encoding device described in claim 7 or claim 8, characterized in that the atlas information includes parameters indicating the width of each component for each submesh of the mesh displacement divided into submeshes, and the atlas information encoding unit encodes the parameters indicating the width of each component for each submesh of the mesh displacement divided into submeshes depending on the value of a flag indicating whether the mesh displacement is to be divided into submeshes (always if there is no value).

Images