3D decoder and 3D encoder

Abstract

To highly efficiently encode / decode 3D information by clarifying correspondence relation between tile information and sub-mesh information of an atlas stream and performing mesh separation / mesh reconfiguration of an arbitrary sub-mesh in encoding / decoding the 3D information using a moving image encoding system.SOLUTION: The 3D decoding device according to claim 1, further comprising: an atlas information decoding unit configured to decode atlas information from encoded information obtained by encoding the mesh information or the point cloud information; and a mesh reconstruction unit configured to decode a mesh from the encoded information and the atlas information, wherein the mesh reconstruction unit decodes an arbitrary mesh / sub-mesh from the encoded information using a parameter indicating a correspondence relationship between tile information having an arbitrary tile ID and sub-mesh information having an arbitrary sub-mesh ID, the tile information and the sub-mesh information being decoded by the atlas information decoding unit. 3D.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a 3D data encoding device and a 3D data decoding device. [Background technology]

[0002] To efficiently transmit or record 3D data, the 3D data is converted into 2D images and video There are 3D data encoding devices that encode data using an image encoding method to generate encoded data, and 3D data decoding devices that decode the encoded data into a 2D image and reconstruct 3D data.

[0003] Specific examples of 3D data encoding methods include MPEG-I's ISO / IEC 23090-5 V3C (Volumetric Video-based Coding) and V-PCC (Video-based Point Cloud Compression). V3C encodes and decodes point clouds that consist of point positions and attribute information. Furthermore, ISO / IEC 23090-12 (MPEG Immersive Video, MIV) and ISO / IEC 23090-29 (Video-based Dynamic Mesh Coding, V-DMC), which are currently being standardized, are enabling the encoding of multi-viewpoint video, It is also used for encoding and decoding mesh images. A soft document has been disclosed.

[0004] In these 3D data coding methods, the geometry and attributes that make up the 3D data are encoded and decoded as images using video coding methods such as H.265 / HEVC (High Efficiency Video Coding) and H.266 / VVC (Versatile Video Coding).

[0005] In the case of a point cloud, the geometry image is the depth to the projection plane, and the attribute image is the image of the attributes projected onto the projection plane.

[0006] 3D data (mesh) as in Non-Patent Document 1 consists of a base mesh, mesh displacement, and texture mapping image. The encoding of the base mesh is done using vertex coding such as Draco. The mesh displacement is encoded as a two-dimensional mesh. In addition to encoding the displacement image using a video codec, there is also a method of encoding it directly using arithmetic coding. The texture mapping image is encoded as an attribute image using a video codec. The video codec can be the above-mentioned HEVC or VVC. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] Text of ISO / IEC CD 23090-29 Video-based mesh coding, ISO / IEC JTC 1 / SC 29 / WG 7 N0885, April 2024 Summary of the Invention [Problem to be solved by the invention]

[0008] In the 3D data encoding method described in Non-Patent Document 1, atlas frames that make up 3D data (meshes) can be encoded and decoded in tile units. Base meshes that make up 3D data (meshes) are encoded and decoded in sub-mesh units, but the correspondence between tile information and sub-mesh information in the atlas stream is unclear, which poses a problem: it is not possible to encode and decode only specific sub-meshes (mesh separation and mesh reconstruction).

[0009] The purpose of this invention is to efficiently encode and decode 3D data using a video coding method by clarifying the correspondence between tile information in the atlas stream and sub-mesh information, and by performing mesh separation and mesh reconstruction of any sub-mesh. [Means for solving the problem]

[0010] In order to solve the above problem, a 3D data decoding device according to one embodiment of the present invention is a 3D data decoding device that decodes mesh data or point cloud data, and is equipped with an atlas information decoding unit that decodes atlas information from encoded data in which the mesh data or point cloud data is encoded, and a mesh reconstruction unit that decodes a mesh from the encoded data and the atlas information, and is characterized in that the mesh reconstruction unit decodes an arbitrary mesh / submesh from the encoded data using parameters that indicate the correspondence between tile information having an arbitrary tile ID decoded by the atlas information decoding unit and submesh information having an arbitrary submesh ID.

[0011] In order to solve the above problem, a 3D data encoding device according to one embodiment of the present invention is a 3D data encoding device that encodes mesh data or point cloud data, and is equipped with a mesh separation unit that separates meshes and an atlas information encoding unit that encodes atlas information, wherein the atlas information includes parameters that indicate the correspondence between tile information having an arbitrary tile ID and submesh information having an arbitrary submesh ID, and the mesh separation unit encodes the mesh / submesh using the parameters. [Effects of the Invention]

[0012] According to one aspect of the present invention, the encoding efficiency of mesh displacement can be improved, and 3D data can be encoded and decoded with high quality. [Brief explanation of the drawings]

[0013] [Figure 1]1 is a schematic diagram showing the configuration of a 3D data transmission system according to the present embodiment. [Figure 2] FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. [Figure 3] FIG. 2 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31. [Figure 4] FIG. 2 is a functional block diagram showing the configuration of an atlas information decoding unit 302. [Figure 5] FIG. 2 is a functional block diagram showing the configuration of a base mesh decoding unit 303. [Figure 6] FIG. 10 is a functional block diagram showing the configuration of a mesh displacement decoding unit 305. [Figure 7] FIG. 2 is a functional block diagram showing the configuration of a mesh reconstruction unit 307. [Figure 8] 10 is an example of a syntax for transmitting coordinate transformation parameters at the sequence level (ASPS). [Figure 9] 10 is an example of a syntax for transmitting coordinate transformation parameters of mesh data at the sequence level (ASPS). [Figure 10] 10 is an example of a syntax for transmitting coordinate transformation parameters at the picture / frame level (AFPS). [Figure 11] This is an example of the syntax for transmitting frame tile information at the picture / frame level (AFPS). [Figure 12] 10 is an example of a syntax for transmitting coordinate transformation parameters of mesh data at the picture / frame level (AFPS). [Figure 13] This is an example of the syntax for transmitting mesh and submesh information at the picture / frame level (AFPS). [Figure 14] 10 is an example of a syntax for transmitting mesh patch information at the picture / frame level (AFPS). [Figure 15] FIG. 10 is a diagram for explaining the operation of the mesh reconstruction unit 307. [Figure 16] 1 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11. FIG. [Figure 17] FIG. 2 is a functional block diagram showing the configuration of an atlas information encoding unit 101. [Figure 18] FIG. 2 is a functional block diagram showing the configuration of a base mesh encoding unit 103. [Figure 19] FIG. 2 is a functional block diagram showing the configuration of a mesh displacement encoding unit 107. [Figure 20] FIG. 2 is a functional block diagram showing the configuration of a mesh separation unit 115. [Figure 21] 10 is a diagram for explaining the operation of the mesh separating unit 115. FIG. [Figure 22] This is an example of the syntax for transmitting mesh and submesh information at the picture / frame level (AFPS). [Figure 23] This is an example of the syntax for transmitting mesh and submesh information at the picture / frame level (AFPS). [Figure 24] This is an example of the syntax for transmitting mesh and submesh information at the picture / frame level (AFPS). [Figure 25] 10 is an example of a syntax for transmitting tile-submesh mapping information in SEI. [Figure 26] 10 is an example of a syntax for transmitting tile-submesh mapping information in SEI. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] FIG. 1 is a schematic diagram showing the configuration of a 3D data transmission system 1 according to this embodiment.

[0016] The 3D data transmission system 1 is a system that transmits an encoded stream obtained by encoding 3D data to be encoded, decodes the transmitted encoded stream, and displays the 3D data. The 3D data transmission system 1 includes a 3D data encoding device 11, a network 21, a 3D data decoding device 31, and a 3D data display device 41.

[0017] The 3D data T is input to the 3D data encoding device 11.

[0018] The network 21 transmits the coded stream Te generated by the 3D data coding device 11 to the 3D data decoding device 31. The network 21 may be the Internet, a wide area network (WAN), a local area network (LAN), or The network 21 is not necessarily limited to a two-way communication network, but may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may also be a network that transmits DVD (Digital Versatile Disc: registered trademark), BD (Blu-ray Disc: registered trademark), etc. It may be replaced by a storage medium on which the coded stream Te, such as a standard (or a DVD-ROM drive), is recorded.

[0019] The 3D data decoding device 31 decodes each of the coded streams Te transmitted by the network 21, and generates one or more decoded 3D data Td.

[0020] The 3D data display device 41 displays all or part of one or more pieces of decoded 3D data Td generated by the 3D data decoding device 31. The 3D data display device 41 is equipped with a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. The display may be in the form of a stationary display, a mobile display, an HMD, or the like. When the device has a high processing power, it displays a high quality image, and when the device has a lower processing power, it displays an image that does not require a high processing power or display power.

[0021] <operator> The operators used in this specification are listed below.

[0022] >> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR , |= is the OR assignment operator, and || indicates logical sum.

[0023] x?y:z is a ternary operator that takes y if x is true (non-zero) and z if x is false (zero). y..z denotes the set of integers from y to z.

[0024] Log2(x) is the logarithm to the base 2.

[0025] Ceil(x) is the smallest integer greater than or equal to x.

[0026] <Structure of the coded stream Te> Before proceeding to a detailed description of the 3D data encoding device 11 and the 3D data decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the 3D data encoding device 11 and decoded by the 3D data decoding device 31 will be described.

[0027] FIG. 2 shows the hierarchical structure of data in the coded stream Te. The frame Te has either a V3C sample stream or a V3C unit stream data structure. A V3C sample stream includes a sample stream header and a V3C unit. A V3C unit stream includes a V3C unit.

[0028] A V3C unit includes a V3C unit header and a V3C unit payload. The V3C unit header is a Unit Type, which is an ID that indicates the type of V3C unit, and can be V3C_VPS, V3C_AD, V3C_AVD, V3C_GVD, V3C_OVD, etc. It takes the value indicated by the label.

[0029] If the Unit Type is V3C_VPS (Video Parameter Set), the V3C unit contains a V3C parameter set.

[0030] If the Unit Type is V3C_AD (Atlas Data), the V3C unit contains a VPS ID, an atlasID, a sample stream nal header, and multiple NAL units. The atlasID is an ID (Identification). Yes, takes an integer value of 0 or greater.

[0031] A NAL unit includes a NALUnitType, a layerID, a TemporalID, and an RBSP (Raw byte sequence payload).

[0032] NAL units are identified by NALUnitType, and are classified as ASPS (Atlas Sequence Parameter Set), AAPS (Atlas Adaptation Parameter Set), ATL (Atlas Tile layer), SEI (Supplemental Enhancement Information), etc.

[0033] The ATL includes an ATL header and an ATL data unit, and the ATL data unit includes information such as the patch position and size, such as patch information data.

[0034] The SEI includes payloadType, which indicates the type of the SEI, payloadSize, which indicates the size (number of bytes) of the SEI, and sei_payload, which is the data of the SEI.

[0035] If the Unit Type is V3C_AVD (Attribute Video Data), The unit includes the VPS ID, atlasID, attribute image ID attrIdx, partition ID partIdx, map ID mapIdx, flag auxFlag indicating whether it is auxiliary data or not, and video stream. The video stream is data encoded using HEVC, VVC, etc. Attribute data corresponds to a texture image in V-DMC.

[0036] If NalUnitType is V3C_GVD (Geometry Video Data), the V3C unit contains VPS ID, atlasID, mapIdx, auxFlag, and video stream. Geometry data corresponds to mesh displacement in V-DMC.

[0037] If Unit Type is V3C_OVD (Occupancy Video Data), V3C u nit includes VPS ID, atlasID, and video stream.

[0038] If the Unit Type is V3C_MD (Mesh data), the V3C unit contains the VPS ID, atlas ID, and mesh_payload. In V-DMC, it corresponds to the base mesh.

[0039] (Configuration of 3D data decoding device according to the first embodiment) 3 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31 according to the first embodiment. The 3D data decoding device 31 includes a demultiplexing unit 301, a submesh information decoding unit 3024, an atlas information decoding unit 3026, a demultiplexing unit 3028, a demultiplexing unit 3029, a demultiplexing unit 3030, a demultiplexing unit 3031, a demultiplexing unit 3032, a demultiplexing unit 3033, a demultiplexing unit 3034, a demultiplexing unit 3035, a demultiplexing unit 3036, a demultiplexing unit 3037, a demultiplexing unit 3038, a demultiplexing unit 3039 ... The 3D data decoding device 31 comprises an information decoding unit 302, a base mesh decoding unit 303, a mesh displacement decoding unit 305, a mesh reconstruction unit 307, an attribute decoding unit 306, and a color space conversion unit 308. The 3D data decoding device 31 inputs coded 3D data and outputs atlas information, meshes, and attribute images.

[0040] The demultiplexing unit 301 receives encoded data multiplexed in a byte stream format, ISOBMFF (ISO Base Media File Format), or the like, and demultiplexes it to generate an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a base mesh encoded stream (V3C_MD mesh_payload), a mesh displacement encoded stream (V3C_GVD video stream), an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a mesh displacement encoded stream (V3C_MD video stream), an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a mesh displacement encoded stream (V3C_MD video stream), an atlas information encoded stream (V3C_MD ... Outputs a live video stream (V3C_AVD video stream).

[0041] The atlas information decoding unit 302 receives the atlas information coded stream output from the demultiplexing unit 301 and decodes the atlas information.

[0042] 3 decodes coordinate system transformation information displacementCoordinateSystem (mdu_displacement_coordinate_system) indicating the coordinate system from the encoded data. Note that a separate gating flag may be provided, and each piece of coordinate system transformation information may be decoded only when the gating flag is 1. The gating flag may be, for example, mdu_displacement_coordinate_system_enable_flag.

[0043] The base mesh decoding unit 303 decodes the base mesh coded stream coded by vertex coding (3D data compression coding method, for example, Draco), and outputs a base mesh. The base mesh will be described later.

[0044] The mesh displacement decoding unit 305 decodes a geometry video stream (mesh displacement coded stream) coded using VVC, HEVC, or the like, and outputs mesh displacement. The type of codec (video codec) used for coding is determined by decoding the V3C parameter set of the coded data. It can also be indicated by the Four CC code (four-character code, 4CC code) indicated by gi_geometry_codec_id[atlasID] in the V3C parameter set. gi_geometry_codec_id[atlasID] indicates the decoding of the geometry video stream in the atlas ID. The set indicating the correspondence between the codec ID (ccm_codec_id) and its 4CC code (ccm_codec_4cc[ccm_codec_id]) may be transmitted in a separate codec mapping SEI (component_codec_mapping SEI). The codec may decode mesh displacement in segments (slices) that are further divided into frames. HEVC and VVC can divide frames into slices. Rice is coded in units of Coded Tree Units (CTUs), not slices. Subpictures or tile divisions may be used as segments. These subpictures, tiles, and slices can be decoded independently, making it possible to decode only a portion of a frame without decoding the entire frame. When subpictures or tiles are used, slices are replaced with subpictures or tiles.

[0045] The mesh reconstruction unit 307 receives the base mesh and the mesh displacement and reconstructs the mesh in 3D space. Reconstruct the cache.

[0046] The attribute decoding unit 306 decodes an attribute video stream coded by VVC, HEVC, or the like, and outputs an attribute image. The attribute image is a texture image expanded on the UV axis (a texture mapping image converted by the UV atlas method) in YCbCr frame. The type of codec used for encoding is indicated by ptl_profile_codec_group_idc, which is obtained by decoding the V3C parameter set of the encoded data. It may also be indicated by the Four CC code indicated by ai_geometry_codec_id[atlasID] in the V3C parameter set. ai_geometry_codec_id[atlasID] indicates the index corresponding to the codec ID of the decoder used to decode the attribute video stream in the atlas ID.

[0047] A color space conversion unit 308 converts the attribute image from the YCbCr format to the RGB format. Color space conversion is performed. Note that the attribute video stream is encoded as RGB format. It is also possible to decode the image data and omit the color space conversion.

[0048] (Decoding atlas information) 4 is a functional block diagram showing the configuration of the atlas information decoding unit 302. The atlas information decoding unit 302 includes a parameter decoding unit 3021, a tile information decoding unit 3022, an extended information decoding unit 3023, a submemory decoding unit 3024, and a tile information decoding unit 3025. The image processing unit 3020 is configured of a mesh patch information decoding unit 3024 and a mesh patch information decoding unit 3025.

[0049] (Decoding and derivation of coding parameters) The parameter decoding unit 3021 decodes coding parameters from the atlas information coded stream. The coding parameters include an ASPS (Atlas Sequence Parameter Set), which is a sequence-level parameter set, and an AFPS (Atlas Frame Parameter Set), which is a picture / frame-level parameter set.

[0050] Figure 8 shows an example of the syntax of the ASPS (Atlas Sequence Parameter Set), which is a sequence-level parameter set. ASPS is one of the NAL units of atlas information. Contains syntax elements that apply to the data stream. The semantics of each field are as follows:

[0051] asps_geometry_3d_bit_depth_minus1: Add 1 to asps_geometry_3d_bit_depth_minus1 The returned value indicates the bit depth of the geometric coordinates of the reconstructed volume content.

[0052] asps_geometry_2d_bit_depth_minus1: Add 1 to asps_geometry_2d_bit_depth_minus1 The value indicates the bit depth of the geometry when projected onto a 2D image.

[0053] Figure 9 shows an example of the syntax of ASVE (ASPS Vdmc Extension), which is a sequence-level mesh data extension encoding parameter set. The semantics of each field are as follows: As stated above.

[0054] asve_subdivision_iteration_count: Indicates the number of mesh division iterations.

[0055] asve_1d_displacement_flag: A flag indicating whether the mesh displacement is one-dimensional. If the value is true, it indicates that the mesh displacement is one-dimensional. If the value is false, it indicates that the mesh displacement is three-dimensional.

[0056] Figure 10 shows an example of the syntax of AFPS (Atlas Frame Parameter Set), which is a picture / frame level parameter set. AFPS is one of the NAL units of atlas information. Contains syntax elements that apply to the atlas information coded stream. The mechanics are as follows: AFPS includes atlas_frame_tile_information() and atlas_frame_mesh_information().

[0057] afps_atlas_frame_parameter_set_id: The atlas referenced by other syntax elements. Identifies the frame parameter set AFPS.

[0058] afps_atlas_sequence_parameter_set_id: Specifies the value of asps_atlas_sequence_parameter_set_id of the active atlas sequence parameter set ASPS.

[0059] (Decoding and derivation of tile-level coding parameters) The tile-level coding parameters decoded from the coded data by the tile information decoding unit 3022 will be described.

[0060] Figure 11 shows an example of the syntax of tile information in AFPS, a picture / frame-level parameter set. The semantics of each field are as follows:

[0061] afti_single_tile_in_atlas_frame_flag: Flag indicating whether there is exactly one tile in each atlas frame that references the atlas frame parameter set AFPS. A value of true indicates that there is exactly one tile in each atlas frame that references AFPS. Value If false, each atlas frame that references AFPS has multiple tiles (greater than 1). Indicates existence.

[0062] afti_single_partition_per_tile_flag: See Atlas Frame Parameter Set AFPS A flag indicating whether each tile referencing the AFPS contains only one tile partition. A value of true indicates that each tile referencing the AFPS contains only one tile partition; a value of false indicates that each tile referencing the AFPS contains multiple tile partitions ( If not present, the value of afti_single_partition_per_tile_flag is inferred to be equal to 1.

[0063] afti_num_tiles_in_atlas_frame_minus1: Specifies the number of tiles in each atlas frame that references the atlas frame parameter set AFPS. The value of afti_num_tiles_in_atlas_frame_minus1 must be in the range from 0 to NumPartitionsInAtlasFrame-1. If not present and afti_single_partition_per_tile_flag is equal to 1, the value of afti_num_tiles_in_atlas_frame_minus1 is inferred to be equal to NumPartitionsInAtlasFrame-1.

[0064] afti_signalled_tile_id_flag: Indicates whether the tile ID of each tile is signaled. If the flag is set to 1, the tile ID of each tile is signaled. If the flag is set to 0, the tile ID is not signaled.

[0065] afti_signalled_tile_id_length_minus1:afti_signalled_tile_id_length_minus1+1 is the length of the syntax element afti_tile_id[i] (if present) and the syntax Specifies the number of bits used to represent the afti_signalled_tile_id element afti_signalled_tile_id_length_minus1. The value of afti_signalled_tile_id_length_minus1 must be in the range 0 to 15.

[0066] afti_tile_id[i]: Specifies the tile ID of the ith tile. If not present, the value of afti_tile_id[i] is inferred to be equal to i for each i in the range 0 to afti_num_tiles_in_atlas_frame_minus1. afti_tile_id[i] is inferred to be equal to afti_tile_id[j] for all i != j. The requirement for bitstream conformance is that they are not equal (there should be no cases where they are equal). The 3D data decoding device 31 decodes bitstreams that satisfy the conformance requirement (and so on).

[0067] When decoding and encoding afti_single_tile_in_atlas_frame_flag and afti_single_partition_per_tile_flag, the tile information decoding unit 3022 decodes a syntax element afti_num_tiles_in_atlas_frame_minus2 indicating the number of tiles minus 2 (the value obtained by subtracting 2 from the number of tiles). Alternatively, if the value of afti_single_tile_in_atlas_frame_flag is false or Only if the value of afti_single_partition_per_tile_flag is false, decode and encode the syntax element afti_num_tiles_in_atlas_frame_minus2, which indicates the number of tiles referenced minus 2. Semantics may use the following examples:

[0068] afti_num_tiles_in_atlas_frame_minus2: Specifies the number of tiles in each atlas frame that references the atlas frame parameter set AFPS. The value of afti_num_tiles_in_atlas_frame_minus1 must be in the range from 0 to NumPartitionsInAtlasFrame-2. If not present and afti_single_partition_per_tile_flag is equal to 1, the value of afti_num_tiles_in_atlas_frame_minus2 is inferred to be equal to NumPartitionsInAtlasFrame-2.

[0069] In this configuration, the case where one tile is referenced is indicated by afti_single_tile_in_atlas_frame_flag Therefore, the syntax element indicating the number of tiles minus 2 is decoded and coded. By encoding, the overhead of the amount of coding can be reduced.

[0070] (Decoding and derivation of extended coding parameters) The extended coding parameters decoded from the coded data by the extended information decoding unit 3023 will be described.

[0071] FIG. 12 shows an example of the syntax of mesh information in AFPS, which is a picture / frame level parameter set.

[0072] afve_overriden_flag: A flag indicating whether or not to update the coordinate system of the mesh displacement. If this flag is true, the coordinate system of the mesh displacement is updated based on the value of mdu_displacement_coordinate_system described below. If this flag is false, the coordinate of the mesh displacement is updated. The system is not updated.

[0073] afve_subdivision_iteration_count: Indicates the number of mesh division iterations.

[0074] (Mesh-level coding parameter decoding and derivation) The mesh-level coding parameters decoded from the coded data by the sub-mesh information decoding unit 3024 will be described.

[0075] Figure 13 shows an example of the syntax structure of the submesh information atlas_frame_mesh_information() transmitted by AFPS. In the example syntax structure of Figure 13, the number of submesh IDs is coded and decoded regardless of the number of submeshes referenced. atlas_frame_mesh_information() may contain any of the following syntax elements. The semantics of each field are as follows:

[0076] afmi_use_single_mesh_flag: In each atlas frame that references AFPS, a mesh A flag indicating whether the patch references a single submesh or not (greater than 1). A value of true indicates that there is only one referenced submesh. A value of false indicates that there are multiple referenced submeshes (greater than 1).

[0077] afmi_num_submeshes_minus2: The number of submeshes referenced by a mesh patch. If the value of afmi_use_single_mesh_flag is true, the sub-mesh The number is 1. If the value of afmi_use_single_mesh_flag is false, the number of submeshes is afmi_num_submeshes_minus2+2.

[0078] afmi_signalled_submesh_id_flag: Flag indicating whether the submesh ID referenced by the mesh patch is signaled. If the value is true, it indicates that the submesh ID is signaled. If the value is false, it indicates that the submesh ID is not signaled. show.

[0079] afmi_signalled_submesh_id_length_minus1: The value obtained by adding 1 to afmi_signalled_submesh_id_length_minus1 is the value of the syntax element mdu_submesh_id[ tileID ][ patchIdx ] and the syntax element afmi_submesh_id[ i Specifies the number of bits used to represent ] in the current atlas tile whose tile ID is equal to tileID. The value of afmi_signalled_submesh_id_length_minus1 must be in the range 0 to 15. If not present, its value is inferred to be equal to Ceil( Log2( NumSubMeshes ) ) - 1.

[0080] afmi_submesh_id[i]: a parameter that indicates the submesh ID of the i-th submesh If the value of afmi_signalled_submesh_id_flag is false, i.e., afmi_submesh_id[i] is If not present, the value of afmi_submesh_id[i] is inferred to be equal to i, for each i in the range 0 to NumSubMeshes-1. It is a bitstream conformance requirement that afmi_submesh_id[i] is not equal to afmi_submesh_id[j] for all i != j. The variable FirstSubmeshID is derived as follows:

[0081] FirstSubmeshID = afmi_submesh_id[0] for ( i = 1; i < NumSubMeshes; i++ ) FirstSubmeshID = Min( FirstSubmeshID, afmi_submesh_id[ i ] ) The atlas information decoding unit 302 (submesh information decoding unit 3024) decodes the AFPS code of the atlas information. Decode frame mesh information from the encoded data. For example, afmi_use_single_mesh_flag , afmi_num_submeshes_minus2, afmi_signalled_submesh_id_flag, afmi_signalled_submesh_id_length_minus1, and afmi_submesh_id. The atlas information encoding unit 101 (submesh information encoding unit 1012) encodes frame mesh information into AFPS encoded data of the atlas information.

[0082] When afmi_signalled_submesh_id_flag is true, the submesh information decoding unit 3024 decodes afmi_submesh_id[i] in the range of i = 0.. NumSubMeshes-1 for the number of submeshes NumSubMeshes, and derives the arrays SubMeshIDToIndex and SubMeshIndextoID for i = 0.. NumSubMeshes-1 as follows:

[0083] SubMeshIDToIndex[ afmi_submesh_id[ i ] ] = i SubMeshIndextoID[ i ] = afmi_submesh_id[ i ] In the syntax configuration of Figure 13, afmi_signalled_submesh_id_flag, if (afmi_signalled_submesh_id_flag), and the brackets {} following it may be configured to exist only in the case of if (!afmi_use_single_mesh_flag). Alternatively, the above sections may be configured to exist when the number of submeshes, NumSubmeshes, is greater than 1. That is, afmi_signalled_submesh_id_flag if ( afmi_signalled_submesh_id_flag || NumSubmeshes > 1 ) { … } else { … } Instead of if ( !afmi_use_single_mesh_flag ) { afmi_signalled_submesh_id_flag if ( afmi_signalled_submesh_id_flag ) { … } else { … } } It may also be possible to use the following.

[0084] In this case, if afmi_use_single_mesh_flag is false (or the number of submeshes NumSubmeshes is greater than 1) and afmi_signalled_submesh_id_flag is true, the submesh information decoding unit 3024 decodes afmi_signalled_submesh_id_length_minus1 and afmi_submesh_id[ i ] in the range of i = 0..NumSubMeshes-1 for the number of submeshes minus 1 (NumSubMeshes-1). Also, if afmi_use_single_mesh_flag is true (or if the sub-mesh If the number of submeshes is 1, then SubMeshIDToIndex[0] = 0, SubMeshIndexToID[0] = 0, The ID may always be set to 0.

[0085] In addition, in the syntax configuration of Figure 13, if the value of afmi_use_single_mesh_flag is true, In this case, the value of afmi_signalled_submesh_id_flag may always be set to false, and the ID may always be set to 0, i.e., SubMeshIDToIndex[0] = 0 and SubMeshIndexToID[0] = 0.

[0086] The submesh information decoding unit 3024 does not decode afmi_submesh_id[i] when afmi_signalled_submesh_id_flag is true, and derives the array as follows for i = 0..NumSubMeshes-1: do.

[0087] SubMeshIDToIndex[ i ] = i SubMeshIndextoID[ i ] = i (Operation of mesh patch information decoding unit) Here, the patch level coding parameters decoded from the coded data by the patch information decoding unit 3025 will be described.

[0088] Figure 14 shows an example of the syntax of mesh patch information in AFPS, a picture / frame level parameter set. The semantics of each field are as follows:

[0089] mdu_submesh_id[ tileID ][ patchIdx ]: indicates the submesh ID associated with the current mesh patch with index patchIdx in the current atlas tile whose tile ID is equal to tileID. The value of mdu_submesh_id[ tileID ][ patchIdx ] is one of afmi_submesh_id[ i ]. It must be.

[0090] mdu_displ_id[ tileID ][ patchIdx ]: The current attribute with a tile ID equal to tileID mdu_disp_id[tileID][patchIdx] indicates the mesh displacement ID associated with the current mesh patch with index patchIdx in the tile. The value of mdu_disp_id[tileID][patchIdx] ranges from 0 to 65535.

[0091] mdu_face_count_minus1[ tileID ][ patchIdx ]: Atlas with tile ID equal to tileID Specifies the number of faces associated with the current mesh patch with index patchIdx in the tile.

[0092] mdu_2d_pos_x[ tileID ][ patchIdx ]: Specifies the x-coordinate of the upper-left corner of the bounding box of the current mesh patch with index patchIdx in the current atlas tile whose tile ID is equal to tileID, expressed as a multiple of PatchPackingBlockSize.

[0093] mdu_2d_pos_y[ tileID ][ patchIdx ]: Specifies the y coordinate of the upper-left corner of the bounding box of the current mesh patch with index patchIdx in the current atlas tile whose tile ID is equal to tileID, expressed as a multiple of PatchPackingBlockSize.

[0094] mdu_2d_size_x_minus1[ tileID ][ patchIdx ]: The value of mdu_2d_size_x_minus1[ tileID ][ patchIdx ] plus 1 is the index of the tile in the current atlas whose tile ID is equal to tileID. Specifies the width value of the bounding box of the current mesh patch for the mesh patch with the given patchIdx.

[0095] mdu_2d_size_y_minus1[ tileID ][ patchIdx ]: The value of mdu_2d_size_y_minus1[ tileID ][ patchIdx ] plus 1 is the index of the tile in the current atlas whose tile ID is equal to tileID. Specifies the height value of the bounding box of the current mesh patch for the mesh patch with the given patchIdx.

[0096] mdu_displacement_coordinate_system[ tileID ][ patchIdx ]: Specifies the identifier of the coordinate system of the submesh (part of a mesh) associated with the mesh patch with index patchIdx in the current atlas tile whose tile ID is equal to tileID.

[0097] The mesh patch information decoding unit 3025 performs mesh reconstruction processing in the mesh reconstruction unit 307. can be done using the tile information directly, we may decode the tile-level mesh patch information as follows:

[0098] for( t = 0; t<= afti_num_tiles_in_atlas_frame_minus1; t++ ) { tileID = TileIndexToID( t ) AtduTotalNumMeshpatches[ tileID ] = MaxNumMeshpatches for( p = 0; p <= AtduTotalNumMeshpatches[ tileID ]; p++ ) { TileMeshpatch2dPosX[ tileID ][ p ] = mdu_2d_pos_x[ tileID ][ p ] * PatchPackingBlockSize TileMeshpatch2dPosY[ tileID ][ p ] = mdu_2d_pos_y[ tileID ][ p ] * PatchPackingBlockSize TileMeshpatch2dSizeX[ tileID ][ p ] = (mdu_2d_size_x_minus1[ tileID ][ p ] + 1) * PatchSizeXQuantizer TileMeshpatch2dSizeY[ tileID ][ p ] = (mdu_2d_size_y_minus1[ tileID ][ p ] + 1) * PatchSizeYQuantizer for( attrIdx = 0; attrIdx < asve_num_attribute_video; attrIdx){ TileMeshpatchAttributes2dPosX[ tileID ][ p ][ attrIdx ] = mdu_attributes_2d_pos_x[ tileID ][ p ][ attrIdx ] TileMeshpatchAttributes2dPosY[ tileID ][ p ][ attrIdx ] = mdu_attributes_2d_pos_y[ tileID ][ p ][ attrIdx ] TileMeshpatchAttributes2dSizeX[ tileID ][ p ][ attrIdx ] = (mdu_attributes_2d_size_x_minus1[ tileID ][ p ][ attrIdx ] + 1) * PatchSizeXQuantizer TileMeshpatchAttributes2dSizeY[ tileID ][ p ][ attrIdx ] = (mdu_attributes_2d_size_y_minus1[ tileID ][ p ][ attrIdx ] + 1) * PatchSizeYQuantizer } TileMeshpatchSubmeshID[ tileID ][ p ] = mdu_submesh_id[ tileID ][ p ] TileMeshpatchDisplID[ tileID ][ p ] = mdu_displ_id[ tileID ][ p ] TileMeshpatchSubdivCount[ tileID ][ p ] = PatchSubdivisionCount[ tileID ][ patchIdx ] for( i = 0; i < TileMeshpatchSubdivCount[ tileID ][ p ] ; i++ ){ TileMeshpatchSubdivMethod[ tileID ][ p ][ i ] = PatchSubdivisionMethod[ tileID ][ p ][ i ] } TileMeshpatchDispCoordSys[ tileID ][ p ] = mdu_displacement_coordinate_system[ tileID ][ p ] TileMeshpatchTransformMethod[ tileID ][ p ] = mdu_transform_method[ tileID ][ p ] ... } } Here, the mesh reconstruction process in the mesh reconstruction unit 307 uses all tile information. When the mesh patch information is converted to atlas-level mesh patch information as follows:

[0099] AtlasTotalNumMeshpatches = 0 atlasPatchIdx = 0 for( t = 0; t <= afti_num_tiles_in_atlas_frame_minus1; t++ ) { tileID = TileIndexToID[ t ] tileOffsetX = TileOffsetX[ t ] tileOffsetY = TileOffsetY[ t ] for( p = 0; p < AtduTotalNumMeshpatches[ tileID ]; p++ ) { AtlasMeshpatch2dSizeX[ atlasPatchIdx ] = TileMeshpatch2dSizeX[ tileID ][ p ] AtlasMeshpatch2dSizeY[ atlasPatchIdx ] = TileMeshpatch2dSizeY[ tileID ][ p ] AtlasMeshpatch2dPosX[ atlasPatchIdx ] = TileMeshpatch2dPosX[ tileID ][ p ] + tileOffsetX AtlasMeshpatch2dPosY[ atlasPatchIdx ] = TileMeshpatch2dPosY[ tileID ][ p ] + tileOffsetY AtlasMeshpatchSubmeshID[ atlasPatchIdx ] = TileMeshpatchSubmeshID[ tileID ][ p ] AtlasMeshpatchDisplID[ atlasPatchIdx ] = TileMeshpatchDisplID[ tileID ][ p ] AtlasMeshpatchVertexCount[ atlasPatchIdx ] = TileMeshpatchVertexCount[ tileID ][ p ] AtlasMeshpatchFaceCount[ [ atlasPatchIdx ] = TileMeshpatchFaceCount[ tileID ][ p ] + 1 AtlasMeshpatchSubdivCount[ atlasPatchIdx ] = TileMeshpatchSubdivCount[ tileID ][ p ] for( i = 0; i < AtlasMeshpatchSubdivCount[ atlasPatchIdx ]; i++ ){ AtlasMeshpatchSubdivMethod[ atlasPatchIdx ][ i ] = TileMeshpatchSubdivMethod[ tileID ][ p ][ i ] } AtlasMeshpatchDispCoordSys[ atlasPatchIdx ] = TileMeshpatchDispCoordSys[ tileID ][ p ] AtlasMeshpatchTransformMethod[ atlasPatchIdx ] = TileMeshpatchTransformMethod[ tileID ][ p ] for( i=0; i<= AtlasPatchSubdivisionCount[ atlasPatchIdx ]; i++){ AtlasPatchVertexBlockCount[ atlasPatchIdx ][ i ] = TilePatchVertexBlockCount[ tileID ][ p ][ i ] AtlasPatchVertexCountLast[ atlasPatchIdx ][ i ] = TilePatchVertexCountLast[ tileID ][ p ][ i ] AtlasPatchVertexCount[ atlasPatchIdx ][ i ] = TileVertexCount[ tileID ][ p ][ i ] AtlasPatchTotalVertexCount[ atlasPatchIdx ] = TilePatchTotalVertexCount[ tileID ][ p ] } if( asve_num_attribute_video > 0 ){ AttributeTileMeshpatchParamsToAtlas( atlasPatchIdx, t, p ) } AtlasMeshpatchTexcoordProjectionFlag[ atlasPatchIdx ] = TileMeshpatchTexcoordProjectionFlag[ tileID ][ p ] AtlasMeshpatchTexcoordProjectionWidthNormalization[ atlasPatchIdx ] = TileMeshpatchTexcoordProjectionWidthNormalization[ tileID ][ p ] AtlasMeshpatchTexcoordProjectionHeightNormalization[ atlasPatchIdx ] = TileMeshpatchTexcoordProjectionHeightNormalization[ tileID ][ p ] AtlasMeshpatchTexcoordProjectionGutter[ atlasPatchIdx ] = TileMeshpatchTexcoordProjectionGutter[ tileID ][ p ] if( AtlasMeshpatchTexcoordProjectionFlag[ atlasPatchIdx ] ) SubpatchTileParamToAtlas( atlasPatchIdx, tileID, p ) atlasPatchIdx += 1 } } AtlasTotalNumMeshpatches = atlasPatchIdx (Operation of the tile-based sub-mesh information decoder and mesh patch information decoder) Figure 22 shows an example of the syntax for mesh and submesh information.

[0100] As shown in the syntax configuration of FIG. 22 , the submesh information decoding unit 3024 loops through tile index i and submesh index j, and calculates a flag afmi_submesh_in_tile_flag[tileID][submeshID] (where tileID=afti_tile_id[ i ], submeshID=afmi_submesh_id[ j ]) indicating whether or not submesh information having the submesh ID (submeshID) of the submesh with index j is included in the tile information having the tile ID (tileID) of the tile with index i. ). The semantics may be:

[0101] afmi_submesh_in_tile_flag[ tileID ][ submeshID ]: The tile of the tile with index i. The tile information with the tile ID of the submesh with index j is If the flag is true (e.g., 1), it indicates that the tile information with the tile ID (afti_tile_id[i]) of the tile with index i contains the submesh information with the submesh ID (afmi_submesh_id[j]) of the submesh with index j. If the flag is false (e.g., 0), it indicates that the tile ID of the tile with index i contains the submesh information with the submesh ID (afmi_submesh_id[j]). This indicates that the tile information with (afti_tile_id[ i ]) does not include the submesh information with the submesh ID (afmi_submesh_id[ j ]) of the submesh with index j.

[0102] Here, the tile information with the tile ID of the tile with index i is a submetadata of the tile with index j. The submesh information with the submesh ID of the same mesh is included if the type of index i The tile and mesh patch data decoded using the parameters decoded from the tile information with the tile ID of the submesh, and the submesh of the submesh with index j that can be reconstructed are This means that the sub-mesh has the same ID as the index i. This means that there is no correspondence between the decoding of a tile with the tile ID of the tile with index j and the reconstruction of a submesh with the submesh ID of the submesh with index j.

[0103] Here, the two-dimensional array afmi_submesh_in_tile_flag[afti_tile_id[i]][afmi_submesh_id[j]] is the flag of the submesh with the submesh ID of index j (afmi_submesh_id[j]). The cache information is included in only one tile of index i, and the other tile information contains Conformance requirements may also be set to the exclusion of submesh information (exclusion constraints). In other words, multiple tile information must not contain submesh information with the same submesh ID, as in the following example:

[0104] afmi_submesh_in_tile_flag[ afti_tile_id

[0000] ][ afmi_submesh_id

[0000] ] = 0 afmi_submesh_in_tile_flag[ afti_tile_id

[0000] ][ afmi_submesh_id

[0001] ] = 1 afmi_submesh_in_tile_flag[ afti_tile_id

[0000] ][ afmi_submesh_id

[0002] ] = 1 afmi_submesh_in_tile_flag[ afti_tile_id

[0001] ][ afmi_submesh_id

[0000] ] = 1 afmi_submesh_in_tile_flag[ afti_tile_id

[0001] ][ afmi_submesh_id

[0001] ] = 1 afmi_submesh_in_tile_flag[ afti_tile_id

[0001] ][ afmi_submesh_id

[0002] ] = 0 In this example, the tile information with the tile IDs of tiles with indexes 0 and 1 contains submesh information with the submesh ID of the submesh with index 1. This violates performance requirements.

[0105] Here, the two-dimensional array afmi_submesh_in_tile_flag[ afti_tile_id[ i ] ][ afmi_submesh_id[ j ] ] specifies that if tile information with the tile ID (afti_tile_id[ i ]) of index i is included in multiple submesh information, the index of that submesh information, SubMeshIDToIndex[ aftmi_submesh_id[ j ] ], must be consecutive indices such as 0, 1, 2, and the conformance requirement may be that submesh information with discontinuous indices such as 1, 3, 5 or 1, 2, 4 does not contain (or must not contain) the same tile information.

[0106] In the syntax configuration of Figure 22, the number of tiles, NumTiles, is derived by decoding afti_num_tiles_in_atlas_frame_minus2 and calculating NumTiles = afti_num_tiles_in_atlas_frame_minus2 + 2. Alternatively, afti_num_tiles_in_atlas_frame_minus1 may be decoded and NumTiles = afti_num_tiles_in_atlas_frame_minus1+1 may be derived. Here, the syntax element afti_single_tile_in_atlas_frame_flag or the syntax If the value of the attribute afti_single_partition_per_tile_flag is false (for example, 0), the tile The number of tiles is calculated with NumTiles=1.

[0107] Here, the following shows an example of the values ​​of afmi_submesh_in_tile_flag[ afti_tile_id[ i ] ][ afmi_submesh_id[ j ] ] when numTiles = 2, numSubMeshes = 3, afmi_signalled_submesh_id_flag = 0, afti_tile_id[0] includes afmi_submehs_id[1] and afmi_submesh_id[2], and afti_tile_id[1] includes afmi_submesh_id[0].

[0108] afmi_submesh_in_tile_flag[0][0] = 0 afmi_submesh_in_tile_flag[0][1] = 1 afmi_submesh_in_tile_flag[0][2] = 1 afmi_submesh_in_tile_flag[1][0] = 1 afmi_submesh_in_tile_flag[1][1] = 0 afmi_submesh_in_tile_flag[1][2] = 0 Here, the mesh patch information decoding unit 3025 derives mesh patch information at the tile level as follows.

[0109] for( t = 0; t <= afti_num_tiles_in_atlas_frame_minus1; t++ ) { tileID = TileIndexToID[ t ] AtduTotalNumMeshpatches[ tileID ] = MaxNumMeshpatches for( si = 0; si < afmi_num_submeshes_minus2 + 2; si++ ) { submeshID = SubMeshIndexToID[ si ] if( afmi_submesh_in_tile_flag[ tileID ][ submeshID ] ) { for( p = 0; p <= AtduTotalNumMeshpatches[ tileID ]; p++ ) { if( submeshID == mdu_submesh_id[ tileID ][ p ] ) { ... TileMeshpatchSubmeshID[ tileID ][ p ] = submeshID ... } } } } } Furthermore, the mesh patch information decoding unit 3025 may derive mesh patch information corresponding to sub-meshes at the tile level as follows.

[0110] for( t = 0; t <= afti_num_tiles_in_atlas_frame_minus1; t++ ) { tileID = TileIndexToID[ t ] for( si = 0; si < afmi_num_submeshes_minus2 + 2; si++ ) { submeshID = SubMeshIndexToID[ si ] if( afmi_submesh_in_tile_flag[ tileID ][ submeshID ] ) { AtduTotalNumMeshpatches[ tileID ][ submeshID ] = MaxNumMeshpatches for( p = 0; p <= AtduTotalNumMeshpatches[ tileID ][ submeshID ]; p++ ){ TileMeshpatch2dPosX[ tileID ][ submeshID ][ p ] = mdu_2d_pos_x[ tileID ][ submeshID ][ p ] * PatchPackingBlockSize TileMeshpatch2dPosY[ tileID ][ submeshID ][ p ] = mdu_2d_pos_y[ tileID ][ submeshID ][ p ] * PatchPackingBlockSize TileMeshpatch2dSizeX[ tileID ][ submeshID ][ p ] = (mdu_2d_size_x_minus1[ tileID ][ submeshID ][ p ] + 1) * P PatchSizeXQuantizer TileMeshpatch2dSizeY[ tileID ][ submeshID ][ p ] = (mdu_2d_size_y_minus1[ tileID ][ submeshID ][ p ] + 1) * PatchSizeYQuantizer for( attrIdx = 0; attrIdx < asve_num_attribute_video; attrIdx){ TileMeshpatchAttributes2dPosX[tileID][submeshID][p][attrIdx] = mdu_attributes_2d_pos_x[ tileID ][ submeshID ][ p ][ attrIdx ] TileMeshpatchAttributes2dPosY[tileID][submeshID][p][attrIdx] = mdu_attributes_2d_pos_y[ tileID ][ submeshID ][ p ][ attrIdx ] TileMeshpatchAttributes2dSizeX[tileID][submeshID][p][attrIdx] = (mdu_attributes_2d_size_x_minus1[tileID][submeshID][p][attrIdx] + 1) * PatchSizeXQuantizer TileMeshpatchAttributes2dSizeY[tileID][submeshID][p][attrIdx] = (mdu_attributes_2d_size_y_minus1[tileID][submeshID][p][attrIdx] + 1) * PatchSizeYQuantizer } TileMeshpatchSubmeshID[ tileID ][ submeshID ][ p ] = submeshID TileMeshpatchDisplID[ tileID ][ submeshID ][ p ] = mdu_displ_id[ tileID ][ submeshID ][ p ] TileMeshpatchSubdivCount[ tileID ][ submeshID ][ p ] = PatchSubdivisionCount[ tileID ][ patchIdx ] for( i = 0; i<TileMeshpatchSubdivCount[tileID][submeshID][p] ; i++ ){ TileMeshpatchSubdivMethod[ tileID ][ submeshID ][ p ][ i ] = PatchSubdivisionMethod[ tileID ][ submeshID ][ p ][ i ] } TileMeshpatchDispCoordSys[ tileID ][ submeshID ][ p ] = mdu_displacement_coordinate_system[ tileID ][ submeshID ][ p ] TileMeshpatchTransformMethod[ tileID ][ submeshID ][ p ] = mdu_transform_method[ tileID ][ submeshID ][ p ] ... } } } } Here, the tile-level mesh patch information may be converted to atlas-level mesh patch information as follows.

[0111] AtlasTotalNumMeshpatches = 0 atlasPatchIdx = 0 for( t = 0; t <= afti_num_tiles_in_atlas_frame_minus1; t++ ) { tileID = TileIndexToID[ t ] tileOffsetX = TileOffsetX[ t ] tileOffsetY = TileOffsetY[ t ] for( si = 0; si < afmi_num_submeshes_minus2 + 2; si++ ) { submeshID = SubMeshIndexToID[ si ] if( afmi_submesh_in_tile_flag[ tileID ][ submeshID ] ) { for( p = 0; p < AtduTotalNumMeshpatches[ tileID ][ submeshID ]; p++ ) { AtlasMeshpatch2dSizeX[ atlasPatchIdx ] = TileMeshpatch2dSizeX[ tileID ][ submeshID ][ p ] ... atlasPatchIdx += 1 } } } } AtlasTotalNumMeshpatches = atlasPatchIdx Here, TileIndexToID[i] is a table that derives the tile ID (tileID) from the index i. be.

[0112] Here, the syntax element afmi_submesh_in_tile_flag can be specified as follows: The encoding and decoding may be performed using an index tileIndex derived from the mesh ID and an index submeshIndex derived from the submesh ID.

[0113] atlas_frame_mesh_information() { ... for( i = 0; i < NumTiles; i++ ) { for( j = 0; j < NumSubMeshes; j++ ) { tileIndex = TileIDToIndex[ afti_tile_id[ i ] ] submeshIndex = SubMeshIDToIndex[ afmi_submesh_id[ j ] ] afmi_submesh_in_tile_flag[ tileIndex ][ submeshIndex ] } } ... } Also, without using the IDs of the syntax elements afti_tile_id[ i ] and afmi_submesh_id[ j ], The syntax element afmi_submesh_in_tile_flag[ i ][ j ] may be encoded and decoded.

[0114] atlas_frame_mesh_information() { ... for( i = 0; i < NumTiles; i++ ) { for( j = 0; j < NumSubMeshes; j++ ) { afmi_submesh_in_tile_flag[ i ][ j ] } } ... } Here, the tile ID of the tile with index i can be derived from TileIndexToID[i], and the submesh ID of the submesh with index j can be derived from SubMeshIndexToID[j].

[0115] According to the above configuration, the coded data can derive the correspondence relationships between tile information and submesh information, between tile information and mesh patch information, and between submesh information and mesh patch information on a tile basis using flags that indicate whether or not there is a correspondence relationship between tile information having an arbitrary tile ID decoded by the submesh information decoding unit and submesh information having an arbitrary submesh ID. Therefore, by identifying the tile information and mesh patch information corresponding to an arbitrary submesh, it is possible to decode only the tile information and mesh patch information necessary to reconstruct the arbitrary submesh.

[0116] Also, as shown in the syntax configuration of Figure 26, the tile index i, submesh It loops with index j of the tile with index i and checks whether the tile information with the tile ID of the tile with index i contains submesh information with the submesh ID of the submesh with index j. A flag indicating the submesh in tile may be transmitted by the SEI: tmsm_submesh_in_tile_flag[ tmsm_tile_id[ i ] ][ tmsm_submesh_id[ j ] ]. The following example semantics may be used:

[0117] tmsm_persistance_mapping_flag: If the value of tmsm_persistance_mapping_flag is equal to 1, If flag is equal to 0, it indicates that tile submesh mapping will persist. Indicates that tile submesh mapping is valid for the current frame only.

[0118] tmsm_num_tiles_minus1: The number of tiles in the coded atlas sequence (CAS) (tmsm_num_tiles_minus1+1). tmsm_num_tiles_minus1 is equal to afti_num_tiles_in_atlas_frame_minus1.

[0119] tmsm_tile_id_length_minus1: The value of tmsm_tile_id_length_minus1 plus 1 is The bitmap used to represent the tmsm_tile_id[ i ] element, if present. The value of tmsm_tile_id_length_minus1 must be in the range of 0 to 15.

[0120] tmsm_tile_id[ i ]: Specifies the tile ID of the i-th tile. tmsm_tile_id[ i ] exists If not specified, the value of tmsm_tile_id[ i ] is inferred to be equal to i for each i in the range 0 to tmsm_num_tiles_minus1, inclusive. tmsm_tile_id[ k ] for all i != k is not a bitstream conformance requirement. is.

[0121] tmsm_use_single_mesh_flag: If the flag is equal to 1, there is only one submesh. If tmsm_use_single_mesh_flag is equal to 0, it specifies that there are multiple sub-meshes.

[0122] tmsm_num_submeshes_minus2: The value of tmsm_num_submeshes_minus2 plus 2 is Specifies the number of submeshes, NumSubMeshes. If tmsm_num_submeshes_minus2 is not present and tmsm_use_single_mesh_flag is equal to 1, the value of NumSubMeshes is inferred to be equal to 1.

[0123] tmsm_submesh_id_length_minus1: The value of tmsm_submesh_id_length_minus1 plus 1 is Specifies the number of bits used to represent the syntax element tmsm_submesh_id[ j ]. The value of tmsm_signalled_submesh_id_length_minus1 shall be in the range 0 to 15. If not present, The value of is estimated to be equal to Ceil( Log2( NumSubMeshes ) - 1).

[0124] tmsm_submesh_id[ j ]: Specifies the submesh ID of the jth submesh. If not, the value of tmsm_submesh_id[j] is in the range 0 to NumSubMeshes - 1 for each j. The length of tmsm_submesh_id[ j ] is tmsm_submesh_id_length_minus1 + 1 bits.

[0125] tmsm_submesh_in_tile_flag[ tmsm_tile_id[ i ] ][ tmsm_submesh_id[ j ] ]: Indicates whether the tile information with the tile ID of the tile with index i contains submesh information with the submesh ID of the submesh with index j. The flag is true (e.g., 1). In this case, the tile information with the tile ID (tmsm_tile_id[ i ]) of the tile with index i is used to find the submesh with the submesh ID (tmsm_submesh_id[ j ]) of the submesh with index j. Indicates that mesh information is included. If the flag is false (e.g., 0), the tile information with the tile ID (tmsm_tile_id[ i ]) of the tile with index i is used to find the tile with index j. Indicates that submesh information with the submesh ID (tmsm_submesh_id[ j ]) of the submesh is not included.

[0126] Here, the mesh patch information decoding unit 3025 decodes the mesh patch information at the tile level into the syntax element tmsm_submesh_in_tile_flag[ tmsm_tile_id[ i ] ][ tmsm_submesh_id[ j ] ] may also be derived.

[0127] Another configuration uses the IDs of the syntax elements tmsm_tile_id[ i ] and tmsm_submesh_id[ j ]. Instead, the syntax element tmsm_submesh_in_tile_flag[ i ][ j ] may be coded and decoded using indexes such as i and j as follows:

[0128] tile_submesh_mapping( payloadSize ) { ... for( i = 0; i < tmsm_num_tiles_minus1 + 1; i++ ) { for( j = 0; j < NumSubMeshes; j++ ) { tmsm_submesh_in_tile_flag[ i ][ j ] } } ... } Here, the tile ID of the tile with index i can be derived from TileIdxToID[i], and the submesh ID of the submesh with index j can be derived from SubmeshIdxToID[j].

[0129] Figure 23 shows an example of the syntax for mesh and submesh information.

[0130] In another configuration, as shown in the syntax configuration of FIG. 23, the submesh information decoding unit 3024 decodes a flag afmi_use_single_mesh_flag[tileId] indicating whether the number of submesh information having the submesh ID included in the tile is one or not, and if the value of afmi_use_single_mesh_flag[tileId] is false, The syntax element that indicates the number of NumSubMeshes is afmi_num_submeshes_minus2[tileId]. and derive NumSubMeshes = afmi_num_submeshes_minus2[tileId] + 2. When the value of afmi_use_single_mesh_flag[tileId] is true and afmi_num_submeshes_minus2[tileId] does not appear, it infers NumSubMeshes = 1. As shown in Fig. 23, when the value of afmi_signalled_submesh_id_flag[tileId] is true, the submesh information decoding unit 3024 may decode one or more j submesh IDs (afmi_submesh_id[tileId][j]) included in the tile information of a certain i as submesh information. Regarding the decoded tile ID and submesh ID, a secondary submesh with the tileId and submesh ID (afmi_submesh_id[tileId][j]) as subscripts may be generated. The original array may be derived. The two-dimensional array indicating the correspondence between this submesh and tile may be SubmeshIDToInTileIndex and SubmeshInTileIndexToID. If afmi_signalled_submesh_id_flag[tileId] is true, afmi_submesh_id[tileId][j] in the range j = 0..NumSubMeshes-1 is decoded and arrayed by the number of submeshes included in the tile, NumSubMeshes. Derive the columns SubmeshIDToInTileIndex and SubmeshInTileIndexToID for i = 0.. NumSubMeshes-1 as follows:

[0131] SubmeshIDToInTileIndex[ tileId ][ afmi_submesh_id[ tileId ][ j ] ] = j SubmeshInTileIndexToID[ tileId ][ j ] = afmi_submesh_id[ tileId ][ j ] Here, the semantics may use the following example:

[0132] afmi_submesh_id[ tileID ][ i ]: the i-th submesh of the tile whose tile ID is equal to tileID Specifies the submesh ID of the base mesh. If not present, the value of afmi_submesh_id[ tileID ][ i ] is inferred to be equal to i for each i in the range 0 to NumSubMeshes[ tileId ]. The length of the afmi_submesh_id[ i ] syntax element is asve_signalled_submesh_id_length_minus1+1 bits. If afmi_submesh_id[ tileId ][ i ] is The requirement is that it must be in the stream.

[0133] mdu_submesh_intile_index[ tileID ][ patchIdx ]: indicates the submesh index associated with the current patch with index patchIdx in the current atlas tile with tile ID equal to tileID. The value of mdu_submesh_intile_index[ tileID ][ patchIdx ] is 0. The length of the mdu_submesh_intile_index[ tileID ][ patchIdx ] syntax element must be in the range of Ceil( Log2( NumSubMeshes[ tileID ]) ) - 1. If mdu_submesh_intile_index[ tileID ][ patchIdx ] does not exist, mdu_submesh_intile_index[ tileID ][ patchIdx ] is derived as patchIdx.

[0134] Also, if the value of afmi_signalled_submesh_id_flag[ tileId ] is always false, as in another example: Then, the value of afmi_submesh_id[ tileId ][ j ] may be derived as in the following syntax configuration:

[0135] for( j = 0; j < NumSubMeshes; j++ ) { afmi_submesh_id[ tileId ][ j ] = j } Here, the mesh patch information decoding unit 3025 derives the mesh patch submesh ID (TileMeshpatchSubmeshID) at the tile level as follows.

[0136] TileMeshpatchSubmeshID[ tileID ][ p ] = afmi_submesh_id[ tileID ][ mdu_submesh_intile_index[ tileID ][ p ] ] According to the above configuration, the encoded data can derive the correspondence relationships between tile information and sub-mesh information, between tile information and mesh patch information, and between sub-mesh information and mesh patch information on a tile basis, using the index of the sub-mesh information corresponding to the tile information with an arbitrary tile ID decoded by the mesh patch information decoding unit and the mesh patch with a certain patch index. Therefore, by identifying the tile information and mesh patch information corresponding to an arbitrary sub-mesh, it is possible to decode only the tile information and mesh patch information necessary to reconstruct the arbitrary sub-mesh.

[0137] Also, as shown in the syntax configuration of FIG. 25, the flag tmsm_use_single_mesh_flag[ i ] indicating whether the number of submesh information having the submesh ID included in the tile is one or not is decoded, and if the value of tmsm_use_single_mesh_flag[ i ] is false, the number of submesh information included in the tile is one. The syntax element tmsm_num_submeshes_minus2[ i ], which indicates the number of submeshes with submesh IDs, NumSubMeshes, may be decoded, and NumSubMeshes = afmi_num_submeshes_minus2[ tileId ] + 2 may be derived and transmitted in the SEI. Alternatively, the syntax element smtm_num_submeshes_minus1, which indicates the number of submeshes associated with a tile, may be decoded, and each submesh ID (smtm_submesh_id[ i ]) and the associated tile ID (smtm_tile_id[ i ]) may be decoded, and SubmeshIdxToID[ i ] and SubmeshIdxToAtlasTileID[ i ] may be derived and transmitted in the SEI. The semantics may also be expressed using the following example: good.

[0138] smtm_persistance_mapping_flag: If the flag is equal to 1, it indicates that the submesh tile mapping is persistent. If the flag is equal to 0, it indicates that the submesh tile mapping is persistent. Indicates that it is valid only for the current frame.

[0139] smtm_num_submeshes_minus1: Indicates the number of submeshes associated with tiles present in the CAS, smtm_num_submeshes_minus1+1.

[0140] smtm_submesh_id_length_minus1: The value of smtm_submesh_id_length_minus1 plus 1 is Specifies the number of bits used to represent the syntax element smtm_submesh_id[ i ], if present.

[0141] smtm_tile_id_length_minus1: If the syntax element smtm_tile_id[ i ] is present, the value of smtm_tile_id_length_minus1 plus 1 specifies the number of bits used to represent smtm_tile_id[ i ].

[0142] smtm_submesh_id[ i ]: Specifies the i-th submesh ID. smtm_submesh_id[ i ] is It is a bitstream conformance requirement that smtm_submesh_id[ k ] must not be equal to smtm_submesh_id[ k ] for all i != k.

[0143] smtm_tile_id[ i ]: Specifies the tile ID associated with the submesh with index i.

[0144] Here, the mesh patch information decoding unit 3025 may derive the mesh patch submesh ID (TileMeshpatchSubmeshID) at the tile level as follows.

[0145] TileMeshpatchSubmeshID[ tileID ][ p ] = tmsm_submesh_id[ tileID ][ mdu_submesh_intile_index[ tileID ][ p ] ] In another configuration, as shown in the syntax configuration of mesh / submesh information in FIG. 24, a submesh with index i corresponding to tile information having the tile ID of a tile with index tileID is The flag mdu_submesh_in_tile_flag[ SubmeshInTileIndexToID[ tileID ][ i ] ][ patchIdx ] indicating the correspondence between the submesh information with the submesh ID and the mesh patch with the index patchIdx is decoded, and if there is a correspondence between the submesh information whose submesh ID is equal to SubmeshInTileIndexToID[ tileID ][ i ] and the mesh patch with the patch index patchIdx, the flag mdu_submesh_in_tile_flag[ SubmeshInTileIndexToID[ tileID ][ i ] ][ patchIdx ] is decoded. patchIdx] is decoded as a value (for example, 1) indicating that there is a correspondence. The submesh information and patch index whose submesh ID is equal to SubmeshInTileIndexToID[tileID][i] are If the mesh patch with patchIdx is not in correspondence, mdu_submesh_in_tile_flag[ SubmeshInTileIndexToID[ tileID ][ i ] ][ patchIdx ] is set to a value indicating that there is no correspondence. (e.g., 0). There may be multiple mesh patches corresponding to submesh information whose submesh ID is equal to SubmeshInTileIndexToID[tileID][i], but there must be only one submesh information corresponding to a mesh patch with patch index patchIdx. Semantics may use the following examples:

[0146] mdu_submesh_in_tile_flag[ SubmeshInTileIndexToID[ tileID ][ i ] ][ patchIdx ] :Submesh information whose submesh ID is equal to SubmeshInTileIndexToID[ tileID ][ i ] Indicates whether there is a correspondence between the submesh information whose submesh ID is equal to SubmeshInTileIndexToID[tileID][i] and the mesh patch having the patch index patchIdx. If the flag is 1, it indicates that there is a correspondence between the submesh information whose submesh ID is equal to SubmeshInTileIndexToID[tileID][i] and the mesh patch having the patch index patchIdx, and if the flag is 0, it indicates that there is no correspondence between the submesh information whose submesh ID is equal to SubmeshInTileIndexToID[tileID][i] and the mesh patch having the patch index patchIdx.

[0147] Here, the mesh patch information decoding unit 3025 derives mesh patch information at the tile level as follows.

[0148] for( t = 0; t <= afti_num_tiles_in_atlas_frame_minus1; t++ ) { tileID = TileIndexToID[ t ] AtduTotalNumMeshpatches[ tileID ] = MaxNumMeshpatches for( si = 0; si < afmi_num_submeshes_minus2[ tileID ] + 2; si++ ) { submeshID = SubmeshInTileIndexToID[ tileID ][ si ] for( p = 0; p <= AtduTotalNumMeshpatches[ tileID ]; p++ ) { if( mdu_submesh_in_tile_flag[ submeshID ] ][ p ] ) { ... TileMeshpatchSubmeshID[ tileID ][ p ] = submeshID ... } } } } According to the above configuration, the coded data can derive the correspondence between tile information and sub-mesh information, between tile information and mesh patch information, and between sub-mesh information and mesh patch information on a tile basis using a flag indicating whether or not sub-mesh information having an arbitrary sub-mesh ID decoded by the mesh patch information decoding unit corresponds to a mesh patch having a certain patch index. Therefore, by identifying the tile information and mesh patch information corresponding to an arbitrary sub-mesh, the tile information required for reconstructing the arbitrary sub-mesh can be derived. This also has the effect of making it possible to decode only the mesh patch information.

[0149] (Operation of the sub-mesh information decoding unit and mesh patch information decoding unit based on the sub-mesh) Another construction is to use the submesh index i, the time stamp, and the It loops through the tile index j and checks whether the submesh with submesh ID (submeshID) contains a tile with tile ID (tileID) of the tile with index j. The flag afmi_tile_in_submesh_flag[submeshID][tileID] (where submeshID = afmi_submesh_id[ i ], tileID = afti_tile_id[ j ]) indicating the index i The submesh information with the submesh ID of the submesh j is The tile information with the submesh ID of index i is included. When reconstructing the cache, the tile information with the tile ID of the tile with index j is restored. It means that the tile and mesh patch data must be decoded using the decoded parameters. There is no correspondence between the reconstruction and the decoding of the tile with the tile ID of the tile with index j. means.

[0150] atlas_frame_mesh_information() { ... for( i = 0; i < NumSubMeshes; i++ ) { for( j = 0; j < NumTiles; j++ ) { afmi_tile_in_submesh_flag[ afmi_submesh_id[ i ] ][ afti_tile_id[ j ] ] } } ... } Here, the semantics may use the following example:

[0151] afmi_tile_in_submesh_flag[afmi_submesh_id[i]][afti_tile_id[j]]: Indicates whether the submesh information with the submesh ID of the submesh with index i contains the tile information with the tile ID of the tile with index j. If the flag is 1, the index Submesh with submesh ID (afmi_submesh_id[ i ]) of submesh of block i If the flag is 0, the tile information with the tile ID (afti_tile_id[ j ]) of the tile with index j is included. If the flag is 0, the tile information with the submesh ID (afmi_submesh_id[ i ]) of the submesh with index i is included. Indicates that tile information with tile ID (afti_tile_id[ j ]) is not included.

[0152] Here, the two-dimensional array afmi_tile_in_submesh_flag[submeshID][tileID] may have the conformance requirement that tile information having the tile ID (afmi_tile_id[j]) of the tile with index j is included in only one submesh information and not in other submesh information (exclusive constraint). In other words, as in the following example, tile information with the same tile ID must not be included in multiple submesh information.

[0153] afmi_tile_in_submesh_flag[ afmi_submesh_id

[0000] ][ afti_tile_id

[0000] ] = 0 afmi_tile_in_submesh_flag[ afmi_submesh_id

[0000] ][ afti_tile_id

[0001] ] = 1 afmi_tile_in_submesh_flag[ afmi_submesh_id

[0000] ][ afti_tile_id

[0002] ] = 1 afmi_tile_in_submesh_flag[ afmi_submesh_id

[0001] ][ afti_tile_id

[0000] ] = 1 afmi_tile_in_submesh_flag[ afmi_submesh_id

[0001] ][ afti_tile_id

[0001] ] = 1 afmi_tile_in_submesh_flag[ afmi_submesh_id

[0001] ][ afti_tile_id

[0002] ] = 0 In this example, the submesh information with the submesh IDs of the submeshes with indexes 0 and 1 contains the tile information with the tile ID of the tile with index 1. This violates performance requirements.

[0154] Here, the two-dimensional array afmi_tile_in_submesh_flag[afmi_submesh_id[i]][afti_tile_id[j]] is the submesh ID of the submesh with index i (afmi_submesh_id[i] ) is included in multiple tile information, the index of that tile information, TileIDToIndex[afti_tile_id[j]], must be consecutive indices such as 0, 1, 2, etc. A conformance requirement may be that tile information with discontinuous indexes such as 1, 3, 5 or 1, 2, 4 is not included in the same submesh information (there should be no cases where this is the case).

[0155] Here, the following shows an example of the values ​​of afmi_tile_in_submesh_flag[ afmi_submesh_id[ i ] ][ afti_tile_id[ j ] ] when numTiles = 3, numSubMeshes = 2, afmi_signalled_submesh_id_flag = 0, afmi_submesh_id[0] includes afti_tile_id[1] and afti_tile_id[2], and afmi_submesh_id[1] includes afti_tile_id[0].

[0156] afmi_tile_in_submesh_flag[0][0] = 0 afmi_tile_in_submesh_flag[0][1] = 1 afmi_tile_in_submesh_flag[0][2] = 1 afmi_tile_in_submesh_flag[1][0] = 1 afmi_tile_in_submesh_flag[1][1] = 0 afmi_tile_in_submesh_flag[1][2] = 0 Here, the mesh patch information decoding unit 3025 derives mesh patch information at the tile level as follows.

[0157] for( si = 0; si < afmi_num_submeshes_minus2 + 2; si++ ) { submeshID = SubMeshIndexToID[ si ] for( t = 0; t <= afti_num_tiles_in_atlas_frame_minus1; t++ ) { tileID = TileIndexToID[ t ] if( afmi_tile_in_submesh_flag[ submeshID ][ tileID ] ) { AtduTotalNumMeshpatches[ tileID ] = MaxNumMeshpatches for( p = 0; p <= AtduTotalNumMeshpatches[ tileID ]; p++ ) { ... TileMeshpatchSubmeshID[ tileID ][ p ] = submeshID ... } } } } Furthermore, in the mesh patch information decoding unit 3025, when multiple mesh patches correspond to one sub-mesh, tile-level mesh patch information may be derived as follows.

[0158] for( si = 0; si < afmi_num_submeshes_minus2+2; si++ ) { submeshID = SubMeshIndexToID[ si ] for( t = 0; t <= afti_num_tiles_in_atlas_frame_minus1; t++ ) { tileID = TileIndexToID[ t ] if( afmi_submesh_in_tile_flag[ submeshID ][ tileID ] ) { AtduTotalNumMeshpatches[ submeshID ][ tileID ] = MaxNumMeshpatches for( p = 0; p <= AtduTotalNumMeshpatches[ submeshID ][ tileID ]; p++ ){ TileMeshpatch2dPosX[ submeshID ][ tileID ][ p ] = mdu_2d_pos_x[ submeshID ][ tileID ][ p ] * PatchPackingBlockSize TileMeshpatch2dPosY[ submeshID ][ tileID ][ p ] = mdu_2d_pos_y[ submeshID ][ tileID ][ p ] * PatchPackingBlockSize TileMeshpatch2dSizeX[ submeshID ][ tileID ][ p ] = (mdu_2d_size_x_minus1[ submeshID ][ tileID ][ p ] + 1) * P PatchSizeXQuantizer TileMeshpatch2dSizeY[ submeshID ][ tileID ][ p ] = (mdu_2d_size_y_minus1[ submeshID ][ tileID ][ p ] + 1) * PatchSizeYQuantizer for( attrIdx = 0; attrIdx < asve_num_attribute_video; attrIdx){ TileMeshpatchAttributes2dPosX[submeshID][tileID][p][attrIdx] = mdu_attributes_2d_pos_x[ submeshID ][ tileID ][ p ][ attrIdx ] TileMeshpatchAttributes2dPosY[submeshID][tileID][p][attrIdx] = mdu_attributes_2d_pos_y[ submeshID ][ tileID ][ p ][ attrIdx ] TileMeshpatchAttributes2dSizeX[submeshID][tileID][p][attrIdx] = (mdu_attributes_2d_size_x_minus1[submeshID][tileID][p][attrIdx] + 1) * PatchSizeXQuantizer TileMeshpatchAttributes2dSizeY[submeshID][tileID][p][attrIdx] = (mdu_attributes_2d_size_y_minus1[submeshID][tileID][p][attrIdx] + 1) * PatchSizeYQuantizer } TileMeshpatchSubmeshID[ submeshID ][ tileID ][ p ] = submeshID TileMeshpatchDisplID[ submeshID ][ tileID ][ p ] = mdu_displ_id[ submeshID ][ tileID ][ p ] TileMeshpatchSubdivCount[ submeshID ][ tileID ][ p ] = PatchSubdivisionCount[ tileID ][ patchIdx ] for( i = 0; i <TileMeshpatchSubdivCount[submeshID][tileID][p] ; i++ ){ TileMeshpatchSubdivMethod[ submeshID ][ tileID ][ p ][ i ] = PatchSubdivisionMethod[ submeshID ][ tileID ][ p ][ i ] } TileMeshpatchDispCoordSys[ submeshID ][ tileID ][ p ] = mdu_displacement_coordinate_system[ submeshID ][ tileID ][ p ] TileMeshpatchTransformMethod[ submeshID ][ tileID ][ p ] = mdu_transform_method[ submeshID ][ tileID ][ p ] ... } } } } Also, always use indexes instead of IDs, such as afmi_tile_in_submesh_flag[ SubMeshIDToIndex[ afmi_submesh_id[ i ] ] ][ TileIDToIndex[ afti_tile_id[ j ] ] ] It may be encoded and decoded.

[0159] In addition, in the syntax configuration of Figure 22, instead of using the IDs of the syntax elements afmi_submesh_id[ i ] and afti_tile_id[ j ], the syntax is written using indexes such as i and j as follows: The submesh element afmi_tile_in_submesh_flag[ i ][ j ] may be encoded and decoded.

[0160] atlas_frame_mesh_information() { ... for( i = 0; i < NumSubMeshes; i++ ) { for( j = 0; j < NumTiles; j++ ) { afmi_tile_in_submesh_flag[ i ][ j ] } } ... } Here, the submesh ID of the submesh with index i can be derived from SubMeshIndexToID[i], and the tile ID of the tile with index j can be derived from TileIndexToID[j].

[0161] According to the above configuration, the coded data can derive the correspondence relationships between tile information and submesh information, between tile information and mesh patch information, and between submesh information and mesh patch information on a submesh basis using flags that indicate whether or not there is a correspondence relationship between tile information having an arbitrary tile ID decoded by the submesh information decoding unit and submesh information having an arbitrary submesh ID. Therefore, by identifying the tile information and mesh patch information corresponding to an arbitrary submesh, it is possible to decode only the tile information and mesh patch information necessary to reconstruct the arbitrary submesh.

[0162] Also, the submesh information with the submesh ID of the submesh with index i is A flag tmsm_tile_in_submesh_flag[ tmsm_submesh_id[ i ] ][ tmsm_tile_id[ j ] ] indicating whether tile information with the tile ID of the tile in index j is included may be transmitted by the SEI. The following example semantics may be used:

[0163] tmsm_tile_in_submesh_flag[ tmsm_submesh_id[ i ] ][ tmsm_tile_id[ j ] ]: Indicates whether the submesh information with the submesh ID of the submesh with index i contains the tile information with the tile ID of the tile with index j. If the flag is 1, the index Submesh with submesh ID (tmsm_submesh_id[ i ]) of submesh of submesh i If the flag is 0, the tile information with the tile ID (tmsm_tile_id[ j ]) of the tile with index j is included. If the flag is 0, the tile information with the submesh ID (tmsm_submesh_id[ i ]) of the submesh with index i is included. Indicates that tile information with the tile ID (tmsm_tile_id[ j ]) is not included.

[0164] Here, the mesh patch information decoding unit 3025 decodes mesh patch information at the tile level into the syntax element tmsm_tile_in_submesh_flag[ tmsm_submesh_id[ i ] ][ tmsm_tile_id[ j ] ] may also be derived.

[0165] (Decoding the base mesh) 5 is a functional block diagram showing the configuration of the base mesh decoding unit 303. The base mesh decoding unit 303 includes a mesh decoding unit 3031, a motion information decoding unit 3032, a mesh motion compensation unit 3033, a reference The base mesh decoding unit 303 is configured with a reference mesh memory 3034, a switch 3035, a switch 3036, and a skip decoding unit 3037. The base mesh decoding unit 303 performs a base mesh decoding (not shown) before outputting the base mesh. The switches 3035 and 3036 may be configured to include a mesh dequantization unit. If the base mesh to be decoded has been coded (intra-coded) without reference to other base meshes (for example, base meshes that have already been coded and decoded), the switches 3035 and 3036 are connected to the mesh decoding unit 3031. If the base mesh to be decoded has been coded (inter-coded) with reference to other base meshes, the switches are connected to the side that performs motion compensation. When motion compensation is performed, the target vertex coordinates are derived by referencing already decoded vertex coordinates and motion information. If the base mesh to be decoded has been skipped and another base mesh has been coded (skip coded), the switches are connected to the skip decoding unit 3037.

[0166] Each base mesh is made up of one or more sub-meshes. If a mesh exists, the tile header in the atlas data sub-bitstream needs the ID to find the sub-mesh corresponding to the tile. A mesh is a subset of a mesh that is defined by specifying a part of the original model, and is a mesh that is created by dividing a mesh into multiple parts. By dividing the mesh into subsets, specific areas of the mesh can be defined individually. Each sub-mesh has its own vertex coordinates, normal vectors, texture coordinates, etc., and can be manipulated and edited individually. The mesh of a frame is called a mesh frame.

[0167] The mesh decoding unit 3031 decodes the intra-coded base mesh coded stream and outputs the base mesh (base mesh vertex positions, base mesh vertex position vectors). As the coding method, Draco, Edge Breaker, etc. are used.

[0168] The motion information decoding unit 3032 decodes the inter-coded base mesh coded stream and outputs motion information (mesh motion information, mesh motion vectors) for each vertex of a reference mesh (described later). Entropy coding such as arithmetic coding is used as the coding method.

[0169] The mesh motion compensation unit 3033 performs motion compensation on each vertex of the reference mesh input from the reference mesh memory 3034 based on the motion information, and outputs a motion-compensated mesh.

[0170] The reference mesh memory 3034 is a memory that holds the decoded mesh for reference in subsequent decoding processes.

[0171] (Mesh displacement decoding) FIG. 6 is a functional block diagram showing the configuration of the mesh displacement decoding unit 305. The mesh displacement decoding unit 305 is composed of a displacement unmapping unit 3052 (image unpacking unit, displacement decoding unit), an inverse quantization unit 3053, an inverse transformation unit 3054, and a coordinate system transformation unit 3055. As shown in the figure, the mesh displacement decoding unit 305 further The mesh displacement decoding unit 305 may include the decoding unit 3051, or the mesh displacement decoding unit 305 may include the video decoding unit 3051. Alternatively, the mesh displacement decoding unit 305 may be configured to use the video decoding device 31 to decode the displaced image (displacement sequence). Also, the mesh displacement decoding unit 305 does not include the inverse quantization unit 3053, and image quality control is performed only by the video decoding device 31. The following configuration may also be used.

[0172] The atlas information decoding unit 302 decodes the coordinate system transformation information displacementCoordinateSystem (mdu_displacement_coordinate_system) indicating the coordinate system from the coded data. The submesh division information of the displacement (displacement submesh division parameter, displacement segment parameter, displacement submesh division flag, displacement segment flag) may be decoded. The submesh division information is the displacementSubmeshFlag that indicates whether to divide into segments. Furthermore, the sub-mesh division information may include a component height origHeight. Furthermore, the sub-mesh division information may include a component width origWidth. In addition, the sub-mesh division information is a syntax that indicates the top position of each LOD of the mesh displacement. dispPos[lodIdx] or the number of mesh displacements per LOD, dispCount[lodIdx] The sub-mesh division information may also include an index dispCountIdx[lodIdx] indicating the number of mesh displacements. The sub-mesh division information may also include the size of the block for sub-mesh alignment, ctuSize, or an index ctuSizeIdx indicating ctuSize. The component height is a parameter indicating the height of the image corresponding to each component (e.g., n, t, b) of the mesh displacement 3D vector.

[0173] Alternatively, a gating flag may be provided separately, and each piece of coordinate system transformation information may be decoded only when the gating flag is 1. The gating flag may be, for example, afve_overriden_flag. Also, a gating flag may be provided in the sub-mesh division information, and the sub-mesh division information may be decoded only when the gating flag is 1. The gating flag may be, for example, For example, afve_displacement_submesh_alignment_flag.

[0174] (Coordinate system) The following two types of coordinate systems are used for mesh displacement (3D vector). Cartesian coordinate system (canonical): A rectangular coordinate system commonly defined throughout the entire 3D space. (X, Y, Z) coordinate system. A rectangular coordinate system whose direction does not change at the same time (within the same frame, within the same tile). Local coordinate system: A Cartesian coordinate system defined per region or per vertex in 3D space. A Cartesian coordinate system whose direction can change at the same time (within the same frame, within the same tile). A coordinate system with normal (D), tangent (U), and bi-tangent (V) axes. In other words, a certain vertex (a certain vertex) The first axis (D) is indicated by the normal vector n_vec on the surface containing the normal vector n_vec, and the second axis (U) and third axis (V) are indicated by two tangent vectors t_vec and b_vec that are perpendicular to the normal vector n_vec. n_vec, t_vec, b_vec are three-dimensional vectors. (D, U, V) coordinates The reference system may also be called the (n, t, b) coordinate system.

[0175] (Operation of mesh displacement decoding unit) The video decoder 3051 receives a geometry video stream (V3C_GVD) encoded by VVC, HEVC, or the like. The decoded image (mesh) is generated by decoding the (quantized) mesh displacement as pixel values. The output is a mesh displacement image, a mesh displacement array, and a color component of the geometry in DecGeoChromaFormat. The image may be in YCbCr4:2:0 format. The mesh displacement image may also be a transformed mesh displacement image, or a residual of the mesh displacement image.

[0176] The displacement unmapping unit 3052 generates mesh displacement from the mesh displacement image. Specifically, it derives mesh displacement dispQuantCoeffArray[v][d], a one-dimensional signal in component d units, from dispQuantCoeffFrame[x][y][d] of the two-dimensional mesh displacement image according to the correspondence of coordinate positions. Note that dispQuantCoeffFrame is a video The image array DecGeoFrames[mapIdx][frameIdx] or GeoFramesNF[mapIdx][compTimeIdx] may be decoded by a codec from the stream. Here, the correspondence of coordinate positions may be a Z-order scan in block units. NF is a nominal format ) and is the image after adjusting the image size, color sampling, etc. frameIdx and compTimeIdx are composition time indexes.

[0177] The displacement unmapper 3052 determines whether or not to use the one-dimensional displacement decoded from the encoded data. DisplacementDim is derived according to the value of the flag asve_1d_displacement_flag.

[0178] DisplacementDim = (asve_1d_displacement_flag) ? 1 : 3 Here, asve_1d_displacement_flag=1 means that only one dimension of the three-dimensional displacement is transmitted. Indicates that the normal component, or x-component (first component) of the displacement is present in the (compressed) geometry image. If the 1D flag is true, the displacement unmapper 3052 The other two components are assumed to be 0. asve_1d_displacement_flag=0 indicates that all three components of the displacement are present in the (compressed) geometry image.

[0179] The displacement unmapper 3052 generates coded data of the NAL units of the atlas, e.g., ASPS You can also decode the ctuSizeIdx (videoBlockSizeIdx) value and find the value 16< <ctuSizeIdxもしくは32<<ctuSizeIdx、64<<ctuSizeIdxから、ctuSize may be derived.

[0180] The displacement unmapper 3052 uses, for example, gi_geometry_codec_id[DecAtlasID] as follows: However, 64 may be used for HEVC and 128 for VVC.

[0181] ctuSize = ptl_profile_codec_group_idc == 3 (VVC) ? 128 : 64 Here, the value of ptl_profile_codec_group_idc is 0: AVC Progressive High, 1: HEVC Main 10 , 2:HEVC Main 444, 3:VVC Main 10.

[0182] ctuSize = gi_geometry_codec_id[DecAtlasID] 4CC code indicates HEVC ? 64 : 128 Here, the 4CC code strings indicating HEVC and VVC are "hev1" and "vvi1", respectively.

[0183] Alternatively, the displacement unmapper 3052 may be configured to set the maximum CTU size of HEVC to 64 and the maximum CTU size of VVC to 64. The larger of the maximum size values ​​of 128 may be fixed to 128.

[0184] In one configuration, the displacement unmapper 3052 receives a three-dimensional array dispQuantCoeffFrame of size asps_frame_width × asps_frame_height × DisplacementDim, variables patch2dSizeX, patch2dSizeY, patch2dPosX, patch2dPosY, bitDepth, subdivisionIterationCount, vertex Using the number of vertices verCoordCount and the array levelOfDetailVertexCounts of size subdivisionIterationCount + 1 as input, a two-dimensional array dispQuantCoeffArray of size verCoordCount × DisplacementDim is derived, which indicates the quantized displacement wavelet coefficients. The two-dimensional array dispQuantCoeffArray is initialized to 0. Also, a one-dimensional array of size subdivisionIterationCount + 1 is All elements of the column, vStart, vEnd, startBlock are initialized as 0 and the variable blockCount is set as 0.

[0185] where patch2dSizeX specifies the width of the patch's bounding box, patch2dSizeY specifies the height of the patch's bounding box, and patch2dPosX specifies the width of the patch's bounding box. Specify the x coordinate of the upper left corner of the bounding box of the patch, and patch2dPosY specifies the y coordinate of the upper left corner of the bounding box of the patch. Also, for example, from ASPS, set bitDepth=asps_geometry_3d_bit_depth_minus1+1, subdivisionIterationCount=asve_subdivision_iteration_count. The parameters may be derived: blockSize may be ctuSize.

[0186] Each variable may be derived as follows:

[0187] patchWidthInBlocks = (patch2dSizeX + blockSize - 1) / blockSize pixelsPerBlock = blockSize * blockSize shift = (1 << bitDepth) >> 1 if ( subdivisionIterationCount = 0 ){ vEnd

[0000] = verCoordCount blockCount = ( verCoordCount + blockSize - 1 ) / blockSize } else { for( i = 0; i < subdivisionIterationCount + 1; i++ ){ vStart[ i ] = i == 0? 0 : levelOfDetailVertexCounts[ i - 1 ] vEnd[ i ] = levelOfDetailVertexCounts[ i ] blockCountLevel [ i ] = (vEnd[ i ] - vStart[ i ] + blockSize - 1) / blockSize startBlock[ i ] = i == 0 ? 0 : (startBlock[ i - 1 ] + blockCount [ i ]) blockCount += blockCountLevel[ i ] } } Here, the variable dispPackingOrder is the syntax element asve_p decoded from the coded data. acking_method is set, and the variable videoChromaFormat is set to the variable DecGeoChromaFormat of the decoded geometry video component. dispQuantCoeffArray is set as follows: It may be derived.

[0188] heightInBlocks = (blockCount + widthInBlocks - 1) / patchWidthInBlocks origHeight = heightInBlocks * blockSize totalBlocksInPatch = ( patch2dSizeX * origHeight ) / pixelsPerBlock for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for( v = vStart[ lodIdx ]; v < vEnd[ lodIdx ]; v++ ) { blockIndex = (v-vStart[ lodIdx ]) / pixelsPerBlock + startBlock[ lodIdx ] indexWithinBlock = ( v - vStart[ lodIdx ] ) % pixelsPerBlock if( dispPackingOrder ){ blockIndex = totalBlocksInVideoFrame - 1 - blockIndex indexWithinBlock = pixelsPerBlock - 1 - indexWithinBlock } x0 = ( blockIndex % widthInBlocks ) * blockSize y0 = ( blockIndex / widthInBlocks ) * blockSize ( x, y ) = computeMorton2D( indexWithinBlock ) x1 = x0 + x + patch2dPosX y1 = y0 + y + patch2dPosX for( d = 0; d < DisplacementDim; d++ ) { if ( videoChromaFormat == 4:2:0 || videoChromaFormat == 4:2:2 || videoChromaFormat == 4:0:0 ) { if( dispPackingOrder ) dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ x1 ][ d * origHeight + y1 ]

[0000] - shift } else { dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ x1 ][ y1 ][ d ] - shift } } } } where asve_packing_method=0 means that the displacement component samples are packed in ascending order. asve_packing_method=1 indicates that the displacement component samples are packed in descending order. computeMorton2D is a function for implementing Z-order scanning and is defined below.

[0189] x = extracOddBits(x) { x = x & 0x55555555 x = (x | (x >> 1)) & 0x33333333 x = (x | (x >> 2)) & 0x0F0F0F0F x = (x | (x >> 4)) & 0x00FF00FF x = (x | (x >> 8)) & 0x0000FFFF } (x, y) = computeMorton2D(i) { x = extracOddBits(i>>1) y = extracOddBits(i) } Alternatively, the displacement unmapper 3052 may decode and derive the mesh displacement for each sub-mesh.

[0190] The displacement unmapper 3052 may add a conformance constraint that origHeight and / or origWidth derived from afve_displacement_component_height_submesh[i], afve_displacement_component_width_submesh[i], etc., must be an integer multiple of ctuSize.

[0191] heightInBlocks = (blockCount[submeshIdx]+widthInBlocks-1) / patchWidthInBlocks origHeight = heightInBlocks * blockSize In another configuration, the displacement unmapper 3052 does not decode or encode afve_displacement_component_width_submesh[i], which indicates the width of each component of the submesh corresponding to the index i of the mesh displacement divided into submeshes, but instead performs the submesh division as shown below. The width of each component of the sub-mesh corresponding to the index i of the mesh displacement is derived. You can put it out.

[0192] widthInBlocks = (blockCount[submeshIdx]+heightInBlocks-1) / patchHeightInBlocks origWidth = widthInBlocks * blockSize The inverse quantization unit 3053 performs inverse quantization based on the quantization scale value iscale to derive the mesh displacement dispCoeffArray after transformation (e.g., wavelet transformation). dispCoeffArray may be in a Cartesian coordinate system or a local coordinate system. iscale is a value derived from the quantization parameter of each component of the mesh displacement image.

[0193] Vcount0 = 0 for( i = 0; i < subdivisionIterationCount; i++ ) { vcount1 = levelOfDetailCounts[ i ] for( v = vcount0; v < vcount1; v++ ) { for( d = 0; d < DisplacementDim; d++ ) { dispCoeffArray[v][d] = dispQuantCoeffArray[ v ][ d ] * iscale[ i ][ d ] } } vcount0 = vcount1 } Here, iscale is derived as follows:

[0194] lodQuantizationFlag = vqp_lod_quantization_flag[ QpIndex ] directQuantizationEnableFlag = vqp_direct_quantization_enabled_flag[QpIndex] for( lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++ ) { for( dimIdx = 0; dimIdx < DisplacementDim; dimIdx ++ ) { iscale[ lodIdx ][ dimIdx ] = InverseScale[ QpIdx ][ lodIdx ][ dimIdx ] } } } else { for( dimIdx = 0; dimIdx < DisplacementDim; dimIdx++ ) { iscale

[0000] [ dimIdx ] = InverseScale[ QpIdx ]

[0000] [ dimIdx ] levelOfDetailInverseScale[ dimIdx ] = 1 << vqp_log2_lod_inverse_scale[ QpIdx ][ dimIdx ] } for( lodIdx = 1; lodIdx < lodCount; lodIdx++ ) { for( dimIdx = 0; dimIdx < DisplacementDim; dimIdx++ ) { iscale[ lodIdx ][ dimIdx ] = iscale[ lodIdx - 1 ][ dimIdx ] * levelOfDetailInverseScale[ dimIdx ] } } } The inverse transform unit 3054 performs an inverse transform g (for example, an inverse wavelet transform) to derive a mesh displacement d.

[0195] d[d][v] = g(dispCoeffArray[v][d]) The coordinate system conversion unit 3055 converts the mesh displacement (coordinate system of the mesh displacement) into a Cartesian coordinate system based on the value of the coordinate system conversion information displacementCoordinateSystem. Specifically, when displacementCoordinateSystem = 1, the displacement in the local coordinate system is converted into a displacement in the Cartesian coordinate system. Here, d is a three-dimensional vector indicating the mesh displacement before the coordinate system conversion. disp is a three-dimensional vector indicating the mesh displacement after the coordinate system conversion, and is a Cartesian coordinate system. n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target region or target vertex.

[0196] if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } The derivation method shown above for vector multiplication can be expressed individually as scalars as follows:

[0197] if (displacementCoordinateSystem == 0) { for (i = 0; i < 3; i++) { disp[i] = d[i] } } else if (displacementCoordinateSystem == 1){ for (i = 0; i < 3; i++) { disp[i] = d[0] * n_vec[i] + d[1] * t_vec[i] + d[2] * b_vec[i] } } Note that disp=d is used to assign the same variable name before and after the transformation, and the value of d is updated by the coordinate transformation. The configuration may be such that:

[0198] Alternatively, the following configuration may be used.

[0199] if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 2){ disp = d[0] * n_vec2 + d[1] * t_vec2 + d[2] * b_vec2 } Here, n_vec2, t_vec2, and b_vec2 are three-dimensional vectors (in the Cartesian coordinate system) corresponding to the axes of the local coordinate system of the adjacent region.

[0200] The following configuration may also be used.

[0201] if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } Here, n_vec3, t_vec3, b_vec3 are three-dimensional vectors (in Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target area where the fluctuation is suppressed. For example, the previous coordinates as follows: A vector of the coordinate system to be used for decoding is derived from the coordinate system and the current coordinate system.

[0202] n_vec3 = (w*n_vec3 + (WT - w)*n_vec)>>wShift t_vec3 = (w*t_vec3 + (WT - w)*t_vec)>>wShift b_vec3 = (w*b_vec3 + (WT - w)*b_vec)>>wShift Here, each variable can also have values such as wShift = 2, 3, 4, WT = 1<<wShift, w = 1..WT - 1, etc. For example, when w = 3 and wShift = 3, the following may also be applicable.

[0203] n_vec3 = (3*n_vec3 + 5*n_vec)>>3 t_vec3 = (3*t_vec3 + 5*t_vec)>>3 b_vec3 = (3*b_vec3 + 5*b_vec)>>3 Also, a configuration may be adopted such that it can be selected according to the value of the coordinate system conversion information displacementCoordinateSystem decoded from the encoded data, as in the following configuration.

[0204] if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 6){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } (Mesh Reconstruction) Figure 7 is a functional block diagram showing the configuration of the mesh reconstruction unit 307. The mesh reconstruction unit 307 is composed of a mesh division unit 3071 and a mesh deformation unit 3072.

[0205] The mesh dividing unit 3071 divides the base mesh output from the base mesh decoding unit 303. Divide the mesh into smaller pieces and generate a divided mesh.

[0206] FIG. 15(a) shows a part (triangle) of the base mesh, and the triangle is composed of vertices v1, v2, and v3. v1, v2, and v3 are three-dimensional vectors. The mesh division unit 3071 generates the divided mesh by adding new vertices v12, v13, and v23 to the middle of each side of the triangle. and output (Fig. 15(b)).

[0207] v12 = (v1 + v2) / 2 v13 = (v1 + v3) / 2 v23 = (v2 + v3) / 2 The following is also possible:

[0208] v12 = (v1 + v2 + 1) >> 1 v13 = (v1 + v3 + 1) >> 1 v23 = (v2 + v3 + 1) >> 1 The mesh deformation unit 3072 receives the division mesh and the mesh displacement, and outputs the mesh displacement d12, A deformed mesh is generated and output by adding d13 and d23 (FIG. 15(c)). The mesh displacement is the output of the mesh displacement decoding unit 305 (coordinate system conversion unit 3055). d12, d13, and d23 are mesh displacements corresponding to the vertices v12, v13, and v23 added by the mesh division unit 3071.

[0209] v12' = v12 + d12 v13' = v13 + d13 v23' = v23 + d23 It should be noted that d12 = disp[0][], d23 = disp[1][], and d23 = disp[3][] may also be used.

[0210] (Configuration of 3D data encoding device according to the first embodiment) FIG. 16 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11 according to the first embodiment. The 3D data encoding device 11 includes an atlas information encoding unit 101, a base mesh encoding unit 103, a base mesh decoding unit 104, a mesh displacement updating unit 106, a mesh displacement encoding unit 107, a mesh The 3D data encoding device 11 is composed of a mesh displacement decoding unit 108, a mesh reconstruction unit 109, an attribute update unit 110, a padding unit 111, a color space conversion unit 112, an attribute encoding unit 113, a multiplexing unit 114, and a mesh separation unit 115. The 3D data encoding device 11 inputs atlas information, a base mesh, a mesh displacement, a mesh, and an attribute image as 3D data, and outputs encoded data.

[0211] The atlas information encoding unit 101 encodes the atlas information and outputs the atlas information encoded stream. Output the program.

[0212] The base mesh encoding unit 103 encodes the base mesh and generates a base mesh encoding sequence. The stream is output using an encoding method such as Draco.

[0213] The base mesh decoding unit 104 is similar to the base mesh decoding unit 303, and therefore a description thereof will be omitted.

[0214] The mesh displacement update unit 106 updates the (original) base mesh and the decoded base mesh. Based on the mesh, adjust the mesh displacement and output the updated mesh displacement.

[0215] The mesh displacement encoding unit 107 encodes the updated mesh displacement and generates a mesh displacement code The encoded stream is output using VVC, HEVC, or other encoding methods.

[0216] The mesh displacement decoding unit 108 is similar to the mesh displacement decoding unit 305, and therefore a description thereof will be omitted.

[0217] The mesh reconstruction unit 109 is similar to the mesh reconstruction unit 307, and therefore a description thereof will be omitted.

[0218] The attribute update unit 110 inputs the (original) mesh, the reconstructed mesh output from the mesh reconstruction unit 109 (mesh deformation unit 3072), and the attribute image, updates the attribute image to match the position (coordinates) of the reconstructed mesh, and outputs the updated attribute image.

[0219] The padding unit 111 receives the attribute image and pads the pixel values ​​in the empty area. Perform the processing.

[0220] The color space conversion unit 112 performs color space conversion from the RGB format to the YCbCr format.

[0221] The attribute encoding unit 113 encodes the YCbCr formatted image data output from the color space conversion unit 112. The attribute image is encoded and an attribute video stream is output. The encoding method is VVC or HEVC.

[0222] The multiplexing unit 114 multiplexes the atlas information coded stream, the base mesh coded stream, The mesh displacement coding stream and the attribute video stream are multiplexed and output as coded data. Multiplexing methods include byte stream format and ISOBMFF. is used.

[0223] (Mesh separation unit operation) The mesh separation unit 115 generates a base mesh and a mesh displacement from the mesh.

[0224] 20 is a functional block diagram showing the configuration of the mesh separation unit 115. The mesh separation unit 115 is made up of a mesh thinning unit 1151, a mesh division unit 1152, and a mesh displacement derivation unit 1153.

[0225] The mesh thinning unit 1151 generates a base mesh by thinning out some of the vertices from the mesh.

[0226] Figure 21(a) shows a part of a mesh, which has vertices v1, v2, v3, v4, v5, 21(b)).

[0227] The mesh dividing unit 1152 divides the base mesh to generate divided meshes, similar to the mesh dividing unit 3071 (FIG. 21(c)).

[0228] v4' = (v1 + v2) / 2 v5' = (v1 + v3) / 2 v6' = (v2 + v3) / 2 The mesh displacement calculation part calculates the displacement for vertices v4', v5', and v6' based on the mesh and the divided mesh. The displacements d4, d5, and d6 of the vertices v4, v5, and v6 are calculated and output as mesh displacements (Fig. 21(d)). .

[0229] d4 = v4 - v4' d5 = v5 - v5' d6 = v6 - v6' (Encoding of atlas information) 17 is a functional block diagram showing the configuration of the atlas information encoding unit 101. The encoding unit 101 includes a mesh patch information encoding unit 1011, a sub-mesh information encoding unit 1012, and an extension information encoding unit 1013. The encoding unit 1014 is made up of a shared information encoding unit 1013, a tile information encoding unit 1014, and a parameter encoding unit 1015.

[0230] The mesh patch information encoding unit 1011 encodes mesh patch information including mesh patch data.

[0231] The mesh patch information encoding unit 1011 may encode the correspondence between sub-mesh information having an arbitrary sub-mesh ID and a mesh patch having a certain patch index.

[0232] The submesh information encoding unit 1012 encodes the number of submeshes and submesh IDs referenced at the picture / frame level.

[0233] The submesh information encoding unit 1012 may encode the number of submeshes and submesh IDs referenced at the tile level of the picture / frame, or if the referenced submesh includes tiles, may encode the number of referenced tiles and tile IDs so that they correspond to the submesh IDs.

[0234] The submesh information encoding unit 1012 encodes tile information having an arbitrary tile ID and an arbitrary submesh information. The correspondence between the sub-mesh information and the mesh ID may be coded.

[0235] The extended information encoding unit 1013 encodes extended encoding parameters related to the mesh data.

[0236] The tile information encoding unit 1014 encodes the number of tiles and tile IDs referenced at the picture / frame level.

[0237] The parameter encoding unit 1015 encodes encoding parameters related to the 3D data.

[0238] (Base mesh encoding) FIG. 18 is a functional block diagram showing the configuration of the base mesh encoding unit 103. The mesh encoding unit 103 is composed of a mesh encoding unit 1031, a mesh decoding unit 1032, a motion information encoding unit 1033, a motion information decoding unit 1034, a mesh motion compensation unit 1035, a reference mesh memory 1036, a switch 1037, and a switch 1038. The base mesh encoding unit 103 may also include a base mesh quantization unit (not shown) after the base mesh is input. When encoding a base mesh without referring to other base meshes (e.g., an already encoded base mesh) (intra-coding), the switches 1037 and 1038 are connected to the side that does not perform motion compensation. When encoding a base mesh with reference to other base meshes (inter-coding), the switches 1037 and 1038 are connected to the side that performs motion compensation.

[0239] The mesh encoding unit 1031 has an intra-encoding function, performs intra-encoding on the base mesh, and outputs a base mesh encoded stream. There are.

[0240] The mesh decoding unit 1032 is similar to the mesh decoding unit 3031, and therefore a description thereof will be omitted.

[0241] The motion information encoding unit 1033 has an inter-encoding function, performs inter-encoding on the base mesh, and outputs a base mesh encoded stream. The encoding method used is entropy encoding such as arithmetic encoding.

[0242] The motion information decoding unit 1034 is similar to the motion information decoding unit 3032, and therefore a description thereof will be omitted.

[0243] The mesh motion compensation unit 1035 is similar to the mesh motion compensation unit 3033, and therefore a description thereof will be omitted.

[0244] The reference mesh memory 1036 is similar to the reference mesh memory 3034, and therefore a description thereof will be omitted.

[0245] (Mesh displacement encoding) 19 is a functional block diagram showing the configuration of the mesh displacement encoding unit 107. The encoding unit 107 includes a coordinate system conversion unit 1071, a conversion unit 1072, a quantization unit 1073, and a displacement mapping unit 1074. As shown in the figure, the mesh displacement coding unit 107 may further include a moving image coding unit 1075. Alternatively, the moving image coding unit 1075 may include The mesh displacement encoding unit 107 does not include the mesh displacement encoding unit 107, and the encoding of the displaced image is performed by an external image encoding device. It may be configured to use

[0246] The coordinate system conversion unit 1071 converts the coordinate system of the mesh displacement from a Cartesian coordinate system to a coordinate system that encodes the displacement (for example, a local coordinate system) based on the value of the coordinate conversion information displacementCoordinateSystem. Here, disp is a three-dimensional vector indicating the mesh displacement before the coordinate system conversion, d is a three-dimensional vector indicating the mesh displacement after the coordinate system conversion, and n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) indicating each axis of the local coordinate system.

[0247] if (displacementCoordinateSystem == 0) { d = disp } else if (displacementCoordinateSystem == 1){ d = (disp * n_vec, disp * t_vec, disp * b_vec) } The mesh displacement coding unit 107 encodes the value of the displacementCoordinateSystem into the picture / frame. It may be updated at the system level.

[0248] When encoding displacementCoordinateSystem at the sequence level, the sequence of the configuration in Figure 9 The asve_displacement_coordinate_system uses the Cartesian coordinate system. Set to 0 if the coordinate system is in the local coordinate system, and 1 if the coordinate system is in the local coordinate system.

[0249] When changing the displacementCoordinateSystem at the picture / frame level, use the syntax in the configuration in Figure 12. For afve_overriden_flag, set 1 if you want to update the coordinate system, or 0 if you do not want to update the coordinate system. For afve_displacement_coordinate_system, set 0 if you want to use a Cartesian coordinate system, or 1 if you want to use a local coordinate system.

[0250] The transformation unit 1072 performs a transformation f (for example, a wavelet transformation) to derive the transformed mesh displacement Tdisp, for pos=0..NumDisp-1, where NumDisp is the mesh vertex displacement. Number of points.

[0251] dispCoeffArray[v][d] = f(d[d][v]) The quantization unit 1073 performs quantization based on the quantization scale value "scale" derived from the quantization parameter of each component of the mesh displacement, and derives the mesh displacement dispQuantCoeffArray after quantization.

[0252] Vcount0 = 0 for( i = 0; i < subdivisionIterationCount; i++ ) { vcount1 = levelOfDetailCounts[ i ] for( v = vcount0; v < vcount1; v++ ) { for( d = 0; d < DisplacementDim; d++ ) { dispQuantCoeffArray[v][d] = dispCoeffArray[ v ][ d ] / iscale[ i ][ d ]} } vcount0 = vcount1 } Alternatively, you can approximate the scale value as a power of 2 and derive dispQuantCoeffArray using the following formula: .

[0253] Vcount0 = 0 for( i = 0; i < subdivisionIterationCount; i++ ) { scale[i] = 1 << scale2[i] vcount1 = levelOfDetailCounts[ i ] for( v = vcount0; v < vcount1; v++ ) { for( d = 0; d < DisplacementDim; d++ ) { dispQuantCoeffArray[v][d] = dispCoeffArray[v][d] >> scale2[i][d] } } vcount0 = vcount1 } The displacement mapping unit 1074 generates an image dispQuantCoeffFrame from the quantized mesh displacements dispQuantCoeffArray based on the value of the displacement mapping parameter displacementChromaLocationType.

[0254] The displacement mapper 1074 calculates the first component of the (quantized) mesh displacement array as follows: The min dispQuantCoeffArray[v][0] may be mapped to the luminance (Y) image component. For image width W and height H (y=0..H-1, x=0..W-1), apply the following:

[0255] H = origHeight shift = (1 << bitDepth) >> 1 dispQuantCoeffFrame[x][ y][0] = dispQuantCoeffArray[v][0] + shift dispQuantCoeffFrame[x][ H+y][0] = dispQuantCoeffArray[v][1] + shift dispQuantCoeffFrame[x][2*H+y][0] = dispQuantCoeffArray[v][2] + shift v++ dispQuantCoeffFrame[x / 2][ y / 2][1] = shift dispQuantCoeffFrame[x / 2][H / 2+y / 2][1] = shift dispQuantCoeffFrame[x / 2][ H+y / 2][1] = shift dispQuantCoeffFrame[x / 2][ y / 2][2] = shift dispQuantCoeffFrame[x / 2][H / 2+y / 2][2] = shift dispQuantCoeffFrame[x / 2][ H+y / 2][2] = shift Alternatively, the displacement mapping unit 1074 may encode the mesh displacement for each sub-mesh.

[0256] The processing may be switched depending on DecGeoChromaFormat. That is, when DecGeoChromaFormat=1 (4:2:0), the above processing is performed, and when DecGeoChromaFormat=3 (4:4:4), the following processing is performed.

[0257] dispQuantCoeffFrame[x][y][d] = dispQuantCoeffArray[v][0] dispQuantCoeffFrame[x][y][d] = dispQuantCoeffArray[v][1] dispQuantCoeffFrame[x][y][d] = dispQuantCoeffArray[v][2] v++ The mesh displacement encoding unit 107 may update the values ​​of origHeight and origWidth at the picture / frame level.

[0258] The video encoding unit 1075 encodes the image in YCbCr4:2:0 format including the (quantized) mesh displacement image, and outputs a mesh displacement encoded stream. The encoding method used is VVC, HEVC, or the like.

[0259] The video encoding unit 1075 may divide the mesh displaced image into slices for each origHeight and encode the slices. Alternatively, the origHeight may be aligned to a predetermined size according to the CTU size. .

[0260] The video encoding unit 1075 converts a first component (e.g., D) of the mesh displacement into a first slide. slice, the second component (e.g., U) to the second slice, the third component (e.g., V ) may be assigned to the third slice and coded (displacementSliceType=1).

[0261] Alternatively, the video encoding unit 1075 may encode the mesh displacement by allocating the first component to the first slice and the second and third components to the second slice (displacementSliceType=2).

[0262] As described above, by assigning different slices to each component of the mesh displacement, the decoding device can decode only some of the components, thereby simplifying the processing. Furthermore, the decoding device can decode slices including the second and third components of the mesh displacement as needed, thereby realizing scalability. Furthermore, even if an error is introduced into the encoded data, the decoding device can decode only the slices (components) without errors, thereby enhancing error resilience.

[0263] One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like can be made within the scope that does not deviate from the gist of the present invention.

[0264] [Application example] The above-described 3D data encoding device 11 and 3D data decoding device 31 can be mounted on various devices that transmit, receive, record, and play back 3D data. The 3D data may be natural 3D data captured by a camera or the like, or artificial 3D data (including CG and GUI) generated by a computer or the like.

[0265] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. In other words, embodiments obtained by combining technical means modified appropriately within the scope of the claims are also included in the technical scope of the present invention. [Industrial Applicability]

[0266] The embodiments of the present invention can be suitably applied to a 3D data decoding device that decodes coded data in which 3D data has been coded, and a 3D data coding device that generates coded data in which 3D data has been coded, and can also be suitably applied to the data structure of coded data that is generated by the 3D data coding device and referenced by the 3D data decoding device. [Explanation of symbols]

[0267] 11 3D data encoding device 101 Atlas Information Encoding Unit 1011 Mesh patch information encoding unit 1012 Submesh information encoding unit 1013 Extended information encoding unit 1014 Tile information encoding unit 1015 Parameter Encoding Unit 103 Base mesh coding unit 1031 Mesh coding unit 1032 Mesh Decoding Unit 1033 Motion information encoding unit 1034 Motion information decoding unit 1035 Mesh motion compensation unit 1036 reference mesh memory 1037 Switch 1038 Switch 1039 Skip Coding 104 Base mesh decoding unit 106 Mesh displacement update section 107 Mesh displacement coding unit 1071 Coordinate system conversion unit 1072 Conversion Unit 1073 Quantization section 1074 Binarization section 1075 Arithmetic encoder 1076 Context Selection 1077 Context Initialization Unit 108 Mesh displacement decoding unit 109 Mesh reconstruction unit 110 Attribute Update Section 111 Padding section 112 Color space conversion unit 113 Attribute Encoding Unit 114 Multiplexer 115 mesh separation section 1151 Mesh thinning section 1152 Mesh division section 1153 Mesh displacement derivation part 21 Network 31 3D data decoding device 301 Demultiplexer 302 Atlas Information Decoding Unit 3021 Parameter Decoding Unit 3022 Tile information decoding unit 3023 Extended Information Decoding Unit 3024 Submesh information decoding unit 3025 Mesh patch information decoding unit 303 Base mesh decoding unit 3031 Mesh Decoding Unit 3032 Motion information decoding unit 3033 Mesh Motion Compensation Unit 3034 Reference Mesh Memory 3035 Switch 3036 Switch 3037 Skip Decoding Unit 305 Mesh displacement decoding unit 3051 Arithmetic Decoding Unit 3052 Multilevel conversion section 3053 Inverse quantization section 3054 Inverse Conversion Unit 3055 Coordinate system conversion unit 3056 Context Selection Section 3057 Context Initialization Unit 307 Mesh Reconstruction Unit 306 Attribute Decoding Unit 3071 Mesh division section 3072 Mesh deformation part 308 Color Space Conversion Unit 41 3D data display device

Claims

1. A 3D data decoding device that decodes mesh data or point cloud data comprises an atlas information decoding unit that decodes atlas information from encoded data in which the mesh data or point cloud data is encoded, and a mesh reconstruction unit that decodes a mesh from the encoded data and the atlas information, wherein the mesh reconstruction unit decodes an arbitrary mesh / submesh from the encoded data using a parameter that indicates the correspondence between tile information having an arbitrary tile ID decoded by the atlas information decoding unit and submesh information having an arbitrary submesh ID.

2. 3D data decoding device according to claim 1, characterized in that the atlas information includes tile information, submesh information, and mesh patch information, the atlas information decoding unit comprises a submesh information decoding unit and a mesh patch information decoding unit, and the mesh patch information decoding unit decodes the correspondence between the tile information and the mesh patch information using a flag indicating a value indicating a correspondence if tile information having an arbitrary tile ID decoded by the submesh information decoding unit and submesh information having an arbitrary submesh ID correspond to each other, and a value indicating a non-correspondence if there is no correspondence, thereby deriving the correspondence between the tile information, submesh information, and mesh patch information.

3. The 3D data decoding device according to claim 1, characterized in that the atlas information includes tile information, submesh information, and mesh patch information, and the atlas information decoding unit includes a mesh patch information decoding unit that decodes tile information having an arbitrary tile ID and an index of submesh information corresponding to a mesh patch having a certain patch index, and derives a correspondence relationship between the tile information, submesh information, and mesh patch information.

4. 2. The 3D data decoding device according to claim 1, wherein the atlas information includes tile information, submesh information, and mesh patch information, and the atlas information decoding unit includes a mesh patch information decoding unit, which decodes a value indicating that submesh information having an arbitrary submesh ID and a mesh patch having a certain patch index correspond to each other if they correspond to each other, and decodes a flag indicating a value indicating that they do not correspond to each other if they do not correspond to each other, thereby deriving the correspondence between the tile information, submesh information, and mesh patch information.

5. A 3D data encoding device that encodes mesh data or point cloud data, comprising: a mesh separation unit that separates meshes; and an atlas information encoding unit that encodes atlas information, wherein the atlas information includes parameters that indicate the correspondence between tile information having an arbitrary tile ID and submesh information having an arbitrary submesh ID, and the mesh separation unit encodes the mesh / submesh using the parameters.

6. 6. The 3D data encoding device according to claim 5, wherein the atlas information includes tile information, submesh information, and mesh patch information, the atlas information encoding unit includes a submesh information encoding unit and a mesh patch information encoding unit, the mesh patch information includes a flag indicating a value representing a correspondence between tile information having an arbitrary tile ID and submesh information having an arbitrary submesh ID when they correspond to each other, and a value representing a non-correspondence when they do not correspond to each other, and the mesh patch information encoding unit encodes the correspondence between the tile information and the mesh patch information using the mesh patch information.

7. the atlas information includes tile information, sub-mesh information, and mesh patch information; The 3D data encoding device according to claim 5, characterized in that the atlas information encoding unit includes a mesh patch information encoding unit, and the mesh patch information encoding unit encodes tile information having an arbitrary tile ID and an index of sub-mesh information corresponding to a mesh patch having a certain patch index.

8. The 3D data encoding device of claim 5, characterized in that the atlas information includes tile information, submesh information, and mesh patch information, the atlas information encoding unit includes a mesh patch information encoding unit, and the mesh patch information encoding unit encodes a value indicating that submesh information having an arbitrary submesh ID and a mesh patch having a certain patch index correspond to each other if they correspond to each other, and encodes a value indicating that they do not correspond to each other if they do not correspond to each other.

Images