Mesh decoding device, mesh decoding method, and program

Abstract

The present invention performs mesh decoding by adding, to a base mesh, the amount of displacement obtained by arithmetic decoding. A mesh decoding device 200 according to the present invention comprises a mesh reconstruction unit 206B which is configured to reconstruct a mesh on the basis of an atlas, a base mesh, and a level value image or a first coefficient sequence that are obtained from a bit stream, and output the mesh. The mesh reconstruction unit 206 is provided with a patch association unit 206A which is configured to use the atlas, the base mesh, and the level value image or the first coefficient sequence to generate and output an associated base mesh and an associated level value image or an associated first coefficient sequence, and a submesh reconstruction unit 206B which is configured to use the associated base mesh and the associated level value image or the associated first coefficient sequence to generate and output a submesh sequence.

Description

Mesh decoding device, mesh decoding method and program

[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program.

[0002] Non-Patent Document 1 discloses a technique for dividing a point cloud into partial patches and decoding the divided patches.

[0003] Non-Patent Document 2 discloses a technique that extends Non-Patent Document 1 to mesh decoding. A mesh is decoded by dividing it into a rough basic mesh and a detailed displacement. The mesh is decoded by dividing it into partial sub-meshes. The displacement is decoded by a video codec or arithmetic decoding.

[0004] “Information technology - Coded Representation of Immersive Media - Part 5: Visual Volumetric Video-based Coding (V3C) and Video-based Point Cloud Compression (V-PCC),” ISO / IEC JTC 1 / SC 29 / WG 7, ISO / IEC 23090-5:2021(2E). “WD 5.0 ​​of V-DMC,” ISO / IEC JTC 1 / SC 29 / WG 7 N00744.

[0005] However, the method of Non-Patent Document 2 has a problem in that when decoding the displacement amount by arithmetic decoding, there is no process for decoding the mesh from the basic mesh and the displacement amount, and therefore the mesh cannot be decoded.

[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a mesh decoding device, a mesh decoding method, and a program that enable decoding of a mesh by adding the amount of displacement decoded by arithmetic decoding to a basic mesh.

[0007] A first feature of the present invention is summarized as a mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a basic mesh, and a level value image or a first coefficient sequence obtained from a bitstream, wherein the mesh reconstruction unit comprises: a patch correspondence unit configured to generate and output a corresponding base mesh and a corresponding level value image or a corresponding first coefficient sequence using the atlas, the base mesh, and the level value image or the first coefficient sequence; and a submesh reconstruction unit configured to generate and output a submesh sequence using the corresponding base mesh and the corresponding level value image or the corresponding first coefficient sequence.

[0008] A second feature of the present invention is summarized as a mesh decoding method comprising a step of reconstructing and outputting a mesh based on an atlas, a basic mesh, and a level value image or a first coefficient sequence obtained from a bitstream, wherein in the step, a corresponding basic mesh and a corresponding level value image or a corresponding first coefficient sequence are generated and output using the atlas, the basic mesh, and the level value image or the first coefficient sequence, and a sub-mesh sequence is generated and output using the corresponding basic mesh and the corresponding level value image or the corresponding first coefficient sequence.

[0009] A third feature of the present invention is summarized as a program for causing a computer to function as a mesh decoding device, the mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a basic mesh, and a level value image or a first coefficient sequence obtained from a bitstream, the mesh reconstruction unit comprising: a patch correspondence unit configured to generate and output a corresponding basic mesh and a corresponding level value image or a corresponding first coefficient sequence using the atlas, the basic mesh, and the level value image or the first coefficient sequence; and a submesh reconstruction unit configured to generate and output a submesh sequence using the corresponding basic mesh and the corresponding level value image or the corresponding first coefficient sequence.

[0010] According to the present invention, it is possible to provide a mesh decoding device, a mesh decoding method and a program that are capable of decoding a mesh by adding the amount of displacement decoded by arithmetic decoding to a basic mesh.

[0011] FIG. 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment. FIG. 2 is a diagram showing an example of functional blocks of a mesh decoding device 200 according to an embodiment. FIG. 3 is a diagram showing an example of the configuration of a V3C unit. FIG. 4 is a diagram showing an example of the syntax configuration of a V3C unit header. FIG. 5 is a diagram showing an example of the configuration of a bitstream. FIG. 6 is a diagram showing an example of the syntax configuration of a V3C parameter set. FIG. 7 is a diagram showing an example of the syntax configuration of profile_tier_level. FIG. 8 is a diagram showing an example of the syntax configuration of afps_vdmc_extension. FIG. 9 is a diagram showing an example of the syntax configuration of atlas_frame_displ_information. FIG. 10A is a diagram showing an example of afdi_displ_id. FIG. 10B is a diagram showing an example of DisplIDToIndex in the example of FIG. 10A. Fig. 10C is a diagram showing an example of DisplIndexToID in the example of Fig. 10A. Fig. 11 is a diagram showing an example of the syntax configuration of patch_data_unit. Fig. 12 is a diagram showing an example of the configuration of a displacement bitstream. Fig. 13 is a diagram showing an example of the syntax configuration of displ_nal_unit_header. Fig. 14 is a diagram showing an example of the correspondence between displ_nal_unit_type and the contents of the payload. Fig. 15 is a diagram showing an example of the syntax configuration of displ_layer_rbsp. Fig. 16 is a diagram showing an example of the syntax configuration of displ_header. Fig. 17 is a diagram showing an example of functional blocks of the displacement decoding unit 204 of the mesh decoding device 200 according to an embodiment. Fig. 18 is a diagram showing an example of functional blocks of a modified example of the displacement decoding unit 204 of the mesh decoding device 200 according to an embodiment. Fig. 19 is a diagram showing an example of functional blocks of the mesh reconstruction unit 206 of the mesh decoding device 200 according to an embodiment. Fig. 20 is a diagram showing an example of functional blocks of the patch association unit 206A of the mesh reconstruction unit 206 of the mesh decoding device 200 according to an embodiment.Fig. 21 is a diagram showing an example of functional blocks of the sub-mesh reconstruction unit 206B of the mesh reconstruction unit 206 of the mesh decoding device 200 according to an embodiment. Fig. 22 is a flowchart showing an example of processing by the mesh reconstruction unit 206 of the mesh decoding device 200 according to an embodiment.

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0013] First Embodiment This embodiment is based on V3C described in Non-Patent Document 1. The definitions of atlas, patch, and tile in V3C are shown below.

[0014] An atlas is a collection of rectangular regions and associated information arranged on a frame, and corresponds to a three-dimensional space.

[0015] A patch is a rectangular region that corresponds to the three-dimensional information contained in the atlas.

[0016] A tile is an independent, decodable rectangular region in the atlas, which is a different concept from a rectangular region in a patch.

[0017] Hereinafter, a mesh processing system 1 according to this embodiment will be described with reference to FIGS.

[0018] 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to this embodiment. As shown in FIG. 1, the mesh processing system 1 includes a mesh encoding device 100 and a mesh decoding device 200.

[0019] FIG. 2 is a diagram showing an example of functional blocks of a mesh decoding device 200 according to this embodiment.

[0020] As shown in FIG. 2, the mesh decoding device 200 includes a V3C unit separation unit 201, an atlas decoding unit 202, a basic mesh decoding unit 203, a displacement amount decoding unit 204, a texture decoding unit 205, and a mesh reconstruction unit 206.

[0021] The V3C unit separation unit 201 is configured to separate the bitstream into an atlas bitstream, a basic mesh bitstream, a displacement bitstream, and a texture bitstream.

[0022] The atlas decoding unit 202 is configured to decode and output the atlas from the atlas bitstream.

[0023] The base mesh decoding unit 203 is configured to decode and output base meshes from the base mesh bitstream.

[0024] The displacement amount decoding unit 204 is configured to decode and output the level value image or the first coefficient sequence from the displacement amount bit stream.

[0025] The texture decoding unit 205 is configured to decode and output the texture from the texture bitstream. The texture decoding unit 205 may decode the texture using HEVC.

[0026] The mesh reconstruction unit 206 is configured to reconstruct and output a mesh using the atlas decoded by the atlas decoding unit 201, the basic mesh decoded by the basic mesh decoding unit 203, and the level value image or first coefficient sequence decoded by the displacement amount decoding unit 204.

[0027] (Bitstream Structure) A bitstream may be formed by combining a plurality of V3C units.

[0028] 3 is a diagram showing an example of the configuration of a V3C unit. A V3C unit may be composed of a V3C unit header and a V3C unit payload.

[0029] 4 is a diagram showing an example of the syntax configuration of a V3C unit header. The V3C unit header may include vuh_unit_type, which indicates the type of the V3C unit.

[0030] 5 is a diagram showing an example of the configuration of a bitstream. The bitstream may be composed of a V3C parameter set, an atlas bitstream, a base mesh bitstream, a displacement bitstream, and a texture bitstream.

[0031] vuh_unit_type may store any of the following: V3C_VPS indicating a V3C parameter set, V3C_AD indicating an atlas bitstream, V3C_BMD indicating a basic mesh bitstream, V3C_GVD indicating a displacement bitstream, and V3C_AVD indicating a texture bitstream.

[0032] 6 is a diagram showing an example of the syntax configuration of a V3C parameter set. The V3C parameter set may include profile_tier_level indicating profile information and the like.

[0033] 7 is a diagram showing an example of the syntax configuration of profile_tier_level. Profile_tier_level may include ptl_profile_codec_group_idc indicating a compression method for the amount of displacement.

[0034] It may be defined that when ptl_profile_codec_group_idc is 0, the displacement amount bitstream contains compressed data of the displacement amount obtained by arithmetic decoding, and when ptl_profile_codec_group_idc is 1, the displacement amount bitstream contains compressed data of the displacement amount obtained by a video codec.

[0035] (Configuration of Atlas Bitstream) The atlas bitstream may use afps_vdmc_extension, which indicates frame parameters that are an extension of the V3C frame parameter set.

[0036] FIG. 8 is a diagram illustrating an example of the syntax configuration of afps_vdmc_extension.

[0037] The afps_vdmc_extension may include atlas_frame_displ_information indicating information on the amount of displacement.

[0038] FIG. 9 is a diagram illustrating an example of the syntax configuration of atlas_frame_displ_information.

[0039] atlas_frame_displ_information may include afdi_use_single_displ_flag indicating whether a single displacement amount is used, afdi_num_displs_minus1 indicating a value obtained by subtracting 1 from the number of displacement amounts, afdi_signaled_displ_id_flag indicating whether a displacement amount ID is transmitted, afdi_signaled_displ_id_length_minus1 indicating a value obtained by subtracting 1 from the number of bits representing the displacement amount ID, and afdi_displ_id indicating the displacement amount ID.

[0040] 10A is a diagram showing an example of afdi_displ_id, in which the 0th afdi_displ_id is 101, the 1st afdi_displ_id is 201, and the 2nd afdi_displ_id is 001.

[0041] The atlas_frame_displ_information may use afdi_displ_id to create a DisplayIDToIndex that indicates a correspondence table that allows a displacement amount index to be referenced from a displacement amount ID.

[0042] Fig. 10B is a diagram showing an example of DisplayIDToIndex in the example of Fig. 10A. Atlas_frame_displ_information may use afdi_displ_id to create DisplayIndexToID indicating a correspondence table that allows a displacement amount ID to be referenced from a displacement amount index.

[0043] Fig. 10C is a diagram showing an example of DisplIndexToID in the example of Fig. 10A. The atlas bitstream may use an extended patch_data_unit indicating a V3C data unit.

[0044] FIG. 11 is a diagram illustrating an example of the syntax configuration of patch_data_unit.

[0045] The patch_data_unit may be defined for each tile identified by tileID. The patch_data_unit may be defined for each patch identified by patchIdx.

[0046] The patch_data_unit may include a pdu_submesh_id indicating the submesh ID associated with the patch and a pdu_displ_id indicating the displacement ID associated with the patch, thereby enabling the correspondence between the submesh ID and the displacement ID.

[0047] patch_data_unit may include pdu_2d_pos_x indicating the x-coordinate of the top left corner of the rectangular area of ​​the patch, pdu_2d_pos_y indicating the y-coordinate of the top left corner of the rectangular area of ​​the patch, pdu_2d_size_x_minus1 indicating the value obtained by subtracting 1 from the horizontal width of the rectangular area of ​​the patch, and pdu_2d_size_y_minus1 indicating the value obtained by subtracting 1 from the height of the rectangular area of ​​the patch.

[0048] patch_data_unit may include pdu_attributes_2d_pos_x indicating the x-coordinate of the top left corner of the rectangular area in the texture image of the patch, pdu_attributes_2d_pos_y indicating the y-coordinate of the top left corner of the rectangular area in the texture image of the patch, pdu_attributes_2d_size_x_minus1 indicating the value obtained by subtracting 1 from the width of the rectangular area in the texture image of the patch, and pdu_attributes_2d_size_y_minus1 indicating the value obtained by subtracting 1 from the height of the rectangular area in the texture image of the patch.

[0049] (Configuration of Displacement Amount Bitstream) The displacement amount bitstream may include data of the displacement amount compressed by arithmetic decoding or data of the displacement amount compressed by a video codec.

[0050] FIG. 12 is a diagram showing an example of the configuration of a displacement amount bit stream.

[0051] When the displacement bitstream includes compressed data of the displacement obtained by arithmetic decoding, the V3C unit payload may be configured with displ_nal_unit indicating the NAL unit.

[0052] The displ_nal_unit may be composed of a displ_nal_unit_header indicating the header of the NAL unit in the displacement amount bitstream and the payload of the NAL unit in the displacement amount bitstream.

[0053] FIG. 13 is a diagram illustrating an example of the syntax configuration of the display_nal_unit_header.

[0054] The display_nal_unit_header may be composed of a display_nal_forbidden_zero_bit indicating the leading write-protected section and a display_nal_unit_type indicating the type of payload data.

[0055] FIG. 14 is a diagram showing an example of the correspondence between the display_nal_unit_type and the contents of the payload.

[0056] The contents of the payload may be defined as displ_sequence_parameter_set_rbsp when displ_nal_unit_type is 0, as displ_frame_parameter_set_rbsp when displ_nal_unit_type is 1, and as displ_layer_rbsp when displ_nal_unit_type is 2.

[0057] FIG. 15 is a diagram illustrating an example of the syntax configuration of display_layer_rbsp.

[0058] The display_layer_rbsp may include a display_header indicating the header of the displacement arithmetic decoding, a display_data_unit indicating the data of the displacement arithmetic decoding, and an rbsp_trailing_bit indicating the trailing zero bits of the displacement arithmetic decoding.

[0059] 16 is a diagram showing an example of the syntax configuration of the displ_header. The displ_header may include a dh_id indicating an ID of the displacement amount.

[0060] (Displacement amount decoding unit 204) As shown in Figure 17, the displacement amount decoding unit 204 includes an arithmetic decoding unit 204A, an adder 204B, a frame buffer 204C, an inter prediction unit 204D, a first inverse quantization unit 204E, and a video decoding unit 204F.

[0061] The video decoding unit 204F is configured to decode the level value image from the displacement amount bitstream using a video codec and output the level value image. The video decoding unit 204F may decode the level value image using HEVC.

[0062] The arithmetic decoding unit 204A is configured to decode and output a level value sequence from the displacement amount bit stream by arithmetic decoding. The arithmetic decoding unit 204A may decode the level value sequence as described in Japanese Patent Application No. 2022-212452.

[0063] The frame buffer 204C is configured to acquire and store a level value sequence from the arithmetic decoding unit 204A or the adder 204B. The frame buffer 204C is configured to output a level value sequence in a reference frame in accordance with control information (not shown).

[0064] The inter prediction unit 204D is configured to perform inter prediction using the level value sequence of the reference frame read out from the frame buffer 204C, thereby generating and outputting a predicted level value sequence.

[0065] The inter prediction unit 204D may determine the predicted level value of each frequency in the current frame by directly referring to the level value of the corresponding frequency in the reference frame.

[0066] The adder 204B is configured to obtain a prediction level value sequence from the inter prediction unit 204D and a level value sequence (actually, a prediction residual sequence) from the arithmetic decoding unit 204A, and add them together to generate a level value sequence. The generated level value sequence is output to the first inverse quantization unit 204E and the frame buffer 204C.

[0067] The first inverse quantization unit 204E is configured to inversely quantize the level value sequence to generate and output a first coefficient sequence.

[0068] Fig. 18 is a diagram showing a modified example of the configuration of the displacement amount decoding unit 204. The following description will focus on the differences from the configuration in Fig. 17.

[0069] The frame buffer 204C is configured to acquire and store the first coefficient sequence from the first inverse quantization unit 204E or the adder 204B, and is configured to output the first coefficient sequence in the reference frame in accordance with control information (not shown).

[0070] The inter prediction unit 204D is configured to perform inter prediction using the first coefficient sequence of the reference frame read from the frame buffer 204C, thereby generating and outputting a prediction coefficient sequence.

[0071] The inter prediction unit 204D may determine the prediction coefficient of each frequency in the current frame by directly referring to the first coefficient of the corresponding frequency in the reference frame.

[0072] The adder 204B is configured to receive the sequence of predicted coefficients from the inter prediction unit 204D and the sequence of first coefficients (actually, the sequence of predicted residuals) from the first inverse quantization unit, and add them together to generate the first sequence of coefficients, which is output to the frame buffer 204C.

[0073] (Mesh Reconstruction Unit) As shown in FIG. 19, the mesh reconstruction unit 206 includes a patch association unit 206A, a submesh reconstruction unit 206B, and a submesh connection unit 206C.

[0074] The patch matching unit 206A is configured to generate and output a matching base mesh and a matching level value image or a matching first coefficient sequence using the decoded atlas, the decoded base mesh, and the level value image or the first coefficient sequence.

[0075] The submesh reconstructing unit 206B is configured to generate and output a submesh sequence using the association basic mesh and the association level value image or the association first coefficient sequence.

[0076] The submesh combining unit 206C is configured to combine the submesh rows to generate and output a single mesh.

[0077] Fig. 20 is a diagram showing an example of the configuration of the patch association unit 206 A. As shown in Fig. 20, the patch association unit 206 A includes a basic mesh association unit 206 A1, a displacement amount type determination unit 206 A2, and a displacement amount association unit 206 A3.

[0078] The basic mesh association unit 206A1 is configured to associate the patch with the basic mesh, thereby generating and outputting an associated basic mesh.

[0079] The displacement amount type determination unit 206A2 is configured to determine the type of the input level value image or first coefficient sequence, and output the determined one of the level value image and the first coefficient sequence as the displacement amount.

[0080] If ptl_profile_codec_group_idc is 0, the displacement amount type determination unit 206A2 may determine that the type of the input level value image or first coefficient sequence is the first coefficient sequence, and if ptl_profile_codec_group_idc is 1, the displacement amount type determination unit 206A2 may determine that the type of the input level value image or first coefficient sequence is a level value image.

[0081] The displacement amount associating unit 206A2 is configured to associate the patch with the displacement amount (level value image or first coefficient sequence) to generate and output the associated level value image or the associated first coefficient sequence, thereby enabling the association between the patch and the displacement amount.

[0082] The displacement amount association unit 206A2 may refer to displ_nal_unit in which dh_id is equal to pdu_displ_id, and output the first coefficient sequence obtained from the displ_nal_unit as the associated first coefficient sequence.

[0083] Fig. 21 is a diagram showing an example of the configuration of the submesh reconstructing unit 206B. As shown in Fig. 21, the submesh reconstructing unit 206B includes a subdivision unit 206B1, a displacement amount type determining unit 206B2, an image developing unit 206B3, a second inverse quantization unit 206B4, an inverse transform unit 206B5, and a displacement amount adding unit 206B6.

[0084] The subdivision unit 206B1 is configured to generate the added subdivision vertices and their connection information from the associated base mesh using the subdivision method indicated by the control information.

[0085] The subdivision unit 206B1 outputs a subdivision mesh that combines the basic mesh, the added subdivision vertices, and their connection information.

[0086] The displacement amount type determination unit 206B2 is configured to determine the type of the input displacement amount (level value image or first coefficient sequence), and output the determined one of the level value image and the first coefficient sequence as the displacement amount.

[0087] The displacement amount type determination unit 206B2 may determine that the type of the input displacement amount is a first coefficient sequence when ptl_profile_codec_group_idc is 0, and may determine that the type of the input displacement amount is a level value image when ptl_profile_codec_group_idc is 1. This makes it possible to appropriately generate a submesh sequence regardless of whether a level value image or a first coefficient sequence is input.

[0088] The image development unit 206B3 is configured to develop the associated level value image to generate and output a level value sequence.

[0089] The second inverse quantization unit 206B4 is configured to inversely quantize the level value sequence to generate and output a second coefficient sequence.

[0090] The inverse transform unit 206B5 is configured to generate and output a displacement sequence by inverse transforming the first coefficient sequence obtained from the displacement type determination unit 206B2 or the second coefficient sequence obtained from the second inverse quantization unit 206B4. This makes it possible to decode a mesh by inverse transforming the displacement decoded by arithmetic decoding and adding it to a basic mesh in a displacement addition unit 206B6 (described later).

[0091] The inverse transform unit 206B5 may inverse transform the first coefficient sequence or the second coefficient sequence using an inverse DCT or an inverse wavelet transform.

[0092] The displacement amount adding unit 206B6 is configured to generate and output sub-meshes by adding a sequence of displacement amounts to the finely divided meshes.

[0093] (Processing of Mesh Reconstruction Unit 206) FIG. 22 is a flowchart showing an example of processing of the mesh reconstruction unit 206.

[0094] As shown in FIG. 22, in step S101, the basic mesh association unit 206A1 associates the patch with the basic mesh.

[0095] In step S102, the displacement amount type determining unit 206A2 determines whether ptl_profile_codec_group_idc is 1 or 0.

[0096] If ptl_profile_codec_group_idc=1, the process proceeds to step S103, and if ptl_profile_codec_group_idc=0, the process proceeds to step S104.

[0097] In step S103, the displacement amount association unit 206A3 associates the patch with the decoded level value image.

[0098] In step S104, the displacement amount association unit 206A3 associates the patch with the first coefficient sequence.

[0099] In step S105, the subdivision unit 206B1 performs subdivision.

[0100] In step S106, the displacement amount type determining unit 206 determines whether ptl_profile_codec_group_idc is 1 or 0.

[0101] If ptl_profile_codec_group_idc=1, the process proceeds to step S107, and if ptl_profile_codec_group_idc=0, the process proceeds to step S109.

[0102] In step S107, the image development unit 206B3 performs image development.

[0103] In step S108, the second inverse quantization unit 206B4 performs inverse quantization.

[0104] In step S109, the inverse transform unit 206B5 performs inverse transform.

[0105] In step S110, the displacement amount addition unit 206B6 performs displacement amount addition.

[0106] In step S111, the submesh reconstruction unit 206 determines whether the patch currently being processed is the last one.

[0107] If the answer is Yes, the process proceeds to step S112; if the answer is No, the process proceeds to step S113.

[0108] In step S112, the sub-mesh combining unit 206C performs sub-mesh combining, and this operation ends.

[0109] According to the mesh decoding device 200 of this embodiment, it is possible to decode a mesh by adding the amount of displacement decoded by arithmetic decoding to the base mesh.

[0110] Furthermore, according to the mesh decoding device 200 of this embodiment, by including the submesh ID and the displacement amount ID in the syntax of the patch, it becomes possible to associate the submesh ID with the displacement amount ID.

[0111] The mesh encoding device 100 and mesh decoding device 200 described above may be realized as a program that causes a computer to execute each function (each step).

[0112] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation."

[0113] 1...Mesh processing system 100...Mesh encoding device 200...Mesh decoding device 201...V3C unit separation unit 202...Atlas decoding unit 203...Basic mesh decoding unit 204...Displacement amount decoding unit 204A...Arithmetic decoding unit 204B...Adder 204C...Frame buffer 204D...Inter prediction unit 204E...First inverse quantization unit 204F...Video decoding unit 205...Texture decoding unit 206...Mesh reconstruction unit 206A...Patch correspondence unit 206A1...Basic mesh correspondence unit 206A2...Displacement amount type determination unit 206A3...Displacement amount correspondence unit 206B...Submesh reconstruction unit 206B1...Subdivision unit 206B2...Displacement amount type determination unit 206B3...Image expansion unit 206B4...Second inverse quantization unit 206B5...Inverse transformation unit 206B6...Displacement amount addition unit 206C...Submesh connection unit

Claims

1. A mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a base mesh, and a level value image or a first coefficient sequence obtained from a bitstream, wherein the mesh reconstruction unit comprises: a patch correspondence unit configured to generate and output a corresponding base mesh, a corresponding level value image, or a corresponding first coefficient sequence using the atlas, the base mesh, and the level value image or the first coefficient sequence; and a submesh reconstruction unit configured to generate and output a submesh sequence using the corresponding base mesh, the corresponding level value image, or the corresponding first coefficient sequence.

2. The mesh decoding device of claim 1, wherein the submesh reconstruction unit comprises: a subdivision unit configured to generate and output a re-division mesh including subdivision vertices added from the corresponding basic mesh and their connection information; a displacement type determination unit configured to determine the type of the corresponding level value image or the corresponding first coefficient sequence and output the determined one of the level value image and the first coefficient sequence; an image expansion unit configured to generate and output a level value sequence by expanding the level value image; a second inverse quantization unit configured to generate and output a second coefficient sequence by inverse quantizing the level value sequence; an inverse transformation unit configured to generate and output a displacement sequence by inverse transforming the first coefficient sequence or the second coefficient sequence; and a displacement addition unit configured to generate and output a submesh sequence by adding the displacement sequence to the subdivision mesh.

3. The mesh decoding device described in claim 2, characterized in that the displacement amount type determination unit determines that the type is the first coefficient sequence when the control information is 0, and determines that the type is the level value image when the control information is 1.

4. A mesh decoding method comprising a step of reconstructing and outputting a mesh based on an atlas, a basic mesh, and a level value image or a first coefficient sequence obtained from a bit stream, wherein in said step, a corresponding basic mesh and a corresponding level value image or a corresponding first coefficient sequence are generated and output using said atlas, said basic mesh, and said level value image or said first coefficient sequence, and a sub-mesh sequence is generated and output using said corresponding basic mesh and said corresponding level value image or said corresponding first coefficient sequence.

5. A program that causes a computer to function as a mesh decoding device, wherein the mesh decoding device comprises a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a basic mesh, and a level value image or a first coefficient sequence obtained from a bit stream, and the mesh reconstruction unit comprises: a patch correspondence unit configured to generate and output a corresponding basic mesh and a corresponding level value image or a corresponding first coefficient sequence using the atlas, the basic mesh, and the level value image or the first coefficient sequence, and a submesh reconstruction unit configured to generate and output a submesh sequence using the corresponding basic mesh and the corresponding level value image or the corresponding first coefficient sequence.

Images