Mesh decoding device, mesh decoding method, and program
The mesh decoding device and method address the incomplete mesh decoding issue by integrating displacement amounts into the basic mesh through a patch correspondence and submesh reconstruction process, achieving accurate mesh reconstruction.
Patent Information
- Application Number
- JP2024046821
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-03-22
- Publication Date
- 2025-10-03
AI Technical Summary
Existing mesh decoding methods fail to decode the displacement amount due to the lack of a process for combining the basic mesh and displacement amount, preventing complete mesh reconstruction.
A mesh decoding device and method that reconstructs a mesh by generating and outputting correspondence meshes and submesh sequences using an atlas, a base mesh, and level value or coefficient sequences, incorporating a patch correspondence unit and a submesh reconstruction unit to integrate displacement amounts into the basic mesh.
Enables complete mesh decoding by adding the displacement amount decoded by arithmetic decoding to the basic mesh, ensuring accurate mesh reconstruction.
Smart Images
Figure 2025146181000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program. [Background technology]
[0002] Non-Patent Document 1 discloses a technique for dividing a point group into partial patches and decoding the divided patches.
[0003] Non-Patent Document 2 discloses a technology that extends Non-Patent Document 1 to mesh decoding. A mesh is decoded by dividing it into a rough basic mesh and a detailed displacement. The mesh is decoded by dividing it into partial sub-meshes. The displacement is decoded by a video codec or arithmetic decoding. [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] “Information technology - Coded Representation of Immersive Media - Part 5: Visual Volumetric Video-based Coding (V3C) and Video-based Point Cloud Compression (V-PCC),” ISO / IEC JTC 1 / SC 29 / WG 7, ISO / IEC 23090-5:2021(2E). [Non-patent document 2] “WD 5.0 of V-DMC,” ISO / IEC JTC 1 / SC 29 / WG 7 N00744. Summary of the Invention [Problem to be solved by the invention]
[0005] However, the method of Non-Patent Document 2 has a problem in that when decoding the displacement amount by arithmetic decoding, there is no process for decoding the mesh from the basic mesh and the displacement amount, and therefore the mesh cannot be decoded.
[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a mesh decoding device, a mesh decoding method, and a program that enable decoding of a mesh by adding the amount of displacement decoded by arithmetic decoding to a basic mesh. [Means for solving the problem]
[0007] A first feature of the present invention is summarized as a mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a base mesh, and a level value image or a first coefficient sequence obtained from a bitstream, wherein the mesh reconstruction unit comprises: a patch correspondence unit configured to generate and output a correspondence base mesh, a correspondence level value image, or a correspondence first coefficient sequence, using the atlas, the base mesh, and the level value image or the first coefficient sequence; and a submesh reconstruction unit configured to generate and output a submesh sequence, using the correspondence base mesh, and the correspondence level value image or the correspondence first coefficient sequence.
[0008] A second feature of the present invention is summarized as a mesh decoding method comprising a step of reconstructing and outputting a mesh based on an atlas, a basic mesh, and a level value image or a first coefficient sequence obtained from a bitstream, wherein in the step, a corresponding basic mesh and a corresponding level value image or a corresponding first coefficient sequence are generated and output using the atlas, the basic mesh, and the level value image or the first coefficient sequence, and a sub-mesh sequence is generated and output using the corresponding basic mesh and the corresponding level value image or the corresponding first coefficient sequence.
[0009] A third feature of the present invention is summarized as a program for causing a computer to function as a mesh decoding device, the mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a basic mesh, and a level value image or a first coefficient sequence obtained from a bitstream; the mesh reconstruction unit comprising: a patch correspondence unit configured to generate and output a corresponding basic mesh and a corresponding level value image or a corresponding first coefficient sequence using the atlas, the basic mesh, and the level value image or the first coefficient sequence; and a submesh reconstruction unit configured to generate and output a submesh sequence using the corresponding basic mesh and the corresponding level value image or the corresponding first coefficient sequence. [Effects of the Invention]
[0010] According to the present invention, it is possible to provide a mesh decoding device, a mesh decoding method and a program that are capable of decoding a mesh by adding the amount of displacement decoded by arithmetic decoding to a basic mesh. [Brief explanation of the drawings]
[0011] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks of a mesh decoding device 200 according to an embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of a V3C unit. [Figure 4] FIG. 4 is a diagram showing an example of the syntax configuration of a V3C unit header. [Figure 5] FIG. 5 is a diagram showing an example of the structure of a bitstream. [Figure 6] FIG. 6 is a diagram illustrating an example of the syntax configuration of a V3C parameter set. [Figure 7]FIG. 7 is a diagram illustrating an example of the syntax configuration of profile_tier_level. [Figure 8] FIG. 8 is a diagram illustrating an example of the syntax configuration of afps_vdmc_extension. [Figure 9] FIG. 9 is a diagram illustrating an example of the syntax configuration of atlas_frame_displ_information. [Figure 10A] FIG. 10A is a diagram illustrating an example of afdi_displ_id. [Figure 10B] FIG. 10B is a diagram showing an example of DisplIDToIndex in the example of FIG. 10A. [Figure 10C] FIG. 10C is a diagram showing an example of DisplIndexToID in the example of FIG. 10A. [Figure 11] FIG. 11 is a diagram illustrating an example of the syntax configuration of patch_data_unit. [Figure 12] FIG. 12 is a diagram showing an example of the configuration of a displacement amount bit stream. [Figure 13] FIG. 13 is a diagram illustrating an example of the syntax configuration of displ_nal_unit_header. [Figure 14] FIG. 14 is a diagram showing an example of the correspondence between displ_nal_unit_type and the contents of the payload. [Figure 15] FIG. 15 is a diagram illustrating an example of the syntax configuration of displ_layer_rbsp. [Figure 16] FIG. 16 is a diagram illustrating an example of the syntax configuration of displ_header. [Figure 17] FIG. 17 is a diagram illustrating an example of functional blocks of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 18] FIG. 18 is a diagram showing an example of functional blocks of a modified example of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 19]FIG. 19 is a diagram showing an example of functional blocks of the mesh reconstruction unit 206 of the mesh decoding device 200 according to an embodiment. [Figure 20] FIG. 20 is a diagram showing an example of functional blocks of a patch association unit 206A of the mesh reconstruction unit 206 of the mesh decoding device 200 according to an embodiment. [Figure 21] FIG. 21 is a diagram showing an example of functional blocks of the sub-mesh reconstructing unit 206B of the mesh reconstructing unit 206 of the mesh decoding device 200 according to an embodiment. [Figure 22] FIG. 22 is a flowchart showing an example of processing by the mesh reconstruction unit 206 of the mesh decoding device 200 according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION
[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.
[0013] First Embodiment This embodiment is based on V3C described in Non-Patent Document 1. The definitions of atlas, patch, and tile in V3C are shown below.
[0014] An atlas is a collection of rectangular regions and associated information placed on a frame, corresponding to a three-dimensional space.
[0015] A patch is a rectangular area that corresponds to the 3D information contained in the atlas.
[0016] A tile is an independent, decodable rectangular region in the atlas, which is a different concept from a rectangular region in a patch.
[0017] Hereinafter, a mesh processing system 1 according to this embodiment will be described with reference to FIGS.
[0018] 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to this embodiment. As shown in FIG. 1, the mesh processing system 1 includes a mesh encoding device 100 and a mesh decoding device 200.
[0019] FIG. 2 is a diagram showing an example of functional blocks of the mesh decoding device 200 according to this embodiment.
[0020] As shown in FIG. 2, the mesh decoding device 200 includes a V3C unit separation unit 201, an atlas decoding unit 202, a basic mesh decoding unit 203, a displacement amount decoding unit 204, a texture decoding unit 205, and a mesh reconstruction unit 206.
[0021] The V3C unit separation unit 201 is configured to separate the bitstream into an atlas bitstream, a base mesh bitstream, a displacement bitstream, and a texture bitstream.
[0022] The atlas decoding unit 202 is configured to decode and output the atlas from the atlas bitstream.
[0023] The base mesh decoding unit 203 is configured to decode and output the base mesh from the base mesh bitstream.
[0024] The displacement decoding unit 204 is configured to decode and output the level value image or the first coefficient sequence from the displacement bitstream.
[0025] The texture decoding unit 205 is configured to decode and output texture from the texture bitstream. The texture decoding unit 205 may decode the texture using HEVC.
[0026] The mesh reconstruction unit 206 is configured to reconstruct and output a mesh using the atlas decoded by the atlas decoding unit 201, the basic mesh decoded by the basic mesh decoding unit 203, and the level value image or first coefficient sequence decoded by the displacement amount decoding unit 204.
[0027] (Bitstream configuration) A bitstream may be composed of a combination of multiple V3C units.
[0028] 3 is a diagram showing an example of the configuration of a V3C unit. A V3C unit may be composed of a V3C unit header and a V3C unit payload.
[0029] 4 is a diagram showing an example of the syntax configuration of a V3C unit header. The V3C unit header may include vuh_unit_type, which indicates the type of the V3C unit.
[0030] 5 is a diagram showing an example of the configuration of a bitstream. The bitstream may be composed of a V3C parameter set, an atlas bitstream, a base mesh bitstream, a displacement bitstream, and a texture bitstream.
[0031] vuh_unit_type may store any of the following: V3C_VPS indicating a V3C parameter set, V3C_AD indicating an atlas bitstream, V3C_BMD indicating a basic mesh bitstream, V3C_GVD indicating a displacement bitstream, or V3C_AVD indicating a texture bitstream.
[0032] 6 is a diagram showing an example of the syntax configuration of a V3C parameter set. The V3C parameter set may include profile_tier_level indicating profile information and the like.
[0033] 7 is a diagram illustrating an example of the syntax configuration of profile_tier_level. The profile_tier_level may include ptl_profile_codec_group_idc, which indicates a compression method for the amount of displacement.
[0034] It may be defined that when ptl_profile_codec_group_idc is 0, the displacement bitstream contains compressed data of displacement by arithmetic decoding, and when ptl_profile_codec_group_idc is 1, the displacement bitstream contains compressed data of displacement by video codec.
[0035] (Atlas bitstream configuration) The atlas bitstream may use afps_vdmc_extension, which indicates frame parameters that are an extension of the V3C frame parameter set.
[0036] FIG. 8 is a diagram illustrating an example of the syntax configuration of afps_vdmc_extension.
[0037] The afps_vdmc_extension may include atlas_frame_displ_information indicating information about the amount of displacement.
[0038] FIG. 9 is a diagram illustrating an example of the syntax configuration of atlas_frame_displ_information.
[0039] The atlas_frame_displ_information may include afdi_use_single_displ_flag indicating whether a single displacement amount is used, afdi_num_displs_minus1 indicating the value obtained by subtracting 1 from the number of displacement amounts, afdi_signalled_displ_id_flag indicating whether to transmit a displacement amount ID, afdi_signalled_displ_id_length_minus1 indicating the value obtained by subtracting 1 from the number of bits representing the displacement amount ID, and afdi_displ_id indicating the displacement amount ID.
[0040] 10A is a diagram showing an example of afdi_displ_id, in which the 0th afdi_displ_id is 101, the 1st afdi_displ_id is 201, and the 2nd afdi_displ_id is 001.
[0041] The atlas_frame_displ_information may use afdi_displ_id to create DisplIDToIndex, which indicates a correspondence table that allows a displacement amount index to be referenced from a displacement amount ID.
[0042] Fig. 10B is a diagram showing an example of DisplIDToIndex in the example of Fig. 10A. Atlas_frame_displ_information may use afdi_displ_id to create DisplIndexToID indicating a correspondence table that allows a displacement amount ID to be referenced from a displacement amount index.
[0043] Fig. 10C is a diagram showing an example of DisplIndexToID in the example of Fig. 10A. The atlas bitstream may use an extended patch_data_unit indicating a V3C data unit.
[0044] FIG. 11 is a diagram illustrating an example of the syntax configuration of patch_data_unit.
[0045] The patch_data_unit may be defined for each tile identified by tileID. The patch_data_unit may be defined for each patch identified by patchIdx.
[0046] The patch_data_unit may include a pdu_submesh_id indicating a submesh ID associated with the patch and a pdu_displ_id indicating a displacement ID associated with the patch, thereby enabling the association between the submesh ID and the displacement ID.
[0047] The patch_data_unit may include pdu_2d_pos_x, which indicates the x-coordinate of the upper left corner of the rectangular area of the patch; pdu_2d_pos_y, which indicates the y-coordinate of the upper left corner of the rectangular area of the patch; pdu_2d_size_x_minus1, which indicates the value obtained by subtracting 1 from the width of the rectangular area of the patch; and pdu_2d_size_y_minus1, which indicates the value obtained by subtracting 1 from the height of the rectangular area of the patch.
[0048] The patch_data_unit may include pdu_attributes_2d_pos_x indicating the x-coordinate of the upper left corner of the rectangular area in the texture image of the patch, pdu_attributes_2d_pos_y indicating the y-coordinate of the upper left corner of the rectangular area in the texture image of the patch, pdu_attributes_2d_size_x_minus1 indicating the value obtained by subtracting 1 from the width of the rectangular area in the texture image of the patch, and pdu_attributes_2d_size_y_minus1 indicating the value obtained by subtracting 1 from the height of the rectangular area in the texture image of the patch.
[0049] (Structure of displacement bitstream) The displacement bitstream may include data of the displacement compressed by arithmetic decoding or data of the displacement compressed by a video codec.
[0050] FIG. 12 is a diagram showing an example of the configuration of a displacement amount bit stream.
[0051] When the displacement bitstream includes compressed data of the displacement obtained by arithmetic decoding, the V3C unit payload may be configured with displ_nal_unit indicating the NAL unit.
[0052] The displ_nal_unit may be composed of a displ_nal_unit_header indicating the header of the NAL unit in the displacement bitstream and the payload of the NAL unit in the displacement bitstream.
[0053] FIG. 13 is a diagram illustrating an example of the syntax configuration of displ_nal_unit_header.
[0054] The displ_nal_unit_header may be composed of a displ_nal_forbidden_zero_bit indicating the leading write-protected section and a displ_nal_unit_type indicating the type of data in the payload.
[0055] FIG. 14 is a diagram showing an example of the correspondence between displ_nal_unit_type and the contents of the payload.
[0056] The contents of the payload may be defined as displ_sequence_parameter_set_rbsp if displ_nal_unit_type is 0, displ_frame_parameter_set_rbsp if displ_nal_unit_type is 1, and displ_layer_rbsp if displ_nal_unit_type is 2.
[0057] FIG. 15 is a diagram illustrating an example of the syntax configuration of displ_layer_rbsp.
[0058] The displ_layer_rbsp may include a displ_header indicating the header of the displacement arithmetic decoding, a displ_data_unit indicating the data of the displacement arithmetic decoding, and an rbsp_trailing_bit indicating the trailing zero bits of the displacement arithmetic decoding.
[0059] 16 is a diagram showing an example of the syntax configuration of displ_header. The displ_header may include dh_id indicating the ID of the displacement amount.
[0060] (Displacement amount decoding unit 204) As shown in FIG. 17, the displacement decoding unit 204 includes an arithmetic decoding unit 204A, an adder 204B, a frame buffer 204C, an inter prediction unit 204D, a first inverse quantization unit 204E, and a video decoding unit 204F.
[0061] The video decoding unit 204F is configured to decode the level value image from the displacement amount bitstream using a video codec and output the level value image. The video decoding unit 204F may decode the level value image using HEVC.
[0062] The arithmetic decoding unit 204A is configured to decode and output a level value sequence from the displacement amount bitstream by arithmetic decoding. The arithmetic decoding unit 204A may decode the level value sequence as described in Japanese Patent Application No. 2022-212452.
[0063] The frame buffer 204C is configured to acquire and store a level value sequence from the arithmetic decoding unit 204A or the adder 204B. The frame buffer 204C is configured to output a level value sequence in a reference frame in accordance with control information (not shown).
[0064] The inter prediction unit 204D is configured to perform inter prediction using the level value sequence of the reference frame read out from the frame buffer 204C, thereby generating and outputting a predicted level value sequence.
[0065] The inter prediction unit 204D may determine the predicted level value of each frequency in the current frame by directly referring to the level value of the corresponding frequency in the reference frame.
[0066] The adder 204B is configured to obtain a prediction level value sequence from the inter prediction unit 204D and a level value sequence (actually, a prediction residual sequence) from the arithmetic decoding unit 204A, and add them together to generate a level value sequence. The generated level value sequence is output to the first inverse quantization unit 204E and the frame buffer 204C.
[0067] The first inverse quantization unit 204E is configured to inversely quantize the level value sequence to generate and output a first coefficient sequence.
[0068] 18 is a diagram showing a modified example of the configuration of the displacement amount decoding unit 204. The following description focuses on the differences from the configuration in FIG.
[0069] The frame buffer 204C is configured to acquire and store the first coefficient sequence from the first inverse quantization unit 204E or the adder 204B, and is configured to output the first coefficient sequence in the reference frame in accordance with control information (not shown).
[0070] The inter prediction unit 204D is configured to perform inter prediction using the first coefficient sequence of the reference frame read from the frame buffer 204C, thereby generating and outputting a prediction coefficient sequence.
[0071] The inter prediction unit 204D may determine the prediction coefficient of each frequency in the current frame by directly referring to the first coefficient of the corresponding frequency in the reference frame.
[0072] The adder 204B is configured to obtain the prediction coefficient sequence from the inter prediction unit 204D and the first coefficient sequence (actually, the prediction residual sequence) from the first inverse quantization unit, add them together, and generate the first coefficient sequence. The generated first coefficient sequence is output to the frame buffer 204C.
[0073] (Mesh reconstruction part) As shown in FIG. 19, the mesh reconstructing unit 206 includes a patch associating unit 206A, a submesh reconstructing unit 206B, and a submesh combining unit 206C.
[0074] The patch matching unit 206A is configured to generate and output a matching basic mesh and a matching level value image or a matching first coefficient sequence using the decoded atlas, the decoded basic mesh, and the level value image or the first coefficient sequence.
[0075] The submesh reconstructing unit 206B is configured to generate and output a submesh sequence using the association basic mesh and the association level value image or the association first coefficient sequence.
[0076] The submesh combining unit 206C is configured to combine the submesh rows to generate and output a single mesh.
[0077] Fig. 20 is a diagram showing an example of the configuration of the patch association unit 206A. As shown in Fig. 20, the patch association unit 206A includes a basic mesh association unit 206A1, a displacement amount type determination unit 206A2, and a displacement amount association unit 206A3.
[0078] The basic mesh association unit 206A1 is configured to associate the patch with the basic mesh, thereby generating and outputting an associated basic mesh.
[0079] The displacement amount type determination unit 206A2 is configured to determine the type of the input level value image or the first coefficient sequence, and output the determined one of the level value image and the first coefficient sequence as the displacement amount.
[0080] The displacement amount type determination unit 206A2 may determine that the type of the input level value image or first coefficient sequence is the first coefficient sequence when ptl_profile_codec_group_idc is 0, and may determine that the type of the input level value image or first coefficient sequence is the level value image when ptl_profile_codec_group_idc is 1.
[0081] The displacement amount associating unit 206A2 is configured to associate the patch with the displacement amount (level value image or first coefficient sequence) to generate and output the associated level value image or the associated first coefficient sequence, thereby enabling the association between the patch and the displacement amount.
[0082] The displacement amount association unit 206A2 may refer to a displ_nal_unit in which dh_id is equal to pdu_displ_id, and output the first coefficient sequence obtained from the displ_nal_unit as an associated first coefficient sequence.
[0083] Fig. 21 is a diagram showing an example of the configuration of the submesh reconstructing unit 206B. As shown in Fig. 21, the submesh reconstructing unit 206B includes a subdivision unit 206B1, a displacement amount type determining unit 206B2, an image developing unit 206B3, a second inverse quantization unit 206B4, an inverse transformation unit 206B5, and a displacement amount adding unit 206B6.
[0084] The subdivision unit 206B1 is configured to generate the added subdivision vertices and their connection information from the associated basic mesh using the subdivision method indicated by the control information.
[0085] The subdivision unit 206B1 outputs a subdivision mesh that combines the basic mesh, the added subdivision vertices, and their connection information.
[0086] The displacement amount type determination unit 206B2 is configured to determine the type of the input displacement amount (level value image or first coefficient sequence), and output the determined one of the level value image and the first coefficient sequence as the displacement amount.
[0087] The displacement amount type determination unit 206B2 may determine that the type of the input displacement amount is a first coefficient sequence when ptl_profile_codec_group_idc is 0, and may determine that the type of the input displacement amount is a level value image when ptl_profile_codec_group_idc is 1. This makes it possible to appropriately generate a sub-mesh sequence regardless of whether a level value image or a first coefficient sequence is input.
[0088] The image development unit 206B3 is configured to develop the associated level value image to generate and output a level value sequence.
[0089] The second inverse quantization unit 206B4 is configured to inversely quantize the level value sequence to generate and output a second coefficient sequence.
[0090] The inverse transform unit 206B5 is configured to generate and output a displacement sequence by inverse transforming the first coefficient sequence obtained from the displacement type determination unit 206B2 or the second coefficient sequence obtained from the second inverse quantization unit 206B4. This makes it possible to decode a mesh by inverse transforming the displacement decoded by arithmetic decoding and adding it to a basic mesh in a displacement adder 206B6 (described later).
[0091] The inverse transform unit 206B5 may inverse transform the first coefficient sequence or the second coefficient sequence using an inverse DCT or an inverse wavelet transform.
[0092] The displacement amount adding unit 206B6 is configured to generate and output sub-meshes by adding the displacement amount sequence to the finely divided mesh.
[0093] (Processing of the mesh reconstruction unit 206) FIG. 22 is a flowchart showing an example of the processing of the mesh reconstruction unit 206.
[0094] As shown in FIG. 22, in step S101, the basic mesh association unit 206A1 associates the patch with the basic mesh.
[0095] In step S102, the displacement amount type determining unit 206A2 determines whether ptl_profile_codec_group_idc is 1 or 0.
[0096] If ptl_profile_codec_group_idc=1, the process proceeds to step S103, and if ptl_profile_codec_group_idc=0, the process proceeds to step S104.
[0097] In step S103, the displacement amount association unit 206A3 associates the patch with the decoded level value image.
[0098] In step S104, the displacement amount association unit 206A3 associates the patch with the first coefficient sequence.
[0099] In step S105, the subdivision unit 206B1 performs subdivision.
[0100] In step S106, the displacement amount type determining unit 206 determines whether ptl_profile_codec_group_idc is 1 or 0.
[0101] If ptl_profile_codec_group_idc=1, the process proceeds to step S107, and if ptl_profile_codec_group_idc=0, the process proceeds to step S109.
[0102] In step S107, the image development section 206B3 performs image development.
[0103] In step S108, the second inverse quantization unit 206B4 performs inverse quantization.
[0104] In step S109, the inverse conversion unit 206B5 performs inverse conversion.
[0105] In step S110, the displacement amount addition unit 206B6 performs displacement amount addition.
[0106] In step S111, the submesh reconstruction unit 206 determines whether the patch currently being processed is the last one.
[0107] If the answer is Yes, the process proceeds to step S112; if the answer is No, the process proceeds to step S113.
[0108] In step S112, the sub-mesh combining unit 206C performs sub-mesh combining, and this operation ends.
[0109] According to the mesh decoding device 200 of this embodiment, it is possible to decode a mesh by adding the amount of displacement decoded by arithmetic decoding to the base mesh.
[0110] Furthermore, according to the mesh decoding device 200 of this embodiment, by including the sub-mesh ID and the displacement amount ID in the patch syntax, it becomes possible to associate the sub-mesh ID with the displacement amount ID.
[0111] The mesh encoding device 100 and the mesh decoding device 200 described above may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]
[0112] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which will contribute to the achievement of Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]
[0113] 1...Mesh processing system 100...Mesh coding device 200...Mesh decoding device 201...V3C unit separation section 202...Atlas decoding unit 203...Basic mesh decoding unit 204...Displacement amount decoding unit 204A...Arithmetic decoding unit 204B...adder 204C...Frame buffer 204D…Inter prediction section 204E...first inverse quantization section 204F...Video decoding section 205...Texture decoding unit 206...Mesh reconstruction unit 206A...Patch matching section 206A1...Basic mesh matching section 206A2...Displacement amount type determination unit 206A3...Displacement amount corresponding section 206B...Submesh reconstruction section 206B1…Subdivision section 206B2...Displacement amount type determination unit 206B3...Image development section 206B4…Second inverse quantization unit 206B5…Inverse conversion unit 206B6...Displacement amount adding unit 206C...Submesh joint
Claims
1. A mesh decoding device, comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on the atlas obtained from the bitstream, the basic mesh, and the level value image or the first coefficient sequence; The mesh reconstruction unit a patch matching unit configured to generate and output a matching basic mesh and a matching level value image or a matching first coefficient sequence using the atlas, the basic mesh, and the level value image or the first coefficient sequence; A mesh decoding device characterized by comprising a submesh reconstruction unit configured to generate and output a submesh sequence using the correspondence basic mesh and the correspondence level value image or the correspondence first coefficient sequence.
2. The submesh reconstruction unit a subdivision unit configured to generate and output a re-division mesh including the added subdivision vertices and their connection information from the associated base mesh; a displacement amount type determining unit configured to determine a type of the associated level value image or the associated first coefficient sequence, and output the determined one of the level value image and the first coefficient sequence; an image development unit configured to develop the level value image to generate and output a level value sequence; a second inverse quantization unit configured to inverse quantize the level value sequence to generate and output a second coefficient sequence; an inverse transform unit configured to generate and output a displacement sequence by inverse transforming the first coefficient sequence or the second coefficient sequence; 2. The mesh decoding device according to claim 1, further comprising: a displacement amount adding unit configured to generate and output a sequence of sub-meshes by adding the sequence of displacement amounts to the subdivision mesh.
3. The mesh decoding device according to claim 2, characterized in that the displacement amount type determination unit determines that the type is the first coefficient sequence when the control information is 0, and determines that the type is the level value image when the control information is 1.
4. 1. A mesh decoding method, comprising: a step of reconstructing and outputting a mesh based on the atlas obtained from the bitstream, the basic mesh, and the level value image or the first coefficient sequence; In the above step, generating and outputting a corresponding basic mesh and a corresponding level value image or a corresponding first coefficient sequence using the atlas, the basic mesh, and the level value image or the first coefficient sequence; A mesh decoding method, characterized in that a sub-mesh sequence is generated and output using the correspondence basic mesh and the correspondence level value image or the correspondence first coefficient sequence.
5. A program that causes a computer to function as a mesh decoding device, The mesh decoding device comprises: a mesh reconstruction unit configured to reconstruct and output a mesh based on the atlas obtained from the bitstream, the basic mesh, and the level value image or the first coefficient sequence; The mesh reconstruction unit a patch matching unit configured to generate and output a matching basic mesh and a matching level value image or a matching first coefficient sequence using the atlas, the basic mesh, and the level value image or the first coefficient sequence; a submesh reconstruction unit configured to generate and output a submesh sequence using the correspondence basic mesh and the correspondence level value image or the correspondence first coefficient sequence.