Mesh decoding device, mesh decoding method, and program
The mesh decoding device and method address syntax mismatches between atlas and displacement bitstreams by using a syntax matching unit to align and combine submeshes, ensuring accurate mesh reconstruction.
Patent Information
- Application Number
- JP2024065030
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-04-12
- Publication Date
- 2025-10-24
AI Technical Summary
Existing mesh decoding methods face issues with inconsistencies between the atlas bitstream and displacement bitstream, leading to incorrect mesh decoding when syntax mismatches occur.
A mesh decoding device and method that includes a syntax matching unit to generate matching base meshes, displacement vectors, and subdivisions, followed by submesh reconstruction and combining to resolve inconsistencies between the atlas and displacement bitstreams.
Ensures correct mesh decoding by aligning syntax elements, thereby resolving inconsistencies and ensuring accurate mesh reconstruction.
Smart Images
Figure 2025161658000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program. [Background technology]
[0002] Non-Patent Document 1 discloses a technique for decoding a point group using video information and additional information called an atlas.
[0003] Non-Patent Document 2 discloses a technique that extends Non-Patent Document 1 to mesh decoding applications.
[0004] Specifically, Non-Patent Document 2 first decodes a rough base mesh and a detailed displacement, and then decodes a mesh from both of them. Here, a syntax indicating a reference list structure is defined for inter-prediction of the displacement. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] “Information technology - Coded Representation of Immersive Media - Part 5: Visual Volumetric Video-based Coding (V3C) and Video-based Point Cloud Compression (V-PCC),” ISO / IEC JTC 1 / SC 29 / WG 7, ISO / IEC 23090-5:2021(2E). [Non-patent document 2] “WD 6.0 of V-DMC,” ISO / IEC JTC 1 / SC 29 / WG 7 N00822. Summary of the Invention [Problem to be solved by the invention]
[0006] However, the method of Non-Patent Document 2 has a problem in that if a syntax mismatch occurs between the decoded atlas and the displacement (the number of vectors of the displacement or the number of subdivisions), the mesh cannot be decoded correctly.
[0007] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a mesh decoding device, a mesh decoding method, and a program that can eliminate inconsistencies between the atlas bitstream and the displacement amount bitstream. [Means for solving the problem]
[0008] A first feature of the present invention is summarized as a mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a base mesh, a coefficient sequence, a number of displacement vectors, and a number of subdivisions obtained from a bitstream, wherein the mesh reconstruction unit comprises: a syntax matching unit configured to generate and output a matching base mesh, a matching coefficient sequence, a matching displacement vector number, and a matching subdivision number using the atlas, the base mesh, the coefficient sequence, the number of displacement vectors, and the number of subdivisions; a submesh reconstruction unit configured to generate and output a submesh sequence using the matching base mesh, the matching coefficient sequence, the matching displacement vector number, and the matching subdivision number; and a submesh combining unit configured to generate and output a single mesh by combining the submesh sequences.
[0009] A second feature of the present invention is a mesh decoding method comprising a step A of reconstructing and outputting a mesh based on an atlas, a base mesh, a coefficient sequence, a number of displacement vectors, and a number of subdivisions obtained from a bitstream, wherein the step A comprises a step of generating and outputting a matching base mesh, a matching coefficient sequence, a matching displacement vector number, and a matching subdivision number using the atlas, the base mesh, the coefficient sequence, the number of displacement vectors, and the number of subdivisions; a step of generating and outputting a submesh sequence using the matching base mesh, the matching coefficient sequence, the number of displacement vectors, and the number of matching subdivisions; and a step of generating and outputting a single mesh by combining the submesh sequences.
[0010] A third feature of the present invention is summarized as a program that causes a computer to function as a mesh decoding device, the mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a base mesh, a coefficient sequence, a number of displacement vectors, and a number of subdivisions obtained from a bitstream, the mesh reconstruction unit comprising: a syntax matching unit configured to generate and output a matching base mesh, a matching coefficient sequence, a matching displacement vector number, and a matching number of subdivisions using the atlas, the base mesh, the coefficient sequence, the number of displacement vectors, and the number of subdivisions; a submesh reconstruction unit configured to generate and output a submesh sequence using the matching base mesh, the matching coefficient sequence, the matching displacement vector number, and the matching number of subdivisions; and a submesh combining unit configured to generate and output a single mesh by combining the submesh sequences. [Effects of the Invention]
[0011] According to the present invention, it is possible to provide a mesh decoding device, a mesh decoding method and a program that can resolve the inconsistency between the atlas bitstream and the displacement bitstream. [Brief explanation of the drawings]
[0012] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks of a mesh decoding device 200 according to an embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of a V3C unit. [Figure 4] FIG. 4 is a diagram showing an example of the syntax configuration of a V3C unit header. [Figure 5] FIG. 5 is a diagram showing an example of the structure of a bitstream. [Figure 6] FIG. 6 is a diagram illustrating an example of the syntax configuration of profile_tier_level. [Figure 7] FIG. 7 is a diagram illustrating an example of the syntax configuration of asps_vdmc_extension. [Figure 8] FIG. 8 is a diagram showing an example of the configuration of a displacement amount bit stream. [Figure 9] FIG. 9 is a diagram illustrating an example of the syntax configuration of displ_nal_unit_header. [Figure 10] FIG. 10 is a diagram showing an example of the correspondence between displ_nal_unit_type and the contents of the payload. [Figure 11] FIG. 11 is a diagram illustrating an example of the syntax configuration of displ_sequence_parameter_set_rbsp. [Figure 12] FIG. 12 is a diagram illustrating an example of the syntax configuration of displ_layer_rbsp. [Figure 13] FIG. 13 is a diagram illustrating an example of the syntax configuration of displ_header. [Figure 14] FIG. 14 is a diagram illustrating an example of the syntax configuration of displ_ref_list_struct. [Figure 15]FIG. 15 is a diagram illustrating an example of the syntax configuration of displ_data_unit. [Figure 16] FIG. 16 is a diagram illustrating an example of functional blocks of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 17] FIG. 17 is a diagram showing an example of functional blocks of a displacement amount header decoder 204B of the displacement amount decoder 204 of the mesh decoding device 200 according to an embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a prediction structure and a reference list constructed therefrom. [Figure 19] FIG. 19 is a diagram showing an example of marking of a reference list. [Figure 20] FIG. 20 is a diagram showing an example of functional blocks of a displacement amount data decoding unit 204C of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 21] FIG. 21 is a diagram showing a modified example of the functional blocks of the displacement amount data decoding unit 204C of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 22] FIG. 22 is a diagram showing an example of functional blocks of the mesh reconstruction unit 206 of the mesh decoding device 200 according to an embodiment. [Figure 23] FIG. 23 is a diagram showing an example of functional blocks of the sub-mesh reconstructing unit 206B of the mesh reconstructing unit 206 of the mesh decoding device 200 according to an embodiment. [Figure 24] FIG. 24 is a flowchart showing an example of processing by the ASVE decoding unit 202A1 of the atlas sequence parameter set decoding unit 202A of the atlas decoding unit 202 of the mesh decoding device 200 according to an embodiment. [Figure 25] FIG. 25 is a flowchart showing an example of processing by the reference list constructor 204B2 of the displacement amount header decoder 204B of the displacement amount decoder 204 of the mesh decoding device 200 according to an embodiment. [Figure 26]FIG. 26 is a flowchart showing an example of processing by the reference frame marking unit 204B3 of the displacement amount header decoding unit 204B of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 27] FIG. 27 is a diagram showing an example of the configuration of the atlas data decoding unit 202. DETAILED DESCRIPTION OF THE INVENTION
[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.
[0014] First Embodiment This embodiment is based on V3C described in Non-Patent Document 1. An atlas in V3C is a collection of rectangular regions and related information arranged on a frame, and corresponds to a three-dimensional space.
[0015] Hereinafter, a mesh processing system 1 according to this embodiment will be described with reference to FIGS.
[0016] 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to this embodiment. As shown in FIG. 1, the mesh processing system 1 includes a mesh encoding device 100 and a mesh decoding device 200.
[0017] FIG. 2 is a diagram showing an example of functional blocks of the mesh decoding device 200 according to this embodiment.
[0018] As shown in FIG. 2, the mesh decoding device 200 includes a V3C unit separation unit 201, an atlas decoding unit 202, a basic mesh decoding unit 203, a displacement amount decoding unit 204, a texture decoding unit 205, and a mesh reconstruction unit 206.
[0019] The V3C unit separation unit 201 is configured to separate the bitstream into an atlas bitstream, a base mesh bitstream, a displacement bitstream, and a texture bitstream.
[0020] The atlas decoding unit 202 is configured to decode and output the atlas from the atlas bitstream.
[0021] The base mesh decoding unit 203 is configured to decode and output the base mesh from the base mesh bitstream.
[0022] The displacement decoding unit 204 is configured to decode and output the coefficient sequence, the number of displacement vectors, and the number of subdivisions from the displacement bitstream.
[0023] The texture decoding unit 205 is configured to decode and output texture from the texture bitstream. The texture decoding unit 205 may decode the texture using HEVC.
[0024] The mesh reconstruction unit 206 is configured to reconstruct and output a mesh using the atlas decoded by the atlas decoding unit 201, the basic mesh decoded by the basic mesh decoding unit 203, and the coefficient sequence, number of displacement vectors, and number of subdivisions decoded by the displacement decoding unit 204.
[0025] (Bitstream configuration) A bitstream may be composed of a combination of multiple V3C units.
[0026] 3 is a diagram showing an example of the configuration of a V3C unit. A V3C unit may be composed of a V3C unit header and a V3C unit payload.
[0027] 4 is a diagram showing an example of the syntax configuration of a V3C unit header. The V3C unit header may include vuh_unit_type, which indicates the type of the V3C unit.
[0028] 5 is a diagram showing an example of the configuration of a bitstream. The bitstream may be composed of a V3C parameter set, an atlas bitstream, a base mesh bitstream, a displacement bitstream, and a texture bitstream.
[0029] vuh_unit_type may store any of the following: V3C_VPS indicating a V3C parameter set, V3C_AD indicating an atlas bitstream, V3C_BMD indicating a basic mesh bitstream, V3C_GVD indicating a displacement bitstream, or V3C_AVD indicating a texture bitstream.
[0030] A V3C parameter set may contain a profile_tier_level that indicates the profile tier level.
[0031] FIG. 6 is a diagram illustrating an example of the syntax configuration of profile_tier_level.
[0032] The profile_tier_level may include a ptl_profile_codec_group_idc that indicates the codec ID of the displacement amount.
[0033] (Atlas bitstream structure) The atlas bitstream may use afps_vdmc_extension, which indicates frame parameters that are an extension of the V3C frame parameter set.
[0034] 7 is a diagram showing an example of the syntax configuration of asps_vdmc_extension. asps_vdmc_extension may include asve_displacement_num, which indicates the number of vectors of the displacement amount. asps_vdmc_extension may also include asve_subdivision_iteration_count, which indicates the number of subdivisions.
[0035] The asps_vdmc_extension may include asve_packing_method, which indicates an image packing method for the displacement. The asve_packing_method is decoded only if the codec of the displacement is a video codec, and may not be decoded otherwise.
[0036] The determination that the codec of the displacement amount is a video codec may be made when ptl_profile_codec_group_idc is an integer in the range of 0 to 4, or when ptl_profile_codec_group_idc is 127.
[0037] (Structure of displacement bitstream) 8 is a diagram showing an example of the configuration of a displacement amount bitstream. The displacement amount bitstream may have a V3C unit payload configured with displ_nal_unit indicating an NAL unit.
[0038] The displ_nal_unit may be composed of a displ_nal_unit_header indicating the header of the NAL unit in the displacement bitstream and the payload of the NAL unit in the displacement bitstream.
[0039] FIG. 9 is a diagram illustrating an example of the syntax configuration of displ_nal_unit_header.
[0040] The displ_nal_unit_header may be composed of a displ_nal_forbidden_zero_bit indicating the leading write-protected section and a displ_nal_unit_type indicating the type of data in the payload.
[0041] FIG. 10 is a diagram showing an example of the correspondence between displ_nal_unit_type and the contents of the payload.
[0042] The contents of the payload may be defined as displ_sequence_parameter_set_rbsp indicating the sequence parameter set of the displacement amount when displ_nal_unit_type is 0, displ_frame_parameter_set_rbsp indicating the frame parameter set of the displacement amount when displ_nal_unit_type is 1, and displ_layer_rbsp indicating the displacement amount data when displ_nal_unit_type is 2.
[0043] FIG. 11 is a diagram illustrating an example of the syntax configuration of displ_sequence_parameter_set_rbsp.
[0044] The displ_sequence_parameter_set_rbsp may include dsps_displacement_num, which indicates the number of vectors of the displacement amount.
[0045] The displ_sequence_parameter_set_rbsp may include dsps_log2_max_displ_frame_order_cnt_lsb, which indicates the maximum value of the frame display order of the displacement amount.
[0046] The displ_sequence_parameter_set_rbsp may include dsps_max_dec_displ_frame_buffering, which indicates the maximum value of the displacement buffer size.
[0047] The displ_sequence_parameter_set_rbsp may include dsps_num_ref_displ_frame_lists_in_dsps, which indicates the number of reference lists for displacement amounts.
[0048] The displ_sequence_parameter_set_rbsp may include a displ_ref_list_struct that indicates a reference list of displacement amounts.
[0049] 12 is a diagram showing an example of the syntax configuration of displ_layer_rbsp. displ_layer_rbsp may include displ_header indicating a displacement amount header and displ_data_unit indicating displacement amount data.
[0050] FIG. 13 is a diagram illustrating an example of the syntax configuration of displ_header.
[0051] The displ_header may include dh_id indicating an ID of the displacement amount, and dh_frm_order_cnt_lsb indicating the display order of the displacement amount frames.
[0052] The displ_header may contain dh_ref_displ_frame_list_dsps_flag which indicates whether the reference list is derived from the displ_sequence_parameter_set_rbsp or from the displ_header.
[0053] The displ_header may contain the displ_ref_list_struct described above.
[0054] The displ_header may contain ref_displ_frame_list_idx, which indicates the index of the reference list in the displ_sequence_parameter_set_rbsp.
[0055] FIG. 14 is a diagram illustrating an example of the syntax configuration of displ_ref_list_struct.
[0056] The displ_ref_list_struct may contain drl_num_ref_entries, which indicates the number of frames in the displ_ref_list_struct.
[0057] The displ_ref_list_struct may contain a drl_st_ref_displ_frame_flag that indicates whether a frame is a short-term or long-term reference frame.
[0058] The displ_ref_list_struct may contain a drl_abs_delta_dfoc_st that indicates the absolute value of the difference between the display order of a frame and the display order of a short-term reference frame.
[0059] The displ_ref_list_struct may include a drl_dfoc_lsb_lt that indicates the difference between the display order of a frame and the maximum value of the display order of frames of the displacement amount.
[0060] 15 is a diagram showing an example of the syntax configuration of displ_data_unit. displ_data_unit may include ddu_lod_count, which indicates the number of subdivisions.
[0061] (Displacement amount decoding unit 204) Fig. 16 is a diagram showing an example of the configuration of the displacement amount decoding unit 204. As shown in Fig. 16, the displacement amount decoding unit 204 includes a payload decoding unit 204A, a displacement amount header decoding unit 204B, and a displacement amount data decoding unit 204C.
[0062] The payload decoder 204A is configured to decode and output the payload from the displacement bitstream.
[0063] The displacement amount header decoder 204B is configured to decode and output the reference list from the payload.
[0064] The displacement data decoding unit 204C is configured to decode and output the coefficient sequence, the number of displacement vectors, and the number of subdivisions from the payload and the reference list.
[0065] Fig. 17 is a diagram showing an example of the configuration of the displacement amount header decoding unit 204B. As shown in Fig. 17, the displacement amount header decoding unit 204B includes a frame display order derivation unit 204B1, a reference list creation unit 204B2, a reference frame marking unit 204B3, and a reference list storage unit 204B4.
[0066] The frame display order derivation unit 204B1 is configured to derive and output a frame display order from the payload.
[0067] The reference list storage unit 204B4 is configured to store the reference list constructed by the reference list construction unit 204B2. The reference list is configured from a list indicating, for each target frame, whether each frame other than the target frame is a reference frame.
[0068] The reference list construction unit 204B2 is configured to construct and output a reference list based on the frame display order.
[0069] The reference list construction unit 204B2 is configured to construct a reference list using the frame display order and displ_ref_list_struct.
[0070] The reference list constructing section 204B2 may construct the reference list using drl_abs_delta_dfoc_st, or may construct the reference list using drl_dfoc_lsb_lt.
[0071] The reference list constructor 204B2 may construct a reference list using drl_abs_delta_dfoc_st when the reference frame is a short-term reference frame, and may construct a reference list using drl_dfoc_lsb_lt when the reference frame is a long-term reference frame.
[0072] 18 is a diagram showing an example of a prediction structure and a reference list constructed therefrom. The example in Fig. 18 shows a reference list when frame 6 is the target frame.
[0073] The reference frame marking unit 204B3 is configured to mark the status of the reference frames in the reference list using the frame display order.
[0074] The reference frame marking unit 204B3 may mark each reference frame in the reference list as either a "long-term reference frame," a "short-term reference frame," or a "non-reference frame" using the frame display order.
[0075] Reference frames marked as "long-term reference frames" are not removed from the reference list. Reference frames marked as "non-reference frames" may be removed from the reference list.
[0076] Fig. 19 is a diagram showing an example of marking of a reference list. The example in Fig. 19 shows a reference list when frame 6 is the target frame and a reference list when frame 7 is the target frame.
[0077] When frame 6 is the target frame, reference frame marking section 204B3 marks frames 1 and 4 as "long-term reference frames" and marks frame 5 as "short-term reference frame."
[0078] When frame 7 is the target frame, the reference frame marking unit 204B3 marks frames 1 and 4 as "long-term reference frames", marks frame 5 as "non-reference frame", and marks frame 6 as "short-term reference frame".
[0079] The above-described processing by the displacement amount header decoding unit 204B constructs a reference list, enabling inter prediction, which will be described later.
[0080] Fig. 20 is a diagram showing an example of the configuration of the displacement amount data decoding unit 204C. As shown in Fig. 20, the displacement amount data decoding unit 204C includes an arithmetic decoding unit 204C1, a frame buffer 204C2, an inter prediction unit 204C3, an adder 204C4, and an inverse quantization unit 204C5.
[0081] The arithmetic decoding unit 204C1 is configured to decode the level value sequence, the number of displacement vectors, and the number of subdivisions from the displacement bit stream by arithmetic decoding, and output the decoded level value sequence, the number of displacement vectors, and the number of subdivisions.
[0082] The arithmetic decoding unit 204C1 may decode the level value sequence, the number of displacement vectors, and the number of subdivisions as in Japanese Patent Application No. 2022-212452.
[0083] The frame buffer 204C2 is configured to acquire and store a level value sequence from the arithmetic decoding unit 204C1 or the adder 204C4. The frame buffer 204C2 is configured to output a level value sequence in a reference frame in accordance with control information (not shown).
[0084] The inter prediction unit 204C3 is configured to read out the reference frame indicated by the reference list from the frame buffer, and perform inter prediction using the level value sequence, thereby generating and outputting a prediction level value sequence.
[0085] The inter prediction unit 204C3 may determine the predicted level value of each frequency in the current frame by directly referring to the level value of the corresponding frequency in the reference frame.
[0086] The adder 204C4 is configured to obtain a prediction level value sequence from the inter prediction unit 204C3 and a level value sequence (actually, a prediction residual sequence) from the arithmetic decoding unit 204C1, and add them together to generate a level value sequence. The generated level value sequence is output to the inverse quantization unit 204C5 and the frame buffer 204C2.
[0087] The inverse quantization unit 204C5 is configured to inversely quantize the level value sequence to generate and output a coefficient sequence.
[0088] Fig. 21 is a diagram showing a modified example of the configuration of the displacement amount data decoding unit 204C. Hereinafter, with reference to Fig. 21, this modified example will be described focusing on the differences from the displacement amount data decoding unit 204C shown in Fig. 20.
[0089] The inter prediction unit 204C3 is configured to read out the reference frame indicated by the reference list from the frame buffer, and perform inter prediction using the coefficient sequence, thereby generating and outputting a prediction coefficient sequence.
[0090] The inter prediction unit 204C3 may determine prediction coefficients for each frequency in the current frame by directly referencing the coefficients for the corresponding frequency in the reference frame.
[0091] The adder 204C4 is configured to obtain a prediction coefficient sequence from the inter prediction unit 204C3 and a coefficient sequence (actually, a prediction residual sequence) from the inverse quantization unit 204C5, and add them together to generate a coefficient sequence. The generated coefficient sequence is output to the frame buffer 204C2.
[0092] (Mesh reconstruction unit 206) Fig. 22 is a diagram showing an example of the configuration of the mesh reconstructing unit 206. As shown in Fig. 22, the mesh reconstructing unit 206 includes a syntax matching unit 206A, a submesh reconstructing unit 206B, and a submesh combining unit 206C.
[0093] The syntax matching unit 206A is configured to generate and output a matching base mesh, a matching coefficient sequence, a matching displacement vector number, and a matching subdivision number using the atlas, the base mesh, the coefficient sequence, the displacement vector number, and the subdivision number.
[0094] The syntax matching unit 206A may output the smaller of asve_displacement_num and the number of displacement vectors as the number of matching displacement vectors.
[0095] The syntax matching unit 206A may hold a compatibility requirement such that asve_displacement_num and the number of displacement vectors match.
[0096] The syntax matching unit 206A may output the smaller of asve_subdivision_iteration_count and the number of subdivisions as the matching number of subdivisions.
[0097] The syntax matching unit 206A may hold a compatibility requirement such that asve_subdivision_iteration_count and the number of subdivisions match.
[0098] The above-described processing by the syntax matching unit 206A can resolve inconsistencies between the atlas bitstream and the displacement amount bitstream.
[0099] The submesh reconstructing unit 206B is configured to generate and output a submesh sequence using the matching basic mesh, the matching coefficient sequence, the number of matching displacement vectors, and the number of matching subdivisions.
[0100] The submesh combining unit 206B is configured to combine the submesh rows to generate and output a single mesh.
[0101] Fig. 23 is a diagram showing an example of the configuration of the submesh reconstructing unit 206B. As shown in Fig. 23, the submesh reconstructing unit 206B includes a subdivision unit 206B1, an inverse conversion unit 206B2, and a displacement amount adding unit 206B3.
[0102] The subdivision unit 206B1 is configured to generate the added subdivision vertices and their connection information from the matching base mesh according to the subdivision method indicated by the control information.
[0103] The subdivision unit 206B1 outputs a subdivision mesh that combines the basic mesh, the added subdivision vertices, and their connection information.
[0104] The inverse transform unit 206B2 is configured to inversely transform the coefficient sequence to generate and output a displacement sequence. The inverse transform unit 206B2 may inversely transform the coefficient sequence using an inverse DCT or an inverse wavelet transform.
[0105] The displacement amount adding unit 206B3 is configured to generate and output sub-meshes by adding the displacement amount sequence to the finely divided mesh.
[0106] (Atlas decoding unit 202) Fig. 27 is a diagram showing an example of the configuration of the atlas data decoding unit 202. As shown in Fig. 271, the atlas data decoding unit 202 includes an atlas sequence parameter set decoding unit 202A, an atlas frame parameter set decoding unit 202B, and an atlas tile decoding unit 202C.
[0107] The atlas sequence parameter set decoding unit 202A is configured to decode the atlas sequence parameter set (ASPS), which is a parameter related to a sequence, and its extension, the atlas sequence parameter set V-DMC extension (ASVE), for 3D mesh compression.
[0108] As shown in FIG. 27, the atlas sequence parameter set decoding unit 202A includes an ASPS decoding unit and an ASVE decoding unit 202A1.
[0109] The atlas frame parameter set decoding unit 207B is configured to decode the Atlas Sequence Parameter Set (AFPS), which is a parameter related to a frame, and its extension AFVE (Atlas Frame Parameter Set V-DMC Extension) for 3D mesh compression.
[0110] The atlas tile decoder 207C is configured to decode various information about tiles and about patches within tiles.
[0111] (ASVE decoding unit 202A1) The ASVE decoding unit 202A1 includes a displacement codec determining unit 202A1a, a displacement video codec dedicated parameter decoding unit 202A1b, and other ASVE parameters 202A1c.
[0112] The ASVE decoding unit 202A1 is configured to decode the ASVE parameters using a displacement codec determining unit 202A1a and another ASVE parameter 202A1c.
[0113] The displacement video codec dedicated parameter decoder 202A1b decodes parameters that are required only when the displacement video codec is used.
[0114] For example, the displacement video codec-specific parameter decoder 202A1b may decode the image packing method asve_packing_method.
[0115] The other ASVE parameter decoder 202A1c decodes parameters commonly used when the displacement amount video codec is used or parameters not required when the displacement amount video codec is used.
[0116] For example, the other ASVE parameter decoder 202A1c may decode asve_subdivision_method, which indicates the subdivision method, and asve_num_attribute_video, which indicates the number of videos of the attribute.
[0117] (Processing of ASVE decoding unit 202A1) FIG. 24 is a flowchart showing an example of the processing of the ASVE decoding unit 202A1.
[0118] In step S101, the displacement codec determination unit 202A1a of the ASVE decoding unit 202A1 determines whether the displacement is decoded using a video codec. If it is decoded using a video codec (for example, if ptl_profile_codec_group_idc is an integer ranging from 0 to 4 or ptl_profile_codec_group_idc is 127), the operation proceeds to step S102; otherwise, the operation proceeds to step S103.
[0119] In step S102, the displacement video codec-specific parameter decoding unit 202A1b of the ASVE decoding unit 202A1 decodes the image packing method asve_packing_method.
[0120] In step S103, the other ASVE parameter decoding unit 202A1c of the ASVE decoding unit 202A1 decodes the syntax in the other ASVE, and this operation ends.
[0121] The image packing method is described as the syntax of ASVE, but it is the same even if it is moved to another parameter set. For example, even if the image packing method is moved to the displacement amount bit stream, the same processing is performed by replacing the atlas with the displacement amount. (Processing of the reference list construction unit 204B2) FIG. 25 is a flowchart showing an example of the processing of the reference list creating section 204B2.
[0122] As shown in FIG. 25, in step S101, the reference list constructor 204B2 determines whether drl_st_ref_displ_frame_flag is 1 or 0.
[0123] If drl_st_ref_displ_frame_flag=1, the operation proceeds to step S102, and if drl_st_ref_displ_frame_flag=0, the operation proceeds to step S103.
[0124] In step S102, the reference list constructor 204B2 calculates a reference frame using drl_abs_delta_dfoc_st.
[0125] In step S103, the reference list constructor 204B2 calculates a reference frame using drl_dfoc_lsb_lt.
[0126] In step S104, the reference list creator 204B2 registers the reference frame.
[0127] In step S105, reference list creator 204B2 determines whether the currently processed frame is the last frame. If Yes, the operation ends, and if No, the operation proceeds to step S106.
[0128] In step S106, reference list creator 204B2 proceeds to processing the next frame.
[0129] (Processing of reference frame marking section 204B3) FIG. 26 is a flowchart showing an example of the processing of the reference frame marking unit 204B3.
[0130] 26, in step S101, the reference frame marking unit 204B3 determines whether or not it is a long-term reference frame. If it is a long-term reference frame, the operation proceeds to step S102, and if not, the operation proceeds to step S103.
[0131] In step S102, the reference frame marking unit 204B3 marks it as a "long-term reference frame."
[0132] In step S103, the reference frame marking unit 204B3 determines whether it is a short-term reference frame. If it is a short-term reference frame, the operation proceeds to step S104, otherwise the operation proceeds to step S105.
[0133] In step S104, the reference frame marking unit 204B3 marks it as a "short-term reference frame."
[0134] In step S105, the reference frame marking unit 204B3 marks the frame as a "non-reference frame."
[0135] In step S106, the reference frame marking unit 204B3 determines whether the currently processed frame is the last one. If yes, the operation ends, and if no, the operation proceeds to step S107.
[0136] In step S107, the reference frame marking unit 204B3 proceeds to process the next frame.
[0137] According to this embodiment, the reference list is constructed by the processing of the displacement amount header decoding unit 204B, thereby enabling inter prediction.
[0138] Furthermore, according to this embodiment, the processing of the syntax matching unit 206A can resolve inconsistencies between the atlas bitstream and the displacement amount bitstream.
[0139] The mesh encoding device 100 and the mesh decoding device 200 described above may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]
[0140] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which will contribute to the achievement of Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]
[0141] 1...Mesh processing system 100...Mesh coding device 200...Mesh decoding device 201...V3C unit separation section 202...Atlas decoding unit 202A... Atlas sequence parameter set decoding unit 202A1...ASVE decoding unit 202 202A1a...Displacement amount codec determination unit 202A1b...Displacement amount video codec dedicated parameter decoding unit 202A1c...Other ASVE parameters 202B... Atlas frame parameter set decoding unit 202C...Atlas Style Decoding Unit 203...Basic mesh decoding unit 204...Displacement amount decoding unit 204A...Payload decoding unit 204B...Displacement amount header decoding unit 204B1...Frame display order derivation unit 204B2...Reference list construction section 204B3...Reference frame marking section 204B4...Reference list storage section 204C...Displacement amount data decoding unit 204C1...Arithmetic decoding unit and 204C2...frame buffer 204C3...Inter prediction section 204C4...adder 204C5...Dequantization section 205...Texture decoding unit 206...Mesh reconstruction unit 206A...Syntax matching section 206B...Submesh reconstruction section 206B1…Subdivision section 206B2...inverse conversion unit 206B3...Displacement amount adding unit 206C...Submesh joint
Claims
1. A mesh decoding device, comprising: a mesh reconstructing unit configured to reconstruct and output a mesh based on the atlas, basic mesh, coefficient sequence, number of displacement vectors, and number of subdivisions obtained from the bit stream; The mesh reconstruction unit a syntax matching unit configured to generate and output a matching base mesh, a matching coefficient sequence, a matching displacement vector number, and a matching subdivision number using the atlas, the base mesh, the coefficient sequence, the displacement vector number, and the subdivision number; a submesh reconstructing unit configured to generate and output a submesh sequence using the matched basic mesh, the matched coefficient sequence, the number of matched displacement vectors, and the number of matched subdivisions; a submesh combining unit configured to combine the submesh sequences to generate and output a single mesh.
2. The mesh decoding device according to claim 1 , wherein the syntax matching unit outputs, as the number of matching displacement vectors, a smaller one of the number of displacement vectors defined in the atlas and the number of displacement vectors.
3. The mesh decoding device according to claim 1 , wherein the syntax matching unit outputs, as the matched subdivision number, a smaller one of a subdivision number defined in the atlas and the subdivision number.
4. 1. A mesh decoding method, comprising: a step A of reconstructing and outputting a mesh based on the atlas, basic mesh, coefficient sequence, number of displacement vectors, and number of subdivisions obtained from the bitstream; The step A is a step of generating and outputting a matching base mesh, a matching coefficient sequence, a matching displacement vector number, and a matching subdivision number using the atlas, the base mesh, the coefficient sequence, the displacement vector number, and the subdivision number; a step of generating and outputting a submesh sequence using the matched basic mesh, the matched coefficient sequence, the number of matched displacement vectors, and the number of matched subdivisions; and generating and outputting a single mesh by combining the submesh sequences.
5. A program that causes a computer to function as a mesh decoding device, The mesh decoding device comprises: a mesh reconstructing unit configured to reconstruct and output a mesh based on the atlas, basic mesh, coefficient sequence, number of displacement vectors, and number of subdivisions obtained from the bit stream; The mesh reconstruction unit a syntax matching unit configured to generate and output a matching base mesh, a matching coefficient sequence, a matching displacement vector number, and a matching subdivision number using the atlas, the base mesh, the coefficient sequence, the displacement vector number, and the subdivision number; a submesh reconstructing unit configured to generate and output a submesh sequence using the matched basic mesh, the matched coefficient sequence, the number of matched displacement vectors, and the number of matched subdivisions; a submesh combining unit configured to combine the submesh strings to generate and output a single mesh.