Mesh decoding device, mesh decoding method, and program

Abstract

The present invention makes it possible to eliminate inconsistency between an atlas bit stream and a displacement amount bit stream. In a mesh decoding device 200 according to the present invention, a mesh reconfiguration unit 206 includes: a syntax matching unit 206A configured to generate and output a matching basic mesh, a matching coefficient sequence, a matching displacement amount vector number, and a matching subdivision frequency by using an atlas, a basic mesh, a coefficient sequence, a displacement amount vector number, and a subdivision number; a sub-mesh reconfiguration unit 206B configured to generate and output a sub-mesh sequence using the matching basic mesh, the matching coefficient sequence, the matching displacement amount vector number, and the matching subdivision number; and a sub-mesh combining unit 206C configured to generate and output a single mesh by combining the sub-mesh sequences.

Description

Mesh decoding device, mesh decoding method and program

[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program.

[0002] Non-Patent Document 1 discloses a technique for decoding a point group using video information and additional information called an atlas.

[0003] Non-Patent Document 2 discloses a technique that extends Non-Patent Document 1 to mesh decoding applications.

[0004] Specifically, Non-Patent Document 2 first decodes a rough base mesh and a detailed displacement, and then decodes a mesh from both of them. Here, a syntax indicating a reference list structure is defined for inter-prediction of displacement.

[0005] “Information technology - Coded Representation of Immersive Media - Part 5: Visual Volumetric Video-based Coding (V3C) and Video-based Point Cloud Compression (V-PCC),” ISO / IEC JTC 1 / SC 29 / WG 7, ISO / IEC 23090-5:2021(2E). “WD 6.0 of V-DMC,” ISO / IEC JTC 1 / SC 29 / WG 7 N00822.

[0006] However, the method of Non-Patent Document 2 had a problem in that if there was a syntax inconsistency (number of vectors of the displacement amount or number of subdivisions) between the decoded atlas and the displacement amount, the mesh could not be decoded correctly.

[0007] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a mesh decoding device, a mesh decoding method, and a program that can eliminate inconsistencies between the atlas bitstream and the displacement amount bitstream.

[0008] A first feature of the present invention is summarized as a mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a base mesh, a coefficient sequence, and a displacement vector number obtained from a bitstream, wherein the mesh reconstruction unit comprises: a syntax matching unit configured to generate and output a matching base mesh, a matching coefficient sequence, and a matching displacement vector number using the atlas, the base mesh, the coefficient sequence, and the displacement vector number; a submesh reconstruction unit configured to generate and output a submesh sequence using the matching base mesh, the matching coefficient sequence, and the matching displacement vector number; and a submesh combining unit configured to generate and output a single mesh by combining the submesh sequences.

[0009] A second feature of the present invention is summarized as a mesh decoding method comprising a step A of reconstructing and outputting a mesh based on an atlas, a base mesh, a coefficient sequence, and a displacement vector number obtained from a bitstream, wherein the step A comprises a step of generating and outputting a matching base mesh, a matching coefficient sequence, and a matching displacement vector number using the atlas, the base mesh, the coefficient sequence, and the displacement vector number; a step of generating and outputting a submesh sequence using the matching base mesh, the matching coefficient sequence, and the matching displacement vector number; and a step of generating and outputting a single mesh by combining the submesh sequences.

[0010] A third feature of the present invention is summarized as a program that causes a computer to function as a mesh decoding device, the mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a base mesh, a coefficient sequence, and a displacement vector number obtained from a bitstream, the mesh reconstruction unit comprising: a syntax matching unit configured to generate and output a matching base mesh, a matching coefficient sequence, and a matching displacement vector number using the atlas, the base mesh, the coefficient sequence, and the displacement vector number; a submesh reconstruction unit configured to generate and output a submesh sequence using the matching base mesh, the matching coefficient sequence, and the matching displacement vector number; and a submesh combining unit configured to generate and output a single mesh by combining the submesh sequences.

[0011] According to the present invention, it is possible to provide a mesh decoding device, a mesh decoding method and a program that can resolve the inconsistency between the atlas bitstream and the displacement bitstream.

[0012] FIG. 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment. FIG. 2 is a diagram showing an example of functional blocks of a mesh decoding device 200 according to an embodiment. FIG. 3 is a diagram showing an example of the configuration of a V3C unit. FIG. 4 is a diagram showing an example of the syntax configuration of a V3C unit header. FIG. 5 is a diagram showing an example of the configuration of a bit stream. FIG. 6 is a diagram showing an example of the syntax configuration of profile_tier_level. FIG. 7 is a diagram showing an example of the syntax configuration of asps_vdmc_extension. FIG. 8 is a diagram showing an example of the configuration of a displacement amount bit stream. FIG. 9 is a diagram showing an example of the syntax configuration of displ_nal_unit_header. FIG. 10 is a diagram showing an example of the correspondence between displ_nal_unit_type and the contents of the payload. Fig. 11 is a diagram showing an example of the syntax configuration of displ_sequence_parameter_set_rbsp. Fig. 12 is a diagram showing an example of the syntax configuration of displ_layer_rbsp. Fig. 13 is a diagram showing an example of the syntax configuration of displ_header. Fig. 14 is a diagram showing an example of the syntax configuration of displ_ref_list_struct. Fig. 15 is a diagram showing an example of the syntax configuration of displ_data_unit. Fig. 16 is a diagram showing an example of functional blocks of a displacement amount decoding unit 204 of a mesh decoding device 200 according to an embodiment. Fig. 17 is a diagram showing an example of functional blocks of a displacement amount header decoding unit 204B of the displacement amount decoding unit 204 of a mesh decoding device 200 according to an embodiment. Fig. 18 is a diagram showing an example of a prediction structure and a reference list constructed therefrom. Fig. 19 is a diagram showing an example of marking of a reference list. Fig. 20 is a diagram showing an example of functional blocks of a displacement amount data decoding unit 204C of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. Fig. 21 is a diagram showing a modified example of the functional blocks of the displacement amount data decoding unit 204C of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment.FIG. 22 is a diagram showing an example of functional blocks of the mesh reconstructing unit 206 of the mesh decoding device 200 according to an embodiment. FIG. 23 is a diagram showing an example of functional blocks of the sub-mesh reconstructing unit 206B of the mesh reconstructing unit 206 of the mesh decoding device 200 according to an embodiment. FIG. 24 is a flowchart showing an example of processing by the ASVE decoding unit 202A1 of the atlas sequence parameter set decoding unit 202A of the atlas decoding unit 202 of the mesh decoding device 200 according to an embodiment. FIG. 25 is a flowchart showing an example of processing by the reference list constructing unit 204B2 of the displacement header decoding unit 204B of the displacement decoding unit 204 of the mesh decoding device 200 according to an embodiment. FIG. 26 is a flowchart showing an example of processing by the reference frame marking unit 204B3 of the displacement header decoding unit 204B of the displacement decoding unit 204 of the mesh decoding device 200 according to an embodiment. FIG. 27 is a diagram showing an example of the configuration of the atlas data decoding unit 202.

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0014] First Embodiment This embodiment is based on V3C described in Non-Patent Document 1. An atlas in V3C is a collection of rectangular regions and related information arranged on a frame, and corresponds to a three-dimensional space.

[0015] Hereinafter, a mesh processing system 1 according to this embodiment will be described with reference to FIGS.

[0016] 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to this embodiment. As shown in FIG. 1, the mesh processing system 1 includes a mesh encoding device 100 and a mesh decoding device 200.

[0017] FIG. 2 is a diagram showing an example of functional blocks of a mesh decoding device 200 according to this embodiment.

[0018] As shown in FIG. 2, the mesh decoding device 200 includes a V3C unit separation unit 201, an atlas decoding unit 202, a basic mesh decoding unit 203, a displacement amount decoding unit 204, a texture decoding unit 205, and a mesh reconstruction unit 206.

[0019] The V3C unit separation unit 201 is configured to separate the bitstream into an atlas bitstream, a basic mesh bitstream, a displacement bitstream, and a texture bitstream.

[0020] The atlas decoding unit 202 is configured to decode and output the atlas from the atlas bitstream.

[0021] The base mesh decoding unit 203 is configured to decode and output base meshes from the base mesh bitstream.

[0022] The displacement decoding unit 204 is configured to decode and output the coefficient sequence, the number of displacement vectors, and the number of subdivisions from the displacement bitstream.

[0023] The texture decoding unit 205 is configured to decode and output the texture from the texture bitstream. The texture decoding unit 205 may decode the texture using HEVC.

[0024] The mesh reconstruction unit 206 is configured to reconstruct and output a mesh using the atlas decoded by the atlas decoding unit 201, the basic mesh decoded by the basic mesh decoding unit 203, and the coefficient sequence, number of displacement vectors, and number of subdivisions decoded by the displacement decoding unit 204.

[0025] (Bitstream Structure) A bitstream may be formed by combining a plurality of V3C units.

[0026] 3 is a diagram showing an example of the configuration of a V3C unit. A V3C unit may be composed of a V3C unit header and a V3C unit payload.

[0027] 4 is a diagram showing an example of the syntax configuration of a V3C unit header. The V3C unit header may include vuh_unit_type, which indicates the type of the V3C unit.

[0028] 5 is a diagram showing an example of the configuration of a bitstream. The bitstream may be composed of a V3C parameter set, an atlas bitstream, a base mesh bitstream, a displacement bitstream, and a texture bitstream.

[0029] vuh_unit_type may store any of the following: V3C_VPS indicating a V3C parameter set, V3C_AD indicating an atlas bitstream, V3C_BMD indicating a basic mesh bitstream, V3C_GVD indicating a displacement bitstream, and V3C_AVD indicating a texture bitstream.

[0030] The V3C parameter set may include a profile_tier_level that indicates the profile tier level.

[0031] FIG. 6 is a diagram illustrating an example of the syntax configuration of profile_tier_level.

[0032] The profile_tier_level may include a ptl_profile_codec_group_idc that indicates the codec ID of the displacement amount.

[0033] (Configuration of Atlas Bitstream) The atlas bitstream may use afps_vdmc_extension, which indicates frame parameters that are an extension of the V3C frame parameter set.

[0034] 7 is a diagram showing an example of the syntax configuration of asps_vdmc_extension. asps_vdmc_extension may include asve_displacement_num, which indicates the number of vectors of the displacement amount. asps_vdmc_extension may include asve_subdivision_iteration_count, which indicates the number of subdivisions.

[0035] The asps_vdmc_extension may include an asve_packing_method that indicates an image packing method for the displacement. The asve_packing_method is decoded only when the codec of the displacement is a video codec, and may not be decoded otherwise.

[0036] The codec of the displacement amount may be determined to be a video codec when ptl_profile_codec_group_idc is an integer in the range of 0 to 4, or when ptl_profile_codec_group_idc is 127.

[0037] (Structure of Displacement Amount Bitstream) Fig. 8 is a diagram showing an example of the structure of a displacement amount bitstream. The displacement amount bitstream may have a V3C unit payload configured with displ_nal_unit indicating a NAL unit.

[0038] The displ_nal_unit may be composed of a displ_nal_unit_header indicating the header of the NAL unit in the displacement amount bitstream and the payload of the NAL unit in the displacement amount bitstream.

[0039] FIG. 9 is a diagram illustrating an example of the syntax configuration of the display_nal_unit_header.

[0040] The display_nal_unit_header may be composed of a display_nal_forbidden_zero_bit indicating the leading write-protected section and a display_nal_unit_type indicating the type of payload data.

[0041] FIG. 10 is a diagram showing an example of the correspondence between the display_nal_unit_type and the contents of the payload.

[0042] The contents of the payload may be defined as displ_sequence_parameter_set_rbsp indicating the sequence parameter set of the displacement amount when displ_nal_unit_type is 0, as displ_frame_parameter_set_rbsp indicating the frame parameter set of the displacement amount when displ_nal_unit_type is 1, and as displ_layer_rbsp indicating the displacement amount data when displ_nal_unit_type is 2.

[0043] FIG. 11 is a diagram illustrating an example of the syntax configuration of display_sequence_parameter_set_rbsp.

[0044] The displ_sequence_parameter_set_rbsp may include dsps_displacement_num, which indicates the number of vectors of the displacement amount.

[0045] The display_sequence_parameter_set_rbsp may include dsps_log2_max_displ_frame_order_cnt_lsb indicating the maximum value of the frame display order of the displacement amount.

[0046] The display_sequence_parameter_set_rbsp may include dsps_max_dec_displ_frame_buffering, which indicates the maximum value of the displacement buffer size.

[0047] The display_sequence_parameter_set_rbsp may include dsps_num_ref_displ_frame_lists_in_dsps, which indicates the number of reference lists for the displacement amount.

[0048] The display_sequence_parameter_set_rbsp may include a display_ref_list_struct that indicates a reference list of displacement amounts.

[0049] 12 is a diagram showing an example of the syntax configuration of display_layer_rbsp. display_layer_rbsp may include display_header indicating a displacement amount header and display_data_unit indicating displacement amount data.

[0050] FIG. 13 is a diagram showing an example of the syntax configuration of the display_header.

[0051] The display_header may include a dh_id that indicates the ID of the displacement amount. The display_header may include a dh_from_order_cnt_lsb that indicates the display order of the displacement amount frames.

[0052] The display_header may contain a dh_ref_displ_frame_list_dsps_flag that indicates whether the reference list is derived from the display_sequence_parameter_set_rbsp or from the display_header.

[0053] The displ_header may contain the displ_ref_list_struct described above.

[0054] The display_header may contain ref_displ_frame_list_idx, which indicates the index of the reference list in the display_sequence_parameter_set_rbsp.

[0055] FIG. 14 is a diagram illustrating an example of the syntax configuration of displ_ref_list_struct.

[0056] The displ_ref_list_struct may contain drl_num_ref_entries, which indicates the number of frames in the displ_ref_list_struct.

[0057] The displ_ref_list_struct may contain a drl_st_ref_displ_frame_flag that indicates whether a frame is a short-term or long-term reference frame.

[0058] The display_ref_list_struct may include a drl_abs_delta_dfoc_st that indicates the absolute value of the difference between the display order of a frame and the display order of a short-term reference frame.

[0059] The display_ref_list_struct may include a drl_dfoc_lsb_lt that indicates the difference between the display order of a certain frame and the maximum value of the display order of frames of the displacement amount.

[0060] 15 is a diagram showing an example of the syntax configuration of displ_data_unit. The displ_data_unit may include ddu_lod_count, which indicates the number of subdivisions.

[0061] (Displacement Amount Decoding Unit 204) Fig. 16 is a diagram showing an example of the configuration of the displacement amount decoding unit 204. As shown in Fig. 16, the displacement amount decoding unit 204 includes a payload decoding unit 204A, a displacement amount header decoding unit 204B, and a displacement amount data decoding unit 204C.

[0062] The payload decoder 204A is configured to decode and output the payload from the displacement bitstream.

[0063] The displacement amount header decoder 204B is configured to decode and output the reference list from the payload.

[0064] The displacement data decoding unit 204C is configured to decode and output the coefficient sequence, the number of displacement vectors, and the number of subdivisions from the payload and the reference list.

[0065] Fig. 17 is a diagram showing an example of the configuration of the displacement amount header decoding unit 204B. As shown in Fig. 17, the displacement amount header decoding unit 204B includes a frame display order derivation unit 204B1, a reference list construction unit 204B2, a reference frame marking unit 204B3, and a reference list storage unit 204B4.

[0066] The frame display order derivation unit 204B1 is configured to derive and output a frame display order from the payload.

[0067] The reference list storage unit 204B4 is configured to store the reference list constructed by the reference list construction unit 204B2. The reference list is configured of a list indicating, for each target frame, whether each frame other than the target frame is a reference frame.

[0068] The reference list creation unit 204B2 is configured to create and output a reference list based on the frame display order.

[0069] The reference list construction unit 204B2 is configured to construct a reference list using the frame display order and display_ref_list_struct.

[0070] The reference list constructing unit 204B2 may construct the reference list using drl_abs_delta_dfoc_st, or may construct the reference list using drl_dfoc_lsb_lt.

[0071] The reference list construction unit 204B2 may construct a reference list using drl_abs_delta_dfoc_st when the reference frame is a short-term reference frame, and may construct a reference list using drl_dfoc_lsb_lt when the reference frame is a long-term reference frame.

[0072] 18 is a diagram showing an example of a prediction structure and a reference list constructed therefrom. The example in Fig. 18 shows a reference list when frame 6 is the target frame.

[0073] The reference frame marking unit 204B3 is configured to use the frame display order to mark the status of the reference frames in the reference list.

[0074] The reference frame marking unit 204B3 may mark each reference frame in the reference list as either a "long-term reference frame," a "short-term reference frame," or a "non-reference frame" using the frame display order.

[0075] Reference frames marked as "long-term reference frames" are not removed from the reference list. Reference frames marked as "non-reference frames" may be removed from the reference list.

[0076] 19 is a diagram showing an example of markings in a reference list, in which frame 6 is the target frame and frame 7 is the target frame.

[0077] When frame 6 is the target frame, the reference frame marking unit 204B3 marks frames 1 and 4 as "long-term reference frames" and marks frame 5 as "short-term reference frame."

[0078] When frame 7 is the target frame, the reference frame marking unit 204B3 marks frames 1 and 4 as "long-term reference frames," marks frame 5 as "non-reference frame," and marks frame 6 as "short-term reference frame."

[0079] The above-described processing by the displacement amount header decoding unit 204B constructs a reference list, enabling inter prediction, which will be described later.

[0080] Fig. 20 is a diagram showing an example of the configuration of the displacement amount data decoding unit 204 C. As shown in Fig. 20, the displacement amount data decoding unit 204 C includes an arithmetic decoding unit 204 C1, a frame buffer 204 C2, an inter prediction unit 204 C3, an adder 204 C4, and an inverse quantization unit 204 C5.

[0081] The arithmetic decoding unit 204C1 is configured to decode the level value sequence, the number of displacement vectors, and the number of subdivisions from the displacement bit stream by arithmetic decoding, and output the decoded level value sequence, the number of displacement vectors, and the number of subdivisions.

[0082] The arithmetic decoding unit 204C1 may decode the level value sequence, the number of displacement vectors, and the number of subdivisions as in Japanese Patent Application No. 2022-212452.

[0083] The frame buffer 204C2 is configured to acquire and store a level value sequence from the arithmetic decoding unit 204C1 or the adder 204C4, and is configured to output a level value sequence in a reference frame in accordance with control information (not shown).

[0084] The inter prediction unit 204C3 is configured to read out the reference frame indicated by the reference list from the frame buffer, and perform inter prediction using the level value sequence, thereby generating and outputting a prediction level value sequence.

[0085] The inter prediction unit 204C3 may determine the predicted level value of each frequency in the current frame by directly referring to the level value of the corresponding frequency in the reference frame.

[0086] The adder 204C4 is configured to obtain a prediction level value sequence from the inter prediction unit 204C3 and a level value sequence (actually, a prediction residual sequence) from the arithmetic decoding unit 204C1, and add them together to generate a level value sequence. The generated level value sequence is output to the inverse quantization unit 204C5 and the frame buffer 204C2.

[0087] The inverse quantization unit 204C5 is configured to inversely quantize the level value sequence to generate and output a coefficient sequence.

[0088] Fig. 21 is a diagram showing a modified example of the configuration of the displacement amount data decoding unit 204C. Hereinafter, with reference to Fig. 21, this modified example will be described, focusing on the differences from the displacement amount data decoding unit 204C shown in Fig. 20.

[0089] The inter prediction unit 204C3 is configured to read the reference frame indicated by the reference list from the frame buffer, and perform inter prediction using the coefficient sequence, thereby generating and outputting a prediction coefficient sequence.

[0090] The inter prediction unit 204C3 may determine prediction coefficients for each frequency in the current frame by directly referencing the coefficients for the corresponding frequency in the reference frame.

[0091] The adder 204C4 is configured to receive the prediction coefficient sequence from the inter prediction unit 204C3 and the coefficient sequence (actually, the prediction residual sequence) from the inverse quantization unit 204C5, and add them together to generate a coefficient sequence, which is output to the frame buffer 204C2.

[0092] (Mesh Reconstruction Unit 206) Fig. 22 is a diagram showing an example of the configuration of the mesh reconstruction unit 206. As shown in Fig. 22, the mesh reconstruction unit 206 includes a syntax matching unit 206A, a submesh reconstruction unit 206B, and a submesh combining unit 206C.

[0093] The syntax matching unit 206A is configured to generate and output a matching base mesh, a matching coefficient sequence, a matching displacement vector number, and a matching subdivision number using the atlas, the base mesh, the coefficient sequence, the displacement vector number, and the subdivision number.

[0094] The syntax matching unit 206A may output the smaller of asve_displacement_num and the number of displacement vectors as the number of matching displacement vectors.

[0095] The syntax matching unit 206A may hold a compatibility requirement such that asve_displacement_num matches the number of displacement vectors.

[0096] The syntax matching unit 206A may output the smaller of asve_subdivision_iteration_count and the number of subdivisions as the matching number of subdivisions.

[0097] The syntax matching unit 206A may hold a conformance requirement such that asve_subdivision_iteration_count and the number of subdivisions match.

[0098] The above-described processing by the syntax matching unit 206A can resolve inconsistencies between the atlas bitstream and the displacement amount bitstream.

[0099] The submesh reconstructing unit 206B is configured to generate and output a submesh sequence using the matching basic mesh, the matching coefficient sequence, the number of matching displacement vectors, and the number of matching subdivisions.

[0100] The submesh combining unit 206B is configured to combine the submesh rows to generate and output a single mesh.

[0101] Fig. 23 is a diagram showing an example of the configuration of the submesh reconstruction unit 206B. As shown in Fig. 23, the submesh reconstruction unit 206B includes a subdivision unit 206B1, an inverse conversion unit 206B2, and a displacement amount addition unit 206B3.

[0102] The subdivision unit 206B1 is configured to generate the added subdivision vertices and their connection information from the matching base mesh according to the subdivision method indicated by the control information.

[0103] The subdivision unit 206B1 outputs a subdivision mesh that combines the basic mesh, the added subdivision vertices, and their connection information.

[0104] The inverse transform unit 206B2 is configured to inversely transform the coefficient sequence to generate and output a displacement sequence. The inverse transform unit 206B2 may inversely transform the coefficient sequence using an inverse DCT or an inverse wavelet transform.

[0105] The displacement amount adding unit 206B3 is configured to generate and output sub-meshes by adding a sequence of displacement amounts to the finely divided meshes.

[0106] (Atlas Decoding Unit 202) Fig. 27 is a diagram showing an example of the configuration of the atlas data decoding unit 202. As shown in Fig. 271, the atlas data decoding unit 202 includes an atlas sequence parameter set decoding unit 202A, an atlas frame parameter set decoding unit 202B, and an atlas tile decoding unit 202C.

[0107] The atlas sequence parameter set decoding unit 202A is configured to decode the atlas sequence parameter set (ASPS), which is a parameter related to a sequence, and its extension ASVE (Atlas Sequence Parameter Set V-DMC Extension) for 3D mesh compression.

[0108] As shown in FIG. 27, the atlas sequence parameter set decoding unit 202A includes an ASPS decoding unit and an ASVE decoding unit 202A1.

[0109] The atlas frame parameter set decoding unit 207B is configured to decode AFPS (Atlas Sequence Parameter Set), which is a parameter related to a frame, and its extension AFVE (Atlas Frame Parameter Set V-DMC Extension) for 3D mesh compression.

[0110] The atlas tile decoder 207C is configured to decode various information about tiles and patches within tiles.

[0111] (ASVE Decoding Unit 202A1) The ASVE decoding unit 202A1 includes a displacement amount codec determining unit 202A1a, a displacement amount video codec dedicated parameter decoding unit 202A1b, and other ASVE parameters 202A1c.

[0112] The ASVE decoding unit 202A1 is configured to decode ASVE parameters using a displacement codec determining unit 202A1a and other ASVE parameters 202A1c.

[0113] The displacement video codec-specific parameter decoder 202A1b decodes parameters that are required only when the displacement video codec is used.

[0114] For example, the displacement video codec-specific parameter decoder 202A1b may decode the image packing method asve_packing_method.

[0115] The other ASVE parameter decoder 202A1c decodes parameters commonly used when the displacement amount video codec is used or parameters not required when the displacement amount video codec is used.

[0116] For example, another ASVE parameter decoder 202A1c may decode asve_subdivision_method indicating the subdivision method and asve_num_attribute_video indicating the number of videos of the attribute.

[0117] (Processing of ASVE Decoding Unit 202A1) FIG. 24 is a flowchart showing an example of processing of the ASVE decoding unit 202A1.

[0118] In step S101, the displacement codec determination unit 202A1a of the ASVE decoding unit 202A1 determines whether the displacement is decoded using a video codec. If it is decoded using a video codec (for example, if ptl_profile_codec_group_idc is an integer ranging from 0 to 4 or ptl_profile_codec_group_idc is 127), the operation proceeds to step S102; otherwise, the operation proceeds to step S103.

[0119] In step S102, the displacement video codec-specific parameter decoding unit 202A1b of the ASVE decoding unit 202A1 decodes asve_packing_method, which is the image packing method.

[0120] In step S103, another ASVE parameter decoding unit 202A1c of the ASVE decoding unit 202A1 decodes the syntax in the other ASVE, and this operation ends.

[0121] The image packing method is described as ASVE syntax, but it is the same even if it is moved to another parameter set. For example, even if the image packing method is moved to the displacement amount bit stream, the same processing is performed by replacing the atlas with the displacement amount. (Processing of the reference list construction unit 204B2) Figure 25 is a flowchart showing an example of the processing of the reference list construction unit 204B2.

[0122] As shown in FIG. 25, in step S101, the reference list creating unit 204B2 determines whether drl_st_ref_displ_frame_flag is 1 or 0.

[0123] If drl_st_ref_displ_frame_flag=1, the operation proceeds to step S102, and if drl_st_ref_displ_frame_flag=0, the operation proceeds to step S103.

[0124] In step S102, the reference list constructor 204B2 calculates a reference frame using drl_abs_delta_dfoc_st.

[0125] In step S103, the reference list creation unit 204B2 calculates a reference frame using drl_dfoc_lsb_lt.

[0126] In step S104, the reference list creation section 204B2 registers the reference frame.

[0127] In step S105, reference list creation section 204B2 determines whether the currently processed frame is the last frame. If Yes, this operation ends, and if No, this operation proceeds to step S106.

[0128] In step S106, reference list creation section 204B2 proceeds to processing the next frame.

[0129] (Processing of Reference Frame Marking Unit 204B3) FIG. 26 is a flowchart showing an example of processing of the reference frame marking unit 204B3.

[0130] 26, in step S101, the reference frame marking unit 204B3 determines whether or not the frame is a long-term reference frame. If it is a long-term reference frame, the operation proceeds to step S102; if not, the operation proceeds to step S103.

[0131] In step S102, the reference frame marking unit 204B3 marks the frame as a "long-term reference frame."

[0132] In step S103, the reference frame marking unit 204B3 determines whether it is a short-term reference frame. If it is, the operation proceeds to step S104, otherwise the operation proceeds to step S105.

[0133] In step S104, the reference frame marking unit 204B3 marks the frame as a "short-term reference frame."

[0134] In step S105, the reference frame marking unit 204B3 marks the frame as a "non-reference frame."

[0135] In step S106, the reference frame marking unit 204B3 determines whether the currently processed frame is the last frame. If Yes, the operation ends, and if No, the operation proceeds to step S107.

[0136] In step S107, the reference frame marking unit 204B3 proceeds to process the next frame.

[0137] According to this embodiment, the reference list is constructed by the processing of the displacement amount header decoding unit 204B, thereby enabling inter prediction.

[0138] Furthermore, according to this embodiment, the processing of the syntax matching unit 206A can resolve inconsistencies between the atlas bitstream and the displacement amount bitstream.

[0139] The mesh encoding device 100 and mesh decoding device 200 described above may be realized as a program that causes a computer to execute each function (each step).

[0140] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation."

[0141] 1...Mesh processing system 100...Mesh encoding device 200...Mesh decoding device 201...V3C unit separation unit 202...Atlas decoding unit 202A...Atlas sequence parameter set decoding unit 202A1...ASVE decoding unit 202 202A1a...Displacement codec determination unit 202A1b...Displacement video codec dedicated parameter decoding unit 202A1c...Other ASVE parameters 202B...Atlas frame parameter set decoding unit 202C...Atlas style decoding unit 203...Basic mesh decoding unit 204...Displacement decoding unit 204A...Payload decoding unit 204B...Displacement header decoding unit 204B1...Frame display order derivation unit 204B2...Reference list construction unit 204B3...Reference frame marking unit 204B4...Reference list storage unit 204C...Displacement data decoding unit 204C1...arithmetic decoding unit and 204C2...frame buffer 204C3...inter prediction unit 204C4...adder 204C5...inverse quantization unit 205...texture decoding unit 206...mesh reconstruction unit 206A...syntax matching unit 206B...submesh reconstruction unit 206B1...subdivision unit 206B2...inverse transformation unit 206B3...displacement amount addition unit 206C...submesh combination unit

Claims

1. A mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a base mesh, a coefficient sequence, and a displacement vector number obtained from a bitstream, wherein the mesh reconstruction unit comprises: a syntax matching unit configured to generate and output a matching base mesh, a matching coefficient sequence, and a matching displacement vector number using the atlas, the base mesh, the coefficient sequence, and the displacement vector number; a submesh reconstruction unit configured to generate and output a submesh sequence using the matching base mesh, the matching coefficient sequence, and the matching displacement vector number; and a submesh combining unit configured to generate and output a single mesh by combining the submesh sequences.

2. The mesh decoding device according to claim 1, characterized in that the syntax matching unit outputs the smaller of the number of displacement vectors defined in the atlas and the number of displacement vectors as the number of matched displacement vectors.

3. A mesh decoding method comprising step A of reconstructing and outputting a mesh based on an atlas, a base mesh, a coefficient sequence, and a displacement vector number obtained from a bitstream, wherein step A comprises: a step of generating and outputting a matching base mesh, a matching coefficient sequence, and a matching displacement vector number using the atlas, the base mesh, the coefficient sequence, and the displacement vector number; a step of generating and outputting a sub-mesh sequence using the matching base mesh, the matching coefficient sequence, and the matching displacement vector number; and a step of generating and outputting a single mesh by combining the sub-mesh sequences.

4. A program that causes a computer to function as a mesh decoding device, the mesh decoding device comprising: a mesh reconstruction unit configured to reconstruct and output a mesh based on an atlas, a base mesh, a coefficient sequence, and a displacement vector number obtained from a bitstream; the mesh reconstruction unit comprising: a syntax matching unit configured to generate and output a matching base mesh, a matching coefficient sequence, and a matching displacement vector number using the atlas, the base mesh, the coefficient sequence, and the displacement vector number; a submesh reconstruction unit configured to generate and output a submesh sequence using the matching base mesh, the matching coefficient sequence, and the matching displacement vector number; and a submesh combining unit configured to generate and output a single mesh by combining the submesh sequences.

Images