Mesh decoding apparatus, mesh decoding method, and program

Abstract

To enable inter-prediction by constructing a reference list.SOLUTION: In a mesh decoding apparatus 200 according to the present invention, a displacement decoding unit 204 comprises: a payload decoding unit 204A configured to decode and output a payload from a displacement bitstream; a displacement header decoding unit 204B configured to decode and output a reference list from the payload; and a displacement data decoding unit 204C configured to decode and output a coefficient sequence, a displacement vector number, and a subdivision count from the payload and the reference list.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program. [Background technology]

[0002] Non-Patent Document 1 discloses a technique for decoding a point group using video information and additional information called an atlas.

[0003] Non-Patent Document 2 discloses a technique that extends Non-Patent Document 1 to mesh decoding applications.

[0004] Specifically, Non-Patent Document 2 first decodes a rough base mesh and a detailed displacement, and then decodes a mesh from both of them. Here, a syntax indicating a reference list structure is defined for inter-prediction of the displacement. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] “Information technology - Coded Representation of Immersive Media - Part 5: Visual Volumetric Video-based Coding (V3C) and Video-based Point Cloud Compression (V-PCC),” ISO / IEC JTC 1 / SC 29 / WG 7, ISO / IEC 23090-5:2021(2E). [Non-patent document 2] “WD 6.0 of V-DMC,” ISO / IEC JTC 1 / SC 29 / WG 7 N00822. Summary of the Invention [Problem to be solved by the invention]

[0006] However, the method of Non-Patent Document 2 has a problem in that it does not allow inter prediction because it does not include a process for constructing an actual reference list structure from the syntax indicating the reference list structure of the displacement amount.

[0007] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a mesh decoding device, a mesh decoding method, and a program that enable inter-prediction by constructing a reference list. [Means for solving the problem]

[0008] a displacement amount decoding unit configured to decode and output a coefficient sequence, a displacement amount vector count, and a subdivision count from the displacement amount bitstream, wherein the displacement amount decoding unit comprises: a payload decoding unit configured to decode and output a payload from the displacement amount bitstream; a displacement amount header decoding unit configured to decode and output a reference list from the payload; and a displacement amount data decoding unit configured to decode and output the coefficient sequence, the displacement amount vector count, and the subdivision count from the payload and the reference list, wherein the displacement amount header decoding unit comprises: a frame display order derivation unit configured to derive and output a frame display order from the payload; a reference list construction unit configured to construct and output a reference list from the frame display order, the reference list being configured for each target frame to indicate whether each frame other than the target frame is a reference frame; and a reference frame marking unit configured to mark the status of the reference frame in the reference list using the frame display order.

[0009] A second feature of the present invention is summarized as a mesh decoding method comprising: step A of separating a bitstream into an atlas bitstream, a base mesh bitstream, and a displacement bitstream; and step B of decoding and outputting a coefficient sequence, a displacement vector count, and a subdivision count from the displacement bitstream, wherein step B comprises step B1 of decoding and outputting a payload from the displacement bitstream; step B2 of decoding and outputting a reference list from the payload; and step B3 of decoding and outputting the coefficient sequence, the displacement vector count, and the subdivision count from the payload and the reference list, wherein step B2 comprises a step of deriving and outputting a frame display order from the payload; a step of constructing and outputting a reference list from the frame display order, the reference list being configured for each target frame to indicate whether each frame other than the target frame is a reference frame; and a step of marking the status of the reference frame in the reference list using the frame display order.

[0010] A third aspect of the present invention is a program for causing a computer to function as a mesh decoding device, the mesh decoding device comprising: a V3C unit separating unit configured to separate a bitstream into an atlas bitstream, a basic mesh bitstream, and a displacement bitstream; and a displacement decoding unit configured to decode and output a coefficient sequence, a displacement vector number, and a subdivision count from the displacement bitstream, the displacement decoding unit comprising: a payload decoding unit configured to decode and output a payload from the displacement bitstream; a displacement header decoding unit configured to decode and output a reference list from the payload; and a displacement amount data decoding unit configured to decode and output a coefficient sequence, a displacement amount vector number, and a subdivision count from the reference list, wherein the displacement amount header decoding unit comprises: a frame display order derivation unit configured to derive and output a frame display order from the payload; a reference list configured to construct and output a reference list from the frame display order and a reference list composed of a list indicating, for each target frame, whether each frame other than the target frame is a reference frame; and a reference frame marking unit configured to mark the state of a reference frame in the reference list using the frame display order. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide a mesh decoding device, a mesh decoding method and a program that enable inter prediction by constructing a reference list. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks of a mesh decoding device 200 according to an embodiment. [Figure 3]FIG. 3 is a diagram illustrating an example of the configuration of a V3C unit. [Figure 4] FIG. 4 is a diagram showing an example of the syntax configuration of a V3C unit header. [Figure 5] FIG. 5 is a diagram showing an example of the structure of a bitstream. [Figure 6] FIG. 6 is a diagram illustrating an example of the syntax configuration of profile_tier_level. [Figure 7] FIG. 7 is a diagram illustrating an example of the syntax configuration of asps_vdmc_extension. [Figure 8] FIG. 8 is a diagram showing an example of the configuration of a displacement amount bit stream. [Figure 9] FIG. 9 is a diagram illustrating an example of the syntax configuration of displ_nal_unit_header. [Figure 10] FIG. 10 is a diagram showing an example of the correspondence between displ_nal_unit_type and the contents of the payload. [Figure 11] FIG. 11 is a diagram illustrating an example of the syntax configuration of displ_sequence_parameter_set_rbsp. [Figure 12] FIG. 12 is a diagram illustrating an example of the syntax configuration of displ_layer_rbsp. [Figure 13] FIG. 13 is a diagram illustrating an example of the syntax configuration of displ_header. [Figure 14] FIG. 14 is a diagram illustrating an example of the syntax configuration of displ_ref_list_struct. [Figure 15] FIG. 15 is a diagram illustrating an example of the syntax configuration of displ_data_unit. [Figure 16] FIG. 16 is a diagram illustrating an example of functional blocks of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 17]FIG. 17 is a diagram showing an example of functional blocks of a displacement amount header decoder 204B of the displacement amount decoder 204 of the mesh decoding device 200 according to an embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a prediction structure and a reference list constructed therefrom. [Figure 19] FIG. 19 is a diagram showing an example of marking of a reference list. [Figure 20] FIG. 20 is a diagram showing an example of functional blocks of a displacement amount data decoding unit 204C of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 21] FIG. 21 is a diagram showing a modified example of the functional blocks of the displacement amount data decoding unit 204C of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 22] FIG. 22 is a diagram showing an example of functional blocks of the mesh reconstruction unit 206 of the mesh decoding device 200 according to an embodiment. [Figure 23] FIG. 23 is a diagram showing an example of functional blocks of the sub-mesh reconstructing unit 206B of the mesh reconstructing unit 206 of the mesh decoding device 200 according to an embodiment. [Figure 24] FIG. 24 is a flowchart showing an example of processing by the ASVE decoding unit 202A1 of the atlas sequence parameter set decoding unit 202A of the atlas decoding unit 202 of the mesh decoding device 200 according to an embodiment. [Figure 25] FIG. 25 is a flowchart showing an example of processing by the reference list constructor 204B2 of the displacement amount header decoder 204B of the displacement amount decoder 204 of the mesh decoding device 200 according to an embodiment. [Figure 26] FIG. 26 is a flowchart showing an example of processing by the reference frame marking unit 204B3 of the displacement amount header decoding unit 204B of the displacement amount decoding unit 204 of the mesh decoding device 200 according to an embodiment. [Figure 27] FIG. 27 is a diagram showing an example of the configuration of the atlas data decoding unit 202. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0014] First Embodiment This embodiment is based on V3C described in Non-Patent Document 1. An atlas in V3C is a collection of rectangular regions and related information arranged on a frame, and corresponds to a three-dimensional space.

[0015] Hereinafter, a mesh processing system 1 according to this embodiment will be described with reference to FIGS.

[0016] 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to this embodiment. As shown in FIG. 1, the mesh processing system 1 includes a mesh encoding device 100 and a mesh decoding device 200.

[0017] FIG. 2 is a diagram showing an example of functional blocks of the mesh decoding device 200 according to this embodiment.

[0018] As shown in FIG. 2, the mesh decoding device 200 includes a V3C unit separation unit 201, an atlas decoding unit 202, a basic mesh decoding unit 203, a displacement amount decoding unit 204, a texture decoding unit 205, and a mesh reconstruction unit 206.

[0019] The V3C unit separation unit 201 is configured to separate the bitstream into an atlas bitstream, a base mesh bitstream, a displacement bitstream, and a texture bitstream.

[0020] The atlas decoding unit 202 is configured to decode and output the atlas from the atlas bitstream.

[0021] The base mesh decoding unit 203 is configured to decode and output the base mesh from the base mesh bitstream.

[0022] The displacement decoding unit 204 is configured to decode and output the coefficient sequence, the number of displacement vectors, and the number of subdivisions from the displacement bitstream.

[0023] The texture decoding unit 205 is configured to decode and output texture from the texture bitstream. The texture decoding unit 205 may decode the texture using HEVC.

[0024] The mesh reconstruction unit 206 is configured to reconstruct and output a mesh using the atlas decoded by the atlas decoding unit 201, the basic mesh decoded by the basic mesh decoding unit 203, and the coefficient sequence, number of displacement vectors, and number of subdivisions decoded by the displacement decoding unit 204.

[0025] (Bitstream configuration) A bitstream may be composed of a combination of multiple V3C units.

[0026] 3 is a diagram showing an example of the configuration of a V3C unit. A V3C unit may be composed of a V3C unit header and a V3C unit payload.

[0027] 4 is a diagram showing an example of the syntax configuration of a V3C unit header. The V3C unit header may include vuh_unit_type, which indicates the type of the V3C unit.

[0028] 5 is a diagram showing an example of the configuration of a bitstream. The bitstream may be composed of a V3C parameter set, an atlas bitstream, a base mesh bitstream, a displacement bitstream, and a texture bitstream.

[0029] vuh_unit_type may store any of the following: V3C_VPS indicating a V3C parameter set, V3C_AD indicating an atlas bitstream, V3C_BMD indicating a basic mesh bitstream, V3C_GVD indicating a displacement bitstream, or V3C_AVD indicating a texture bitstream.

[0030] A V3C parameter set may contain a profile_tier_level that indicates the profile tier level.

[0031] FIG. 6 is a diagram illustrating an example of the syntax configuration of profile_tier_level.

[0032] The profile_tier_level may include a ptl_profile_codec_group_idc that indicates the codec ID of the displacement amount. (Atlas bitstream configuration) The atlas bitstream may use afps_vdmc_extension, which indicates frame parameters that are an extension of the V3C frame parameter set.

[0033] 7 is a diagram showing an example of the syntax configuration of asps_vdmc_extension. asps_vdmc_extension may include asve_displacement_num, which indicates the number of vectors of the displacement amount. asps_vdmc_extension may also include asve_subdivision_iteration_count, which indicates the number of subdivisions.

[0034] The asps_vdmc_extension may include asve_packing_method, which indicates an image packing method for the displacement. The asve_packing_method is decoded only if the codec of the displacement is a video codec, and may not be decoded otherwise.

[0035] The determination that the codec of the displacement amount is a video codec may be made when ptl_profile_codec_group_idc is an integer in the range of 0 to 4, or when ptl_profile_codec_group_idc is 127.

[0036] (Structure of displacement bitstream) 8 is a diagram showing an example of the configuration of a displacement amount bitstream. The displacement amount bitstream may have a V3C unit payload configured with displ_nal_unit indicating an NAL unit.

[0037] The displ_nal_unit may be composed of a displ_nal_unit_header indicating the header of the NAL unit in the displacement amount bitstream and the payload of the NAL unit in the displacement amount bitstream.

[0038] FIG. 9 is a diagram illustrating an example of the syntax configuration of displ_nal_unit_header.

[0039] The displ_nal_unit_header may be composed of a displ_nal_forbidden_zero_bit indicating the leading write-protected section and a displ_nal_unit_type indicating the type of data in the payload.

[0040] FIG. 10 is a diagram showing an example of the correspondence between displ_nal_unit_type and the contents of the payload.

[0041] The contents of the payload may be defined as displ_sequence_parameter_set_rbsp indicating the sequence parameter set of the displacement amount when displ_nal_unit_type is 0, displ_frame_parameter_set_rbsp indicating the frame parameter set of the displacement amount when displ_nal_unit_type is 1, and displ_layer_rbsp indicating the displacement amount data when displ_nal_unit_type is 2.

[0042] FIG. 11 is a diagram illustrating an example of the syntax configuration of displ_sequence_parameter_set_rbsp.

[0043] The displ_sequence_parameter_set_rbsp may include dsps_displacement_num, which indicates the number of vectors of the displacement amount.

[0044] The displ_sequence_parameter_set_rbsp may include dsps_log2_max_displ_frame_order_cnt_lsb, which indicates the maximum value of the frame display order of the displacement amount.

[0045] The displ_sequence_parameter_set_rbsp may include dsps_max_dec_displ_frame_buffering, which indicates the maximum value of the displacement buffer size.

[0046] The displ_sequence_parameter_set_rbsp may include dsps_num_ref_displ_frame_lists_in_dsps, which indicates the number of reference lists for displacement amounts.

[0047] The displ_sequence_parameter_set_rbsp may include a displ_ref_list_struct that indicates a reference list of displacement amounts.

[0048] 12 is a diagram showing an example of the syntax configuration of displ_layer_rbsp. displ_layer_rbsp may include displ_header indicating a displacement amount header and displ_data_unit indicating displacement amount data.

[0049] FIG. 13 is a diagram illustrating an example of the syntax configuration of displ_header.

[0050] The displ_header may include dh_id indicating an ID of the displacement amount, and dh_frm_order_cnt_lsb indicating the display order of the displacement amount frames.

[0051] The displ_header may contain dh_ref_displ_frame_list_dsps_flag which indicates whether the reference list is derived from the displ_sequence_parameter_set_rbsp or from the displ_header.

[0052] The displ_header may contain the displ_ref_list_struct described above.

[0053] The displ_header may contain ref_displ_frame_list_idx, which indicates the index of the reference list in the displ_sequence_parameter_set_rbsp.

[0054] FIG. 14 is a diagram illustrating an example of the syntax configuration of displ_ref_list_struct.

[0055] The displ_ref_list_struct may contain drl_num_ref_entries, which indicates the number of frames in the displ_ref_list_struct.

[0056] The displ_ref_list_struct may contain a drl_st_ref_displ_frame_flag that indicates whether a frame is a short-term or long-term reference frame.

[0057] The displ_ref_list_struct may contain a drl_abs_delta_dfoc_st that indicates the absolute value of the difference between the display order of a frame and the display order of a short-term reference frame.

[0058] The displ_ref_list_struct may include a drl_dfoc_lsb_lt that indicates the difference between the display order of a frame and the maximum value of the display order of frames of the displacement amount.

[0059] 15 is a diagram showing an example of the syntax configuration of displ_data_unit. displ_data_unit may include ddu_lod_count, which indicates the number of subdivisions.

[0060] (Displacement amount decoding unit 204) Fig. 16 is a diagram showing an example of the configuration of the displacement amount decoding unit 204. As shown in Fig. 16, the displacement amount decoding unit 204 includes a payload decoding unit 204A, a displacement amount header decoding unit 204B, and a displacement amount data decoding unit 204C.

[0061] The payload decoder 204A is configured to decode and output the payload from the displacement bitstream.

[0062] The displacement amount header decoder 204B is configured to decode and output the reference list from the payload.

[0063] The displacement data decoding unit 204C is configured to decode and output the coefficient sequence, the number of displacement vectors, and the number of subdivisions from the payload and the reference list.

[0064] Fig. 17 is a diagram showing an example of the configuration of the displacement amount header decoding unit 204B. As shown in Fig. 17, the displacement amount header decoding unit 204B includes a frame display order derivation unit 204B1, a reference list creation unit 204B2, a reference frame marking unit 204B3, and a reference list storage unit 204B4.

[0065] The frame display order derivation unit 204B1 is configured to derive and output a frame display order from the payload.

[0066] The reference list storage unit 204B4 is configured to store the reference list constructed by the reference list construction unit 204B2. The reference list is configured from a list indicating, for each target frame, whether each frame other than the target frame is a reference frame.

[0067] The reference list construction unit 204B2 is configured to construct and output a reference list based on the frame display order.

[0068] The reference list construction unit 204B2 is configured to construct a reference list using the frame display order and displ_ref_list_struct.

[0069] The reference list constructing section 204B2 may construct the reference list using drl_abs_delta_dfoc_st, or may construct the reference list using drl_dfoc_lsb_lt.

[0070] The reference list constructor 204B2 may construct a reference list using drl_abs_delta_dfoc_st when the reference frame is a short-term reference frame, and may construct a reference list using drl_dfoc_lsb_lt when the reference frame is a long-term reference frame.

[0071] 18 is a diagram showing an example of a prediction structure and a reference list constructed therefrom. The example in Fig. 18 shows a reference list when frame 6 is the target frame.

[0072] The reference frame marking unit 204B3 is configured to mark the status of the reference frames in the reference list using the frame display order.

[0073] The reference frame marking unit 204B3 may mark each reference frame in the reference list as either a "long-term reference frame," a "short-term reference frame," or a "non-reference frame" using the frame display order.

[0074] Reference frames marked as "long-term reference frames" are not removed from the reference list. Reference frames marked as "non-reference frames" may be removed from the reference list.

[0075] Fig. 19 is a diagram showing an example of marking of a reference list. The example in Fig. 19 shows a reference list when frame 6 is the target frame and a reference list when frame 7 is the target frame.

[0076] When frame 6 is the target frame, reference frame marking section 204B3 marks frames 1 and 4 as "long-term reference frames" and marks frame 5 as "short-term reference frame."

[0077] When frame 7 is the target frame, the reference frame marking unit 204B3 marks frames 1 and 4 as "long-term reference frames", marks frame 5 as "non-reference frame", and marks frame 6 as "short-term reference frame".

[0078] The above-described processing by the displacement amount header decoding unit 204B constructs a reference list, enabling inter prediction, which will be described later.

[0079] Fig. 20 is a diagram showing an example of the configuration of the displacement amount data decoding unit 204C. As shown in Fig. 20, the displacement amount data decoding unit 204C includes an arithmetic decoding unit 204C1, a frame buffer 204C2, an inter prediction unit 204C3, an adder 204C4, and an inverse quantization unit 204C5.

[0080] The arithmetic decoding unit 204C1 is configured to decode the level value sequence, the number of displacement vectors, and the number of subdivisions from the displacement bit stream by arithmetic decoding, and output the decoded level value sequence, the number of displacement vectors, and the number of subdivisions.

[0081] The arithmetic decoding unit 204C1 may decode the level value sequence, the number of displacement vectors, and the number of subdivisions as in Japanese Patent Application No. 2022-212452.

[0082] The frame buffer 204C2 is configured to acquire and store a level value sequence from the arithmetic decoding unit 204C1 or the adder 204C4. The frame buffer 204C2 is configured to output a level value sequence in a reference frame in accordance with control information (not shown).

[0083] The inter prediction unit 204C3 is configured to read out the reference frame indicated by the reference list from the frame buffer, and perform inter prediction using the level value sequence, thereby generating and outputting a prediction level value sequence.

[0084] The inter prediction unit 204C3 may determine the predicted level value of each frequency in the current frame by directly referring to the level value of the corresponding frequency in the reference frame.

[0085] The adder 204C4 is configured to obtain a prediction level value sequence from the inter prediction unit 204C3 and a level value sequence (actually, a prediction residual sequence) from the arithmetic decoding unit 204C1, and add them together to generate a level value sequence. The generated level value sequence is output to the inverse quantization unit 204C5 and the frame buffer 204C2.

[0086] The inverse quantization unit 204C5 is configured to inversely quantize the level value sequence to generate and output a coefficient sequence.

[0087] Fig. 21 is a diagram showing a modified example of the configuration of the displacement amount data decoding unit 204C. Hereinafter, with reference to Fig. 21, this modified example will be described focusing on the differences from the displacement amount data decoding unit 204C shown in Fig. 20.

[0088] The inter prediction unit 204C3 is configured to read out the reference frame indicated by the reference list from the frame buffer, and perform inter prediction using the coefficient sequence, thereby generating and outputting a prediction coefficient sequence.

[0089] The inter prediction unit 204C3 may determine prediction coefficients for each frequency in the current frame by directly referencing the coefficients for the corresponding frequency in the reference frame.

[0090] The adder 204C4 is configured to obtain a prediction coefficient sequence from the inter prediction unit 204C3 and a coefficient sequence (actually, a prediction residual sequence) from the inverse quantization unit 204C5, and add them together to generate a coefficient sequence. The generated coefficient sequence is output to the frame buffer 204C2.

[0091] (Mesh reconstruction unit 206) Fig. 22 is a diagram showing an example of the configuration of the mesh reconstructing unit 206. As shown in Fig. 22, the mesh reconstructing unit 206 includes a syntax matching unit 206A, a submesh reconstructing unit 206B, and a submesh combining unit 206C.

[0092] The syntax matching unit 206A is configured to generate and output a matching base mesh, a matching coefficient sequence, a matching displacement vector number, and a matching subdivision number using the atlas, the base mesh, the coefficient sequence, the displacement vector number, and the subdivision number.

[0093] The syntax matching unit 206A may output the smaller of asve_displacement_num and the number of displacement vectors as the number of matched displacement vectors.

[0094] The syntax matching unit 206A may hold a compatibility requirement such that asve_displacement_num and the number of displacement vectors match.

[0095] The syntax matching unit 206A may output the smaller of asve_subdivision_iteration_count and the number of subdivisions as the matching number of subdivisions.

[0096] The syntax matching unit 206A may hold a compatibility requirement such that asve_subdivision_iteration_count and the number of subdivisions match.

[0097] The above-described processing by the syntax matching unit 206A can resolve inconsistencies between the atlas bitstream and the displacement amount bitstream.

[0098] The submesh reconstructing unit 206B is configured to generate and output a submesh sequence using the matching basic mesh, the matching coefficient sequence, the number of matching displacement vectors, and the number of matching subdivisions.

[0099] The submesh combining unit 206B is configured to combine the submesh rows to generate and output a single mesh.

[0100] Fig. 23 is a diagram showing an example of the configuration of the submesh reconstructing unit 206B. As shown in Fig. 23, the submesh reconstructing unit 206B includes a subdivision unit 206B1, an inverse conversion unit 206B2, and a displacement amount adding unit 206B3.

[0101] The subdivision unit 206B1 is configured to generate the added subdivision vertices and their connection information from the matching base mesh according to the subdivision method indicated by the control information.

[0102] The subdivision unit 206B1 outputs a subdivision mesh that combines the basic mesh, the added subdivision vertices, and their connection information.

[0103] The inverse transform unit 206B2 is configured to inversely transform the coefficient sequence to generate and output a displacement sequence. The inverse transform unit 206B2 may inversely transform the coefficient sequence using an inverse DCT or an inverse wavelet transform.

[0104] The displacement amount adding unit 206B3 is configured to generate and output sub-meshes by adding the displacement amount sequence to the finely divided mesh.

[0105] (Atlas decoding unit 202) Fig. 27 is a diagram showing an example of the configuration of the atlas data decoding unit 202. As shown in Fig. 271, the atlas data decoding unit 202 includes an atlas sequence parameter set decoding unit 202A, an atlas frame parameter set decoding unit 202B, and an atlas tile decoding unit 202C.

[0106] The atlas sequence parameter set decoding unit 202A is configured to decode the atlas sequence parameter set (ASPS), which is a parameter related to a sequence, and its extension, the atlas sequence parameter set V-DMC extension (ASVE), for 3D mesh compression.

[0107] As shown in FIG. 27, the atlas sequence parameter set decoding unit 202A includes an ASPS decoding unit and an ASVE decoding unit 202A1.

[0108] The atlas frame parameter set decoding unit 207B is configured to decode the Atlas Sequence Parameter Set (AFPS), which is a parameter related to a frame, and its extension AFVE (Atlas Frame Parameter Set V-DMC Extension) for 3D mesh compression.

[0109] The atlas tile decoder 207C is configured to decode various information about tiles and about patches within tiles.

[0110] (ASVE decoding unit 202A1) The ASVE decoding unit 202A1 includes a displacement codec determining unit 202A1a, a displacement video codec dedicated parameter decoding unit 202A1b, and other ASVE parameters 202A1c.

[0111] The ASVE decoding unit 202A1 is configured to decode the ASVE parameters using a displacement codec determining unit 202A1a and another ASVE parameter 202A1c.

[0112] The displacement video codec dedicated parameter decoder 202A1b decodes parameters that are required only when the displacement video codec is used.

[0113] For example, the displacement video codec-specific parameter decoder 202A1b may decode the image packing method asve_packing_method.

[0114] Another ASVE parameter decoder 202A1c decodes parameters commonly used when the displacement amount video codec is used or parameters not required when the displacement amount video codec is used.

[0115] For example, the other ASVE parameter decoder 202A1c may decode asve_subdivision_method, which indicates the subdivision method, and asve_num_attribute_video, which indicates the number of videos of the attribute.

[0116] (Processing of ASVE decoding unit 202A1) FIG. 24 is a flowchart showing an example of the processing of the ASVE decoding unit 202A1.

[0117] In step S101, the displacement codec determination unit 202A1a of the ASVE decoding unit 202A1 determines whether the displacement is decoded using a video codec. If it is decoded using a video codec (for example, if ptl_profile_codec_group_idc is an integer ranging from 0 to 4 or ptl_profile_codec_group_idc is 127), the operation proceeds to step S102; otherwise, the operation proceeds to step S103.

[0118] In step S102, the displacement video codec-specific parameter decoding unit 202A1b of the ASVE decoding unit 202A1 decodes the image packing method asve_packing_method.

[0119] In step S103, the other ASVE parameter decoding unit 202A1c of the ASVE decoding unit 202A1 decodes the syntax in the other ASVE, and this operation ends.

[0120] The image packing method is described as the syntax of ASVE, but it is the same even if it is moved to another parameter set. For example, even if the image packing method is moved to the displacement amount bit stream, the same processing is performed by replacing the atlas with the displacement amount. (Processing of the reference list construction unit 204B2) FIG. 25 is a flowchart showing an example of the processing of the reference list creating section 204B2.

[0121] As shown in FIG. 25, in step S101, the reference list constructor 204B2 determines whether drl_st_ref_displ_frame_flag is 1 or 0.

[0122] If drl_st_ref_displ_frame_flag=1, the operation proceeds to step S102, and if drl_st_ref_displ_frame_flag=0, the operation proceeds to step S103.

[0123] In step S102, the reference list constructor 204B2 calculates a reference frame using drl_abs_delta_dfoc_st.

[0124] In step S103, the reference list constructor 204B2 calculates a reference frame using drl_dfoc_lsb_lt.

[0125] In step S104, the reference list creator 204B2 registers the reference frame.

[0126] In step S105, reference list creator 204B2 determines whether the currently processed frame is the last frame. If Yes, the operation ends, and if No, the operation proceeds to step S106.

[0127] In step S106, reference list creator 204B2 proceeds to processing the next frame.

[0128] (Processing of reference frame marking section 204B3) FIG. 26 is a flowchart showing an example of the processing of the reference frame marking unit 204B3.

[0129] 26, in step S101, the reference frame marking unit 204B3 determines whether or not it is a long-term reference frame. If it is a long-term reference frame, the operation proceeds to step S102, and if not, the operation proceeds to step S103.

[0130] In step S102, the reference frame marking unit 204B3 marks it as a "long-term reference frame."

[0131] In step S103, the reference frame marking unit 204B3 determines whether it is a short-term reference frame. If it is a short-term reference frame, the operation proceeds to step S104, otherwise the operation proceeds to step S105.

[0132] In step S104, the reference frame marking unit 204B3 marks it as a "short-term reference frame."

[0133] In step S105, the reference frame marking unit 204B3 marks the frame as a "non-reference frame."

[0134] In step S106, the reference frame marking unit 204B3 determines whether the currently processed frame is the last frame. If yes, the operation ends, and if no, the operation proceeds to step S107.

[0135] In step S107, the reference frame marking unit 204B3 proceeds to process the next frame.

[0136] According to this embodiment, the reference list is constructed by the processing of the displacement amount header decoding unit 204B, thereby enabling inter prediction.

[0137] Furthermore, according to this embodiment, the processing of the syntax matching unit 206A can resolve inconsistencies between the atlas bitstream and the displacement amount bitstream.

[0138] The mesh encoding device 100 and the mesh decoding device 200 described above may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]

[0139] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0140] 1. Mesh processing system 100...Mesh coding device 200...Mesh decoding device 201...V3C unit separation section 202...Atlas decoding unit 202A... Atlas sequence parameter set decoding unit 202A1...ASVE decoding unit 202 202A1a...Displacement amount codec determination unit 202A1b...Displacement amount video codec dedicated parameter decoding unit 202A1c...Other ASVE parameters 202B... Atlas frame parameter set decoding unit 202C...Atlas Style Decoding Unit 203...Basic mesh decoding unit 204...Displacement amount decoding unit 204A...Payload decoding unit 204B...Displacement amount header decoding unit 204B1...Frame display order derivation unit 204B2...Reference list construction section 204B3...Reference frame marking section 204B4...Reference list storage section 204C...Displacement amount data decoding unit 204C1...Arithmetic decoding unit and 204C2...frame buffer 204C3...Inter prediction section 204C4...adder 204C5...Inverse quantization section 205...Texture decoding unit 206...Mesh reconstruction unit 206A...Syntax matching section 206B...Submesh reconstruction section 206B1…Subdivision section 206B2...inverse conversion unit 206B3...Displacement amount adding unit 206C...Submesh joint

Claims

1. A mesh decoding device, comprising: a V3C unit separator configured to separate the bitstream into an atlas bitstream, a base mesh bitstream, and a displacement bitstream; a displacement decoding unit configured to decode and output a coefficient sequence, a displacement vector number, and a subdivision count from the displacement bitstream; The displacement amount decoding unit a payload decoder configured to decode and output a payload from the displacement bitstream; a displacement header decoder configured to decode and output a reference list from the payload; a displacement data decoding unit configured to decode and output a coefficient sequence, a displacement vector number, and a subdivision count from the payload and the reference list; The displacement amount header decoding unit a frame display order derivation unit configured to derive and output a frame display order from the payload; a reference list construction unit configured to construct and output a reference list, which is configured to indicate, for each target frame, whether each frame other than the target frame is a reference frame, and the reference list from the frame display order; a reference frame marking unit configured to mark the status of a reference frame in the reference list using the frame display order.

2. The reference list construction unit If the reference frame is a short-term reference frame, constructing the reference list using the absolute value of the difference between the display order of a frame and the display order of the short-term reference frame; The mesh decoding device according to claim 1, characterized in that, when the reference frame is a long-term reference frame, the reference list is constructed using the difference between the display order of the certain frame and the maximum value of the display order of the frame of the displacement amount.

3. The mesh decoding device described in claim 1, characterized in that the reference frame marking unit marks each reference frame in the reference list as either a ``long-term reference frame,'' a ``short-term reference frame,'' or a ``non-reference frame'' using the frame display order.

4. 1. A mesh decoding method, comprising: A step A of separating the bitstream into an atlas bitstream, a base mesh bitstream, and a displacement bitstream; a step B of decoding and outputting a coefficient sequence, a number of displacement vectors, and a number of subdivisions from the displacement bitstream; The step B is a step B1 of decoding and outputting a payload from the displacement bitstream; a step B2 of decoding and outputting a reference list from the payload; a step B3 of decoding and outputting a coefficient sequence, a displacement vector number, and a subdivision count from the payload and the reference list; The step B2 is deriving and outputting a frame display order from the payload; a step of constructing and outputting a reference list for each target frame, the reference list being a list indicating whether each frame other than the target frame is a reference frame, and the reference list being constructed from the frame display order; and using said frame display order to mark the status of reference frames in said reference list.

5. A program that causes a computer to function as a mesh decoding device, The mesh decoding device comprises: a V3C unit separator configured to separate the bitstream into an atlas bitstream, a base mesh bitstream, and a displacement bitstream; a displacement decoding unit configured to decode and output a coefficient sequence, a displacement vector number, and a subdivision count from the displacement bitstream; The displacement amount decoding unit a payload decoder configured to decode and output a payload from the displacement bitstream; a displacement header decoder configured to decode and output a reference list from the payload; a displacement data decoding unit configured to decode and output a coefficient sequence, a displacement vector number, and a subdivision count from the payload and the reference list; The displacement amount header decoding unit a frame display order derivation unit configured to derive and output a frame display order from the payload; a reference list construction unit configured to construct and output a reference list, which is configured to indicate, for each target frame, whether each frame other than the target frame is a reference frame, and the reference list from the frame display order; a reference frame marking unit configured to mark a status of a reference frame in the reference list using the frame display order.

Images