Mesh decoding device, mesh decoding method and program

Abstract

To improve coding efficiency.SOLUTION: A displacement amount decoding unit 205 of a mesh decoding device 200 according to the present invention includes: a first decoding unit 205B that generates a displacement amount by arithmetic decoding; a second decoding unit 205C that generates a displacement amount by video codec; and a decoding method determination unit 205A that generates a decoding method for the displacement amount on the basis of a syntax included in a displacement amount bit stream and switches between the first decoding unit 205B and the second decoding unit 205C to generate the displacement amount on the basis of the decoding method.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program. [Background technology]

[0002] Non-Patent Document 1 discloses a technique in which a mesh is divided into a rough basic mesh and a detailed displacement amount, and the mesh is decoded, and the displacement amount is converted into a two-dimensional image and then decoded by a video codec. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Khaled Mammou, Jungsun Kim, Alexis Tourapis, Dimitri Podborski, Krasimir Kolarov, “[V-CG] Apple's Dynamic Mesh Coding CfP Response,” ISO / IEC JTC 1 / SC 29 / WG 7 m5928, April 2022. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology disclosed in Non-Patent Document 1 has a problem that the generated 2D video is not suitable for encoding by a video codec, and the encoding efficiency is reduced. Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a mesh decoding device, a mesh decoding method, and a program that can improve the encoding efficiency. [Means for solving the problem]

[0005] A first feature of the present invention is a mesh decoding device comprising a displacement amount decoding unit configured to decode a displacement amount bitstream to generate and output a displacement amount, wherein the displacement amount decoding unit comprises a first decoding unit configured to generate the displacement amount by arithmetic decoding, a second decoding unit configured to generate the displacement amount by a video codec, and a decoding method determination unit configured to generate a decoding method for the displacement amount by a syntax included in the displacement amount bitstream and to switch whether the first decoding unit or the second decoding unit generates the displacement amount based on the decoding method.

[0006] A second feature of the present invention is a mesh decoding method comprising the steps of decoding a displacement amount bitstream to generate and output displacement amounts, the steps comprising: step A of generating the displacement amount by arithmetic decoding; step B of generating the displacement amount by a video codec; and step C of generating a decoding method for the displacement amount by a syntax included in the displacement amount bitstream and switching between step A and step B for generating the displacement amount based on the decoding method.

[0007] A third feature of the present invention is summarized as a program for causing a computer to function as a mesh decoding device, the mesh decoding device comprising a displacement amount decoding unit configured to decode a displacement amount bitstream to generate and output a displacement amount, the displacement amount decoding unit comprising a first decoding unit configured to generate the displacement amount by arithmetic decoding, a second decoding unit configured to generate the displacement amount by a video codec, and a decoding method determination unit configured to generate a decoding method for the displacement amount by a syntax included in the displacement amount bitstream and to switch whether the first decoding unit or the second decoding unit generates the displacement amount based on the decoding method. Effect of the Invention

[0008] According to the present invention, it is possible to provide a mesh decoding device, a mesh decoding method, and a program capable of improving the encoding efficiency. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of functional blocks of a mesh decoding device 200 according to an embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of functional blocks of the displacement amount decoding unit 205 of the mesh decoding device 200 according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a configuration of a displacement amount bit stream. [Diagram 5] FIG. 5 is a diagram illustrating an example of a configuration of a displacement amount bit stream. [Figure 6] FIG. 6 is a diagram illustrating an example of a configuration of a displacement amount bit stream. [Figure 7] FIG. 7 is a diagram illustrating an example of a configuration of a displacement amount bit stream. [Figure 8] FIG. 8 is a diagram for explaining a modified example of the configuration of the displacement amount bit stream. [Figure 9] FIG. 9 is a flowchart showing an example of the operation of the mesh decoding device 200 according to an embodiment. [Figure 10] FIG. 10 is a flowchart showing a modified example of the operation of the mesh decoding device 200 according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0011] First Embodiment The mesh processing system according to this embodiment will be described below with reference to FIGS.

[0012] 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to this embodiment. As shown in FIG.

[0013] FIG. 2 is a diagram showing an example of functional blocks of the mesh decoding device 200 according to this embodiment.

[0014] As shown in FIG. 2, the mesh decoding device 200 includes a demultiplexing unit 201 , a basic mesh decoding unit 202 , a subdivision unit 203 , a mesh decoding unit 204 , a displacement amount decoding unit 205 , and a video decoding unit 206 .

[0015] The demultiplexing unit 201 is configured to separate the multiplexed bit stream into a base mesh bit stream, a displacement amount bit stream, and a texture bit stream.

[0016] The base mesh decoding unit 202 is configured to decode the base mesh bitstream and generate and output a base mesh.

[0017] The subdivision unit 203 is configured to generate and output added subdivision vertices and their connection information from the basic mesh decoded by the basic mesh decoding unit 202 using a subdivision method indicated by the control information.

[0018] Here, the basic mesh, the added subdivision vertices, and their connection information are collectively referred to as a "subdivision mesh."

[0019] The mesh decoding unit 204 is configured to generate and output a decoded mesh using the subdivision mesh generated by the subdivision unit 203 and the displacement amount decoded by the displacement amount decoding unit 205 .

[0020] The displacement amount decoding unit 205 is configured to decode the displacement amount bitstream to generate and output the displacement amount.

[0021] The video decoding unit 206 is configured to decode the texture by video encoding and output the decoded texture. For example, the video decoding unit 206 may use HEVC described in Non-Patent Document 1.

[0022] (Displacement amount decoding unit 205) Fig. 3 is a diagram showing an example of the configuration of the displacement amount decoding unit 205. As shown in Fig. 3, the displacement amount decoding unit 205 includes a decoding method determination unit 205A, a first decoding unit 205B, and a second decoding unit 205C.

[0023] The decoding method determination unit 205A is configured to generate and output a decoding method for the amount of displacement using a syntax to be described later.

[0024] Here, the decoding method determination unit 205A is configured to switch whether the displacement amount is generated by the first decoding unit 205B or the second decoding unit 205C, based on the decoding method.

[0025] The first decoding unit 205B is configured to decode and generate the amount of displacement by arithmetic decoding. The decoding unit using arithmetic decoding may generate the amount of displacement in the same manner as in Japanese Patent Application No. 2022-212452.

[0026] The second decoding unit 205C is configured to generate the amount of displacement by decoding using a video codec. The second decoding unit 205C may generate the amount of displacement using a method similar to that described in Non-Patent Document 1.

[0027] (Bitstream configuration) Hereinafter, an example of the configuration of a displacement amount bit stream will be described with reference to FIGS.

[0028] Here, the displacement bit stream may be formed by combining a plurality of V3C units (see ISO / IEC 23090-5(2E):2023).

[0029] As shown in Fig. 4, the V3C unit may be composed of a V3C unit header and a V3C unit payload. As shown in Fig. 5, the V3C unit header may include vuh_unit_type indicating the type of the V3C unit.

[0030] The displacement amount bitstream may also include a displacement amount sub-bitstream 2 in which compressed data of the displacement amount is stored.

[0031] The displacement amount sub-bitstream 2 may include displacement amount compressed data obtained by arithmetic decoding or displacement amount compressed data obtained by a video codec.

[0032] As shown in FIG. 6, the V3C unit payload in the displacement amount sub-bitstream 2 may be configured with displ_nal_unit indicating a NAL unit.

[0033] The displ_nal_unit may be composed of a displ_nal_unit_header indicating a header, a payload, and trailing zero bits. The displ_nal_unit_header may be composed of a displ_nal_forbidden_zero_bit indicating a write-prohibited section at the beginning, and a displ_nal_unit_type indicating the type of data in the payload. The displ_nal_unit_type may be defined, for example, as 6 bits, with one value indicating compressed data of the displacement amount and another value indicating a sequence parameter set.

[0034] The V3C unit header may include a displacement_codec_id that indicates a compression method of the displacement. If the value of the displacement_codec_id is "0", the displacement data compressed by arithmetic decoding is included, and if the value of the displacement_codec_id is "1", the displacement data compressed by video codec is included.

[0035] This allows compressed data to be stored in a single sub-bitstream even if the displacement amount is decoded differently.

[0036] FIG. 7 is a diagram showing an example of the configuration of a displacement amount bit stream.

[0037] For example, as shown in FIG. 7, the displacement bitstream may be composed of a sequence parameter set, an atlas sub-bitstream, a base mesh sub-bitstream, a displacement sub-bitstream 2, and a texture sub-bitstream.

[0038] vuh_unit_type may store any of V3C_VPS indicating a sequence parameter set, V3C_AD indicating an atlas sub-bitstream, V3C_BMD indicating a basic mesh sub-bitstream, V3C_GVD indicating a displacement sub-bitstream 2, and V3C_AVD indicating a texture sub-bitstream.

[0039] <Modification> The displacement amount bitstream may include either a displacement amount sub-bitstream 1 in which compressed data of the displacement amount is stored or a displacement amount sub-bitstream 2 in which compressed data of the displacement amount is stored.

[0040] The displacement amount sub-bitstream 1 may include data of the displacement amount compressed by arithmetic decoding, and the displacement amount sub-bitstream 2 may include data of the displacement amount compressed by a video codec.

[0041] FIG. 8 is a diagram showing an example of the configuration of a displacement amount bit stream.

[0042] For example, as shown in FIG. 8, the displacement bitstream may be composed of a sequence parameter set, an atlas sub-bitstream, a base mesh sub-bitstream, a displacement sub-bitstream 1, a displacement sub-bitstream 2, and a texture sub-bitstream.

[0043] vuh_unit_type may store any of V3C_VPS indicating a sequence parameter set, V3C_AD indicating an atlas sub-bitstream, V3C_BMD indicating a basic mesh sub-bitstream, V3C_DD indicating a displacement amount sub-bitstream 1, V3C_GVD indicating a displacement amount sub-bitstream 2, and V3C_AVD indicating a texture sub-bitstream.

[0044] When vuh_unit_type is V3C_DD, compressed data of the displacement amount by arithmetic decoding is included, and when vuh_unit_type is V3C_GVD, compressed data of the displacement amount by video codec is included.

[0045] (Mesh Decoder Operation) Hereinafter, an example of the operation of the mesh decoding device 200 according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the operation of the mesh decoding device according to this embodiment.

[0046] 9, in step S101, the displacement decoding method determination unit 205A determines whether vuh_unit_type is V3C_GVD. If Yes, the operation proceeds to step S102, and if No, the operation proceeds to step S103.

[0047] In step S102, the decoding method determination unit 205A determines whether the value of displacement_codec_id is “0” or not. If Yes, the operation proceeds to step S104, and if No, the operation proceeds to step S105.

[0048] In step S103, the mesh decoding device 200 performs processing for each V3C unit.

[0049] In step S104, the first decoding unit 205B performs decoding by arithmetic decoding.

[0050] In step S105, the second decoding unit 205C performs decoding using a video codec.

[0051] In step S106, the demultiplexing unit 201 determines whether the V3C unit currently being processed is the last V3C unit or not If Yes, this operation ends, and if No, this operation proceeds to step S107.

[0052] In step S107, the demultiplexing unit 201 proceeds to processing the next V3C unit.

[0053] <Modification> Hereinafter, a modified example of the operation of the mesh decoding device 200 according to this embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing a modified example of the operation of the mesh decoding device according to this embodiment.

[0054] 10, in step S201, the decoding method determination unit 205A determines whether or not vuh_unit_type is V3C_DD. If Yes, the operation proceeds to step S202, and if No, the operation proceeds to step S203.

[0055] In step S202, the first decoding unit 205B performs decoding by arithmetic decoding.

[0056] In step S203, the decoding method determination unit 205A determines whether or not vuh_unit_type is V3C_GVD. If Yes, the operation proceeds to step S204, and if No, the operation proceeds to step S205.

[0057] In step S204, the second decoding unit 205C performs decoding using a video codec.

[0058] In step S205, the mesh decoding device 200 performs processing for each V3C unit.

[0059] In step S206, the demultiplexing unit 201 determines whether the currently processed V3C unit is the last V3C unit or not If Yes, this operation ends, and if No, this operation proceeds to step S207.

[0060] In step S207, the demultiplexing unit 201 proceeds to processing the next V3C unit.

[0061] According to this embodiment, by utilizing the syntax, it becomes possible to store compressed data in a single sub-bitstream even when the method for decoding the displacement amount is different.

[0062] The above-mentioned mesh encoding device 100 and mesh decoding device 200 may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]

[0063] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0064] 1. Mesh processing system 100...Mesh coding device 200…Mesh decoding unit 201…Demultiplexer 202...Basic mesh decoding unit 203…Subdivision part 204…Mesh decoding unit 205...Displacement amount decoding unit 205A... Decoding method determination unit 205B…First decoding unit 205C…Second decoding unit 206...Video decoding section

Claims

1. A mesh decoding device, comprising: a displacement amount decoding unit configured to decode the displacement amount bitstream to generate and output the displacement amount; The displacement amount decoding unit a first decoding unit configured to generate the displacement amount by arithmetic decoding; a second decoding unit configured to generate the displacement by a video codec; A mesh decoding device characterized by comprising: a decoding method for the displacement amount using a syntax included in the displacement amount bit stream; and a decoding method determination unit configured to switch whether the displacement amount is generated by the first decoding unit or the second decoding unit based on the decoding method.

2. 2. The mesh decoding device according to claim 1, wherein the displacement bit stream is composed of a V3C unit that is composed of a V3C unit header and a V3C unit payload.

3. The mesh decoding device according to claim 2 , wherein a V3C unit payload in the displacement bit stream is composed of a NAL unit.

4. 4. The mesh decoding device according to claim 3, wherein the V3C unit header includes a syntax indicating a type of the V3C unit and a syntax indicating a compression method of the displacement amount.

5. 4. The mesh decoding device according to claim 3, wherein the V3C unit header includes a syntax indicating a compression method for the displacement amount.

6. 1. A mesh decoding method, comprising: Decoding the displacement bitstream to generate and output a displacement, The process comprises: A step A of generating the displacement amount by arithmetic decoding; A step B of generating the displacement amount by a video codec; A mesh decoding method characterized by having a process C of generating a decoding method for the displacement amount using syntax contained in the displacement amount bit stream, and switching whether the displacement amount is generated in process A or process B based on the decoding method.

7. A program for causing a computer to function as a mesh decoding device, comprising: The mesh decoding device comprises a displacement decoding unit configured to decode a displacement bitstream to generate and output a displacement, The displacement amount decoding unit a first decoding unit configured to generate the displacement amount by arithmetic decoding; a second decoding unit configured to generate the displacement by a video codec; A program comprising: a decoding method for the displacement amount using a syntax included in the displacement amount bitstream; and a decoding method determination unit configured to switch whether the displacement amount is generated by the first decoding unit or the second decoding unit based on the decoding method.

Images