Mesh decoding device, mesh encoding device, mesh decoding method, and program

Abstract

To improve encoding efficiency.SOLUTION: An intra basic mesh decoding block 202A of a basic mesh decoding unit 202 of a mesh decoding device 200 according to the present invention includes: a multiple separation block 202A01 that separates an intra frame bit stream into a tracking basic mesh bit stream and a non-tracking basic mesh bit stream; a static mesh decoding block 202A02 that decodes the tracking basic mesh bit stream and the non-tracking basic mesh bit stream, respectively, to obtain a reconstructed tracking basic mesh and a reconstructed non-tracking basic mesh; and a mesh merging block 202A03 that merges the tracking basic mesh and the non-tracking basic mesh into a single basic mesh.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to a mesh decoding device, a mesh encoding device, a mesh decoding method, and a program. [Background technology]

[0002] In the current video-based dynamic mesh coding (V-DMC) model, there are two coding modes: (1) intra-frame coding mode and (2) inter-frame coding mode (see, for example, Non-Patent Document 1).

[0003] The choice of different modes is determined by whether the input dynamic mesh is tracked or not.

[0004] A tracked mesh has both one-to-one vertex and edge correspondences between frames, as shown in Figure 1(a), while a non-tracked mesh does not, as shown in Figure 1(b).

[0005] For tracking meshes, V-DMC skips coding of the current base mesh and codes the difference between the current base mesh and the previous base mesh, allowing for better temporal prediction.

[0006] On the other hand, in V-DMC, non-tracked meshes are coded separately, resulting in poor coding efficiency. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] Khaled Mammou, Jungsun Kim, Alexis M Tourapis, Dimitri Podborski, and Krasimir Kolarov, “[V-CG] Apple's Dynamic Mesh Coding CfP Response,” April 2022, ISO / IEC JTC 1 / SC 29 / WG 7. Summary of the Invention [Problem to be solved by the invention]

[0008] In the above-mentioned prior art, there are only two coding modes: intra-frame coding mode and inter-frame coding mode, depending on whether the input dynamic mesh is tracked or not.

[0009] However, in the non-tracked mesh shown in FIG. 2(a), as shown in FIG. 2(b), some parts may be tracked (tracked part in FIG. 2(b)) and some parts may not be tracked (non-tracked part in FIG. 2(b)).

[0010] In such a case, the conventional technology can only apply intra-frame coding, resulting in low coding efficiency.

[0011] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a mesh encoding device, a mesh decoding device, a mesh encoding method, and a program that can improve encoding efficiency. [Means for solving the problem]

[0012] A first feature of the present invention is a mesh decoding device comprising a base mesh decoding unit configured to decode one of the bit streams of a base mesh in intra mode and decode the other bit stream of the base mesh in partial inter mode, the base mesh decoding unit comprising: a demultiplexing block configured to separate the bit stream of the base mesh into a bit stream of an intra frame and a bit stream of a partial inter frame; an intra base mesh decoding block configured to decode the bit stream of the intra frame in intra mode; and a partial inter base mesh decoding block configured to decode the bit stream of the partial inter frame in partial inter mode, the intra base mesh decoding block comprising: a demultiplexing block configured to separate the bit stream of the intra frame into a bit stream of a tracking base mesh and a bit stream of a non-tracking base mesh; a static mesh decoding block configured to decode each of the bit streams of the tracking base mesh and the bit stream of the non-tracking base mesh to obtain reconstructed tracking base mesh and non-tracking base mesh; and a mesh merging block configured to merge the tracking base mesh and the non-tracking base mesh into one base mesh.

[0013] A second feature of the present invention is a mesh decoding device comprising a base mesh decoding unit configured to decode one of the bit streams of a base mesh in intra mode and decode the other bit stream of the base mesh in partial inter mode, the base mesh decoding unit comprising: a demultiplexing block configured to separate the bit stream of the base mesh into a bit stream of an intra frame and a bit stream of a partial inter frame; an intra base mesh decoding block configured to decode the bit stream of the intra frame in intra mode; and a partial inter base mesh decoding block configured to decode the bit stream of the partial inter frame in partial inter mode, the partial inter base mesh decoding block comprising: a demultiplexing block configured to separate the bit stream of the partial inter frame into a motion bit stream and a base mesh bit stream; a motion decoding block configured to decode the motion bit stream and update vertex positions of a tracking base mesh; a static mesh decoding block configured to decode the base mesh bit stream and generate a non-tracking base mesh; and a mesh merging block configured to merge the tracking base mesh and the non-tracking base mesh into one base mesh.

[0014] A third feature of the present invention is summarized as a mesh encoding device comprising: a pre-processing block configured to generate two base meshes, a non-tracking base mesh and a tracking base mesh; an intra-coding block configured to encode the non-tracking base mesh in an intra-coding mode; and a partial inter-coding block configured to encode the tracking base mesh in a partial inter-coding mode, wherein the intra-coding block comprises: a static mesh encoding block configured to encode the non-tracking base mesh and the tracking base mesh separately; a mesh merging block configured to merge the reconstructed non-tracking base mesh and the reconstructed tracking base mesh into one base mesh; and a displacement amount update block configured to update a displacement amount according to the merged base mesh.

[0015] A fourth feature of the present invention is a mesh decoding method comprising the steps of decoding one of the bit streams of a base mesh in intra mode and decoding the other bit stream of the base mesh in partial inter mode, the preceding steps comprising step A of separating the bit stream of the base mesh into a bit stream of an intra frame and a bit stream of a partial inter frame, step B of decoding the bit stream of the intra frame in intra mode, and step C of decoding the bit stream of the partial inter frame in partial inter mode, wherein in step B, the bit stream of the intra frame is separated into a bit stream of a tracking base mesh and a bit stream of a non-tracking base mesh, each of the bit streams of the tracking base mesh and the bit stream of the non-tracking base mesh is decoded to obtain a reconstructed tracking base mesh and a non-tracking base mesh, and the tracking base mesh and the non-tracking base mesh are merged into one base mesh.

[0016] A fifth aspect of the present invention is a program for making a computer function as a mesh decoding device, the mesh decoding device comprising a basic mesh decoding unit configured to decode one of a bit stream of a basic mesh in an intra mode and to decode the other bit stream of the basic mesh in a partial inter mode, the basic mesh decoding unit including a demultiplexing block configured to separate the bit stream of the basic mesh into a bit stream of an intra frame and a bit stream of a partial inter frame, an intra basic mesh decoding block configured to decode the bit stream of the intra frame in an intra mode, and a partial inter frame bit stream in a partial inter mode. The present invention relates to a method for decoding a bit stream of an intra frame in a static mesh decoding mode, and a partial inter base mesh decoding block configured to decode the bit stream of the intra frame in a static mesh decoding mode, the intra base mesh decoding block comprising: a multiplexing separation block configured to separate the bit stream of the intra frame into a bit stream of a tracking base mesh and a bit stream of a non-tracking base mesh; a static mesh decoding block configured to decode each of the bit streams of the tracking base mesh and the non-tracking base mesh to obtain reconstructed tracking base mesh and non-tracking base mesh; and a mesh merging block configured to merge the tracking base mesh and the non-tracking base mesh into one base mesh. Effect of the Invention

[0017] According to the present invention, it is possible to provide a mesh decoding device, a mesh encoding device, a mesh decoding method, and a program capable of improving encoding efficiency. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram for explaining the prior art. [Diagram 2] FIG. 2 is a diagram for explaining the prior art. [Diagram 3]FIG. 3 is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of functional blocks of the mesh encoding device 100 according to an embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of functional blocks of the pre-processing unit 101 of the mesh encoding device 100 according to an embodiment. [Figure 6] FIG. 6 is a diagram for explaining an example of the pre-processing sub-block 101A of the pre-processing unit 101 of the mesh encoding device 100 according to an embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of the operation of the mesh separation sub-block 101B of the pre-processing unit 101 of the mesh encoding device 100 according to an embodiment. [Figure 8] FIG. 8 is a diagram showing an example of functional blocks of the intra-coding block 102 of the mesh coding device 100 according to an embodiment. [Figure 9] FIG. 9 is a diagram for explaining an example of the operation of the mesh merging block 102C of the intra-coding block 102 of the mesh coding device 100 according to an embodiment. [Figure 10] FIG. 10 is a diagram showing an example of functional blocks of the partial inter-coding block 103 of the mesh coding device 100 according to an embodiment. [Figure 11] FIG. 11 is a diagram showing an example of functional blocks of the mesh decoding device 200 according to this embodiment. [Figure 12] FIG. 12 is a diagram showing an example of functional blocks of the basic mesh decoding unit 202 of the mesh decoding device 200 according to this embodiment. [Figure 13] FIG. 13 is a diagram showing an example of functional blocks of the intra base mesh decoding block 202A of the base mesh decoding unit 202 of the mesh decoding device 200 according to this embodiment. [Figure 14] FIG. 14 is a diagram showing an example of functional blocks of the partial inter basic mesh decoding block 202B of the basic mesh decoding unit 202 of the mesh decoding device 200 according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0020] First Embodiment The mesh processing system 1 according to this embodiment will be described below with reference to FIGS.

[0021] 3 is a diagram showing an example of the configuration of a mesh processing system 1 according to this embodiment. As shown in FIG.

[0022] Hereinafter, in this embodiment, the frame is synonymous with the mesh.

[0023] <Mesh encoding device 100> FIG. 4 is a diagram showing an example of functional blocks of the mesh coding device 100 according to this embodiment.

[0024] As shown in FIG. 4, the mesh coding device 100 includes a pre-processing block 101, an intra-coding block 102, and a partial inter-coding block 103.

[0025] The pre-processing block 101 is configured to generate two base meshes: a non-tracked base mesh and a tracked base mesh.

[0026] Here, as shown in FIG. 4, the tracking base mesh q(j) generated from the reference mesh M(j) is transmitted to a partial inter-coding block 103 for inter-frame coding. (Pre-processing block 101) An example of the pre-processing block 101 will now be described with reference to FIGS.

[0027] As shown in FIG. 5, the pre-processing block 101 includes a pre-processing sub-block 101A and a mesh separation sub-block 101B.

[0028] The pre-processing sub-block 101A is the same as that disclosed in Non-Patent Document 1, and is configured to generate basic meshes m(i) / m(j) and displacements d(i) / d(j) from input meshes M(i) / M(j), as shown in Figs. 5 and 6.

[0029] As shown in FIG. 5, the mesh separation sub-block 101B is configured to receive a base mesh m(i) / m(j) as input and separate the base mesh m(i) / m(j) into two base meshes: a non-tracking base mesh p(i) / p(j) and a tracking base mesh q(i) / q(j).

[0030] An example of the mesh separation sub-block 101B will be described below with reference to FIG.

[0031] As shown in FIG. 7, in step S101B-1, the mesh separation block 101B determines whether or not all the faces of the basic mesh m(i) / m(j) have been processed.

[0032] If the answer is Yes, the mesh separation block 101B ends this process. If the answer is No, the mesh separation block 101B proceeds to step S101B-2.

[0033] In step S101B-2, the mesh separation block 101B calculates the average height h of the three vertices of the current face. c Calculate.

[0034] For example, if the coordinates of the three vertices of the current face are (x1, y1, z1), (x2, y2, z2) and (x3, y3, z3), the average height of these three vertices h c is (z1+z2+z3) / 3.

[0035] In step S101B-3, the mesh separation block 101B is divided into 100 pieces of the mesh separation block 101B having the above-mentioned average height h c is the threshold h t It is determined whether the value is greater than .

[0036] If Yes, the mesh separation block 101B adds the current face to the tracking base mesh q(i) / q(j) in step S101B-4.

[0037] On the other hand, if the answer is No, then mesh separation block 101B adds the current face to the non-tracked base mesh p(i) / p(j) in step S101B-5.

[0038] Mesh separation block 101B then proceeds to process the next face in step S101B-6.

[0039] (Intra coding block 102) FIG. 8 shows an example of functional blocks of the intra-coding block 102.

[0040] As shown in FIG. 8, the intra-coding block 102 includes a quantization block 102A, a static mesh coding block 102B, a static mesh decoding block 102C, a mesh merging block 102D, a displacement amount update block 102E, a displacement amount coding block 102F, a texture coding block 102G, and a MUX 102H.

[0041] The non-tracked base mesh p(j) is compressed into a compressed non-tracked base mesh bitstream by the quantization block 102A and the static mesh coding block 102B.

[0042] Similarly, the tracking base mesh q(j) is compressed into a compressed tracking base mesh bitstream by the quantization block 102A and the static mesh coding block 102B.

[0043] The static mesh decoder 102C then reconstructs the compressed non-tracked base mesh bitstream into a reconstructed non-tracked base mesh p''(j).

[0044] The static mesh decoding unit 102C also reconstructs the compressed tracking base mesh bit stream into a reconstructed tracking base mesh q''(j).

[0045] As shown in FIG. 9, the base mesh merging block 102D merges the reconstructed non-tracked base mesh p''(j) and the tracked base mesh q''(j) into a single base mesh m'(j).

[0046] The displacement update block 102E updates the displacement d(j) according to the merged base mesh m'(j).

[0047] In addition, the quantization block 102A, static mesh encoding block 102B, static mesh decoding block 102C, displacement amount update block 102E, displacement amount encoding block 102F, texture encoding block 102G and MUX 102H are the same as those described in non-patent document 1.

[0048] The mesh merging block 102D merges the separated meshes (the non-tracked basic mesh p(j) and the tracked basic mesh q(j)) into a single mesh (the merged basic mesh m'(j)).

[0049] Here, if the meshes input to the mesh merge block 102D are {V1, F1} and {V2, F2}, where V1 and V2 are sets of vertices and F1 and F2 are sets of faces, then the merged base mesh m'(j) generated by the mesh merge block 102D is {V1 ∪ V2, F1 ∪ (F2 + |V1|)}, where |V1| is the total number of vertices in V1.

[0050] (Partial inter-coding block 103) FIG. 10 is a diagram showing an example of functional blocks of the partial inter-coding block 103.

[0051] As shown in FIG. 10, the partial inter-coding block 103 includes a quantization block 103A, a static mesh coding block 103B, a motion coding block 103C, a static mesh decoding block 103D, a mesh merging block 103E, a displacement amount update block 103F, a displacement amount coding block 103G, a texture coding block 103H, and a MUX 103I.

[0052] The motion coding block 103C is the same as that described in Non-Patent Document 1. Also, the blocks in Figure 10 basically have the same functions as the blocks with the same names in Figure 8.

[0053] A feature of the partial inter coding block 103 is that the current non-tracked base mesh p(i) is compressed by the static mesh coding block 103B, while the tracking bases q(i) are used to be compressed by the motion coding block 103C.

[0054] <Mesh Decoding Device 200> FIG. 11 is a diagram showing an example of functional blocks of the mesh decoding device 200 according to this embodiment.

[0055] As shown in FIG. 11, the mesh decoding device 200 includes a demultiplexing unit 201, a basic mesh decoding unit 202, a displacement amount decoding unit 203, a video decoding unit 204, a subdivision unit 205, and a mesh decoding unit 206.

[0056] The demultiplexing unit 201 is configured to separate the bit stream into three: a base mesh bit stream, a displacement amount bit stream, and a texture bit stream.

[0057] (Basic mesh decoding unit 202) FIG. 12 is a diagram showing an example of functional blocks of the basic mesh decoding unit 202. As shown in FIG.

[0058] As shown in FIG. 12, the base mesh decoding unit 202 includes an intra base mesh decoding block 202A and a partial inter base mesh decoding block 202B.

[0059] The intra base mesh decoding block 202A is configured to decode the bitstream of base meshes of an intra frame and output a reconstructed intra frame base mesh.

[0060] The partial inter base mesh decoding block 202B is configured to decode the bitstream of the base mesh of the current frame, and output the bitstream of the reconstructed base mesh of the current frame.

[0061] FIG. 13 is a diagram showing an example of functional blocks of the intra base mesh decoding block 202A.

[0062] As shown in FIG. 13, the intra basic mesh decoding block 202A includes a demultiplexing block 202A01, a static mesh decoding block 202A02, and a mesh merging block 202A03.

[0063] The demultiplexing block 202A01 is configured to separate the intra-frame bitstream into two: a tracking base mesh bitstream and a non-tracking base mesh bitstream.

[0064] The static mesh decoding block 202A02 decodes the tracked base mesh bitstream and the non-tracked base mesh bitstream separately to obtain a reconstructed tracked base mesh and a reconstructed non-tracked base mesh.

[0065] The mesh merging block 202A03 merges the reconstructed tracked base mesh with the non-tracked base mesh and outputs a merged base mesh m'(j).

[0066] Here, the output of the intra basis mesh decoding block 202A is the reconstructed tracking basis mesh q''(j) and the merged basis mesh m'(j).

[0067] FIG. 14 is a diagram showing an example of functional blocks of the partial inter basic mesh decoding block 202B.

[0068] As shown in FIG. 14, the partial inter basic mesh decoding block 202B includes a demultiplexing block 202B01, a motion decoding block 202B02, a static mesh decoding block 202B03, and a mesh merging block 202B04.

[0069] The demultiplexing block 202B01 is configured to separate the partial inter-frame bitstream into two: a motion bitstream and a base mesh bitstream.

[0070] The motion decoding block 202B02 is configured to decode the motion bitstream into motion vectors and add the motion vectors to the tracking basis mesh q''(j) of the intra-frame to obtain the tracking basis mesh q''(i) of the current frame.

[0071] The motion decoding block 202B02 is the same as that described in Non-Patent Document 1.

[0072] The static mesh decoding block 202B03 decodes the base mesh bitstream to obtain the non-tracked base mesh p''(i) of the current frame.

[0073] The mesh merging block 202B04 merges the tracked base mesh q''(i) of the current frame (the reconstructed tracked base mesh) and the non-tracked base mesh p''(i) of the current frame (the reconstructed non-tracked base mesh) into a single mesh m'(i).

[0074] The above-mentioned mesh encoding device 100 and mesh decoding device 200 may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]

[0075] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0076] 1. Mesh processing system 100...Mesh coding device 101…Pre-processing block 101A…Pre-treatment sub-block 101B…Mesh separation sub-block 102...Intra coding block 102A, 103A...Quantization block 102B, 103B...Static mesh coding block 102C, 103D...Static mesh decoding block 102D, 103E, 202B04…Mesh merge block 102E, 103F...Displacement update block 102F, 103G... Displacement encoding block 102G, 103H...Texture coding block 102H, 103I…MUX 103...Partial inter-coding block 103C...Motion coding block 200…Mesh decoding device 201…Demultiplexer 202...Basic mesh decoding unit 202A... Intra basic mesh decoding block 202A01, 202B01...Demultiplexing block 202A02, 202B03...Static mesh decoding block 202B...Partial inter-basic mesh decoding block 202B02...Motion decoding block 203...Displacement amount decoding unit 204...Video decoding section 205…Subdivision part 206…Mesh decoding unit

Claims

1. A mesh decoding device, comprising: A base mesh decoding unit configured to decode one of the bit streams of the base meshes in intra mode and to decode the other of the bit streams of the base meshes in partial inter mode, The basic mesh decoding unit a demultiplexing block configured to separate said base mesh bitstream into an intraframe bitstream and a partial interframe bitstream; an intra basic mesh decoding block configured to decode the bitstream of said intra frames in intra mode; a partial inter basic mesh decoding block configured to decode the partial inter frame bitstream in a partial inter mode, The intra-element mesh decoding block is a demultiplexing block configured to separate the intraframe bitstream into a tracking base mesh bitstream and a non-tracking base mesh bitstream; a static mesh decoding block configured to decode the bitstream of the tracked base mesh and the bitstream of the non-tracked base mesh, respectively, to obtain a reconstructed tracked base mesh and a non-tracked base mesh; a mesh merging block configured to merge the tracked base mesh and the non-tracked base mesh into one base mesh.

2. A mesh decoding device, comprising: A base mesh decoding unit configured to decode one of the bit streams of the base meshes in intra mode and to decode the other of the bit streams of the base meshes in partial inter mode, The basic mesh decoding unit a demultiplexing block configured to separate said base mesh bitstream into an intraframe bitstream and a partial interframe bitstream; an intra basic mesh decoding block configured to decode the bitstream of said intra frames in intra mode; a partial inter basic mesh decoding block configured to decode the partial inter frame bitstream in a partial inter mode, The partial inter-basic mesh decoding block includes: a demultiplexing block configured to separate the partial inter frame bitstream into a motion bitstream and a base mesh bitstream; a motion decoding block for decoding the motion bitstream and updating the vertex positions of a tracking base mesh; a static mesh decoding block configured to decode the base mesh bitstream and generate a non-tracked base mesh; a mesh merging block configured to merge the tracked base mesh and the non-tracked base mesh into one base mesh.

3. A mesh encoding device, comprising: a pre-processing block configured to generate two base meshes, a non-tracked base mesh and a tracked base mesh; an intra-coding block configured to code the non-tracked base mesh in an intra-coding mode; a partial inter coding block configured to code the tracking base mesh in a partial inter coding mode, The intra-coded block is a static mesh coding block configured to separately code the non-tracked base mesh and the tracked base mesh; a mesh merging block configured to merge the reconstructed non-tracked base mesh and the reconstructed tracked base mesh into one base mesh; a displacement update block configured to update a displacement in response to the merged base mesh.

4. 4. The mesh encoding device according to claim 3, wherein said pre-processing block comprises a mesh separation sub-block capable of separating said base mesh into said non-tracked base mesh and said tracked base mesh.

5. The partial inter-coded block is a static mesh coding block configured to code the non-tracked base mesh; a motion coding block configured to code said tracking basis mesh; a mesh merging block configured to merge the reconstructed non-tracked base mesh and the reconstructed tracked base mesh into one base mesh; 4. The mesh encoding device according to claim 3, further comprising a displacement update block configured to update a displacement in response to the merged base mesh.

6. 1. A mesh decoding method, comprising: Decoding one of the bitstreams of the base meshes in intra mode and decoding the other of the bitstreams of the base meshes in partial inter mode, In the front-end process, A step A of separating said basic mesh bit stream into an intraframe bit stream and a partial interframe bit stream; a step B of decoding said intra frame bitstream in intra mode; C. decoding the partial inter frame bitstream in partial inter mode, In the step B, Separating the intraframe bitstream into a tracking base mesh bitstream and a non-tracking base mesh bitstream; Decoding the bit stream of the tracked base mesh and the bit stream of the non-tracked base mesh, respectively, to obtain reconstructed tracked base mesh and non-tracked base mesh; A method for decoding meshes, comprising merging the tracked base meshes and the non-tracked base meshes into one base mesh.

7. A program for causing a computer to function as a mesh decoding device, comprising: The mesh decoding device comprises: A base mesh decoding unit configured to decode one of the bit streams of the base meshes in intra mode and to decode the other of the bit streams of the base meshes in partial inter mode, The basic mesh decoding unit a demultiplexing block configured to separate said base mesh bitstream into an intraframe bitstream and a partial interframe bitstream; an intra basic mesh decoding block configured to decode the bitstream of said intra frames in intra mode; a partial inter basic mesh decoding block configured to decode the partial inter frame bitstream in a partial inter mode, The intra-element mesh decoding block is a demultiplexing block configured to separate the intraframe bitstream into a tracking base mesh bitstream and a non-tracking base mesh bitstream; a static mesh decoding block configured to decode the bitstream of the tracked base mesh and the bitstream of the non-tracked base mesh, respectively, to obtain a reconstructed tracked base mesh and a non-tracked base mesh; a mesh merging block configured to merge the tracked base mesh and the non-tracked base mesh into one base mesh.

Images