Mesh decoding device, mesh decoding method and program
The mesh decoding device addresses redundancy in existing methods by omitting unnecessary control signals during mesh decoding, thereby reducing bit usage and improving efficiency.
Patent Information
- Application Number
- JP2024003512
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-01-12
- Publication Date
- 2025-07-25
AI Technical Summary
Existing mesh decoding methods encode control signals related to displacement amounts even when the displacement amounts themselves are not encoded, leading to redundancy and inefficiency in bit usage.
A mesh decoding device and method that includes an atlas data decoding unit which, when a geometry information flag is set, does not encode syntax related to displacement amounts and subdivisions, utilizing units such as a basic mesh decoding unit, subdivision unit, mesh decoding unit, and boundary correction unit to generate and correct meshes.
Reduces the number of bits required for decoding meshes by omitting redundant control signals, enhancing efficiency and reducing data overhead.
Smart Images
Figure 2025109551000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program.
Background Art
[0002] In Non-Patent Document 1, a mesh is divided into a basic mesh representing rough information and a displacement amount representing detailed information and decoded, and the displacement amount is decoded by a video codec.
Prior Art Documents
Non-Patent Documents
[0003]
Non-Patent Document 1
Non-Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, in the existing method, even when the displacement amount is not encoded, a control signal related to the encoding of the displacement amount is encoded, so there is a problem of redundancy. Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide a mesh decoding device, a mesh decoding method, and a program capable of reducing the number of bits.
Means for Solving the Problems
[0005] A first feature of the present invention is a mesh decoding device, which includes an atlas data decoding unit configured to decode an atlas bit stream to generate and output first control information, a basic mesh decoding unit configured to input a basic mesh bit stream to generate and output a basic mesh and second control information, a subdivision unit configured to input the first control information, the second control information, and the basic mesh to output a subdivided mesh and a subdivided vertex normal, a mesh decoding unit configured to input the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normal to generate a decoded mesh, and a boundary correction unit configured to input the first control information and the decoded mesh to output a boundary-corrected decoded mesh. The gist of the atlas data decoding unit is that when the geometry information flag is "1", it does not encode the syntax related to the displacement amount and subdivision.
[0006] A second feature of the present invention is a mesh decoding method, which includes step A of decoding an atlas bit stream to generate and output first control information, step B of inputting a basic mesh bit stream to generate and output a basic mesh and second control information, step C of inputting the first control information, the second control information, and the basic mesh to output a subdivided mesh and a subdivided vertex normal, step D of inputting the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normal to generate a decoded mesh, and step E of inputting the first control information and the decoded mesh to output a boundary-corrected decoded mesh. The gist of step A is that when the geometry information flag is "1", it does not encode the syntax related to the displacement amount and subdivision.
[0007] A third feature of the present invention is a program that causes a computer to function as a mesh decoding device, where the mesh decoding device includes an atlas data decoding unit configured to decode an atlas bitstream and generate and output first control information, a basic mesh decoding unit configured to input a basic mesh bitstream and generate and output a basic mesh and second control information, a subdivision unit configured to input the first control information, the second control information, and the basic mesh and output a subdivided mesh and subdivided vertex normals, a mesh decoding unit configured to input the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normals and generate a decoded mesh, and a boundary correction unit configured to input the first control information and the decoded mesh and output a boundary-corrected decoded mesh. The gist of the atlas data decoding unit is that when the geometry information flag is "1", it does not encode the syntax related to the displacement amount and subdivision.
Advantages of the Invention
[0008] According to the present invention, it is possible to provide a mesh decoding device, a mesh decoding method, and a program capable of reducing the amount of bits.
Brief Description of the Drawings
[0009]
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.
[0011] <First Embodiment> Hereinafter, with reference to FIGS. 1 to 10, the mesh processing system 1 according to the present embodiment will be described.
[0012] FIG. 1 is a diagram showing an example of the configuration of the mesh processing system 1 according to the present embodiment. As shown in FIG. 1, the mesh processing system 1 includes a mesh encoding device 100 and a mesh decoding device 200.
[0013] FIG. 2 is a diagram showing an example of the functional blocks of the mesh decoding device 200 according to the present embodiment.
[0014] As shown in FIG. 2, the mesh decoding device 200 includes a multiplex separation unit 201, a basic mesh decoding unit 202, a subdivision unit 203, a mesh decoding unit 204, a displacement amount decoding unit 205, a video decoding unit 206, an atlas data decoding unit 207, and a boundary correction unit 208.
[0015] The multiplex separation unit 201 is configured to separate the multiplexed bit stream into an atlas bit stream, a basic mesh bit stream, a displacement amount bit stream, and a texture bit stream.
[0016] The basic mesh decoding unit 202 is configured to decode the basic mesh bit stream based on control information (first control information) to generate and output control information (second control information) and a basic mesh.
[0017] The subdivision unit 203 is configured to generate and output subdivision vertices and their connection information from the basic mesh decoded by the basic mesh decoding unit 202 by the subdivision method indicated by the control information (first control information and second control information). The basic mesh is composed of one sub-mesh or a plurality of sub-meshes.
[0018] Here, the basic mesh, the added subdivision vertices, and their connection information are collectively referred to as a "subdivided mesh". Also, the sub-mesh, the added subdivision vertices, and their connection information are collectively referred to as a "subdivided sub-mesh".
[0019] The mesh decoding unit 204 is configured to generate and output a decoded mesh using the control information, the subdivided mesh, the subdivision vertex normal, and the displacement amount.
[0020] The displacement amount decoding unit 205 is configured to decode the displacement amount bit stream based on the control information to generate and output a displacement amount.
[0021] The video decoding unit 206 is configured to decode and output a texture by a video codec.
[0022] The atlas data decoding unit 207 is configured to decode the atlas bit stream and output control information (first control information).
[0023] The boundary correction unit 208 is configured to correct the sub-mesh boundary for the input decoded mesh and output a boundary-corrected decoded mesh.
[0024] (Atlas data decoding unit 207) Hereinafter, with reference to FIGS. 3 to 7, the control information decoded by the atlas data decoding unit 207 will be described.
[0025] First, the atlas bitstream may include AVE (asps vdmc extension), which is a set of control information of the atlas.
[0026] Second, the atlas bitstream may include AFMI (afps frame mesh information), which is a set of frame control information of the atlas.
[0027] Third, the atlas bitstream may include Zippering, which is a set of control information related to boundary correction.
[0028] AVE, AFMI, and Zippering may be defined for each mesh data, or may be defined at a fine granularity such as frames, patches, or tiles.
[0029] Hereinafter, with reference to FIGS. 3 to 7, the configuration of the syntax decoded by the atlas data decoding unit 207 and its decoding method will be described.
[0030] FIG. 3 is a diagram showing an example of the configuration of the syntax of AVE. Here, if the functions of the syntax are the same, a syntax name different from the syntax name shown in FIG. 3 may be used.
[0031] In the syntax structure of AVE shown in Figure 3, the Description field indicates how each syntax is encoded. Also, ue(v) means it is an unsigned 0th order exponential Golomb code, and u(n) means it is an n-bit flag.
[0032] Description is just an example, and encoding methods other than those in the Description of the charts such as u(n), ue(v), ae(v), etc. may be selected.
[0033] The same applies to the syntax structures of AFMI and zippering.
[0034] AVE is configured such that when the geometry information flag vps_geometry_video_present_flag is "1", asve_subdivision_method, asve_subdivision_iteration_count, asve_displacement_coordinate_system, vdmc_lifting_transform_parameters(0,AspsSubdivisionCount), asve_1d_displacement_flag, and asve_displacement_frame_qp_minus_N are not defined.
[0035] Here, asve_subdivision_method is a control signal representing a subdivision method, asve_subdivision_iteration_count is a control signal representing the number of subdivision iterations, asve_displacement_coordinate_system is a control signal representing a coordinate system for defining a displacement amount, vdmc_lifting_transform_parameters(0, AspsSubdivisionCount) is a control signal related to the displacement amount, asve_1d_displacement_flag is a control signal representing one-dimensional displacement amount encoding, and asve_displacement_frame_qp_minus_N is a control signal representing the quantization value of the displacement amount.
[0036] Figure 4 is a diagram showing an example of the configuration of the syntax of AFMI. Here, if the function of the syntax is the same, a syntax name different from the syntax name shown in Figure 4 may be used.
[0037] AFMI includes a single mesh control signal afmi_use_single_mesh_flag.
[0038] When afmi_use_single_mesh_flag is "0", AFMI includes afmi_num_submeshes_minus2, and NumSubMeshes can be decoded by adding 2 to afmi_num_submeshes_minus2.
[0039] When afmi_use_single_mesh_flag is "1", NumSubMeshes is decoded as 1.
[0040] Hereinafter, the method for decoding boundary information will be described with reference to FIGS. 5 to 7.
[0041] Figure 5 is a diagram showing an example of the configuration of the syntax of Zippering.
[0042] Zippering includes a control signal zp_persistence_flag that indicates whether to send boundary information for each frame.
[0043] Zippering includes a control signal zp_border_vertex_pair_count_minus1 that represents the number of boundary pairs for each frame.
[0044] Here, a boundary pair is a combination of vertices with the same coordinates. Details of the boundary pairs will be described later.
[0045] Zippering includes a control signal zp_border_vertex_pair_length_minus1[i] that represents the number of vertices constituting the i-th boundary pair for each frame.
[0046] Zippering includes a control signal zp_border_submesh_idx[i][j] that represents the submesh index of the j-th vertex constituting the i-th boundary pair for each frame.
[0047] Zippering includes a control signal zp_border_vertex_idx[i][j] that represents the vertex index of the j-th vertex constituting the i-th boundary pair for each frame.
[0048] Zippering includes a submesh subdivision vertex calculation control signal zp_subdivision_vertex_border_calculation_flag.
[0049] Figure 6 shows an example of a submesh decoded by the basic mesh decoding unit 202.
[0050] A displacement amount is defined for the vertices and their subdivision vertices on the submesh shown in Figure 6. However, even if the coordinates of the vertices are the same, the displacement amount may be different when the submesh indices are different. Therefore, there was a problem that holes would occur in the submesh boundary when the displacement amount was added in the mesh decoding unit 204.
[0051] To solve such problems, the boundary correction unit 208 corrects the vertices of the decoded mesh output from the mesh decoding unit 204 based on the boundary information.
[0052] Here, the boundary information is information regarding vertices having the same coordinates, even though they are vertices on different sub-meshes at the time of sub-mesh decoding.
[0053] Note that the boundary information is obtained by decoding the above-described atlas bitstream. Also, the boundary vertices can be decoded as a set of boundary pairs.
[0054] FIG. 7 is an example of zp_border_submesh_idx and zp_border_vertex_idx.
[0055] In the example of FIG. 7, when the index of the boundary pair is "2", zp_border_submesh_idx[2]=[0,1,2] and zp_border_vertex_idx[2]=[5,2,0].
[0056] This indicates that the sub-mesh vertex v(0,5), the sub-mesh vertex v(1,2), and the sub-mesh vertex v(2,0) are the same vertex. Here, v(i,j) indicates the vertex j of the sub-mesh i.
[0057] These combinations of the same vertices are called boundary pairs. In the above example, the boundary pair is 2, and the same applies to all other boundary pairs.
[0058] When the boundary pair index is i, the vertex pairs are obtained as v(zp_border_submesh_idx[i][0], zp_border_vertex_idx[i][0]), …, v(zp_border_submesh_idx[i][zp_border_vertex_pair_length_minus1[i]], zp_border_vertex_idx[i][zp_border_vertex_pair_length_minus1[i]]).
[0059] By repeating the above procedure for all boundary pair indices from 0 to zp_border_pair_count_minus1, the boundary pairs included in all boundary information can be obtained.
[0060] Any boundary information can be decoded by the same procedure as the above example.
[0061] (Basic mesh decoding unit 202) The basic mesh decoding unit 202 is configured to decode the basic mesh bitstream based on the control information and generate and output the basic mesh.
[0062] Here, the basic mesh is composed of one submesh or a plurality of submeshes. A submesh is composed of a plurality of vertices in three-dimensional space and sides connecting such a plurality of vertices.
[0063] As shown in FIG. 8, the basic mesh is configured by combining basic faces represented by three vertices.
[0064] The basic mesh decoding unit 202 may be configured to decode the basic mesh bitstream using, for example, Draco shown in Non-Patent Document 2 or other known mesh decoding techniques.
[0065] The basic mesh decoding unit 202 may be configured to independently decode a plurality of sub-meshes using Draco or other known mesh decoding techniques.
[0066] (Subdivision unit 203) Referring to FIG. 9, a method for decoding a subdivided mesh in the subdivision unit 203 will be described.
[0067] The subdivision unit 203 is configured to output a subdivided mesh, subdivided vertex normals, and control information based on the control information and the basic mesh.
[0068] FIG. 10 is an example of a subdivided mesh obtained by performing one-time subdivision by the Mid-edge subdivision method. The Mid-edge subdivision method generates a subdivided mesh by dividing the midpoints of the edges that make up the mesh.
[0069] By repeating the above operations, a finer subdivided mesh can be generated.
[0070] The subdivision unit 203 is configured to store, at the time of generating the subdivided vertices, which vertices the subdivided vertices are points between, and output that information as control information to the boundary correction unit 208.
[0071] The subdivision unit 203 may be configured not to record the above information when the zp_subdivision_vertex_border_calculation_flag is "1".
[0072] The subdivided mesh may be configured to be independently decoded for each sub-mesh.
[0073] (Displacement decoding unit 205) The displacement decoding unit 205 is configured to decode a displacement bit stream to generate and output a displacement.
[0074] The displacement may be defined to be independently decoded for each sub-mesh.
[0075] (Mesh Decoding Unit 204) The mesh decoding unit 204 is configured to output a decoded mesh based on the input control information, the subdivided mesh, and the displacement amount.
[0076] The mesh decoding unit 204 decodes the decoded mesh by adding the displacement amount to the vertices of the input subdivided mesh.
[0077] At this time, the mesh decoding unit 204 may output a plurality of decoded meshes for each sub-mesh based on the displacement amount corresponding to the sub-mesh and the subdivided mesh.
[0078] Also, the mesh decoding unit 204 may be configured to set the coordinate system of the displacement amount based on a control signal representing the coordinate system of the displacement amount defined for each sub-mesh, add the displacement amount to the subdivided vertices on the set coordinate system, and calculate and output the decoded mesh.
[0079] (Boundary Correction Unit 208) Referring to FIG. 10, the decoding method of the boundary correction decoded mesh in the boundary correction unit 208 will be described.
[0080] The boundary correction unit 208 is configured to correct the sub-mesh boundary for the input decoded mesh and output the boundary correction decoded mesh.
[0081] When the zp_persistence_flag is "1", the boundary correction unit 208 uses the sub-mesh boundary information of the reference frame as the sub-mesh boundary information as it is.
[0082] The boundary correction unit 208 may be configured to skip the boundary correction when the frame to be corrected is not an inter-frame.
[0083] When there is no index of the sub-mesh or vertex index included in the boundary information, the boundary correction unit 208 may be configured to skip the boundary correction and output the decoded mesh as it is.
[0084] Alternatively, when there is no index of the sub-mesh or vertex index included in the boundary information, the boundary correction unit 208 may be configured to skip the correction only for the boundary pairs of these indexes.
[0085] FIG. 10 is a diagram schematically showing a method of boundary correction. Hereinafter, the procedure of boundary correction will be described with reference to FIG. 10.
[0086] When the zp_subdivision_vertex_border_calculation_flag is "0", the atlas data decoding unit 207 has only obtained the boundary pairs of the sub-mesh vertices. Therefore, in order to perform correction on the subdivided vertices as well, it is necessary to obtain the boundary pairs of the subdivided vertices.
[0087] However, when the zp_subdivision_vertex_border_calculation_flag is "1", the boundary correction unit 208 may perform the correction process only on the boundary pairs obtained by the atlas data decoding unit 207.
[0088] Here, the procedure for decoding the boundary pairs of the subdivided vertices will be described.
[0089] First, the boundary correction unit 208 obtains the midpoint of the sub-mesh boundary vertices.
[0090] In the example of FIG. 10, the midpoint of the boundary vertex v(0, 3) and the boundary vertex v(0, 4) is v(0, 5).
[0091] Next, the boundary correction unit 208 obtains the respective boundary vertex pairs of the boundary vertices v(0, 3) and v(0, 4). The respective boundary vertex pairs of the boundary vertices v(0, 3) and v(0, 4) are the boundary vertices v(1, 0) and v(1, 1).
[0092] Next, the boundary correction unit 208 obtains the midpoint of the boundary vertices v(1, 0) and v(1, 1). Such a midpoint is v(1, 4).
[0093] Finally, the boundary correction unit 208 sets the boundary vertices v(0, 5) and v(1, 4) as a boundary vertex pair.
[0094] When the number of subdivision times is 2 or more, the boundary correction unit 208 can obtain boundary vertex pairs by repeating the above procedure.
[0095] The boundary correction unit 208 obtains the average value of the vertex coordinates of the vertices constituting each boundary vertex pair, and corrects the vertices constituting the boundary vertex pair with the average value.
[0096] According to the present embodiment, when the geometry information flag vps_geometry_video_present_flag is "1", the bit amount can be reduced by not encoding the syntax related to the displacement amount and the subdivision.
[0097] Also, according to the present embodiment, Zippering is introduced, and when Zippering (sub-mesh subdivision vertex calculation control signal) is "1", the calculation of the boundary information of the subdivision vertices is skipped, and the boundary correction of the sub-mesh is performed based only on the boundary information stored in the bit stream. Thereby, even when the number of subdivision times is different for each sub-mesh, correction can be performed.
[0098] Furthermore, according to this embodiment, when the single mesh control signal afmi_use_single_mesh_flag is "1", 1 is added to the signal representing the number of sub-meshes, and in other cases, 2 is added to the control signal representing the number of sub-meshes to decode the number of sub-meshes. In this way, by using afmi_use_single_mesh_flag in combination, the range of the number of expressible sub-meshes can be expanded.
[0099] The above-described mesh encoding device 100 and mesh decoding device 200 may be realized by a program that causes a computer to execute each function (each process).
Industrial Applicability
[0100] Note that according to this embodiment, for example, since an overall improvement in service quality can be realized in moving image communication, it is possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization and foster innovation."
Explanation of Signs
[0101] 1... Mesh processing system 100... Mesh encoding device 200... Mesh decoding device 201... Multiplex separation unit 202... Basic mesh decoding unit 203... Subdivision unit 204... Mesh decoding unit 205... Displacement amount decoding unit 206... Video decoding unit 207... Atlas data decoding unit 208... Boundary correction unit
Claims
1. A mesh decoding device, comprising: An atlas data decoding unit configured to decode an atlas bit stream and generate and output first control information; A basic mesh decoding unit configured to input a basic mesh bit stream and generate and output a basic mesh and second control information; A subdivision unit configured to input the first control information, the second control information, and the basic mesh and output a subdivided mesh and a subdivided vertex normal; A mesh decoding unit configured to input the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normal and generate a decoded mesh; A boundary correction unit configured to input the first control information and the decoded mesh and output a boundary-corrected decoded mesh, wherein the atlas data decoding unit does not encode the syntax related to the displacement amount and subdivision when a geometry information flag is "1".
2. A mesh decoding method, comprising: Step A of decoding an atlas bit stream and generating and outputting first control information; Step B of inputting a basic mesh bit stream and generating and outputting a basic mesh and second control information; Step C of inputting the first control information, the second control information, and the basic mesh and outputting a subdivided mesh and a subdivided vertex normal; Step D of inputting the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normal and generating a decoded mesh; Step E of inputting the first control information and the decoded mesh and outputting a boundary-corrected decoded mesh, wherein in Step A, when a geometry information flag is "1", the syntax related to the displacement amount and subdivision is not encoded.
3. A program for causing a computer to function as a mesh decoding device, wherein the mesh decoding device comprises: An atlas data decoding unit configured to decode an atlas bit stream and generate and output first control information; A basic mesh decoding unit configured to input a basic mesh bit stream and generate and output a basic mesh and second control information; A subdivision unit configured to input the first control information, the second control information, and the basic mesh and output a subdivided mesh and a subdivided vertex normal; A mesh decoding unit configured to generate a decoded mesh using the first control information, the displacement amount, the subdivided mesh, and the subdivided vertex normal as inputs; A boundary correction unit configured to input the first control information and the decoded mesh and output a boundary-corrected decoded mesh; A program characterized in that the atlas data decoding unit does not encode the syntax related to the displacement amount and subdivision when the geometry information flag is "1".