Mesh decoding device, mesh decoding method and program
The mesh decoding device addresses the issue of overlapping sub-mesh representation ranges by adjusting control signals to expand the number of representable sub-meshes, enhancing mesh decoding efficiency.
Patent Information
- Application Number
- JP2024003514
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-01-12
- Publication Date
- 2025-07-25
AI Technical Summary
Existing methods face issues with overlapping ranges of sub-mesh representation due to single mesh control signals, limiting the expandability of sub-meshes.
A mesh decoding device comprising an atlas data decoding unit, basic mesh decoding unit, subdivision unit, mesh decoding unit, and boundary correction unit, which decodes atlas bit streams and generates control information to expand the range of representable sub-meshes by adjusting the number of sub-meshes based on control signals.
Enables the expansion of the range of representable sub-meshes, improving the efficiency and flexibility in mesh decoding processes.
Smart Images
Figure 2025109553000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program.
Background Art
[0002] In Non-Patent Document 1, a mesh is divided into a basic mesh representing rough information and a displacement amount representing detailed information for decoding, and the displacement amount is decoded by a video codec.
Prior Art Documents
Non-Patent Documents
[0003]
Non-Patent Document 1
Non-Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, existing methods have a problem that the ranges of the number of sub-meshes represented by a single mesh control signal and a sub-mesh number control signal overlap. Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide a mesh decoding device, a mesh decoding method, and a program capable of expanding the range of the number of representable sub-meshes.
Means for Solving the Problems
[0005] A first feature of the present invention is a mesh decoding device, comprising an atlas data decoding unit configured to decode an atlas bit stream and generate and output first control information including boundary information, a basic mesh decoding unit configured to take a basic mesh bit stream as an input and generate and output a basic mesh and second control information, a subdivision unit configured to take the first control information, the second control information, and the basic mesh as inputs and output a subdivided mesh and a subdivided vertex normal, a mesh decoding unit configured to take the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normal as inputs and generate a decoded mesh, and a boundary correction unit configured to take the first control information and the decoded mesh as inputs and output a boundary-corrected decoded mesh. The atlas data decoding unit is characterized in that when a single mesh control signal is "1", it decodes with the number of sub-meshes being 1, and in other cases, it decodes by adding "2" to a control signal representing the number of sub-meshes.
[0006] A second feature of the present invention is a mesh decoding method, comprising step A of decoding an atlas bit stream and generating and outputting first control information including boundary information, step B of taking a basic mesh bit stream as an input and generating and outputting a basic mesh and second control information, step C of taking the first control information, the second control information, and the basic mesh as inputs and outputting a subdivided mesh and a subdivided vertex normal, step D of taking the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normal as inputs and generating a decoded mesh, and step E of taking the first control information and the decoded mesh as inputs and outputting a boundary-corrected decoded mesh. In step A, when a single mesh control signal is "1", it is decoded with the number of sub-meshes being 1, and in other cases, it is decoded by adding "2" to a control signal representing the number of sub-meshes.
[0007] A third feature of the present invention is a program that causes a computer to function as a mesh decoding device, wherein the mesh decoding device is configured to decode an atlas bit stream to generate and output first control information including boundary information, an atlas data decoding unit configured to generate and output a basic mesh and second control information using a basic mesh bit stream as an input, a subdivision unit configured to input the first control information, the second control information, and the basic mesh and output a subdivided mesh and subdivided vertex normals, a mesh decoding unit configured to input the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normals and generate a decoded mesh, and a boundary correction unit configured to input the first control information and the decoded mesh and output a boundary-corrected decoded mesh. The gist of the atlas data decoding unit is to decode with the number of sub-meshes being 1 when a single mesh control signal is "1", and otherwise to decode by adding "2" to a control signal representing the number of sub-meshes.
Advantages of the Invention
[0008] According to the present invention, it is possible to provide a mesh decoding device, a mesh decoding method, and a program capable of expanding the range of the number of representable sub-meshes.
Brief Description of the Drawings
[0009]
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components and the like, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.
[0011] <First Embodiment> Hereinafter, with reference to FIGS. 1 to 10, the mesh processing system 1 according to the present embodiment will be described.
[0012] FIG. 1 is a diagram showing an example of the configuration of the mesh processing system 1 according to the present embodiment. As shown in FIG. 1, the mesh processing system 1 includes a mesh encoding device 100 and a mesh decoding device 200.
[0013] FIG. 2 is a diagram showing an example of the functional blocks of the mesh decoding device 200 according to the present embodiment.
[0014] As shown in FIG. 2, the mesh decoding device 200 includes a multiplex separation unit 201, a basic mesh decoding unit 202, a subdivision unit 203, a mesh decoding unit 204, a displacement amount decoding unit 205, a video decoding unit 206, an atlas data decoding unit 207, and a boundary correction unit 208.
[0015] The multiplex separation unit 201 is configured to separate the multiplexed bit stream into an atlas bit stream, a basic mesh bit stream, a displacement amount bit stream, and a texture bit stream.
[0016] The basic mesh decoding unit 202 is configured to decode the basic mesh bit stream based on control information (first control information) to generate and output control information (second control information) and a basic mesh.
[0017] The subdivision unit 203 is configured to generate and output subdivision vertices and their connection information from the basic mesh decoded by the basic mesh decoding unit 202 by a subdivision method indicated by control information (first control information and second control information). The basic mesh is composed of one sub-mesh or a plurality of sub-meshes.
[0018] Here, the basic mesh, the added subdivision vertices, and their connection information are collectively referred to as a "subdivided mesh". Also, the sub-mesh, the added subdivision vertices, and their connection information are collectively referred to as a "subdivided sub-mesh".
[0019] The mesh decoding unit 204 is configured to generate and output a decoded mesh using control information, the subdivided mesh, the subdivision vertex normal, and the displacement amount.
[0020] The displacement amount decoding unit 205 is configured to decode the displacement amount bit stream based on control information to generate and output a displacement amount.
[0021] The video decoding unit 206 is configured to decode and output a texture by a video codec.
[0022] The atlas data decoding unit 207 is configured to decode an atlas bit stream and output control information (first control information).
[0023] The boundary correction unit 208 is configured to correct the sub-mesh boundary for the input decoded mesh and output a boundary-corrected decoded mesh.
[0024] (Atlas data decoding unit 207) Hereinafter, with reference to FIGS. 3 to 7, the control information decoded by the atlas data decoding unit 207 will be described.
[0025] First, the atlas bit stream may include AVE (asps vdmc extension), which is a set of control information of the atlas.
[0026] Second, the atlas bit stream may include AFMI (afps frame mesh information), which is a set of frame control information of the atlas.
[0027] Third, the atlas bit stream may include Zippering, which is a set of control information related to boundary correction.
[0028] AVE, AFMI, and Zippering may be defined for each mesh data, or may be defined at a fine granularity such as a frame, a patch, or a tile.
[0029] Hereinafter, with reference to FIGS. 3 to 7, the configuration of the syntax decoded by the atlas data decoding unit 207 and its decoding method will be described.
[0030] FIG. 3 is a diagram showing an example of the configuration of the syntax of AVE. Here, if the functions of the syntax are the same, a syntax name different from the syntax name shown in FIG. 3 may be used.
[0031] In the syntax structure of AVE shown in Figure 3, the Description field indicates how each syntax is encoded. Also, ue(v) means it is an unsigned zero-order exponential Golomb code, and u(n) means it is an n-bit flag.
[0032] Description is just an example, and encoding methods other than those in the Description of charts such as u(n), ue(v), and ae(v) may be selected.
[0033] The same applies to the syntax structures of AFMI and zippering.
[0034] AVE is configured such that when the geometry information flag vps_geometry_video_present_flag is "1", asve_subdivision_method, asve_subdivision_iteration_count, asve_displacement_coordinate_system, vdmc_lifting_transform_parameters(0,AspsSubdivisionCount), asve_1d_displacement_flag, and asve_displacement_frame_qp_minus_N are not defined.
[0035] Here, asve_subdivision_method is a control signal representing a subdivision method, asve_subdivision_iteration_count is a control signal representing the number of subdivision iterations, asve_displacement_coordinate_system is a control signal representing a coordinate system for defining a displacement amount, vdmc_lifting_transform_parameters(0,AspsSubdivisionCount) is a control signal related to the displacement amount, asve_1d_displacement_flag is a control signal representing one-dimensional displacement amount encoding, and asve_displacement_frame_qp_minus_N is a control signal representing the quantization value of the displacement amount.
[0036] Figure 4 is a diagram showing an example of the configuration of the AFMI syntax. Here, if the functions of the syntax are the same, a syntax name different from the one shown in Figure 4 may be used.
[0037] AFMI includes a single mesh control signal afmi_use_single_mesh_flag.
[0038] When afmi_use_single_mesh_flag is "0", AFMI includes afmi_num_submeshes_minus2, and NumSubMeshes can be decoded by adding 2 to afmi_num_submeshes_minus2.
[0039] When afmi_use_single_mesh_flag is "1", NumSubMeshes is decoded as 1.
[0040] Hereinafter, the method for decoding boundary information will be described with reference to FIGS. 5 to 7.
[0041] Figure 5 is a diagram showing an example of the configuration of the Zippering syntax.
[0042] Zippering includes a control signal zp_persistence_flag that indicates whether to send boundary information for each frame.
[0043] Zippering includes a control signal zp_border_vertex_pair_count_minus1 that represents the number of boundary pairs for each frame.
[0044] Here, a boundary pair is a combination of vertices with the same coordinates. Details of the boundary pairs will be described later.
[0045] Zippering includes a control signal zp_border_vertex_pair_length_minus1[i] that represents the number of vertices constituting the i-th boundary pair for each frame.
[0046] Zippering includes a control signal zp_border_submesh_idx[i][j] that represents the submesh index of the j-th vertex constituting the i-th boundary pair for each frame.
[0047] Zippering includes a control signal zp_border_vertex_idx[i][j] that represents the vertex index of the j-th vertex constituting the i-th boundary pair for each frame.
[0048] Zippering includes a submesh subdivision vertex calculation control signal zp_subdivision_vertex_border_calculation_flag.
[0049] Figure 6 shows an example of a submesh decoded by the basic mesh decoder 202.
[0050] A displacement amount is defined for the vertices and their subdivision vertices on the submesh shown in Figure 6. However, even if the coordinates of the vertices are the same, the displacement amount may be different when the submesh indices are different. Therefore, there was a problem that holes would occur in the submesh boundary when the displacement amount was added in the mesh decoder 204.
[0051] To solve such problems, the boundary correction unit 208 corrects the vertices of the decoded mesh output from the mesh decoding unit 204 based on the boundary information.
[0052] Here, the boundary information is information regarding vertices having the same coordinates, even though they are vertices on different sub-meshes at the time of sub-mesh decoding.
[0053] Note that the boundary information is obtained by decoding the above-described atlas bit stream. Also, the boundary vertices can be decoded as a set of boundary pairs.
[0054] FIG. 7 is an example of zp_border_submesh_idx and zp_border_vertex_idx.
[0055] In the example of FIG. 7, when the index of the boundary pair is "2", zp_border_submesh_idx[2] = [0, 1, 2] and zp_border_vertex_idx[2] = [5, 2, 0].
[0056] This indicates that the sub-mesh vertex v(0, 5), the sub-mesh vertex v(1, 2), and the sub-mesh vertex v(2, 0) are the same vertex. Here, v(i, j) indicates the vertex j of the sub-mesh i.
[0057] These combinations of the same vertices are called boundary pairs. In the above example, the boundary pair is 2, and the same applies to all other boundary pairs.
[0058] When the boundary pair index is i, the vertex pairs are obtained as v(zp_border_submesh_idx[i][0], zp_border_vertex_idx[i][0]), …, v(zp_border_submesh_idx[i][zp_border_vertex_pair_length_minus1[i]], zp_border_vertex_idx[i][zp_border_vertex_pair_length_minus1[i]]).
[0059] By repeating the above procedure for all boundary pair indices from 0 to zp_border_pair_count_minus1, the boundary pairs included in all boundary information can be obtained.
[0060] For any boundary information, it can be decoded by the same procedure as the above example.
[0061] (Basic mesh decoding unit 202) The basic mesh decoding unit 202 is configured to decode a basic mesh bit stream based on control information and generate and output a basic mesh.
[0062] Here, the basic mesh is composed of one sub-mesh or a plurality of sub-meshes. A sub-mesh is composed of a plurality of vertices in three-dimensional space and edges connecting such a plurality of vertices.
[0063] As shown in FIG. 8, the basic mesh is configured by combining basic faces represented by three vertices.
[0064] The basic mesh decoding unit 202 may be configured to decode a basic mesh bit stream using, for example, Draco shown in Non-Patent Document 2 or other known mesh decoding techniques.
[0065] The basic mesh decoding unit 202 may be configured to independently decode a plurality of sub-meshes using Draco or other known mesh decoding techniques.
[0066] (Subdivision unit 203) Referring to FIG. 9, a method for decoding a subdivided mesh in the subdivision unit 203 will be described.
[0067] The subdivision unit 203 is configured to output a subdivided mesh, subdivided vertex normals, and control information based on the control information and the basic mesh.
[0068] FIG. 10 is an example of a subdivided mesh obtained by performing one-time subdivision using the Mid-edge subdivision method. The Mid-edge subdivision method generates a subdivided mesh by dividing the midpoints of the edges that make up the mesh.
[0069] By repeating the above operations, a finer subdivided mesh can be generated.
[0070] The subdivision unit 203 is configured to store, at the time of generating the subdivided vertices, which vertices the subdivided vertices are points between, and output that information as control information to the boundary correction unit 208.
[0071] The subdivision unit 203 may be configured not to record the above information when the zp_subdivision_vertex_border_calculation_flag is "1".
[0072] The subdivided mesh may be configured to be independently decoded for each sub-mesh.
[0073] (Displacement decoding unit 205) The displacement decoding unit 205 is configured to decode a displacement bitstream to generate and output a displacement.
[0074] The displacement may be defined to be independently decoded for each sub-mesh.
[0075] (Mesh Decoding Unit 204) The mesh decoding unit 204 is configured to output a decoded mesh based on the input control information, the subdivided mesh, and the displacement amount.
[0076] The mesh decoding unit 204 decodes the decoded mesh by adding the displacement amount to the vertices of the input subdivided mesh.
[0077] At this time, the mesh decoding unit 204 may output a plurality of decoded meshes for each sub-mesh based on the displacement amount corresponding to the sub-mesh and the subdivided mesh.
[0078] Alternatively, the mesh decoding unit 204 may be configured to calculate and output the decoded mesh by setting the coordinate system of the displacement amount based on a control signal representing the coordinate system of the displacement amount defined for each sub-mesh, and adding the displacement amount to the subdivided vertices on the set coordinate system.
[0079] (Boundary Correction Unit 208) With reference to FIG. 10, a method for decoding the boundary-corrected decoded mesh in the boundary correction unit 208 will be described.
[0080] The boundary correction unit 208 is configured to correct the sub-mesh boundary for the input decoded mesh and output a boundary-corrected decoded mesh.
[0081] When zp_persistence_flag is "1", the boundary correction unit 208 uses the sub-mesh boundary information of the reference frame as the sub-mesh boundary information as it is.
[0082] The boundary correction unit 208 may be configured to skip the boundary correction when the frame to be corrected is not an inter-frame.
[0083] When there is no index of the sub-mesh or vertex index included in the boundary information, the boundary correction unit 208 may be configured to skip the boundary correction and output the decoded mesh as it is.
[0084] Also, when there is no index of the sub-mesh or vertex index included in the boundary information, the boundary correction unit 208 may be configured to skip the correction only for the boundary pairs of these indexes.
[0085] FIG. 10 is a diagram schematically showing a method of boundary correction. Hereinafter, the procedure of boundary correction will be described with reference to FIG. 10.
[0086] When the zp_subdivision_vertex_border_calculation_flag is "0", the atlas data decoding unit 207 has only obtained the boundary pairs of the sub-mesh vertices. Therefore, in order to perform correction on the subdivided vertices as well, it is necessary to obtain the boundary pairs of the subdivided vertices.
[0087] However, when the zp_subdivision_vertex_border_calculation_flag is "1", the boundary correction unit 208 may perform the correction process only on the boundary pairs obtained by the atlas data decoding unit 207.
[0088] Here, the procedure for decoding the boundary pairs of the subdivided vertices will be described.
[0089] First, the boundary correction unit 208 obtains the midpoint of the sub-mesh boundary vertices.
[0090] In the example of FIG. 10, the midpoint of the boundary vertex v(0, 3) and the boundary vertex v(0, 4) is v(0, 5).
[0091] Next, the boundary correction unit 208 obtains the boundary vertex pairs of the boundary vertices v(0, 3) and v(0, 4) respectively. The boundary vertex pairs of the boundary vertices v(0, 3) and v(0, 4) are the boundary vertices v(1, 0) and v(1, 1).
[0092] Next, the boundary correction unit 208 obtains the midpoint of the boundary vertices v(1, 0) and v(1, 1). Such a midpoint is v(1, 4).
[0093] Finally, the boundary correction unit 208 sets the boundary vertices v(0, 5) and v(1, 4) as a boundary vertex pair.
[0094] When the number of subdivision times is 2 or more, the boundary correction unit 208 can obtain boundary vertex pairs by repeating the above procedure.
[0095] The boundary correction unit 208 obtains the average value of the vertex coordinates of the vertices constituting each boundary vertex pair, and corrects the vertices constituting the boundary vertex pair with the average value.
[0096] According to the present embodiment, when the geometry information flag vps_geometry_video_present_flag is "1", the syntax related to the displacement amount and subdivision is not encoded, so that the number of bits can be reduced.
[0097] Also, according to the present embodiment, Zippering is introduced. When Zippering (sub-mesh subdivision vertex calculation control signal) is "1", the calculation of the boundary information of the subdivision vertices is skipped, and the boundary correction of the sub-mesh is performed based only on the boundary information stored in the bitstream. Thereby, even when the number of subdivision times is different for each sub-mesh, correction can be performed.
[0098] Furthermore, according to this embodiment, when the single mesh control signal afmi_use_single_mesh_flag is "1", 1 is added to the signal representing the number of sub-meshes, and in other cases, 2 is added to the control signal representing the number of sub-meshes, thereby decoding the number of sub-meshes. In this way, by using afmi_use_single_mesh_flag in combination, the range of the number of representable sub-meshes can be extended.
[0099] The above-described mesh encoding device 100 and mesh decoding device 200 may be realized by a program that causes a computer to execute each function (each process).
Industrial Applicability
[0100] Note that according to this embodiment, for example, since an overall improvement in service quality can be realized in moving image communication, it is possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization, and foster innovation."
Explanation of Signs
[0101] 1... Mesh processing system 100... Mesh encoding device 200... Mesh decoding device 201... Multiplex separation unit 202... Basic mesh decoding unit 203... Subdivision unit 204... Mesh decoding unit 205... Displacement amount decoding unit 206... Video decoding unit 207... Atlas data decoding unit 208... Boundary correction unit
Claims
1. A mesh decoding device, comprising: An atlas data decoding unit configured to decode an atlas bit stream and generate and output first control information including boundary information; A basic mesh decoding unit configured to input a basic mesh bit stream and generate and output a basic mesh and second control information; A subdivision unit configured to input the first control information, the second control information, and the basic mesh and output a subdivided mesh and a subdivided vertex normal; A mesh decoding unit configured to input the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normal and generate a decoded mesh; A boundary correction unit configured to input the first control information and the decoded mesh and output a boundary-corrected decoded mesh, wherein the atlas data decoding unit decodes with the number of sub-meshes being 1 when a single mesh control signal is "1", and otherwise decodes by adding "2" to a control signal representing the number of sub-meshes.
2. A mesh decoding method, comprising: Step A of decoding an atlas bit stream and generating and outputting first control information including boundary information; Step B of inputting a basic mesh bit stream and generating and outputting a basic mesh and second control information; Step C of inputting the first control information, the second control information, and the basic mesh and outputting a subdivided mesh and a subdivided vertex normal; Step D of inputting the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normal and generating a decoded mesh; Step E of inputting the first control information and the decoded mesh and outputting a boundary-corrected decoded mesh, wherein in Step A, when a single mesh control signal is "1", decoding is performed with the number of sub-meshes being 1, and otherwise decoding is performed by adding "2" to a control signal representing the number of sub-meshes.
3. A program for causing a computer to function as a mesh decoding device, wherein the mesh decoding device comprises an atlas data decoding unit configured to decode an atlas bit stream and generate and output first control information including boundary information; a basic mesh decoding unit configured to input a basic mesh bit stream and generate and output a basic mesh and second control information; A subdivision unit configured to output a subdivided mesh and a subdivided vertex normal using the first control information, the second control information, and the basic mesh as inputs; A mesh decoding unit configured to generate a decoded mesh using the first control information, a displacement amount, the subdivided mesh, and the subdivided vertex normal as inputs; A boundary correction unit configured to input the first control information and the decoded mesh and output a boundary-corrected decoded mesh, and The atlas data decoding unit decodes such that the number of sub-meshes is 1 when the single mesh control signal is "1", and otherwise decodes by adding "2" to the control signal representing the number of sub-meshes. A program characterized by that.
Citation Information
Cited By
Adhesive bond between a thermoplastic material and an elastomer composition
US12448552B2