Mesh decoding device, mesh decoding method, and program

Abstract

The present invention correctly performs a decoding process even when a sub-mesh count and boundary vertex count defined by boundary correction syntax are less than a sub-mesh count and boundary vertex count decoded otherwise. A mesh decoding device 200 according to the present invention comprises a boundary correction unit 208 that sets a correction distance threshold value for all boundary vertices.

Description

Mesh decoding device, mesh decoding method and program

[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program.

[0002] In Non-Patent Document 1, meshes are decoded by dividing them into basic meshes that represent rough information and displacements that represent detailed information, and the displacements are decoded using a video codec.

[0003] Khaled Mammou, Jungsun Kim, Alexis Tourapis, Dimitri Podborski, Krasimir Kolarov, “[V-CG] Apple's Dynamic Mesh Coding CfP Response,” ISO / IEC JTC 1 / SC 29 / WG 7 m59281, April 2022. Google Draco, accessed May 26, 2022 [Online], https: / / google.github.io / draco

[0004] However, in the conventional technology, when the number of sub-meshes and the number of boundary vertices defined in the boundary correction syntax are less than the number of sub-meshes and the number of boundary vertices decoded by other means, the decoding process cannot be performed correctly. Therefore, the present invention has been made in consideration of the above-mentioned problem, and it is an object of the present invention to provide a mesh decoding device, a mesh decoding method, and a program that can perform the decoding process correctly even when the number of sub-meshes and the number of boundary vertices defined in the boundary correction syntax are less than the number of sub-meshes and the number of boundary vertices decoded by other means.

[0005] A first feature of the present invention is summarized as a mesh decoding device including a boundary correction unit that sets a correction distance threshold for all boundary vertices.

[0006] A second aspect of the present invention is a mesh decoding method, which includes a step of setting a corrected distance threshold for all boundary vertices.

[0007] A third feature of the present invention is a program that causes a computer to function as a mesh decoding device, wherein the mesh decoding device includes a boundary correction unit that sets a correction distance threshold for all boundary vertices.

[0008] According to the present invention, it is possible to provide a mesh decoding device, a mesh decoding method, and a program that can perform decoding processing correctly even when the number of sub-meshes and the number of boundary vertices defined in the boundary correction syntax are less than the number of sub-meshes and the number of boundary vertices decoded elsewhere.

[0009] FIG. 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment. FIG. 2 is a diagram showing an example of functional blocks of a mesh decoding device 200 according to an embodiment. FIG. 3 is a diagram showing an example of a Zippering syntax configuration. FIG. 4 is a flowchart showing a method for decoding a control signal for distance-based boundary correction when zp_method_type is 1. FIG. 5 is a diagram showing an example of a basic mesh. FIG. 6 is a diagram for explaining an example of a method for decoding a subdivision mesh in the subdivision unit 203. FIG. 7 is a flowchart for explaining an example of a method for calculating a correction distance threshold in the boundary correction unit 208. FIG. 8 is a flowchart showing an example of processing for calculating boundary pairs of subdivision vertices in the boundary correction unit 208. FIG. 9 is a flowchart showing an example of boundary correction. FIG. 10 is a diagram schematically showing a method of boundary correction.

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0011] First Embodiment A mesh processing system 1 according to this embodiment will be described below with reference to FIGS.

[0012] 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to this embodiment. As shown in FIG. 1, the mesh processing system 1 includes a mesh encoding device 100 and a mesh decoding device 200.

[0013] FIG. 2 is a diagram showing an example of functional blocks of a mesh decoding device 200 according to this embodiment.

[0014] As shown in Figure 2, the mesh decoding device 200 has a demultiplexing unit 201, a basic mesh decoding unit 202, a subdivision unit 203, a mesh decoding unit 204, a displacement amount decoding unit 205, a video decoding unit 206, an atlas data decoding unit 207, a boundary correction unit 208, and an SEI message decoding unit 209.

[0015] The demultiplexing unit 201 is configured to separate the multiplexed bitstream into an atlas bitstream, a base mesh bitstream, a displacement bitstream, a texture bitstream, and an SEI bitstream.

[0016] The base mesh decoder 202 is configured to decode the base mesh bitstream, generate and output a base mesh.

[0017] The base mesh is a mesh that has been simplified by reducing the number of faces compared to the mesh before encoding, and is used to represent the basic shape of the mesh.

[0018] Furthermore, the basic mesh is made up of one submesh or multiple submeshes.

[0019] The subdivision unit 203 is configured to generate and output a subdivision mesh, subdivision vertex normals, and subdivision information from the basic mesh decoded by the basic mesh decoding unit 202 using the subdivision method indicated by the atlas control information.

[0020] Here, the basic mesh, the added subdivision vertices, and the connection information therebetween are collectively referred to as a "subdivision mesh."

[0021] The subdivision vertex normal is a normal defined for each subdivision vertex, and the subdivision information is information indicating which vertices are subdivided to generate which vertices.

[0022] The mesh decoding unit 204 is configured to generate and output a decoded mesh using the subdivision mesh and subdivision vertex normals generated by the subdivision unit 203 and the displacement amount decoded by the displacement amount decoding unit 205.

[0023] The displacement amount decoding unit 205 is configured to decode the displacement amount bitstream to generate and output the displacement amount.

[0024] The video decoding unit 206 is configured to decode the texture using a video codec and output the decoded texture.

[0025] The atlas data decoder 207 is configured to decode the atlas bitstream and output the atlas control information.

[0026] The boundary correction unit 208 is configured to correct the submesh boundaries of the input decoded mesh and output the boundary-corrected decoded mesh.

[0027] The SEI message decoder 209 is configured to decode the SEI message bitstream and output the corrected submesh number, the corrected submesh boundary vertex number, and the boundary pair.

[0028] Here, the number of correction submeshes is the number of submeshes to be corrected, the number of correction submesh boundary vertices is the number of vertices to be corrected in the submesh, and the boundary pair is a list of vertices to be corrected.

[0029] (SEI Message Decoding Unit 209) The SEI message decoding unit 209 is configured to decode the SEI bitstream and output the corrected submesh number, the corrected submesh boundary vertex number, and the boundary pair.

[0030] The control information decoded by the SEI message decoder 209 will be described with reference to FIG.

[0031] First, the SEI bitstream includes a correction syntax Zippering, which is a collection of control information related to sub-mesh boundary correction.

[0032] Zippering may be defined for each mesh data, or may be defined at a finer granularity such as frame, patch, or tile.

[0033] The structure of the syntax decoded by the SEI message decoder 209 and the decoding method thereof will be described below with reference to FIG.

[0034] 3 is a diagram showing an example of a syntax configuration of Zippering. Here, if the syntax functions are similar, syntax names different from those shown in FIG. 3 may be used.

[0035] In the Zippering syntax structure shown in Figure 3, the Description column indicates how each syntax is coded. Also, ue(v) indicates an unsigned zeroth-order exponential-Golomb code, and u(n) indicates a fixed code length of n bits.

[0036] The Description is merely an example, and coding methods other than the Description of the diagram, such as u(n), ue(v), and ae(v), may be selected.

[0037] Zippering includes a control signal zp_persistence_flag that indicates whether to use the correction syntax of the previous frame as is.

[0038] Zippering includes a control signal zp_send_distance_per_submesh that indicates whether to send a correction distance threshold for each submesh.

[0039] Zippering includes a control signal zp_number_of_submeshes_minus1 that indicates the number of corrected submeshes.

[0040] Zippering includes a control signal zp_send_distance_per_border_point[k][p] that indicates whether to send a correction distance threshold for each vertex.

[0041] Zippering includes a control signal zp_number_of_border_points that represents the number of border vertices within the same submesh that defines the correction distance threshold.

[0042] Zippering includes a control signal zp_border_point_distance that represents a correction distance threshold for each vertex.

[0043] Zippering includes a control signal zp_max_match_distance_per_submesh that indicates the correction distance threshold for each submesh.

[0044] Zippering includes a control signal zp_max_match_distance that indicates a correction distance threshold for each identical frame.

[0045] A specific procedure for decoding a control signal for distance-based boundary correction will be described below with reference to Fig. 4. Fig. 4 is a flowchart showing a method for decoding a control signal for distance-based boundary correction when zp_method_type is 1.

[0046] As shown in FIG. 4, in step S209A-1, the SEI message decoding unit 209 determines whether zp_send_distance_per_submesh=1 is satisfied.

[0047] If it is determined that the condition is met, the process proceeds to step S209A-3, and if it is determined that the condition is not met, the process proceeds to step S209A-2.

[0048] In step S209A-2, the SEI message decoding unit 209 decodes zp_max_match_distance and ends this processing.

[0049] In step S209A-3, the SEI message decoding unit 209 initializes i to 0, decodes zp_number_of_submeshes_minus1, and the process proceeds to step S209A-4.

[0050] In step S209A-4, the SEI message decoding unit 209 determines whether i<zp_number_of_submeshes_minus1+1 is satisfied.

[0051] If it is determined that the condition is met, the process proceeds to step S209A-5, and if it is determined that the condition is not met, the process ends.

[0052] In step S209A-5, the SEI message decoding unit 209 decodes zp_send_distance_per_border_point, and the process proceeds to step S209A-6.

[0053] In step S209A-6, the SEI message decoding unit 209 determines whether zp_send_distance_per_border_point=1 is satisfied.

[0054] If it is determined that the condition is met, the process proceeds to step S209A-8, and if it is determined that the condition is not met, the process proceeds to step S209A-7.

[0055] In step S209A-7, the SEI message decoding unit 209 decodes zp_max_match_distance_per_submesh, adds 1 to i, and the processing proceeds to step S209A-4.

[0056] In step S209A-8, the SEI message decoding unit 209 initializes j to 0, decodes zp_number_of_border_points, and the process proceeds to step S209A-9.

[0057] In step S209A-9, the SEI message decoding unit 209 determines whether the condition j<zp_number_of_border_points is satisfied.

[0058] If it is determined that the condition is met, the process proceeds to step S209A-10; if it is determined that the condition is not met, the SEI message decoding unit 209 adds 1 to i, and the process proceeds to step S209A-6.

[0059] In step S209A-10, the SEI message decoding unit 209 decodes zp_max_match_distance_per_point, adds 1 to j, and the processing proceeds to step S209A-9.

[0060] Zippering includes a control signal zp_number_of_border_points_minus1 that represents the correct submesh boundary vertex number.

[0061] The corrected submesh boundary vertex number numBorderPoints can be decoded by adding 1 to zp_number_of_border_points_minus1.

[0062] In this case, numBorderPoints is defined to be equal to or less than the number of vertices of the sub-meshes decoded by other decoding units such as the atlas data decoding unit 207, the basic mesh decoding unit 202, and the subdivision unit 203.

[0063] In addition, if numBorderPoints exceeds the number of vertices decoded by other decoding units such as the atlas data decoding unit 207, the basic mesh decoding unit 202, or the subdivision unit 203, numBorderPoints may be configured to be corrected to be equal to or less than the number of vertices of the submesh decoded by the other decoding units.

[0064] For example, the corrected number of submesh boundary vertices may be corrected to the same value as the number of vertices of the submeshes decoded by other decoding units.

[0065] Zippering includes a control signal zp_subdivision_vertex_border_calculation_flag that indicates whether the boundary pairs of subdivision vertices are calculated based on the subdivision information.

[0066] Zippering contains the submesh index of the boundary pair zp_border_point_match_submesh_index[k][p][b].

[0067] Zippering contains the border vertex indices zp_border_point_match_border_point_inex[k][p][b] of the border pair.

[0068] Here, when zp_border_point_match_submesh_index[k][p][b] = q and zp_border_point_match_border_point_inex[k][p][b] = c, it indicates that vertex v(p, b) and vertex v(q, c) are a boundary pair.

[0069] In this case, v(i, j) represents the vertex with boundary vertex index j on submesh i, where i and j are integers equal to or greater than 0.

[0070] (Basic Mesh Decoding Unit 202) The basic mesh decoding unit 202 is configured to decode the basic mesh bit stream, generate a basic mesh, and output it.

[0071] Here, the basic mesh is composed of one submesh or multiple submeshes.

[0072] A submesh is made up of a plurality of vertices in three-dimensional space and edges connecting these vertices.

[0073] As shown in FIG. 5, the basic mesh is formed by combining basic faces each represented by three vertices.

[0074] The base mesh decoder 202 may be configured to decode the base mesh bitstream using, for example, Draco as described in Non-Patent Document 2 or other known mesh decoding techniques.

[0075] Multiple sub-meshes may be configured to be independently decoded using Draco or other known mesh decoding techniques.

[0076] (Subdivision Unit 203) A method for decoding a subdivision mesh in the subdivision unit 203 will be described with reference to FIG.

[0077] The subdivision unit 203 is configured to output a subdivision mesh, subdivision vertex normals, and subdivision information based on the base mesh.

[0078] 6 shows an example of a subdivision mesh that has been subdivided once using the mid-edge division method. In the mid-edge division method, a subdivision mesh is generated by dividing the midpoints of the edges that make up the mesh.

[0079] By repeating the above process, a finer refined mesh can be generated.

[0080] The subdivision unit 203 is configured to store, at the time of generating a subdivision vertex, which vertex the subdivision vertex is located between, and to output this information to the boundary correction unit 208 as subdivision information.

[0081] The subdivision unit 203 may be configured not to record the above information when zp_subdivision_vertex_border_calculation_flag is 1.

[0082] The subdivision unit 203 may be configured to perform decoding independently for each submesh.

[0083] (Displacement Amount Decoding Unit 205) The displacement amount decoding unit 205 is configured to decode the displacement amount bitstream to generate and output a displacement amount.

[0084] The displacement may be defined so that it can be decoded independently for each sub-mesh.

[0085] (Mesh Decoding Unit 204) The mesh decoding unit 204 is configured to output a decoded mesh based on the input atlas control information, subdivision mesh, and displacement amount.

[0086] The mesh decoding unit 204 adds the displacement amounts to the vertices of the input subdivision mesh, and decodes the decoded mesh.

[0087] At this time, the mesh decoding unit 204 may output a plurality of decoded meshes for each submesh based on the displacement amount corresponding to the submesh and the subdivision mesh.

[0088] In addition, the mesh decoding unit 204 may be configured to set a coordinate system for the displacement amount based on a control signal representing a coordinate system for the displacement amount defined for each submesh, and calculate and output the decoded mesh by adding the displacement amount to the subdivision vertex on the set coordinate system.

[0089] (Boundary Correction Unit 208) A method for decoding a boundary-corrected decoded mesh in the boundary correction unit 208 will be described with reference to FIGS.

[0090] The boundary correction unit 208 performs distance-based boundary correction when zp_method_type=1, and performs index-based boundary correction when zp_method_type=2.

[0091] First, distance-based boundary correction is described.

[0092] In distance-based boundary correction, boundary vertices v that exist on all sub-meshes are ｉ vertex v that is the boundary vertex that is the shortest distance to the ｊ and divide the vertices into boundary pairs (v ｉ , v ｊ ) is calculated as

[0093] A method for determining the corrected distance threshold will be described below with reference to FIG.

[0094] 7, in step S209B-1, the boundary correction unit 208 determines whether zp_send_distance_per_submesh = 1. If Yes, the boundary correction unit 208 proceeds to step S209B-3, and if No, the boundary correction unit 208 proceeds to step S209B-2.

[0095] In step S209B-2, the boundary correction unit 208 sets the correction threshold for all submesh indices smIdx and all boundary vertex indices bIdx to zipBoundaryDistance[smIdx][bIdx]=zp_max_match_distance, and then ends the processing.

[0096] In step S209B-3, the boundary correction unit 208 sets smIdx=0, and the process proceeds to step S209B-4.

[0097] In step S209B-4, the boundary correction unit 208 determines whether smIdx<numSubmesh is satisfied.

[0098] Here, numSubmesh is the number of submeshes defined in the Zippering syntax.

[0099] If the condition is satisfied, the boundary correction unit 208 proceeds to step S209B-5, and if the condition is not satisfied, the boundary correction unit 208 proceeds to step S209B-7.

[0100] In step S209B-5, the boundary correction unit 208 determines whether zp_send_distance_per_border_point=1.

[0101] If the condition is satisfied, the boundary correction unit 208 proceeds to step S209B-6, and if the condition is not satisfied, the boundary correction unit 208 proceeds to step S209B-9.

[0102] In step S209B-6, the boundary correction unit 208 sets the correction threshold for all boundary vertex indices bIdx of the boundary vertices included in the submesh whose submesh index is smIdx to zipBoundaryDistance[smIdx][bIdx] = zp_max_match_distance_per_submesh[smIdx], adds 1 to smIdx, and proceeds to step S209B-4.

[0103] In step S209B-7, the boundary correction unit 208 determines whether numSubmesh<submeshCount is satisfied.

[0104] Here, submeshCount is the number of submeshes in the decoded mesh or the number of submeshes decoded by other decoding units.

[0105] If the condition is met, the boundary correction unit 208 proceeds to step S209B-8, and if the condition is not met, the boundary correction unit 208 ends the process.

[0106] In step S209B-8, the boundary correction unit 208 sets the correction threshold for the submesh indices smIdx equal to or greater than numSubmesh and all boundary vertex indices bIdx to zipBoundaryDistance[smIdx][bIdx]=zp_max_match_distance, and then ends the processing.

[0107] In step S209B-9, the boundary correction unit 208 initializes bIdx to 0, and the process proceeds to step S209B-10.

[0108] In step S209B-10, the boundary correction unit 208 determines whether bIdx<bCount is satisfied.

[0109] Here, bCount is the number of boundary vertices defined in the Zippering syntax.

[0110] If the boundary correction unit 208 determines that the condition is met, the process proceeds to step S209B-11. If the condition is not met, the process proceeds to step S209B-12.

[0111] In step S209B-11, the boundary correction unit 208 sets zipBoundaryDistance[smIdx][bIdx]=zp_max_match_distance_per_point[smIdx][bIdx] for smIdx and bIdx, adds 1 to bIdx, and proceeds to step S209B-10.

[0112] In step S209B-12, the boundary correction unit 208 determines whether bCount<numBorderCount.

[0113] Here, numBorderCount is the number of boundary vertices of the decoded mesh or the number of boundary vertices decoded by another decoder.

[0114] If the boundary correction unit 208 determines that the condition is met, the process proceeds to step S209B-13. If the condition is not met, the process proceeds to step S209B-4.

[0115] In step S209B-13, the boundary correction unit 208 sets the correction threshold for the submesh index smIdx and all boundary vertex indices bIdx equal to or greater than bCount as zipBoundaryDistance[smIdx][bIdx]=zp_max_match_distance_per_submesh, adds 1 to smIdx, and proceeds to step S209B-4.

[0116] Additionally, the boundary correction unit 208 may be configured to initially initialize all correction distance thresholds with a constant value.

[0117] By defining correction thresholds for all thresholds, the process of finding boundary pairs can be terminated without an error and the mesh can be decoded correctly.

[0118] In distance-based boundary correction, the boundary correction unit 208 calculates the boundary vertices v that exist on all sub-meshes. ｉ Explore and ｉ vertex v that is the boundary vertex that is the shortest distance from the ｊ and divide the vertices into boundary pairs (v ｉ , v ｊ ) is calculated as

[0119] At this time, v ｉ The shortest distance is v ｊ and both vertices have been searched, the boundary correction unit 208 ｉ and the boundary vertex with the second closest distance is the boundary pair (v ｉ , v ｋ ), or by allowing overlapping vertices, we can define the boundary pair as (v ｉ , v ｊ ) can also be used.

[0120] The boundary correction unit 208 corrects the vertices v included in the boundary pair. ｉ , v ｊ The boundary vertices may be corrected using a statistical value such as the average value of the coordinates included in the boundary pair.

[0121] The boundary correction unit 208 corrects the vertices v included in the boundary pair. ｉ , v ｊ When the vertices are corrected and uncorrected, the vertex v ｉ , v ｊ The coordinates of the points may be corrected to either coordinate value.

[0122] The boundary correction unit 208 may also be configured to determine vertices that are closest to each other up to the zp_coordination_border_point_count-th distance as a boundary pair, and correct the boundary vertex using the average value of these vertices during correction.

[0123] This makes it possible to correct multiple vertices to vertices with the same coordinates, and to decode a mesh that is completely free of holes.

[0124] Second, an index-based boundary correction scheme is described.

[0125] The boundary correction unit 208 is configured to correct the submesh boundaries of the input decoded mesh and output the boundary-corrected decoded mesh.

[0126] When zp_persistence_flag is 1, the boundary correction unit 208 uses the sub-mesh boundary information of the reference frame as it is as the sub-mesh boundary information.

[0127] The boundary correction unit 208 may be configured to skip boundary correction when the frame to be corrected is not an interframe.

[0128] The boundary correction unit 208 may be configured to skip boundary correction and output the decoded mesh as is when the sub-mesh index or vertex index included in the boundary information does not exist.

[0129] Furthermore, the boundary correction unit 208 may be configured to skip correction of only the boundary pairs of sub-mesh indexes or vertex indexes when these indexes are not included in the boundary information.

[0130] When zp_subdivision_vertex_border_calculation_flag is 0, the SEI message decoder 209 only finds boundary pairs of submesh vertices. Therefore, in order to perform correction on subdivision vertices as well, it is necessary to find boundary pairs of subdivision vertices.

[0131] However, when zp_subdivision_vertex_border_calculation_flag is 0, the boundary correction unit 208 may perform correction processing only on the boundary pair determined by the SEI message decoding unit 209 .

[0132] 8 is a flowchart showing the process of calculating boundary pairs of subdivision vertices in the boundary correction unit 208. A method of calculating boundary pairs of subdivision vertices will be described below with reference to FIG.

[0133] In step S208A-1, the boundary correction unit 208 assigns 0 to each of the variables i and j, and then proceeds to step S208A-2.

[0134] In step S208A-2, the boundary correction unit 208 determines whether the variable i satisfies i<numSubmeshes.

[0135] If the condition is satisfied, the boundary correction unit 208 proceeds to step S208A-3, and if the condition is not satisfied, the boundary correction unit 208 ends the process.

[0136] In step S208A-3, the boundary correction unit 208 determines whether the variable j satisfies j<numBorderPoints.

[0137] If the condition is met, the boundary correction unit 208 proceeds to step S208A-4, and if the condition is not met, the boundary correction unit 208 proceeds to step S208A-9.

[0138] In step S208A-4, the boundary correction unit 208 determines which side was sub-divided to generate the vertex v(i, j), and then proceeds to step S208A-5.

[0139] If vertex v(i, j) is a vertex generated by subdividing edge e(v(i0, j0), v(i0, j1)), the vertices to be found are v(i0, j0) and v(i0, j1).

[0140] Here, the edge e(v(i0, j0), v(i0, j1)) represents an edge whose endpoints are v(i0, j0) and v(i0, j1).

[0141] In step S208A-5, the boundary correction unit 208 determines the vertices that are paired with the vertices that make up the calculated side, and then the process proceeds to step S208A-6.

[0142] Assume that the vertices calculated in step S208A-4 are v(i0, j0) and v(i0, j1). If v(i0, j0) and v(i1, j0) are a vertex pair and v(i0, j1) and v(i1, j1) are a vertex pair, the vertices to be found are v(i1, j0) and v(i1, j1).

[0143] In step S208A-6, the boundary correction unit 208 determines vertices to be generated by subdividing the side whose endpoints are the vertices v(i1, j0) and v(i1, j1) calculated in step S208A-5, and then proceeds to step S208A-7.

[0144] When the edge e(v(i1, j0), v(i1, j1)) is subdivided to generate the vertex v(i2, j2), the vertex obtained is v(i2, j2).

[0145] In step S208A-7, the boundary correction unit 208 adds the vertex found in step S208A-4 and the vertex found in step S208A-6 to the boundary pair, and then proceeds to step S208A-8.

[0146] In step S208A-8, the boundary correction unit 208 adds 1 to the variable j, and the process proceeds to step S208A-4.

[0147] In step S208A-9, the boundary correction unit 208 adds 1 to the variable i, and the process proceeds to step S208A-2.

[0148] In the above example, the boundary correction unit 208 calculates boundary pairs based on subdivision information, but may be configured to decode only the boundary pairs indicated by the submesh indexes of the boundary pairs and the boundary vertex indexes of the boundary pairs stored in the Zippering syntax as boundary pairs.

[0149] FIG. 9 is a flowchart showing an example of boundary correction.

[0150] As shown in FIG. 9, in step S209C-1, the boundary correction unit 208 initializes i to 0, and the process proceeds to step S209C-2.

[0151] In step S209C-2, the boundary correction unit 208 determines whether i<the number of vertices is satisfied.

[0152] If the condition is met, the boundary correction unit 208 proceeds to step S209C-3, and if the condition is not met, the boundary correction unit 208 ends the process.

[0153] In step S209C-3, the boundary correction unit 208 determines whether the vertex v(i0, j0) indicated by the index i is a boundary vertex.

[0154] If the boundary correction unit 208 determines that the condition is met, the process proceeds to step S209C-4. If the condition is not met, the process proceeds to step S209C-6.

[0155] In step S209C-5, the boundary correction unit 208 corrects the vertex v(i0, j0) and the vertices that belong to the same boundary pair as the vertex v(i0, j0).

[0156] Next, the boundary correction method in step S209C-5 will be described with reference to FIG.

[0157] FIG. 10 is a diagram showing a schematic representation of the boundary correction method, where arrows indicate that vertices form boundary pairs.

[0158] Here, in boundary correction, the boundary correction unit 208 applies correction processing to the coordinates of vertices having boundary pairs.

[0159] When applying correction processing to the boundary pair of vertices v(i0, j0) and v(i1, j0), the boundary correction unit 208 calculates the average value of the coordinates of v(i0, j0) and v(i1, j0) and sets the two vertices to the calculated average value.

[0160] Furthermore, if vertex v(i1, j0) that belongs to the same boundary pair as vertex v(i0, j0) has already been corrected, the boundary correction unit 208 may correct the two vertices using the average value of vertex v(i0, j0) and vertex v(i1, j0), or may correct the two vertices to the coordinates of either vertex.

[0161] The boundary correction unit 208 may be configured to correct the boundary vertices v(i0, j0) and v(i1, j0) included in the boundary pair using a statistical value such as the average value of the coordinates included in the boundary pair.

[0162] In addition, when the vertices v(i0, j0) and v(i1, j0) included in the boundary pair are corrected and uncorrected vertices, the boundary correction unit 208 may correct the coordinates of the vertices v(i0, j0) and v(i1, j0) to either of the coordinate values.

[0163] Furthermore, when the number of vertices in a boundary pair is three or more, the boundary correction unit 208 may correct the vertices belonging to the boundary pair by using the average value of these vertices.

[0164] This makes it possible to correct multiple vertices to vertices with the same coordinates, and to decode a mesh that is completely free of holes.

[0165] Here, when the submesh index exceeds the number of submeshes defined in other bitstreams such as the atlas bitstream or the base mesh bitstream, the boundary correction unit 208 may not calculate the correction process for the corresponding boundary pair.

[0166] Furthermore, when the boundary pair index exceeds the number of vertices of a submesh defined in another bitstream such as an atlas bitstream, the boundary correction unit 208 may not calculate the correction process for the corresponding boundary pair.

[0167] The boundary correction unit 208 repeats the above process for the boundary vertices of all boundary pairs to correct the vertex coordinates.

[0168] This is expected to have the effect of filling holes even if they exist before the correction process.

[0169] According to this embodiment, when a correction distance threshold is sent for each vertex, only the correction distance threshold for each vertex is decoded, and when a correction distance threshold is sent for each submesh, only the correction distance threshold for each submesh is decoded, thereby reducing the amount of coding.

[0170] Furthermore, according to this embodiment, by setting some kind of correction distance threshold for all boundary vertices, it is possible to prevent the decoding process from ending with an error.

[0171] Furthermore, according to this embodiment, by correcting multiple vertices to vertices with the same coordinates, it is possible to decode a mesh that is completely free of holes.

[0172] The mesh encoding device 100 and mesh decoding device 200 described above may be realized as a program that causes a computer to execute each function (each step).

[0173] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation."

[0174] REFERENCE SIGNS LIST 1... Mesh processing system 100... Mesh encoding device 200... Mesh decoding device 201... Demultiplexing unit 202... Basic mesh decoding unit 203... Subdivision unit 204... Mesh decoding unit 205... Displacement amount decoding unit 206... Video decoding unit 207... Atlas data decoding unit 208... Boundary correction unit 209... SEI message decoding unit

Claims

1. A mesh decoding device comprising a boundary correction unit that sets a correction distance threshold for all boundary vertices.

2. The mesh decoding device described in claim 1, characterized in that when the number of submeshes defined in the correction syntax is less than the number of submeshes of the decoded mesh, the boundary correction unit sets the correction distance threshold for each boundary vertex present in submeshes that are equal to or greater than the number of submeshes defined in the correction syntax to the correction distance threshold for each frame.

3. The mesh decoding device described in claim 1, characterized in that when the number of boundary vertices defined in the correction syntax is less than the number of boundary vertices of the decoded mesh, the boundary correction unit sets the correction distance threshold for each boundary vertex that is equal to or greater than the number of boundary vertices defined in the correction syntax as the correction distance threshold for each frame or submesh.

4. The mesh decoding device according to claim 1, wherein the boundary correction unit sets the same correction distance threshold for all boundary vertices.

5. A mesh decoding method comprising the step of setting a corrected distance threshold for all boundary vertices.

6. A program that causes a computer to function as a mesh decoding device, the mesh decoding device comprising a boundary correction unit that sets a correction distance threshold for all boundary vertices.

Images