Mesh decoding device, mesh decoding method and program

Abstract

To efficiently achieve coding of a displacement magnitude.SOLUTION: A mesh decoding device 200 includes: a basic face division part 203A5 for subdividing a basic mesh and outputting an initial subdivision mesh; a subdivision vertex normal calculation part 203C for calculating a normal for each vertex constituting the initial subdivision mesh; and a subdivision mesh adjustment part 203B for moving the positions of the vertexes constituting the surface of the initial subdivision mesh, then dividing a subdivision surface, and thereby adjusting a subdivision mesh.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a mesh decoding device, a mesh decoding method, and a program. [Background technology]

[0002] In Non-Patent Document 1, a mesh is decoded by dividing it into a rough basic mesh and a detailed displacement amount, and the displacement amount is decoded by a video codec. The displacement amount is calculated based on a normal defined for each vertex, and the direction of the normal varies for each vertex.

[0003] Furthermore, in Non-Patent Document 1, a decoding method is defined for each patch according to parameters set for each patch. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Khaled Mammou, Jungsun Kim, Alexis Tourapis, Dimitri Podborski, Krasimir Kolarov, “[V-CG] Apple's Dynamic Mesh Coding CfP Response,” ISO / IEC JTC 1 / SC 29 / WG 7 m5928, April 2022. Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Non-Patent Document 1, the direction of the normals differs for each vertex, which causes a problem that the displacement amount cannot be efficiently compressed. Therefore, the present invention has been made in consideration of the above-mentioned problem, and aims to provide a mesh decoding device, a mesh encoding device, a mesh decoding method, and a program that can efficiently encode the displacement amount. [Means for solving the problem]

[0006] A first feature of the present invention is summarized as a mesh decoding device comprising: a basic surface division unit configured to subdivide a basic mesh and output an initial subdivision mesh; a subdivision vertex normal calculation unit configured to calculate normals for each vertex constituting the initial subdivision mesh; and a subdivision mesh adjustment unit configured to adjust the subdivision mesh by dividing the subdivision surface after moving positions of vertices constituting the faces of the initial subdivision mesh.

[0007] A second feature of the present invention is summarized as a mesh decoding method comprising the steps of: subdividing a basic mesh to output an initial subdivision mesh; calculating normals for each vertex constituting the initial subdivision mesh; and adjusting the subdivision mesh by moving positions of vertices constituting faces of the initial subdivision mesh and then dividing the subdivision faces.

[0008] A third feature of the present invention is summarized as a program for causing a computer to function as a mesh decoding device, the mesh decoding device comprising: a basic surface division unit configured to subdivide a basic mesh and output an initial subdivision mesh; a subdivision vertex normal calculation unit configured to calculate normals for each vertex constituting the initial subdivision mesh; and a subdivision mesh adjustment unit configured to adjust the subdivision mesh by dividing the subdivision surface after moving positions of vertices constituting the faces of the initial subdivision mesh. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a mesh decoding device, a mesh encoding device, a mesh decoding method and a program that can efficiently encode the amount of displacement. [Brief explanation of the drawings]

[0010] [Figure 1]FIG. 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks of a mesh decoding device 200 according to an embodiment. [Figure 3] FIG. 3 is a diagram showing an example of the syntax structure of an atlas bitstream. [Figure 4] FIG. 4 is a diagram illustrating an example of a syntax configuration of a PDU. [Figure 5] FIG. 5 is a diagram illustrating an example of a syntax configuration of an FPS. [Figure 6A] FIG. 6A is a diagram showing an example of a basic mesh and a subdivision mesh. [Figure 6B] FIG. 6B is a diagram showing an example of a basic mesh and a subdivision mesh. [Figure 7] FIG. 7 is a diagram showing an example of functional blocks of the subdivision unit 203 of the mesh decoding device 200 according to an embodiment. [Figure 8] FIG. 8 is a diagram showing an example of functional blocks of the basic mesh subdivision unit 203A of the subdivision unit 203 of the mesh decoding device 200 according to an embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a method for dividing a basic surface by the basic surface dividing unit 203A5 of the basic mesh subdivision unit 203A of the subdivision unit 203 of the mesh decoding device 200 according to an embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of the operation of the basic mesh subdivision unit 203A of the subdivision unit 203 of the mesh decoding device 200 according to an embodiment. [Figure 11] FIG. 11 is a diagram showing an example of functional blocks of the subdivision mesh adjustment unit 203B of the subdivision unit 203 of the mesh decoding device 200 according to an embodiment. [Figure 12] FIG. 12 is a diagram showing an example of a case in which an edge division point on a basic surface ABC is moved by the edge division point moving unit 701 of the subdivision mesh adjustment unit 203B of the subdivision unit 203 of the mesh decoding device 200 according to one embodiment. [Figure 13]FIG. 13 is a diagram showing an example of a case in which subdivision is performed again on a subdivision surface X within a basic surface by the subdivision surface division unit 702 of the subdivision mesh adjustment unit 203B of the subdivision unit 203 of the mesh decoding device 200 according to one embodiment. [Figure 14] FIG. 14 is a diagram showing an example of a case in which all subdivision surfaces are re-subdivided by the subdivision surface division unit 702 of the subdivision mesh adjustment unit 203B of the subdivision unit 203 of the mesh decoding device 200 according to one embodiment. [Figure 15] FIG. 15 is a flowchart showing an example of the operation of the subdivision vertex normal calculation unit 203C of the subdivision unit 203 of the mesh decoding device 200 according to an embodiment. [Figure 16A] FIG. 16A is a diagram for explaining an example of a method for calculating subdivision vertex normals by the subdivision vertex normal calculation unit 203C. [Figure 16B] FIG. 16B is a diagram for explaining an example of a method for calculating subdivision vertex normals by the subdivision vertex normal calculation unit 203C. [Figure 17] FIG. 17 is a diagram showing an example of functional blocks of the displacement amount decoding unit 206 of the mesh decoding device 200 according to an embodiment. [Figure 18] FIG. 18 is a flowchart showing an example of the operation of the inverse quantization unit 206J of the displacement decoding unit 206 of the mesh decoding device 200 according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0012] First Embodiment Hereinafter, a mesh processing system 1 according to this embodiment will be described with reference to FIGS.

[0013] 1 is a diagram showing an example of the configuration of a mesh processing system 1 according to this embodiment. As shown in FIG. 1, the mesh processing system 1 includes a mesh encoding device 100 and a mesh decoding device 200.

[0014] FIG. 2 is a diagram showing an example of functional blocks of the mesh decoding device 200 according to this embodiment.

[0015] As shown in Figure 2, the mesh decoding device 200 includes a demultiplexing unit 201, a basic mesh decoding unit 202, a subdivision unit 203, a mesh decoding unit 204, a patch integration unit 205, a displacement amount decoding unit 206, a video decoding unit 207, and an atlas data decoding unit 208.

[0016] The demultiplexing unit 201 is configured to separate the multiplexed bit stream into a basic mesh bit stream, a displacement amount bit stream, a texture bit stream, and an atlas bit stream.

[0017] The base mesh decoding unit 202 is configured to decode the base mesh bitstream to generate and output a base mesh.

[0018] The subdivision unit 203 is configured to generate and output the added subdivision vertices and their connection information from the basic mesh decoded by the basic mesh decoding unit 202 using the subdivision method indicated by the patch information.

[0019] Here, the basic mesh, the added subdivision vertices, and the connection information therebetween are collectively referred to as a "subdivision mesh."

[0020] The mesh decoding unit 204 is configured to generate and output a decoded mesh using the subdivision mesh generated by the subdivision unit 203 and the displacement amount decoded by the displacement amount decoding unit 206 .

[0021] The patch integration unit 205 is configured to integrate the decoded mesh generated by the mesh decoding unit 206 by the number of patches and output the integrated mesh.

[0022] The displacement decoding unit 206 is configured to decode the displacement bitstream to generate and output a displacement.

[0023] The video decoding unit 207 is configured to decode and output textures by video coding. For example, the video decoding unit 207 may use HEVC, which is disclosed in Reference 1, "Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 2: High efficiency video coding, ISO / IEC 23008-2:2013."

[0024] The atlas data decoder 208 is configured to decode the atlas bitstream and output frame information and patch information.

[0025] (Atlas data decoding unit 208) As described above, the atlas data decoder 208 is configured to decode the atlas bitstream and output frame information and patch information.

[0026] The frame information and patch information decoded by the atlas data decoding unit 208 will be described below with reference to FIGS.

[0027] FIG. 3 is a diagram showing an example of the syntax structure of an atlas bitstream.

[0028] As shown in FIG. 3, first, the atlas bitstream includes a PDU (Patch data unit), which is a collection of control information corresponding to a patch.

[0029] The patch information is a set of control signals defined for each patch obtained by decoding the PDU.

[0030] As described above, the atlas bitstream is configured such that each patch corresponds to one PDU.

[0031] As shown in FIG. 3, the frame parameter set (FPS) is also included, which is a set of control information corresponding to the frame.

[0032] The frame information is a set of control signals defined for each frame obtained by decoding the FPS.

[0033] The configuration in FIG. 3 is merely an example, and elements other than those described above may be added as components of the atlas bitstream, as long as each patch data is configured to correspond to a PDU.

[0034] For example, as shown in FIG. 3, the atlas bitstream may include a Sequence Parameter Set (SPS).

[0035] The syntax structure of the PDU will be described below with reference to FIG.

[0036] 4 is a diagram showing an example of a syntax configuration of a PDU. Here, if the syntax functions are similar, different syntax names may be used in the syntax shown in FIG.

[0037] In the PDU syntax structure shown in Figure 4, the Description column indicates how each syntax is coded. Also, ue(v) indicates an unsigned zero-order exponential-Golomb code, and u(n) indicates an n-bit flag.

[0038] The PDU contains a control signal mdu_triangle_count_minus1 that specifies the number of elementary planes included per patch.

[0039] The PDU includes a flag mdu_subdivision_enable_flag that specifies whether subdivision is performed for each patch.

[0040] The PDU may include a flag mdu_lod_change_flag that specifies whether or not to change the quantized value of the displacement amount for each subdivision level of the displacement amount for each patch.

[0041] The PDU may include a control signal (mdu_subdivision_method) that specifies the type of subdivision method for the base mesh for each patch.

[0042] For example, when mdu_subdivision_method=0, it may be defined that the initial refinement mesh is generated using the Mid-edge division method, and when mdu_subdivision_method=1, it may be defined that the initial refinement mesh is generated using the N2 division method.

[0043] The PDU may include a control signal (mdu_subdivision_num_method) that specifies the type of subdivision number generation method for each patch.

[0044] The PDU may include a control signal (mdu_subdivision_residuals) that specifies the error in the predicted value of the number of subdivisions for each patch.

[0045] The PDU may include a control signal (mdu_max_subdivision_iteration_count) that indicates the number of subdivision processes for each patch.

[0046] The PDU may contain a control signal (mdu_subdivision_flag) that specifies whether to recursively sub-divide the elementary plane for each index i (i=0,...,mdu_triangle_count_minus1) and j (j=0,...,mdu_subdivision_iteration_count).

[0047] The PDU may include a control signal (mdu_subdivision_num) that specifies the number of divisions of the basic surface in one subdivision process for each patch.

[0048] The PDU may include a control signal (mdu_normal_method) that specifies the type of calculation method for subdivision vertex normals for each patch.

[0049] The PDU may include a control signal (mdu_lod_qp) that specifies the quantization value of the displacement amount for each subdivision level for each patch.

[0050] The subdivision level is the number of subdivision processes at the current stage. For example, when the subdivision level is 1, it means that the subdivision process has been performed once on the basic surface.

[0051] The syntax structure of the FPS will be described below with reference to FIG.

[0052] 5 is a diagram showing an example of the syntax configuration of an FPS. Here, if the syntax functions are similar, different syntax names may be used in the syntax shown in FIG.

[0053] In the syntax configuration of the FPS shown in FIG. 5, the Description field is the same as that of the PDU.

[0054] The FPS may include a flag (mdu_border_adaption_flag) indicating whether or not subdivision adjustment is performed for each frame.

[0055] The FPS may include a control signal (mdu_border_adaption_method) that specifies the subdivision adjustment method for each frame.

[0056] The FPS may include a flag (mdu_vertex_adaption_flag) that indicates whether or not the edge division points are to be moved in the subdivision mesh adjustment unit for each frame.

[0057] (Subdivision section 203) As described above, the subdivision unit 203 is configured to generate and output added subdivision vertices and their connection information from the basic mesh decoded by the basic mesh decoding unit 202 using the subdivision method indicated by the control information.

[0058] Here, the basic mesh, the added subdivision vertices, and the connection information therebetween are collectively referred to as a "subdivision mesh."

[0059] The subdivision unit 202 is configured to identify the type of subdivision method from subdivision_method, which is control information generated by decoding the atlas bitstream.

[0060] The subdivision unit 202 will be described below with reference to FIGS. 6A and 6B.

[0061] 6A and 6B are diagrams for explaining an example of the operation of generating subdivision vertices from a basic mesh.

[0062] FIG. 6A is a diagram showing an example of a basic mesh made up of five vertices.

[0063] Here, the subdivision may be performed using, for example, a mid-edge division method in which the midpoints of the sides of each basic face are connected to each other, resulting in a basic face being divided into four faces.

[0064] Figure 6B shows an example of a subdivision mesh obtained by dividing a base mesh consisting of five vertices. In the subdivision mesh shown in Figure 6B, eight subdivision vertices (white circles) are generated in addition to the original five vertices (black circles).

[0065] The displacement decoding unit 206 decodes the displacement for each subdivision vertex thus generated, which is expected to improve coding performance.

[0066] The subdivision unit 202 may also apply a different subdivision method to each patch. This allows the displacement decoded by the displacement decoding unit 206 to be adaptively changed for each patch, which is expected to improve coding performance. Information about the divided patches is received as patch_id, which is control information.

[0067] The subdivision section 203 will be described below with reference to Fig. 7 to Fig. 9. Fig. 7 is a diagram showing an example of functional blocks of the subdivision section 203.

[0068] As shown in FIG. 7, the subdivision unit 203 has a basic mesh subdivision unit 203A, a subdivision mesh adjustment unit 203B, and a subdivision vertex normal calculation unit 203C.

[0069] (Basic mesh refinement section 203A) The basic mesh subdivision unit 203A is configured to calculate the number of divisions (number of subdivisions) for each basic surface and basic patch based on the input basic mesh and division information of the basic mesh, subdivide the basic mesh based on this number of divisions, and output an initial subdivision mesh.

[0070] Here, the subdivision surfaces are surfaces that make up the subdivision mesh, and the subdivision mesh is a mesh that has been obtained by subdividing a basic surface at least once.

[0071] That is, the basic mesh subdivision unit 203A may be configured so that the number of divisions can be changed for each basic surface and each basic patch.

[0072] Here, a basic face is a face that constitutes a basic mesh, and a basic patch is a set of several basic faces.

[0073] In addition, the basic mesh subdivision unit 203A may be configured to predict the number of subdivisions of the basic surface and calculate the number of subdivisions of the basic surface by adding the predicted subdivision number residual to the predicted number of subdivisions of the basic surface.

[0074] The basic mesh subdivision unit 203A may also be configured to calculate the number of subdivisions of a basic surface based on the number of subdivisions of adjacent basic surfaces of the basic surface.

[0075] The basic mesh subdivision unit 203A may also be configured to calculate the number of subdivisions of a basic surface based on the number of subdivisions of the basic surface that was accumulated immediately before.

[0076] The basic mesh subdivision unit 203A may also be configured to calculate the number of subdivisions of the basic surface based on the number of subdivisions mdu_subdivision_num defined for each patch.

[0077] The basic mesh subdivision unit 203A may also be configured to generate vertices that divide the three sides that make up the basic surface, and to connect the generated vertices to subdivide the basic surface.

[0078] As shown in FIG. 7, a subdivision mesh adjustment unit 203B and a subdivision vertex normal calculation unit 203C, which will be described later, are provided downstream of a basic mesh subdivision unit 203A.

[0079] An example of the processing of the basic mesh subdivision unit 203A will be described below with reference to FIGS.

[0080] FIG. 8 is a diagram showing an example of functional blocks of the basic mesh subdivision unit 203A, and FIG. 10 is a flowchart showing an example of the operation of the basic mesh subdivision unit 203A.

[0081] As shown in FIG. 8, the basic mesh subdivision unit 203A has a basic surface division number buffer unit 203A1, a basic surface division number reference unit 203A2, a basic surface division number prediction unit 203A3, an addition unit 203A4, and a basic surface division unit 203A5.

[0082] The basic surface division number buffer unit 203A1 stores division information of basic surfaces including the division number of the basic surface, and is configured to output the division information of the basic surface to the basic surface division number reference unit 203A2.

[0083] Here, the size of the basic surface division number buffer unit 203A1 may be set to 1, and the basic surface division number buffer unit 203A1 may be configured to output the division number of the basic surface that was most recently accumulated to the basic surface division number reference unit 203A2.

[0084] That is, by setting the size of the basic surface division number buffer unit 203A1 to 1, it may be configured to refer to only the fine division number decoded last (the subdivision number decoded immediately before).

[0085] The basic surface division number reference unit 203A2 is configured to output a reference not possible to the basic surface division number prediction unit 203A3 when there is no adjacent basic surface to the basic surface to be decoded, or when there is an adjacent basic surface to the basic surface to be decoded but the division number has not been determined.

[0086] On the other hand, if there is a basic surface adjacent to the basic surface to be decoded and the number of divisions has been determined, the basic surface division number reference unit 203A2 is configured to output the number of divisions to the basic surface division number prediction unit 203A3.

[0087] The basic surface division number prediction unit 203A3 is configured to predict the division number (number of subdivisions) of the basic surface based on one or more input division numbers, and output the predicted division number (predicted division number) to the addition unit 203A4.

[0088] Here, the basic surface division number prediction unit 203A3 is configured to output 0 as the predicted number of divisions to the addition unit 203A4 when only reference impossible is input from the basic surface division number reference unit 203A2.

[0089] In addition, when one or more division numbers are input, the basic surface division number prediction unit 203A3 may be configured to generate a predicted division number using any of the statistical values ​​such as the average value, maximum value, minimum value, or mode of the input division numbers.

[0090] The basic surface division number prediction unit 203A3 may be configured to generate the division number of the most adjacent surface as the predicted division number when one or more division numbers are input.

[0091] The addition unit 203A4 is configured to output the division number obtained by adding the predicted division number residual decoded from the prediction residual bitstream and the predicted division number obtained from the basic surface division number prediction unit 203A3 to the basic mesh division number buffer unit 203A1 and the basic surface division unit 203A5.

[0092] The basic surface dividing unit 203A5 is configured to subdivide the basic surface based on the division number input from the addition unit 203A4.

[0093] 9 shows an example of a case where a basic surface is divided into 9. The division method of the basic surface by the basic surface dividing section 203A5 will be described with reference to FIG.

[0094] As shown in FIG. 9, the basic surface dividing unit 203A5 generates points A_1, . . . , A_(N-1) that divide the side AB that constitutes the basic surface into N equal parts (N=3).

[0095] Similarly, the basic surface division unit 203A5 divides the sides BC and CA into N equal parts, and generates points B_1, . . . , B_(N-1) and C_1, . . . , C_(N-1), respectively.

[0096] Hereinafter, the points on sides AB, BC, and CA will be called "side division points."

[0097] For all i (i = 1, 2, …, N - 1), the basic surface division part 203A5 generates the sides \(A_iB_{N - i}\), \(B_iC_{N - i}\), \(C_iA_{N - i}\), and generates \(N^2\) sub - divided surfaces. This division method is hereinafter referred to as the \(N^2\) division method. The \(N^2\) division method is equivalent to the Mid - edge division method when N = 2.

[0098] Next, referring to FIG. 10, an example of the operation of the basic mesh division part 203A will be described.

[0099] As shown in FIG. 10, in step S2201, the basic mesh division part 203A determines whether the division processing for the last basic surface has been completed. If all the division processing has been completed, this operation ends; otherwise, this operation proceeds to step S2202.

[0100] In step S2202, the basic mesh division part 203A determines whether Depth < mdu_max_subdivision_iteration_count.

[0101] Here, Depth is a variable representing the current depth, with an initial value of 0, and mdu_max_subdivision_iteration_count represents the maximum depth determined for each basic surface.

[0102] If the condition in step S2202 is satisfied, this operation proceeds to step S2203; if the condition is not satisfied, this operation returns to step S2201.

[0103] In step S2203, the basic mesh division part 203A determines whether mdu_subdivision_flag at the current depth is 1.

[0104] If Yes, this operation returns to step S2201; if No, this operation proceeds to step S2204.

[0105] In step S2204, the basic mesh subdivision unit 203A further subdivides all subdivision surfaces within the basic surface.

[0106] Here, if the subdivision process has never been performed on the basic surface, the basic mesh subdivision unit 203A subdivides the basic surface.

[0107] The subdivision method is the same as that described with reference to FIG.

[0108] Specifically, if the basic surface has never been subdivided, the basic mesh subdivision unit 203A subdivides the basic surface as shown in FIG.

[0109] On the other hand, if the subdivision surface has been subdivided at least once, the basic mesh subdivision unit 203A subdivides the subdivision surface into N2 pieces.

[0110] Taking FIG. 9 as an example, a face consisting of vertices A_2, B, and B_1 is divided into N2 faces using the N2 division method in the same way as when dividing the basic face.

[0111] When the subdivision process is completed, the operation proceeds to step S2205.

[0112] In step S2205, the basic mesh subdivision unit 203A adds 1 to Depth, and the operation returns to step S2202.

[0113] The basic mesh subdivision unit 203A may also perform subdivision processing so that all basic faces are subdivided the same maximum number of times of subdivision mdu_max_subdivision_iteration_count. In this case, the subdivision processing per time may be configured to perform subdivision using the N2 subdivision method or the Mid-edge subdivision method based on the number of subdivisions mdu_subdivision_num.

[0114] (Fine division mesh adjustment unit 203B) Next, a specific example of the processing performed by the finely divided mesh adjustment unit 203B will be described. An example of the processing performed by the finely divided mesh adjustment unit 203B will be described below with reference to Figs.

[0115] FIG. 11 is a diagram showing an example of functional blocks of the finely divided mesh adjustment unit 203B.

[0116] The subdivision mesh adjustment unit 203B may be configured to adjust the subdivision mesh based on the subdivision adjustment method specified in the frame information output by the atlas data decoding unit 208.

[0117] Here, adjustment refers to an operation of generating subdivision meshes so that no holes are created in the mesh when the mesh is decoded by the mesh decoding unit 204.

[0118] In the example of FIG. 11, the subdivision mesh adjustment unit 203B uses a subdivision adjustment method in which the vertices of the subdivision faces of the initial subdivision mesh are moved and the subdivision faces are further divided to adjust the mesh.

[0119] As shown in FIG. 11, the subdivision mesh adjustment unit 203B has an edge division point moving unit 701 and a subdivision surface division unit 702.

[0120] (Edge division point moving unit 701) The edge division point moving unit 701 is configured to move an edge division point of a basic face to any of the edge division points of an adjacent basic face for an input initial subdivision face, and output a subdivision face.

[0121] The edge division point moving unit 701 may be configured to output the initial subdivision mesh as is to the subdivision surface dividing unit 702 without moving the edge division points when mdu_vertex_adaption_flag is 1.

[0122] Fig. 12 shows an example of a case where an edge division point on a basic face ABC is moved. For example, as shown in Fig. 12, the edge division point moving unit 701 may be configured to move an edge division point of the basic face ABC to an edge division point of the nearest adjacent basic face.

[0123] (Subdivision surface division part 702) The subdivision surface division unit 702 is configured to again subdivide the input subdivision surface and output a decoded subdivision surface.

[0124] FIG. 13 is a diagram showing an example of a case where subdivision is performed again on a subdivision surface X within a basic surface.

[0125] As shown in FIG. 13, the subdivision surface division unit 702 may be configured to generate a new subdivision surface within the basic surface by connecting the vertices that make up the subdivision surface X with the edge division points of the adjacent basic surface.

[0126] FIG. 14 is a diagram showing an example of a case where the above-described subdivision processing is performed on all subdivision surfaces.

[0127] (Subdivision vertex normal calculation unit 203C) The subdivision vertex normal calculation unit 203C is configured to output normals (subdivision vertex normals) defined for each subdivision vertex that constitutes the input subdivision surface.

[0128] An example of the processing performed by the subdivision vertex normal calculation unit 203C will be described below with reference to FIGS.

[0129] FIG. 15 is a flowchart showing an example of the operation of the subdivision vertex normal calculation unit 203C.

[0130] As shown in FIG. 15, in step S203B-1, the subdivision vertex normal calculation unit 203C determines whether or not the calculation of subdivision vertex normals has been completed for all patches.

[0131] If yes, the operation ends, and if no, the operation proceeds to step S203B-2.

[0132] In step S203B-2, the subdivision vertex normal calculation unit 203C determines the calculation method for the subdivision vertex normals based on mdu_normal_method, and the process proceeds to step S203B-3.

[0133] In step S203B-3, the subdivision vertex normal calculation unit 203C determines whether or not the calculation of subdivision vertex normals has been completed for all basic faces in the patch.

[0134] If yes, the operation proceeds to step S203B-1, and if no, the operation proceeds to step S203B-4.

[0135] In step S203B-4, the subdivision vertex normal calculation unit 203C determines whether or not calculation of subdivision vertex normals has been completed for all subdivision vertices within the base surface.

[0136] If yes, the operation proceeds to step S203B-3, and if no, the operation proceeds to step S203B-5.

[0137] In step S203B-5, the subdivision vertex normal calculation unit 203C calculates the subdivision vertex normals of the unprocessed subdivision vertices, and the operation proceeds to step S203B-4.

[0138] 16A and 16B show the subdivision vertex normals calculated by the subdivision vertex normal calculation unit 203C in step S203B-5 for the subdivision vertex X in the base face ABC.

[0139] An example of a method for calculating subdivision vertex normals will be described with reference to FIG. 16A.

[0140] The subdivision vertex normal calculation unit 203C may be configured to calculate a normal n (vector) of the subdivision vertex X on the basic surface ABC based on the sum of the basic point normals, as shown in the following (Equation 1).

[0141]

number

[0142] Also, with reference to FIG. 16B, another example of a method for calculating subdivision vertex normals will be described.

[0143] The subdivision vertex normal calculation unit 203C may be configured to calculate a normal n (vector) of the subdivision vertex X on the base face ABC based on the sum of the normals of the base faces adjacent to the base face ABC, as shown in the following (Equation 2).

[0144]

number

[0145] Furthermore, the subdivision vertex normal calculation unit 203C may simply use the surface normal of the base surface ABC as the normal of the subdivision vertex X (subdivision point normal).

[0146] By performing calculations in this way, the same normal is basically used for subdivision point normals that exist on the same basic surface.

[0147] If the subdivision vertex normals have already been calculated, the subdivision vertex normal calculation unit 203C does not need to perform the calculation process for the subdivision vertex normals.

[0148] Similarly, the subdivision vertex normal calculation unit 203C may also calculate the base point normals using the above-described method for calculating subdivision vertex normals.

[0149] At this time, the same normal is basically used as the subdivision point normal and the base point normal that exist on the same base surface.

[0150] (Displacement amount decoding unit 206) As shown in FIG. 17, the displacement amount decoding unit 206 includes a decoding skip determination unit 206A, an arithmetic decoding unit 206B, a context value update unit 206C, a context buffer 206D, a context selection unit 206E, a multi-value conversion unit 206F, an inter prediction unit 206G, a frame buffer 206H, an adder 206I, an inverse quantization unit 206J, and a displacement amount prediction addition unit 206K.

[0151] The control information decoder 206A is configured to output control information by performing variable length decoding on the received displacement amount bit stream.

[0152] The arithmetic decoding unit 206B is configured to perform arithmetic decoding on the received displacement amount bit stream, thereby outputting binarized coefficient level values.

[0153] The context value update unit 206C is configured to update the context value using the binarization coefficient level value and output it to the context buffer 206D.

[0154] The context selection unit 206E is configured to generate and output a context value (output context value) using the context value, bit position, and syntax read from the context buffer 206D.

[0155] The multi-value conversion unit 206F is configured to convert the binarized coefficient level values ​​into multi-values ​​to generate and output coefficient level values. The generated (calculated) coefficient level values ​​are also output to the context buffer 206D as bit positions and syntax.

[0156] The inter prediction unit G is configured to generate and output a predicted displacement amount using a reference frame read from the frame buffer 206H.

[0157] The frame buffer H is configured to acquire and store the decoded displacement amount, and to output the decoded displacement amount at the corresponding vertex in the reference frame in accordance with control information (not shown).

[0158] The adder 206I is configured to obtain the inter prediction residual and the inter prediction displacement from the inter prediction unit 206G. The adder 206I is configured to add them to generate and output a quantized intra prediction residual. The generated (calculated) quantized intra prediction residual is also output to the frame buffer 206H.

[0159] The inverse quantization unit 206J is configured to perform inverse quantization on the quantized intra prediction residual obtained from the addition unit 206I, and output the intra prediction residual. Details will be described later.

[0160] The displacement amount prediction addition unit 206K is configured to intra-predict the displacement amount of the subdivision vertices based on the basic mesh output from the basic mesh decoding unit 202, calculate an intra-prediction value, and decode the displacement amount by adding the calculated intra-prediction value and the intra-prediction residual output from the inverse quantization unit 206J.

[0161] (Inverse quantization section 206J) 18 is a diagram showing an example of a flowchart illustrating an example of the operation of the inverse quantization unit 206 J. Hereinafter, an example of the operation of the inverse quantization unit 206 J will be described with reference to FIG.

[0162] As shown in FIG. 18, in step S203J-1, the inverse quantization unit 206J determines whether mdu_lod_change_flag[tileID][patchIdx]!=0.

[0163] If the answer is Yes, the operation proceeds to step S203J-3, and if the answer is No, the operation proceeds to step S203J-2.

[0164] In step S203J-2, the inverse quantization unit 206J performs inverse quantization on all the displacement amounts, and then ends the process.

[0165] In this case, the inverse quantization unit 206J may perform inverse quantization using the uniform quantization disclosed in Reference 2, "Jinhua Yu, "Advantages of uniform scalar dead-zone quantization in image coding system," in 2004 International Conference on Communications, Circuits and Systems (IEEE Cat. No. 04EX914). IEEE, 2004, vol. 2, pp. 805-808," or may perform inverse quantization using any other known inverse quantization method.

[0166] In step S203J-3, the inverse quantization unit 206J determines whether or not the process has been completed for all patches.

[0167] If yes, the operation ends, and if no, the operation proceeds to step S203J-4.

[0168] In step S203J-4, the inverse quantization unit 206J determines whether processing has been completed for all levels in the patch.

[0169] If the answer is Yes, the operation proceeds to step S203J-3, and if the answer is No, the operation proceeds to step S203J-5.

[0170] In step S203J-5, the inverse quantization unit 206J determines the quantization value of the current level based on mdu_lod_qp[tileID][patchIdx][i], and the process proceeds to step S203J-6.

[0171] In step S203J-6, the inverse quantization unit 206J determines whether or not the inverse quantization process has been completed for all displacement amounts at the current level.

[0172] If the answer is Yes, the operation proceeds to step S203J-4, and if the answer is No, the operation proceeds to step S203J-7.

[0173] In step S203J-7, the inverse quantization unit 206J inversely quantizes the displacement amount using the quantization value calculated in step S203J-5, and the process proceeds to step S203J-6.

[0174] At this time, the inverse quantization unit 206J may perform inverse quantization using the uniform quantization disclosed in the above-mentioned Reference 2, or may perform inverse quantization using other known inverse quantization methods.

[0175] According to this embodiment, the same subdivision vertex normals are used within the same base surface, making it possible to efficiently encode the amount of displacement.

[0176] Furthermore, according to this embodiment, it is possible to set a decoding method according to the characteristics of the meshes in the patch, so that the number of bits of the displacement amount can be reduced.

[0177] The mesh encoding device 100 and the mesh decoding device 200 described above may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]

[0178] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which will contribute to the achievement of Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0179] 1...Mesh processing system 100...Mesh coding device 200...Mesh decoding device 201…Demultiplexer 202...Basic mesh decoding unit 203…Subdivision part 203A...Basic mesh refinement section 203A1...Basic face division number buffer section 203A2…Basic surface division number reference section 203A3...Basic surface division number prediction unit 203A4...Addition section 203A5…Basic surface division part 203B…Fine division mesh adjustment section 701...Edge division point moving part 702…Subdivision surface division part 203C...Subdivision vertex normal calculation unit 204...Mesh decoding unit 205…Patch Integration Department 206...Displacement amount decoding unit 206A...Control information decoding unit 206B...Arithmetic decoding unit 206C...Context value update unit 206D...Context buffer 206E…Context selection section 206F...Multi-level conversion section 206F2...coefficient level value decoding unit 206G…Inter prediction section 206H...Frame buffer 206I...adder 206J…Inverse quantization section 206K…Displacement prediction addition section 207...Video decoding unit 208...Atlas data decoding unit

Claims

1. A mesh decoding device, comprising: a basic surface division unit configured to subdivide the basic mesh and output an initial refinement mesh; a subdivision vertex normal calculation unit configured to calculate a normal for each vertex constituting the initial subdivision mesh; a subdivision mesh adjustment unit configured to adjust the subdivision mesh by dividing the subdivision surfaces after moving the positions of the vertices that constitute the surfaces of the initial subdivision mesh.

2. 2. The mesh decoding device according to claim 1, wherein the subdivision vertex normal calculation unit is configured to calculate subdivision vertex normals within the base surface based on the sum of base point normals defined for each base point, which is a vertex that constitutes a base surface within the initial subdivision mesh.

3. 2. The mesh decoding device according to claim 1, wherein the subdivision vertex normal calculation unit is configured to calculate subdivision vertex normals within the base surface based on the sum of normals of base surfaces adjacent to the base surface within the initial subdivision mesh.

4. 2. The mesh decoding device according to claim 1, wherein the subdivision vertex normal calculation unit is configured to use the same normal for the base points, which are vertices within a base face within the initial subdivision mesh, and the subdivision points.

5. 2. The mesh decoding device according to claim 1, wherein the subdivision vertex normal calculation unit is configured to, when normals of base points and subdivision points in a base face in the initial subdivision mesh have already been calculated in decoding order, use the already calculated normals as normals of the base points and the subdivision points.

6. 1. A mesh decoding method, comprising: a step of subdividing the basic mesh to output an initial refinement mesh; calculating normals for each vertex constituting the initial subdivision mesh; and adjusting the subdivision mesh by dividing the subdivision surfaces after moving the positions of the vertices that constitute the surfaces of the initial subdivision mesh.

7. A program that causes a computer to function as a mesh decoding device, The mesh decoding device comprises: a basic surface division unit configured to subdivide the basic mesh and output an initial refinement mesh; a subdivision vertex normal calculation unit configured to calculate a normal for each vertex constituting the initial subdivision mesh; a subdivision mesh adjustment unit configured to adjust the subdivision mesh by dividing the subdivision surfaces after moving the positions of the vertices that constitute the surfaces of the initial subdivision mesh.

Images