Mesh decoding device, mesh decoding method, and program product

Abstract

A mesh decoding device (200) according to the present invention is provided with: a basic plane division unit (203A5) that subdivides a basic mesh and outputs an initial subdivided mesh; a sub-division vertex normal calculation unit (203C) that calculates a normal for each vertex constituting the initial sub-division mesh; and a sub-division mesh adjustment unit (203B) that adjusts the sub-division mesh by dividing the sub-division surface after moving the position of the apex of the surface constituting the initial sub-division mesh.

Description

Technical Field

[0001] The invention relates to a grid decoding device, a grid decoding method and a program product. Background Art

[0002] In Non-Patent Document 1, a mesh is divided into a simple base mesh and a detailed displacement, which is decoded by a video codec. The displacement is calculated based on a normal defined for each vertex, but the direction of the normal is different for each vertex.

[0003] Furthermore, in Non-Patent Literature 1, a decoding method is defined for each patch using parameters set for each patch.

[0004] Prior art literature

[0005] Non-patent literature

[0006] Non-patent document 1: Khaled Mammou, Jungsun Kim, Alexis Tourapis, DimitriPodborski, Krasimir Kolarov, “[V-CG] Apple's Dynamic Mesh Coding CfP Response,” ISO / IEC JTC 1 / SC 29 / WG 7m5928, April 2022. Summary of the invention

[0007] Problems to be solved by the invention

[0008] However, in non-patent document 1, since the direction of the normal line is different for each vertex, there is a problem that the displacement cannot be compressed efficiently. Therefore, the present invention is made in view of the above-mentioned problem, and its purpose is to provide a grid decoding device, grid encoding device, grid decoding method and program product that can efficiently realize the encoding of the displacement.

[0009] Means for solving problems

[0010] The first feature of the present invention is a mesh decoding device, the main purpose of which is to have: a basic surface segmentation unit, which is configured to sub-divide the basic mesh and output an initial sub-divided mesh; a sub-divided vertex normal calculation unit, which is configured to calculate normals for each vertex constituting the initial sub-divided mesh; and a sub-divided mesh adjustment unit, which is configured to adjust the sub-divided mesh by segmenting the sub-divided surface after moving the position of the vertices of the surface constituting the initial sub-divided mesh.

[0011] The second feature of the present invention is a grid decoding method, the main purpose of which is to include: a process of sub-dividing a basic grid and outputting an initial sub-divided grid; a process of calculating a normal for each vertex constituting the initial sub-divided grid; and a process of adjusting the sub-divided grid by dividing the sub-divided surface after moving the position of the vertices of the surface constituting the initial sub-divided grid.

[0012] The third feature of the present invention is a program product, the main purpose of which is that it includes a computer program, which implements the function of a mesh decoding device when the computer program is executed, and the mesh decoding device has: a basic surface segmentation unit, which is configured to sub-divide the basic mesh and output an initial sub-divided mesh; a sub-divided vertex normal calculation unit, which is configured to calculate normals for each vertex constituting the initial sub-divided mesh; and a sub-divided mesh adjustment unit, which is configured to adjust the sub-divided mesh by segmenting the sub-divided surface after moving the position of the vertices of the surface constituting the initial sub-divided mesh.

[0013] Effects of the Invention

[0014] According to the present invention, it is possible to provide a trellis decoding device, a trellis encoding device, a trellis decoding method and a program product that can efficiently implement the encoding of the displacement amount.

[0015] Brief description of the attached figure

[0016] Figure 1 It is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment.

[0017] Figure 2 This is a diagram showing an example of functional blocks of a trellis decoding device 200 according to an embodiment.

[0018] Figure 3 This is a diagram showing an example of the syntax structure of the Atlas bit stream.

[0019] Figure 4 This is a diagram showing an example of the syntax structure of PDU.

[0020] Figure 5 This is a diagram showing an example of the syntax structure of FPS.

[0021] Fig. 6A This is a diagram showing an example of a basic grid and sub-divided grids.

[0022] Figure 6B This is a diagram showing an example of a basic grid and sub-divided grids.

[0023] Figure 7 1 is a diagram showing an example of a functional block of the sub-division unit 203 of the trellis decoding device 200 according to an embodiment.

[0024] Figure 8 This is a diagram showing an example of a functional block of a base trellis sub-dividing unit 203A of the sub-dividing unit 203 of the trellis decoding device 200 according to an embodiment.

[0025] Fig. 9 This is a diagram for explaining an example of a base plane division method implemented by a base plane division unit 203A5 of a base mesh sub-division unit 203A of a sub-division unit 203 of a mesh decoding device 200 according to an embodiment.

[0026] Fig.10 This is a flowchart showing an example of the operation of the base grid sub-dividing unit 203A of the sub-dividing unit 203 of the grid decoding device 200 according to one embodiment.

[0027] Fig.11 1 is a diagram showing an example of a functional block of a sub-division grid adjustment unit 203B of the sub-division unit 203 of the grid decoding device 200 according to an embodiment.

[0028] Fig.12 1 is a diagram showing an example of a case where the side division point moving unit 701 of the sub division grid adjustment unit 203B of the sub division unit 203 of the grid decoding device 200 according to one embodiment moves the side division points on the base plane ABC.

[0029] Fig.13 1 is a diagram showing an example of a case where the sub-division plane X in the base plane is sub-divided again by the sub-division plane dividing unit 702 of the sub-division grid adjustment unit 203B of the sub-division unit 203 of the grid decoding device 200 according to one embodiment.

[0030] Fig.14 1 is a diagram showing an example of a case where all sub-division planes are sub-divided again by the sub-division plane dividing unit 702 of the sub-division grid adjusting unit 203B of the sub-division unit 203 of the trellis decoding device 200 according to one embodiment.

[0031] Fig.15 This is a flowchart showing an example of the operation of the sub-divided vertex normal calculation unit 203C of the sub-dividing unit 203 of the mesh decoding device 200 according to one embodiment.

[0032] Fig.16A This is a diagram for explaining an example of a method for calculating sub-division vertex normals implemented by the sub-division vertex normal calculation unit 203C.

[0033] Fig. 16B This is a diagram for explaining an example of a method for calculating sub-division vertex normals implemented by the sub-division vertex normal calculation unit 203C.

[0034] Fig.17 1 is a diagram showing an example of a functional block of the displacement amount decoding unit 206 of the trellis decoding device 200 according to an embodiment.

[0035] Fig.18 This is a flowchart showing an example of the operation of the inverse quantization unit 206J of the displacement amount decoding unit 206 of the trellis decoding device 200 according to one embodiment. DETAILED DESCRIPTION

[0036] Below, while referring to the attached Figure 1 The embodiments of the present invention are described below. In addition, the structural elements in the following embodiments can be appropriately replaced with existing structural elements, etc., and various changes including combinations with other existing structural elements can be made. Therefore, the content of the invention described in the claims is not limited based on the description of the following embodiments.

[0037] <First Embodiment>

[0038] Below, refer to Figures 1 to 18 , the grid processing system involved in this embodiment is described.

[0039] Figure 1 1 is a diagram showing an example of the structure of the grid processing system 1 according to the present embodiment. Figure 1 As shown, the grid processing system 1 has a grid encoding device 100 and a grid decoding device 200 .

[0040] Figure 2 This is a diagram showing an example of functional blocks of the trellis decoding device 200 according to the present embodiment.

[0041] like Figure 2 As shown, the trellis decoding device 200 has a demultiplexing unit 201, a basic trellis decoding unit 202, a sub-division unit 203, a trellis decoding unit 204, a patch integration unit 205, a displacement decoding unit 206, a video decoding unit 207 and an atlas data decoding unit 208.

[0042] The demultiplexing unit 201 is configured to separate the multiplexed bit stream into a base grid bit stream, a displacement bit stream, a texture bit stream, and an atlas bit stream.

[0043] The base grid decoding unit 202 is configured to decode the base grid bit stream, generate and output a base grid.

[0044] The sub-division unit 203 is configured to generate and output the added sub-division vertices and connection information thereof from the basic mesh decoded by the basic mesh decoding unit 202 using the sub-division method indicated by the patch information.

[0045] Here, the base mesh, the added sub-division vertices, and the connection information thereof are collectively referred to as a “sub-division mesh”.

[0046] The mesh decoding unit 204 is configured to generate and output a decoded mesh using the sub-division mesh generated by the sub-division unit 203 and the displacement amount decoded by the displacement amount decoding unit 206 .

[0047] The patch integration unit 205 is configured to integrate and output a plurality of patches of the decoded meshes generated by the mesh decoding unit 206 .

[0048] The displacement amount decoding unit 206 is configured to decode the displacement amount bit stream and generate and output the displacement amount.

[0049] The video decoding unit 207 is configured to decode and output textures by video coding. For example, the video decoding unit 207 may use HEVC disclosed in Reference 1 "Information technology-High efficiency coding and media delivery in heterogeneous environments-Part 2: High efficiency video coding, ISO / IEC 23008-2:2013."

[0050] The Atlas data decoding unit 208 is configured to decode the Atlas bit stream and output frame information and patch information.

[0051] (Atlas data decoding unit 208)

[0052] As described above, the Atlas data decoding unit 208 is configured to decode the Atlas bit stream and output frame information and patch information.

[0053] Below, refer to Figure 3 to Figure 5 , describing the frame information and patch information decoded by the Atlas data decoding unit 208.

[0054] Figure 3 This is a diagram showing an example of the syntax structure of the Atlas bit stream.

[0055] like Figure 3 As shown, first, the Atlas bit stream includes a PDU (Patch data unit) which is a control information set corresponding to the patch.

[0056] The patch information is a set of control signals defined by each patch obtained by decoding the PDU.

[0057] As described above, the atlas bit stream is structured such that each PDU corresponds to each patch.

[0058] In addition, if Figure 3 As shown, FPS (Frame Parameter Set) which is a control information set corresponding to the frame is included.

[0059] The frame information is a set of control signals defined for each frame obtained by decoding the FPS.

[0060] also, Figure 3 The structure is just an example, and as long as the PDU is configured to correspond to each patch data, elements other than the above may be added as structural elements of the atlas bit stream.

[0061] For example, Figure 3 As shown, the Atlas bitstream may contain an SPS (Sequence Parameter Set).

[0062] Below, refer to Figure 4 Describe the syntactic structure of PDU.

[0063] Figure 4 is a diagram showing an example of the syntax structure of a PDU. Here, as long as the syntax functions are the same, you can use Figure 4 The syntactic companions shown have different syntactic names.

[0064] exist Figure 4 In the syntax structure of the BPH shown, the descriptor column indicates how each syntax is encoded. In addition, ue(v) indicates an uncoded 0th-order exponential-Golomb code, and u(n) indicates an n-bit flag.

[0065] The PDU contains a control signal mdu_triangle_count_minus1 that specifies the number of base surfaces contained in each patch.

[0066] The PDU includes a flag mdu_subdivision_enable_flag that specifies whether to perform subdivision by patch.

[0067] The PDU may include a flag mdu_lod_change_flag for designating whether to change the quantization value of the displacement amount at each sub-division level of the displacement amount of each patch.

[0068] The PDU may include a control signal (mdu_subdivision_method) that specifies the type of subdivision method of the base mesh per patch.

[0069] For example, when mdu_subdivision_method=0, it is defined as using the Mid-edge segmentation method to generate the initial sub-division grid, and when mdu_subdivision_method=1, it can be defined as using the N2 segmentation method to generate the initial sub-division grid.

[0070] The PDU may include a control signal (mdu_subdivision_num_method) for designating the type of subdivision number generation method for each patch.

[0071] The PDU may contain a control signal (mdu_subdivision_residuals) that specifies the error in the predicted value of the number of subdivisions per patch.

[0072] The PDU may include a control signal (mdu_max_subdivision_iteration_count) indicating the number of subdivision processing times per patch.

[0073] The PDU may include a control signal (mdu_subdivision_flag) that specifies whether to recursively sub-divide the base plane by index i (i=0, ..., mdu_triangle_count_minus1) and j (j=0, ..., mdu_subdivision_iteration_count).

[0074] The PDU may include a control signal (mdu_subdivision_num) for specifying the number of divisions of the base plane in the subdivision process for each patch.

[0075] The PDU may include a control signal (mdu_normal_method) that specifies the type of method for calculating sub-partition vertex normals on a per-patch basis.

[0076] The PDU may contain a control signal (mdu_lod_qp) that specifies the quantized value of the displacement amount for each sub-division level on a per-patch basis.

[0077] The sub-division level is the number of sub-division processes in the current stage. For example, when the sub-division level is 1, it means that the base surface is sub-divided once.

[0078] Below, refer to Figure 5 The syntactic structure of FPS is described.

[0079] Figure 5 is a diagram showing an example of the syntax structure of FPS. Here, as long as the syntax functions are the same, you can use Figure 5 The syntactic companions shown have different syntactic names.

[0080] exist Figure 5 In the syntax structure of the FPS shown, the descriptor column is the same as the PDU.

[0081] The FPS may include a flag (mdu_border_adaption_flag) indicating whether sub-division adjustment is performed on a per-frame basis.

[0082] The FPS may contain a control signal (mdu_border_adaption_method) that specifies the sub-division adjustment method on a per-frame basis.

[0083] The FPS may include a flag (mdu_vertex_adaption_flag) indicating whether to move the edge division points in the sub-division grid adjustment part per frame.

[0084] (Sub-dividing unit 203)

[0085] As described above, the sub-division unit 203 is configured to generate and output the added sub-division vertices and the connection information thereof based on the basic mesh decoded by the basic mesh decoding unit 202 according to the sub-division method indicated by the control information.

[0086] Here, the base mesh, the added sub-division vertices, and the connection information thereof are collectively referred to as a “sub-division mesh”.

[0087] The sub-division unit 202 is configured to specify the type of sub-division method based on subdivision_method, which is control information generated by decoding the atlas bit stream.

[0088] Below, refer to Fig. 6A and Figure 6B The sub-dividing unit 202 will be described.

[0089] Fig. 6A and Figure 6B This is a diagram for explaining an example of an operation of generating sub-division vertices from a base mesh.

[0090] Fig. 6A This is a diagram showing an example of a basic mesh composed of five vertices.

[0091] Here, the subdivision may be performed using, for example, a mid-edge segmentation method that connects the midpoints of the edges of each base surface to each other, thereby segmenting one base surface into four surfaces.

[0092] Figure 6B An example of a sub-divided mesh formed by dividing a base mesh consisting of five vertices is shown. Figure 6B In the sub-division mesh shown, eight sub-division vertices (hollow circles) are generated in addition to the original five vertices (solid circles).

[0093] By decoding the displacement amount for each sub-divided vertex generated in this way, the displacement amount decoding unit 206 can expect improvement in encoding performance.

[0094] In addition, the sub-division unit 202 may apply different sub-division methods to each patch. Thus, the displacement decoded by the displacement decoding unit 206 is adaptively changed for each patch, and improvement in encoding performance can be expected. Information on the divided patch is received as control information, namely, patch_id.

[0095] Below, refer to Figure 7 to Figure 9 The sub-dividing unit 203 will be described. Figure 7 2 is a diagram showing an example of functional blocks of the sub-dividing unit 203 .

[0096] like Figure 7 As shown, the sub-division unit 203 includes a base mesh sub-division unit 203A, a sub-division mesh adjustment unit 203B, and a sub-division vertex normal calculation unit 203C.

[0097] (Basic mesh sub-division unit 203A)

[0098] The base mesh sub-division unit 203A is configured to calculate the number of divisions (sub-division number) for each base surface and each base patch based on the input base mesh and the division information of the base mesh, sub-divide the base mesh according to the division number, and output an initial sub-divided mesh.

[0099] Here, the sub-division surface is a surface constituting a sub-division mesh, and the sub-division mesh is a mesh obtained by sub-dividing the base surface at least once.

[0100] That is, the base mesh sub-division unit 203A may be configured to be able to change the above-mentioned number of divisions in units of base surfaces and base patches.

[0101] Here, a base face is a face that constitutes a base mesh, and a base patch is a collection of several base faces.

[0102] In addition, the basic mesh sub-division unit 203A may be configured to calculate the number of sub-divisions of the basic surface by predicting the number of sub-divisions of the basic surface and adding the predicted number of sub-divisions of the basic surface to a residual of the predicted number of sub-divisions of the basic surface.

[0103] In addition, the basic mesh sub-division unit 203A may be configured to calculate the number of sub-divisions of the basic surface based on the number of sub-divisions of adjacent basic surfaces of the basic surface.

[0104] In addition, the basic mesh sub-division unit 203A may be configured to calculate the number of sub-divisions of the basic surface based on the previously accumulated number of sub-divisions of the basic surface.

[0105] In addition, the basic mesh sub-division unit 203A may be configured to calculate the number of sub-divisions of the basic surface based on the number of sub-divisions mdu_subdivision_num defined for each patch.

[0106] In addition, the basic mesh sub-division unit 203A may be configured to generate vertices for dividing three sides constituting the basic surface and connect the generated vertices, thereby performing sub-division on the basic surface.

[0107] like Figure 7 As shown, a sub-division mesh adjustment unit 203B and a sub-division vertex normal calculation unit 203C, which will be described later, are provided at the subsequent stage of the basic mesh sub-division unit 203A.

[0108] Below, use Figure 8 to Figure 10 An example of the processing performed by the basic mesh sub-division unit 203A will be described.

[0109] Figure 8 is a diagram showing an example of a functional block of the basic mesh sub-division unit 203A. Fig.10 This is a flowchart showing an example of the operation of the basic mesh sub-division unit 203A.

[0110] like Figure 8 As shown, the basic mesh sub-division unit 203A includes a basic surface division number buffer unit 203A1, a basic surface division number reference unit 203A2, a basic surface division number prediction unit 203A3, an addition unit 203A4, and a basic surface division unit 203A5.

[0111] The base surface division number buffer unit 203A1 is configured to store the base surface division information including the division number of the base surface, and output the base surface division information to the base surface division number reference unit 203A2.

[0112] Here, the size of the base surface division number buffer unit 203A1 may be set to 1, and the base surface division number buffer unit 203A1 may output the previously accumulated base surface division number to the base surface division number reference unit 203A2.

[0113] That is, the size of the base face division number buffer unit 203A1 may be set to 1 so that only the last decoded sub-division number (the sub-division number decoded before) is referenced.

[0114] The base plane division number reference unit 203A2 is configured to output "cannot be referenced" to the base plane division number prediction unit 203A3 when there is no adjacent base plane to the decoding target base plane or when there is an adjacent base plane to the decoding target base plane but the division number is uncertain.

[0115] On the other hand, when there is an adjacent base plane to the decoding target base plane and the number of divisions is determined, the base plane division number reference unit 203A2 is configured to output the number of divisions to the base plane division number prediction unit 203A3.

[0116] The base surface division number prediction unit 203A3 is configured to predict the number of divisions of the base surface (the number of sub-divisions) based on one or more input division numbers, and output the predicted number of divisions (predicted number of divisions) to the addition unit 203A4.

[0117] Here, the base surface division number prediction unit 203A3 is configured to output 0 as the predicted division number to the addition unit 203A4 when only the reference impossibility is input from the base surface division number reference unit 203A2.

[0118] Furthermore, the base surface division number prediction unit 203A3 may be configured to generate a predicted division number using any statistical value such as an average value, maximum value, minimum value, mode, etc. of the input division numbers when more than one division number is input.

[0119] Furthermore, the base surface division number prediction unit 203A3 may be configured to generate the number of divisions of the most adjacent surface as the predicted number of divisions when one or more numbers of divisions are input.

[0120] The adding unit 203A4 is configured to output the number of divisions obtained by adding the predicted number of divisions residual decoded from the prediction residual bit stream and the predicted number of divisions acquired from the base plane division number predicting unit 203A3 to the base mesh division number buffer unit 203A1 and the base plane division unit 203A5.

[0121] The base surface dividing unit 203A5 is configured to divide the base surface into sub-divisions according to the number of divisions input from the adding unit 203A4.

[0122] Fig. 9 This is an example of dividing the base surface into nine parts. Fig. 9 The method of dividing the base surface by the base surface dividing unit 203A5 will be described.

[0123] like Fig. 9 As shown, the base surface dividing unit 203A5 generates points A_1, ..., A_(N-1) for dividing the side AB constituting the base surface into N equal parts (N=3).

[0124] Similarly, the base surface dividing unit 203A5 divides the side BC or the side CA into N equal parts, and generates points B_1, ..., B_(N-1), C_1, ..., C_(N-1), respectively.

[0125] Hereinafter, points on the side AB, the side BC, and the side CA are referred to as “side dividing points”.

[0126] The base surface division unit 203A5 generates the edges A_i B_(N-i), B_i C_(N-i), and C_i A_(N-i) for all i (i = 1, 2, …, N - 1), generating N2 sub-division surfaces. Thereafter, this division method is called the N2 division method. The N2 division method is equal to the Mid-edge division method when N = 2.

[0127] Next, with reference to Fig.10 an example of the operation of the base mesh sub-division unit 203A will be described.

[0128] As Fig.10 shown, in step S2201, the base mesh sub-division unit 203A determines whether sub-division processing of the last base surface has been completed. If all sub-division processing has been completed, this operation ends; if not, this operation proceeds to step S2202.

[0129] In step S2202, the base mesh sub-division unit 203A determines whether Depth < mdu_max_subdivision_iteration_count.

[0130] Here, Depth is a variable representing the current depth, with an initial value of 0, and mdu_max_subdivision_iteration_count represents the maximum depth determined for each base surface.

[0131] If the condition of step S2202 is satisfied, this operation proceeds to step S2203; if the condition is not satisfied, this operation returns to step S2201.

[0132] In step S2203, the base mesh sub-division unit 203A determines whether the mdu_subdivision_flag of the current depth is 1.

[0133] If it is, this operation returns to step S2201; if not, this operation proceeds to step S2204.

[0134] In step S2204, the base mesh sub-division unit 203A further sub-divides all sub-division surfaces within the base surface.

[0135] Here, if the base mesh sub-division unit 203A has not performed sub-division processing on the base surface once, it sub-divides the base surface.

[0136] In addition, the sub-division method is the same as the method described with reference to Fig. 9 above.

[0137] Specifically, when the base surface is not subdivided once, the base mesh subdivision unit 203A performs subdivision on the base surface. Fig. 9 The sub-division is performed as shown.

[0138] On the one hand, when the basic mesh sub-division unit 203A performs sub-division at least once, it sub-divides the sub-division plane into N2 pieces.

[0139] In Fig. 9 For example, when the face composed of vertex A_2, vertex B and vertex B_1 is divided from the base face, the N2 division method is also used to generate N2 faces.

[0140] When the sub-division processing is completed, the action proceeds to step S2205.

[0141] In step S2205, the basic mesh sub-division unit 203A adds 1 to Depth, and the operation returns to step S2202.

[0142] In addition, the basic mesh subdivision unit 203A can perform subdivision processing on all basic surfaces in a manner of subdividing the same upper limit of subdivision times mdu_max_subdivision_iteration_count. At this time, each subdivision processing can be configured to perform subdivision using N2 segmentation method according to the subdivision number mdu_subdivision_num, or can be configured to perform subdivision using Mid-edge segmentation method.

[0143] (Sub-division grid adjustment unit 203B)

[0144] Next, a specific example of the processing performed by the sub-division grid adjustment unit 203B will be described. Figure 11 to Figure 14 An example of the processing performed by the sub-division grid adjustment unit 203B will be described.

[0145] Fig.11 2 is a diagram showing an example of the functional blocks of the sub-division grid adjustment unit 203B.

[0146] The sub-division grid adjustment unit 203B may be configured to adjust the sub-division grid according to a sub-division adjustment method specified by the frame information output by the atlas data decoding unit 208 .

[0147] Here, adjustment refers to an operation of generating sub-divided meshes without holes in the mesh when the mesh is decoded by the mesh decoding unit 204 .

[0148] exist Fig.11 In the example of , the sub-division mesh adjustment unit 203B uses a method of further dividing the sub-division faces by moving the vertices of the sub-division faces of the initial sub-division mesh as the sub-division adjustment method.

[0149] like Fig.11 As shown, the sub-division grid adjustment unit 203B includes an edge division point moving unit 701 and a sub-division surface dividing unit 702 .

[0150] (Edge Division Point Moving Unit 701)

[0151] The edge division point moving unit 701 is configured to move the edge division point of the base surface to any one of the edge division points of the adjacent base surface with respect to the input initial sub-division surface, and output the sub-division surface.

[0152] The side division point moving unit 701 may be configured to directly output the initial sub-division mesh to the sub-division surface dividing unit 702 without moving the side division points when mdu_vertex_adaption_flag is 1.

[0153] Fig.12 is an example of moving the edge splitting point on the base surface ABC. Fig.12 As shown, the side division point moving unit 701 may be configured to move the side division point of the base surface ABC to the side division point of the nearest adjacent base surface.

[0154] (Sub-dividing surface dividing unit 702)

[0155] The sub-divided plane dividing unit 702 is configured to sub-divide the input sub-divided plane again and output a decoded sub-divided plane.

[0156] Fig.13 1 is a diagram showing an example of a case where the sub-divided surface X in the base surface is further sub-divided.

[0157] like Fig.13 As shown, the sub-divided surface dividing unit 702 may be configured to generate a new sub-divided surface in the base surface by connecting the vertices constituting the sub-divided surface X with the edge division points of the adjacent base surface.

[0158] Fig.14 1 is a diagram showing an example of a case where the above-mentioned sub-division process is performed on all sub-division surfaces.

[0159] (Sub-divided Vertex Normal Calculation Unit 203C)

[0160] The sub-division vertex normal calculation unit 203C is configured to output normals defined for each sub-division vertex (sub-division vertex normals) to sub-division vertices constituting the input sub-division surface.

[0161] Below, use Figure 15 to Figure 1 6 describes an example of processing performed by the sub-division vertex normal calculation unit 203C.

[0162] Fig.15This is a flowchart showing an example of the operation of the sub-division vertex normal calculation unit 203C.

[0163] like Fig.15 As shown, in step S203B-1, the sub-division vertex normal calculation unit 203C determines whether the calculation of sub-division vertex normals is completed for all patches.

[0164] If yes, the action ends, and if no, the action proceeds to step S203B-2.

[0165] In step S203B-2, the sub-division vertex normal calculation unit 203C determines the sub-division vertex normal calculation method according to mdu_normal_method, and the process proceeds to step S203B-3.

[0166] In step S203B-3, the sub-division vertex normal calculation unit 203C determines whether the calculation of sub-division vertex normals has been completed for all base surfaces in the patch.

[0167] If yes, the action proceeds to step S203B-1, and if no, the action proceeds to step S203B-4.

[0168] In step S203B-4, the sub-divided vertex normal calculation unit 203C determines whether calculation of sub-divided vertex normals has been completed for all sub-divided vertices in the base surface.

[0169] If yes, the action proceeds to step S203B-3, and if no, the action proceeds to step S203B-5.

[0170] In step S203B-5, the sub-divided vertex normal calculation unit 203C calculates the sub-divided vertex normals of the unprocessed sub-divided vertices, and the operation proceeds to step S203B-4.

[0171] exist Fig.16A and Fig. 16B In step S203B-5, the sub-division vertex normal calculation unit 203C calculates the sub-division vertex normal for the sub-division vertex X in the base plane ABC.

[0172] Reference Fig.16A An example of a method for calculating subdivided vertex normals will be described.

[0173] The subdivided vertex normal calculation unit 203C may be configured to calculate the normal n (vector) of the subdivided vertex X on the base plane ABC from the total value of the base point normals as shown in the following (Equation 1).

[0174] [Mathematical formula 1]

[0175] n=(a+b+c) / (||a+b+c||)...(Formula 1)

[0176] At this time, a, b, and c (vectors) are normals (normals at the base point) defined by the base points (vertices constituting the base surface).

[0177] In addition, refer to Fig. 16B Another example of the method of calculating the sub-division vertex normals will be described.

[0178] The subdivided vertex normal calculation unit 203C may be configured to calculate the normal n (vector) of the subdivided vertex X on the base plane ABC from the total value of the normals of the base planes adjacent to the base plane ABC, as shown in the following (Equation 2).

[0179] [Mathematical formula 2]

[0180] n=(a+b+c+d+e) / (||a+b+c+d+e||)...(Formula 2)

[0181] At this time, a, b, c, d, and e (vectors) are surface normals (vectors) of base surfaces adjacent to the base surface ABC.

[0182] In addition, the sub-division vertex normal calculation unit 203C may simply use the surface normal of the base surface ABC as the normal of the sub-division vertex X (sub-division point normal).

[0183] By calculating in this way, the normals of the sub-division points existing on the same base surface use approximately the same normal.

[0184] The sub-divided vertex normal calculation unit 203C may not perform the sub-divided vertex normal calculation process if the sub-divided vertex normals have already been calculated.

[0185] In addition, the sub-division vertex normal calculation unit 203C can also calculate the base point normal using the above-mentioned sub-division vertex normal calculation method.

[0186] At this time, as the normal line of the sub-division point and the normal line of the base point existing on the same base surface, substantially the same normal line is used.

[0187] (Displacement Decoding Unit 206)

[0188] like Fig.17 As shown, the displacement decoding unit 206 has a decoding omission determination unit 206A, an arithmetic decoding unit 206B, a context value update unit 206C, a context buffer 206D, a context selection unit 206E, a multi-numbering unit 206F, an inter-frame prediction unit 206G, a frame buffer 206H, an adder 206I, an inverse quantization unit 206J and a displacement prediction addition unit 206K.

[0189] The control information decoding unit 206A is configured to output control information by performing variable length decoding on the received displacement bit stream.

[0190] The arithmetic decoding unit 206B is configured to output binarized coefficient level values ​​by performing arithmetic decoding on the received displacement bit stream.

[0191] The context value updating unit 206C is configured to update the context value using the binarized coefficient level value and output it to the frame buffer 206D.

[0192] The context selection unit 206E is configured to generate and output a context value (output context value) using the context value read from the context buffer 206D, the bit position, and the syntax.

[0193] The multi-byte conversion unit 206F is configured to generate and output coefficient level values ​​by multi-byteing the binarized coefficient level values. The generated (calculated) coefficient level values ​​are also output to the context buffer 206D as bit positions and syntax.

[0194] The inter-frame prediction unit G is configured to generate and output a predicted displacement using a reference frame read from the frame buffer 206D.

[0195] The frame buffer H is configured to acquire and accumulate the decoded displacement. The frame buffer H is configured to output the decoded displacement of the corresponding vertex in the reference frame in response to control information (not shown).

[0196] The adder 206I is configured to obtain the inter prediction residual and the inter prediction displacement from the inter prediction unit 206G. The adder 206I is configured to add these and generate and output a quantized intra prediction residual. The generated (calculated) quantized intra prediction residual is also output to the frame buffer 206H.

[0197] The inverse quantization unit 206J is configured to inversely quantize the quantized intra prediction residual obtained from the addition unit 206I, and output the intra prediction residual. The details will be described later.

[0198] The displacement prediction addition unit 206K calculates an intra-frame prediction value by performing intra-frame prediction on the displacement of the sub-divided vertices according to the basic grid output from the basic grid decoding unit 202, and adds the calculated intra-frame prediction value to the intra-frame prediction residual output from the inverse quantization unit 206J, thereby decoding the displacement.

[0199] (Inverse quantization unit 206J)

[0200] Fig.18 2 is a diagram showing an example of a flowchart, showing an example of the operation of the inverse quantization unit 206J. Fig.18 An example of the operation of the inverse quantization unit 206J will be described.

[0201] like Fig.18 As shown, in step S203J-1, the inverse quantization unit 206J determines whether mdu_lod_change_flag[tileID][patchIdx]!=0.

[0202] If yes, the action goes to step S203J-3, and if no, the action goes to step S203J-2.

[0203] In step S203J-2, the inverse quantization unit 206J inverse quantizes all displacement amounts and ends the processing.

[0204] At this time, the inverse quantization unit 206J can use the uniform quantization disclosed in reference 2 "Jinhua Yu, "Advantages of uniform scalar dead-zone quantization in image coding system," in 2004 International Conference on Communications, Circuits and Systems (IEEE Cat. No. 04EX914). IEEE, 2004, vol. 2, pp. 805-808" for inverse quantization, or use other known inverse quantization methods for inverse quantization.

[0205] In step S203J-3, the inverse quantization unit 206J determines whether the processing is completed for all patches.

[0206] If yes, the action ends, and if no, the action goes to step S203J-4.

[0207] In step S203J-4, the inverse quantization unit 206J determines whether the processing is completed for all levels in the patch.

[0208] If yes, the action goes to step S203J-3, and if no, the action goes to step S203J-5.

[0209] In step S203J-5, the inverse quantization unit 206J determines the quantization value of the current level according to mdu_lod_qp[tileID][patchIdx][i] and proceeds to step S203J-6.

[0210] In step S203J-6, the inverse quantization unit 206J determines whether the inverse quantization process has been completed for all displacement amounts of the current level.

[0211] If yes, the action goes to step S203J-4, and if no, the action goes to step S203J-7.

[0212] In step S203J-7, the inverse quantization unit 206J uses the quantization value obtained in step S203J-5 to perform inverse quantization of the displacement amount and proceeds to step S203J-6.

[0213] At this time, the inverse quantization unit 206J may use the uniform quantization disclosed in the above-mentioned reference 2 for inverse quantization, or may use other known inverse quantization methods for inverse quantization.

[0214] According to the present embodiment, by using the same normals as the sub-division vertices within the same base plane, the displacement amount can be efficiently encoded.

[0215] Furthermore, according to the present embodiment, since the decoding method can be set according to the characteristics of the mesh within the patch, the number of bits of the shift amount can be reduced.

[0216] The above-mentioned trellis encoding device 100 and trellis decoding device 200 can be implemented as a program product that causes a computer to execute each function (each process).

[0217] Industrial Applicability

[0218] In addition, according to this embodiment, for example, since comprehensive service quality can be improved in moving image communications, it can contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Improve the infrastructure of the information environment, promote sustainable industrialization and achieve expansion of innovation."

[0219] Explanation of symbols

[0220] 1: Grid processing system;

[0221] 100: grid encoding device;

[0222] 200: grid decoding device;

[0223] 201: multiplexing and separation unit;

[0224] 202: basic grid decoding unit;

[0225] 203: sub-division part;

[0226] 203A: basic grid subdivision part;

[0227] 203A1: basic surface division number buffer unit;

[0228] 203A2: Reference to the number of base surface divisions;

[0229] 203A3: Basic surface segmentation number prediction unit;

[0230] 203A4: Addition Department;

[0231] 203A5: basic surface segmentation;

[0232] 203B: sub-division grid adjustment unit;

[0233] 701: edge segmentation point moving unit;

[0234] 702: sub-division surface division part;

[0235] 203C: sub-division vertex normal calculation unit;

[0236] 204: grid decoding unit;

[0237] 205: Patch Integration Department;

[0238] 206: displacement decoding unit;

[0239] 206A: control information decoding unit;

[0240] 206B: Arithmetic decoding unit;

[0241] 206C: context value updating unit;

[0242] 206D: context buffer;

[0243] 206E: context selection unit;

[0244] 206F: Multi-numbering unit;

[0245] 206F2: coefficient level value decoding unit;

[0246] 206G: inter-frame prediction unit;

[0247] 206H: frame buffer;

[0248] 206I: adder;

[0249] 206J: Anti-quantification Department;

[0250] 206K: displacement prediction addition unit;

[0251] 207: Image decoding unit;

[0252] 208: Atlas data decoding department.

Claims

1. A grid decoding device, characterized in that: have: A base surface segmentation unit, configured to sub-divide the base mesh and output an initial sub-divided mesh; a sub-division vertex normal calculation unit configured to calculate a normal for each vertex constituting the initial sub-division mesh; and The sub-divided mesh adjustment unit is configured to adjust the sub-divided mesh by dividing the sub-divided faces after moving the positions of the vertices of the faces constituting the initial sub-divided mesh.

2. The grid decoding device according to claim 1, characterized in that: The sub-division vertex normal calculation unit is configured to calculate the sub-division vertex normals in the base surface based on a total value of base point normals defined by base points that are vertices constituting the base surface in the initial sub-division mesh.

3. The grid decoding device according to claim 1, characterized in that: The sub-division vertex normal calculation unit is configured to calculate the sub-division vertex normals in the base surface based on a total value of normals of base surfaces adjacent to the base surface in the initial sub-division mesh.

4. The grid decoding device according to claim 1, characterized in that: The sub-division vertex normal calculation unit is configured to use the same normals of base points and sub-division points as vertices in the base surface in the initial sub-division mesh.

5. The grid decoding device according to claim 1, characterized in that: The sub-division vertex normal calculation unit is configured to use, when normals of a base point and a sub-division point in a base plane in the initial sub-division mesh have already been calculated in a decoding order, the already calculated normals as the normals of the base point and the sub-division point.

6. A grid decoding method, characterized in that: have: A process of subdividing the base mesh and outputting an initial subdivided mesh; A step of calculating a normal for each vertex constituting the initial sub-divided mesh; and A step of adjusting the sub-divided meshes by dividing the sub-divided faces after moving the positions of the vertices of the faces constituting the initial sub-divided meshes.

7. A program product, comprising a computer program, which, when executed, implements the function of a trellis decoding device, characterized in that: The grid decoding device comprises: A base surface segmentation unit, configured to sub-divide the base mesh and output an initial sub-divided mesh; a sub-division vertex normal calculation unit configured to calculate a normal for each vertex constituting the initial sub-division mesh; and The sub-divided mesh adjustment unit is configured to adjust the sub-divided mesh by dividing the sub-divided faces after moving the positions of the vertices of the faces constituting the initial sub-divided mesh.

Images