Mesh decoding device, mesh decoding method, and program product

Abstract

A mesh decoding device (200) according to the present invention is provided with: an ordering unit (202B2) that gathers, as one vertex, repeated vertices, which are a plurality of vertices having consistent coordinates in a base mesh to be decoded, and then rearranges the same in a prescribed order.

Description

Technical Field

[0001] The present invention relates to a grid decoding device, a grid encoding device, a grid decoding method and a program product. Background Art

[0002] Non-patent document 1 discloses a technology for encoding a grid using non-patent document 2.

[0003] Prior art literature

[0004] Non-patent literature

[0005] Non-patent literature 1: Cfp for Dynamic Mesh Coding, ISO / IECJTC1 / SC29 / WG7 N00231, MPEG136-Online

[0006] Non-patent document 2: Google Draco, accessed on May 26, 2022 [Online], https: / / google.github.io / draco Summary of the invention

[0007] Problems to be solved by the invention

[0008] However, in the prior art, since the coordinates or connection information of all vertices constituting the dynamic mesh are losslessly encoded, the amount of information cannot be reduced even under the condition of allowing loss, and there is a problem of low encoding efficiency.

[0009] In addition, in the prior art, there is a problem that the motion vector coding efficiency is reduced due to the presence of repeated vertices in the decoded basic mesh. Therefore, the present invention is made in view of the above-mentioned problems, and its purpose is to provide a mesh decoding device, a mesh coding device, a mesh decoding method and a program product that can improve the coding efficiency of the mesh.

[0010] Means for solving problems

[0011] A first technical solution of the present invention is a mesh decoding device, which has: a sorting unit that aggregates multiple vertices with consistent coordinates in a decoded basic mesh, that is, repeated vertices, into one vertex and then rearranges them into a predetermined order.

[0012] A second technical solution of the present invention is a mesh decoding method, which has a step of aggregating a plurality of vertices having the same coordinates with respect to a decoded basic mesh, that is, repeated vertices, into one vertex and then rearranging them into a predetermined order.

[0013] The third technical solution of the present invention is a program product, which includes a program that enables a computer to function as a grid decoding device. The main point of the program product is that the grid decoding device has a sorting unit, which aggregates multiple vertices, i.e., repeated vertices, whose coordinates are consistent with those of the decoded basic grid into one vertex, and then rearranges them into a specified order.

[0014] Effects of the Invention

[0015] According to the present invention, a grid decoding device, a grid encoding device, a grid decoding method and a program product that can improve the coding efficiency of the grid can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a diagram showing an example of the configuration of a mesh processing system 1 according to an embodiment.

[0017] Figure 2 This is a diagram showing an example of functional blocks of a trellis decoding device 200 according to an embodiment.

[0018] Figure 3A This is a diagram showing an example of a basic grid and sub-divided grids.

[0019] Figure 3B This is a diagram showing an example of a basic grid and sub-divided grids.

[0020] Figure 4 This is a diagram showing an example of the syntax structure of the basic grid bit stream.

[0021] Figure 5 This is a diagram showing an example of the syntax structure of BPH.

[0022] Figure 6 This is a diagram showing an example of a functional block of the basic trellis decoding unit 202 of the trellis decoding device 200 according to one embodiment.

[0023] Figure 7 This is a diagram showing an example of a functional block of the intra-frame decoding unit 202B of the base trellis decoding unit 202 of the trellis decoding device 200 according to one embodiment.

[0024] Figure 8 This is a flowchart showing an example of the operation of the arranging unit 202B2 of the intra decoding unit 202B of the trellis decoding device 200 according to one embodiment.

[0025] Fig. 9 This is a diagram for explaining an example of the operation of the arranging unit 202B2 of the intra decoding unit 202B of the trellis decoding device 200 according to the embodiment.

[0026] Fig.10 This is a diagram showing an example of the correspondence relationship between the vertices of the base mesh of the P frame and the vertices of the base mesh of the I frame.

[0027] Fig.11 This is a diagram showing an example of a functional block of the inter-frame decoding unit 202E of the basic trellis decoding unit 202 of the trellis decoding device 200 according to one embodiment.

[0028] Fig.12 This is a diagram showing an example of a functional block of the basic trellis decoding unit 202 of the trellis decoding device 200 according to one embodiment.

[0029] Fig.13 is a graph representing the vertices of a base mesh of reference frames with multiple motion vectors.

[0030] Fig.14 This is a diagram for explaining an example of a method for calculating the MVP of a vertex of a decoding target implemented by the motion vector prediction unit 202E3 of the inter decoding unit 202E of the base trellis decoding unit 202 of the trellis decoding device 200 according to one embodiment.

[0031] Fig.15 A flowchart is shown showing an example of the operation of the motion vector prediction unit 202E3 of the inter-frame decoding unit 202E of the trellis decoding device 200 according to one embodiment.

[0032] Fig.16 1 is a diagram showing an example of a functional block of the sub-division unit 203 of the trellis decoding device 200 according to an embodiment.

[0033] Fig.17 This is a diagram showing an example of a functional block of a base trellis sub-dividing unit 203A of the sub-dividing unit 203 of the trellis decoding device 200 according to an embodiment.

[0034] Fig.18 This is a diagram for explaining an example of a base plane division method implemented by a base plane division unit 203A5 of a base mesh sub-division unit 203A of a sub-division unit 203 of a mesh decoding device 200 according to an embodiment.

[0035] Fig.19 This is a flowchart showing an example of the operation of the base grid sub-dividing unit 203A of the sub-dividing unit 203 of the grid decoding device 200 according to one embodiment.

[0036] Fig. 20 1 is a diagram showing an example of a functional block of a sub-division grid adjustment unit 203B of the sub-division unit 203 of the grid decoding device 200 according to an embodiment.

[0037] Fig.21 1 is a diagram showing an example of a case where the side division point moving unit 701 of the sub division grid adjustment unit 203B of the sub division unit 203 of the grid decoding device 200 according to one embodiment moves the side division points on the base plane ABC.

[0038] Fig. 22 1 is a diagram showing an example of a case where the sub-division plane X in the base plane is sub-divided again by the sub-division plane dividing unit 702 of the sub-division grid adjustment unit 203B of the sub-division unit 203 of the grid decoding device 200 according to one embodiment.

[0039] Fig.23 1 is a diagram showing an example of a case where all sub-division planes are sub-divided again by the sub-division plane dividing unit 702 of the sub-division grid adjusting unit 203B of the sub-division unit 203 of the trellis decoding device 200 according to one embodiment.

[0040] Fig.24 This is a diagram showing an example of functional blocks of the displacement amount decoding unit 206 of the trellis decoding device 200 according to one embodiment (a case where inter-frame prediction is performed in a spatial region).

[0041] Fig.25 This is a diagram showing an example of the structure of a displacement bit stream.

[0042] Fig.26 This is a diagram showing an example of the syntax structure of DPS.

[0043] Fig. 27 This is a diagram showing an example of the syntax structure of DPH.

[0044] Fig.28 This is a diagram for explaining an example of the correspondence relationship between sub-division vertices between a reference frame and a decoding target frame when inter-frame prediction is performed in a spatial region.

[0045] Fig.29 This is a diagram showing an example of a functional block of the displacement amount decoding unit 206 of the trellis decoding device 200 according to an embodiment (a case where inter-frame prediction is performed in the frequency domain).

[0046] Fig.30 This is a diagram for explaining an example of the frequency correspondence relationship between a reference frame and a decoding target frame when inter-frame prediction is performed in the frequency domain.

[0047] Fig.31 This is a flowchart showing an example of the operation of the displacement amount decoding unit 206 of the trellis decoding device 200 according to one embodiment.

[0048] Fig.32This is a diagram showing an example of functional blocks of the displacement amount decoding unit 206 according to the first modification.

[0049] Fig.33 This is a diagram showing an example of functional blocks of the displacement amount decoding unit 206 involved in the second modification.

[0050] Explanation of symbols

[0051] 1: Grid processing system;

[0052] 100: grid encoding device;

[0053] 200: grid decoding unit;

[0054] 201: multiplexing and separation unit;

[0055] 202: basic grid decoding unit;

[0056] 202A: separation part;

[0057] 202B: intra-frame decoding unit;

[0058] 202B1: arbitrary intra-frame decoding unit;

[0059] 202B2: Sorting Department;

[0060] 202C: grid buffer section;

[0061] 202D: connection information decoding unit;

[0062] 202E: inter-frame decoding unit;

[0063] 202E1: motion vector decoding unit;

[0064] 202E2: motion vector buffer unit;

[0065] 202E3: Motion vector prediction unit;

[0066] 202E4: motion vector calculation unit;

[0067] 202E5: Adder;

[0068] 202E10: Extended vertex calculation unit;

[0069] 203: sub-division part;

[0070] 203A: basic grid subdivision part;

[0071] 203A1: basic surface division number buffer unit;

[0072] 203A2: Reference to the number of base surface divisions;

[0073] 203A3: Basic surface segmentation number prediction unit;

[0074] 203A4: Addition department;

[0075] 203A5: basic surface segmentation;

[0076] 203B: sub-division grid adjustment unit;

[0077] 701: edge segmentation point moving unit;

[0078] 702: sub-division surface division part;

[0079] 204: grid decoding unit;

[0080] 205: Patch Integration Department;

[0081] 206: displacement decoding unit;

[0082] 206A: decoding unit;

[0083] 206B, 2063: Dequantization Department;

[0084] 206C, 2064: inverse wavelet transform unit;

[0085] 206D: Adder;

[0086] 206E: inter-frame prediction unit;

[0087] 206F: frame buffer;

[0088] 206G: Anti-DCT unit;

[0089] 2062: image development unit;

[0090] 207, 2061: Image decoding unit. DETAILED DESCRIPTION

[0091] Below, while referring to the attached Figure 1 The embodiments of the present invention are described below. In addition, the structural elements in the following embodiments can be appropriately replaced with existing structural elements, etc., and various changes including combinations with other existing structural elements can be made. Therefore, the content of the invention described in the claims is not limited based on the description of the following embodiments.

[0092] <First Embodiment>

[0093] Below, refer to Figure 1 to Figure 31 , the grid processing system involved in this embodiment is described.

[0094] Figure 11 is a diagram showing an example of the structure of the grid processing system 1 according to the present embodiment. Figure 1 As shown, the grid processing system 1 has a grid encoding device 100 and a grid decoding device 200 .

[0095] Figure 2 This is a diagram showing an example of functional blocks of the trellis decoding device 200 according to the present embodiment.

[0096] like Figure 2 As shown, the trellis decoding device 200 includes a demultiplexing unit 201 , a basic trellis decoding unit 202 , a sub-division unit 203 , a trellis decoding unit 204 , a patch integration unit 205 , a displacement decoding unit 206 and a video decoding unit 207 .

[0097] Here, the basic mesh decoding unit 202 , the sub-division unit 203 , the mesh decoding unit 204 , and the displacement decoding unit 206 are configured to perform processing in units of patches obtained by dividing the mesh, and then the patch integration unit 205 may integrate the processing results.

[0098] exist Figure 3A In the example, the mesh is divided into patch 1 consisting of base surfaces 1 and 2, and patch 2 consisting of base surfaces 3 and 4.

[0099] The demultiplexing unit 201 is configured to separate the multiplexed bit stream into a base grid bit stream, a displacement bit stream, and a texture bit stream.

[0100] <Basic grid decoding unit 202>

[0101] The base grid decoding unit 202 is configured to decode the base grid bit stream, generate and output a base grid.

[0102] Here, the basic mesh is composed of a plurality of vertices in a three-dimensional space and edges connecting the plurality of vertices.

[0103] In addition, if Figure 3A As shown, the basic mesh is constructed by combining basic faces expressed by three vertices.

[0104] The basic grid decoding unit 202 can be configured to decode the basic grid bit stream using Draco as shown in non-patent document 2, for example.

[0105] In addition, the basic grid decoding unit 202 may be configured to generate “subdivision_method_id” described later as control information for controlling the type of the subdivision method.

[0106] Below, refer to Figure 4-5 , the control information decoded by the basic grid decoding unit 202 is explained.

[0107] Figure 4 This is a diagram showing an example of the syntax structure of the basic grid bit stream.

[0108] like Figure 4 As shown, first, the base grid bitstream may include a BPH (Base Patch Header) which is a collection of control information corresponding to the base grid patch. Second, the base grid bitstream may include base grid patch data encoding the base grid patch after the BPH.

[0109] As described above, the basic mesh bitstream has a structure in which each BPH corresponds to each patch data. Figure 4 The structure is just an example, and as long as the structure is such that BPH corresponds to each patch data, elements other than the above-mentioned ones can be added as structural elements of the basic mesh bit stream.

[0110] For example, Figure 4 As shown, the basic mesh bitstream may include an SPS (Sequence Parameter Set), an FH (Frame Header) which is a set of control information corresponding to a frame, and an MH (Mesh Header) which is control information corresponding to a mesh.

[0111] Figure 5 is a diagram showing an example of the syntax structure of BPH. Here, as long as the syntax functions are the same, you can use Figure 5 The syntax companions shown have different syntax names.

[0112] exist Figure 5 In the syntax structure of the BPH shown, the descriptor column indicates how each syntax is encoded. In addition, ue(v) indicates an uncoded 0th-order exponential-Golomb code, and u(n) indicates an n-bit flag.

[0113] The BPH includes at least a control signal (mdu_face_count_minus1) that specifies the number of base faces contained in the base mesh patch.

[0114] In addition, the BPH includes at least a control signal (mdu_subdivision_method_id) that specifies the type of subdivision method of the base mesh for each base patch.

[0115] In addition, the BPH may include a control signal (mdu_subdivision_num_method_id) that specifies the type of subdivision number generation method for each base mesh patch. For example, when mdu_subdivision_num_method_id=0, it is defined as generating the subdivision number of the base surface by predicting the segmentation residual, and when mdu_subdivision_num_method_id=1, it is defined as recursively generating the subdivision number of the base surface.

[0116] When the BPH generates the number of subdivisions of the base plane using the prediction partition residual, it may include a control signal (mdu_subdivision_resuiduals) that specifies the prediction partition residual of the base plane for each index i (i=0, . . . , mdu_face_count_minus1).

[0117] When recursively generating the number of sub-divisions of the base surface, the BPH may include a control signal (mdu_max_depth) for identifying an upper limit of the number of sub-divisions to be recursively performed for each base mesh patch.

[0118] The BPH may include a control signal (mdu_subdivision_flag) that specifies whether to recursively sub-divide the base plane for each index i (i=0, ..., mdu_face_count_minus1) and j (j=0, ..., mdu_subdivision_depth_index).

[0119] like Figure 6 As shown, the basic mesh decoding unit 202 includes a separation unit 202A, an intra-frame decoding unit 202B, a mesh buffer unit 202C, a connection information decoding unit 202D, and an inter-frame decoding unit 202E.

[0120] The separation unit 202A is configured to classify the base grid bit stream into a bit stream of an I frame (reference frame) and a bit stream of a P frame.

[0121] (Intra-frame decoding unit 202B)

[0122] The intra-frame decoding unit 202B is configured to decode the coordinates and connection information of the vertices of the I frame from the bit stream of the I frame using, for example, Draco as described in Non-Patent Document 2.

[0123] Figure 7 This is a diagram showing an example of functional blocks of the intra-frame decoding unit 202B.

[0124] like Figure 7 As shown, the intra-frame decoding unit 202B includes an arbitrary intra-frame decoding unit 202B1 and a sorting unit 202B2.

[0125] The arbitrary intra-frame decoding unit 202B1 is configured to decode the coordinates and connection information of the non-sequential vertices of the I frame from the bit stream of the I frame using an arbitrary method including Draco described in Non-Patent Document 2.

[0126] The sorting unit 202B2 is configured to sort out the out-of-order vertices into a predetermined order and output the vertices.

[0127] As the prescribed order, for example, a Morton code order or a raster scan order may be used.

[0128] In addition, the sorting unit 202B2 may collect a plurality of vertices having the same coordinates in the decoded basic mesh, that is, duplicate vertices, into one vertex, and then rearrange the vertices into a predetermined order.

[0129] Here, refer to Figure 8 An example of the operation of the organizing unit 202B2 will be described.

[0130] like Figure 8 As shown, in step S101, the sorting unit 202B2 determines the list of duplicate vertices. That is, the sorting unit 202B2 determines the list of duplicate vertices existing in the decoded basic mesh. Here, as a method of determining the list of duplicate vertices, at least two methods are envisioned.

[0131] (Determination method 1)

[0132] For example, the organizing unit 202B2 is configured to decode the above-mentioned duplicate vertex list from the bit stream of the I frame in the determination method 1.

[0133] Specifically, first, the organizing unit 202B2 decodes a flag indicating whether or not there are duplicate vertices based on the bit stream of the I frame.

[0134] Second, if the flag is TRUE, the sorting unit 202B2 decodes the number D of repeated vertices.

[0135] Third, the sorting unit 202B2 decodes the pairs of index A(k) and index B(k) of the vertices existing as duplicate vertices one by one according to the bit stream of the I frame and stores them in a specific buffer. A(k), B(k) and k (k=1, 2, ..., D) are integers. Here, the list of pairs is stored in the above-mentioned specific buffer in the order of A(k)→B(k).

[0136] Furthermore, if the above flag is FALSE, the sorting unit 202B2 makes the above specific buffer empty.

[0137] According to the determination method 1, since calculation is not performed, an effect of avoiding an increase in the amount of calculation can be expected.

[0138] (Determination method 2)

[0139] The organizing unit 202B2 is configured to calculate a list of duplicate vertices by searching for duplicate vertices in the decoded basic mesh in the determination method 2.

[0140] Specifically, first, the organizing unit 202B2 searches for the index of the vertices (duplicate vertices) with the same coordinates based on the decoded geometric information of the basic mesh and stores the index in the buffer.

[0141] In addition, the input to the sorting unit 202B2 is the index (decoding order) and position coordinates of each vertex of the decoded basic mesh, and the output to the sorting unit 202B2 is a list of pairs of indices A(k) and B(k) of vertices existing as duplicate vertices. A(k), B(k) and k (k=1, 2, ..., D) are integers. Here, the pair list is stored in the above-mentioned specific buffer in the order of A(k)→B(k).

[0142] When there is no vertex with matching coordinates in the above-mentioned search, the organizing unit 202B2 makes the specific buffer empty (that is, sets D=0).

[0143] According to the determination method 2, since data is not acquired based on the bit stream, an effect of avoiding an increase in the bit rate can be expected.

[0144] In addition, in any of the above-mentioned determination methods 1 and 2, since B(k) is decoded before A(k), the relationship of A(k)>B(k) is achieved.

[0145] In step S102 , the organizing unit 202B2 updates the mesh based on the repeated vertices.

[0146] The sorting unit 202B2 determines the list of duplicate vertices in step S101, and then, Fig. 9 As shown, all or part of the repeated vertices are integrated and the connectivity is updated.

[0147] As a method of implementing the action of updating the connectivity, at least two methods are conceivable.

[0148] (Implementation method 1)

[0149] For example, in implementation method 1, the sorting unit 202B2 processes all duplicate vertices stored in the specific buffer in step S101 described above.

[0150] Specifically, in implementation method 1, first, the organizing unit 202B2 deletes D vertices A(k) (k=1, 2, . . . , D) from the set of vertices of the base mesh.

[0151] Second, the organizing unit 202B2 sequentially switches the index of the vertex A(k) to B(k) in the connectivity. However, if the index of the UV coordinate exists in the connectivity, the organizing unit 202B2 may not change the index.

[0152] Third, the sorting unit 202B2 resets the indexes in a manner that eliminates discontinuous indexes. For example, the sorting unit 202B2 increases all the indexes after the deleted A(k) by one. The sorting unit 202B2 repeats the increase operation until k=1-D.

[0153] In implementation method 1, the number of vertices is reduced, but the number of faces and the number of UV coordinates may not be changed.

[0154] (Implementation method 2)

[0155] In implementation method 2, the organizing unit 202B2 decodes information related to unprocessed duplicate vertices (vertices that do not need to be processed) among the duplicate vertices stored in the specific buffer in the above step S101 based on the bit stream, integrates duplicate vertices other than the unprocessed duplicate vertices, and updates connectivity.

[0156] Specifically, in implementation method 2, first, the sorting unit 202B2 decodes a flag indicating the presence or absence of unprocessed vertices (duplicate vertices) from the bit stream.

[0157] Second, if the flag is TRUE, the sorting unit 202B2 decodes the number N of vertices that do not need to be processed.

[0158] Third, the sorting unit 202B2 decodes the indexes A(k) of the vertices that do not need to be processed or the order k of the indexes A(k) one by one, where k=1, 2, ..., N.

[0159] Fourthly, the sorting unit 202B2 deletes the pair including A(k) or the k-th pair among the pairs of duplicate vertices stored in the specific buffer in the above-mentioned step S101.

[0160] Fifth, the sorting unit 202B2 uses the last updated specific buffer to implement the above-mentioned implementation method 1.

[0161] In implementation method 2, the number of vertices is reduced, but the number of faces and the number of UV coordinates may not be changed.

[0162] The mesh buffer unit 202C is configured to accumulate the coordinates and connection information of the vertices of the I frame decoded by the intra-frame decoding unit 202B. Here, a specific buffer may be provided to store pairs of indexes A(k) and B(k) of vertices existing as duplicate vertices in a predetermined order.

[0163] The connection information decoding unit 202D is configured to use the connection information of the I frame extracted from the mesh buffer unit 202C as the connection information of the P frame.

[0164] The inter-frame decoding unit 202E is configured to decode the coordinates of the vertices of the P frame by adding the coordinates of the vertices of the I frame taken out from the mesh buffer unit 202C to the motion vectors decoded from the bit stream of the P frame.

[0165] Furthermore, the inter-frame decoding unit 202E can adjust the indices of the vertices of the P frame using the pair of indices A(k) and B(k) of the vertices existing as duplicate vertices stored in the specific buffer.

[0166] In this embodiment, if Fig.10 As shown, there is a correspondence between the vertices of the base mesh of the P frame and the vertices of the base mesh of the I frame (reference frame). Here, the motion vector decoded by the inter-frame decoding unit 202E is a differential vector between the coordinates of the vertices of the base mesh of the P frame and the coordinates of the vertices of the base mesh of the I frame.

[0167] As described later, the inter-frame decoding unit 202E may be configured to calculate the indices or coordinates of the vertices added to the base mesh of the inter-frame of the decoding target using the vertices of the base mesh of the reference frame having a plurality of motion vectors.

[0168] (Inter-frame decoding unit 202E)

[0169] Fig.11 This is a diagram showing an example of functional blocks of the inter-frame decoding unit 202E.

[0170] like Fig.11 As shown, the inter-frame decoding unit 202E includes a motion vector residual decoding unit 202E1, a motion vector buffer unit 202E2, a motion vector prediction unit 202E3, a motion vector calculation unit 202E4, and an adder 202E5.

[0171] Furthermore, when the sorting unit 202B2 implements the implementation method 1 (i.e., when all repeated vertices are integrated), Fig.12 As shown, the base mesh decoding unit 202 may include an extended vertex calculation unit 202E10 before the inter-frame decoding unit 202E.

[0172] Here, the extended vertex calculation unit 202E10 is configured to calculate the index of the vertex added to the base mesh of the inter-frame of the decoding target, using the vertices of the base mesh of the reference frame having a plurality of motion vectors.

[0173] in addition, Fig.13 The vertices of the base mesh representing the reference frame with the plurality of motion vectors described above.

[0174] The extended vertex calculation unit 202E10 decodes the index of the vertex added in the base mesh of the inter frame of the decoding target from the bit stream through the vertices of the base mesh of the reference frame having a plurality of motion vectors.

[0175] First, the extended vertex calculation unit 202E10 decodes, from the bit stream, a flag indicating whether or not there are vertices of the base mesh of the reference frame having a plurality of motion vectors that do not need to be processed.

[0176] Second, if the flag is TRUE, the extended vertex calculation unit 202E10 decodes the number N of vertices added in the base mesh of the inter-frame of the decoding target through the vertices of the base mesh of the reference frame that does not need to be processed having a plurality of motion vectors.

[0177] That is, the extended vertex calculation unit 202E10 adds one vertex to the base mesh of the inter-frame of the decoding target through the vertices of the base mesh of the reference frame with two motion vectors, and adds two vertices to the base mesh of the inter-frame of the decoding target through the vertices of the base mesh of the reference frame with three motion vectors.

[0178] Third, the extended vertex calculation unit 202E10 decodes the index A(k) of the vertex added to the base mesh of the inter-frame of the decoding target and the index B(k) of the vertex of the base mesh of the corresponding reference frame that does not need to be processed or the order k of the index B(k) one by one, and stores them in the above-mentioned specific buffer, where k=1, 2, ..., N.

[0179] Furthermore, if the flag indicating whether or not there is a vertex of the base mesh having the reference frame having the above-mentioned plurality of motion vectors is FALSE, the extended vertex calculation unit 202E10 makes the specific buffer empty.

[0180] Then, the expanded vertex calculation unit 202E10 deletes the pair including A(k) or the k-th pair among the pairs of duplicate vertices stored in the specific buffer in step S101 described above.

[0181] The motion vector residual decoding unit 202E1 is configured to generate MVR (Motion Vector Residual) from the bit stream of the P frame.

[0182] Here, MVR is a motion vector residual representing the difference between MV (Motion Vector) and MVP (Motion Vector Prediction). MV is a difference vector (motion vector) between the coordinates of the vertex of the corresponding I frame and the coordinates of the vertex of the P frame. MVP is the value of the MV of the vertex of the prediction target using MV (predicted value of the motion vector).

[0183] The motion vector buffer unit 202E2 is configured to sequentially store the MV output by the motion vector calculation unit 202E4.

[0184] The motion vector prediction unit 202E3 is configured to obtain the decoded MV from the motion vector buffer unit 202E2 for the vertex connected to the vertex of the decoding target, such as Fig.14 As shown, the MVP of the vertex of the decoding target is output using all or part of the acquired decoded MV.

[0185] The motion vector calculation unit 202E4 is configured to add the MVR generated by the motion vector residual decoding unit 202E1 to the MVP output from the motion vector prediction unit 202E3, and output the MV of the vertex of the decoding target.

[0186] The adder 202E5 is configured to add the coordinates of the vertex corresponding to the vertex of the decoding target obtained from the decoded basic mesh of the corresponding I frame (reference frame) and the motion vector MV output from the motion vector calculation unit 202E3, and output the coordinates of the vertex of the decoding target.

[0187] The following describes the details of each part of the inter-frame decoding unit 202E.

[0188] exist Fig.15 , a flowchart showing an example of the operation of the motion vector prediction unit 202E3 is shown.

[0189] like Fig.15 As shown, in step S1001, the motion vector prediction unit 202E3 sets MVP and N to 0.

[0190] In step S1002, the motion vector prediction unit 202E3 obtains a set of MVs of vertices around the vertex of the decoding target from the motion vector buffer unit 202E2, identifies vertices for which subsequent processing has not been completed, and switches to No; if subsequent processing has been completed for all vertices, switches to Yes.

[0191] In step S1003 , the motion vector prediction unit 202E3 switches to No when the MV of the vertex of the processing target has not been decoded, and switches to Yes when the MV of the vertex of the processing target has been decoded.

[0192] In step S1004, the motion vector prediction unit 202E3 adds MV and MVP, and increases N by 1.

[0193] In step S1005, the motion vector prediction unit 202E3 divides MVP by N when N is greater than 0 and outputs the result, and outputs 0 and terminates the process when N is 0.

[0194] That is, the motion vector prediction unit 202E3 is configured to output the MVP of the decoding target by averaging the decoded motion vectors of the vertices located around the vertex of the decoding target.

[0195] Furthermore, the motion vector prediction unit 202E3 may be configured to set the MVP to 0 when the set of decoded motion vectors is an empty set.

[0196] The motion vector calculation unit 202E4 may be configured to calculate the MV of the vertex of the decoding target based on the MVP output by the motion vector prediction unit 202E3 and the MVR generated by the motion vector residual decoding unit 202E1 using equation (1).

[0197] MV(k)=MVP(k)+MVR(k) … (1)

[0198] Here, k is the index of the vertex. MV, MVR, and MVP are vectors having an x ​​component, a y component, and a z component.

[0199] According to the above configuration, since MVP is used instead of MV and only MVR is encoded, the effect of improving the encoding efficiency can be expected.

[0200] The adder 202E5 is configured to calculate the coordinates of the vertex by adding the MV of the vertex calculated by the motion vector calculation unit 202E4 and the coordinates of the vertex of the reference frame corresponding to the vertex, thereby maintaining the connection information (Connectivity) in the reference frame.

[0201] Specifically, the adder 202E5 can be configured to use equation (2) to calculate the coordinate v' of the kth vertex. i (k) Perform calculations.

[0202] v' i (k) = v' j (k)+MV(k) … (2)

[0203] Here, v' i (k) is the coordinate of the kth vertex decoded in the decoding target frame, v' j (k) is the coordinate of the k-th vertex after decoding of the reference frame, MV(k) is the k-th MV of the frame of the decoding target, k=1, 2, . . . , K.

[0204] However, when the arrangement unit 202B2 implements the implementation method 1, the motion vector is decoded for the vertex expanded in the base mesh of the inter frame of the decoding target decoded by the expanded vertex calculation unit 202E10.

[0205] For example, when k in equation (2) is consistent with B(k) in a pair of A(k) and B(k) stored in a specific buffer, adder 202E5 uses equation (2) for the first motion vector, but uses equation (3) instead of equation (2) for the second and subsequent u-th motion vectors.

[0206] v' i (A(k)) = v' j (B(k))+MV u (B(k)) … (3)

[0207] In addition, the connection information of the decoding target frame is the same as the connection information of the reference frame.

[0208] Furthermore, since the motion vector prediction unit 202E3 calculates the MVP using the decoded MV, the decoding order does not affect the MVP.

[0209] The decoding order is the order of decoding the vertices of the base mesh of the reference frame. Generally speaking, if a decoding method is used that uses a constant repetition pattern to increase the base surface one by one from the edge as the starting point, the order of the vertices of the decoded base mesh is determined by the decoding process.

[0210] For example, the motion vector prediction unit 202E3 may use Edgebreaker (compression algorithm) to determine the order of decoding vertices in the base mesh of the reference frame.

[0211] According to the above configuration, since the MV from the reference frame is encoded instead of the coordinates of the vertices, it is possible to expect an effect of improving the encoding efficiency.

[0212] <Sub-division Unit 203>

[0213] The sub-division unit 203 is configured to generate and output the added sub-division vertices and connection information thereof based on the basic mesh decoded by the basic mesh decoding unit 202 according to the sub-division method indicated by the control information.

[0214] Here, the base mesh, the added sub-division vertices, and the connection information thereof are collectively referred to as a “sub-division mesh”.

[0215] The sub-division unit 202 is configured to specify the type of sub-division method based on subdivision_method_id, which is control information generated by decoding the basic grid bit stream.

[0216] Below, refer to Figure 3A and Figure 3B The sub-dividing unit 202 will be described.

[0217] Figure 3A and Figure 3B This is a diagram for explaining an example of an operation of generating sub-division vertices from a base mesh.

[0218] Figure 3A This is a diagram showing an example of a basic mesh composed of five vertices.

[0219] Here, the subdivision may be performed using, for example, a mid-edge segmentation method that connects the midpoints of the edges of each base surface to each other, thereby segmenting one base surface into four surfaces.

[0220] Figure 3B An example of a sub-divided mesh formed by dividing a base mesh consisting of five vertices is shown. Figure 3B In the sub-division mesh shown, eight sub-division vertices (hollow circles) are generated in addition to the original five vertices (solid circles).

[0221] By decoding the displacement amount for each of the sub-divided vertices generated in this way by the displacement amount decoding unit 206 , it is expected that the encoding performance will be improved.

[0222] In addition, different sub-division methods may be applied to each patch. In this way, the displacement decoded by the displacement decoding unit 206 is adaptively changed for each patch, and improvement in encoding performance can be expected. Information on the divided patch is received as control information, namely, patch_id.

[0223] Below, refer to Fig.16 The sub-dividing unit 203 will be described. Fig.16 2 is a diagram showing an example of functional blocks of the sub-dividing unit 203 .

[0224] like Fig.16 As shown, the sub-division unit 203 includes a basic grid sub-division unit 203A and a sub-division grid adjustment unit 203B.

[0225] (Basic mesh sub-division unit 203A)

[0226] The base mesh sub-division unit 203A is configured to calculate the number of divisions (sub-division number) for each base surface and each base patch based on the input base mesh and the division information of the base mesh, sub-divide the base mesh according to the division number, and output the sub-divided surfaces.

[0227] That is, the base mesh sub-division unit 203A may be configured to be able to change the above-mentioned number of divisions in units of base surfaces and base patches.

[0228] Here, a base face is a face that constitutes a base mesh, and a base patch is a collection of several base faces.

[0229] In addition, the basic mesh sub-division unit 203A may be configured to calculate the number of sub-divisions of the basic surface by predicting the number of sub-divisions of the basic surface and adding the predicted number of sub-divisions of the basic surface to a residual of the predicted number of sub-divisions of the basic surface.

[0230] In addition, the basic mesh sub-division unit 203A may be configured to calculate the number of sub-divisions of the basic surface based on the number of sub-divisions of adjacent basic surfaces of the basic surface.

[0231] In addition, the basic mesh sub-division unit 203A may be configured to calculate the number of sub-divisions of the basic surface based on the previously accumulated number of sub-divisions of the basic surface.

[0232] In addition, the basic mesh sub-division unit 203A may be configured to generate vertices for dividing three sides constituting the basic surface and connect the generated vertices, thereby performing sub-division on the basic surface.

[0233] like Fig.16 As shown, the basic mesh sub-division unit 203A has 203B at the rear stage, which has a sub-division mesh adjustment unit described later.

[0234] Below, use Figure 17 to Figure 19 An example of the processing performed by the basic mesh sub-division unit 203A will be described.

[0235] Fig.17 is a diagram showing an example of a functional block of the basic mesh sub-division unit 203A. Fig.19 This is a flowchart showing an example of the operation of the basic mesh sub-division unit 203A.

[0236] like Fig.17 As shown, the basic mesh sub-division unit 203A includes a basic surface division number buffer unit 203A1, a basic surface division number reference unit 203A2, a basic surface division number prediction unit 203A3, an addition unit 203A4, and a basic surface division unit 203A5.

[0237] The base surface division number buffer unit 203A1 is configured to store the base surface division information including the division number of the base surface, and output the base surface division information to the base surface division number reference unit 203A2.

[0238] Here, the size of the base surface division number buffer unit 203A1 may be set to 1, and the previously accumulated base surface division number may be output to the base surface division number reference unit 203A2.

[0239] That is, the size of the base face division number buffer unit 203A1 may be set to 1 so that only the last decoded sub-division number (the sub-division number decoded before) is referenced.

[0240] The base plane division number reference unit 203A2 is configured to output "cannot be referenced" to the base plane division number prediction unit 203A3 when there is no adjacent base plane to the decoding target base plane or when there is an adjacent base plane to the decoding target base plane but the division number is uncertain.

[0241] On the other hand, when there is an adjacent base plane to the decoding target base plane and the number of divisions is determined, the base plane division number reference unit 203A2 is configured to output the number of divisions to the base plane division number prediction unit 203A3.

[0242] The base surface division number prediction unit 203A3 is configured to predict the number of divisions of the base surface (the number of sub-divisions) based on one or more input division numbers, and output the predicted number of divisions (predicted number of divisions) to the addition unit 203A4.

[0243] Here, the base surface division number prediction unit 203A3 is configured to output 0 to the addition unit 203A4 when only the input of "cannot be referenced" is input from the base surface division number reference unit 203A2.

[0244] Furthermore, the base surface division number prediction unit 203A3 may be configured to generate a predicted division number using any statistical value such as an average value, maximum value, minimum value, mode, etc. of the input division numbers when more than one division number is input.

[0245] Furthermore, the base surface division number prediction unit 203A3 may be configured to generate the number of divisions of the most adjacent surface as the predicted number of divisions when one or more numbers of divisions are input.

[0246] The adding unit 203A4 is configured to output, to the base plane partitioning unit 203A5, a number of divisions obtained by adding the prediction number of divisions residual decoded from the prediction residual bit stream and the prediction number of divisions acquired from the base plane partitioning number predicting unit 203A3.

[0247] The base surface dividing unit 203A5 is configured to divide the base surface into sub-divisions according to the number of divisions input from the adding unit 203A4.

[0248] Fig.18 This is an example of dividing the base surface into nine parts. Fig.18 The method of dividing the base surface by the base surface dividing unit 203A5 will be described.

[0249] The base surface dividing unit 203A5 generates points A_1, . . . , A_(N-1) for dividing the side AB constituting the base surface into N equal parts (N=3).

[0250] Similarly, the base surface dividing section 203A5 also N-equal-divides the opposite side BC or the side CA of the triangle, and generates points B_1, …, B_(N-1), C_1, …, C_(N-1) respectively.

[0251] After that, the points on the sides AB, BC, and CA are called "side division points".

[0252] The base surface dividing section 203A5 generates sides A_i B_(N-i), B_i C_(N-i), C_i A_(N-i) for all i (i = 1, 2, …, N-1), and generates N 2 sub-dividing surfaces.

[0253] Next, refer to Fig.19 to illustrate the processing sequence of the base grid sub-dividing section 203A.

[0254] In step S2201, it is determined whether the re-division processing of the last base surface is completed. If the processing is completed, it ends; if not, it proceeds to step S2202.

[0255] In step S2202, the base grid sub-dividing section 203A determines whether Depth < mdu_max_depth.

[0256] Here, Depth is a variable representing the current depth, with an initial value of 0, and mdu_max_depth represents the maximum depth determined for each base surface.

[0257] If the condition of step S2202 is satisfied, the processing sequence proceeds to step S2203; if the condition is not satisfied, the processing sequence returns to step S2201.

[0258] In step S2203, the base grid sub-dividing section 203A determines whether the mdu_subdivision_flag of the current depth is 1.

[0259] If yes, the processing sequence returns to step S2201; if no, the processing sequence proceeds to step S2204.

[0260] In step S2204, the base grid sub-dividing section 203A further sub-divides all the sub-dividing surfaces within the base surface.

[0261] Here, if the base grid sub-dividing section 203A has not performed a single sub-division processing on the base surface, it sub-divides the base surface.

[0262] In addition, the sub-division method is the same as the method described in step S2204.

[0263] Specifically, when the base surface is not subdivided once, the base surface is Fig.18 When at least one sub-division is performed, the sub-division face is divided into N 2 In Fig.18 For example, the face composed of vertices A_2, B, and B_1 is further segmented to generate N 2 Noodles.

[0264] When the sub-division processing is completed, the processing sequence enters step S2205.

[0265] In step S2205, the basic mesh sub-division unit 203A adds 1 to Depth, and the processing sequence returns to step S2202.

[0266] (Sub-division grid adjustment unit 203B)

[0267] Next, a specific example of the processing performed by the sub-division grid adjustment unit 203B will be described. Figures 20-24 An example of the processing performed by the sub-division grid adjustment unit 203B will be described.

[0268] Fig. 20 2 is a diagram showing an example of the functional blocks of the sub-division grid adjustment unit 203B.

[0269] like Fig. 20 As shown, the sub-division grid adjustment unit 203B includes an edge division point moving unit 701 and a sub-division surface dividing unit 702 .

[0270] (Edge Division Point Moving Unit 701)

[0271] The edge division point moving unit 701 is configured to move the edge division point of the base surface to any one of the edge division points of the adjacent base surface with respect to the input initial sub-division surface, and output the sub-division surface.

[0272] Fig.21 is an example of moving the edge splitting point on the base surface ABC. Fig.21 As shown, the side division point moving unit 701 may be configured to move the side division point of the base surface ABC to the side division point of the nearest adjacent base surface.

[0273] (Sub-dividing surface dividing unit 702)

[0274] The sub-divided plane dividing unit 702 is configured to sub-divide the input sub-divided plane again and output a decoded sub-divided plane.

[0275] Fig. 221 is a diagram showing an example of a case where the sub-divided surface X in the base surface is further sub-divided.

[0276] like Fig. 22 As shown, the sub-divided surface dividing unit 702 may be configured to generate new sub-divided surfaces within the base surface by connecting vertices constituting the sub-divided surfaces with edge division points of the adjacent base surface.

[0277] Fig.23 1 is a diagram showing an example of a case where the above-mentioned sub-division process is performed on all sub-division surfaces.

[0278] The mesh decoding unit 204 is configured to generate and output a decoded mesh using the sub-division mesh generated by the sub-division unit 203 and the displacement amount decoded by the displacement amount decoding unit 206 .

[0279] Specifically, the mesh decoding unit 204 is configured to generate a decoded mesh by adding the corresponding displacement amount to each sub-division vertex. Here, information on which sub-division vertex each displacement amount corresponds to is indicated by control information.

[0280] The patch integration unit 205 is configured to integrate and output a plurality of patches of the decoded meshes generated by the mesh decoding unit 206 .

[0281] Here, the patch segmentation method is defined by the grid encoding device 100. For example, the patch segmentation method may be configured to calculate the normal vector for each base surface in advance, select the base surface with the most similar normal vector from adjacent base surfaces, aggregate the two base surfaces into the same patch, and repeat the above sequence for the next base surface.

[0282] The video decoding unit 207 is configured to decode and output textures by video coding. For example, the video decoding unit 207 can use HEVC of Non-Patent Document 1.

[0283] <Displacement Decoding Unit 206>

[0284] The displacement amount decoding unit 206 is configured to decode the displacement amount bit stream and generate and output the displacement amount.

[0285] Figure 3B is a diagram showing an example of the displacement amount relative to a sub-division vertex. Figure 3B In the example of , since there are 8 sub-division vertices, the displacement amount decoding unit 206 is configured to define 8 displacement amounts expressed by scalars or vectors for each sub-division vertex.

[0286] Below, refer to Fig.24 , the displacement decoding unit 206 is explained. Fig.24 2 is a diagram showing an example of functional blocks of the displacement amount decoding unit 206 .

[0287] like Fig.24 As shown, the displacement amount decoding unit 206 includes a decoding unit 206A, an inverse quantization unit 206B, an inverse wavelet transform unit 206C, an adder 206D, an inter-frame prediction unit 206E, and a frame buffer 206F.

[0288] The decoding unit 206A is configured to decode and output the level value and control information by performing variable length decoding on the received displacement bit stream. Here, the level value obtained by variable length decoding is output to the inverse quantization unit 206B, and the control information is output to the inter-frame prediction unit 206E.

[0289] Below, refer to Fig.25 , an example of the structure of the displacement bit stream is explained. Fig.25 This is a diagram showing an example of the structure of a displacement bit stream.

[0290] like Fig.25 As shown, first, the displacement bit stream may include a set of control information related to the decoding of the displacement, namely, DPS (Displacement Parameter Set).

[0291] Second, the displacement bitstream may include a set of control information corresponding to the patch, namely, DPH (DisplacementPatch Header).

[0292] Third, the displacement bitstream may contain the encoded displacements that make up the patch after the DPH.

[0293] As described above, the structure of the displacement bit stream is that DPH and DPS correspond to each encoded displacement.

[0294] also, Fig.25 The structure is just an example. As long as DPH and DPS correspond to the encoded displacement amounts, elements other than the above may be added as structural elements of the displacement amount bit stream.

[0295] For example, Fig.25 As shown, the displacement bit stream may include an SPS (Sequence Parameter Set).

[0296] Fig.26 This is a diagram showing an example of the syntax structure of DPS.

[0297] exist Fig.26 In the descriptor column, the descriptor column indicates how each syntax is encoded.

[0298] In addition, Fig.26 In it, ue(v) represents the codeless 0th order exponential-Golomb code, and u(n) represents the n-bit tag.

[0299] When there are a plurality of DPSs, the DPS includes at least DPS id information (dps_displacement_parameter_set_id) for identifying each DPS.

[0300] In addition, the DPS may include a flag (interprediction_enabled_flag) for controlling whether to perform inter-frame prediction.

[0301] For example, when the interpretation_enabled_flag is 0, it can be defined as not performing inter-frame prediction, and when the interpretation_enabled_flag is 1, it can be defined as performing inter-frame prediction. When the interpretation_enabled_flag is not included, it can be defined as not performing inter-frame prediction.

[0302] DPS may include a flag (dct_enabled_flag) that controls whether to perform inverse DCT.

[0303] For example, when dct_enabled_flag is 0, it can be defined as not performing inverse DCT, and when dct_enabled_flag is 1, it can be defined as performing inverse DCT. When dct_enabled_flag is not included, it can be defined as not performing inverse DCT.

[0304] Fig. 27 This is a diagram showing an example of the syntax structure of DPH.

[0305] like Fig. 27 As shown, the DPH includes at least DPS id information for specifying the DPS corresponding to each DPH.

[0306] The inverse quantization unit 206B is configured to generate and output a transform coefficient by inverse quantizing the level value decoded by the decoding unit 206A.

[0307] The inverse wavelet transform unit 206C is configured to generate and output a prediction residual by performing an inverse wavelet transform on the transform coefficients generated by the inverse quantization unit 206B.

[0308] (Inter-frame prediction unit 206E)

[0309] The inter-frame prediction unit 206E is configured to perform inter-frame prediction using the decoded displacement of the reference frame read from the frame buffer 206F, and to generate and output a predicted displacement.

[0310] The inter-frame prediction unit 206E is configured to perform the inter-frame prediction only when interprediction_enabled_flag is 1.

[0311] The inter-frame prediction unit 206E can perform inter-frame prediction in the spatial domain or in the frequency domain. The inter-frame prediction can use a reference frame in the past and a reference frame in the future to perform bidirectional prediction.

[0312] Fig.24 This is an example of a functional block of the inter-frame prediction unit 206E when inter-frame prediction is performed in the spatial region.

[0313] When the inter-frame prediction unit 206E performs inter-frame prediction in the spatial region, it may directly refer to the decoded displacement of the corresponding sub-divided vertex in the reference frame to determine the predicted displacement of the sub-divided vertex in the target frame.

[0314] Alternatively, the predicted displacement of a sub-divided vertex in the target frame can be determined probabilistically using the decoded displacements of the corresponding sub-divided vertices in multiple reference frames according to a normal distribution with an estimated mean and dispersion. In this case, the dispersion can be uniquely determined to be zero only by the mean.

[0315] Alternatively, the predicted displacement of a sub-divided vertex in the target frame can be determined using decoded displacements of corresponding sub-divided vertices in multiple reference frames, based on a regression curve that estimates time using explanatory variables and displacement as target variables.

[0316] In the trellis coding apparatus 100, in order to improve the coding efficiency of each frame, the order of the decoding displacements may be rearranged.

[0317] In this case, the inter-frame prediction unit 206E may be configured to perform inter-frame prediction on the rearranged decoded displacement.

[0318] The correspondence relationship between the sub-division vertices between the reference frame and the decoding target frame is represented by the control information.

[0319] Fig.28 This is a diagram for explaining an example of the correspondence relationship between sub-division vertices between a reference frame and a decoding target frame when inter-frame prediction is performed in a spatial region.

[0320] Fig.29 This is an example of a functional block of the inter-frame prediction unit 206E when inter-frame prediction is performed in the frequency domain.

[0321] When the inter-frame prediction unit 206E performs inter-frame prediction in the frequency domain, it may directly refer to the decoded wavelet transform coefficient of the corresponding frequency in the reference frame to determine the predicted wavelet transform coefficient of the frequency in the decoding target frame.

[0322] The inter prediction unit 206E may perform inter prediction according to a normal distribution probability in which the average and the dispersion are estimated, using the decoded displacement amounts or decoded wavelet transform coefficients of the sub-division vertices in the plurality of reference frames.

[0323] The inter prediction unit 206E may perform inter prediction based on a regression curve that estimates time using explanatory variables and displacement amounts as target variables, using decoded displacement amounts or decoded wavelet transform coefficients of sub-divided vertices in a plurality of reference frames.

[0324] The inter-frame prediction unit 206E may be configured to perform inter-frame prediction bidirectionally using a temporally past reference frame and a future reference frame.

[0325] In the trellis encoding device 100, in order to improve the encoding efficiency of each frame, the order of the decoded wavelet transform coefficients may be rearranged.

[0326] The frequency correspondence between the reference frame and the decoding target frame is represented by the control information.

[0327] Fig.30 This is a diagram for explaining an example of the frequency correspondence relationship between a reference frame and a decoding target frame when inter-frame prediction is performed in the frequency domain.

[0328] Furthermore, when the sub-dividing unit 203 divides the base grid into a plurality of patches, the inter-frame prediction unit 206E also performs inter-frame prediction for each of the divided patches. This increases the temporal correlation between frames and can be expected to improve the encoding performance.

[0329] The adder 206D receives an input of a prediction residual from the inverse wavelet transform unit 206C and an input of a prediction displacement from the inter-frame prediction unit 206E.

[0330] The adder 206D is configured to calculate and output a decoded displacement by adding the prediction residual to the prediction displacement.

[0331] The decoded displacement calculated by the adder 206D is also output to the frame buffer 206F.

[0332] The frame buffer 206F is configured to acquire and accumulate the decoded displacement amount from the adder 206D.

[0333] Here, the frame buffer 206F outputs the decoded displacement of the corresponding vertex in the reference frame in response to control information (not shown).

[0334] Fig.31 This is a flowchart showing an example of the operation of the displacement amount decoding unit 206.

[0335] like Fig.31As shown, in step S3501, the displacement amount decoding unit 206 determines whether the processing is completed for all patches.

[0336] If yes, the action ends, and if no, the action proceeds to step S3502.

[0337] In step S3502 , the displacement decoding unit 206 performs inverse DCT on the patch of the decoding target, and then performs inverse quantization and inverse wavelet transform.

[0338] In step S3503 , the displacement amount decoding unit 206 determines whether interpretation_enabled_flag is 1.

[0339] In the case of yes, the action proceeds to step S3504, and in the case of no, the action returns to step S3501.

[0340] In step S3504, the displacement decoding unit 206 performs the above-mentioned inter-frame prediction and addition.

[0341] <Modification 1>

[0342] Below, refer to Fig.32 , focusing on the differences between the above-mentioned modification example 1 of the first embodiment and the above-mentioned first embodiment.

[0343] Fig.32 This is a diagram showing an example of functional blocks of the displacement amount decoding unit 206 according to the first modification.

[0344] like Fig.32 As shown, the displacement decoding unit 206 according to the first modification includes an inverse DCT unit 206G after the decoding unit 206A, that is, between the decoding unit 206A and the inverse quantization unit 206B.

[0345] That is, in the present modification example 1, the inverse quantization unit 206B is configured to generate a prediction residual by performing an inverse wavelet transform on the level value output from the inverse DCT unit 202G.

[0346] <Modification 2>

[0347] Below, refer to Fig.33 , focusing on the differences between the modified example 2 of the above-mentioned first embodiment and the above-mentioned first embodiment.

[0348] like Fig.33 As shown, the displacement amount decoding unit 206 according to the second modification includes a video decoding unit 2061 , an image development unit 2062 , an inverse quantization unit 2063 , and an inverse wavelet transformation unit 2064 .

[0349] The video decoding unit 2061 is configured to output a video by decoding the displacement bit stream received by the video encoding.

[0350] For example, the video decoding unit 2061 can use HEVC of Non-Patent Document 1.

[0351] In addition, the video decoding unit 2061 may use a video encoding method that always sets the motion vector to 0. For example, the video decoding unit 2061 may always set the motion vector of HEVC to 0 and use inter-frame prediction that is always at the same position.

[0352] In addition, the video decoding unit 2061 may use a video coding method that always skips transformation. For example, the video decoding unit 2061 may always set the transformation of HEVC to the transformation skip mode and use the video coding method without performing transformation.

[0353] The image expansion unit 2062 is configured to expand the image decoded by the image decoding unit 2061 as a level value for each image (frame) and output the expanded image.

[0354] In the above-described expansion method, the image expansion unit 2062 can perform back-calculation and identification based on the arrangement method of the level values ​​with respect to the image indicated by the control information.

[0355] The image expansion unit 2062 may, for example, arrange the level values ​​from high-frequency components to low-frequency components in the image in a raster operation order as a method of arranging the level values.

[0356] The inverse quantization unit 2063 is configured to generate and output a transform coefficient by inverse quantizing the level value generated by the image development unit 2062 .

[0357] The inverse wavelet transform unit 2064 is configured to generate and output a decoded displacement by performing an inverse wavelet transform on the transform coefficients generated by the inverse quantization unit 2063 .

[0358] The above-mentioned trellis encoding device 100 and trellis decoding device 200 can be implemented as a program that causes a computer to execute each function (each process).

[0359] Industrial Applicability

[0360] In addition, according to this embodiment, for example, since comprehensive service quality can be improved in moving image communications, it can contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Improve the infrastructure of the information environment, promote sustainable industrialization and achieve expansion of innovation."

Claims

1. A grid decoding device, characterized in that: The grid decoding device has: The sorting unit collects a plurality of vertices having the same coordinates in the decoded basic mesh, that is, duplicate vertices, into one vertex and then rearranges the vertices into a predetermined order.

2. The grid decoding device according to claim 1, characterized in that: The grid decoding device has: The inter-frame decoding unit calculates the index or coordinates of the vertices added to the base mesh of the inter-frame of the decoding target by using the vertices of the base mesh of the reference frame having the plurality of motion vectors.

3. The grid decoding device according to claim 1 or 2, characterized in that: The sorting unit decodes the list of duplicate vertices according to a bit stream of an I frame.

4. The trellis decoding device according to claim 1 or 2, characterized in that: The sorting unit calculates a list of the duplicate vertices by searching the base mesh for the duplicate vertices.

5. The grid decoding device according to claim 1 or 2, characterized in that: After determining the list of duplicate vertices, the sorting unit integrates all or part of the duplicate vertices to update connectivity.

6. The grid decoding device according to claim 5, characterized in that: After determining the list of duplicate vertices, the sorting unit decodes information of unprocessed duplicate vertices according to a bit stream, integrates duplicate vertices other than the unprocessed duplicate vertices, and updates connectivity.

7. The grid decoding device according to claim 2, characterized in that: The inter decoding unit decodes, from the bitstream, indices of vertices added to a base mesh of an inter frame of a decoding target through vertices of a base mesh of a reference frame having a plurality of motion vectors when all duplicate vertices are integrated by the sorting unit.

8. The grid decoding device according to claim 2, characterized in that: The inter decoding unit decodes motion vectors for the extended vertices of the decoded base mesh of the inter frame of the decoding target when all the duplicate vertices are integrated by the sorting unit.

9. A grid decoding method, characterized in that: This mesh decoding method includes a step of aggregating a plurality of vertices whose coordinates match those of the decoded basic mesh, that is, duplicate vertices, into one vertex and then rearranging them into a predetermined order.

10. A program product comprising a program for causing a computer to function as a trellis decoding device, the program product being characterized in that: The mesh decoding device includes a sorting unit that collects a plurality of vertices having the same coordinates with respect to the decoded basic mesh, that is, duplicate vertices, into one vertex and then rearranges the vertices into a predetermined order.

Images