Apparatus and method for dynamic mesh coding

Abstract

An apparatus and method for dynamic mesh coding is provided. [Solution] The dynamic mesh encoding / decoding method and apparatus of the present invention additionally uses motion information existing between multiple frames that make up the dynamic mesh in order to remove temporal redundancy in the dynamic mesh and improve encoding efficiency.

Description

Technical Field

[0001] The present invention relates to an apparatus and method for dynamic mesh coding.

Background Art

[0002] The following description merely provides background information related to the present invention and does not constitute prior art.

[0003] Generally, meshes are classified into static meshes and dynamic meshes. A static mesh means three-dimensional information at a moment and includes mesh information in one single frame. On the other hand, a dynamic mesh means mesh information during a specific time period and includes mesh information that remains distributed among a number of frames according to time changes.

[0004] Conventional mesh compression methods encode and decode meshes in frame units regardless of the dependency between the previous frame and the current frame. That is, even for a number of frames constituting a dynamic mesh, each frame is individually encoded and decoded. Therefore, when encoding / decoding a dynamic mesh, a coding method and apparatus using inter-frame dependency should be considered.

Summary of the Invention

Problems to be Solved by the Invention

[0005] The present invention has been made in view of the above prior art, and an object of the present invention is to provide a dynamic mesh encoding / decoding method and apparatus that additionally use motion information existing between frames.

Means for Solving the Problems

[0006] A decoding method for decoding a dynamic mesh performed by a dynamic mesh decoding device according to one aspect of the present invention, made to achieve the above objective, comprises the steps of: acquiring a bitstream, separating a first bitstream and a second bitstream from the bitstream; and decoding the bitstream, wherein the first bitstream is a bitstream in which a predetermined key-frame among a number of frames representing the dynamic mesh is encoded; the second bitstream is a bitstream in which one of the remaining frames excluding the key-frame among the number of frames is encoded; the decoding step, if the bitstream is the first bitstream, includes the steps of: decoding the first bitstream to restore the mesh of the key-frame; and saving the mesh of the key-frame as the previous frame in the mesh storage unit; and if the bitstream is the second bitstream, decoding the second bitstream to restore the motion data of the current frame. The method is characterized by including the steps of: restoring data; applying the motion data to the previous frame to restore the mesh of the current frame; and saving the mesh of the current frame as the previous frame in the mesh storage unit.

[0007] A dynamic mesh decoding device according to one aspect of the present invention, made to achieve the above objective, comprises a bitstream separation unit that, after acquiring a bitstream, separates a first bitstream and a second bitstream from the bitstream, wherein the first bitstream is a bitstream in which a preset key-frame among a number of frames representing the dynamic mesh is encoded, and the second bitstream is a bitstream in which one of the remaining frames excluding the key-frame among the number of frames is encoded, and the dynamic mesh decoding device further comprises a mesh decoding unit that, when the bitstream is the first bitstream, decodes the first bitstream to restore the mesh of the key-frame, a motion decoding unit that, when the bitstream is the second bitstream, decodes the second bitstream to restore the motion data of the current frame, a motion compensation unit that applies the motion data to the previous frame to restore the mesh of the current frame, and a mesh storage unit that saves the mesh of the key-frame and the mesh of the current frame as the previous frame.

[0008] An encoding method for encoding a dynamic mesh, performed by a dynamic mesh encoding device according to one aspect of the present invention made to achieve the above objective, comprises the steps of: acquiring the current frame constituting the dynamic mesh; confirming whether the current frame is a preset key-frame; and encoding the current frame, wherein the encoding step includes, if the current frame is the key-frame, encoding the key-frame to generate a first bitstream, generating a restored mesh of the key-frame from the first bitstream, and saving the restored mesh of the key-frame as the previous frame in the mesh storage unit, and if the current frame is not the key-frame, extracting motion data using the mesh of the current frame and the restored mesh of the previous frame; encoding the motion data to generate a second bitstream, generating restored motion data from the second bitstream; applying the restored motion data to the previous frame to generate a restored current frame; and saving the restored current frame as the previous frame in the mesh storage unit. [Effects of the Invention]

[0009] According to the present invention, by providing a dynamic mesh coding / decoding method and apparatus that additionally utilizes motion information present between a large number of frames constituting a dynamic mesh, it is possible to eliminate the temporal redundancy of the dynamic mesh and improve coding efficiency. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram conceptually illustrating a dynamic mesh coding device according to one embodiment of the present invention. [Figure 2] This is a block diagram conceptually illustrating a dynamic mesh decoding device according to one embodiment of the present invention. [Figure 3]This is a block diagram conceptually illustrating a dynamic mesh coding device using coordinate system transformation according to another embodiment of the present invention. [Figure 4] This is a block diagram conceptually illustrating a dynamic mesh decoding device using inverse coordinate system transformation according to another embodiment of the present invention. [Figure 5] This is a block diagram conceptually illustrating a dynamic mesh coding device using coordinate system transformation according to yet another embodiment of the present invention. [Figure 6] This is a block diagram conceptually illustrating a dynamic mesh decoding device using inverse coordinate system transformation according to yet another embodiment of the present invention. [Figure 7] This is a block diagram showing the motion encoding unit within an encoding device according to one embodiment of the present invention. [Figure 8] This is an illustrative diagram showing motion map generation and downsampling using one embodiment of the present invention. [Figure 9] This is a block diagram showing the movement of the decoding unit within a decoding device according to one embodiment of the present invention. [Figure 10] This is a flowchart showing a dynamic mesh coding method according to one embodiment of the present invention. [Figure 11] This is a flowchart showing a dynamic mesh decoding method according to one embodiment of the present invention. [Figure 12] Another embodiment of the present invention presents an attractive method for dynamic mesh coding. [Figure 13] This flowchart shows a dynamic mesh decoding method according to another embodiment of the present invention. [Modes for carrying out the invention]

[0011] Hereinafter, specific examples of embodiments for implementing the present invention will be described in detail while referring to the drawings. Note that when adding reference signs to the components in each drawing, for the same components, even if they are shown on other drawings, they are made to have the same signs as much as possible. In addition, in describing this embodiment, when it is determined that a specific description of a related known configuration or function obscures the gist of the present invention, the detailed description thereof will be omitted.

[0012] This embodiment discloses content related to an apparatus and method for dynamic mesh coding. More specifically, in order to eliminate the temporal redundancy of a dynamic mesh, a dynamic mesh encoding / decoding method and apparatus are provided that additionally use motion information existing between a large number of frames constituting the dynamic mesh.

[0013] In the following description, a mesh represents a static mesh including one frame. Therefore, a mesh and a frame are used interchangeably. On the other hand, a dynamic mesh includes a large number of frames and includes at least one pre-set key frame. For example, a key frame represents a frame that does not refer to other frames when encoding / decoding is performed. The remaining frames except the key frame are represented as non-key frames.

[0014] Hereinafter, motion information and motion data are used interchangeably.

[0015] Hereinafter, a dynamic mesh encoding apparatus (hereinafter, "encoding apparatus") for encoding a dynamic mesh and a dynamic mesh decoding apparatus (hereinafter, "decoding apparatus") for decoding a dynamic mesh will be described using the diagrams shown in FIGS. 1 and 2.

[0016] FIG. 1 is a block diagram conceptually showing a dynamic mesh encoding apparatus according to an embodiment of the present invention.

[0017] After obtaining the original dynamic mesh, the encoding device according to this embodiment encodes it to generate a bitstream. The encoding device includes all or part of a mesh encoding unit 102, a mesh storage unit 104, a motion extraction unit 106, a motion encoding unit 108, a motion compensation unit 110, and a bitstream synthesis unit 112.

[0018] The encoding device checks whether the current frame is a key frame for the frame constituting the input dynamic mesh. When the current frame is a key frame, the corresponding mesh is transmitted to the mesh encoding unit 102, and when it is a non-key frame, the corresponding mesh is transmitted to the motion extraction unit 106.

[0019] The mesh encoding unit 102 encodes the transmitted mesh to generate a bitstream and generates a mesh restored from the bitstream. The bitstream is transmitted to the bitstream synthesis unit 112, and the restored mesh is stored in the mesh storage unit 104. To encode the mesh, the mesh encoding unit 102 uses an existing mesh encoding method. For example, a method of encoding each of the vertices, edges, and attribute information constituting the mesh is used. On the other hand, so that the restored mesh is referenced by the next frame, the mesh encoding unit 102 restores the mesh before generating the final bitstream related to the mesh using a decoding method corresponding to the encoding method.

[0020] The mesh storage unit 104 stores the restored mesh of the immediately previous frame. Here, the immediately previous frame is one of the restored key frame or the restored non-key frame. When the current frame is a non-key frame, the mesh stored in the motion extraction and restoration of the current frame is used. Therefore, the stored mesh is used by the motion extraction unit 106 and the motion compensation unit 110.

[0021] The motion extraction unit 106 extracts motion information using the mesh of the current frame that has been input and the restored mesh of the previous frame stored in the mesh storage unit 104. For example, the motion extraction unit 106 extracts motion data for each vertex of the mesh. As another example, the motion extraction unit 106 extracts at least one motion data for one face of the mesh. In this case, the motion data of the vertices that make up one face is interpolated, and motion data is extracted for the interpolated points of the vertices. Therefore, the number and resolution of the motion data are determined by the number and resolution of the interpolated points. As yet another example, the motion extraction unit 106 extracts motion data in units of patches containing many faces and vertices. Here, the motion data of a vertex, face, or patch is a three-dimensional motion vector that shows how they move. The extracted motion information is transmitted to the motion encoding unit 108.

[0022] The motion encoding unit 108 encodes the transmitted motion information and generates a bitstream. The generated bitstream is transmitted to the bitstream synthesis unit 112. The motion encoding unit 108 also generates restored motion information from the bitstream. The generated restored motion information is transmitted to the motion compensation unit 110. The detailed operation of the motion encoding unit 108 will be described later.

[0023] The motion compensation unit 110 restores the current frame by applying the restored motion information to the previous frame stored in the mesh storage unit 104 to compensate for the motion. The restored current frame is stored in the mesh storage unit 104 for encoding the next frame.

[0024] The bitstream synthesis unit 112 combines the input bitstreams into a single bitstream. The encoding device either saves the synthesized bitstream or transmits it to the decoding device.

[0025] Figure 2 is a block diagram conceptually illustrating a dynamic mesh decoding device according to one embodiment of the present invention.

[0026] The decoding device according to this embodiment acquires a bitstream, decodes it, and restores the original dynamic mesh. The decoding device includes all or part of a bitstream separation unit 202, a mesh decoding unit 204, a mesh storage unit 206, a motion decoding unit 208, and a motion compensation unit 210.

[0027] The bitstream separation unit 202 separates the bitstream based on the header information of the input bitstream. Bitstreams corresponding to keyframes are transmitted to the mesh decoding unit 204, and bitstreams corresponding to non-keyframes are transmitted to the motion decoding unit 208.

[0028] The mesh decoding unit 204 decodes the bitstream corresponding to the keyframes to restore the mesh. To decode the mesh, the mesh decoding unit 204 uses a decoding method corresponding to the encoding method used by the mesh encoding unit 102 in the encoding device. The restored mesh is stored in the mesh storage unit 206.

[0029] The mesh storage unit 206 saves the restored mesh of the previous frame, where the previous frame is either a restored keyframe or a restored non-keyframe. The mesh storage unit 206 outputs the saved mesh, for example, for display. Furthermore, if the current frame is a non-keyframe, the saved mesh is used to restore the current frame. Thus, the saved mesh is used by the motion compensation unit 210.

[0030] The motion decoding unit 208 decodes the bitstream corresponding to non-keyframes and restores the motion information of the current frame. The restored motion information is transmitted to the motion compensation unit 210. The detailed operation of the motion decoding unit 208 will be described later.

[0031] The motion compensation unit 210 restores the current frame by applying the restored motion information to the previous frame stored in the mesh storage unit 206. The restored current frame is then stored in the mesh storage unit 206 for the restoration of the next frame.

[0032] On the other hand, the coordinate system in which the input dynamic mesh resides may not be the optimal coordinate system in terms of mesh coding efficiency. Therefore, before coding is performed, the original coordinate system of the input mesh is transformed into a first coordinate system suitable for mesh coding. For example, the coordinate system transformation is from the original orthogonal coordinate system to a cylindrical coordinate system, or from an orthogonal coordinate system to a spherical coordinate system. Another example is when the original coordinate system is a cylindrical coordinate system; in this case, the coordinate system transformation is from a cylindrical coordinate system to an orthogonal coordinate system, or from a cylindrical coordinate system to a spherical coordinate system. Yet another example is when the original coordinate system is a spherical coordinate system; in this case, the coordinate system transformation is from a spherical coordinate system to an orthogonal coordinate system, or from a spherical coordinate system to a cylindrical coordinate system.

[0033] The following describes the coding and decoding devices that use coordinate system transformations, using the diagrams shown in Figures 3 and 4.

[0034] Figure 3 is a block diagram conceptually illustrating a dynamic mesh coding device using coordinate system transformation according to another embodiment of the present invention.

[0035] In another embodiment, the encoding device encodes the dynamic mesh using coordinate system transformation. For this purpose, as illustrated in Figure 3, the encoding device additionally includes a first coordinate system transformation unit 302.

[0036] The first coordinate system transformation unit 302 transforms the original coordinate system to the first coordinate system for the vertices of the input dynamic mesh. At this time, the encoding device transmits information related to such coordinate system transformation to the decoding device in the form of a bitstream. Alternatively, the coordinate system transformation method is shared between the encoding device and the decoding device using information from the higher-level step.

[0037] The operation of the remaining components that encode the current frame, which has been transformed from its coordinate system to the first coordinate system, is identical to that of the components in the encoding device illustrated in Figure 1, so a detailed explanation is omitted.

[0038] Figure 4 is a conceptual block diagram showing a dynamic mesh decoding device using inverse coordinate system transformation according to another embodiment of the present invention.

[0039] In another embodiment, the decoding device decodes the dynamic mesh using an inverse coordinate system transform. For this purpose, as illustrated in Figure 4, the decoding device additionally includes a first inverse coordinate system transform unit 402.

[0040] The operation of the remaining components that decode the current frame in the first coordinate system from the bitstream is identical to that of the components in the decoding device illustrated in Figure 2, so a detailed explanation is omitted.

[0041] The first coordinate system inverse transformation unit 402 inversely transforms the first coordinate system of the mesh back to the original coordinate system before outputting the mesh stored in the mesh storage unit 206. At this time, the decoding device obtains information related to the coordinate system transformation from the encoding device in the form of a bitstream. Alternatively, the decoding device performs the coordinate system transformation of the restored mesh using information transmitted from a higher step. Or, the decoding device performs the coordinate system transformation of the restored mesh using a new coordinate system unrelated to the original coordinate system and the first coordinate system used by the encoding device.

[0042] Even with a single coordinate system transformation as described above, the first coordinate system may not be optimal in terms of encoding efficiency considering motion information extraction and motion compensation. Therefore, before extracting motion information, the first coordinate system of the mesh is transformed into a second coordinate system. Here, the second coordinate system differs from the original coordinate system and the first coordinate system in that it is a coordinate system that is suitable for motion prediction and compensation in terms of encoding efficiency.

[0043] The following describes an encoding and decoding device that uses two coordinate system transformations, using the illustrations in Figures 5 and 6.

[0044] Figure 5 is a block diagram conceptually illustrating a dynamic mesh coding device using coordinate system transformation according to yet another embodiment of the present invention.

[0045] In yet another embodiment, the encoding device encodes the dynamic mesh using two coordinate system transformations. For this purpose, as illustrated in Figure 5, the encoding device additionally includes a first coordinate system transformation unit 302, a second coordinate system transformation unit 502, and a second inverse coordinate system transformation unit 504.

[0046] As described above, the first coordinate system transformation unit 302 transforms the original coordinate system to the first coordinate system for the vertices of the input dynamic mesh.

[0047] The encoding device checks whether the current frame is a keyframe for the frame whose coordinate system has been transformed. If the current frame is a keyframe, the corresponding mesh is transmitted to the mesh encoding unit 102; if it is not a keyframe, the corresponding mesh is transmitted to the second coordinate system transformation unit 502.

[0048] The mesh encoding unit 102 encodes the transmitted mesh and generates a bitstream. The generated bitstream is transmitted to the bitstream synthesis unit 112. The mesh encoding unit 102 also generates a restored mesh from the bitstream. The restored mesh is stored in the mesh storage unit 104.

[0049] The mesh storage unit 104 stores the restored mesh of the previous frame. Here, the previous frame is either a restored keyframe or a restored non-keyframe. If the current frame is a non-keyframe, the stored mesh is used for motion extraction and restoration of the current frame. Therefore, the stored mesh is used for motion extraction and motion compensation, and is transmitted to the second coordinate system transformation unit 502 for coordinate system transformation before that.

[0050] The second coordinate system transformation unit 502 transforms the first coordinate system of the current frame, which is a non-keyframe, into the second coordinate system prior to motion extraction and compensation. The second coordinate system transformation unit 502 also transforms the first coordinate system of the previous frame stored in the mesh storage unit 104 into the second coordinate system. The current frame, whose coordinate system has been transformed into the second coordinate system, is transmitted to the motion extraction unit 106, and the previous frame, whose coordinate system has been transformed, is transmitted to both the motion extraction unit 106 and the motion compensation unit 110.

[0051] The motion extraction unit 106 extracts motion data using the transmitted current frame and previous frame. The extracted motion information is transmitted to the motion encoding unit 108.

[0052] The motion encoding unit 108 encodes the transmitted motion information and generates a bitstream. The generated bitstream is transmitted to the bitstream synthesis unit 112. The motion encoding unit 108 also generates motion data restored from the bitstream. The generated restored motion information is transmitted to the motion compensation unit 110.

[0053] The motion compensation unit 110 restores the current frame by applying the restored motion information to the frame immediately preceding the coordinate system transformation to compensate for the motion. Before being saved to the mesh storage unit 104, the restored current frame is transmitted to the second coordinate system inverse transformation unit 504 for the inverse coordinate system transformation.

[0054] The second coordinate system inverse transformation unit 504 inversely transforms the second coordinate system of the restored current frame back to the first coordinate system. The restored current frame, whose coordinate system has been inversely transformed back to the first coordinate system, is stored in the mesh storage unit 104 for encoding the next frame.

[0055] The bitstream synthesis unit 112 combines the input bitstreams into a single bitstream. The encoding device either saves the synthesized bitstream or transmits it to the decoding device.

[0056] Figure 6 is a block diagram conceptually illustrating a dynamic mesh decoding device using inverse coordinate system transformation according to yet another embodiment of the present invention.

[0057] In another embodiment, the decoding device decodes the dynamic mesh using two inverse coordinate transformations. For this purpose, as illustrated in Figure 6, the decoding device additionally includes a first inverse coordinate transformation unit 402, a second coordinate transformation unit 602, and a second inverse coordinate transformation unit 604.

[0058] The bitstream separation unit 202 separates the bitstream based on the header information of the input bitstream. Bitstreams corresponding to keyframes are transmitted to the mesh decoding unit 204, and bitstreams corresponding to non-keyframes are transmitted to the motion decoding unit 208. As described above, the bitstream is generated by the encoding device on the first coordinate system and then transmitted to the decoding device.

[0059] The mesh decoding unit 204 decodes the bitstream corresponding to the keyframes to restore the mesh. To decode the mesh, the mesh decoding unit 204 uses a decoding method corresponding to the encoding method used by the mesh encoding unit 102 in the encoding device. The restored mesh is stored in the mesh storage unit 206.

[0060] The mesh storage unit 206 stores the transmitted restored mesh. Here, the previous frame is either a restored keyframe or a restored non-keyframe. The mesh storage unit 206 outputs the mesh saved for, for example, display, and transmits the mesh saved earlier for coordinate system transformation to the first inverse coordinate system transformation unit 402. Furthermore, if the current frame is a non-keyframe, the saved mesh is used to restore the current frame. Thus, the saved mesh is used for motion compensation and is transmitted earlier for coordinate system transformation to the second coordinate system transformation unit 602.

[0061] The second coordinate system transformation unit 602 transforms the first coordinate system of the preceding frame into the second coordinate system prior to motion compensation. The preceding frame, whose coordinate system has been transformed into the second coordinate system, is transmitted to the motion compensation unit 210.

[0062] The motion decoding unit 208 decodes the bitstream corresponding to non-keyframes and restores the motion information of the current frame. As described above, the motion information is generated by the encoding device on the second coordinate system and then transmitted to the decoding device. The restored motion information is then transmitted to the motion compensation unit 210.

[0063] The motion compensation unit 210 restores the current frame by applying the restored motion information to the frame immediately preceding the coordinate system transformation. Before being saved to the mesh storage unit 206, the restored current frame is transmitted to the second coordinate system inverse transformation unit 604 for the inverse coordinate system transformation.

[0064] The second coordinate system inverse transformation unit 604 inversely transforms the second coordinate system of the restored current frame back to the first coordinate system. The restored current frame, whose coordinate system has been inversely transformed back to the first coordinate system, is saved in the mesh storage unit 206 for the restoration of the next frame.

[0065] The first coordinate system inverse transformation unit 402 generates a restored dynamic mesh by inversely transforming the coordinate system of the mesh from the first coordinate system back to the original coordinate system before outputting the mesh stored in the mesh storage unit 206.

[0066] The operation of the motion encoding unit 108 and the motion decoding unit 208 will be described below using the diagrams shown in Figures 7 to 9.

[0067] Figure 7 is a block diagram showing the motion coding unit within an coding device according to one embodiment of the present invention.

[0068] As described above, the motion encoding unit 108 encodes motion information to generate a bitstream and generates motion information restored from the generated bitstream. In this case, the motion information is a three-dimensional motion vector for all vertices of the mesh. Alternatively, the motion information is a motion space for all positions in three-dimensional space. The motion encoding unit 108 includes all or part of the motion map generation unit 702, the motion map downsampling unit 704, and the motion map encoding unit 706.

[0069] The motion map generation unit 702 maps the transmitted motion information in two dimensions to generate one or more motion maps. The generated one or more motion maps are transmitted to the motion map downsampling unit 704.

[0070] The motion map downsampling unit 704 downsamples the transmitted motion map to a smaller size. At this time, one of several filters with various lengths, such as 4-tap, 6-tap, or 8-tap filters, is used as the filter for downsampling. Alternatively, common methods such as bicubic filters or sub-sampling filters are used. The downsampled motion map is then transmitted to the motion map encoding unit 706.

[0071] The motion map encoding unit 706 encodes the transmitted motion map to generate a bitstream. To encode the motion map, the motion map encoding unit 706 uses an existing image or video compression method. For example, image compression methods such as JPEG, JPEG2000, HEIF, and PNG are used. Alternatively, video compression methods such as H.264 / AVC (Advanced Video Coding), H.265 / HEVC (High Efficiency Video Coding), and H.266 / VVC (Versatile Video Coding) are used. At this time, the motion map compression method used in the higher step is transmitted from the encoding device to the decoding device as encoded. Based on the transmitted motion map compression method, the decoding device uses a decoding method corresponding to the encoding method used in the encoding device to restore the motion map.

[0072] Figure 8 is an illustrative diagram showing motion map generation and downsampling according to one embodiment of the present invention.

[0073] Since a mesh represents the relationship between vertices in 3D space and a texture map, the motion map, which corresponds to 3D motion information, also has a 2D map similar to the texture map. For example, motion data extracted for each vertex of the mesh is mapped to a motion map. As another example, vertex motion data is interpolated to generate at least one motion data for each polygonal face of the mesh, and then such motion data is mapped to a motion map. As mentioned above, since motion data is a 3D vector, a motion map is generated using the size values ​​of the 3D vector for each x, y, and z axis. Then, each motion map mapped for each axis is combined in the form of an image with three channels, as shown in the example in Figure 8.

[0074] On the other hand, when motion data is mapped to vertices, the motion map has sparse properties, so downsampling is performed by the motion map downsampling unit 704 to improve the encoding efficiency of the motion map encoding unit 706.

[0075] When motion data is mapped to the mesh surface, downsampling is implicitly performed in parallel during the interpolation process, thus omitting the operation of the motion map downsampling unit 704.

[0076] Figure 9 is a block diagram showing the movement of the decoding unit within a decoding device according to one embodiment of the present invention.

[0077] As described above, the motion decoding unit 208 decodes the bitstream corresponding to non-keyframes to restore motion information for the current frame. The motion decoding unit 208 includes all or part of the motion map decoding unit 902, the motion map upsampling unit 904, and the motion vector generation unit 906.

[0078] The motion map decoding unit 902 decodes the transmitted bitstream to restore the motion map. At this time, the decoding device restores the motion map using a decoding method corresponding to the encoding method used by the encoding device, based on the motion map compression method transmitted from the higher step. The restored motion map is transmitted to the motion map upsampling unit 904.

[0079] The motion map upsampling unit 904 upsamples the transmitted restored motion map to restore it to its original size. The upsampled motion map is then transmitted to the motion vector generation unit 906.

[0080] The motion vector generation unit 906 converts the transmitted motion map into motion vectors so that it can be used in subsequent motion compensation steps.

[0081] The dynamic mesh coding method and dynamic mesh decoding method will be described below using the diagrams shown in Figures 10 and 11.

[0082] Figure 10 is a flowchart showing a dynamic mesh coding method according to one embodiment of the present invention.

[0083] The encoding device acquires the current frame that constitutes the dynamic mesh (S1000).

[0084] The encoding device checks whether the current frame is a pre-configured keyframe (S1004).

[0085] If the current frame is a keyframe, the encoding device performs the following steps:

[0086] The encoding device encodes the keyframes to generate a first bitstream and generates a reconstructed mesh of the keyframes from the first bitstream (S1006). To encode the mesh, the encoding device uses an existing mesh encoding method. Furthermore, the encoding device reconstructs the mesh using a decoding method corresponding to the encoding method before generating a final bitstream associated with the mesh, so that the reconstructed mesh is referenced by the next frame.

[0087] The encoding device saves the restored mesh of the keyframe as the previous frame in the mesh storage unit (S1008). Later, the keyframes saved in the mesh storage unit 104 are used for encoding the next frame.

[0088] If the current frame is not a keyframe but one of the non-keyframes, the encoding device performs the following steps:

[0089] The encoding device extracts motion data using the mesh of the current frame and the reconstructed mesh of the previous frame (S1010). For example, the encoding device extracts motion data for each vertex of the mesh. As another example, the encoding device extracts at least one motion data for each face of the mesh. As yet another example, the encoding device extracts motion data in units of patches containing multiple faces and vertices. Here, the motion data for a vertex, face, or patch is a three-dimensional motion vector that shows how they move.

[0090] The encoding device encodes the motion data to generate a second bitstream, and then generates reconstructed motion data from the second bitstream (S1012).

[0091] The encoding device generates a second bitstream by mapping the motion vectors to two dimensions to produce at least one motion map, downsampling the motion map, and then encoding the downsampled motion map using a video or image compression method. Furthermore, the encoding device generates reconstructed motion data from the second bitstream by reversing the steps described above.

[0092] The encoding device generates a restored current frame by applying the restored motion data to the previous frame (S1014). The encoding device restores the current frame by applying the restored motion data to the previous frame and compensating for the motion.

[0093] The encoding device saves the restored current frame as the previous frame in the mesh storage unit (S1016). Later, the current frame saved in the mesh storage unit 104 is used for encoding the next frame.

[0094] Finally, the encoding device combines the first bitstream and the second bitstream to form a single bitstream (S1018). The encoding device either stores the combined bitstream or transmits it to the decoding device.

[0095] Figure 11 is a flowchart showing a dynamic mesh decoding method according to one embodiment of the present invention.

[0096] After acquiring the bitstream, the decoding device separates the bitstream into a first bitstream and a second bitstream (S1100). Here, the first bitstream is a bitstream in which pre-set keyframes from a number of frames representing the dynamic mesh are encoded, and the second bitstream represents a bitstream in which one of the remaining frames, i.e., non-keyframes, from a number of frames is encoded. The decoding device uses the bitstream header information to separate the bitstreams.

[0097] The decoding device checks whether the bitstream is the first bitstream (S1102), and if it is the first bitstream, it performs the next step.

[0098] The decoding device decodes the first bitstream to restore the mesh of keyframes (S1104). To decode the mesh, the decoding device uses a decoding method corresponding to the encoding method used by the encoding device.

[0099] The decoding device saves the keyframe mesh as the previous frame in the mesh storage unit (S1106). Later, the keyframes saved in the mesh storage unit 206 are used to decode the next frame.

[0100] If the bitstream is a second bitstream, the decryption device performs the following steps:

[0101] The decoding device decodes the second bitstream to reconstruct the motion data of the current frame (S1108). As mentioned above, the reconstructed motion data is a three-dimensional motion vector.

[0102] The decoding device decodes the second bitstream and restores the motion map using a decoding method corresponding to the encoding method used by the dynamic mesh coding device. The decoding device then upsamples the restored motion map and converts the upsampled motion map into motion vectors.

[0103] The decoding device applies motion data to the previous frame to reconstruct the mesh of the current frame (S1110). The decoding device then reconstructs the current frame by applying reconstructed motion data to the previous frame to compensate for the motion.

[0104] The decoding device saves the mesh of the current frame in the mesh storage unit for the previous frame (S1112). Later, the current frame saved in the mesh storage unit 206 is used for decoding the next frame.

[0105] The dynamic mesh coding method and dynamic mesh decoding method using two coordinate system transformations will be described below using the figures shown in Figures 12 and 13.

[0106] Figure 12 is a flowchart showing a dynamic mesh coding method according to another embodiment of the present invention.

[0107] The encoding device acquires the current frame that constitutes the dynamic mesh (S1200).

[0108] The encoding device transforms the original coordinate system of the vertices in the current frame into a first coordinate system that is different from the original coordinate system (S1202).

[0109] The encoding device checks whether the current frame is a pre-configured keyframe (S1204).

[0110] If the current frame is a keyframe, the encoding device performs the following steps:

[0111] The encoding device encodes the keyframes to generate a first bitstream and generates a reconstructed mesh of the keyframes from the first bitstream (S1206). To encode the mesh, the encoding device uses an existing mesh encoding method. Furthermore, the encoding device reconstructs the mesh before generating the final bitstream associated with the mesh so that the reconstructed mesh is referenced by the next frame.

[0112] The encoding device saves the restored mesh of the keyframe as the previous frame in the mesh storage unit (S1208). Later, the keyframes saved in the mesh storage unit 104 are used for encoding the next frame.

[0113] If the current frame is not a keyframe but one of the non-keyframes, the encoding device performs the following steps:

[0114] The encoding device converts the first coordinate system of the current frame to the second coordinate system, and also converts the first coordinate system of the previous frame stored in the mesh storage unit to the second coordinate system (S1210).

[0115] The encoding device extracts motion data using the mesh of the current frame and the reconstructed mesh of the previous frame (S1212). As described above, the motion data is a three-dimensional motion vector.

[0116] The encoding device encodes the motion data to generate a second bitstream, and then generates reconstructed motion data from the second bitstream (S1214).

[0117] The encoding device generates a second bitstream by mapping the motion vectors to two dimensions to produce at least one motion map, downsampling the motion map, and then encoding the downsampled motion map using a video or image compression method. Furthermore, the encoding device generates reconstructed motion data from the second bitstream by reversing the steps described above.

[0118] The encoding device generates a restored current frame by applying the restored motion data to the previous frame in which the coordinate system was transformed (S1216). The encoding device restores the current frame by applying the restored motion data to the previous frame and compensating for the motion.

[0119] The encoding device transforms the second coordinate system of the restored current frame back to the first coordinate system (S1218).

[0120] The encoding device saves the restored current frame as the previous frame in the mesh storage unit (S1220). Later, the current frame saved in the mesh storage unit 104 is used for encoding the next frame.

[0121] Finally, the encoding device combines the first bitstream and the second bitstream to form a single bitstream (S1222). The encoding device either saves the combined bitstream or transmits it to the decoding device.

[0122] Figure 13 is a flowchart showing a dynamic mesh decoding method according to another embodiment of the present invention.

[0123] After acquiring the bitstream, the decoding device separates the first bitstream and the second bitstream from the bitstream (S1300). The decoding device uses the bitstream header information to separate the bitstreams.

[0124] The decoding device checks whether the bitstream is the first bitstream (S1302), and if it is the first bitstream, it performs the following steps.

[0125] The decoding device decodes the first bitstream to restore the mesh of keyframes (S1304). To decode the mesh, the decoding device uses a decoding method corresponding to the encoding method used by the encoding device.

[0126] The decoding device saves the mesh of the keyframe as the previous frame in the mesh storage unit (S1306). Later, the keyframe saved in the mesh storage unit 206 is used to decode the next frame.

[0127] If the bitstream is a second bitstream, the decoder performs the following steps:

[0128] The decoding device converts the first coordinate system of the previous frame to the second coordinate system (S1308).

[0129] The decoding device decodes the second bitstream to reconstruct the motion data of the current frame (S1310). As mentioned above, the reconstructed motion data is a three-dimensional motion vector.

[0130] The decoding device decodes the second bitstream and restores the motion map using a decoding method corresponding to the encoding method used by the dynamic mesh coding device. The decoding device then upsamples the restored motion map and converts the upsampled motion map into motion vectors.

[0131] The decoding device applies motion data to the previous frame to reconstruct the mesh of the current frame (S1312). The decoding device reconstructs the current frame by applying reconstructed motion data to the previous frame to compensate for the motion.

[0132] The decoding device reverses the second coordinate system of the restored current frame back to the first coordinate system (S1314).

[0133] The decoding device saves the mesh of the current frame in the mesh storage unit of the previous frame (S1316). Later, the current frame saved in the mesh storage unit 206 is used for decoding the next frame.

[0134] Finally, the decoding device performs an inverse coordinate transformation on the frames stored in the mesh storage unit, from the first coordinate system to the original coordinate system (S1318).

[0135] Although the flowcharts / timing diagrams in this specification describe each process as being executed sequentially, this is merely an illustrative example of the technical concept of the embodiments of the present invention. In other words, the flowcharts / timing diagrams are not limited to a chronological order, as they can be modified and adapted in various ways by changing the order described in the flowcharts / timing diagrams, or by executing one or more of the processes in parallel, without departing from the essential characteristics of the present invention.

[0136] It should be understood that the exemplary embodiments described above can be embodied in many different ways. The functions or methods described in one or more examples may be embodied in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to particularly emphasize their independent implementation.

[0137] On the other hand, the various functions or methods described in this embodiment are embodied in instruction words stored on a non-temporary recording medium that are read and executed by one or more processors. The non-temporary recording medium includes, for example, any kind of recording device in which data is stored in a form that can be read by a computer system. For example, the non-temporary recording medium includes storage media such as EPROM (erasable programmable read-only memory), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0138] The above description is merely illustrative of the technical concept of the present invention, and any person with ordinary skill in the art to which this embodiment belongs will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Therefore, this embodiment is for illustrative purposes only and not to limit the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention should be interpreted in accordance with the claims, and all technical concepts within the equivalent scope should be interpreted as being included in the scope of rights of the present invention.

[0139] CROSS-REFERENCE TO RELATED APPLICATION This patent application claims priority over patent application No. 10-2021-0043654 filed in Korea on April 2, 2021, and patent application No. 10-2022-0038209 filed in Korea on March 28, 2022, and all contents of those applications are merged into this patent application as references. [Explanation of Symbols]

[0140] 102 Mesh coding section 104, 206 Mesh storage section 106 Motion extraction unit 108 Motion encoding unit 110, 210 Motion compensation unit 112-bitstream composition unit 202 Bitstream Separation Unit 204 Mesh Decoding Unit 208 Motion Decoding Unit 302 First Coordinate System Transformation Unit 402 First Coordinate System Inverse Transform Section 502, 602 Second Coordinate System Transformation Section 504, 604 Second inverse coordinate system transformation section 702 Motion Map Generation Unit 704 Motion Map Downsampling Unit 706 Motion Map Encoding Unit 902 Motion Map Decoding Unit 904 Motion Map Upsampling Unit 906 Motion vector generation unit

Claims

1. A decoding method for decoding a dynamic mesh performed by a dynamic mesh decoding device, After acquiring the bitstream, the first bitstream and the second bitstream are separated from the bitstream. The step of decoding the bitstream is included, The first bitstream is a bitstream in which pre-defined keyframes (key-frames) from a number of frames representing the dynamic mesh are encoded. The second bitstream is a bitstream in which one of the remaining frames, excluding the keyframe, from the large number of frames is encoded. The aforementioned decryption step is, If the bitstream is the first bitstream, The steps include decoding the first bitstream and restoring the mesh of the keyframes, The step includes saving the mesh of the keyframe as the previous frame in the mesh storage unit, If the bitstream is the second bitstream, The steps include decoding the second bitstream to restore the motion data of the current frame, The steps include: applying the motion data to the preceding frame to restore the mesh of the current frame; A decoding method characterized by comprising the step of saving the mesh of the current frame as the previous frame in the mesh storage unit.

2. The decoding method according to claim 1, characterized in that the step of restoring the motion data is to restore the motion data for each vertex of the mesh of the current frame.

3. The decoding method according to claim 1, characterized in that the step of restoring the motion data restores at least one motion data for one face of the mesh of the current frame.

4. The decoding method according to claim 1, characterized in that the step of restoring the motion data is to restore a three-dimensional motion vector as the motion data.

5. The step of restoring the motion data is as follows: The steps include: decoding the second bitstream using a decoding method corresponding to the encoding method used in the dynamic mesh coding device to restore the motion map; The steps include upsampling the restored motion map, The decoding method according to claim 4, further comprising the step of converting the upsampled motion map into the motion vector.

6. An encoding method for encoding a dynamic mesh performed by a dynamic mesh encoding device, The steps include obtaining the current frame that constitutes the dynamic mesh and confirming whether the current frame is a pre-set keyframe, The step of encoding the current frame, The encoding step is, If the current frame is the keyframe, The steps include: encoding the keyframes to generate a first bitstream, and generating a restored mesh of the keyframes from the first bitstream; The step includes saving the restored mesh of the keyframe as the previous frame in the mesh storage unit, If the current frame is not the keyframe, The steps include: extracting motion data using the mesh of the current frame and the reconstructed mesh of the previous frame; The steps include encoding the motion data to generate a second bitstream, and generating reconstructed motion data from the second bitstream, The steps include: applying the restored motion data to the preceding frame to generate the restored current frame; An encoding method characterized by including the step of saving the restored current frame as the immediately preceding frame in the mesh storage unit.

7. The encoding method according to claim 6, further comprising the step of combining the first bitstream and the second bitstream to form a single bitstream.

8. The encoding method according to claim 6, characterized in that the step of extracting motion data is performed by extracting motion data for each vertex of the mesh of the current frame.

9. The encoding method according to claim 6, characterized in that the step of extracting motion data includes extracting at least one motion data for one face of the mesh of the current frame, and interpolating the motion data of the vertices constituting the one face to extract the at least one motion data.

10. The encoding method according to claim 6, characterized in that the step of extracting motion data is to extract a three-dimensional motion vector as the motion data.

11. The step of generating the aforementioned restored motion data is: The steps include: mapping the motion vectors in two dimensions to generate at least one motion map; The steps include downsampling the aforementioned motion map, The encoding method according to claim 10, further comprising the step of generating the second bitstream by encoding the downsampled motion map using a video or image compression method.

12. The encoding method according to claim 11, characterized in that the step of generating the motion map is to generate three motion maps by projecting the motion vector onto the x, y, and z axes.

13. A method for transmitting a bitstream generated by an encoding method for a dynamic mesh, The method for transmitting the bitstream includes the step of transmitting the bitstream, The aforementioned encoding method is The steps include obtaining the current frame that constitutes the dynamic mesh and confirming whether the current frame is a pre-set keyframe, The step of encoding the current frame, The encoding step is, If the current frame is the keyframe, The steps include: encoding the keyframes to generate a first bitstream, and generating a restored mesh of the keyframes from the first bitstream; The step includes saving the restored mesh of the keyframe as the previous frame in the mesh storage unit, If the current frame is not the keyframe, The steps include: extracting motion data using the mesh of the current frame and the reconstructed mesh of the previous frame; The steps include encoding the motion data to generate a second bitstream, and generating reconstructed motion data from the second bitstream, The steps include: applying the restored motion data to the preceding frame to generate the restored current frame; A method for transmitting a bitstream, comprising the step of saving the restored current frame as the previous frame in the mesh storage unit.

Images