Method and apparatus for mesh decoding / encoding

The mesh decoding/encoding method optimizes video compression by predicting subdivision values and using probability tables, addressing inefficiencies in existing methods and enhancing coding efficiency for video/image data transmission and storage.

WO2024242429A1PCT designated stage expired Publication Date: 2024-11-28INTELLECTUAL DISCOVERY CO LTD

Patent Information

Application Number
PCT/KR2024/006810
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-19
Filing Date
2024-05-20
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Current video compression methods fail to efficiently encode and decode mesh data, leading to suboptimal coding efficiency and increased computational resources due to inadequate handling of spatial-temporal and inter-view redundancies.

Method used

A mesh decoding/encoding method and device that processes bitstreams containing sub-bitstreams for meshes and difference vector transform coefficients, employing subdivision techniques and probability tables to predict and encode mesh structures, thereby optimizing coding efficiency.

Benefits of technology

The proposed method achieves high coding efficiency by predicting subdivision values and using initial probability tables, reducing computational resources and improving video/image transmission and storage efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2024006810_28112024_PF_FP_ABST
    Figure KR2024006810_28112024_PF_FP_ABST
Patent Text Reader

Abstract

A mesh decoding method and apparatus according to the present disclosure may: receive a bitstream including one or more sub-bitstreams; reconstruct a base mesh of the current patch by decoding a sub-bitstream related to a mesh; reconstruct a difference vector by decoding a sub-bitstream related to a difference vector transform coefficient; and reconstruct a mesh of the current patch on the basis of at least one of the base mesh and the difference vector.
Need to check novelty before this filing date? Find Prior Art

Description

Mesh decryption / encoding method and device

[0001] The present invention relates to an encoder and a decoding device, and more particularly, to a method and device for mesh encoding and decoding.

[0002] Video images are compressed and encoded to remove spatiotemporal redundancy and inter-view redundancy, so that they can be transmitted over communication lines or stored in a form suitable for storage media.

[0003] The present invention proposes a method and device for decoding / encoding a mesh.

[0004] A mesh decoding method and device according to the present disclosure can receive a bitstream including one or more sub-bitstreams. Here, the one or more sub-bitstreams can include at least one of a sub-bitstream regarding a mesh, a sub-bitstream regarding difference vector transform coefficients, or a sub-bitstream regarding a texture map.

[0005] The mesh decoding method and device according to the present disclosure can restore a base mesh of a current patch by decoding a sub-bitstream regarding the mesh, restore a difference vector by decoding a sub-bitstream regarding the difference vector transform coefficients, and restore a mesh of the current patch based on at least one of the base mesh or the difference vector.

[0006] In the mesh decoding method and device according to the present disclosure, the step of restoring the mesh of the current patch may include the step of performing sub-division on the restored basic mesh based on a predetermined sub-division value. Here, the sub-division value may indicate the number of times the sub-division is performed.

[0007] In the mesh decoding method and device according to the present disclosure, the sub-division value can be signaled in patch units from the bitstream.

[0008] In the mesh decoding method and device according to the present disclosure, the sub-division value can be signaled from the bitstream based on a patch-based sub-division flag. Here, the patch-based sub-division flag can indicate whether the patch-based sub-division value is used.

[0009] In the mesh decoding method and device according to the present disclosure, the sub-division value can be derived based on the sub-division value of the previous patch and the delta sub-division value signaled for the current patch.

[0010] In the mesh decryption method and device according to the present disclosure, the previous patch may mean the first patch in the decryption order or a patch having a decryption order immediately before the current patch.

[0011] In the mesh decoding method and device according to the present disclosure, any one of the plurality of triangles belonging to the current patch may have a sub-division value different from another one.

[0012] In the mesh decoding method and device according to the present disclosure, the sub-division value of the current triangle among the plurality of triangles can be derived based on the sub-division values ​​of the surrounding triangles adjacent to the current triangle.

[0013] In the mesh decoding method and device according to the present disclosure, the peripheral triangle can be specified based on information signaled from the bitstream. Here, the information can indicate any one of a plurality of peripheral triangles adjacent to the current triangle.

[0014] In the mesh decoding method and device according to the present disclosure, a probability table stored in any one of the pre-restored patches can be used as an initial probability table for the current patch.

[0015] In the mesh decoding method and device according to the present disclosure, any one of the pre-restored patches can be selected based on reference patch index information signaled through the bitstream.

[0016] In the mesh decryption method and device according to the present disclosure, any one of the pre-restored patches may be a patch having a decryption order prior to the current patch.

[0017] In the mesh decoding method and device according to the present disclosure, the pre-restored patch may belong to a frame different from the current frame to which the current patch belongs, and any one of the pre-restored patches may be a patch having the same patch index as the current patch.

[0018] The mesh encoding method and device according to the present disclosure can generate a sub-bitstream regarding the mesh by encoding a basic mesh of a current patch, generate a sub-bitstream regarding a difference vector transform coefficient by encoding a difference vector, and generate a bitstream including at least one of the sub-bitstream regarding the mesh, the sub-bitstream regarding the difference vector transform coefficient, or the sub-bitstream regarding a texture map.

[0019] A computer-readable digital storage medium is provided having stored thereon video / image information that causes a method of decrypting a message according to the present disclosure to be performed.

[0020] A computer-readable digital storage medium storing video / image information generated by a mesh encoding method according to the present disclosure is provided.

[0021] A method and device for transmitting video / image information generated by a mesh encoding method according to the present disclosure are provided.

[0022] According to the multi-subdivision method proposed in the present invention, the number of subdivisions of the current patch can be predicted from surrounding patches, thereby providing high encoding efficiency for the number of subdivisions of the current patch.

[0023] The entropy coding method proposed in the present invention can provide high encoding efficiency by predicting an initial probability value for entropy coding of the current patch and using this for entropy coding of the current patch.

[0024] FIG. 1 illustrates a dynamic mesh decryption device according to the present disclosure.

[0025] Figure 2 illustrates a mesh decryption method according to the present disclosure.

[0026] Figure 3 illustrates patch division of a mesh according to the present disclosure.

[0027] FIG. 4 is an example of a connected triangular surface inside a patch according to the present disclosure.

[0028] Figure 5 illustrates a subdivision of one triangle according to the present disclosure.

[0029] Figure 6 illustrates a case where the sub-division value of the current triangle is smaller than the sub-division values ​​of the surrounding triangles.

[0030] Figure 7 illustrates a case where the sub-division value of the current triangle is smaller than the sub-division values ​​of the surrounding triangles.

[0031] Figure 8 is a drawing illustrating a case where the sub-division value of the current triangle is greater than the sub-division values ​​of the surrounding triangles.

[0032] Figure 9 illustrates a case where there are two surrounding triangles and the subdivision value of the current triangle is smaller than the subdivision value of the surrounding triangles.

[0033] Figure 10 illustrates a three-dimensional patch divided into various sizes.

[0034] FIG. 11 illustrates a method for initializing a probability table of patch units in a static mesh decoding unit according to the present disclosure.

[0035] FIG. 12 illustrates a method for initializing a patch-unit optional probability table in a static mesh decoding unit according to the present disclosure.

[0036] FIG. 13 illustrates a method for initializing an adaptive probability table per patch in a static mesh decoding unit according to the present disclosure.

[0037] FIG. 14 illustrates a method of referencing a patch unit probability table in a static mesh decoding unit according to the present disclosure.

[0038] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0039] Throughout this specification, when a part is said to be 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with another element in between.

[0040] Additionally, whenever a part throughout this specification is said to "include" a component, this does not mean that other components are excluded, but rather that other components may be included, unless specifically stated otherwise.

[0041] Additionally, while terms such as first, second, etc. may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0042] Additionally, in the embodiments of the devices and methods described herein, some components of the devices or some steps of the methods may be omitted. Furthermore, the order of some components of the devices or some steps of the methods may be changed. Furthermore, other components or other steps may be inserted into some components of the devices or some steps of the methods.

[0043] Additionally, some components or some steps of the first embodiment of the present invention may be added to the second embodiment of the present invention, or some components or some steps of the second embodiment may be replaced.

[0044] In addition, the components shown in the embodiments of the present invention are independently depicted to represent different characteristic functions, and this does not mean that each component is composed of separate hardware or a single software component. That is, each component is described by listing each component for convenience of explanation, and at least two components among each component may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0045] First, the terms used in this application are briefly explained as follows.

[0046] The decoding device (Video Decoding Apparatus) described below may be a device included in a server terminal such as a civilian security camera, a civilian security system, a military security camera, a military security system, a personal computer (PC), a laptop computer, a portable multimedia player (PMP), a wireless communication terminal, a smart phone, a TV application server, a service server, etc., and may refer to various devices including user terminals such as various devices, communication devices such as a communication modem for communicating with a wired or wireless communication network, a memory for storing various programs and data for decoding a video or for inter or intra prediction for decoding, and a microprocessor for executing a program for calculation and control.

[0047] In addition, an image encoded into a bitstream by an encoder can be transmitted to an image decoding device through wired or wireless communication networks such as the Internet, a local area network, a wireless LAN, a WiBro network, a mobile communication network, etc. in real time or non-real time, or through various communication interfaces such as a cable or a universal serial bus (USB), and can be decoded, restored into an image, and played back. Alternatively, the bitstream generated by the encoder can be stored in memory. The memory can include both volatile memory and non-volatile memory. In this specification, the memory can be expressed as a recording medium that stores a bitstream.

[0048] Typically, a video may be composed of a series of pictures, and each picture may be divided into coding units such as blocks. In addition, those skilled in the art to which this embodiment pertains will understand that the term "picture" described below may be replaced with other terms having equivalent meanings, such as "image" or "frame." In addition, those skilled in the art to which this embodiment pertains will understand that the term "coding unit" may be replaced with other terms having equivalent meanings, such as "unit block" or "block."

[0049] Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, duplicate descriptions of identical components will be omitted.

[0050] FIG. 1 illustrates a dynamic mesh decryption device according to the present disclosure.

[0051] A dynamic mesh decoding device according to the present disclosure can decode a bitstream to restore at least one of a mesh or a texture map.

[0052] The dynamic mesh decoding device may include at least one of a bitstream distribution unit, a motion decoding unit, a basic mesh restoration unit, a static mesh decoding unit, a mesh storage unit, an inverse quantization unit, a first video decoding unit, a second video decoding unit, an image unpacking unit, an inverse quantization unit, an inverse transformation unit, an enhanced mesh restoration unit, or a color space transformation unit.

[0053] The bitstream distribution unit can output a plurality of sub-bitstreams from the bitstream. The plurality of sub-bitstreams can include at least one of a sub-bitstream regarding a mesh (hereinafter referred to as a first sub-bitstream), a sub-bitstream regarding a difference vector transform coefficient (hereinafter referred to as a second sub-bitstream), or a sub-bitstream regarding a texture map (hereinafter referred to as a third sub-bitstream).

[0054] The first sub-bitstream can be passed to a static mesh decoder and / or a motion decoder. For example, if the current patch is an intra patch, the first sub-bitstream can be passed to the static mesh decoder. Alternatively, if the current patch is an inter patch, the first sub-bitstream can be passed to the motion decoder. The second sub-bitstream can be passed to the first video decoder. The third sub-bitstream can be passed to the second video decoder.

[0055] The static mesh decoding unit can obtain a base mesh from the first sub-bitstream. The base mesh can be transmitted to a mesh storage unit and / or a dequantization unit. In this case, the base mesh can be obtained based on a static mesh compression method based on an edge-breaker algorithm.

[0056] The mesh storage unit can store the obtained basic mesh. In addition, the stored basic mesh can be transmitted to the motion decoding unit and / or the basic mesh restoration unit.

[0057] The motion decoding unit can obtain motion for vertices of the base mesh based on at least one of the first sub-bitstream or the base mesh. The first sub-bitstream can include motion information for vertices of the base mesh. Here, the motion information can include a motion vector. The motion for vertices of the base mesh can be transmitted to the base mesh restoration unit.

[0058] The basic mesh restoration unit may restore the basic mesh of the current frame based on at least one of the motion of the vertices of the basic mesh or the basic mesh received from the mesh storage unit. Here, the process of restoring the basic mesh may include a process of adding the vertex coordinates of the basic mesh and the motion vector for the vertices of the basic mesh. The restored basic mesh may be transmitted to the inverse quantization unit.

[0059] The dequantization unit can perform dequantization on the restored base mesh. For example, dequantization can be performed on the vertex coordinates of the base mesh. The dequantized base mesh can be passed to the enhanced mesh restoration unit.

[0060] The first video decoding unit can obtain a difference image from the second sub-bitstream. At this time, the first video decoding unit may use a video decoding device equipped with a codec such as H.264 / AVC, H.265 / HEVC, or H.266 / VVC. Alternatively, the first video decoding unit may use an image decoding device equipped with a codec such as PNG, TIF, JPEG, or JPEG2000. Information about the decoding device may be included in the bitstream in the form of upper-level information. The difference image may be transmitted to the image unpacking unit.

[0061] The image unpacking unit can extract each sample value of the difference image according to a pre-defined scan order to obtain a quantized difference vector transform coefficient. The quantized difference vector transform coefficient can be passed to the difference vector dequantization unit.

[0062] Alternatively, the second sub-bitstream may be entropy decoded to obtain the quantized difference vector transform coefficients without going through the first video decoding unit and the image unpacking unit. The quantized difference vector transform coefficients may be passed to the difference vector dequantization unit.

[0063] The difference vector inverse quantization unit can perform inverse quantization on the quantized difference vector transform coefficients to generate difference vector transform coefficients. At this time, the quantized difference vector transform coefficients may be a vector having one or more dimensions. Inverse quantization may be performed using different quantization parameters for each dimension. The generated difference vector transform coefficients may be transmitted to the inverse transform unit.

[0064] The inverse transform unit can restore the difference vector by performing an inverse transform on the difference vector transform coefficients. Any of the wavelet transforms can be used for the inverse transform. The restored difference vector can be passed to the enhanced mesh restoration unit.

[0065] The enhanced mesh restoration unit can restore a mesh based on at least one of the restored base mesh and the difference vector. The restored mesh can be output from the dynamic mesh decoding device.

[0066] The second video decoding unit can generate a texture map by decoding the third sub-bitstream. At this time, the second video decoding unit can use a video decoding device equipped with a codec such as H.264 / AVC, H.265 / HEVC, or H.266 / VVC. Alternatively, an image decoding device equipped with a codec such as PNG, TIF, JPEG, or JPEG2000 can be used. Information about the decoding device can be included in the bitstream in the form of upper-level information. The generated texture map can be transmitted to a color space conversion unit.

[0067] The color space conversion unit can perform color space conversion on the texture map. At this time, the color space conversion may be YUV-RGB conversion. Alternatively, the color space conversion may be YCgCo-RGB or YCoCg-RGB conversion. The color space converted texture map can be output from the dynamic mesh decoding device.

[0068] Figure 2 illustrates a mesh decryption method according to the present disclosure.

[0069] Referring to Fig. 2, a bitstream can be received (S200).

[0070] A bitstream according to the present disclosure may include one or more sub-bitstreams. The one or more sub-bitstreams may include at least one of a sub-bitstream regarding a mesh, a sub-bitstream regarding difference vector transform coefficients, or a sub-bitstream regarding a texture map.

[0071] Referring to FIG. 2, the sub-bitstream for the mesh can be decoded to restore the basic mesh of the current patch (S210). The method for restoring the basic mesh is the same as that described with reference to FIG. 1, and a detailed description thereof will be omitted here.

[0072] Referring to FIG. 2, the sub-bitstream regarding the difference vector transform coefficients can be decoded to restore the difference vector (S220). The method for restoring the difference vector is the same as that described with reference to FIG. 1, and a detailed description thereof will be omitted here.

[0073] Referring to FIG. 2, a mesh can be restored based on at least one of the base mesh or the difference vector (S230). The mesh according to the present disclosure can be restored based on multiple subdivisions of the base mesh. Hereinafter, the multiple subdivision method will be described with reference to FIGS. 3 to 9.

[0074] Figure 3 illustrates patch division of a mesh according to the present disclosure.

[0075] A mesh according to the present disclosure may be referred to as a dynamic mesh. Here, the dynamic mesh may be data representing the surface of a three-dimensional object. In addition, the dynamic mesh may have one or more frames and / or texture maps. The frame may be geometric information of the outermost surface of a three-dimensional object expressed by connecting a plurality of triangular faces, and the texture map may be attribute information of the outermost surface of the three-dimensional object. At this time, for each of the frames and the texture map, one or more patches may be generated by grouping a plurality of triangular faces constituting the outermost surface of the three-dimensional object. Each patch may include a plurality of triangular faces connected to each other.

[0076] A one-to-one mapping between the triangles of a frame and the triangles of a texture map may be possible. The multiple triangles that make up the texture map may be grouped into triangles corresponding to each patch of the frame. The same number of patches may be generated for the texture map as there are frames.

[0077] Information about the above patch can be transmitted from the encoding device to the decoding device via high-level information of the bitstream. Here, the information about the patch can include three-dimensional and / or two-dimensional coordinates of the patch. The information about the patch can include three-dimensional size information and / or two-dimensional size information of the patch.

[0078] FIG. 4 is an example of a connected triangular surface inside a patch according to the present disclosure.

[0079] Referring to FIG. 4, a triangular face expressed as a basic mesh is illustrated within a patch. One or more basic meshes may be included within a single 3D patch. The basic mesh may have information about multiple connected triangular faces. Each triangular face may be composed of a basic mesh vertex and a basic mesh edge. A basic mesh vertex may be defined as a point where two line segments forming a triangular face meet, and a basic mesh edge may be defined as a line segment forming a triangular face.

[0080] Figure 5 illustrates a subdivision of one triangle according to the present disclosure.

[0081] Referring to FIG. 5, the enhanced mesh restoration unit may perform sub-division on the restored base mesh based on the sub-division value. The sub-division value may refer to the number of sub-divisions. The number of sub-divisions may be referred to as the number of repetitions of the sub-division. The sub-division may be performed recursively and / or hierarchically. As in the example of FIG. 5, sub-division may generate sub-division vertices at the midpoints of each base mesh edge. In addition, sub-division may be recursively performed on edges to which sub-division vertices and base mesh vertices are connected. In other words, sub-division may be recursively performed on edges to which base mesh vertices are connected and / or edges to which sub-division vertices are connected. Then, the 3D position of the base mesh may be corrected based on the 3D position values ​​(or coordinates) of the base mesh vertices and one or more sub-division vertices and the restored difference vector received from the inverse transformation unit. Through this correction, the mesh may be restored. For example, the 3D position of the base mesh can be corrected by adding the 3D position values ​​of the base mesh vertices and one or more sub-division vertices and the restored difference vector for each vertex.

[0082] A sub-division value indicating the number of repetitions of the above sub-division can be transmitted from an encoding device to a decoding device. The sub-division value can be encoded on a patch basis.

[0083] For example, a patch-based sub-division flag can be obtained from a bitstream. The patch-based sub-division flag can indicate whether a patch-based sub-division value is used. Alternatively, the patch-based sub-division flag can indicate whether a patch-based encoded sub-division value is parsed from the bitstream. When the value of the patch-based sub-division flag is 1, a patch-based encoded sub-division value can be parsed from the bitstream. On the other hand, when the value of the patch-based sub-division flag is 0, a frame-based sub-block value can be used as a current patch's sub-division value. Table 1 below is an example of a syntax for decoding a sub-division value and the semantics of the sub-division value.

[0084]

[0085] mdu_subdivision_iteration_count[ tileID ][ patchIdx ] indicates the number of iterations used for the subdivision in a patch with index patchIdx of the current atlas tile, with tile ID equal to tileID. When mdu_subdivision_iteration_count[ tileID ][ patchIdx ] is not present, its values ​​are inferred to be equal to afps_vmc_ext_subdivision_iteration_count.

[0086] The sub-segmentation value can change depending on the importance of the patch. In other words, the sub-segmentation value can adaptively change depending on the classification, type, and properties of the patch. For example, a patch representing a part such as a human face can express the details of the face by performing a relatively large number of sub-segments. Conversely, a flat part such as a human back can perform a relatively small number of sub-segments. In such cases, the encoding device and the decoding device can derive the sub-segmentation value in the same way without signaling per patch. In addition, the more similar the location and size of the patches in 3D space, the closer the encoding / decoding order of the patches can be. Furthermore, when the encoding / decoding order of the patches is close, the sub-segmentation structure can be relatively more similar. Therefore, encoding the sub-segmentation value using the delta sub-segmentation value can be more efficient.

[0087] For example, if the sub-division value indicating the number of sub-divisions is determined on a patch basis, the triangle of the basic mesh in the patch can be sub-divided by the maximum number of sub-divisions determined above. That is, if the sub-division value of the current patch is determined (or signaled) as 2, the triangle of the basic mesh in the current patch can be recursively / hierarchically sub-divided up to 2 times. At this time, the number of sub-divisions can be determined within the range of the number of sub-divisions per patch for each triangle included in the basic mesh. That is, the number of sub-divisions determined on a patch basis can be the maximum number of sub-divisions.

[0088] Alternatively, if the sub-division value indicating the number of sub-divisions is determined on a patch basis, all triangles of the base mesh within the patch can be sub-divided by the number of sub-divisions determined on a patch basis.

[0089] Alternatively, for the first patch of the current frame, the subdivision value can be transmitted from the encoder to the decoder. After all the current patch information has been parsed, the current subdivision value can be stored. Furthermore, for subsequent patches, delta subdivision values ​​(delta_subdivisions) with respect to the previous patch can be transmitted from the encoder. The subdivision value for the current patch can be derived by adding the previously stored subdivision values ​​of the previous patch and the transmitted delta subdivision values.

[0090] The delta sub-division value may be the difference between a specific patch within a frame (e.g., the first patch in the sub-coding / decoding order) and the sub-division value of the current patch. Alternatively, the delta sub-division value may be the difference between the sub-division value of the current patch and the immediately previous patch in the sub-coding / decoding order of the patches within the frame.

[0091] Alternatively, for the first patch of the current frame, the subdivision value may be transmitted from the encoding device to the decoding device. After all current patch information is parsed, the current subdivision value may be stored. Then, for subsequent patches, the delta subdivision value (delta_subdivisions) of the previous patch may be transmitted from the encoding device. The subdivision value for the current patch may be derived by adding the pre-stored subdivision value of the previous patch and the delta subdivision value. At this time, the delta subdivision flag (delta_subdivisions_flag) may be parsed before parsing the delta subdivision value. If the value of the delta subdivision flag is 1, the delta subdivision value may be parsed. In this case, the subdivision value for the current patch may be derived by adding the pre-stored subdivision value of the previous patch and the delta subdivision value. On the other hand, if the value of the delta subdivision flag is 0, the delta subdivision value may be regarded as 0. In this case, the pre-stored subdivision value may be derived as the subdivision value for the current patch. Tables 3 and 4 below are examples of syntax in which delta subdivision values ​​are signaled.

[0092]

[0093]

[0094] Different sub-division values ​​can be used for each triangle of a patch. The static mesh decoder can sequentially decode triangles according to a predetermined encoding / decoding order. For example, the encoding / decoding order of triangles can be determined based on an edge-breaker algorithm. Based on the encoding / decoding order, delta coding can be applied to the sub-division values ​​of each triangle. Tables 5 and 6 below are examples of syntax for using different sub-division values ​​for each triangle. Information indicating whether to use different sub-division values ​​for each triangle can be parsed, and the delta sub-division values ​​can be parsed for each triangle based on the value of the information. In this case, the sub-division value for the current triangle can be derived by adding the parsed delta sub-division value for the current triangle to the sub-division value of the triangle with the previous encoding / decoding order. Information indicating whether to use different sub-division values ​​for each triangle can be encoded for each patch. That is, one of the multiple patches can use different subdivision values ​​for the triangles that make up the patch, and another of the multiple patches can use the same subdivision values ​​for the triangles that make up the patch.

[0095]

[0096]

[0097] Depending on the encoding / decoding order of the triangle, there may be one or more surrounding triangles adjacent to the current triangle. In this case, a prediction can be performed on the sub-division value of the current triangle based on the sub-division values ​​of the pre-encoded surrounding triangles. The difference between the original sub-division value of the current triangle and the sub-division values ​​of the surrounding triangles can be transmitted from the encoding device to the decoding device. That is, the sub-division value of the current triangle can be derived based on the sub-division values ​​of the pre-decoded surrounding triangles and the signaled differential sub-division value. If the current triangle does not have surrounding triangles, the sub-division value transmitted in a patch unit can be used as a prediction value for the sub-division value of the current triangle. Alternatively, if the current triangle has one surrounding triangle, the sub-division value of the corresponding surrounding triangle can be used as the prediction value. If the current triangle has two or more surrounding triangles, the average value of the sub-division values ​​of the two or more surrounding triangles can be used as the prediction value. Table 7 below is an example of a method for deriving a prediction value for the subdivision value of the current triangle based on the number of surrounding triangles adjacent to the current triangle.

[0098]

[0099] In order to obtain a predicted value for the sub-division value of the current triangle, the number of pre-restored surrounding triangles adjacent to the current triangle can be checked. If the number of surrounding triangles is 0, the sub-division value of the patch including the current triangle can be used as the predicted value. If the number of surrounding triangles is 1, the sub-division value of the corresponding triangle can be used as the predicted value. If the number of surrounding triangles is 2, information (nb_pred_flag) indicating the position of the surrounding triangle that has the predicted value for the sub-division value of the current triangle can be parsed. If the value of the parsed information is 0, the sub-division value of the first surrounding triangle can be used as the predicted value. On the other hand, if the value of the information is 1, the sub-division value of the second surrounding triangle can be used as the predicted value. Here, the first surrounding triangle can refer to a triangle adjacent to an edge connected to the left of a reference edge, and the second surrounding triangle can refer to a triangle adjacent to an edge connected to the right of a reference edge. Alternatively, the first peripheral triangle may refer to a triangle adjacent to an edge connected to the right of the reference edge, and the second peripheral triangle may refer to a triangle adjacent to an edge connected to the left of the reference edge. For example, a syntax such as the following Table 8 may be used.

[0100]

[0101] Figure 6 illustrates a case where the sub-division value of the current triangle is smaller than the sub-division values ​​of the surrounding triangles.

[0102] Figure 6 is an example of a case where the sub-division value of the current triangle is 1 less than the sub-division values ​​of the surrounding triangles.

[0103] Based on the sub-division values ​​of the above-derived patch units, sub-division can be performed on the current triangle. At this time, if the sub-division values ​​of the current triangle and the sub-division values ​​of the surrounding triangles are different, the triangles can be additionally divided to generate edges and triangles.

[0104] As in the example of Fig. 6, if there is one surrounding triangle of the current triangle and the sub-division value of the current triangle is smaller than the sub-division value of the surrounding triangle, an edge connecting the vertices adjacent to the current triangle among the first sub-division vertices of the surrounding triangle and one vertex diagonally opposite to those vertices in the current triangle can be created to divide the current triangle into two triangles.

[0105] Figure 7 illustrates a case where the sub-division value of the current triangle is smaller than the sub-division values ​​of the surrounding triangles.

[0106] Figure 7 is an example of a case where the sub-division value of the current triangle is 2 smaller than the sub-division values ​​of the surrounding triangles.

[0107] As in the example of Fig. 7, if there is one surrounding triangle of the current triangle and the sub-division value of the current triangle is smaller than the sub-division value of the surrounding triangle and the difference is 2, the current triangle can be divided into multiple triangles by creating multiple edges connecting the vertices adjacent to the current triangle among the first sub-division vertices and the second sub-division vertices of the surrounding triangle and the vertices diagonally to the corresponding vertices in the current triangle.

[0108] Figure 8 is a drawing illustrating a case where the sub-division value of the current triangle is greater than the sub-division values ​​of the surrounding triangles.

[0109] Figure 8 is an example of a case where the sub-division value of the current triangle is 1 greater than the sub-division values ​​of the surrounding triangles.

[0110] If the subdivision value of the current triangle is greater than the subdivision value of the surrounding triangle, the surrounding triangle can be additionally divided.

[0111] As in the example of Fig. 8, if there is one surrounding triangle of the current triangle, and the sub-division value of the current triangle is greater than the sub-division value of the surrounding triangle and the difference is 1, the neighboring triangle can be divided by creating an edge by connecting the second sub-division vertices of the current triangle adjacent to the surrounding triangle and the first sub-division vertices of the neighboring triangle that are not adjacent to the current triangle.

[0112] Figure 9 illustrates a case where there are two surrounding triangles and the subdivision value of the current triangle is smaller than the subdivision value of the surrounding triangles.

[0113] Figure 9 is an example of a case where there are two surrounding triangles and the sub-division value of the current triangle is smaller than the sub-division value of the surrounding triangles.

[0114] As in the case of Fig. 9, if the sub-division value of the current triangle is smaller than the sub-division values ​​of the surrounding triangles, additional sub-division can be performed on the current triangle to divide it into four faces, and then an edge can be created to further divide it into two faces.

[0115] Here, additional subdivisions can be performed by the difference between the minimum value of the subdivision values ​​of the surrounding triangles and the subdivision value of the current triangle. Subsequently, additional subdivisions using edges can be performed.

[0116] Figure 10 illustrates a three-dimensional patch divided into various sizes.

[0117] Dynamic meshes are typically data representing the surface of a three-dimensional object. As illustrated in the example of Fig. 10, the surface of a three-dimensional object can be divided into one or more patches. The size of each patch may vary depending on surface property information. Alternatively, the patch sizes may be determined differently based on the surface curvature of the three-dimensional object or user parameters. Additionally, adjacent patches may have identical / similar property information and motion information.

[0118] FIG. 11 illustrates a method for initializing a probability table of patch units in a static mesh decoding unit according to the present disclosure.

[0119] The static mesh decoding unit can decode the base mesh based on patch information signaled through the upper level of the bitstream. Decoding of the base mesh can be performed in the order of patch indices. The coordinates of the 3D vertices, the coordinates of the 2D vertices, and the triangles of the base mesh corresponding to each patch can be restored.

[0120] As in the example of Fig. 11, probability values ​​for entropy decoding at the point in time when decoding for a patch is completed can be stored and used in the next patch. At this time, probability values ​​for all symbols can be stored in the form of a table, and there can be multiple probability values ​​for a single symbol depending on the context.

[0121] For consecutive patches, the stored probability value for the current patch can be used as the initial probability value for the next patch, allowing entropy decoding of the symbols in the patch. Encoding devices typically determine the order of patches based on their size and proximity. Therefore, patch indices that are closer together tend to have similar probability values, enabling more efficient symbol compression.

[0122] FIG. 12 illustrates a method for initializing a patch-unit optional probability table in a static mesh decoding unit according to the present disclosure.

[0123] Referring to FIG. 12, after decryption of a patch is completed, the probability values ​​of the entropy decoder can be stored, and the stored probability values ​​can be used as initial probability values ​​in subsequent patches. In this case, the probability table stored in any of the previously restored patches can be used as the initial probability table for the current patch.

[0124] For example, one of the pre-restored patches can be selected based on reference patch index information signaled from the encoding device on a patch-by-patch basis.

[0125] Alternatively, based on the location information of the patch, one of the pre-restored patches may be selected. The probability table stored in the patch closest to the pre-restored patch in three-dimensional space may be used as the initial probability table for the current patch.

[0126] Alternatively, based on the size information of the patch, one of the previously restored patches may be selected. The probability table stored in the patch with the most similar size to the current patch among the previously restored patches may be used as the initial probability table for the current patch.

[0127] Alternatively, one of the restored patches may be selected based on any one of the values ​​of the width, height, and depth of the patch. The probability table stored in the patch among the restored patches whose values ​​are most similar to the current patch may be used as the initial probability table of the current patch. Here, the value of one of the width, height, and depth of the patch may be defined as a weighted larger or smaller value among the width, height, and depth of the patch. Alternatively, the value may be defined as an average value of at least two of the width, height, or depth of the patch.

[0128] Alternatively, one of the pre-restored patches can be selected based on the vertex density of the base mesh within the patch. The probability table stored in the patch with the density most similar to the current patch among the pre-restored patches can be used as the initial probability table for the current patch.

[0129] Alternatively, one of the pre-restored patches may be selected based on the number of vertices of the base mesh within the patch. The probability table stored in the patch among the pre-restored patches whose number of vertices of the base mesh within the patch is most similar to the current patch may be used as the initial probability table for the current patch.

[0130] FIG. 13 illustrates a method for initializing an adaptive probability table per patch in a static mesh decoding unit according to the present disclosure.

[0131] Referring to FIG. 13, after decryption for one patch is completed, the probability value of the entropy decoder can be stored, and the stored probability value can be used as the initial probability value in a subsequent patch.

[0132] Among the restored patches, a probability table stored in a patch specified by reference patch index information signaled from an encoding device on a patch-by-patch basis can be used as an initial probability table of the current patch.

[0133] Based on properties such as the location, size, density, or number of vertices of the base mesh of the patch, the probability table stored in the patch with the most similar properties to the current patch among the restored patches can be used as the initial probability table for the current patch.

[0134] For example, after a certain patch index, the probability table stored in the previous patch can optionally be used as the initial probability table for the current patch.

[0135] Here, a specific patch index can be signaled on a frame-by-frame basis. Alternatively, the specific patch index can be determined as the index of the first patch whose patch size is smaller than a specific value. In this case, the reference patch size can be any one of width, height, or depth.

[0136] Alternatively, after a specific patch index, a probability table that is identically predefined for the encoding device and the decoding device may be used as the initial probability table of the current patch. In other words, a selection mode may be defined in which the probability table of the patch having the first specific patch index is selectively used as the initial probability table of the current patch. Alternatively, a last mode may be defined in which the probability table of the previous patch in the encoding / decoding order is used as the initial probability table of the current patch. The first specific patch index used in the selection mode may be determined by the method described above.

[0137] As illustrated in FIG. 13, either the selection mode or the last mode may be selectively used based on the second specific patch index. For example, as illustrated in FIG. 13, the selection mode may be applied up to a patch having the second specific patch index, and the last mode may be applied after the patch having the second specific patch index. Alternatively, the last mode may be applied up to a patch having the second specific patch index, and the selection mode may be applied after the patch having the second specific patch index. The second specific patch index may be a separately signaled value, or may be a value determined according to encoding information pre-defined in the encoding / decoding device. For example, the second specific patch index may be the index of a patch having a smallest index value among patches smaller than (or smaller than or equal to) a specific size.

[0138] FIG. 14 illustrates a method of referencing a patch unit probability table in a static mesh decoding unit according to the present disclosure.

[0139] Referring to Fig. 14, after decoding for a patch is completed, the stored probability table can be used in the next frame. Here, if inter prediction is performed for the (T+1)th frame, the probability table stored for a patch with the same patch index can be used as the initial probability table for the current patch.

[0140] Alternatively, a flag indicating whether to use the probability table of the restored patch within the current frame or the probability table of another restored frame can be signaled on a patch-by-patch basis, and one method can be selected to initialize the probability table based on the flag. If it is decided to use another restored frame, information about the frame to reference can be additionally signaled, and the probability table of the frame specified by the information can be used as the initial probability table of the current patch.

[0141] If the current frame and the reference frame have different numbers of patches, information about patch indices can be additionally signaled, and the probability table of the patch specified by the information can be used as the initial probability table of the current patch. Conversely, if the current frame and the reference frame have the same number of patches, the probability table of the patch with the same patch index as the current patch within the reference frame can be used as the initial probability table of the current patch.

[0142] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0143] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0144] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

Claims

1. A step of receiving a bitstream; wherein the bitstream includes one or more sub-bitstreams, and the one or more sub-bitstreams include at least one of a sub-bitstream regarding a mesh, a sub-bitstream regarding a difference vector transform coefficient, or a sub-bitstream regarding a texture map. A step of decoding a sub-bitstream for the above mesh to restore the base mesh of the current patch; A step of decoding a sub-bitstream for the above difference vector transform coefficients to restore the difference vector; and A mesh decoding method comprising a step of restoring a mesh of the current patch based on at least one of the base mesh and the difference vector.

2. In the first paragraph, the step of restoring the mesh of the current patch is: A step of performing subdivision on the restored base mesh based on a predetermined subdivision value, A mesh decoding method, wherein the sub-division value indicates the number of sub-divisions.

3. In paragraph 2, A mesh decoding method wherein the above sub-division value is signaled in a patch unit from the bitstream.

4. In paragraph 3, The above sub-division value is signaled from the bitstream based on a patch unit sub-division flag, A mesh decoding method in which the above patch unit sub-division flag indicates whether the sub-division value of the patch unit is used.

5. In paragraph 2, A mesh decoding method wherein the above sub-division value is derived based on the sub-division value of the previous patch and the delta sub-division value signaled for the current patch.

6. In paragraph 5, A mesh decryption method, wherein the above-mentioned previous patch means the first patch in the decryption order or a patch having a decryption order immediately preceding the current patch.

7. In paragraph 2, A mesh decoding method, wherein any one of a plurality of triangles belonging to the current patch has a sub-division value different from the other triangles.

8. In paragraph 7, A mesh decoding method, wherein the sub-division value of the current triangle among the plurality of triangles is derived based on the sub-division values ​​of the surrounding triangles adjacent to the current triangle.

9. In paragraph 8, The above peripheral triangle is specified based on information signaled from the bitstream, A mesh decoding method, wherein the above information represents any one of a plurality of surrounding triangles adjacent to the current triangle.

10. In paragraph 1, A mesh decryption method, wherein a probability table stored in any one of the restored patches is used as an initial probability table for the current patch.

11. In paragraph 10, A mesh decoding method, wherein any one of the above-described restored patches is selected based on reference patch index information signaled through the bitstream.

12. In paragraph 10, A mesh decryption method, wherein any one of the above-decrypted patches is a patch having a decryption order prior to the current patch.

13. In paragraph 10, The above-mentioned restored patch belongs to a different frame from the current frame to which the above-mentioned current patch belongs, A mesh decryption method, wherein any one of the above-described restored patches is a patch having the same patch index as the current patch.

14. A step of encoding the basic mesh of the current patch to generate a sub-bitstream for the mesh; A step of encoding a difference vector to generate a sub-bitstream for difference vector transform coefficients; and A mesh encoding method comprising the step of generating a bitstream including at least one of a sub-bitstream regarding the mesh, a sub-bitstream regarding the difference vector transform coefficients, or a sub-bitstream regarding the texture map.

Citation Information

Patent Citations

  • Method and apparatus for measuring image quality base on perceptual sensitivity

    KR1020220140669A

  • Electric vehicle transmission

    KR1020230061264A

  • Composition for inhibiting myeloid-derived suppressor cells comprising MITF inhibitor

    KR102352127B1

  • Pp bag feeding machine

    KR102474193B1

  • An apparatus, a method and a computer program for volumetric video

    WO2023041838A1

Cited By

  • Mesh data encoding device, mesh data encoding method, mesh data decoding device, and mesh data decoding method

    WO2026116883A1