Method and apparatus for UV attribute coding for symmetric meshes

Abstract

1. A method comprising: segmenting an input symmetric mesh into a plurality of unconnected UV segments; partitioning at least one UV segment via a plane that partitions the at least one UV segment into a first side and a second side; rearranging a plurality of vertices included in the first side of the at least one UV segment; performing 2D symmetry detection on the at least one UV segment to find a symmetry transformation; determining a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetry transformation of a corresponding vertex on the first side of the at least one UV segment to find a predicted vertex on the first side of the at least one UV segment; and encoding a bitstream including at least the vertices in the first side of the at least one UV segment.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority from U.S. Provisional Application No. 63 / 415,108, filed October 11, 2022, and U.S. Application No. 18 / 483,936, filed October 10, 2023, the entire disclosures of which are incorporated herein by reference.

[0002] This disclosure is directed to a set of advanced video coding techniques. More particularly, this disclosure is directed to UV attribute coding for symmetry meshes. [Background technology]

[0003]

[0003] VMesh is an evolving MPEG standard for compressing dynamic meshes. The current VMesh reference software compresses meshes based on a decimated base mesh, displacement vectors, and a motion field. Displacements are calculated by finding the closest point on the input mesh to each vertex of the subdivided base mesh. The displacement vectors are converted to wavelet coefficients by a linear lifting scheme, and the coefficients are then quantized and coded by a video or arithmetic codec.

[0004]

[0004] Reflection symmetry is a pervasive property of mesh coding, especially computer-generated meshes. Symmetry has been exploited to compress symmetric meshes. Vertices are divided into left and right parts of a symmetry plane. The left part is coded by mesh coding, and the right part is coded by symmetric prediction and displacement coding. UV attributes also have a certain level of symmetry, but UV attributes may have different symmetry properties, such as translation and rotation. However, UV attribute compression for symmetric meshes has not yet been exploited.

[0005]

[0005] Attributes attached to vertices also have a level of symmetry. However, the associated UVs of symmetric vertices do not always display mirror symmetry. Therefore, conventional mesh compression methods do not take advantage of these types of symmetry properties to compress mesh attributes. Summary of the Invention

[0006]

[0006] According to one or more embodiments, a method for encoding an input symmetric mesh includes: segmenting the input symmetric mesh into a plurality of unconnected UV segments in a UV attribute map; partitioning at least one UV segment via a plane that partitions the at least one UV segment into a first side and a second side opposite the first side, the first side including a plurality of vertices occupying a first area; rearranging a plurality of vertices included in the first side of the at least one UV segment, the plurality of vertices occupying a second area smaller than the first area after the rearranging; performing 2D symmetry detection on the at least one UV segment to find a symmetry transformation for the at least one UV segment; determining a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetry transformation of a corresponding vertex on the first side of the at least one UV segment to find a predicted vertex on the first side of the at least one UV segment; and encoding a bitstream including at least the vertices in the first side of the at least one UV segment.

[0007]

[0007] According to one or more embodiments, there is provided an encoder for encoding an input symmetric mesh, the encoder comprising at least one memory configured to store program code and at least one processor configured to read the program code and to operate as instructed by the program code, the program code comprising: segmentation code configured to cause the at least one processor to segment the input symmetric mesh into a plurality of unconnected UV segments in a UV attribute map; partitioning code configured to cause the at least one processor to partition via a plane that partitions at least one UV segment into a first side and a second side opposite the first side, the first side including a plurality of vertices occupying a first area; and partitioning code configured to cause the at least one processor to rearrange a plurality of vertices included in the first side of the at least one UV segment. the at least one UV segment to find a symmetry transformation for the at least one UV segment; determining code configured to cause the at least one processor to determine a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetry transformation and the vertices on the first side of the at least one UV segment to find a predicted vertex that corresponds to a corresponding vertex on the first side of the at least one UV segment; and encoding code configured to cause the at least one processor to encode a bitstream including at least vertices in the first side of the at least one UV segment.

[0008] According to one or more embodiments, a non-transitory computer-readable medium having stored thereon instructions that, when executed by a processor of an encoder for encoding an input symmetric mesh, cause the processor to: segment the input symmetric mesh into a plurality of unconnected UV segments in a UV attribute map; partition at least one UV segment via a plane that partitions the at least one UV segment into a first side and a second side opposite the first side, the first side including a plurality of vertices occupying a first area; and rearrange a plurality of vertices included in the first side of the at least one UV segment, the plurality of vertices being rearranged. after performing the above step, rearranging the plurality of vertices to occupy a second area smaller than the first area; performing 2D symmetry detection on the at least one UV segment to find a symmetry transformation for the at least one UV segment; determining a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetry transformation of a corresponding vertex on the first side of the at least one UV segment to find a predicted vertex on the first side of the at least one UV segment; and encoding a bitstream including at least the vertices in the first side of the at least one UV segment.

[0009]

[0009] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0010] [Figure 1]

[0010] FIG. 1 is a schematic diagram of a block diagram of a communication system according to an embodiment of the present disclosure. [Figure 2]

[0011] FIG. 1 is a schematic diagram of a block diagram of a streaming system, according to an embodiment of the present disclosure. [Figure 3]

[0012] FIG. 10 illustrates an example of UV parameterized mapping from a 3D mesh segment onto a 2D chart, according to an embodiment of the present disclosure.

[0013] [Figure 4]

[0014] FIG. 1 is a schematic diagram of a system for implementing symmetric predictive coding for UV attributes, according to an embodiment of the present disclosure. [Figure 5A]

[0015] FIG. 1 is a diagram of a mesh and a corresponding UV attribute map, according to an embodiment of the present disclosure. [Figure 5B]

[0016] FIG. 1 is a diagram of a half-symmetric mesh and corresponding UV attribute map, according to an embodiment of the present disclosure. [Figure 6]

[0017] FIG. 1 is an illustration of an exemplary algorithm for segmenting a UV mesh based on UV connectivity, according to an embodiment of the present disclosure. [Figure 7]

[0018] FIG. 10 is an illustration of adding one or more UV vertices to a mesh, according to an embodiment of the present disclosure. [Figure 8]

[0019] FIG. 1 is a diagram of an exemplary algorithm for partitioning a mesh, according to an embodiment of the present disclosure. [Figure 9]

[0020] FIG. 2 is a diagram of an exemplary algorithm for determining corresponding pairs of vertices in a mesh, according to an embodiment of the present disclosure. [Figure 10]

[0021] 10A-10C are diagrams of exemplary determinations of displacements between original and predicted vertices, according to embodiments of the present disclosure. [Figure 11]

[0022] FIG. 1 is a flowchart of an exemplary process for encoding a mesh, according to an embodiment of the present disclosure. [Figure 12]

[0023] FIG. 10 is a flowchart of an exemplary process for decoding a mesh, according to an embodiment of the present disclosure. [Figure 13]

[0024] FIG. 1 is a diagram of a computer system suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011]

[0025] The following detailed description of the exemplary embodiments refers to the accompanying drawings, in which the same reference numbers in different drawings may identify the same or similar elements.

[0012]

[0026] The above disclosure provides illustration and description, but is not intended to be exhaustive or to limit implementations to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations. Moreover, one or more features or components of an embodiment may be incorporated into or combined with another embodiment (or one or more features of another embodiment). Furthermore, in the flowcharts and descriptions of operations provided below, it should be understood that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed simultaneously (at least in part), and the order of one or more operations may be permuted.

[0013]

[0027] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods does not limit the implementation. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code, and it should be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0014]

[0028] Although particular combinations of features are recited in the claims and / or disclosed herein, these combinations do not limit the disclosure of possible implementations. Indeed, many of these features may be combined in ways not specifically recited in the claims and / or disclosed herein. Although each dependent claim set forth below may depend directly on only one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim set.

[0015]

[0029] No element, act, or instruction used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles "a" and "an" are intended to include one or more items and may be used interchangeably with "one or more." Where only one item is intended, the term "one" or similar phrases are used. Also, as used herein, terms such as "has," "have," "having," "include," and "including" are intended to be open-ended terms. Furthermore, the phrase "based on" is intended to mean "based at least in part on," unless otherwise specified. Furthermore, phrases such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood to include A only, B only, or both A and B.

[0016]

[0030] References throughout this specification to "one embodiment," "an embodiment," or similar phrases mean that a particular feature, structure, or characteristic described in connection with the illustrated embodiment is included in at least one embodiment of the solution. Thus, throughout this specification, the phrases "in one embodiment," "in an embodiment," and similar phrases may, but do not necessarily, all refer to the same embodiment.

[0017]

[0031] Furthermore, the described features, advantages, and characteristics of the present disclosure may be combined in any suitable manner in one or more embodiments. In light of the description herein, those skilled in the art will recognize that the present disclosure may be practiced without one or more of the specific features or advantages of a particular embodiment. In other cases, additional features and advantages may be recognized in some embodiments, which may not be present in all embodiments of the present disclosure.

[0018]

[0032] The proposed methods may be used separately or combined in any order and may be used for any polygonal mesh. Embodiments of the present disclosure are directed to encoding UV coordinates for symmetric-based mesh coding. In particular, embodiments are directed to reducing and reorganizing meshes. A mesh may be reduced by first cutting the mesh in half and further reducing the mesh into smaller segments. A mesh may be further reorganized and made more compact by utilizing a transformation that allows for a mapping between one side of the coordinates (e.g., the left side) and another side of the coordinates (e.g., the right side). The embodiments result in a more compact mesh being encoded. As a result, at least one side of the mesh is discarded, resulting in a more compact mesh being encoded. Thus, the embodiments result in improved mesh compression, which leads to faster encoding and a smaller bitstream for transmission.

[0019]

[0033] 1-2, one or more embodiments of the present disclosure for implementing the encoding and decoding structure of the present disclosure will be described.

[0020]

[0034] 1 shows a simplified block diagram of a communication system 100 according to one embodiment of the present disclosure. The system 100 may include at least two terminals 110, 120 interconnected via a network 150. For unidirectional transmission of data, a first terminal 110 may code video data, which may include mesh data, at a local location for transmission to the other terminal 120 via the network 150. The second terminal 120 may receive the other terminal's coded video data from the network 150, decode the coded data, and display the recovered video data. Unidirectional data transmission may be common in media serving applications, etc.

[0021]

[0035] 1 shows a second pair of terminals 130, 140 provided to support bidirectional transmission of coded video, such as may occur during a video conference. For the bidirectional transmission of data, each terminal 130, 140 may code video data captured at a local location for transmission to the other terminal over network 150. Each terminal 130, 140 may also receive coded video data transmitted by the other terminal, decode the coded data, and display the recovered video data on a local display device.

[0022]

[0036] In FIG. 1 , terminals 110-140 may be, for example, servers, personal computers, and smartphones, and / or any other type of terminal. For example, terminals 110-140 may be laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. Network 150 represents any number of networks that convey coded video data between terminals 110-140, including, for example, wireline and / or wireless communication networks. Communication network 150 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, the architecture and topology of network 150 may not be important to the operation of the present disclosure unless described herein below.

[0023]

[0037] 2 illustrates the arrangement of a video encoder and a video decoder in a streaming environment as an example of an application for the disclosed subject matter. The disclosed subject matter can be used with other video-enabled applications, including, for example, video conferencing, digital TV, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0024]

[0038] 2, a streaming system 200 may include a capture subsystem 213 that includes a video source 201 and an encoder 203. The streaming system 200 may further include at least one streaming server 205 and / or at least one streaming client 206.

[0025]

[0039] Video source 201 may, for example, create stream 202 including a 3D mesh and metadata associated with the 3D mesh. Video source 201 may, for example, include a 3D sensor (e.g., a depth sensor) or 3D imaging technology (e.g., digital camera(s)) and a computing device configured to generate a 3D mesh using data received from the 3D sensor or 3D imaging technology. Sample stream 202, which may have a high data volume compared to an encoded video bitstream, may be processed by encoder 203 coupled to video source 201. Encoder 203 may include hardware, software, or a combination thereof for enabling or implementing aspects of the disclosed subject matter, as described in more detail below. Encoder 203 may also generate encoded video bitstream 204. Encoded video bitstream 204, which may have a lower data volume compared to uncompressed stream 202, may be stored on streaming server 205 for future use. One or more streaming clients 206 may access the streaming server 205 to retrieve a video bitstream 209 , which may be a copy of the encoded video bitstream 204 .

[0026]

[0040] The streaming client 206 may include a video decoder 210 and a display 212. The video decoder 210 may, for example, decode a video bitstream 209 that is an incoming copy of the encoded video bitstream 204 and create an outgoing video sample stream 211 that can be rendered on the display 212 or another rendering device. In some streaming systems, the video bitstreams 204, 209 may be encoded according to some video coding / compression standard.

[0027]

[0041] According to one or more embodiments, a 3D mesh may be partitioned into several segments (or patches / charts). Each segment may consist of a set of connected vertices, which are associated with their geometry, attributes, and connectivity information. As shown in FIG. 3 , a UV parameterization process maps a mesh segment 300 onto 2D charts (302, 304) in a 2D UV atlas. Each vertex in a mesh segment may be assigned 2D UV coordinates in the 2D UV atlas. The vertices in the 2D charts may form connected components with their 3D counterparts. The geometry and connectivity information of each vertex may also be inherited from their 3D counterparts. Thus, UV coordinates map vertices in 2D space to 3D space based on the associated UV attributes, including the geometry and connectivity information.

[0028]

[0042] A mesh may be a collection of vertices, edges, and faces that define a shape or object. A 3D mesh may include multiple vertices, each associated with a 3D coordinate (e.g., x, y, z) in 3D space. A 2D mesh may include multiple vertices, each associated with a 2D coordinate (e.g., x and y) in 2D space. The vertices may be connected to form edges, where a collection of vertices may define a face.

[0029]

[0043] According to one or more embodiments, Figure 4 illustrates a framework for compressing UV maps of a symmetric mesh. A symmetric mesh M may have a mirror symmetry plane p, which maps vertex V to left vertex V L and the right vertex V R and a set of vertices V in the symmetry plane I The set is separated into face F and left face F. L Set of right face F R and a cross symmetry plane face F CIn one or more examples, a vertex may have multiple UV attributes, and thus the symmetry properties of the vertex do not apply directly to the UV attributes. Thus, according to one or more embodiments, a UV attribute U is separated into a set of disconnected parts U i 402 and apply 2D symmetric prediction. The symmetry property is determined by the symmetry transformation p i If detected, the segment is divided into two parts, and the first half of the UV attribute is

[0030]

number

[0031]

number

[0032]

number

[0033]

number

[0034]

[0044] According to one or more embodiments, segmentation is performed on an input symmetric mesh. The input symmetric mesh may be segmented into multiple disconnected portions in a UV attribute map. For example, FIG. 5(A) shows a UV map of a spoon mesh 500 segmented into three disconnected portions (502, 504, 506) based on the exemplary segmentation algorithm shown in FIG. 6. FIG. 5(B) shows a UV map of a half spoon mesh 508 segmented into three disconnected portions (510, 512, 514). The UV map of the half spoon mesh 508 may result in empty spaces between the disconnected portions. Therefore, the vertices in the UV map of the half spoon mesh may be reorganized to reduce the empty spaces between the disconnected portions, resulting in a more compact mesh. That is, the reorganized vertices occupy a second area that is smaller than the first area after reorganization.

[0035]

[0045] In Figure 6, FindSeg(seed,F) is a function to find all UVs that directly and indirectly connect to the seed UV.

[0036]

number

[0037]

number

[0046] In one or more examples, the segmentation may be based on texture. For example, the disconnected portions may be based on portions of the mesh that have unique textures. For example, with reference to FIG. 5(A), portions 502, 504, and 506 may each have a unique texture relative to one another. In one or more examples, the segmentation is obtained based on symmetry properties of the input mesh. For example, portions of the mesh that are symmetrical relative to one another may be segmented from other portions of the mesh.

[0038]

[0046] According to one or more embodiments, a 2D symmetry detection method is performed on each segmented UV U i is performed on the UV of this segment, i In one or more examples, the symmetry prediction is based only on UV coordinates. Prediction techniques for 2D images may be applied to UV segments. This symmetry prediction with respect to UVs may not depend on corresponding vertex symmetries. In one or more examples, the technique is applied only to segments with mirror symmetry.

[0039]

[0047] According to one or more embodiments, UV coordinates may not maintain mirror symmetry properties, so mirror symmetry may be generalized as a combination of mirror symmetry, rotation, and translation. In one or more examples, embodiments may rely on geometric symmetry to find corresponding UV pairs. Each UV coordinate u j , and its corresponding vertex v j is mapped to

[0040]

number

[0041]

number

[0042]

number

[0043]

[0048] In one or more examples, only the translation, rotation, and mirror are considered. Therefore, only four parameters are used:

[0044]

number

[0045]

[0049] According to one or more embodiments, to partition a symmetric mesh, one or more new vertices and edges may be added for cross symmetry plane connectivity. In one or more examples, associated UVs and UV connectivity may be added for the added vertices and edges. FIG. 7 shows an example segment where vertices u5 and u6 and associated UV connectivity are added. The sets of UV attributes for left, right, and cross symmetry are defined as G = {G R ,G L ,G C}.

[0046]

[0050] FIG. 8 illustrates an exemplary algorithm for adding a new vertex to a segment. In one or more examples, the operation PlaneLineCollision(v i ,v i+1 , p) is a plane p and two vertices v i , v i+1 Find the collision point between the operation num(u new ) but u new represents the number of elements in the newly added vertex v new To find the UVs associated with vertices u1 and u3, an interpolation operator, Interpolate(·), may be used, which may perform linear interpolation. Referring to Figure 7, plane p intersects with the edge between vertices u1 and u3, where vertex u5 is added. Furthermore, plane p intersects with the edge between vertices u2 and u3, where vertex u6 is added. An edge between new vertices u5 and u6 may be added based on the interpolation.

[0047]

[0051] The newly added UVs can be derived as follows: Equation (3): u new =ω*u1+(1-ω)*u2

[0048]

[0052] According to one or more embodiments, the corresponding vertex v i , v i+1 Assume that v new If x is the newly added vertex due to the collapse of the intersecting symmetry plane edge and the symmetry plane, the weights for the new UV prediction can be derived as follows:

[0049]

number

[0050]

[0053] According to one or more embodiments, to extract the half UVs to be coded, UVs associated with the left vertex and the in-plane vertex are kept, and UVs associated with the right vertex are discarded. In one or more examples, the UV connectivity is remapped to new UV indices. In one or more examples, displacement coding using predicted UVs is performed.

[0051]

[0054] According to one or more embodiments, the transformation parameters are estimated based on corresponding UV pairs associated with symmetric vertex pairs of corresponding UV segments. For example, for the i-th UV segment U i is the vertex set V i The i-th UV segment U i are the left and right parts of the vertex, respectively.

[0052]

number

[0053]

number

[0054]

number

[0055] 9, the operations FindVertex(.) and FindUVs(.) may be functions for finding the corresponding vertex of a given UV and vice versa. The operation SymmPred(.) may be a function for performing symmetric prediction in 3D, involving a given vertex and a symmetry plane Q.

[0056] According to one or more embodiments, a regression is performed to find two sets of 2D points.

[0057]

number

[0058]

number

[0059]

number

[0060] According to one or more embodiments, 2D displacement vector coding is performed on the predicted

[0061]

number

[0062]

number

[0063]

number

[0064] According to one or more embodiments, the first half UVs are encoded with the first half of the geometry to take advantage of their correlation. Full and near-full symmetric meshes can be split into left meshes with vertices, faces, and UV attributes, which can be compressed using other codecs. In one or more examples, Draco can be used, as will be appreciated by those skilled in the art.

[0065]

[0059] According to one or more embodiments, the transformation matrix is ​​coded and signaled in the coded bitstream. In one or more examples, the UV segments and the associated transformation matrix are sorted in descending order of the number of UVs in the remaining left mesh. In one or more examples, based on this order, each parameter in the transformation matrix is ​​coded consecutively row by row.

[0066] 11 shows a flowchart of an example process 1100 for encoding a mesh according to an embodiment of the present disclosure. The process begins at operation S1102, where a 3D mesh is segmented into a plurality of 2D segments. For example, the 3D mesh may be segmented according to the algorithm shown in FIG.

[0067] The process continues to operation S1104, where at least one 2D segment is partitioned. For example, the at least one 2D segment is partitioned according to the algorithm shown in Figure 8. The process continues to operation S1106, where a transformation between pairs of vertices in the at least one 2D segment is estimated. For example, the transformation may be estimated based on the algorithm shown in Figure 9.

[0068] The process proceeds to operation S1108, where prediction is performed based on the estimated transformation. For example, the prediction may be performed according to equation (5). The process proceeds to operation S1110 to determine a displacement between the predicted vertex and the original vertex. For example, the displacement may be determined according to FIG. 10. The process proceeds to operation S1112, where the bitstream is encoded. The encoded bitstream may include vertices on one side (e.g., the left side) of the segment, the predicted transformation, and the determined displacement.

[0069] 12 is a flowchart of an example process 1200 for decoding a mesh, according to an embodiment of the present disclosure. The process may begin at operation S1202, where an encoded bitstream is received. The left vertices, translation, and displacement may be decoded from the bitstream.

[0070] The process proceeds to operation S1204, where a right vertex is derived. For example, the right vertex may be derived based on using the predicted transformation and the left vertex to determine the predicted right vertex. The displacement may be added to the predicted right vertex to derive the original right vertex.

[0071]

[0065] The process continues to operation S1206, where a 2D segment is reconstructed based on the decoded left vertices and the derived right vertices. The process continues to operation S1208, where a 3D mesh is reconstructed based on the reconstructed 2D segment.

[0072]

[0066] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 13 illustrates a computer system 1300 suitable for implementing some embodiments of the present disclosure.

[0073]

[0067] Computer software may be coded using any suitable machine code or computer language that may be subjected to assembly, compilation, concatenation, or similar mechanisms to create code including instructions that may be executed by a computer central processing unit (CPU), graphics processing unit (GPU), etc., directly, or by interpretation, through microcode execution, etc.

[0074]

[0068] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0075] 13 for computer system 1300 are examples and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. The arrangement of components should not be interpreted as having a dependency or requirement relating to any one or combination of components illustrated in the non-limiting embodiment of computer system 1300.

[0076] The computer system 1300 may include several human interface input devices. Such human interface input devices may respond to input by one or more human users through, for example, tactile input (such as keystrokes, swipes, data glove movements), audio input (such as voice, clapping), visual input (such as gestures), or olfactory input (not shown). The human interface devices may also be used to capture certain media that do not necessarily involve direct conscious human input, such as audio (such as voice, music, ambient sounds), images (such as scanned images, photographic images obtained from a still image camera), or video (such as two-dimensional video, three-dimensional video including stereoscopic video).

[0077]

[0071] The input human interface devices may include one or more (only one of each shown) of a keyboard 1301, a mouse 1302, a trackpad 1303, a touchscreen 1310, a data glove, a joystick 1305, a microphone 1306, a scanner 1307, and a camera 1308.

[0078] The computer system 1300 may also include several human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen 1310, data gloves, or joystick 1305, although there may also be haptic feedback devices that do not serve as input devices). For example, such devices may be audio output devices (such as speakers 1309, headphones (not shown)), visual output devices (such as screens 1310, which may include CRT screens, LCD screens, plasma screens, OLED screens, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), some of which may be capable of outputting two-dimensional visual output or four or more dimensions of output through means such as stereoscopic output, each with or without touchscreen input capability, each with or without haptic feedback capability), and a printer (not shown).

[0079]

[0073] The computer system 1300 may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW 1320 along with CD / DVD or similar media 1321, thumb drives 1322, removable hard drives or solid state drives 1323, legacy magnetic media such as tape and floppy disks (not shown), and dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0080]

[0074] Those skilled in the art will also understand that the term "computer-readable medium" as used with respect to the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0081] The computer system 1300 may also include interfaces to one or more communication networks. The networks may be wireless, wireline, or optical. The networks may further be local, wide-area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, and the like. Examples of networks include local area networks such as Ethernet and wireless LAN; cellular networks, which may include GSM, 3G, 4G, 5G, LTE, and the like; TV wireline or wireless wide-area digital networks, which may include cable TV, satellite TV, and terrestrial broadcast TV; vehicular and industrial networks, which may include CANBus; and the like. Some networks generally require an external network interface adapter attached to some general-purpose data port (e.g., a USB port on the computer system 1300) or peripheral bus 1349, while other networks are generally integrated into the core of the computer system 1300 by attachment to a system bus, described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system 1300 may communicate with other entities. Such communications may be unidirectional, receive-only (e.g., broadcast TV), unidirectional send-only (e.g., CANbus to some CANbus devices), or bidirectional, for example, to other computer systems using local or wide-area digital networks. Such communications may include communications to cloud computing environment 1355. Several protocols and protocol stacks may be used on each of the networks and network interfaces described above.

[0082]

[0076] The above-mentioned human interface devices, human-accessible storage devices, and network interface 1354 may be attached to the core 1340 of the computer system 1300.

[0083] The core 1340 may include one or more central processing units (CPUs) 1341, graphics processing units (GPUs) 1342, specialized programmable processing units in the form of field programmable gate arrays (FPGAs) 1343, hardware accelerators 1344 for some tasks, etc. These devices may be connected through a system bus 1348, along with read-only memory (ROM) 1345, random access memory 1346, and internal mass storage 1347, such as an internal non-user-accessible hard drive, SSD, etc. In some computer systems, the system bus 1348 may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached either directly to the core's system bus 1348 or through a peripheral bus 1349. Architectures for peripheral buses include PCI, USB, etc. A graphics adapter 1350 may be included in the core 1340.

[0084] The CPU 1341, GPU 1342, FPGA 1343, and accelerator 1344 may execute several instructions, which in combination may make up the computer code described above. That computer code may be stored in ROM 1345 or RAM 1346. Persistent data may be stored, for example, in internal mass storage 1347, while transient data may also be stored in RAM 1346. Fast storage and retrieval to any of the memory devices may be enabled by the use of cache memory, which may be closely associated with one or more of the CPU 1341, GPU 1342, mass storage 1347, ROM 1345, RAM 1346, etc.

[0085]

[0079] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0086] By way of example and not limitation, a computer system having architecture 1300, and core 1340 in particular, may provide functionality as a result of processor(s) (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible, computer-readable media. Such computer-readable media may be media associated with the user-accessible mass storage introduced above, as well as any storage of core 1340 that is non-transitory in nature, such as core internal mass storage 1347 or ROM 1345. Software implementing various embodiments of the present disclosure may be stored on such devices and executed by core 1340. The computer-readable media may include one or more memory devices or chips, as appropriate. Software may cause the core 1340, and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.), to perform particular processes or particular portions of particular processes described herein, including defining data structures stored in RAM 1346 and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of hardwired or otherwise embodied logic in circuitry (e.g., accelerator 1344), which may operate in place of or in conjunction with software to perform particular processes or particular portions of particular processes described herein. Reference to software may encompass logic, and vice versa, where appropriate. Reference to computer-readable medium may encompass circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware and software.

[0087]

[0081] While this disclosure has described several non-limiting embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and therefore are within the spirit and scope of the present disclosure.

[0088]

[0082] The above disclosure also encompasses the embodiments described below.

[0089]

[0083] A segmentation method is used to segment the input symmetric mesh into multiple disconnected parts in the UV attribute map.

[0090]

[0084] (1) A method for encoding an input symmetric mesh, the method comprising: segmenting the input symmetric mesh into a plurality of unconnected UV segments in a UV attribute map; partitioning at least one UV segment via a plane that partitions the at least one UV segment into a first side and a second side opposite the first side, the first side including a plurality of vertices occupying a first area; and rearranging the plurality of vertices included in the first side of the at least one UV segment, the plurality of vertices occupying a second area smaller than the first area after the rearranging. performing 2D symmetry detection on the at least one UV segment to find a symmetry transformation for the at least one UV segment; determining a predicted vertex for each vertex on a second side of the at least one UV segment based on the symmetry transformation of a corresponding vertex on the first side of the at least one UV segment to find a predicted vertex on the first side of the at least one UV segment; and encoding a bitstream including at least the vertices in the first side of the at least one UV segment.

[0091]

[0085] (2) The method described in feature (1), wherein the input symmetric mesh is segmented based on a texture associated with the input symmetric mesh, and at least two non-connected UV segments have textures that are different from each other.

[0092] (3) The method of feature (1), wherein the input symmetric mesh is segmented based on symmetry properties of the input mesh.

[0093]

[0087] (4) A method described in any of features (1) to (3), wherein dividing at least one UV segment includes adding one or more new vertices based on corresponding collision points between the plane and the at least one UV segment.

[0094] (5) The method according to feature (4), wherein partitioning at least one UV segment further comprises adding a new edge between the two new vertices.

[0095] (6) The method according to any one of features (1) to (5), wherein the symmetric transformation is a six-parameter affine transformation.

[0096] (7) The method according to any one of features (1) to (5), wherein the symmetric transformation is a four-parameter affine transformation.

[0097]

[0091] (8) A method according to any one of features (1) to (7), wherein the symmetric transformation is determined based on a regression analysis performed between a vertex on a first side of at least one UV segment and a vertex on a second side of the UV segment.

[0098] (9) The method according to any one of features (1) to (8), wherein the predicted vertices are symmetrical with respect to the corresponding vertices on the first side of at least one UV segment.

[0099]

[0093] (10) The method described in feature (5), further comprising determining a displacement between each vertex on the second side of at least one UV segment and a corresponding predicted vertex, wherein the bitstream further includes each displacement and the estimated transformation.

[0100]

[0094] (11) An encoder for encoding an input symmetric mesh, the encoder comprising at least one memory configured to store program code and at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising: segmentation code configured to cause the at least one processor to segment the input symmetric mesh into a plurality of unconnected UV segments in a UV attribute map; partitioning code configured to cause the at least one processor to partition at least one UV segment via a plane that partitions the at least one UV segment into a first side and a second side opposite the first side, the first side including a plurality of vertices occupying a first area; and partitioning code configured to cause the at least one processor to rearrange a plurality of vertices included in the first side of the at least one UV segment. wherein a plurality of vertices occupy a second area smaller than the first area after reorganization; a symmetry transformation code configured to cause at least one processor to perform 2D symmetry detection on the at least one UV segment to find a symmetry transformation for the at least one UV segment; a determination code configured to cause the at least one processor to determine a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetry transformation and the vertices on the first side of the at least one UV segment to find a predicted vertex that corresponds to a corresponding vertex on the first side of the at least one UV segment; and an encoding code configured to cause the at least one processor to encode a bitstream including at least the vertices in the first side of the at least one UV segment.

[0101]

[0095] (12) An encoder as described in feature (11), wherein the input symmetric mesh is segmented based on a texture associated with the input symmetric mesh, and at least two unconnected UV segments have textures that are different from each other.

[0102] (13) The encoder of feature (11), wherein the input symmetric mesh is segmented based on symmetry properties of the input mesh.

[0103]

[0097] (14) An encoder described in any one of features (11) to (13), wherein dividing at least one UV segment includes adding one or more new vertices based on corresponding collision points between the plane and the at least one UV segment.

[0104] (15) The encoder of feature (14), wherein partitioning at least one UV segment further comprises adding a new edge between the two new vertices.

[0105] (16) The encoder according to any one of features (11) to (15), wherein the symmetric transformation is a six-parameter affine transformation.

[0106] (17) The encoder according to any one of features (11) to (15), wherein the symmetric transformation is a four-parameter affine transformation.

[0107]

[0101] (18) An encoder described in any one of features (11) to (17), wherein the symmetric transformation is determined based on a regression analysis performed between a vertex on a first side of at least one UV segment and a vertex on a second side of the UV segment.

[0108] (19) The encoder of any one of features (11) to (18), wherein the predicted vertices are symmetrical with respect to corresponding vertices on the first side of at least one UV segment.

[0109]

[0103] (20) A non-transitory computer-readable medium having stored thereon instructions, which when executed by a processor of an encoder for encoding an input symmetric mesh, cause the processor to: segment the input symmetric mesh into a plurality of unconnected UV segments in a UV attribute map; partition at least one UV segment via a plane that partitions the at least one UV segment into a first side and a second side opposite the first side, wherein the first side includes a plurality of vertices occupying a first area; and rearrange a plurality of vertices included in the first side of the at least one UV segment, wherein the plurality of vertices are rearranged. A non-transitory computer-readable medium that causes execution of a method including: subsequently rearranging a plurality of vertices, the vertices occupying a second area smaller than the first area; performing 2D symmetry detection on the at least one UV segment to find a symmetry transformation for the at least one UV segment; determining a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetry transformation of a corresponding vertex on the first side of the at least one UV segment to find a predicted vertex on the first side of the at least one UV segment; and encoding a bitstream including at least the vertices in the first side of the at least one UV segment.

Claims

1. 1. A method for encoding an input symmetric mesh, said method comprising the steps of: Segmenting the input symmetric mesh into a plurality of disconnected UV segments in a UV attribute map; Partitioning at least one UV segment via a plane that partitions the at least one UV segment into a first side and a second side opposite the first side, the first side including a plurality of vertices that occupy a first area; rearranging the vertices included in the first side of the at least one UV segment, the vertices occupying a second area smaller than the first area after the rearranging; performing a 2D symmetry detection on the at least one UV segment to find a symmetry transformation for the at least one UV segment; determining a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetric transformation of a corresponding vertex on the first side of the at least one UV segment to find a predicted vertex on the first side of the at least one UV segment; encoding a bitstream including at least the vertices within the first side of the at least one UV segment; A method comprising:

2. The method of claim 1 , wherein the input symmetric mesh is segmented based on a texture associated with the input symmetric mesh, and at least two disconnected UV segments have a texture that differs from each other.

3. The method of claim 1 , wherein the input symmetric mesh is segmented based on symmetry properties of the input mesh.

4. 2. The method of claim 1, wherein the partitioning the at least one UV segment comprises adding one or more new vertices based on corresponding collision points between the plane and the at least one UV segment.

5. The method of claim 4 , wherein the partitioning the at least one UV segment further comprises adding a new edge between two new vertices.

6. The method of claim 1 , wherein the symmetric transformation is a six-parameter affine transformation.

7. The method of claim 1 , wherein the symmetric transformation is a four-parameter affine transformation.

8. The method of claim 1 , wherein the symmetric transformation is determined based on a regression analysis performed between vertices on the first side of the at least one UV segment and vertices on the second side of the UV segment.

9. The method of claim 1 , wherein the predicted vertices are symmetrical with respect to the corresponding vertices on the first side of the at least one UV segment.

10. determining a displacement between each vertex on the second side of the at least one UV segment and a corresponding predicted vertex; further comprising The method of claim 5 , wherein the bitstream further comprises each displacement and an estimated transformation.

11. A program causing at least one processor to carry out the method according to any one of claims 1 to 10.

12. 1. An encoder for encoding an input symmetric mesh, said encoder comprising: at least one memory configured to store program code; at least one processor configured to read the program code and to operate as instructed by the program code; wherein the program code comprises: segmentation code configured to cause the at least one processor to segment the input symmetric mesh into a plurality of disjoint UV segments in a UV attribute map; partitioning code configured to cause the at least one processor to partition at least one UV segment through a plane that partitions at least one UV segment into a first side and a second side opposite the first side, the first side including a plurality of vertices occupying a first area; reorganization code configured to cause the at least one processor to reorganize the vertices included in the first side of the at least one UV segment, the vertices occupying a second area after the reorganization, the second area being smaller than the first area; symmetry transformation code configured to cause the at least one processor to perform a 2D symmetry detection on the at least one UV segment to find a symmetry transformation for the at least one UV segment; determination code configured to cause the at least one processor to determine a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetry transformation and the vertices on the first side of the at least one UV segment to find a predicted vertex that corresponds to a corresponding vertex on the first side of the at least one UV segment; encoding code configured to cause the at least one processor to encode a bitstream including at least the vertices in the first side of the at least one UV segment; Including, an encoder.

13. The encoder of claim 12 , wherein the input symmetric mesh is segmented based on a texture associated with the input symmetric mesh, and at least two disjoint UV segments have a different texture from each other.

14. The encoder of claim 13 , wherein the input symmetric mesh is segmented based on symmetry properties of the input mesh.

15. 13. The encoder of claim 12, wherein the partitioning the at least one UV segment comprises adding one or more new vertices based on corresponding collision points between the plane and the at least one UV segment.

16. The encoder of claim 15 , wherein the partitioning the at least one UV segment further comprises adding a new edge between two new vertices.

17. The encoder of claim 12 , wherein the symmetric transformation is a six-parameter affine transformation.

18. The encoder of claim 12 , wherein the symmetric transformation is a four-parameter affine transformation.

19. 13. The encoder of claim 12, wherein the symmetric transformation is determined based on a regression analysis performed between vertices on the first side of the at least one UV segment and vertices on the second side of the UV segment.

20. The encoder of claim 12 , wherein the predicted vertices are symmetrical with respect to the corresponding vertices on the first side of the at least one UV segment.

21. 1. A non-transitory computer-readable medium having stored thereon instructions that, when executed by a processor of an encoder for encoding an input symmetric mesh, cause the processor to: Segmenting the input symmetric mesh into a plurality of disconnected UV segments in a UV attribute map; Partitioning at least one UV segment via a plane that partitions the at least one UV segment into a first side and a second side opposite the first side, the first side including a plurality of vertices that occupy a first area; rearranging the vertices included in the first side of the at least one UV segment, the vertices occupying a second area smaller than the first area after the rearranging; performing a 2D symmetry detection on the at least one UV segment to find a symmetry transformation for the at least one UV segment; determining a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetric transformation of a corresponding vertex on the first side of the at least one UV segment to find a predicted vertex on the first side of the at least one UV segment; encoding a bitstream including at least the vertices within the first side of the at least one UV segment; A non-transitory computer-readable medium for performing a method comprising:

Images