Method and apparatus for UV attribute coding for symmetric meshes

Abstract

1. A method comprising: segmenting an input symmetric mesh into a plurality of unconnected UV segments; partitioning at least one UV segment via a plane that partitions the at least one UV segment into a first side and a second side; rearranging a plurality of vertices included in the first side of the at least one UV segment; performing 2D symmetry detection on the at least one UV segment to find a symmetry transformation; determining a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetry transformation of a corresponding vertex on the first side of the at least one UV segment to find a predicted vertex on the first side of the at least one UV segment; and encoding a bitstream including at least the vertices in the first side of the at least one UV segment.

Description

[Technical Field]

[0001] Cross-reference of related applications

[0001] This application claims priority from U.S. Provisional Application No. 63 / 415,108, filed October 11, 2022, and U.S. Application No. 18 / 483,936, filed October 10, 2023, the entire disclosure of which is incorporated herein by reference.

[0002]

[0002] This disclosure covers a set of advanced video coding techniques. More specifically, this disclosure covers UV attribute coding for symmetry meshes. [Background technology]

[0003]

[0003] VMesh is an ongoing MPEG standard for compressing dynamic meshes. Current VMesh reference software compresses meshes based on a decimated base mesh, displacement vectors, and motion fields. Displacement is calculated by searching for the nearest point on the input mesh to each vertex of the subdivided base mesh. Displacement vectors are converted to wavelet coefficients by a linear lifting scheme, and these coefficients are then quantized and coded by a video codec or arithmetic codec.

[0004]

[0004] Reflection symmetry is a prevalent characteristic of mesh coding, particularly computer-generated meshes. Symmetry has been used to compress symmetric meshes. Vertices are divided into left and right portions of the symmetry plane. The left portion is coded by mesh coding, and the right portion is coded by symmetry prediction and displacement coding. UV attributes also have a certain level of symmetry, although UV attributes can have different symmetry properties, such as transitions and rotations. However, UV attribute compression for symmetric meshes has not yet been utilized.

[0005]

[0005] Attributes connected to vertices also possess a certain degree of symmetry. However, the associated UVs of symmetrical vertices do not always exhibit mirror symmetry. Therefore, conventional mesh compression methods do not utilize these types of symmetry properties to compress the attributes of a mesh. [Overview of the project]

[0006]

[0006] According to one or more embodiments, a method for encoding an input symmetric mesh, the method comprising: segmenting the input symmetric mesh into a plurality of disconnected UV segments in a UV attribute map; dividing at least one UV segment through a plane dividing it into a first side and a second side opposite to the first side, wherein the first side comprises at least one UV segment containing a plurality of vertices occupying a first area; reorganizing a plurality of vertices contained within the first side of the at least one UV segment, wherein, after the reorganization, the plurality of vertices occupy a second area smaller than the first area; performing a 2D symmetry detection on the at least one UV segment to find a symmetry transformation for the at least one UV segment; determining a predicted vertex for each vertex on the second side of the at least one UV segment based on the symmetry transformation of the corresponding vertex on the first side of the at least one UV segment to find a predicted vertex on the first side of the at least one UV segment; and encoding a bitstream containing at least vertices within the first side of the at least one UV segment.

[0007]

[0007] According to one or more embodiments, an encoder for encoding an input symmetric mesh comprises at least one memory configured to store program code, and at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising: a segmentation code configured to cause at least one processor to segment an input symmetric mesh into a plurality of disconnected UV segments in a UV attribute map; a partitioning code configured to cause at least one processor to partition at least one UV segment through a plane dividing it into a first side and a second side opposite to the first side, wherein the first side includes a plurality of vertices occupying a first area; and a partitioning code configured to cause at least one processor to rearrange a plurality of vertices contained in the first side of at least one UV segment A reorganization code configured such that, after reorganization, a plurality of vertices occupy a second area smaller than a first area; a symmetry transformation code configured to cause at least one processor to perform 2D symmetry detection on at least one UV segment in order to find a symmetry transformation for at least one UV segment; a determination code configured to cause at least one processor to determine a predicted vertex for each vertex on the second side of at least one UV segment, based on the symmetry transformation and the vertex on the first side of at least one UV segment, in order to find a predicted vertex corresponding to the corresponding vertex on the first side of at least one UV segment; and an encoding code configured to cause at least one processor to encode a bitstream containing at least vertices within the first side of at least one UV segment.

[0008]

[0008] According to one or more embodiments, a non-temporary computer-readable medium storing instructions, the instructions, when executed by a processor of an encoder for encoding an input symmetric mesh, instruct the processor to segment the input symmetric mesh into a plurality of disconnected UV segments in a UV attribute map, to divide at least one UV segment through a plane dividing it into a first side and a second side opposite to the first side, wherein the first side comprises a plurality of vertices occupying a first area, and to reorganize the plurality of vertices contained in the first side of the at least one UV segment, wherein the plurality of vertices are reorganized After doing so, perform a method that includes rearranging multiple vertices that occupy a second area smaller than the first area; performing a 2D symmetry detection on at least one UV segment to find a symmetry transformation for at least one UV segment; determining the predicted vertices for each vertex on the second side of at least one UV segment based on the symmetry transformation of the corresponding vertices on the first side of at least one UV segment to find the predicted vertices on the first side of at least one UV segment; and encoding a bitstream containing at least vertices within the first side of at least one UV segment.

[0009]

[0009] Further features, properties, and various advantages of the disclosed subject matter will become clearer from the following detailed description and accompanying drawings. [Brief explanation of the drawing]

[0010] [Figure 1]

[0010] This is a schematic diagram of a block diagram of a communication system according to an embodiment of the present disclosure. [Figure 2]

[0011] This is a schematic diagram of a block diagram of a streaming system according to an embodiment of the present disclosure. [Figure 3]

[0012] A diagram showing an example of UV parameterization mapping from a 3D mesh segment onto a 2D chart according to an embodiment of the present disclosure.

[0013] [Figure 4]

[0014] A schematic diagram of a system for performing symmetric prediction coding for UV attributes according to an embodiment of the present disclosure. [Figure 5A]

[0015] A diagram of a mesh and a corresponding UV attribute map according to an embodiment of the present disclosure. [Figure 5B]

[0016] A diagram of a half-symmetric mesh and a corresponding UV attribute map according to an embodiment of the present disclosure. [Figure 6]

[0017] A diagram of an exemplary algorithm for segmenting a UV mesh based on UV connectivity according to an embodiment of the present disclosure. [Figure 7]

[0018] A diagram of adding one or more UV vertices to a mesh according to an embodiment of the present disclosure. [Figure 8]

[0019] A diagram of an exemplary algorithm for partitioning a mesh according to an embodiment of the present disclosure. [Figure 9]

[0020] A diagram of an exemplary algorithm for determining corresponding pairs of vertices in a mesh according to an embodiment of the present disclosure. [Figure 10]

[0021] A diagram of an exemplary determination of displacement between an original vertex and a predicted vertex according to an embodiment of the present disclosure. [Figure 11]

[0022] A diagram of a flowchart of an exemplary process for encoding a mesh according to an embodiment of the present disclosure. [Figure 12]

[0023] A diagram of a flowchart of an exemplary process for decoding a mesh according to an embodiment of the present disclosure. [Figure 13]

[0024] A diagram of a computer system suitable for implementing an embodiment of the present disclosure. [Modes for carrying out the invention]

[0011]

[0025] A detailed description of exemplary embodiments follows with reference to the accompanying drawings. The same reference numerals in different drawings may identify the same or similar elements.

[0012]

[0026] The above disclosures are illustrative and descriptive, but are not intended to be exhaustive or to limit implementations to the exact forms disclosed. Modifications and variations may be possible in light of the above disclosures or may be derived from the practice of implementations. Furthermore, one or more features or components of one embodiment may be incorporated into or combined with another embodiment (or one or more features of another embodiment). In addition, it should be understood that in the flowcharts and descriptions of operations provided below, one or more operations may be omitted, one or more operations may be added, one or more operations may be performed simultaneously (at least partially), and the order of one or more operations may be changed.

[0013]

[0027] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The specific control hardware or software code used to implement these systems and / or methods is not limiting to the implementation form. Therefore, it should be understood that the operation and behavior of the systems and / or methods are described herein independently of any specific software code, and that software and hardware may be designed to implement the systems and / or methods based on the descriptions herein.

[0014]

[0028] Certain combinations of features are described in the claims and / or disclosed herein, but these combinations do not limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not described in detail in the claims and / or disclosed herein. Each dependent claim described below may depend directly on only one claim, but the disclosure of possible implementations includes each dependent claim combined with any other claims in that claim set.

[0015]

[0029] No element, act, or command used herein should be construed as important or essential unless expressly described as such. Furthermore, the articles “a” and “an” used herein are intended to include one or more items and may be used interchangeably with “one or more.” When only one item is intended, the term “one” or similar phrasing should be used. Also, terms such as “has,” “have,” “having,” “include,” and “including” used herein are open-ended terms. Furthermore, the phrase “based on” means “at least partially based on” unless otherwise specified. Additionally, expressions such as “at least one of [A] and [B]” or “at least one of [A] or [B]” should be understood as including A only, B only, or both A and B.

[0016]

[0030] Throughout this specification, any reference to “one embodiment,” “an embodiment,” or similar phrases means that certain features, structures, or characteristics described in relation to the embodiments shown are included in at least one embodiment of the Solution. Therefore, throughout this specification, the phrases “in one embodiment,” “in an embodiment,” and similar phrases may all, though not necessarily, refer to the same embodiment.

[0017]

[0031] Furthermore, the features, advantages, and characteristics described herein may be combined in any preferred manner in one or more embodiments. Those skilled in the art will recognize, in light of the description herein, that the disclosure may be practiced without one or more of the specific features or advantages of a particular embodiment. In other cases, additional features and advantages that may not be present in all embodiments of the disclosure may be recognized in some embodiments.

[0018]

[0032] The proposed methods may be used separately or in any combination of any order and may be used for any polygonal mesh. Embodiments of this disclosure concern encoding UV coordinates for symmetric-based mesh coding. More specifically, embodiments concern reducing and reorganizing a mesh. A mesh may be reduced by first cutting the mesh in half and further reducing the mesh into smaller segments. A mesh may be further reorganized and made more compact by utilizing a transformation that allows mapping between one side of the coordinates (e.g., left side) and the other side of the coordinates (e.g., right side). Embodiments result in a more compact mesh being encoded. As a result, at least one side of the mesh is discarded and a more compact mesh is encoded. Thus, embodiments result in improved mesh compression, which leads to faster encoding and a smaller bitstream for transmission.

[0019]

[0033] One or more embodiments of the present disclosure for implementing the coding and decoding structures of the present disclosure will be described with reference to Figures 1 and 2.

[0020]

[0034] Figure 1 shows a simplified block diagram of a communication system 100 according to one embodiment of the present disclosure. The system 100 may include at least two terminals 110, 120 interconnected via a network 150. For unidirectional data transmission, the first terminal 110 may code video data, which may include mesh data, at a local location for transmission to the other terminal 120 via the network 150. The second terminal 120 may receive the coded video data from the other terminal via the network 150, decode the coded data, and display the restored video data. Unidirectional data transmission may be common in media serving applications, for example.

[0021]

[0035] Figure 1 shows a second pair of terminals 130, 140 provided to support the bidirectional transmission of coded video, which may occur, for example, during a video conference. For bidirectional data transmission, each terminal 130, 140 may code video data captured at its local location for transmission to the other terminal over the network 150. Each terminal 130, 140 may also receive coded video data transmitted by the other terminal, decode the coded data, and display the restored video data on a local display device.

[0022]

[0036] In Figure 1, terminals 110-140 could be, for example, servers, personal computers, and smartphones, and / or any other type of terminal. For example, terminals (110-140) could be laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 150 represents any number of networks that transmit coded video data between terminals 110-140, including, for example, wireline and / or wireless communication networks. Communication network 150 may exchange data over circuit-switched and / or packet-switched channels. Typical networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. In this discussion, the architecture and topology of network 150 may not be important to the operation of this disclosure unless described herein below.

[0023]

[0037] Figure 2 shows an example of an application of the disclosed subject matter, illustrating the arrangement of a video encoder and video decoder in a streaming environment. The disclosed subject matter can be used in conjunction with other video-enabled applications, such as video conferencing, digital TV, and the storage of compressed video on digital media including CDs, DVDs, and memory sticks.

[0024]

[0038] As shown in Figure 2, the streaming system 200 may include a capture subsystem 213 which includes a video source 201 and an encoder 203. The streaming system 200 may further include at least one streaming server 205 and / or at least one streaming client 206.

[0025]

[0039] The video source 201 may, for example, create a stream 202 that includes a 3D mesh and metadata associated with the 3D mesh. The video source 201 may include, for example, a 3D sensor (e.g., a depth sensor) or 3D imaging technology (e.g., one or more digital cameras) and a computing device configured to generate a 3D mesh using data received from the 3D sensor or 3D imaging technology. A sample stream 202, which may have a higher data volume compared to the encoded video bitstream, may be processed by an encoder 203 coupled to the video source 201. The encoder 203 may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject, as will be described in more detail below. The encoder 203 may also generate an encoded video bitstream 204. The encoded video bitstream 204, which may have a lower data volume compared to the uncompressed stream 202, may be stored in a streaming server 205 for future use. One or more streaming clients 206 may access the streaming server 205 to retrieve a video bitstream 209, which may be a copy of the encoded video bitstream 204.

[0026]

[0040] The streaming client 206 may include a video decoder 210 and a display 212. The video decoder 210 may, for example, decode a video bitstream 209, which is an incoming copy of an encoded video bitstream 204, and create an outgoing video sample stream 211 that can be rendered on the display 212 or another rendering device. In some streaming systems, the video bitstreams 204, 209 may be encoded according to some video coding / compression standard.

[0027]

[0041] According to one or more embodiments, a 3D mesh may be divided into several segments (or patches / charts). Each segment may consist of a set of connected vertices, and these sets of vertices are associated with their geometry, attributes, and connectivity information. As shown in Figure 3, the UV parameterization process maps mesh segment 300 onto 2D charts (302, 304) in a 2D UV atlas. Each vertex in the mesh segment may be assigned 2D UV coordinates in the 2D UV atlas. The vertices in the 2D chart may form connected components as their 3D counterparts. The geometry and connectivity information of each vertex may also be inherited from their 3D counterparts. Thus, the UV coordinates map vertices in 2D space to 3D space based on associated UV attributes, including geometry and connectivity information.

[0028]

[0042] A mesh can be a set of vertices, edges, and faces that define a shape or object. A 3D mesh can contain multiple vertices, each associated with a 3D coordinate in 3D space (e.g., x, y, z). A 2D mesh can contain multiple vertices, each associated with a 2D coordinate in 2D space (e.g., x and y). These vertices can be connected to form edges, where the set of vertices can define a face.

[0029]

[0043] According to one or more embodiments, Figure 4 shows a framework for compressing the UV map of a symmetric mesh. The symmetric mesh M may have a mirror symmetry plane p, which is such that vertex V is mirrored to the left vertex V L The set and the right vertex V R A set of vertices V in the symmetric plane. I The set is separated into two parts, with Face F and left Face F. L The set includes the right face F R The set and the cross symmetry plane face F CIt is separated into a set and a part. In one or more examples, a single vertex may have multiple UV attributes, and therefore the symmetry property of the vertex is not directly applied to the UV attribute. Thus, according to one or more embodiments, the UV attribute U is separated into an unconnected part U i The segment is divided into 402 segments, and a 2D symmetry prediction is applied. The symmetry property is a symmetry transformation p for the i-th segment. i If a 404 is detected, the segment is divided into 406, the first half UV attribute

[0030]

number

[0031]

number

[0032]

number

[0033]

number

[0034]

[0044] According to one or more embodiments, segmentation is performed on an input symmetric mesh. The input symmetric mesh can be segmented into multiple disconnected parts in the UV attribute map. For example, Figure 5(A) shows the UV map of a spoon mesh 500 segmented into three disconnected parts (502, 504, 506) based on the exemplary segmentation algorithm shown in Figure 6. Figure 5(B) shows the UV map of a half-spoon mesh 508 segmented into three disconnected parts (510, 512, 514). The UV map of the half-spoon mesh 508 may result in empty spaces between the disconnected parts. Therefore, the vertices in the UV map of the half-spoon mesh may be reorganized to reduce the empty spaces between the disconnected parts, resulting in a more compact mesh. That is, the reorganized vertices occupy a second area that is smaller than the first area after the reorganization.

[0035]

[0045] In Figure 6, FindSeg(seed,F) is a function for finding all UVs that are directly and indirectly connected to the seed UV. The function FacesContain(

[0036]

number

[0037]

number

[0038]

[0046] According to one or more embodiments, a 2D symmetry detection method is performed on each segmented UV U i to find a symmetry transformation p i for this segment UV. In one or more examples, the symmetry prediction is based on only the UV coordinates. Prediction techniques for 2D images can be applied to the UV segments. This symmetry prediction regarding UV may not depend on the corresponding vertex symmetry. In one or more examples, this approach is applied only to segments with mirror symmetry.

[0039]

[0047] According to one or more embodiments, since the UV coordinates may not maintain the mirror symmetry property, mirror symmetry can be generalized as a combination of mirror symmetry, rotation, and translation. In one or more examples, an embodiment can be based on geometric symmetry to find the corresponding UV pairs. Each UV coordinate u j is mapped to its corresponding vertex v j and

[0040]

Number

[0041]

Number

[0042]

Number

[0043]

[0048] In one or more examples, only transitions, rotations, and mirrors are considered. Therefore, only four parameters are used, as follows:

[0044]

number

[0045]

[0049] According to one or more embodiments, one or more new vertices and edges may be added for cross-symmetric plane connectivity to partition a symmetric mesh. In one or more examples, associated UVs and UV connectivity may be added for the added vertices and edges. Figure 7 shows an exemplary segment to which vertices u5 and u6 and associated UV connectivity are added. The sets of UV attributes in left, right, and cross-symmetry are given by G={G R ,G L ,G C It could be}.

[0046]

[0050] Figure 8 shows an exemplary algorithm for adding a new vertex to a segment. In one or more examples, the operation PlaneLineCollision(v i ,v i+1 ,p) is a plane p and two vertices v i , v i+1 Find the collision point between them. Calculate num(u new ) but, u new Represents the number of elements. Newly added vertex v new To find the associated UVs, the interpolation operator Interpolate(·) can be used, which can perform linear interpolation. Referring to Figure 7, plane p intersects the edge between vertices u1 and u3, where vertex u5 is added. Furthermore, plane p intersects the edge between vertices u2 and u3, where vertex u6 is added. The edge between the new vertex u5 and the new vertex u6 may be added based on the interpolation.

[0047]

[0051] The newly added UV can be derived as follows: Equation (3): u new =ω*u1+(1-ω)*u2

[0048]

[0052] According to one or more embodiments, the corresponding vertex v i , v i+1 Assuming that v new However, if we assume that these are newly added vertices due to the collapse between the intersecting symmetry plane edge and the symmetry plane, then the weights for the new UV prediction can be derived as follows.

[0049]

number

[0050]

[0053] According to one or more embodiments, in order to extract the half-UVs to be coded, the UVs associated with the left vertex and the in-plane vertex are retained, and the UVs associated with the right vertex are discarded. In one or more examples, UV connectivity is remapped to a new UV index. In one or more examples, displacement coding is performed using predicted UVs.

[0051]

[0054] According to one or more embodiments, the transformation parameters are estimated based on the corresponding UV pairs associated with the symmetric vertex pairs of the corresponding UV segments. For example, the i-th UV segment U i is vertex set V i It can be associated with the i-th UV segment U i These are the left and right parts of the vertex, respectively.

[0052]

number

[0053]

number

[0054]

number

[0055]

[0055] In Figure 9, the operations FindVertex(.) and FindUVs(.) may be functions for finding the corresponding vertices of a given UV, and vice versa. The operation SymmPred(.) may be a function for performing symmetry prediction in 3D, given a vertex and a symmetry plane Q.

[0056]

[0056] According to one or more embodiments, regression is performed on two sets of 2D points

[0057]

number

[0058]

number

[0059]

number

[0060]

[0057] According to one or more embodiments, 2D displacement vector coding is predicted

[0061]

number

[0062]

number

[0063]

number

[0064]

[0058] According to one or more embodiments, the first half UV is encoded together with the first half of the geometry in order to take advantage of their correlation. A fully and nearly fully symmetric mesh may be split into a left mesh having vertices, faces and UV attributes, which can be compressed using other codecs. In one or more examples, Draco may be used, as will be understood by those skilled in the art.

[0065]

[0059] According to one or more embodiments, the transformation matrix is ​​encoded and signaled in the encoded bitstream. In one or more examples, the UV segments and their associated transformation matrices are sorted in descending order of the number of UVs in the remaining left mesh. In one or more examples, each parameter in the transformation matrix is ​​encoded row by row based on this order.

[0066]

[0060] Figure 11 shows a flowchart of an exemplary process 1100 for encoding a mesh according to an embodiment of the present disclosure. The process starts in operation S1102, in which a 3D mesh is segmented into a plurality of 2D segments. For example, a 3D mesh may be segmented according to the algorithm shown in Figure 6.

[0067]

[0061] The process proceeds to operation S1104, in which at least one 2D segment is partitioned. For example, at least one 2D segment is partitioned according to the algorithm shown in Figure 8. The process proceeds to operation S1106, in which a transformation between pairs of vertices in at least one 2D segment is estimated. For example, the transformation may be estimated based on the algorithm shown in Figure 9.

[0068]

[0062] The process proceeds to operation S1108, in which a prediction is made based on the estimated transformation. For example, the prediction may be made according to equation (5). The process proceeds to operation S1110 to determine the displacement between the predicted vertex and the original vertex. For example, the displacement may be determined according to Figure 10. The process proceeds to operation S1112, in which the bitstream is encoded. The encoded bitstream may include the vertices on one side of the segment (e.g., the left side), the predicted transformation, and the determined displacement.

[0069]

[0063] Figure 12 is a flowchart of an exemplary process 1200 for decoding a mesh according to an embodiment of the present disclosure. The process may be initiated in operation S1202, in which an encoded bitstream is received. Left vertices, transformations, and displacements can be decoded from that bitstream.

[0070]

[0064] The process proceeds to operation S1204, in which the right vertex is derived. For example, the right vertex may be derived based on the use of the predicted transformation and the left vertex to determine the predicted right vertex. Displacements may be added to the predicted right vertex to derive the original right vertex.

[0071]

[0065] The process proceeds to operation S1206, in which the 2D segment is reconstructed based on the decoded left vertex and the derived right vertex. The process proceeds to operation S1208, in which the 3D mesh is reconstructed based on the reconstructed 2D segment.

[0072]

[0066] The techniques described above may be implemented as computer software that uses computer-readable instructions and is physically stored on one or more computer-readable media. For example, Figure 13 shows a computer system 1300 suitable for implementing some embodiments of the present disclosure.

[0073]

[0067] Computer software can be coded using any suitable machine code or computer language that may be subject to assembly, compilation, concatenation, or similar mechanisms to create code that includes instructions, which can be executed directly or by interpretation by a computer central processing unit (CPU), graphics processing unit (GPU), etc., through microcode execution or the like.

[0074]

[0068] Instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, and Internet of Things devices.

[0075]

[0069] The components shown in Figure 13 for the computer system 1300 are examples and do not imply any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. The configuration of the components should not be construed as having any dependency or requirement relating to any one or combination of components shown in non-limiting embodiments of the computer system 1300.

[0076]

[0070] The computer system 1300 may include several human interface input devices. Such human interface input devices may respond to input from one or more human users through, for example, tactile input (such as keystrokes, swipes, and data glove movements), audio input (such as voice and clapping), visual input (such as gestures), and olfactory input (not shown). Human interface devices may also be used to capture some media that are not necessarily directly related to conscious human input, such as audio (such as voices, music, and ambient sounds), images (such as scanned images and photographic images taken from still image cameras), and video (such as two-dimensional video and three-dimensional video including stereoscopic video).

[0077]

[0071] The input human interface device may include one or more of the following (only one of each as shown): keyboard 1301, mouse 1302, trackpad 1303, touchscreen 1310, data glove, joystick 1305, microphone 1306, scanner 1307, and camera 1308.

[0078]

[0072] The computer system 1300 may also include several human interface output devices. Such human interface output devices may stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include tactile output devices (e.g., touchscreen 1310, data glove, or tactile feedback via joystick 1305, although there may also be tactile feedback devices that do not function as input devices). For example, such devices may include audio output devices (such as speaker 1309, headphones (not shown)), visual output devices (such as screen 1310, CRT screen, LCD screen, plasma screen, OLED screen, virtual reality glasses (not shown), holographic display, and smoke tank (not shown)), and printers (not shown), some of which may be capable of outputting four or more dimensions through means such as two-dimensional visual output or stereoscopic output, each with or without touchscreen input capability, each with or without tactile feedback capability).

[0079]

[0073] The computer system 1300 may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW 1320 along with CD / DVD or similar media 1321, thumb drives 1322, removable hard drives or solid-state drives 1323, legacy magnetic media such as tapes and floppy disks (not shown), and dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0080]

[0074] Those skilled in the art will also understand that the term “computer-readable medium” as used in relation to the subject matter of this disclosure does not include a transmission medium, carrier wave, or other transient signal.

[0081]

[0075] The computer system 1300 may also include interfaces to one or more communication networks. The networks may be wireless, wireline, or optical. The networks may further include local, wide-area, metropolitan, vehicle and industrial, real-time, latency-tolerant, etc. Examples of networks include local area networks such as Ethernet and wireless LANs, cellular networks which should include GSM, 3G, 4G, 5G, LTE, etc., TV wireline or wireless wide-area digital networks which should include cable TV, satellite TV, and terrestrial broadcast TV, and vehicle and industrial networks which should include CANBus, etc. Some networks generally require some general-purpose data ports (such as USB ports on the computer system 1300) or external network interface adapters attached to peripheral bus 1349, while other networks are generally integrated into the core of the computer system 1300 by attachment to system buses as described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system 1300 may communicate with other entities. Such communications may be unidirectional, receive-only (e.g., broadcast TV), unidirectional transmit-only (e.g., CANbus to some CANbus devices), or bidirectional to other computer systems using local or wide-area digital networks. Such communications may include communications to cloud computing environments 1355. Several protocols and protocol stacks may be used on each of the networks and network interfaces described above.

[0082]

[0076] The human interface device, the human-accessible storage device, and the network interface 1354 described above may be mounted on the core 1340 of the computer system 1300.

[0083]

[0077] The core 1340 may include one or more central processing units (CPUs) 1341, graphics processing units (GPUs) 1342, special programmable processing units in the form of field-programmable gate arrays (FPGAs) 1343, hardware accelerators 1344 for several tasks, etc. These devices may be connected via a system bus 1348, along with read-only memory (ROM) 1345, random access memory 1346, internal non-user-accessible hard drive, SSD or other internal mass storage 1347. In some computer systems, the system bus 1348 may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached either directly to the core's system bus 1348 or through a peripheral bus 1349. Architectures for the peripheral bus include PCI, USB, etc. A graphics adapter 1350 may be included in the core 1340.

[0084]

[0078] The CPU 1341, GPU 1342, FPGA 1343, and accelerator 1344 can execute several instructions that, in combination, can create the computer code described above. This computer code can be stored in ROM 1345 or RAM 1346. Transition data can also be stored in RAM 1346, but persistent data can be stored, for example, in internal mass storage 1347. The use of cache memory, which may be closely associated with one or more CPUs 1341, GPUs 1342, mass storage 1347, ROM 1345, RAM 1346, etc., can enable fast storage and retrieval to any of the memory devices.

[0085]

[0079] A computer-readable medium may have computer code thereon for performing various computer implementation operations. The medium and computer code may be specifically designed and constructed for this disclosure, or they may be of a type that is well known to and available to those skilled in the art in computer software technology.

[0086]

[0080] Not as an limitation, but as an example, a computer system having architecture 1300, and more particularly core 1340, may provide functionality as a result of (one or more) processors (including CPUs, GPUs, FPGAs, accelerators, etc.) running software embodied in one or more tangible, computer-readable media. Such computer-readable media may be user-accessible mass storage as described above, as well as media associated with any storage of core 1340 that is non-transient in nature, such as core internal mass storage 1347 or ROM 1345. Software implementing various embodiments of the present disclosure may be stored in such devices and run by core 1340. The computer-readable media may include one or more memory devices or chips, depending on the specific requirements. The software may cause the core 1340, and more specifically the processor within it (including a CPU, GPU, FPGA, etc.), to execute a particular process or a particular part of a particular process as described herein, which includes defining data structures stored in RAM 1346 and modifying such data structures according to a process defined by the software. Furthermore or alternatively, a computer system may provide functionality as a result of logic hardwired or otherwise embodied in a circuit (e.g., accelerator 1344), which may operate in place of or with the software to execute a particular process or a particular part of a particular process as described herein. References to software may encompass logic, and vice versa, where appropriate. References to computer-readable media may encompass circuits (such as integrated circuits (ICs)) that store software for execution, circuits that embody logic for execution, or both, where appropriate. This disclosure encompasses any preferred combination of hardware and software.

[0087]

[0081] While this disclosure has described some non-limiting embodiments, there are many modifications, substitutions, and alternative equivalents that fall within the scope of this disclosure. It will be understood that, as such, a number of systems and methods not expressly shown or described herein but embodying the principles of this disclosure and thus falling within the spirit and scope of this disclosure can be devised by those skilled in the art.

[0088]

[0082] The above disclosure also includes embodiments described below.

[0089]

[0083] A segmentation method is used to segment the input symmetric mesh into multiple disconnected portions in the UV attribute map.

[0090]

[0084] (1) A method for encoding an input symmetric mesh, the method comprising: segmenting the input symmetric mesh into a plurality of disconnected UV segments in a UV attribute map; dividing at least one UV segment through a plane that divides the at least one UV segment into a first side and a second side opposite to the first side, wherein the first side comprises at least one UV segment containing a plurality of vertices occupying a first area; and rearranging the plurality of vertices contained in the first side of the at least one UV segment, wherein after the rearrangement the plurality of vertices occupy a second area smaller than the first area. A method comprising: rearranging multiple vertices occupying an area; performing 2D symmetry detection on at least one UV segment to find a symmetry transformation for at least one UV segment; determining a predicted vertex for each vertex on the second side of at least one UV segment based on the symmetry transformation of the corresponding vertex on the first side of at least one UV segment to find a predicted vertex on the first side of at least one UV segment; and encoding a bitstream containing at least vertices within the first side of at least one UV segment.

[0091]

[0085] (2) The method according to feature (1), wherein the input symmetric mesh is segmented based on the texture associated with the input symmetric mesh, and at least two disconnected UV segments have different textures from each other.

[0092]

[0086] (3) The method according to feature (1), wherein the input symmetric mesh is segmented based on the symmetry properties of the input mesh.

[0093]

[0087] (4) The method according to any one of features (1) to (3), wherein dividing at least one UV segment includes adding one or more new vertices based on corresponding collision points between a plane and at least one UV segment.

[0094]

[0088] (5) The method according to feature (4), further comprising dividing at least one UV segment to add a new edge between two new vertices.

[0095]

[0089] (6) The method according to any one of features (1) to (5), wherein the symmetric transformation is a 6-parameter affine transformation.

[0096]

[0090] (7) The method according to any one of features (1) to (5), wherein the symmetric transformation is a four-parameter affine transformation.

[0097]

[0091] (8) The method according to any one of features (1) to (7), wherein the symmetric transformation is determined based on a regression analysis performed between a vertex on the first side of at least one UV segment and a vertex on the second side of the UV segment.

[0098]

[0092] (9) The method according to any one of features (1) to (8), wherein the predicted vertices are symmetric with respect to the corresponding vertices on the first side of at least one UV segment.

[0099]

[0093] (10) The method according to feature (5), further comprising determining the displacement between each vertex on the second side of at least one UV segment and the corresponding predicted vertex, wherein the bitstream further comprises each displacement and the estimated transformation.

[0100]

[0094] (11) An encoder for encoding an input symmetric mesh, the encoder comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising: a segmentation code configured to cause at least one processor to segment an input symmetric mesh into a plurality of disconnected UV segments in a UV attribute map; a partitioning code configured to cause at least one processor to partition at least one UV segment through a plane that divides at least one UV segment into a first side and a second side opposite to the first side, the first side comprising a plurality of vertices occupying a first area; and a partitioning code configured to cause at least one processor to rearrange a plurality of vertices contained in the first side of at least one UV segment An encoder comprising: a reorganization code configured such that, after reorganization, a plurality of vertices occupy a second area smaller than a first area; a symmetry transformation code configured to cause at least one processor to perform 2D symmetry detection on at least one UV segment in order to find a symmetry transformation for at least one UV segment; a determination code configured to cause at least one processor to determine a predicted vertex for each vertex on the second side of at least one UV segment, based on the symmetry transformation and the vertex on the first side of at least one UV segment, in order to find a predicted vertex corresponding to the corresponding vertex on the first side of at least one UV segment; and an encoding code configured to cause at least one processor to encode a bitstream containing at least vertices within the first side of at least one UV segment.

[0101]

[0095] (12) The encoder according to feature (11), wherein the input symmetric mesh is segmented based on the texture associated with the input symmetric mesh, and at least two disconnected UV segments have different textures from each other.

[0102]

[0096] (13) The encoder according to feature (11), wherein the input symmetric mesh is segmented based on the symmetry properties of the input mesh.

[0103]

[0097] (14) An encoder according to any one of features (11) to (13), wherein dividing at least one UV segment includes adding one or more new vertices based on corresponding collision points between a plane and at least one UV segment.

[0104]

[0098] (15) The encoder according to feature (14), further comprising dividing at least one UV segment to add a new edge between two new vertices.

[0105]

[0099] (16) The encoder according to any one of features (11) to (15), wherein the symmetric transformation is a 6-parameter affine transformation.

[0106]

[0100] (17) An encoder according to any one of features (11) to (15), wherein the symmetric transformation is a four-parameter affine transformation.

[0107]

[0101] (18) An encoder according to any one of features (11) to (17), wherein the symmetric transformation is determined based on a regression analysis performed between a vertex on the first side of at least one UV segment and a vertex on the second side of the UV segment.

[0108]

[0102] (19) The encoder according to any one of features (11) to (18), wherein the predicted vertices are symmetric with respect to the corresponding vertices on the first side of at least one UV segment.

[0109]

[0103] (20) A non-temporary computer-readable medium storing instructions, wherein, when executed by a processor of an encoder for encoding an input symmetric mesh, the processor instructs the processor to segment the input symmetric mesh into a plurality of disconnected UV segments in a UV attribute map, to divide at least one UV segment through a plane that divides at least one UV segment into a first side and a second side opposite to the first side, wherein the first side comprises a plurality of vertices occupying a first area, and to reorganize the plurality of vertices contained in the first side of the at least one UV segment, wherein the plurality of vertices are reorganized A non-temporary computer-readable medium that subsequently performs a method comprising: rearranging multiple vertices that occupy a second area smaller than a first area; performing 2D symmetry detection on at least one UV segment to find a symmetry transformation for at least one UV segment; determining a predicted vertex for each vertex on the second side of at least one UV segment based on the symmetry transformation of the corresponding vertex on the first side of at least one UV segment to find a predicted vertex on the first side of at least one UV segment; and encoding a bitstream containing at least vertices within the first side of at least one UV segment.

Claims

1. A method for encoding an input symmetric mesh, wherein the method is The input symmetric mesh is segmented into multiple disconnected UV segments in the UV attribute map, Dividing at least one UV segment through a plane that divides it into a first side and a second side opposite to the first side, wherein the first side divides at least one UV segment including a plurality of vertices occupying a first area, Reorganizing the plurality of vertices contained within the first side of the at least one UV segment, wherein the plurality of vertices, after the reorganization, occupy a second area smaller than the first area. In order to find the symmetry transformation for the at least one UV segment, 2D symmetry detection is performed on the at least one UV segment, The predicted vertices for each vertex on the second side of the at least one UV segment are determined based on the symmetric transformation of the corresponding vertices on the first side of the at least one UV segment in order to find the predicted vertices on the first side of the at least one UV segment. Encoding a bitstream that includes at least the vertices within the first side of the at least one UV segment. Methods that include...

2. The method according to claim 1, wherein the input symmetric mesh is segmented based on a texture associated with the input symmetric mesh, and at least two disconnected UV segments have different textures from each other.

3. The method according to claim 1, wherein the input symmetric mesh is segmented based on the symmetry properties of the input symmetric mesh.

4. The method according to claim 1, wherein the division of the at least one UV segment includes adding one or more new vertices based on corresponding collision points between the plane and the at least one UV segment.

5. The method according to claim 4, wherein dividing the at least one UV segment further comprises adding a new edge between two new vertices.

6. The method according to claim 1, wherein the symmetric transformation is a six-parameter affine transformation.

7. The method according to claim 1, wherein the symmetric transformation is a four-parameter affine transformation.

8. The method according to claim 1, wherein the symmetric transformation is determined based on a regression analysis performed between the vertices on the first side of the at least one UV segment and the vertices on the second side of the UV segment.

9. The method according to claim 1, wherein the predicted vertices are symmetrical with respect to the corresponding vertices on the first side of the at least one UV segment.

10. Determining the displacement between each vertex on the second side of the at least one UV segment and the corresponding predicted vertex. It further includes, Claim 5 further includes each displacement and the estimated transformation. The method.

11. A program that causes at least one processor to perform the method according to any one of claims 1 to 10.

12. An encoder for encoding an input symmetric mesh, wherein the encoder is At least one memory configured to store program code, A processor configured to read the program code and operate as instructed by the program code, and The program code is provided with, A segmentation code configured to cause at least one processor to segment the input symmetric mesh into a plurality of disconnected UV segments in a UV attribute map, A partition code configured to cause at least one processor to partition at least one UV segment via a plane that divides it into a first side and a second side opposite to the first side, wherein the first side includes a plurality of vertices occupying a first area, Reorganization code configured to cause at least one processor to reorganize the plurality of vertices contained in the first side of the at least one UV segment, wherein the plurality of vertices occupy a second area smaller than the first area after the reorganization, A symmetry transformation code configured to cause the at least one processor to perform 2D symmetry detection on the at least one UV segment in order to find a symmetry transformation on the at least one UV segment, A decision code configured to cause the at least one processor to determine the predicted vertices for each vertex on the second side of the at least one UV segment, based on the symmetry transformation and the vertices on the first side of the at least one UV segment, in order to find the predicted vertices corresponding to the corresponding vertices on the first side of the at least one UV segment, An encoding code configured to cause the at least one processor to encode a bitstream including at least the vertices within the first side of the at least one UV segment. Encoders, including

13. The encoder according to claim 12, wherein the input symmetric mesh is segmented based on a texture associated with the input symmetric mesh, and at least two disconnected UV segments have different textures from each other.

14. The encoder according to claim 13, wherein the input symmetric mesh is segmented based on the symmetry properties of the input symmetric mesh.

15. The encoder according to claim 12, wherein the division of the at least one UV segment includes adding one or more new vertices based on corresponding collision points between the plane and the at least one UV segment.

16. The encoder according to claim 15, wherein the division of the at least one UV segment further comprises adding a new edge between two new vertices.

17. The encoder according to claim 12, wherein the symmetric transformation is a six-parameter affine transformation.

18. The encoder according to claim 12, wherein the symmetric transformation is a four-parameter affine transformation.

19. The encoder according to claim 12, wherein the symmetric transformation is determined based on a regression analysis performed between the vertices on the first side of the at least one UV segment and the vertices on the second side of the UV segment.

20. The encoder according to claim 12, wherein the predicted vertices are symmetrical with respect to the corresponding vertices on the first side of the at least one UV segment.

21. A non-temporary computer-readable medium storing instructions, wherein, when the instructions are executed by the processor of an encoder for encoding an input symmetric mesh, the processor receives: The input symmetric mesh is segmented into multiple disconnected UV segments in the UV attribute map, Dividing at least one UV segment through a plane that divides it into a first side and a second side opposite to the first side, wherein the first side divides at least one UV segment including a plurality of vertices occupying a first area, Reorganizing the plurality of vertices contained within the first side of the at least one UV segment, wherein the plurality of vertices, after the reorganization, occupy a second area smaller than the first area. In order to find the symmetry transformation for the at least one UV segment, 2D symmetry detection is performed on the at least one UV segment, The predicted vertices for each vertex on the second side of the at least one UV segment are determined based on the symmetric transformation of the corresponding vertices on the first side of the at least one UV segment in order to find the predicted vertices on the first side of the at least one UV segment. Encoding a bitstream that includes at least the vertices within the first side of the at least one UV segment. A non-temporary computer-readable medium that enables the execution of a method including [a specific method].

Images