Method for generating global displacement transformation in grid compression

Abstract

A method of encoding a grid includes determining a plane of symmetry dividing the grid into a first side and a second side opposite the first side, the grid including a plurality of vertices, the first side including a first set of vertices and the second side including a second set of vertices; determining, for each vertex in the first group, a predicted vertex on the second side that is symmetric to the corresponding vertex in the first group; determining a transformation function that transforms each predicted vertex on the second side, the transformation function being determined from a loss function; applying a transformation function that transforms each predicted vertex on the second side; determining a displacement vector between each transformed predicted vertex and a corresponding vertex in the second group; and generating a bitstream including each displacement vector and each encoded vertex in the first group.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 456,774, filed April 3, 2023, and U.S. Application No. 18 / 605,306, filed March 14, 2024, the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0002] This disclosure relates to a set of advanced video coding techniques. More specifically, this disclosure relates to generating global translational transformations and vectors. Background Technology

[0003] Similar to data compression in other media formats, prediction is frequently used in mesh compression, where a displacement vector is assigned to each vertex of the mesh or mesh subdivision to indicate the difference between the predicted attributes (e.g., vertex coordinates) and the original attributes. The prediction may come from a previously encoded mesh frame, a corresponding vertex in a lower-resolution mesh, or vertices in different subdivisions of the same mesh frame. The displacements are then encoded into the compressed mesh bitstream.

[0004] Some meshes possess symmetry properties, and therefore half (or a portion) of a mesh can be predicted based on the other half (the other portion). For example, a prediction of the right mesh can be generated first using the reconstructed left mesh and the plane of symmetry. Then, for each vertex in the predicted right mesh, a displacement vector is computed and encoded such that the displacement mesh closely resembles the original right mesh. In the remainder of this disclosure, for simplicity only, the predictor will be referred to as coming from the “left mesh,” and the portion to be predicted will be referred to as the “right mesh.” The disclosed method can be applied to other general cases, such as using a selected first portion of a mesh to predict a second portion.

[0005] It has been observed that the predicted right mesh may sometimes fail to align well with the true right mesh. This can result in large displacement vectors. Suboptimal alignment can be due to several reasons, such as inappropriate symmetry plane detection, or differences between the original left mesh and the reconstructed left mesh. Summary of the Invention

[0006] According to one or more embodiments, a method for encoding a mesh includes: determining a symmetry plane that divides the mesh into a first side and a second side opposite to the first side, the mesh including a plurality of vertices, the first side including a first set of vertices, and the second side including a second set of vertices; for each vertex in the first set, determining a predicted vertex on the second side that is symmetric to a corresponding vertex in the first set; determining a transformation function for transforming each predicted vertex on the second side, the transformation function being determined based on a loss function; applying the transformation function for transforming each predicted vertex on the second side; determining a displacement vector between each transformed predicted vertex and a corresponding vertex in the second set; and generating a bitstream including each displacement vector and each encoded vertex in the first set.

[0007] According to one or more embodiments, a method for encoding a grid includes: generating a bitstream comprising a portion of the grid, wherein: determining a symmetry plane dividing the grid into a first side and a second side opposite to the first side, the grid comprising a plurality of vertices, the first side comprising a first set of vertices, and the second side comprising a second set of vertices; wherein, for each vertex in the first set, determining a predicted vertex on the second side symmetric to a corresponding vertex in the first set; wherein determining a transformation function for transforming each predicted vertex on the second side, the transformation function being determined based on a loss function; wherein applying the transformation function for transforming each predicted vertex on the second side; wherein determining a displacement vector between each transformed predicted vertex and a corresponding vertex in the second set; and wherein a portion of the grid in the bitstream comprises each displacement vector and each vertex in the first set.

[0008] According to one or more embodiments, a method for decoding an encoded mesh includes: receiving a bitstream including an encoded mesh, wherein the mesh is divided by a symmetry plane into a first side and a second side opposite to the first side, the mesh including a plurality of vertices, the first side including a first group of vertices, and the second side including a second group of vertices; wherein, for each vertex in the first group, a predicted vertex on the second side symmetric to a corresponding vertex in the first group is determined; wherein a transformation function for transforming each predicted vertex on the second side is determined, the transformation function being determined based on a loss function; wherein the transformation function for transforming each predicted vertex on the second side is applied; wherein a displacement vector is determined between each transformed predicted vertex and a corresponding vertex in the second group; and wherein the bitstream includes each displacement vector and each encoded vertex in the first group. Attached Figure Description

[0009] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0010] Figure 1 This is a schematic diagram of a communication system according to an embodiment of the present disclosure.

[0011] Figure 2 This is a schematic diagram of a streaming system according to an embodiment of the present disclosure.

[0012] Figure 3 A flowchart illustrating an example process performed by an encoder according to an embodiment of this disclosure is shown.

[0013] Figure 4 An example of a grid in the case of symmetric prediction according to an embodiment of this disclosure is shown.

[0014] Figure 5A The predicted right grid and the original right grid according to an embodiment of this disclosure are shown.

[0015] Figure 5B The transformed predicted right grid and the original right grid are shown.

[0016] Figure 6 The local displacement vector of each vertex of the transformed predicted right grid is shown.

[0017] Figure 7 This is a diagram of a computer system suitable for implementing the embodiments of this disclosure. Detailed Implementation

[0018] The following detailed description of the exemplary embodiments is with reference to the accompanying drawings. The same reference numerals in different drawings may identify the same or similar elements.

[0019] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit the implementation to the exact forms disclosed. Modifications and variations can be made based on the foregoing disclosure, or modifications and variations can be derived from practice of the implementation. Furthermore, one or more features or components of one embodiment may be incorporated into or combined with another embodiment (or one or more features of another embodiment). Additionally, in the flowcharts and descriptions of operations provided below, it should be understood that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed (at least partially) simultaneously, and the order of one or more operations may be switched.

[0020] It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limiting in its implementation. Therefore, this document describes the operation and behavior of the systems and / or methods without reference to any specific software code—it should be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0021] Even if specific combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically recited in the claims and / or not disclosed in the specification. While each dependent claim listed below may directly refer to only one claim, the disclosure of possible implementations includes combinations of each dependent claim with every other claim in the claim set.

[0022] Unless explicitly stated otherwise, no element, action, or instruction used herein should be construed as critical or necessary. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” The term “an” or similar language is used when the intent is only one item. Furthermore, as used herein, the terms “has,” “have,” “having,” “include,” “including,” etc., are intended to be open-ended terms. Furthermore, unless explicitly stated otherwise, the phrase “based on” is intended to mean “at least partially based on.” Furthermore, expressions such as “at least one of [A] and [B]” or “at least one of [A] or [B]” should be understood to include only A, only B, or both A and B.

[0023] Throughout this specification, references to "one embodiment," "implementation," or similar language mean that a particular feature, structure, or characteristic described in connection with the indicated embodiment is included in at least one embodiment of this solution. Therefore, throughout this specification, the phrases "in one embodiment," "in an embodiment," and similar language may, but not necessarily all, refer to the same embodiment.

[0024] Furthermore, the features, advantages, and characteristics described in this disclosure may be combined in any suitable manner in one or more embodiments. Based on the description herein, those skilled in the art will recognize that this disclosure may be practiced without one or more specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments of this disclosure.

[0025] Reference Figures 1 to 2 This describes one or more embodiments of the encoding and decoding structures for implementing the present disclosure.

[0026] Figure 1 A simplified block diagram of a communication system 100 according to an embodiment of the present disclosure is shown. System 100 may include at least two terminals 110 and 120 interconnected via a network 150. For unidirectional data transmission, the first terminal 110 may encode video data, including grid data, at its local location for transmission to the other terminal 120 via the network 150. The second terminal 120 may receive the encoded video data from the other terminal from the network 150, decode the encoded data, and display the recovered video data. Unidirectional data transmission is common in media service applications, etc.

[0027] Figure 1 A second pair of terminals 130 and 140 is shown, provided to support bidirectional transmission of encoded video, for example, during video conferencing. For bidirectional data transmission, each terminal 130 and 140 can encode video data captured at a local location for transmission to the other terminal via network 150. Each terminal 130 and 140 can also receive encoded video data transmitted by the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[0028] exist Figure 1 In this context, terminals 110 to 140 can be, for example, servers, personal computers, and smartphones, and / or any other type of terminal. For example, terminals (110 to 140) can be laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 150 refers to any number of networks transmitting encoded video data between terminals 110 to 140, including, for example, wired and / or wireless communication networks. Communication network 150 can exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, unless otherwise stated below, the architecture and topology of network 150 may be irrelevant to the operation of this disclosure.

[0029] Figure 2 The placement of a video encoder and decoder in a streaming environment is illustrated as an example of an application to the disclosed subject matter. The disclosed subject matter can be used with other video-enabled applications, including, for example, video conferencing, digital television, and storing compressed video on digital media including CDs (Compact Discs), DVDs (Digital Versatile Discs), memory sticks, etc.

[0030] like Figure 2 As shown, the streaming system 200 may include a capture subsystem 213, which includes a video source 201 and an encoder 203. The streaming system 200 may also include at least one streaming server 205 and / or at least one streaming client 206.

[0031] Video source 201 can create a stream 202, for example, including a 3D mesh and metadata associated with the 3D mesh. Video source 201 may include, for example, a 3D sensor (e.g., a depth sensor) or 3D imaging technology (e.g., a digital camera device) and a computing device configured to generate the 3D mesh using data received from the 3D sensor or 3D imaging technology. The sample stream 202, which may have a high data volume compared to the encoded video bitstream, can be processed by an encoder 203 coupled to video source 201. Encoder 203 may include hardware, software, or a combination thereof to implement or enforce aspects of the disclosed subject matter as described in more detail below. Encoder 203 can also generate an encoded video bitstream 204. The encoded video bitstream 204, which may have a lower data volume compared to the uncompressed stream 202, can be stored on streaming server 205 for future use. One or more streaming clients 206 can access streaming server 205 to retrieve video bitstream 209, which may be a copy of encoded video bitstream 204.

[0032] Streaming client 206 may include video decoder 210 and display 212. Video decoder 210 may, for example, decode video bitstream 209 as an incoming copy of encoded video bitstream 204, and create outgoing video sample stream 211 that can be displayed on display 212 or another presentation device (not depicted). In some streaming systems, video bitstreams 204, 209 may be encoded according to certain video encoding / compression standards.

[0033] A mesh consists of polygons that describe the surface of a volumetric object. Each polygon is defined by its vertices in 3D space and information about how these vertices are connected (called connectivity information). Optionally, vertex attributes such as color and normals can be associated with mesh vertices. Attributes can also be associated with the mesh surface using mapping information parameterized by a 2D property map. Such mapping is typically described by a set of parametric coordinates called UV coordinates or texture coordinates associated with the mesh vertices. The 2D property map is used to store high-resolution attribute information such as texture, normals, and displacement. This information can be used for various purposes, such as texture mapping and shading.

[0034] Dynamic mesh sequences can require large amounts of data because they may contain a wealth of information that changes over time. Therefore, efficient compression techniques are needed to store and transmit such content. Previously, MPEG (Moving Picture Experts Group) developed mesh compression standards IC, MESHGRID, and FAMC to handle dynamic meshes with constant connectivity and time-varying geometry and vertex properties. However, these standards do not consider time-varying property graphs and connectivity information. DCC (Digital Content Creation) tools often generate such dynamic meshes. Correspondingly, generating constant connectivity dynamic meshes, especially under real-time constraints, is challenging for volumetric acquisition techniques. Existing standards do not support this type of content. MPEG plans to develop a new mesh compression standard to directly handle dynamic meshes with time-varying connectivity information and optionally time-varying property graphs. This standard targets lossy and lossless compression for various applications such as real-time communication, storage, free-viewpoint video, AR, and VR. Features such as random access and scalable / progressive coding are also considered.

[0035] Mesh geometry information includes vertex connectivity information, 3D coordinates, and 2D texture coordinates. Compression of vertex 3D coordinates (also known as vertex positions) is very important because, in most cases, it constitutes a significant portion of the entire geometry-related data.

[0036] Implementations of this disclosure involve generating global displacement transformations and vectors.

[0037] According to one or more implementations, the suboptimal alignment problem is addressed by introducing a global displacement transformation to align the predicted right grid with the original right grid. The global transformation can handle large displacement errors, and the local displacement vectors can subsequently be used to obtain a tighter match.

[0038] In one or more examples, a global rigid transformation consisting of 3D rotation, 3D scaling, and 3D translation is computed to optimally align the predicted right mesh with the original right mesh. The global transformation is then applied to all vertices of the predicted right mesh to generate a new predicted right mesh (e.g., "transformed predicted right mesh"). Furthermore, a local displacement vector is appended to each vertex of the transformed predicted right mesh. In one or more examples, the global transformation may be computed and applied to a set of partially cut meshes, depending on which approach is preferred. Applying the global transformation to a portion of the mesh can be helpful in scenarios where the mesh exhibits local rigidity. The steps for computed global and local symmetric displacements are detailed below according to one or more implementations.

[0039] Figure 3 A flowchart illustrating an implementation of process 300 performed by the encoder is shown. Process 300 may be performed by encoder 203.

[0040] The process can begin at operation S302, where the predicted right grid is obtained. The predicted right grid can be obtained by reflecting the reconstructed left grid about the symmetry plane. In one or more examples, the symmetry plane is determined and signaled. A portion of the grid is selected as the left grid and encoded. Based on these two, the predicted right grid is generated by mirroring the decoded left grid using the signaled plane.

[0041] Figure 4 An example of symmetric prediction using mirror reflection is shown. Plane p1 is the symmetric plane; vertex V' (right grid) is symmetrically predicted according to V (left grid); x1' and x1, x5 and x5' are co-located vertices on the plane. Vertices X6, X7, and X8 are vertices in the left grid, while vertex X 6' X 7' and X 8' It is the predicted vertex in the right grid.

[0042] The process proceeds to operation S304, where the optimal rigid transformation is found. In one or more examples, the optimal rigid transformation aligns the predicted right mesh with the original right mesh according to the loss function. For this, global and / or local registration algorithms can be used, such as Iterative Closest Point (ICP), Fast Global Registration (FGR), and TEASER. Registration algorithms can consider the vertices of both meshes and ignore connectivity, or they can jointly consider vertices and connectivity. The registration algorithm gives the 3D rigid transformation. (For example, 3D rotation and 3D translation).

[0043] In one or more examples, several loss functions may be considered. In one or more implementations, the loss function is the mean square error between vertex correspondences in the registration. This loss reduces the magnitude of the local displacement vector found in operation S308.

[0044] In another implementation, the loss function includes the cost of encoding the displacement. This implementation finds the optimal transform that uses fewer bits to encode the displacement (e.g., in this case, the size of the local displacement vector may or may not be the minimum possible size). This loss reduces the number of bits required to encode the global transform as well as the local displacement found in operation S308. Multiple loss functions can be considered in combination, for example, defining a new loss function as the sum of the mean squared error and the number of bits required to encode the displacement vector. Similarly, the loss function can be defined as a weighted sum of the mean squared error and the number of bits required to encode the displacement vector.

[0045] The process proceeds to operation S306, where the vertices of the predicted right grid are transformed to obtain the transformed predicted right grid. A 3D transformation can be applied to all vertices of the predicted right grid. The i-th vertex of the predicted right grid... The new vertex coordinates are determined by Provided.

[0046] The process proceeds to operation S308 to find the local displacement vector of each vertex of the transformed predicted right mesh. In one or more examples, the transformed predicted right mesh is compared with the original right mesh and the local displacements are found to further modify the vertices of the transformed predicted right mesh and make it resemble the original right mesh.

[0047] The embodiments of this disclosure decompose displacement information (or prediction error) into two components—(1) a global displacement transformation first applied to all vertices (which is the same for all vertices), followed by (2) a local displacement vector unique to each vertex. This framework may be unaware of the methods used to obtain the global transformation (e.g., registration or other similar methods) and the methods used to calculate the local displacements.

[0048] Figure 5A , Figure 5B and Figure 6 The proposed framework of this disclosure is shown. For simplicity, only the mesh vertices are shown. Figure 4 (A) shows the predicted right grid (dark circle) and the original right grid (light circle). Figure 4 (B) shows the transformed predicted right grid (dark circle) and the original right grid (light circle). Figure 6The local displacement vector of each vertex of the transformed predicted right grid is shown. Therefore, as... Figure 5A , Figure 5B and Figure 6 As shown, a global transformation is used to align the predicted right grid with the original right grid. Then, the local displacement is found.

[0049] According to one or more implementations, global displacement transformation can be estimated at any subdivision level. In one or more examples, the subdivision level may correspond to a decimation process. For example, a decimation process can be performed on the original mesh to reduce the number of vertices, thereby creating a subdivision level. The decimation process can be repeated to create additional subdivision levels.

[0050] The global transformation can be estimated in two ways. One way is to use the mesh at subdivision level X to estimate the global transformation for faster computation. The second way is to use the global transformation obtained from the mesh at subdivision level X as initialization, and then fine-tune it using a mesh with all vertices.

[0051] The methods described above are not limited to symmetric (e.g., left and right) predictions. They can be used to predict one part of a grid (or subdivision grid) based on another part of the grid (or subdivision grid).

[0052] According to one or more implementations, the decoder is, for example, decoder 210 ( Figure 2 The decoder can receive a bitstream generated by the encoder. The bitstream can include encoded vertices on the left side of the grid, displacement vectors, and an indication of the transformation function. The decoder can decode and reconstruct each vertex on the left side of the grid. For each vertex on the left side, the decoder can determine the predicted symmetric vertex on the right side of the grid. The decoder can apply the transformation function to each predicted symmetric vertex. Subsequently, the decoder can determine each original vertex on the right side of the grid by applying displacement vectors to each transformed predicted symmetric vertex.

[0053] The above-described technology can be implemented as computer software that uses computer-readable instructions and is physically stored on one or more computer-readable media. For example, Figure 7 A computer system 700 suitable for implementing certain embodiments of the present disclosure is shown.

[0054] Computer software can be encoded using any suitable machine code or computer language, which can be subjected to mechanisms such as assembly, compilation, and linking to create code containing instructions that can be executed directly by a computer's central processing unit (CPU), graphics processing unit (GPU), or through interpretation, microcode execution, etc.

[0055] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0056] Figure 7 The components shown for computer system 700 are examples and are not intended to impose any limitation on the scope or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement relating to any one or a combination of components shown in a non-limiting embodiment of computer system 700.

[0057] Computer system 700 may include certain human-machine interface input devices. Such human-machine interface input devices can respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, tapping), visual input (e.g., gestures), and olfactory input (not depicted). Human-machine interface devices can also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image capturing devices), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0058] The input human-machine interface device may include one or more of the following (only one of each depicted): keyboard 701, mouse 702, touchpad 703, touch screen 710, data glove, joystick 705, microphone 706, scanner 707, and camera device 708.

[0059] The computer system 700 may also include certain human-machine interface (HMI) output devices. Such HMI output devices can stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. These HMI output devices may include tactile output devices (e.g., tactile feedback via a touchscreen 710, data gloves, or joystick 705, but tactile feedback devices that are not used as input devices may also exist). For example, such devices may be audio output devices (e.g., speakers 709, headphones (not depicted)), visual output devices (e.g., screens 710 including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input capabilities, each with or without tactile feedback capabilities, some of which may output two-dimensional visual output or more than three-dimensional output via methods such as stereoscopic graphics output; virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted)), and printers (not depicted).

[0060] The computer system 700 may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM (Read Only Memory, ROM) / RW 720 with media such as CD / DVD 721, thumb drives 722, removable hard disk drives or solid-state drives 723, conventional magnetic media such as magnetic tapes and floppy disks (not depicted), and devices based on dedicated ROM / ASIC (Application Specific Integrated Circuit, ASIC) / PLD (Programmable Logic Device, PLD) such as security dongles (not depicted), etc.

[0061] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0062] The computer system 700 may also include interfaces to one or more communication networks. These networks can be wireless, wired, or optical. They can also be local area, wide area, metropolitan area, vehicular and industrial, real-time, or latency-tolerant. Examples of networks include: local area networks such as Ethernet; wireless LANs; cellular networks including GSM (Global System for Mobile Communications), 3G (the Third Generation), 4G (the Fourth Generation), 5G (the Fifth Generation), and LTE (Long Term Evolution); wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; and vehicle and industrial networks including CANbus. Some networks typically require external network interface adapters attached to certain general-purpose data ports or peripheral buses 749 (such as, for example, USB ports of computer system 700); other networks are typically integrated into the core of computer system 700 via system buses attached to systems described below (e.g., Ethernet interfaces to PC computer systems or cellular network interfaces to smartphone computer systems). Using any of these networks, computer system 700 can communicate with other entities. Such communication can be one-way receiving (e.g., broadcasting TV), one-way transmitting (e.g., to a CANBus device), or bidirectional, such as to other computer systems using local area digital networks or wide area digital networks. Such communication can include communication to cloud computing environments 755. Certain protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0063] The aforementioned human-machine interface device, human-accessible storage device, and network interface 754 can be attached to the core 740 of the computer system 700.

[0064] Core 740 may include one or more central processing units (CPUs) 741, graphics processing units (GPUs) 742, dedicated programmable processing units in the form of field-programmable gate areas (FPGAs) 743, hardware accelerators 744 for certain tasks, etc. These devices, along with read-only memory (ROM) 745, random access memory 746, and internal mass storage devices 747 such as internal non-user-accessible hard disk drives (HDDs) and solid state drives (SSDs), can be connected via system bus 748. In some computer systems, system bus 748 may be accessed as one or more physical connectors to allow for expansion by adding CPUs, GPUs, etc. Peripheral devices may be attached directly to or via peripheral bus 749 to the core's system bus 748. Peripheral bus architectures include PCI (Peripheral Component Interconnect / Interface), USB (Universal Serial Bus), etc. A graphics adapter 750 may be included in core 740.

[0065] The CPU 741, GPU 742, FPGA 743, and accelerator 744 can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM 745 or RAM (Random Access Memory) 746. Transient data can also be stored in RAM 746, while permanent data can be stored, for example, in an internal mass storage device 747. Fast storage and retrieval of any memory device within the memory device can be achieved using a cache memory, which can be closely associated with one or more CPUs 741, GPUs 742, mass storage devices 747, ROM 745, RAM 746, etc.

[0066] Computer-readable media may have computer code thereon for performing operations of various computer implementations. The media and computer code may be specifically designed and constructed for the purposes of this disclosure, or the media and computer code may be of a type known and available to those skilled in the art of computer software.

[0067] By way of example and not limitation, a computer system 700 with an architecture, and particularly a core 740, can be functionalized by a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage devices as described above, and certain storage devices of the core 740 having non-transitory characteristics, such as internal mass storage device 747 or ROM 745. Software implementing various embodiments of this disclosure can be stored in such devices and executed by the core 740. Depending on specific needs, the computer-readable media may include one or more memory devices or chips. The software can cause the core 740, and particularly its processors (including CPU, GPU, FPGA, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM 746 and modifying such data structures according to the software-defined processes. Alternatively or as an alternative, the computer system may provide functionality resulting from or otherwise embodied in circuitry (e.g., accelerator 744) via hardwired logic, which may replace or operate with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and references to logic may also include software. Where appropriate, references to computer-readable media may include circuitry storing software for execution (e.g., integrated circuits (ICs)), circuitry implementing logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0068] While this disclosure has described several non-limiting embodiments, there are variations, substitutions, and various alternative equivalents that fall within the scope of this disclosure. Therefore, it will be appreciated that those skilled in the art will be able to conceive of many systems and methods that, while not expressly shown or described herein, embody the principles of this disclosure and are therefore within its spirit and scope.

Claims

1. A method of encoding a mesh, comprising: determining a symmetry plane that divides the mesh into a first side and a second side opposite the first side, the mesh comprising a plurality of vertices, the first side comprising a first set of vertices, the second side comprising a second set of vertices; for each vertex in the first set, determining a predicted vertex on the second side that is symmetric to the corresponding vertex in the first set; determining a transformation function that transforms each predicted vertex on the second side, the transformation function determined according to a loss function; applying the transformation function that transforms each predicted vertex on the second side; determining a displacement vector between each transformed predicted vertex and the corresponding vertex in the second set; and generating a bitstream comprising each displacement vector and each encoded vertex in the first set. determining the transformation process comprises performing a registration process between the first set of vertices and the second set of vertices.

2. The method of claim 1, wherein, the registration process outputs a rotation function that rotates each predicted vertex on the second side as the transformation function.

3. The method of claim 2, wherein, the registration process outputs a translation function that translates each predicted vertex on the second side as the transformation function.

4. The method of claim 3, wherein, the loss function is a mean squared error function that minimizes the displacement vector between each transformed predicted vertex and the corresponding vertex in the second set.

5. The method of claim 2, wherein, the loss function is a cost function that minimizes a number of bits used to encode the displacement vector between each transformed predicted vertex and the corresponding vertex in the second set.

6. The method of claim 2, wherein, the loss function comprises: (i) a mean squared error function that minimizes the displacement vector between each transformed predicted vertex and the corresponding vertex in the second set; and (ii) a cost function that minimizes a number of bits used to encode the displacement vector between each transformed predicted vertex and the corresponding vertex in the second set.

7. The method of claim 2, wherein, a weight of the mean squared error function is higher than the cost function.

8. The method of claim 7, wherein, a weight of the cost function is higher than the mean squared error function.

9. The method of claim 7, wherein, 10. The method of claim 1, further comprising: performing decimation on the mesh to reduce a number of vertices in the mesh, wherein the transformation function is applied on the decimated mesh.

11. A method of encoding a mesh, comprising: generating a bitstream comprising a portion of the mesh, wherein a symmetry plane is determined that divides the mesh into a first side and a second side opposite the first side, the mesh comprising a plurality of vertices, the first side comprising a first set of vertices, the second side comprising a second set of vertices; wherein for each vertex in the first set, a predicted vertex on the second side that is symmetric to the corresponding vertex in the first set is determined; wherein a transformation function that transforms each predicted vertex on the second side is determined, the transformation function determined according to a loss function; wherein the transformation function that transforms each predicted vertex on the second side is applied; wherein a displacement vector between each transformed predicted vertex and the corresponding vertex in the second set is determined; and wherein a bitstream is generated comprising each displacement vector and each encoded vertex in the first set. wherein the portion of the mesh in the bitstream includes each displacement vector and each vertex in the first set.

12. The method of claim 11, wherein, The transformation process includes a registration process between the first set of vertices and the second set of vertices.

13. The method of claim 12, wherein, The registration process outputs a rotation function that rotates each predicted vertex on the second side as the transformation function.

14. The method of claim 13, wherein, The registration process outputs a translation function that translates each predicted vertex on the second side as the transformation function.

15. The method of claim 12, wherein, The loss function is a mean squared error function that minimizes displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set.

16. The method of claim 12, wherein, The loss function is a cost function that minimizes a number of bits used to encode displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set.

17. The method of claim 12, wherein, The loss function includes (i) a mean squared error function that minimizes displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set and (ii) a cost function that minimizes a number of bits used to encode the displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set.

18. The method of claim 17, wherein, The weight of the mean squared error function is higher than the cost function.

19. The method of claim 17, wherein, The weight of the cost function is higher than the mean squared error function.

20. A method of decoding an encoded mesh, comprising: receiving a bitstream, the bitstream including the encoded mesh, wherein the mesh is divided by a symmetry plane that divides the mesh into a first side and a second side opposite the first side, the mesh including a plurality of vertices, the first side including a first set of vertices, the second side including a second set of vertices; wherein for each vertex in the first set, a predicted vertex on the second side that is symmetrical to the corresponding vertex in the first set is determined; wherein a transformation function that transforms each predicted vertex on the second side is determined, the transformation function determined according to a loss function; wherein the transformation function that transforms each predicted vertex on the second side is applied; wherein displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set are determined; and wherein the bitstream includes each displacement vector and each encoded vertex in the first set. wherein the portion of the mesh in the bitstream includes each displacement vector and each vertex in the first set. The transformation process includes a registration process between the first set of vertices and the second set of vertices. The registration process outputs a rotation function that rotates each predicted vertex on the second side as the transformation function. The registration process outputs a translation function that translates each predicted vertex on the second side as the transformation function. The loss function is a mean squared error function that minimizes displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set. The loss function is a cost function that minimizes a number of bits used to encode displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set. The loss function includes (i) a mean squared error function that minimizes displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set and (ii) a cost function that minimizes a number of bits used to encode the displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set. The weight of the mean squared error function is higher than the cost function. The weight of the cost function is higher than the mean squared error function.

20. A method of decoding an encoded mesh, comprising: receiving a bitstream, the bitstream including the encoded mesh, wherein the mesh is divided by a symmetry plane that divides the mesh into a first side and a second side opposite the first side, the mesh including a plurality of vertices, the first side including a first set of vertices, the second side including a second set of vertices; wherein for each vertex in the first set, a predicted vertex on the second side that is symmetrical to the corresponding vertex in the first set is determined; wherein a transformation function that transforms each predicted vertex on the second side is determined, the transformation function determined according to a loss function; wherein the transformation function that transforms each predicted vertex on the second side is applied; wherein displacement vectors between each transformed predicted vertex and a corresponding vertex in the second set are determined; and wherein the bitstream includes each displacement vector and each encoded vertex in the first set.

Images