Basic mesh coding using surface reflection symmetry

Abstract

The encoding method includes the step of receiving a polygonal mesh containing a plurality of original vertices. The method further includes the step of deriving an initial basic mesh from the polygonal mesh, wherein the initial basic mesh includes a first basic mesh vertex set. The method further includes the step of performing a symmetry process on the initial basic mesh to generate a symmetric basic mesh including a first side having a first basic mesh vertex set and a second side having a second basic mesh vertex set, wherein each basic mesh vertex in the first basic mesh vertex set has a corresponding symmetric vertex in the second basic mesh vertex set. The method further includes the step of determining a first displacement between each original vertex in the polygonal mesh located on the second side of the symmetric basic mesh and the nearest vertex included in the second basic mesh vertex set.

Description

Technology Field

[0001] This application claims priority to U.S. Provisional Application No. 63 / 406,606 filed September 14, 2022 and U.S. Application No. 18 / 312,224 filed May 4, 2023, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates to a series of advanced video coding techniques. More specifically, the present disclosure relates to basic mesh coding that uses surface reflection symmetry for efficient mesh compression. Background Technology

[0003] Advanced three-dimensional (3D) representations of the world have enabled more immersive forms of interaction and communication. To achieve realism in 3D representations, 3D models are becoming more sophisticated, and a significant amount of data is associated with the creation and consumption of these 3D models. 3D meshes are widely used to 3D model immersive content.

[0004] A 3D mesh can be composed of multiple polygons that describe the surface of a volumetric object. Since a significant amount of information in a mesh sequence can change over time, dynamic mesh sequences may require a large amount of data. Therefore, efficient compression techniques are required to store and transmit this content.

[0005] VMesh is an ongoing MPEG standard for compressing static and dynamic meshes. Current VMesh reference software separates the input mesh into a simplified base mesh and displacement vectors, which are coded independently.

[0006] Symmetry is an attribute of a geometry object when an operation maps the object to itself. In Euclidean metrics, the group of symmetry transformations is called Euclidean isometry, consisting of reflection, translation, rotation, and combinations thereof. Of all these attributes, reflection symmetry, or bilateral symmetry, is the most common symmetry existing in both the biological and non-biological worlds. Every point and edge of a reflection-symmetric mesh has a one-to-one correspondence through the symmetry plane.

[0007] The reflection symmetry planes of a mesh can be detected using simple methods such as principal component analysis (PCA) or advanced techniques such as deep learning. Surface reflection symmetry refers to the case where reflection symmetry appears only on the surface of the mesh and not on xyz points or vertices. Therefore, one-to-one mapping cannot be used in this situation.

[0008] According to one or more embodiments, a video encoding method performed by at least one processor includes the step of receiving a polygon mesh comprising a plurality of original vertices. The video encoding method further includes the step of deriving an initial base mesh from the polygon mesh, wherein the initial base mesh comprises a first base mesh vertex set. The video encoding method further includes the step of performing a symmetrize process on the initial base mesh to generate a symmetrical base mesh comprising a first side having a first base mesh vertex set and a second side having a second base mesh vertex set, wherein each base mesh vertex in the first base mesh vertex set has a corresponding symmetric vertex in the second base mesh vertex set. The video encoding method further includes the step of determining a first displacement between each original vertex in the polygon mesh located on the second side of the symmetrical base mesh and the nearest vertex included in the second base mesh vertex set. The video encoding method further includes the step of generating a coded video bitstream comprising at least a first basic mesh vertex set and each determined first displacement.

[0009] According to one or more embodiments, the encoder comprises at least one memory configured to store program code, and at least one processor configured to read program code and operate as commanded by program code. The program code comprises receiving, configured to cause at least one processor to receive a polygonal mesh comprising a plurality of original vertices. The program code further comprises deriving, configured to cause at least one processor to derive an initial base mesh from the polygonal mesh, wherein the initial base mesh comprises a first base mesh vertex set. The program code further comprises performing, configured to cause at least one processor to perform a symmetry process on the initial base mesh to generate a symmetric base mesh comprising a first side having a first base mesh vertex set and a second side having a second base mesh vertex set, wherein each base mesh vertex in the first base mesh vertex set has a corresponding symmetric vertex in the second base mesh vertex set. The program code further includes a first determining configured to cause at least one processor to determine a first displacement between each original vertex within a polygonal mesh located on the second side of a symmetric base mesh and the nearest vertex included in a second base mesh vertex set. The program code further includes a generating configured to cause at least one processor to generate a coded video bitstream including at least a first base mesh vertex set and each determined first displacement.

[0010] According to one or more embodiments, a computer-readable non-transient medium is provided in which instructions are stored, wherein when the instructions are executed by at least one processor of an encoder, at least one processor performs the steps of: receiving a polygonal mesh comprising a plurality of original vertices; deriving an initial basic mesh from the polygonal mesh, wherein the initial basic mesh comprises a first basic mesh vertex set; performing a symmetry process on the initial basic mesh to generate a symmetric basic mesh comprising a first side having a first basic mesh vertex set and a second side having a second basic mesh vertex set, wherein each basic mesh vertex in the first basic mesh vertex set has a corresponding symmetric vertex in the second basic mesh vertex set; determining a first displacement between each original vertex in the polygonal mesh located on the second side of the symmetric basic mesh and the nearest vertex included in the second basic mesh vertex set; and generating a coded video bitstream comprising at least the first basic mesh vertex set and each determined first displacement. Brief explanation of the drawing

[0011] Additional features, characteristics, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings: Figures 1a through 1f illustrate examples of objects with symmetry. FIG. 2 is a schematic diagram illustrating a block diagram of a communication system according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram illustrating a block diagram of a streaming system according to an embodiment of the present disclosure. FIG. 4 is a schematic diagram illustrating a block diagram of a video encoding mesh framework based on a symmetric basic mesh according to an embodiment of the present disclosure. FIG. 5 is a schematic illustration of a decoding mesh framework based on a symmetric basic mesh according to an embodiment of the present disclosure. FIG. 6 is an example of an exemplary surface symmetric mesh having a symmetric basic mesh according to an embodiment of the present disclosure. FIG. 7 is a diagram showing an example in which the left vertex of a symmetric base mesh is almost perfectly aligned with the symmetric mesh according to an embodiment of the present disclosure. Figure 8 is a diagram illustrating an example of a completed approximate symmetric mesh. FIG. 9 is an exemplary flowchart for encoding a symmetric mesh and generating a coded video bitstream according to an embodiment of the present disclosure. FIG. 10 is an exemplary flowchart for decoding a bitstream and reconstructing a symmetric mesh according to an embodiment of the present disclosure. FIG. 11 is a drawing of a computer system suitable for implementing an embodiment of the present disclosure. Specific details for implementing the invention

[0012] Refer to the accompanying drawings for the following detailed description of exemplary embodiments. Identical reference numbers in different drawings may identify identical or similar elements.

[0013] The foregoing disclosure is intended to provide examples and descriptions, but is not intended to limit or encompass implementations in the exact form disclosed. Modifications and variations are possible in light of the above disclosure or may be achieved through the execution of implementations. Additionally, one or more features or components of one embodiment may be incorporated into or combined with another embodiment (or one or more features of another embodiment). Furthermore, in the flowchart and description of operations provided below, one or more operations may be omitted, one or more operations may be added, one or more operations may be performed simultaneously (at least partially), and one or more operations may be switched.

[0014] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or combinations of hardware and software. The specific control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the operation and behavior of the systems and / or methods have been described herein without reference to specific software code, and it is understood that software and hardware may be designed to implement the systems and / or methods based on the descriptions provided herein.

[0015] Even if a combination of specific features is cited in the claims and / or disclosed in the specification, such combination is not intended to limit the disclosure of possible implementations. In practice, many of these features may be combined in ways not specifically mentioned in the claims and / or disclosed in the specification. Each dependent claim listed below may depend directly on only one claim, but the disclosure of possible implementations includes each dependent claim combined with all other claims in the set of claims.

[0016] Unless explicitly stated otherwise, the elements, acts, or commands used herein should not be interpreted as important or essential. Additionally, the articles "a" and "an" used herein are intended to include one or more items and may be used interchangeably with "one or more." If only one item is intended, the term "one" or similar language is used. Furthermore, terms used herein such as "have," "have," "having," "include," "include," etc., are intended as unlimited terms. Also, the phrase "based on" is intended to mean "at least partially based on" unless otherwise explicitly stated. Additionally, expressions such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood to include only A, only B, or both A and B.

[0017] Throughout this specification, references to “one embodiment,” “an embodiment,” or similar language mean that a specific feature, structure, or characteristic described in relation to the indicated embodiment is included in at least one embodiment of the solution. Accordingly, throughout this specification, phrases such as “in one embodiment,” “in an embodiment,” and similar language may all refer to the same embodiment, but are not necessarily so.

[0018] Additionally, the described features, benefits, and characteristics of this disclosure may be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize, in light of the description herein, that this disclosure may be practiced without one or more of the specific features or benefits of a particular embodiment. In other cases, additional features and benefits in a particular embodiment that may not be present in all embodiments of this disclosure may be recognized.

[0019] Embodiments of the present disclosure relate to providing both lossless mesh coding techniques and lossy mesh coding techniques based on the symmetry properties of mesh content. More specifically, reflection symmetry partitioning, prediction, and coding for mesh content are introduced.

[0020] FIGS. 1a through 1f illustrate examples of objects with symmetry. Various types of objects can be designed with reflection symmetry as illustrated in FIGS. 1a through 1f. As described in detail below, reflection symmetry can be utilized to generate a folding mesh according to one or more exemplary embodiments of the present disclosure.

[0021] With reference to FIGS. 2 and FIGS. 3, one or more embodiments of the present disclosure for implementing the encoding and decoding structure of the present disclosure are described.

[0022] FIG. 2 illustrates a simplified block diagram of a communication system (100) according to an embodiment of the present disclosure. The system (100) may include at least two terminals (110, 120) interconnected via a network (150). For unidirectional transmission of data, the first terminal (110) may code video data, which may include mesh data, at a local location for transmission to another terminal (120) via the network (150). The second terminal (120) may receive the coded video data from the other terminal from the network (150), decode the coded data, and display the restored video data. Unidirectional data transmission may be common in media serving applications, etc.

[0023] FIG. 2 illustrates a second pair of terminals (130, 140) provided to support bidirectional transmission of coded video that may occur, for example, during a video conference. For bidirectional transmission of data, each terminal (130, 140) can code video data captured at a local location for transmission to another terminal via a network (150). Each terminal (130, 140) can also receive coded video data transmitted by another terminal, decode the coded data, and display the restored video data on a local display device.

[0024] In FIG. 2, the terminals (110-140) may be, for example, a server, a personal computer, a smartphone, and / or any other type of terminal. For example, the terminals (110-140) may be a laptop computer, a tablet computer, a media player, and / or dedicated video conferencing equipment. The network (150) represents any number of networks that transmit coded video data between the terminals (110-140), including, for example, wired and / or wireless communication networks. The communication network (150) may exchange data over circuit-switched and / or packet-switched channels. Representative networks include a communication network, a local area network, a wide area network, and / or the Internet. For the purposes of this discussion, the architecture and topology of the network (150) may not be important to the operation of the present disclosure unless described herein below.

[0025] FIG. 3 illustrates the placement of a video encoder and decoder in a streaming environment as an example of an application for the disclosed subject. The disclosed subject may be used with other video-supported applications, such as video conferencing, digital TV, and storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0026] As illustrated in FIG. 3, the streaming system (200) may include a capture subsystem (213) comprising a video source (201) and an encoder (203). The streaming system (200) may further include at least one streaming server (205) and / or at least one streaming client (206).

[0027] A video source (201) may generate a stream (202) containing, for example, a 3D mesh and metadata associated with the 3D mesh. The 3D mesh may consist of multiple polygons describing the surface of a volumetric object. For example, the 3D mesh may contain multiple vertices in 3D space, each vertex being associated with 3D coordinates (e.g., x, y, z). The video source (201) may include, for example, a 3D sensor (e.g., a depth sensor) or 3D imaging technology (e.g., digital camera(s)), and a computing device configured to generate a 3D mesh using data received from the 3D sensor or 3D imaging technology. A sample stream (202), which may have a high data volume compared to an encoded video bitstream, may be processed by an encoder (203) coupled to the video source (201). The encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The encoder (203) may also generate an encoded video bitstream (204). The encoded video bitstream (204), which may have a lower data volume compared to the uncompressed stream (202), may be stored in a streaming server (205) for future use. One or more streaming clients (206) may access the streaming server (205) to retrieve a video bitstream (209), which may be a copy of the encoded video bitstream (204).

[0028] A streaming client (206) may include a video decoder (210) and a display (212). The video decoder (210) may decode a video bitstream (209), which is, for example, an incoming copy of an encoded video bitstream (204), and generate an outgoing video sample stream (211) that can be rendered on a display (212) or another rendering device (not shown). In some streaming systems, the video bitstream (204, 209) may be encoded according to a specific video coding / compression standard.

[0029] According to one or more embodiments, a folding mesh can be generated by utilizing reflection symmetry. Folding mesh is a mutually exclusive domain: It can be segmented. After the folding mesh is generated, the symmetry plane can be represented using a folding tree structure, which can be mesh data with symmetry and asymmetry. While the folding mesh offers some saving potential for symmetric meshes, it cannot handle vertex misalignment and is therefore not suitable for lossless compression. Furthermore, the folding mesh does not provide the efficiency of other compression tools when reflection bit saving is lower than lossless coding, and it is inefficient for signaling symmetry information.

[0030] Current meshing processes also face the following problems: (1) existing mesh coding algorithms primarily rely on the local properties of the mesh, (2) many meshes do not exhibit perfect reflective symmetry where one-to-one mapping cannot be used, (3) meshes can only exhibit symmetry on surfaces, (4) conditions for reflective symmetry are still limited, and (5) base meshes occupy more bits than displacement.

[0031] The proposed method can be used individually or combined in any order and can be used on any polygonal mesh. According to one or more embodiments, the surface symmetry mesh is separated into a symmetry base mesh and a predicted displacement coding.

[0032] According to one or more embodiments, an exemplary encoding framework (400) for encoding surface symmetry is illustrated in FIG. 4. The encoding framework (400) may be implemented by an encoder (203). In the base mesh derivation (402), an initial version of the base mesh may be derived using one or more existing methods known to those skilled in the art. Using a symmetry process (404), the initial mesh may be symmetrized to utilize symmetry properties. This process involves the base mesh and symmetry plane It can output. Symmetric partitioning (406) can be performed by dividing the base mesh into a left part and a right part. In one or more examples, the left base mesh The right base mesh may be referred to as the base mesh to be kept or encoded, and the right base mesh may be referred to as the base mesh to be removed. Displacement coding (408) may be applied to encode the displacement between the right base mesh vertex (408A) and the original base mesh vertex (408B). Displacement coding (408) may also encode the displacement between the left base mesh vertex and the original base mesh vertex. The encoded displacement may be signaled in the bitstream (414). Base mesh encoding (410) may be used to encode the left base mesh vertex, which may be signaled in the bitstream (414). For example, base mesh encoding (410) may compress the left base mesh vertex by any mesh encoder. Plane It can be signaled (412) in the bitstream (414).

[0033] FIG. 5 illustrates an exemplary decoding framework (500) according to one or more embodiments. The decoding framework (500) may be implemented by a decoder (210). A bitstream (502) received by the decoding framework (500) may correspond to a bitstream (414) generated by the encoding framework (400). The bitstream (502) may include a left base mesh vertex, a symmetry plane, an encoded displacement between the left original vertex and the left base mesh vertex, and an encoded displacement between the right original vertex and the right base mesh vertex.

[0034] In the decoding process, the left base mesh can first be reconstructed by the corresponding mesh decoding block (504). Symmetry prediction (508) can be used to predict the right base mesh using the symmetric reflection left base mesh and to be reconstructed by displacement decoding (510). Displacement decoding (510) can be based on a displacement bitstream and a decoded symmetry plane (506). Displacement can be determined between the right base mesh vertex (510A) and the original vertex (510B). Displacement between the left base mesh vertex and the original vertex can also be determined. After displacement decoding is performed, the original vertex can be reconstructed (e.g., restored) based on the displacement decoding of the decoded base mesh and the original vertex.

[0035] FIG. 6 illustrates an exemplary polygonal mesh (600) comprising a left side (600A) with a left original vertex and a right side (600B) with a right original vertex. The polygonal mesh (600) may represent the surface of a three-dimensional object. The polygonal mesh (600) may correspond to a surface-symmetric mesh. A base mesh with left base mesh vertices may be derived from the polygonal mesh (600). Additionally, a corresponding right base mesh vertex may be derived based on the left base mesh vertex, wherein each right base mesh vertex has a corresponding symmetric left base mesh vertex. For each left original vertex, the displacement between the left original vertex and the nearest left base mesh vertex may be encoded. Additionally, for each right original vertex, the displacement between the right original vertex and the nearest right base mesh vertex may be encoded. The displacement may be encoded and included in a bitstream.

[0036] Therefore, the left original vertex and the right original vertex can be derived based on the information contained in the bitstream. For example, the decoder decodes the left original mesh vertex contained in the bitstream, and since the right original mesh vertex and the left original mesh vertex are symmetrical, the right original mesh vertex can be derived from the left original mesh vertex. After the original mesh vertex is derived, the original vertex can be derived (e.g., predicted) based on the displacement contained in the bitstream.

[0037] According to one or more embodiments, the base mesh is symmetrical and becomes a perfect symmetric base mesh average. For example, half of the base mesh can be predicted through a given symmetry plane having zero displacement. Thus, displacement coding for the symmetric base mesh can be removed for half of the base mesh where the displacement is zero.

[0038] According to one or more embodiments, displacement coding for the original vertex may be based on a symmetry plane. For example, the input mesh is a symmetry plane It can be divided into left and right parts based on this. Displacement coding of the left original mesh can be performed first, where the reconstruction of the left original mesh is used together with the reconstructed right base mesh to predict the right vertices.

[0039] In one or more examples, the input mesh may be a nearly perfect symmetric mesh. For example, most of the vertices or at least a subset of the vertices are symmetric pairs. In this scenario, the left original mesh is considered the left base mesh. The left original mesh and its symmetrically reflected counter portion become the base mesh as exemplified in FIG. 7. FIG. 7 illustrates an exemplary polygonal mesh (700) having a left side (700A) and a right side (700b). As exemplified in FIG. 7, the left original vertex is nearly perfectly symmetric with the right base mesh vertex. In this regard, there is zero displacement between the left base mesh vertex and the left original vertex. Therefore, the left original vertex can be treated as the left base mesh, with zero displacement between the left original vertex and the left base mesh. Consequently, during encoding, the displacement of the left side (700A) is removed from the bitstream, allowing for higher efficiency.

[0040] According to one or more embodiments, the input mesh is nearly symmetric and complete. Examples of this type of input mesh include an example of a one-to-one mapping for each vertex through a symmetry plane, where the mapping is also in the normal direction of the symmetry plane. For example, FIG. 8 illustrates an exemplary polygonal mesh (800) having a left side (800A) and a right side (800B). As illustrated in FIG. 8, the left original vertex is nearly symmetrical with the right base mesh vertex. In this regard, the left original mesh vertex and the right base mesh vertex become the base mesh, and one displacement is required to encode each right vertex as follows:

[0041] Formula.(1)

[0042] Here is a scalar displacement offset.

[0043] In this regard, the right original vertex is displaced by magnitude d in one direction (e.g., horizontal direction) from the right base mesh vertex. In contrast to FIG. 8, FIG. 6 and FIG. 7 illustrate an example where the displacement from the base mesh is in two directions (e.g., horizontal and vertical directions). Under these constraints, a significant amount of bits is saved for displacement coding.

[0044] FIG. 9 illustrates a flowchart of an embodiment of an encoding process (900). The encoding process may be performed by an encoder such as an encoder (203) (Fig. 2) that implements an encoding framework (400) (Fig. 4). The process may begin with an operation (S902) in which a polygonal mesh containing a plurality of original vertices is received. For example, referring to FIG. 6, a polygonal mesh (600) having a left original vertex and a right original vertex may be received.

[0045] The process proceeds to an operation (S904) in which an initial basic mesh is derived from a polygonal mesh. For example, referring to FIG. 6, the left basic mesh vertex of side (600A) can be derived as the initial basic mesh. The process proceeds to an operation (S906) in which a symmetry process is performed on the initial basic mesh to generate a symmetric basic mesh. For example, referring to FIG. 6, the right basic mesh vertex of side (600B) can be derived as a symmetric vertex with respect to the left basic mesh vertex.

[0046] The process proceeds to an operation (S908) in which the displacement between the base mesh vertex and the original vertex is determined. For example, referring to FIG. 6a, the displacement between each original vertex of the right side (600B) and the nearest right-side base mesh vertex may be determined. Additionally, if the left side (600A) does not contain zero displacement, the displacement between each original vertex of the left side (600A) and the nearest left-side base mesh vertex may be determined. The process proceeds to an operation (S910) in which a coded video bitstream is generated. The coded video bitstream may include at least the left-side base mesh vertex and the determined displacement.

[0047] FIG. 10 illustrates a flowchart of an embodiment of a decoding process (1000). The decoding process may be performed by a decoder such as a decoder (210) (Fig. 2) that implements a decoding framework (500) (Fig. 5). The process may begin at an operation (S1000) in which a coded video bitstream is received. Referring to FIG. 6 and FIG. 9, the coded video bitstream may correspond to a bitstream generated at operation (S910), which includes a left-side base mesh vertex and a determined displacement.

[0048] The process proceeds to an operation (S1002) in which the first side of the basic mesh is reconstructed. For example, referring to FIG. 6, the left side basic mesh vertex of the side (600A) can be reconstructed. The process proceeds to an operation (S1004) in which the second side of the basic mesh is reconstructed. For example, referring to FIG. 6, the right side basic mesh vertex can be derived based on the left side basic mesh vertex based on the symmetry between the left side basic mesh vertex and the right side basic mesh vertex.

[0049] The process proceeds to an operation (S1006) in which the original vertices of the polygon mesh are reconstructed. For example, referring to FIG. 6, the original vertices on the right side can be reconstructed based on each original right-side mesh vertex and the corresponding displacement contained in the bitstream. Similarly, the original left-side vertices can be reconstructed based on each original left-side mesh vertex and the corresponding displacement contained in the bitstream. The process proceeds to an operation (S1008) in which the polygon mesh is reconstructed. For example, referring to FIG. 6, after reconstructing the original left-side vertices and the original right-side vertices, the polygon mesh (600) is reconstructed.

[0050] The above-described technology may be implemented as computer software that uses computer-readable instructions and is physically stored on one or more computer-readable media. For example, FIG. 11 illustrates a computer system (1100) suitable for implementing a specific embodiment of the present disclosure.

[0051] Computer software may be coded using any suitable machine code or computer language, and may be subject to assembly, compilation, linking, or similar mechanisms to generate code containing instructions that can be executed directly or through interpretation, microcode execution, etc., by a computer central processing unit (CPU), graphics processing unit (GPU), etc.

[0052] The command can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, game devices, Internet of Things devices, etc.

[0053] The components illustrated in FIG. 11 for the computer system (1100) are examples and are not intended to suggest any limitations on the scope of use or function of the computer software implementing the embodiments of the present disclosure. The configuration of the components should not be interpreted as having any dependencies or requirements related to any one or combination of the components exemplified in the non-limiting embodiments of the computer system (1100).

[0054] The computer system (1100) may include a specific human interface input device. This human interface input device may respond to input by one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, applause), visual input (e.g., gestures), and olfactory input (not shown). The human interface device may also be used to capture specific media that are not directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., 2D video, 3D video including stereoscopic video).

[0055] The input human interface device may include one or more of a keyboard (1101), a mouse (1102), a trackpad (1103), a touch screen (1110), a data glove, a joystick (1105), a microphone (1106), a scanner (1107), and a camera (1108) (only one of each illustrated).

[0056] The computer system (1100) may also include a specific human interface output device. This human interface output device may stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. This human interface output device may include a tactile output device (e.g., a tactile feedback device that includes tactile feedback by a touch screen (1110), a data glove, or a joystick (1105), but which does not function as an input device). For example, such devices may include audio output devices (e.g., speakers (1109), headphones (not shown)), visual output devices (e.g., screens (1110) including CRT screens, LCD screens, plasma screens, and OLED screens, each of which may or may not have touch screen input capabilities, each of which may or may not have haptic feedback capabilities, some of which may or may not have 2D visual output or 3D output through means such as virtual reality glasses (not shown), holographic displays and stereographic output of smoke tanks (not shown), and printers (not shown).

[0057] The computer system (1100) may also include human-accessible storage devices and associated media, such as optical media including a CD / DVD ROM / RW (1120) with a CD / DVD or similar media, a thumb drive (1122), a removable hard drive or solid-state drive (1123), legacy magnetic media such as tape and floppy disk (not shown), and specialized ROM / ASIC / PLD-based devices such as a security dongle (not shown).

[0058] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not include transmission media, carrier waves, or other transient signals.

[0059] The computer system (1100) may also include an interface to one or more communication networks. The networks may be wireless, wired, or optical. The networks may also be local, wide-area, metropolitan, automotive and industrial, real-time, latency-tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, automotive and industrial networks including CANBus, etc. A specific network generally requires an external network interface adapter attached to a specific general-purpose data port or peripheral bus (1149) (e.g., a USB port of the computer system (1100)); others are integrated into the core of the computer system (1100) by being attached to a system bus (e.g., an Ethernet interface for a PC computer system or a cellular network interface for a smartphone computer system), as described below. Using any of these networks, the computer system (1100) may communicate with other entities. Such communication may be unidirectional, receive-only (e.g., broadcast TV), unidirectional transmit-only (e.g., from a CANbus to a specific CANbus device), or bidirectional, to another computer system using, for example, a local or wide-area digital network. Such communication may include communication to a cloud computing environment (1155). Specific protocols and protocol stacks may be used in each of the network and network interface as described above.

[0060] The aforementioned human interface device, human-accessible storage device, and network interface (1154) can be attached to the core (1140) of the computer system (1100).

[0061] The core (1140) may include one or more Central Processing Units (CPUs) (1141), Graphics Processing Units (GPUs) (1142), specialized programmable processing units in the form of Field Programmable Gate Areas (FPGAs) (1143), hardware accelerators (1144) for specific tasks, etc. Along with Read-only Memory (ROM) (1145), Random Access Memory (1146), and internal mass storage (1147) such as internal non-user accessible hard drives, SSDs, etc., these devices may be connected via a system bus (1148). In some computer systems, the system bus (1148) may be accessed in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, etc. Peripheral devices may be attached to the core's system bus (1148) directly or via a peripheral bus (1149). Architectures for peripheral buses include PCI, USB, etc. The graphics adapter (1150) can be included in the core (1140).

[0062] The CPU (1141), GPU (1142), FPGA (1143), and accelerator (1144) can be combined to execute specific instructions that can construct the aforementioned computer code. This computer code may be stored in ROM (1145) or RAM (1146). Transitional data may be stored in RAM (1146), but permanent data may be stored, for example, in internal mass storage (1147). Fast storage and retrieval of any memory device may be enabled through the use of a cache memory that may be closely associated with one or more CPUs (1141), GPUs (1142), mass storage (1147), ROM (1145), RAM (1146), etc.

[0063] A computer-readable medium may have computer code for performing various computer-implemented operations. The medium and the computer code may be specifically designed and configured for the purposes of this disclosure, or they may be of a type well known and available to those skilled in the field of computer software.

[0064] As an example not limited to a method, a computer system having an architecture (1100), in particular a core (1140), may provide functionality as a result of processor(s) (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software implemented on one or more types of computer-readable media. Such computer-readable media may be media associated with specific storage of the core (1140) having non-transient characteristics, such as the user-accessible mass storage or internal mass storage (1147) or ROM (1145) introduced above. Software implementing various embodiments of the present disclosure may be stored on such devices and executed by the core (1140). The computer-readable media may include one or more memory devices or chips depending on specific needs. Software may enable the core (1140) and, in particular, the processor within it (including a CPU, GPU, FPGA, etc.) to execute a specific process or part of a specific process described herein, including defining data structures stored in RAM (1146) and modifying such data structures according to a process defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of logic that is hardwired or implemented in a circuit (e.g., an accelerator (1144)) capable of working instead of or with the software to execute a specific process or part of a specific process described herein. References to software may include logic, and where appropriate, vice versa. References to computer-readable media may include circuits that store software for execution (e.g., an integrated circuit (IC)), circuits that implement logic for execution, or, where appropriate, both. The present disclosure covers any suitable combination of hardware and software.

[0065] Although the present disclosure describes some non-limiting embodiments, there are modifications, permutations, and various alternative equivalents that fall within the scope of the disclosure. Accordingly, those skilled in the art will understand that numerous systems and methods can be devised that embody the principles of the present disclosure and thus fall within the spirit and scope of the present disclosure, even though they are not expressly illustrated or described herein.

[0066] The above disclosure also includes the embodiments listed below:

[0067] (1) A video encoding method performed by at least one processor comprises: receiving a polygonal mesh containing a plurality of original vertices; deriving an initial basic mesh from the polygonal mesh, wherein the initial basic mesh includes a first basic mesh vertex set; performing a symmetry process on the initial basic mesh to generate a symmetric basic mesh including a first side having a first basic mesh vertex set and a second side having a second basic mesh vertex set, wherein each basic mesh vertex in the first basic mesh vertex set has a corresponding symmetric vertex in the second basic mesh vertex set; determining a first displacement between each original vertex in the polygonal mesh located on the second side of the symmetric basic mesh and the nearest vertex included in the second basic mesh vertex set; and generating a coded video bitstream including at least the first basic mesh vertex set and each determined first displacement.

[0068] (2) The method according to feature (1) further includes the step of determining a second displacement between each original vertex in a polygonal mesh located on the first side of a symmetrical base mesh and the nearest vertex included in the first base mesh vertex set, and the coded video bitstream further includes each determined second displacement.

[0069] (3) In the method according to feature (1) or (2), at least one determined first displacement includes vertical displacement and horizontal displacement.

[0070] (4) In a method according to any one of features (1) to (3), at least one determined second displacement includes vertical displacement and horizontal displacement.

[0071] (5) In the method according to any one of features (1) to (4), each determined first displacement is limited to either horizontal displacement or vertical displacement.

[0072] (6) In a method according to any one of features (1) to (5), the symmetry process further includes generating a symmetry plane included in the coded video bitstream.

[0073] (7) In a method according to any one of features (1) to (6), each basic vertex in the first basic mesh vertex set has zero displacement for each individual original vertex located on the first side.

[0074] (8) The encoder has at least one memory configured to store program code; The apparatus includes at least one processor configured to read program code and operate as commanded by the program code, wherein the program code comprises: receiving, configured to have at least one processor receive a polygonal mesh containing a plurality of original vertices; derivation, configured to have at least one processor derive an initial basic mesh from the polygonal mesh, wherein the initial basic mesh includes a first basic mesh vertex set; performing, configured to have at least one processor perform a symmetry process on the initial basic mesh to generate a symmetric basic mesh including a first side having a first basic mesh vertex set and a second side having a second basic mesh vertex set, wherein each basic mesh vertex in the first basic mesh vertex set has a corresponding symmetric vertex in the second basic mesh vertex set; first determination, configured to have at least one processor determine a first displacement between each original vertex in the polygonal mesh located on the second side of the symmetric basic mesh and the nearest vertex included in the second basic mesh vertex set; and generation, configured to have at least one processor generate a coded video bitstream including at least the first basic mesh vertex set and each determined first displacement.

[0075] (9) The encoder according to feature (8) further includes a second determination configured to allow at least one processor to determine a second displacement between each original vertex in a polygonal mesh located on the first side of a symmetrical base mesh and the nearest vertex included in the first base mesh vertex set, and the coded video bitstream further includes each determined second displacement.

[0076] (10) In a decoder according to feature (8) or (9), at least one determined first displacement includes a vertical displacement and a horizontal displacement.

[0077] (11) In an encoder according to any one of features (8) to (10), at least one determined second displacement includes a vertical displacement and a horizontal displacement.

[0078] (12) In an encoder according to any one of features (8) to (11), each determined first displacement is limited to either horizontal displacement or vertical displacement.

[0079] (13) In an encoder according to any one of features (8) to (12), the symmetry process further includes generating a symmetry plane included in the coded video bitstream.

[0080] (14) As an encoder according to any one of features (8) to (13), each basic vertex in the first basic mesh vertex set has zero displacement for each individual original vertex located on the first side.

[0081] (15) In a computer-readable non-transient medium in which instructions are stored: when instructions are executed by at least one processor of an encoder, at least one processor performs the steps of: receiving a polygonal mesh containing a plurality of original vertices; deriving an initial basic mesh from the polygonal mesh - the initial basic mesh includes a first basic mesh vertex set -; performing a symmetry process on the initial basic mesh to generate a symmetric basic mesh including a first side having a first basic mesh vertex set and a second side having a second basic mesh vertex set - each basic mesh vertex in the first basic mesh vertex set has a corresponding symmetric vertex in the second basic mesh vertex set -; determining a first displacement between each original vertex in the polygonal mesh located on the second side of the symmetric basic mesh and the nearest vertex included in the second basic mesh vertex set; and generating a coded video bitstream including at least the first basic mesh vertex set and each determined first displacement.

[0082] (16) In a computer-readable non-transient medium according to feature (15), the instruction further comprises: at least one processor performing the step of determining a second displacement between each original vertex in a polygonal mesh located on the first side of a symmetrical base mesh and the nearest vertex included in the first base mesh vertex set, and the coded video bitstream further comprises each determined second displacement.

[0083] (17) In a computer-readable non-transient medium according to either feature (15) or (16), at least one determined first displacement includes a vertical displacement and a horizontal displacement.

[0084] (18) In a computer-readable non-transient medium according to any one of features (15) to (17), at least one determined first displacement includes a vertical displacement and a horizontal displacement.

[0085] (19) In a computer-readable non-transient medium according to any one of features (15) to (19), each determined first displacement is limited to either horizontal displacement and vertical displacement.

[0086] (20) In a computer-readable non-transient medium according to any one of features (15) to (20), the symmetry process further includes generating a symmetry plane included in the coded video bitstream.

Claims

Claim 1 A video encoding method performed by at least one processor, comprising: receiving a polygonal mesh containing a plurality of original vertices; deriving an initial basic mesh from the polygonal mesh, wherein the initial basic mesh includes a first basic mesh vertex set; performing a symmetry process on the initial basic mesh to generate a symmetric basic mesh including a first side having the first basic mesh vertex set and a second side having a second basic mesh vertex set, wherein each basic mesh vertex in the first basic mesh vertex set has a corresponding symmetric vertex in the second basic mesh vertex set; determining a first displacement between each original vertex in the polygonal mesh located on the second side of the symmetric basic mesh and the nearest vertex included in the second basic mesh vertex set; and generating a coded video bitstream including at least the first basic mesh vertex set and each determined first displacement. Claim 2 A video encoding method according to claim 1, further comprising the step of determining a second displacement between each original vertex within the polygon mesh located on the first side of the symmetric base mesh and the nearest vertex included in the first base mesh vertex set, wherein the coded video bitstream further comprises each determined second displacement. Claim 3 A video encoding method according to claim 1, wherein at least one determined first displacement includes a vertical displacement and a horizontal displacement. Claim 4 A video encoding method according to claim 1, wherein at least one determined second displacement includes a vertical displacement and a horizontal displacement. Claim 5 A video encoding method according to claim 1, wherein each determined first displacement is limited to one of horizontal displacement and vertical displacement. Claim 6 A video encoding method according to claim 1, wherein the symmetry process further comprises generating a symmetry plane included in the coded video bitstream. Claim 7 A video encoding method according to claim 1, wherein each basic vertex in the first basic mesh vertex set has a zero displacement with respect to each individual original vertex located on the first side. Claim 8 An encoder comprising: at least one memory configured to store program code; and at least one processor configured to read said program code and operate as commanded by said program code to perform the method of any one of claims 1 to 7. Claim 9 A computer-readable non-transient medium having a stored instruction, wherein the instruction causes the at least one processor of an encoder to execute the method of any one of claims 1 to 7 when the instruction is executed by the at least one processor of an encoder. Claim 10 delete Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images