Method, apparatus, medium for video encoding

Abstract

A method and apparatus for deriving a curve of a mesh sequence from a frame of a video bitstream, the curve comprising a plurality of vertices; subdividing the curve by adding a plurality of subdivision vertices; calculating one or more displacement vectors between each of the plurality of vertices on the curve and each of the plurality of subdivision vertices on the subdivided curve; transforming the one or more displacement vectors to output one or more original coefficients; tracking a current frame from a previous frame to obtain a one-to-one correspondence between a plurality of vertices of the current frame and a plurality of vertices of the previous frame; predicting one or more coefficients for each of the plurality of vertices in the current frame based on the one-to-one correspondence; performing entropy encoding on one or more prediction residuals; and encoding the one or more prediction residuals.

Description

[0001] Cross-referencing

[0002] This application claims priority to U.S. Provisional Application No. 63 / 358,084, filed July 1, 2022, and U.S. Application No. 18 / 315,020, filed May 10, 2023, the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure relates to a set of advanced video coding techniques. More specifically, this disclosure relates to compression techniques for encoding vertex displacements of dynamically subdivided meshes. Background Technology

[0004] Advances in 3D capture, modeling, and rendering have facilitated the ubiquitous presence of 3D content across multiple platforms and devices. Today, it's possible to capture a baby's first steps on one continent and allow grandparents on other continents to see (and perhaps interact with) the child, enjoying a fully immersive experience with them. However, to achieve this realism, the models have become increasingly complex, with vast amounts of data associated with their creation and consumption. 3D meshes are widely used to represent this immersive content.

[0005] Dynamic mesh sequences can require large amounts of data because they may contain a wealth of information that changes over time. Therefore, efficient compression techniques are needed to store and transmit such content. The Moving Pictures Experts Group (MPEG) has previously developed mesh compression standards IC, MESHGRID, and FAMC to handle dynamic meshes with constant connectivity and time-varying geometry and vertex properties. However, these standards do not account for time-varying property mappings and connectivity information. Digital Content Creation (DCC) tools often generate such dynamic meshes. Consequently, generating dynamic meshes with constant connectivity is challenging for volumetric acquisition techniques, especially under real-time constraints. Existing standards do not support this type of content. A new mesh compression standard is needed to directly handle dynamic meshes with time-varying connectivity information and optional time-varying property mappings. This standard will target lossy and lossless compression for various applications such as real-time communication, storage, free-viewpoint video, augmented reality (AR), and virtual reality (VR). Features such as random access and scalable / progressive coding are also considered. Summary of the Invention

[0006] The following presents a simplified overview of one or more embodiments of this disclosure to provide a basic understanding of these embodiments. This overview is not a comprehensive overview of all contemplated embodiments and is intended neither to identify key or core elements of all embodiments nor to describe the scope of any or all embodiments. The sole purpose of this overview is to present some concepts of one or more embodiments of this disclosure in a simplified form as a prelude to the more detailed description that follows.

[0007] This disclosure provides a compression technique for encoding vertex displacements of dynamically subdivided meshes.

[0008] According to some embodiments, a method executed by at least one processor is provided, the method comprising: deriving a curve of a grid sequence from frames of a video stream, the curve including a plurality of vertices; subdividing the curve by adding a plurality of subdivision vertices; calculating one or more displacement vectors between each of the plurality of vertices on the curve and each of the plurality of subdivision vertices on the subdivided curve; transforming the one or more displacement vectors to output one or more original coefficients; and determining whether to encode the current frame using inter-frame picture information or intra-frame picture information. In response to determining to encode the current frame using inter-frame picture information, the method may include: tracing the current frame back through a previous frame to obtain a one-to-one correspondence between a plurality of vertices in the current frame and a plurality of vertices in the previous frame; predicting one or more coefficients for each of the plurality of vertices in the current frame based on the one-to-one correspondence; performing entropy coding on one or more prediction residuals; and encoding the one or more prediction residuals based on the predicted one or more coefficients. In response to determining to encode the current frame using intra-frame picture information, the method may include: encoding one or more coefficients using arithmetic coding.

[0009] According to other aspects of one or more embodiments, an apparatus and a non-transitory computer-readable medium consistent with the method are also provided.

[0010] Additional embodiments will be set forth in the following description, and some of the additional embodiments will be obvious from the description, and / or can be learned by practicing the embodiments presented in this disclosure. Attached Figure Description

[0011] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0012] Figure 1 This is a simplified block diagram of a communication system according to some embodiments;

[0013] Figure 2This is a simplified block diagram of a streaming system according to some embodiments;

[0014] Figure 3 These are illustrations of a detailed scheme based on some embodiments;

[0015] Figure 4 These are examples of ordered displacement coefficients according to some embodiments;

[0016] Figure 5 This is an operational flowchart illustrating the steps performed by a program for encoding vertex displacements of a dynamically subdivided mesh, according to some embodiments; and

[0017] Figure 6 This is a schematic diagram of a computer system suitable for implementing the embodiments. Detailed Implementation

[0018] The following detailed description of the example embodiments is with reference to the accompanying drawings. The same reference numerals in different drawings may identify the same or similar elements.

[0019] A mesh can comprise several polygons describing the surface of a volumetric object. Information about its vertices in 3D space and how these vertices are connected defines each polygon; this is called connectivity information. Optionally, vertex attributes, such as color and normals, can be associated with mesh vertices. Attributes can also be associated with the mesh surface by utilizing mapping information that parametrically modulates the mesh using 2D attribute mapping. This mapping can be defined using a set of parametric coordinates, called UV coordinates or texture coordinates, and this mapping is associated with mesh vertices. 2D attribute mapping can be used to store high-resolution attribute information, such as textures, normals, and displacements. This information can be used for various purposes, such as texture mapping, shading, and mesh reconstruction.

[0020] This disclosure proposes several methods to improve existing methods for compressing vertex displacements of dynamically subdivided meshes. These methods can be used individually or in any combination. Furthermore, each of the methods (or embodiments), encoders, and decoders can be implemented using processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium.

[0021] Reference Figures 1 to 2 This describes embodiments of the encoding and decoding structures of this disclosure for implementing the present disclosure.

[0022] Figure 1A simplified block diagram of a communication system (100) according to an embodiment of the present disclosure is shown. The system (100) may include at least two terminals (110, 120) interconnected via a network (150). For unidirectional data transmission, the first terminal (110) may encode video data, which may include grid data, at a local location for transmission to the other terminal (120) via the network (150). The second terminal (120) may receive the encoded video data from the other terminal from the network (150), decode the encoded data, and display the recovered video data. Unidirectional data transmission can be applied in media service applications, etc.

[0023] Figure 1 A second pair of terminals (130, 140) is shown, provided to support bidirectional transmission of encoded video, for example, during video conferencing. For bidirectional data transmission, each terminal (130, 140) can encode video data captured at a local location for transmission to the other terminal via a network (150). Each terminal (130, 140) can also receive encoded video data sent by the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[0024] exist Figure 1 In this context, terminals (110-140) can be servers, personal computers, and smartphones, and / or any other type of terminal. For example, terminals (110-140) can be laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. A network (150) refers to any number of networks that transmit encoded video data between terminals (110-140), including, for example, wired communication networks and / or wireless communication networks. Communication networks (150) can exchange data in circuit-switched channels and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, the architecture and topology of the network (150) may be of little importance to the operation of this disclosure unless explained below.

[0025] Figure 2 The placement of a video encoder and video decoder in a streaming environment is illustrated as an example of the application of the disclosed subject matter. The disclosed subject matter can also be applied to other video-enabled applications, including, for example, video conferencing, digital television, and storing compressed video on digital media including CDs, DVDs, Memory Sticks, etc.

[0026] like Figure 2As shown, the streaming system (200) may include a capture subsystem (213), which may include a video source (201) and an encoder (203). The streaming system (200) may also include at least one streaming server (205) and / or at least one streaming client (206).

[0027] A video source (201) can create a stream (202) including, for example, a 3D mesh and metadata associated with the 3D mesh. The video source (201) may include, for example, a 3D sensor (e.g., a depth sensor) or a 3D imaging technology (e.g., a digital camera), and a computing device configured to generate the 3D mesh using data received from the 3D sensor or the 3D imaging technology. The sample stream (202) may have a higher data volume compared to an encoded video stream and may be processed by an encoder (203) coupled to the video source (201). The encoder 203 may include hardware, software, or a combination thereof to perform or implement various aspects of the disclosed subject matter, which are described in more detail below. The encoder (203) may also generate an encoded video stream (204). The encoded video stream (204) may have a lower data volume compared to the uncompressed stream (202) and may be stored on a streaming server (205) for future use. One or more streaming clients (206) can access the streaming server (205) to retrieve a video stream (209), which may be a copy of an encoded video stream (204).

[0028] The streaming client (206) may include a video decoder (210) and a display (212). The video decoder (210) may, for example, decode a video stream (209) that is an input copy of an encoded video stream (204), and the video decoder (210) may create an output video sample stream (211) that can be presented on the display (212) or other presentation device (not shown). In some streaming systems, the video streams (204, 209) may be encoded according to certain video encoding / compression standards.

[0029] As mentioned above, dynamic mesh sequences may require a large amount of data because they can contain a great deal of information that changes over time. Subdivision-based methods can be used to compress dynamic mesh sequences, where the displacements of the vertices of the subdivided mesh are written; therefore, efficient displacement encoding is required for high-performance compression of dynamic meshes.

[0030] Subdivision schemes can be used to effectively compress dynamic meshes, in Figure 3The diagram illustrates a 2D representation where the original curve is first extracted and subdivided. The subdivided polylines are then deformed to more closely approximate the original curve. More precisely, a displacement vector is calculated for each vertex of the subdivided mesh, such that the shape of the displacement curve is as close as possible to the shape of the original curve. The main advantage of subdivided curves is their subdivision structure, which allows for efficient compression while faithfully approximating the original curve.

[0031] In some embodiments, it is assumed that the vertex displacements (displacement vectors) of the subdivided mesh have been calculated and transformed such that the output of the transformation (i.e., the coefficients) has lower entropy than the original displacements, and therefore can be compressed more effectively. For example, the amplitudes of the scaling factors (low-frequency components) of the wavelet transform are typically larger than those of the wavelet coefficients (high-frequency components). Therefore, if the coefficients are quantized and ordered sequentially from lower Level of Detail (LOD) to higher LOD, the amplitudes of the quantized coefficients will be statistically arranged in descending order, and most coefficients near the end of the sequence will be zero. Figure 4 An example of the coefficient distribution is shown. Based on these assumptions, the following method for encoding the aforementioned coefficients is proposed. In some embodiments, the above assumptions may not need to be true in order to apply the following method.

[0032] In some embodiments, the current frame is encoded using only intra-frame picture information. Existing methods, such as arithmetic coding, can be used to encode coefficients. Furthermore, if the coefficients have a tree structure, where high-frequency coefficients are descendants of low-frequency components, the tree structure, such as a zero-tree structure, can be utilized to improve coding performance.

[0033] In some embodiments, inter-frame image information is used to encode the current frame. In these embodiments, it is recommended to first predict the coefficients of the current frame, and then encode the prediction residual using the method used in intra-frame coded frames. The residual can be calculated as the difference between the original coefficients and the predicted coefficients. Specifically, in inter-frame mode, the current grid frame is traced back to the previous frame, meaning there is a one-to-one correspondence between the vertices of the current frame and the vertices of the previous frame. Therefore, the coefficients of each vertex in the current frame can be predicted using the coefficients of the corresponding vertices in a reference frame (e.g., the previous frame). Entropy coding can then be performed on the prediction residual. In some embodiments, the prediction residual is encoded; in other embodiments, the original coefficients are encoded. For each frame, a signaling flag can be used to indicate such a choice. Since the distribution of the prediction residual differs from that of the original coefficients, a separate coding context can be used to encode the prediction residual and the original coefficients to further improve coding efficiency.

[0034] In some embodiments, different encoding methods can be employed for the coefficients or coefficient residuals in the inter-frame encoded frames, depending on the distribution of the coefficients / residuals. In some embodiments, a syntax structure is proposed such that coefficients with absolute values ​​greater than 1 are identified, and their positions in the sequence and their values ​​are written in. Syntax elements indicating this purpose may include the number of these large coefficients, the distance between the previous and next large coefficient, etc. The remaining coefficients are always 0, +1, or -1. To encode such a sequence, some embodiments simply use flags to indicate each position to determine whether it is 0; if not, other symbolic flags are written to indicate whether it is a positive 1 or a negative 1.

[0035] Figure 5 This is a flowchart of an example process (500) for encoding vertex displacements in a dynamically subdivided mesh. In some implementations, Figure 5 One or more processing blocks can be executed by any of the elements discussed above.

[0036] like Figure 5 As shown, the process (500) may include: deriving a curve of a grid sequence from the frames of the video bitstream, the curve including multiple vertices (block 510).

[0037] like Figure 5 As further shown in the diagram, the process (500) may include: subdividing the curve by adding multiple subdivision vertices (block 520).

[0038] like Figure 5 As further shown in the diagram, the process (500) may include: calculating one or more displacement vectors between each of the plurality of vertices on the curve and each of the plurality of subdivision vertices on the subdivided curve (block 530).

[0039] like Figure 5 As shown, the process (500) may include: transforming the one or more displacement vectors to output one or more original coefficients (block 540).

[0040] like Figure 5 As further shown in the diagram, the process (500) may include: determining whether to encode the current frame using inter-frame picture information or intra-frame picture information (block 550).

[0041] like Figure 5 As further shown in the figure, in response to determining that the current frame is to be encoded using inter-frame picture information encoding, the process (500) may include: tracing the current frame through the previous frame to obtain a one-to-one correspondence between multiple vertices of the current frame and multiple vertices of the previous frame (block 561).

[0042] like Figure 5As further shown in the diagram, the process (500) may include: predicting one or more coefficients for each of the multiple vertices in the current frame based on the one-to-one correspondence (block 571).

[0043] like Figure 5 As further shown in the diagram, the process (500) may include: performing entropy encoding on one or more prediction residuals (block 581).

[0044] like Figure 5 As further shown in the diagram, the process (500) may include: encoding the one or more predicted residuals based on one or more predicted coefficients (block 591).

[0045] like Figure 5 As further shown in the diagram, in response to determining that the current frame should be encoded using intra-frame picture information, the process (500) may include: encoding one or more coefficients using arithmetic coding (block 562).

[0046] although Figure 5 An example block of process (500) is shown, but in some implementations, compared to Figure 5 The blocks shown in the diagram, the process (500) may include additional blocks, fewer blocks, different blocks, or blocks arranged differently. Additionally, or alternatively, two or more blocks of the process (500) may be executed in parallel.

[0047] The above techniques can be implemented as computer software that uses computer-readable instructions and is physically stored in one or more computer-readable media. For example, Figure 6 A computer system (900) suitable for implementing certain embodiments of the present disclosure is shown.

[0048] Computer software can be coded using any suitable machine code or computer language. Any suitable machine code or computer language can be assembled, compiled, linked, or similarly used to create code containing instructions that can be executed directly by the computer's central processing unit (CPU), graphics processing unit (GPU), or through interpretation, microcode execution, etc.

[0049] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0050] Figure 6The components of the computer system (900) shown are examples and are not intended to impose any limitation on the scope or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement relating to any component or combination thereof shown in the non-limiting embodiments of the computer system (900).

[0051] The computer system (900) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users, for example, through tactile input (e.g., keystrokes, swipes, movement of a data glove), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not depicted). Human-machine interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sound), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video), etc.

[0052] The input human-machine interface device may include one or more of the following devices (only one of each is depicted): keyboard (901), mouse (902), touchpad (903), touch screen (910), data glove, joystick (905), microphone (906), scanner (907), camera (908).

[0053] The computer system (900) may also include certain human-machine interface output devices. Such human-machine interface output devices may, for example, stimulate the senses of one or more human users through tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (910), data gloves, or joystick (905), but may also be tactile feedback devices not used as input devices). For example, such human-machine interface output devices can be audio output devices (e.g., speakers (909), headphones (not depicted)), visual output devices (e.g., screens including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light-emitting diode (OLED) screens (910), each screen having or not having touch screen input functionality, each screen having or not having haptic feedback functionality, and some of these screens may be able to output two-dimensional visual output or more than three-dimensional output via devices such as stereoscopic image output, virtual reality glasses (not depicted), holographic displays and smoke boxes (not depicted), and printers (not depicted).

[0054] The computer system (900) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (920) having CD / DVD or similar media (921), finger drives (922), removable hard disk drives or solid-state drives (923), conventional magnetic media such as magnetic tapes and floppy disks (not depicted), devices based on dedicated ROM / ASIC / PLD such as security dongles (not depicted), etc.

[0055] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0056] The computer system (900) may also include interfaces to one or more communication networks. These networks may be, for example, wireless networks, wired networks, or optical networks. Networks may also be local area networks, wide area networks, metropolitan area networks, vehicle and industrial networks, real-time networks, latency-tolerant networks, and so on. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including Global System for Mobile communications (GSM), 3G, 4G, 5G, Long-Term Evolution (LTE), etc., cable or wireless wide area digital television networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial networks including CAN buses, and so on. Some networks typically require external network interface adapters (e.g., USB ports on the computer system (900)) to be attached to certain general-purpose data ports or peripheral buses (949). Other network interfaces are typically integrated into the core of the computer system (900) by being attached to system buses as described below (e.g., an Ethernet interface connected to a PC computer system or a cellular network interface connected to a smartphone computer system). The computer system (900) can use any of these networks to communicate with other entities. Such communication can be one-way (receive-only, e.g., broadcast television), one-way (transmit-only, e.g., to a CANbus device), or bidirectional, e.g., using a local area network or wide area network to connect to other computer systems. This communication can include communication to a cloud computing environment (955). As mentioned above, certain protocols and protocol stacks can be used on each of those networks and network interfaces.

[0057] The aforementioned human-machine interface device, human-accessible storage device, and network interface (954) can be attached to the kernel (940) of the computer system (900).

[0058] The core (940) may include one or more central processing units (CPU) (941), graphics processing units (GPUs) (942), dedicated programmable processing units (943) in the form of field-programmable gate arrays (FPGAs), hardware accelerators (944) for certain tasks, and so on. These devices, along with read-only memory (ROM) (945), random access memory (946), and internal mass storage (947) such as internal non-user-accessible hard disk drives, SSDs, etc., can be connected via a system bus (948). In some computer systems, the system bus (948) may be accessed in the form of one or more physical plugs to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (948) or attached to the core's system bus (948) via a peripheral bus (949). Peripheral bus architectures include PCI, USB, etc. A graphics adapter (950) may be included in the core (940).

[0059] The CPU (941), GPU (942), FPGA (943), and accelerator (944) can execute certain instructions, which can be combined to form the aforementioned computer code. This computer code can be stored in ROM (945) or RAM (946). Transient data can also be stored in RAM (946), while permanent data can be stored, for example, in internal mass storage (947). Fast storage and retrieval to any storage device can be achieved using a cache, which can be closely associated with one or more CPUs (941), GPUs (942), mass storage (947), ROM (945), RAM (946), etc.

[0060] Computer-readable media may have computer code thereon for performing various computer-implemented operations. The media and computer code may be media and computer code specifically designed and constructed for the purposes of this disclosure, or the media and computer code may be of a type known and available to those skilled in the art of computer software.

[0061] By way of example and not limitation, a computer system having an architecture (900), particularly a kernel (940), can provide functionality because one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) execute software contained in one or more tangible computer-readable media. Such computer-readable media may be media associated with user-accessible mass storage as described above, and some non-transitory memory of the kernel (940), such as internal kernel mass storage (947) or ROM (945). Software implementing various embodiments of this disclosure may be stored in such devices and executed by the kernel (940). Depending on specific needs, the computer-readable media may include one or more storage devices or chips. The software may cause the kernel (940), particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (946) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system may provide functionality, through hard-wired or otherwise embodied logic in the circuitry (e.g., the accelerator (944)), which may replace or co-operate with the software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry storing software for execution (e.g., an integrated circuit (IC)), circuitry embodying logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0062] Although this disclosure has described several non-limiting embodiments, modifications, substitutions, and various equivalent alternatives that fall within the scope of this disclosure exist. Therefore, it should be understood that those skilled in the art will be able to design many systems and methods as described below: although not explicitly shown or described in this disclosure, they embody the principles of this disclosure and thus fall within its spirit and scope.

Claims

1. A method of video coding performed by at least one processor, the method comprising: The method comprises: deriving a curve of a mesh sequence from a frame of a video bitstream, the curve comprising a plurality of vertices; subdividing the curve by adding a plurality of subdivision vertices; calculating one or more displacement vectors between each of the plurality of vertices on the curve and each of the plurality of subdivision vertices on the subdivided curve; transforming the one or more displacement vectors to output one or more original coefficients; determining whether to encode a current frame using inter-picture information or intra-picture information; in response to determining to encode the current frame using inter-picture information: tracking the current frame from a previous frame to obtain a one-to-one correspondence between a plurality of vertices of the current frame and a plurality of vertices of the previous frame; predicting one or more coefficients for each of the plurality of vertices in the current frame based on the one-to-one correspondence; and encoding one or more prediction residuals based on the predicted one or more coefficients; in response to determining to encode the current frame using intra-picture information: encoding the one or more original coefficients using arithmetic coding.

2. The method of claim 1, wherein, in response to determining to encode the current frame using inter-picture information: encoding the one or more original coefficients; wherein, for each frame, a signaling flag is used to indicate whether to encode the one or more prediction residuals or to encode the one or more original coefficients.

3. The method of claim 2, wherein, The prediction residuals and the original coefficients are encoded using one or more separate coding contexts.

4. The method of claim 3, wherein, The prediction residuals or the original coefficients are encoded using a syntax structure such that, for each of the prediction residuals or each of the original coefficients, a respective position and a respective value are written in a respective location in a sequence.

5. The method of claim 4, further comprising: A flag is used to indicate each of the respective positions.

6. The method of claim 1, wherein, The subdividing the curve comprises forming a decimated curve by decimating the curve, and adding the plurality of subdivision vertices to the decimated curve.

7. An apparatus for video coding, the apparatus comprising: Comprise: at least one memory configured to store program code; and at least one processor configured to read the program code and perform actions in accordance with instructions of the program code, the program code comprising: deriving code configured to cause the at least one processor to derive a curve of a mesh sequence from a frame of a video bitstream, the curve comprising a plurality of vertices; subdividing code configured to cause the at least one processor to subdivide the curve by adding a plurality of subdivision vertices; calculating code configured to cause the at least one processor to calculate one or more displacement vectors between each of the plurality of vertices on the curve and each of the plurality of subdivision vertices on the subdivided curve; transforming code configured to cause the at least one processor to transform the one or more displacement vectors to output one or more original coefficients; determining code configured to cause the at least one processor to determine whether to encode a current frame using inter-picture information or intra-picture information; first response code configured to, in response to determining to encode the current frame using inter-picture information, cause the at least one processor to perform the following operations: obtaining a one-to-one correspondence between vertices of the current frame and vertices of the previous frame by tracing the current frame from the previous frame; predicting one or more coefficients for each of the vertices in the current frame based on the one-to-one correspondence; and encoding one or more prediction residuals based on the predicted one or more coefficients. a second response code configured to cause the at least one processor to, in response to determining to encode the current frame using intra-picture information, perform the following operation: encode the one or more original coefficients using arithmetic coding.

8. The apparatus of claim 7, wherein, The first response code is further configured to cause the at least one processor to, in response to determining to encode the current frame using inter-picture information, perform the following operation: encode the one or more original coefficients; wherein, for each frame, a signaling flag is used to indicate whether to encode the one or more prediction residuals or to encode the one or more original coefficients.

9. The apparatus of claim 8, wherein, encode the prediction residuals and the original coefficients using one or more separate coding contexts.

10. The apparatus of claim 9, wherein, encode the prediction residuals or the original coefficients using a syntax structure such that, for each of the prediction residuals or each of the original coefficients, a respective position and a respective value in a sequence are written.

11. The apparatus of claim 10, wherein, The program code further includes an indication code configured to cause the at least one processor to use a flag to indicate each of the respective positions.

12. The apparatus of claim 7, wherein, The subdividing the curve includes forming a decimated curve by decimating the curve, and adding the plurality of subdivided vertices to the decimated curve.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 6.

Images