SEI MESSAGE FOR GENERATIVE FACIAL VIDEO

Description

1 / 147 SEI MESSAGE FOR GENERATIVE FACIAL VIDEO CROSS-REFERENCE TO RELATED ORDERS

[0001] The disclosure claims priority benefits for the U.S. Provisional Application No. 63 / 494,493, filed April 6, 2023, U.S. Provisional Application No. 63 / 511,200, filed June 30, 2023, U.S. Provisional Application No. 63 / 587,763, filed October 4, 2023, U.S. Provisional Application No. 63 / 618,387, filed January 8, 2024, and U.S. Provisional Application No. 18 / 622,621, filed March 29, 2024, the contents of which are incorporated by reference herein in their entirety. TECHNICAL FIELD

[0002] This disclosure generally relates to video processing and, more particularly, to methods and devices for using supplemental enhancement information (SEI) messages to perform generative facial video compression. BACKGROUND

[0003] A video is a collection of still images (or “frames”) that capture visual information. To reduce storage memory and transmission bandwidth, a video can be compressed before storage or transmission, and decompressed before display. The compression process is generally referred to as encoding, and the decompression process is generally referred to as decoding. There are several video encoding formats that use standardized video encoding technologies, generally based on prediction, transformation, quantization, entropy coding, and loop filtering. Video encoding standards, such as the High Efficiency Video Coding Standard (HEVC / H.265), Versatile Video Coding Standard (VVC / H.266), and AVS standards, specify specific encoding formats. Petition 870250090349, dated 03 / 10 / 2025, page 9 / 193 2 / 147 of videos are developed by standardization organizations. With increasingly advanced video encoding technologies being adopted in video standards, the encoding efficiency of the new video encoding standards is becoming ever greater. SUMMARY OF THE DISCLOSURE

[0004] The embodiments of the present disclosure provide methods and apparatus for processing video data using generative facial video supplemental enhancement information (SEI) messages.

[0005] According to some exemplary embodiments, a method is provided for generating a facial image, the method includes: receiving a bitstream; decoding the encoded information from the bitstream to obtain a base image and a supplementary enhancement information (SEI) message; determining whether the SEI message applies to a neural network to generate a facial image; in response to the SEI message applied to the neural network to generate the facial image, determining a mode and a corresponding facial information parameter used to encode the facial image based on the SEI message; and generating the facial image based on the base image and the facial information parameter by the neural network.

[0006] According to some exemplary embodiments, a method is provided for encoding a sequence of videos into a bitstream, the method including: Receiving a sequence of videos; and encoding one or more images from the sequence of videos to generate a bitstream, including: encoding a base image from one or more images and a supplementary enhancement information (SEI) message, the SEI message indicating a mode and a corresponding facial information parameter used to encode a facial image, and wherein the bitstream is used to generate the facial image by a network Petition 870250090349, dated 03 / 10 / 2025, page 10 / 193 3 / 147 neural based on the baseline image and facial information parameter.

[0007] According to some exemplary embodiments, a non-transient computer-readable storage medium is provided that stores a bitstream of a video. The bitstream includes: a base image and a supplementary enhancement information (SEI) message, the SEI message indicating a mode and a corresponding facial information parameter used to encode a facial image, and wherein the bitstream is used to generate the facial image by a neural network based on the base image and the facial information parameter. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The features and various aspects of this disclosure are illustrated in the following detailed description and in the accompanying figures. Several features in the figures are not shown to scale.

[0009] FIG. 1 is a schematic diagram illustrating an exemplary system for encoding image data, according to some embodiments of the present disclosure.

[0010] FIG. 2A is a schematic diagram illustrating an exemplary encoding process of a hybrid video encoding system, consistent with the modalities of dissemination.

[0011] FIG. 2B is a schematic diagram illustrating another coding process exemplifying a hybrid video coding system, consistent with the modalities of dissemination.

[0012] FIG. 3A is a schematic diagram illustrating an exemplary decoding process of a hybrid video encoding system, consistent with the modalities of dissemination.

[0013] FIG. 3B is a schematic diagram illustrating another exemplary decoding process of a hybrid video coding system, consistent with the modalities of dissemination. Petition 870250090349, dated 03 / 10 / 2025, page 11 / 193 4 / 147

[0014] FIG. 4 is a block diagram of an exemplary device for encoding image data, according to some embodiments of the present disclosure.

[0015] FIG. 5 is a flowchart of an exemplary method for processing video based on generative facial video SEI messages, according to some embodiments of the present disclosure.

[0016] FIG. 6 is another flowchart of an exemplary method for processing SEI-based video of generative facial video, according to some embodiments of the present disclosure.

[0017] FIG. 7A is a flowchart of an exemplary method for generating a facial image, according to some embodiments of the present disclosure.

[0018] FIG. 7B is a schematic diagram illustrating a neural network for generating a facial image, according to some embodiments of the present disclosure.

[0019] FIG. 8 is a schematic diagram illustrating an exemplary SEI message, according to some embodiments of the present disclosure.

[0020] FIG. 9 is a schematic diagram illustrating another exemplary SEI message, according to some modalities of the present disclosure.

[0021] FIG. 10 is a schematic diagram illustrating another exemplary SEI message, according to some modalities of the present disclosure.

[0022] FIG. 11 is a schematic diagram illustrating another exemplary SEI message, according to some modalities of the present disclosure.

[0023] FIG. 12 is a schematic diagram illustrating another exemplary SEI message, according to some modalities of the present disclosure. Petition 870250090349, dated 03 / 10 / 2025, p. 12 / 193 5 / 147

[0024] FIG. 13 is a flowchart of an exemplary method for encoding a sequence of videos into a bitstream, according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0025] Detailed reference will now be made to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent identical or similar elements, except where otherwise represented. The implementations defined in the following description of the exemplary embodiments do not represent all implementations consistent with the invention. Instead, they are only examples of apparatus and methods consistent with aspects related to the invention as cited in the accompanying claims. The specific aspects of this disclosure are described in more detail below. The terms and definitions provided herein shall prevail if they conflict with terms and / or definitions incorporated by reference.

[0026] The Joint Video Experts Team (JVET) of the ITU-T Video Coding Expert Group (ITU-T VCEG) and the ISO / IEC Moving Picture Expert Group (ISO / IEC MPEG) is currently developing the Versatile Video Coding (VVC / H.266) standard. The VVC standard aims to double the compression efficiency of its predecessor, the High Efficiency Video Coding (HEVC / H.265) standard. In other words, the goal of the VVC standard is to achieve the same subjective quality as the HEVC / H.265 standard using half the bandwidth. The VVC standard has progressed well since April 2018 and continues to incorporate more coding technologies that provide better compression performance. VVC is based on the same hybrid video coding system that has been used in modern video compression standards such as HEVC, H.264 / AVC, MPEG2, H.263, Petition 870250090349, dated 03 / 10 / 2025, page 13 / 193 6 / 147 etc.

[0027] Neural network post-processing filtering has been adopted in image processing. Some existing techniques use supplementary enhancement information (SEI) messages to specify post-processing filter characteristics for neural network post-processing filtering. However, temporal oversampling in computer vision is not considered when generating SEI messages. However, there is a need to perform temporal oversampling for computer vision based on SEI messages.

[0028] FIG. 1 is a block diagram illustrating a system 100 for preprocessing and encoding image data according to some disclosed modalities. Image data may include a single image (also called a “frame”), multiple images, or a video. A single image is a static image. Multiple images may be related or unrelated, either spatially or temporally. A video is a collection of images arranged in a temporal sequence.

[0029] As shown in FIG. 1, the system 100 includes a source device 120 that provides encoded video data to be subsequently decoded by a destination device 140. Consistent with the disclosed embodiments, each source device 120 and destination device 140 may include any wide variety of devices, including a desktop computer, a notebook computer (e.g., laptop), a server, a handheld computer, a TV box, a mobile phone, a vehicle, a camera, an image sensor, a robot, a television, a camera, a wearable device (e.g., a smartwatch or a wearable camera), a display device, a digital media player, a video game console, a video transmission device, or similar. The source device 120 and the destination device 140 may be equipped to Petition 870250090349, dated 03 / 10 / 2025, page 14 / 193 7 / 147 Wireless or wired communication.

[0030] With reference to FIG. 1. The source device 120 may include an image / video preprocessor 122, an image / video encoder 124, and an output interface 126. The destination device 140 may include an input interface 142, an image / video decoder 144, and one or more computer vision applications 146. The image / video preprocessor 122 preprocesses the image data, i.e., image(s) or video(s), and generates an input bitstream for the image / video encoder 124. The image / video encoder 124 encodes the input bitstream and sends an encoded bitstream 162 through the output interface 126. The encoded bitstream 162 is transmitted through a communication medium 160 and received by the input interface 142. The image / video decoder 144 then decodes the encoded bitstream. 162 to generate decoded data, which can be used by computer vision applications 146.

[0031] More specifically, the source device 120 may further include several devices (not shown) for providing source image data to be pre-processed by the image / video preprocessor 122. The devices for providing the source image data may include an image / video capture device, such as a camera, an image / video file or storage device containing previously captured images / videos, or an image / video feed interface for receiving images / videos from an image / video content provider.

[0032] The 124-image / video encoder and the 144-image / video decoder can be implemented as any variety of encoder and decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), circuits in Petition 870250090349, dated 03 / 10 / 2025, page 15 / 193 8 / 147 Application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When encoding or decoding is partially implemented in software, the 124-image / video encoder or 144-image / video decoder may store instructions for software on a suitable non-transient computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques consistent with the present disclosure. Each 124-image / video encoder or 144-image / video decoder may be included in one or more encoders or decoders, which may be integrated as part of a combined encoder / decoder (CODEC) in a respective device.

[0033] The 124-image / video encoder and the 144-image / video decoder can operate according to any video coding standard, such as Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), AOMedia Video 1 (AV1), Joint Photographic Experts Group (JPEG), Moving Picture Experts Group (MPEG), etc. Alternatively, the 124-image / video encoder and the 144-image / video decoder can be customized devices that do not comply with existing standards. Although not shown in FIG. 1. In some embodiments, the 124-image / video encoder and the 144-image / video decoder may be integrated with an audio encoder and decoder, and may include suitable MUX-DEMUX units, or other hardware and software, to handle audio and video encoding in a common data stream or a separate data stream.

[0034] The 126 output interface can include any type of media. Petition 870250090349, dated 03 / 10 / 2025, page 16 / 193 9 / 147 or device capable of transmitting 162-encoded bitstream from source device 120 to destination device 140. For example, the output interface 126 may include a transmitter or transceiver configured to transmit 162-encoded bitstream from source device 120 directly to destination device 140 in real time. The 162-encoded bitstream may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to destination device 140.

[0035] The communication medium 160 may include transient media, such as a wireless transmission or wired network transmission. For example, the communication medium 160 may include radio frequency (RF) spectrum or one or more physical transmission lines (e.g., a cable). The communication medium 160 may be part of a packet-based network, such as a local area network, a wide area network, or a global network, such as the Internet. In some embodiments, the communication medium 160 may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from the source device 120 to the destination device 140. For example, a network server (not shown) may receive encoded bitstream 162 from the source device 120 and provide encoded bitstream 162 to the destination device 140, for example, via network transmission.

[0036] The media 160 may also be in the form of a storage medium (e.g., non-transient storage medium), such as a hard disk, flash drive, compact disc, digital video disc, Blu-ray disc, volatile or non-volatile memory, or any other digital storage medium suitable for storing encoded image data. In some embodiments, a computing device of a media production facility, such as a disc stamping facility, may Petition 870250090349, dated 03 / 10 / 2025, page 17 / 193 10 / 147 receive encoded image data from source device 120 and produce a disc containing the encoded video data.

[0037] Input interface 142 may include any type of medium or device capable of receiving information from communication medium 160. The information received includes encoded bitstream 162. For example, input interface 142 may include a receiver or a transceiver configured to receive encoded bitstream 162 in real time.

[0038] Computer vision applications 146 include various types of hardware and / or software to utilize the decoded image data generated by the image / video decoder 144. For example, computer vision applications 146 may include a display device that displays the decoded image data to a user and may include any variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.As another example, computer vision applications 146 may include one or more processors configured to use the decoded image data to perform various computer vision applications, such as object recognition and tracking, facial recognition, image matching, image / video search, augmented reality, robotic vision and navigation, autonomous driving, three-dimensional structure construction, stereo matching, motion tracking, etc.

[0039] Next, exemplary techniques for encoding and decoding image data (such as those implemented by encoder 124 and decoder 144 in FIG. 1) are described in connection with FIGS. 2A-2B and FIGS. 3A-3B.

[0040] FIG. 2A illustrates a schematic diagram of a process Petition 870250090349, dated 03 / 10 / 2025, page 18 / 193 11 / 147 exemplary coding 200A, consistent with the embodiments of the disclosure. For example, the 200A encoding process can be performed by an encoder, such as an image / video encoder 124 in FIG. 1. As shown in FIG. 2A, the encoder can encode a sequence of videos 202 into a video bitstream 228 according to the 200A process. The sequence of videos 202 may include a set of images (referred to as “original images”) arranged in a temporal order. Each original image of the sequence of videos 202 can be divided by the encoder into basic processing units, basic processing subunits, or regions for processing. In some embodiments, the encoder can perform the 200A process at the level of basic processing units for each individual image of the sequence of videos 202.For example, the encoder can execute process 200A iteratively, where the encoder can encode a basic processing unit in one iteration of process 200A. In some embodiments, the encoder can execute process 200A in parallel for regions of each original image in the 202 video sequence.

[0041] In FIG. 2A, the encoder can feed a basic processing unit (referred to as an “original BPU”) of an original video sequence image 202 to prediction stage 204 to generate prediction data 206 and predicted BPU 208. The encoder can subtract the predicted BPU 208 from the original BPU to obtain the residual BPU 210. The encoder can feed the residual BPU 210 to transformation stage 212 and quantization stage 214 to generate quantized transformation coefficients 216. The encoder can feed prediction data 206 and quantized transformation coefficients 216 to binary encoding stages 226 to generate video bitstream 228. The components 202, 204, 206, 208, 210, 212, 214, 216, 226 and 228 can be referred to as a "route of Petition 870250090349, dated 03 / 10 / 2025, page 19 / 193 12 / 147 advance”. During process 200A, after quantization stage 214, the encoder can feed in quantized transformation coefficients 216 and inverse quantization stage 218 and inverse transformation stage 220 to generate reconstructed residual BPU 222. The encoder can add reconstructed residual BPU 222 to predicted BPU 208 to generate prediction reference 224, which is used in prediction stage 204 for the next iteration of process 200A. Components 218, 220, 222, and 224 of process 200A can be referred to as a “reconstruction path”. The reconstruction path can be used to ensure that the encoder and decoder use the same reference data for prediction.

[0042] The encoder can execute process 200A iteratively to encode each original BPU of the original image (in the forward path) and generate the predicted reference 224 to encode the next original BPU of the original image (in the reconstruction path). After encoding the original BPUs of the original image, the encoder can proceed to encode the next image in the video sequence 202.

[0043] Referring to process 200A, the encoder may receive the video sequence 202 generated by a video capture device (e.g., a camera). The term “receive” as used in this document may refer to receiving, inserting, acquiring, retrieving, obtaining, reading, accessing, or any action in any way to insert data.

[0044] In the prediction stage 204, in a current iteration, the encoder may receive an original BPU and prediction reference 224, and perform a prediction operation to generate prediction data 206 and predicted BPU 208. The prediction reference 224 may be generated from the reconstruction path of the previous iteration of process 200A. The purpose of the prediction stage 204 is to reduce information redundancy by extracting the prediction data 206 which can be used to reconstruct the original BPU as predicted BPU 208 from Petition 870250090349, dated 03 / 10 / 2025, page 20 / 193 13 / 147 of the prediction data 206 and prediction reference 224.

[0045] Ideally, the predicted BPU 208 can be identical to the original BPU. However, due to non-ideal prediction and reconstruction operations, the predicted BPU 208 is generally slightly different from the original BPU. To account for such differences, after generating the predicted BPU 208, the encoder can subtract it from the original BPU to generate the residual BPU 210. For example, the encoder can subtract values ​​(e.g., grayscale values ​​or RGB values) from pixels of predicted BPU 208 from corresponding pixel values ​​of the original BPU. Each pixel of residual BPU 210 may have a residual value as a result of such subtraction between the corresponding pixels of the original BPU and the predicted BPU 208. Compared to the original BPU, the prediction data 206 and the residual BPU 210 may have fewer bits, but they can be used to reconstruct the original BPU without significant quality deterioration. Thus, the original BPU is compressed.

[0046] To further compress the residual BPU 210, in the transformation stage 212, the encoder can reduce the spatial redundancy of the residual BPU 210 by decomposing it into a set of two-dimensional “basis patterns,” each basis pattern being associated with a “transformation coefficient.” The basis patterns can be the same size (e.g., the size of the residual BPU 210). Each basis pattern can represent a frequency variation component (e.g., brightness variation frequency) of the residual BPU 210. None of the basis patterns can be reproduced from any combinations (e.g., linear combinations) of any other basis patterns. In other words, the decomposition can decompose variations of the residual BPU 210 in the frequency domain. Such decomposition is analogous to a discrete Fourier transform of a function, where the basis patterns are analogous. Petition 870250090349, dated 03 / 10 / 2025, page 21 / 193 14 / 147 to the basis functions (e.g., trigonometric functions) of the discrete Fourier transform, and the transformation coefficients are analogous to the coefficients associated with the basis functions.

[0047] Different transformation algorithms may use different basis patterns. Several transformation algorithms can be used in the 212 transformation stage, such as, for example, a discrete cosine transformation, a discrete sine transformation, or similar. The transformation in the 212 transformation stage is reversible. That is, the encoder can restore the residual BPU 210 by a reverse transformation operation (referred to as an “inverse transformation”). For example, to restore a pixel from residual BPU 210, the inverse transformation can multiply the corresponding pixel values ​​of the basis patterns by their respective associated coefficients and add the products to produce a weighted sum. For a video encoding pattern, both the encoder and the decoder can use the same transformation algorithm (thus, the same basis patterns).Thus, the encoder can only record the transformation coefficients, by which the decoder can reconstruct the residual 210 BPU without receiving the base patterns from the encoder. Compared to the residual 210 BPU, the transformation coefficients may have fewer bits, but they can be used to reconstruct the residual 210 BPU without significant quality deterioration. Thus, the residual 210 BPU is additionally compressed.

[0048] The encoder can further compress the transformation coefficients at quantization stage 214. In the transformation process, different basis patterns can represent different frequencies of variation (e.g., frequencies of brightness variation). Since human eyes are generally better at Petition 870250090349, dated 03 / 10 / 2025, page 22 / 193 15 / 147 recognizing low-frequency variation, the encoder can disregard high-frequency variation information without causing significant quality deterioration in the decoding. For example, at quantization stage 214, the encoder can generate quantized transformation coefficients 216 by dividing each transformation coefficient by an integer value (referred to as a “quantization parameter”) and rounding the quotient to its nearest integer. After such an operation, some transformation coefficients of the high-frequency base patterns can be converted to zero, and the transformation coefficients of the low-frequency base patterns can be converted to smaller integers. The encoder can disregard the zero-value quantized transformation coefficients 216, whereby the transformation coefficients are further compressed.The quantization process is also reversible, in which the quantized transformation coefficients 216 can be reconstructed into the transformation coefficients in an inverse quantization operation (referred to as "inverse quantization").

[0049] Since the encoder disregards the remainder of such divisions in the rounding operation, the quantization stage 214 may contain data loss. Typically, the quantization stage 214 may contribute to the greatest loss of information in process 200A. The greater the loss of information, the fewer bits of the quantized transformation coefficients 216 are needed. To obtain different levels of loss of information, the encoder may use different values ​​of the quantization parameter or any other parameter of the quantization process.

[0050] In the binary encoding stage 226, the encoder can encode the prediction data 206 and quantized transformation coefficients 216 using a binary encoding technique, such as, Petition 870250090349, dated 03 / 10 / 2025, page 23 / 193 16 / 147 for example, entropy coding, variable-length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless or lossy data compression algorithm. In some embodiments, in addition to the prediction data 206 and quantized transformation coefficients 216, the encoder may encode other information in binary coding stage 226, such as, for example, a prediction mode used in prediction stage 204, prediction operation parameters, a transformation type in transformation stage 212, quantization process parameters (e.g., quantization parameters), an encoder control parameter (e.g., a bitrate control parameter), or the like. The encoder may use the output data from binary coding stage 226 to generate video bitstream 228.In some modes, the 228-bit video stream can be further packaged for network transmission.

[0051] Referring to the process reconstruction path 200A, at inverse quantization stage 218, the encoder can perform inverse quantization on the quantized transformation coefficients 216 to generate reconstructed transformation coefficients. At inverse transformation stage 220, the encoder can generate reconstructed residual BPU 222 based on the reconstructed transformation coefficients. The encoder can add reconstructed residual BPU 222 to the predicted BPU 208 to generate prediction reference 224, which is used in the next process iteration 200A.

[0052] It should be noted that other variations of process 200A can be used to encode the video sequence 202. In some embodiments, the stages of process 200A can be performed by the encoder in different orders. In some embodiments, one or more stages of process 200A can be combined into one Petition 870250090349, dated 03 / 10 / 2025, page 24 / 193 17 / 147 single stage. In some embodiments, a single stage of process 200A may be divided into multiple stages. For example, transformation stage 212 and quantization stage 214 may be combined into a single stage. In some embodiments, process 200A may include additional stages. In some embodiments, process 200A may omit one or more stages in FIG. 2A.

[0053] FIG. 2B illustrates a schematic diagram of another exemplary coding process 200B, consistent with the embodiments of the disclosure. For example, coding process 200B may be performed by an encoder, such as an image / video encoder 124 in FIG. 1. Process 200B may be modified from process 200A. For example, the 200B process can be used by an encoder conforming to a hybrid video encoding standard (e.g., H.26x series).Compared to process 200A, the forward path of process 200B additionally includes a mode decision stage 230 and divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044. The reconstruction path of process 200B additionally includes a circuit filter stage 232 and a buffer stage 234.

[0054] Generally, prediction techniques can be categorized into two types: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-image prediction or “intraprediction”) can use pixels from one or more surrounding BPUs already encoded in the same image to predict the current BPU. That is, the 224 prediction reference in spatial prediction can include the surrounding BPUs. Spatial prediction can reduce the inherent spatial redundancy of the image. Temporal prediction (e.g., inter-image prediction or “interprediction”) can use regions from one or more already encoded images to predict the current BPU. That is, the 224 prediction reference in temporal prediction can include the encoded images. A Petition 870250090349, dated 03 / 10 / 2025, page 25 / 193 18 / 147 temporal prediction can reduce the inherent temporal redundancy of images.

[0055] Referring to process 200B, in the forward path, the encoder performs the prediction operation in the spatial prediction stage 2042 and in the temporal prediction stage 2044. For example, in the spatial prediction stage 2042, the encoder may perform intraprediction. For an original BPU of an image being encoded, the prediction reference 224 may include one or more surrounding BPUs that were encoded (in the forward path) and reconstructed (in the reconstructed path) in the same image. The encoder may generate predicted BPU 208 by extrapolating the surrounding BPUs. The extrapolation technique may include, for example, linear extrapolation or interpolation, polynomial extrapolation or interpolation, or similar. In some embodiments, the encoder may perform pixel-level extrapolation, such as extrapolating corresponding pixel values ​​for each pixel of the predicted BPU 208.The surrounding BPUs used for extrapolation can be located relative to the original BPU from various directions, such as in a vertical direction (e.g., at the top of the original BPU), a horizontal direction (e.g., to the left of the original BPU), a diagonal direction (e.g., below and to the left, below and to the right, above and to the left, or above and to the right of the original BPU), or any direction defined in the video encoding standard used. For intraprediction, the 206 prediction data may include, for example, locations (e.g., coordinates) of the surrounding BPUs used, sizes of the surrounding BPUs used, extrapolation parameters, a direction of the surrounding BPUs used relative to the original BPU, or similar.

[0056] For another example, in the 2044 time prediction stage, the encoder can perform interprediction. For an original BPU of Petition 870250090349, dated 03 / 10 / 2025, page 26 / 193 19 / 147 In a current image, the prediction reference 224 may include one or more images (referred to as “reference images”) that have been encoded (in the forward path) and reconstructed (in the reconstructed path). In some modalities, a reference image may be encoded and the BPU reconstructed by the BPU. For example, the encoder may add reconstructed residual BPU 222 to the predicted BPU 208 to generate a reconstructed BPU. When all reconstructed BPUs from the same image are generated, the encoder may generate a reconstructed image as a reference image. The encoder may perform a “motion estimation” operation to search for a matching region within a scope (referred to as a “search window”) of the reference image. The location of the search window in the reference image may be determined based on the location of the original BPU in the current image.For example, the search window can be centered on a location with the same coordinates in the reference image as the original BPU in the current image and can be extended to a predetermined distance. When the encoder identifies (e.g., using a pel-recursive algorithm, a block-matching algorithm, or similar) a region similar to the original BPU in the search window, the encoder can designate such a region as the matching region. The matching region can have different dimensions (e.g., be smaller than, equal to, larger than, or in a different shape) than the original BPU. Since the reference image and the current image are temporally separated in the timeline, the matching region can be considered to “move” toward the location of the original BPU as time passes. The encoder can record the direction and distance of such movement as a “motion vector”.When multiple reference images are used, the encoder can search for a matching region and determine its motion vector. Petition 870250090349, dated 03 / 10 / 2025, page 27 / 193 20 / 147 associated for each reference image. In some modes, the encoder can assign weights to the pixel values ​​of the matching regions of the respective corresponding reference images.

[0057] Motion estimation can be used to identify various types of motion, such as, for example, translations, rotations, zooms, or the like. For interprediction, the prediction data 206 may include, for example, locations (e.g., coordinates) of the matching region, motion vectors associated with the matching region, the number of reference images, weights associated with the reference images, or the like.

[0058] To generate the predicted BPU 208, the encoder can perform a “motion compensation” operation. Motion compensation can be used to reconstruct the predicted BPU 208 based on prediction data 206 (e.g., the motion vector) and the prediction reference 224. For example, the encoder can move the matching region of the reference image according to the motion vector, where the encoder can predict the original BPU of the current image. When multiple reference images are used, the encoder can move the matching regions of the reference images according to the respective motion vectors and average pixel values ​​of the matching regions.In some modes, if the encoder has weights assigned to the pixel values ​​of the matching regions of the respective corresponding reference images, the encoder may add a weighted sum of pixel values ​​from the moved matching regions.

[0059] In some modalities, interprediction can be unidirectional or bidirectional. Unidirectional interpredictions can use Petition 870250090349, dated 03 / 10 / 2025, p. 28 / 193 21 / 147 one or more reference images in the same temporal direction with respect to the current image. Unidirectional interpredictions use a reference image that precedes the current image. Bidirectional interpredictions can use one or more reference images in both temporal directions with respect to the current image.

[0060] Still referring to the advancement path of process 200B, after the spatial prediction stage 2042 and temporal prediction stage 2044, in the mode decision stage 230, the encoder can select a prediction mode (e.g., one from intraprediction or interprediction) for the current iteration of process 200B. For example, the encoder can perform a rate distortion optimization technique, in which the encoder can select a prediction mode to minimize a value of a cost function depending on the bit rate of a candidate prediction mode and distortion of the reconstructed reference image under the candidate prediction mode. Depending on the prediction mode selected, the encoder can generate the predicted BPU 208 and predicted data 206.

[0061] In the reconstruction path of process 200B, if the intraprediction mode was selected in the forward path, after generating prediction reference 224 (e.g., the current BPU that was encoded and reconstructed in the current image), the encoder can directly feed prediction reference 224 to the spatial prediction stage 2042 for later use (e.g., for extrapolation of a next BPU from the current image). If the interprediction mode was selected in the forward path, after generating prediction reference 224 (e.g., the current image in which all BPUs were encoded and reconstructed), the encoder can feed prediction reference 224 to the circuit filter stage 232, where the encoder can apply a circuit filter to prediction reference 224 to reduce or eliminate distortion (e.g., blocking artifacts). Petition 870250090349, dated 03 / 10 / 2025, page 29 / 193 22 / 147 introduced by interprediction. The encoder can apply various circuit filter techniques in the circuit filter stage 232, such as, for example, unblocking, adaptive sample compensation, adaptive circuit filters, or the like. The circuit-filtered reference image can be stored in a buffer 234 (or “decoded image buffer”) for later use (e.g., to be used as an interprediction reference image for a future video sequence image 202). The encoder can store one or more reference images in a buffer 234 to be used in the temporal prediction stage 2044. In some embodiments, the encoder can encode circuit filter parameters (e.g., a circuit filter intensity) in the binary encoding stage 226, along with quantized transformation coefficients 216, prediction data 206, and other information.

[0062] In some modes, the input video sequence 202 is processed block by block according to the coding process 200B. In VVC, a coded tree unit (CTU) is the largest block unit, and can be as large as 128x128 luma samples (plus the corresponding chroma samples depending on the chroma format). A CTU can be further partitioned into coding units (CUs) using quaternary tree, binary tree, or ternary tree. At the leaf nodes of the partitioning structure, coding information, such as coding mode (intra mode and inter mode), motion information (reference index, motion vector difference, etc.), if intercoded, and quantized transformation coefficients 216, are sent. If intraprediction (also called spatial prediction) is used, spatial surrounding samples are used to predict the current block.If interprediction (also called time prediction or motion-compensated prediction) is used, then... Petition 870250090349, dated 03 / 10 / 2025, page 30 / 193 23 / 147 samples of previously encoded images, called reference images, are used to predict the current block. Interprediction can use uniprediction or biprediction. In uniprediction, only one motion vector pointing to a reference image is used to generate the prediction signal for the current block; and in biprediction, two motion vectors, each pointing to its own reference image, are used to generate the prediction signal for the current block. The motion vectors and reference indices are sent to the decoder to identify the origin of the prediction signal(s) for the current block. After intra- or interprediction, the mode decision stage 230 decides the best prediction mode for the current block, for example, based on the rate distortion optimization method. Based on the best prediction mode, the predicted BPU 208 is generated and subtracted from the input video block.

[0063] Still referring to FIG. 2B, the residual prediction BPU 210 is sent to the transformation stage 212 and quantization stage 214 to generate quantized transformation coefficients 216. The quantized transformation coefficients 216 will then be inversely quantized in the inverse quantization stage 218 and inversely transformed in the inverse transformation stage 220 to obtain the reconstructed residual BPU 222. The predicted BPU 208 and the reconstructed residual BPU 222 are added together to form the prediction reference 224 before in-circuit filtering, which is used to provide reference samples for intraprediction.Circuit filtering, such as unblocking, adaptive sample compensation (SAO), and adaptive circuit filtering (ALF), can be applied in the circuit filter stage 232 in the prediction reference 224 to form the reconstructed block, which is stored in the buffer 234 and used to provide reference samples for interprediction. The encoding information, which is generated in the mode decision stage 230,... Petition 870250090349, dated 03 / 10 / 2025, page 31 / 193 24 / 147, such as the encoding mode (intra or interprediction), intraprediction mode, motion information, quantized residual coefficients and the like, is sent to the binary encoding stage 226 to further reduce the bit rate before being packaged into the output video bitstream 228.

[0064] FIG. 3A illustrates a schematic diagram of an exemplary decoding process 300A, consistent with the embodiments of the disclosure. For example, the decoding process 300A can be performed by a decoder, such as an image / video decoder 144 in FIG. 1. The process 300A can be a decompression process corresponding to the compression process 200A in FIG. 2A. In some embodiments, the process 300A can be similar to the reconstruction path of the process 200A. A decoder (e.g., image / video decoder 144 in FIG. 1) can decode the video bitstream 228 into video transmission 304 according to the process 300A. Video transmission 304 can be very similar to video sequence 202. However, due to information loss in the compression and decompression process (e.g., quantization stage 214 in FIGS. 2A-2B), video transmission 304 is generally not identical to video sequence 202.Similar to processes 200A and 200B in FIGS. 2A-2B, the decoder can execute process 300A at the basic processing unit (BPU) level for each image encoded in the 228-bit video stream. For example, the decoder can execute process 300A iteratively, where the decoder can decode one basic processing unit in one iteration of process 300A. In some embodiments, the decoder can execute process 300A in parallel for regions of each image encoded in the 228-bit video stream.

[0065] In FIG. 3A, the decoder can feed a portion of Petition 870250090349, dated 03 / 10 / 2025, page 32 / 193 25 / 147 video bitstream 228 associated with a basic processing unit (referred to as a “coded BPU”) of an image encoded in binary decoding stage 302. In binary decoding stage 302, the decoder can decode the portion into prediction data 206 and quantized transformation coefficients 216. The decoder can feed the quantized transformation coefficients 216 into inverse quantization stage 218 and inverse transformation stage 220 to generate reconstructed residual BPU 222. The decoder can feed prediction data 206 into prediction stage 204 to generate predicted BPU 208. The decoder can add reconstructed residual BPU 222 to predicted BPU 208 to generate predicted reference 224. In some embodiments, the predicted reference 224 can be stored in a buffer (e.g., a decoded image buffer in computer memory).The decoder can feed predicted reference 224 into prediction stage 204 to perform a prediction operation in the next process iteration 300A.

[0066] The decoder can execute process 300A iteratively to decode each encoded BPU of the encoded image and generate predicted reference 224 to encode the next encoded BPU of the encoded image. After decoding all encoded BPUs of the encoded image, the decoder can send the image to the video transmission 304 to display and proceed with decoding on the next encoded image in the video bitstream 228.

[0067] In the binary decoding stage 302, the decoder can perform an inverse operation of the binary encoding technique used by the encoder (e.g., entropy encoding, variable-length encoding, arithmetic encoding, Huffman encoding, context-adaptive binary arithmetic encoding, or any other lossless data compression algorithm). In Petition 870250090349, dated 03 / 10 / 2025, page 33 / 193 26 / 147 In some embodiments, in addition to the prediction data 206 and quantized transformation coefficients 216, the decoder may decode other information in the binary encoding stage 302, such as, for example, a prediction mode, prediction operation parameters, a transformation type, quantization process parameters (e.g., quantization parameters), an encoder control parameter (e.g., a bit rate control parameter), or the like. In some embodiments, if the video bitstream 228 is transmitted over a network in packets, the decoder may unpack the video bitstream 228 before feeding it into the binary decoding stage 302.

[0068] FIG. 3B illustrates a schematic diagram of another exemplary decoding process 300B, consistent with the disclosure embodiments. For example, the 300B decoding process can be performed by a decoder, such as an image / video decoder 144 in FIG. 1. The 300B process can be modified from the 300A process. For example, the 300B process can be used by a decoder that conforms to a hybrid video coding standard (e.g., H.26x series). Compared with the 300A process, the 300B process further divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and additionally includes the circuit filter stage 232 and the buffer 234.

[0069] In process 300B, for a basic encoded processing unit (referred to as a “current BPU”) of an encoded image (referred to as a “current image”) being decoded, the prediction data 206 decoded by the binary decoding stage 302 by the decoder may include various data types, depending on the prediction mode used to encode the current BPU by the encoder. For example, if intraprediction was used Petition 870250090349, dated 03 / 10 / 2025, page 34 / 193 27 / 147 by the encoder to encode the current BPU, the prediction data 206 may include a prediction mode indicator (e.g., an indicative value) of the intraprediction, parameters of the intraprediction operation, or similar. The parameters of the intraprediction operation may include, for example, locations (e.g., coordinates) of one or more surrounding BPUs used as a reference, sizes of the surrounding BPUs, extrapolation parameters, a direction of the surrounding BPUs with respect to the original BPU, or similar. In another example, if interprediction was used by the encoder to encode the current BPU, the prediction data 206 may include a prediction mode indicator (e.g., an indicative value) of the interprediction, parameters of the interprediction operation, or similar.The parameters of the interprediction operation may include, for example, the number of reference images associated with the current BPU, weights respectively associated with the reference images, locations (e.g., coordinates) of one or more matching regions in the respective reference images, one or more motion vectors respectively associated with the matching regions, or similar.

[0070] Based on the prediction mode indicator, the decoder can decide whether to perform a spatial prediction (e.g., intraprediction) at spatial prediction stage 2042 or a temporal prediction (e.g., interprediction) at temporal prediction stage 2044. The details of performing such a spatial or temporal prediction are described in FIG. 2B and will not be repeated herein. After performing such a spatial or temporal prediction, the decoder can generate the predicted BPU 208. The decoder can add the predicted BPU 208 and the reconstructed residual BPU 222 to generate prediction reference 224, as described in FIG. 3A. Petition 870250090349, dated 03 / 10 / 2025, page 35 / 193 28 / 147

[0071] In process 300B, the decoder can feed the predicted reference 224 to spatial prediction stage 2042 or temporal prediction stage 2044 to perform a prediction operation in the next iteration of process 300B. For example, if the current BPU is decoded using intraprediction in spatial prediction stage 2042, after generating prediction reference 224 (e.g., the decoded current BPU), the decoder can directly feed prediction reference 224 into spatial prediction stage 2042 for later use (e.g., for extrapolation of a next BPU from the current image). If the current BPU is decoded using interprediction in the time prediction stage 2044, after generating the prediction reference 224 (e.g., a reference image in which all BPUs have been decoded), the encoder can feed the prediction reference 224 into the circuit filter stage 232 to reduce or eliminate distortion (e.g., blocking artifacts).The decoder can apply a circuit filter to the prediction reference 224, in a manner described in FIG. 2B. The circuit-filtered reference image can be stored in buffer 234 (e.g., decoded image buffer in computer memory) for later use (e.g., to be used as an interprediction reference image for a future encoded video bitstream image 228). The decoder can store one or more reference images in buffer 234 to be used in temporal prediction stage 2044. In some embodiments, when the prediction mode indicator of the prediction data 206 indicates that interprediction was used to encode the current BPU, the prediction data additionally includes circuit filter parameters (e.g., a circuit filter intensity).

[0072] Referring again to FIG. 1, each image / video preprocessor 122, image / video encoder 124, and decoder Petition 870250090349, dated 03 / 10 / 2025, p. 36 / 193 29 / 147 image / video 144 can be implemented as any suitable hardware, software, or a combination thereof. FIG. 4 is a block diagram of an example device 400 for processing image data, consistent with the disclosure modalities. For example, device 400 can be a preprocessor, an encoder, or a decoder. As shown in FIG. 4, device 400 can include processor 402. When processor 402 executes the instructions described herein, device 400 can become a specialized machine for preprocessing, encoding, and / or decoding image data. Processor 402 can be any type of circuit capable of manipulating or processing information.For example, the 402 processor may include any combination of any number of a central processing unit (or “CPU”), a graphics processing unit (or “GPU”), a neural processing unit (“NPU”), a microcontroller unit (“MCU”), an optical processor, a programmable logic controller, a microcontroller, a microprocessor, a digital signal processor, an intellectual property (IP) core, a Programmable Logic Array (PLA), a programmable logic array (PAL), a Generic Logic Array (GAL), a Programmable Complex Logic Device (CPLD), a Field Programmable Gate Array (FPGA), a System on a Chip (SoC), an Application-Specific Integrated Circuit (ASIC), or the like. In some embodiments, the 402 processor may also be an array of processors as a single logic component. For example, as shown in FIG.4. The 402 processor may include multiple 402a processors, a 402b processor, and a 402n processor.

[0073] Device 400 may also include memory 404 configured to store data (for example, a set of instructions, computer code, intermediate data, or the like). By Petition 870250090349, dated 03 / 10 / 2025, page 37 / 193 30 / 147 For example, as shown in FIG. 4, the stored data may include program instructions (e.g., program instructions to implement the stages in processes 200A, 200B, 300A, or 300B) and data for processing (e.g., video sequence 202, video bitstream 228, or video transmission 304). Processor 402 may access the program instructions and data for processing (e.g., via bus 410), and execute the program instructions to perform an operation or manipulation on the data for processing. Memory 404 may include a high-speed random-access storage device or a non-volatile storage device.In some embodiments, 404 memory may include any combination of any number of random access memory (RAM), read-only memory (ROM), optical disk, magnetic disk, hard disk, solid-state drive, flash drive, digital security card (SD), memory stick, compact flash card (CF), or the like. 404 memory may also be a group of memories (not shown in FIG. 4) grouped together as a single logical component.

[0074] The 410 bus can be a communication device that transfers data between components within the 400 appliance, such as an internal bus (e.g., a CPU-memory bus), an external bus (e.g., a universal serial bus port, a peripheral component interconnect express port), or the like.

[0075] To facilitate explanation without causing ambiguity, the 402 processor and other data processing circuits are collectively referred to as a “data processing circuit” in this disclosure. The data processing circuit may be im Petition 870250090349, dated 03 / 10 / 2025, page 38 / 193 31 / 147 implemented completely as hardware, or as a combination of software, hardware, or firmware. Furthermore, the data processing circuitry can be a single, independent module or can be fully or partially integrated into any other component of the 400 device.

[0076] The 400 device may additionally include the 406 network interface to provide wired or wireless communication with a network (for example, the Internet, an intranet, a local area network, a mobile communications network, or the like). In some embodiments, the 406 network interface may include any combination of any number of a network interface controller (NIC), a radio frequency (RF) module, a transponder, a transceiver, a modem, a router, a gateway, a wired network adapter, a wireless network adapter, a Bluetooth adapter, an infrared adapter, a near-field communication (“NFC”) adapter, a cellular network chip, or the like.

[0077] In some embodiments, the 400 apparatus may additionally include a peripheral interface 408 to provide a connection for one or more peripheral devices. As shown in FIG. 4, the peripheral device may include, but is not limited to, a cursor control device (e.g., a mouse, a touchpad, or a touch screen), a keyboard, a display (e.g., a cathode ray tube display, a liquid crystal display, or a light-emitting diode display), a video input device (e.g., a camera or an input interface coupled to a video file), or the like.

[0078] Note that video codecs (for example, a codec running a 200A, 200B, 300A or 300B process) can be implemented as any combination of any software or hardware modules in the 400 device. For example, some or all of the Petition 870250090349, dated 03 / 10 / 2025, page 39 / 193 32 / 147 stages of process 200A, 200B, 300A or 300B can be implemented as one or more modules of device 400, such as program instructions that can be loaded into memory 404. For another example, some or all stages of process 200A, 200B, 300A or 300B can be implemented as one or more hardware modules of device 400, such as a specialized data processing circuit (e.g., an FPGA, an ASIC, an NPU, or similar).

[0079] SEI messages are intended to be transmitted within the video bitstream specifically encoded in a video encoding specification or transmitted by other means determined by the specifications to systems that make use of such encoded video bitstream. SEI messages may contain various types of data indicating the timing of video images or describing various properties of the encoded video or how they can be used or enhanced. SEI messages are also defined as containing arbitrary user-defined data. SEI messages do not affect the main decoding process, but may indicate how the video is recommended to be post-processed or displayed.

[0080] With the emergence of deep generative models, including Variational Autocoding (VAE) and Generative Adversarial Networks (GANs), facial video compression has achieved promising performance improvements. For example, the X2Face architecture can be used to control facial generation via images, audio, and position codes. Furthermore, realistic neural facial models that speak can be used through few-trial adversarial learning. For video-to-video synthesis tasks, the Face-vidtovid architecture (also known as “Face_vid2vid”) can be used. Additionally, schemes that leverage the compact 3D keypoint representation for Petition 870250090349, dated 03 / 10 / 2025, page 40 / 193 33 / 147 triggering a generative model to render the target frame can also be used. Additionally, mobile-friendly video chat systems based on FOMM can be used. VSBNet, which utilizes adversarial learning to reconstruct source frames from reference points, can also be used. Furthermore, an end-to-end facial video compression framework based on compact feature learning (CFTE), designed for high-efficiency facial video compression for ultra-low bandwidth scenarios, can be used. The CFTE scheme leverages compact feature representation to compensate for temporal evolution and reconstruct the target facial video structure end-to-end. Additionally, the CFTE scheme can be incorporated into the video coding framework with rate-distortion objective supervision.Furthermore, facial semantics can be used through the 3DMM model to characterize facial video and equip facial video encoding with facial manipulation capabilities.

[0081] FIG. 5 is a flowchart of an exemplary method 500 for processing video based on generative facial video SEI messages, according to some embodiments of the present disclosure. Method 500 describes the general syntax structure and an order for generating some syntax elements of the generative facial video SEI message. As shown in FIG. 5, method 500 may include steps 502 to 508.

[0082] In step 502, an encoder (e.g., picture / video encoder 124 in FIG. 1 or device 400 in FIG. 4) can generate and signal an identification number indicator in the SEI message to a decoder (e.g., picture / video decoder 144 in FIG. 1 or device 400 in FIG. 4). The signaled identification number indicator can be used to identify the SEI message. Petition 870250090349, dated 03 / 10 / 2025, page 41 / 193 34 / 147 and also to indicate whether the current generative facial video SEI message matches the generative network in the decoder.

[0083] In step 504, the encoder can generate multiple facial information presence indicators (e.g., three) and other parameters and signal them to indicate the presence and length of certain syntax elements associated with the facial video generative compression scheme, when the identification number indicator indicates that the facial video generative compression scheme is used.

[0084] In step 506, the encoder can generate the corresponding facial parameter information for the facial presence indicators in step 504 and signal it to the decoder.

[0085] In step 508, the decoder can use the corresponding facial parameter information signaled to reconstruct or repurpose the related facial image based on a base image.

[0086] FIG. 6 is another flowchart of an illustrative method. 600 for processing video based on generative facial video SEI messages, according to some embodiments of the present disclosure. Method 600 describes the general syntax structure and an order for generating some syntax elements of the generative facial video SEI message. As shown in FIG. 6, method 600 may include steps 602 to 610.

[0087] In step 602, an encoder (e.g., image / video encoder 124 in FIG. 1 or device 400 in FIG. 4) can generate an identification number indicator or other indicator to identify the SEI message. For example, a key indicator can be generated and flagged to identify the analysis network used to generate the syntax elements of the current generative facial video SEI message. The value of the key indicator (e.g., gfv_key as described) Petition 870250090349, dated 03 / 10 / 2025, page 42 / 193 35 / 147 below) can be used to determine if the parsing network in the encoder corresponds to the generative network in the decoder (e.g., image / video decoder 144 in FIG. 1 or device 400 in FIG. 4). Furthermore, the image order counter specifies the display counter module 1<<31 of the generated image with the current SEI message. In some embodiments, since the parsing network in the encoder is also expected to be the generative network in the decoder for decoding the bitstream sent by the encoder, the parsing network in the encoder is also referred to as a target (neural) network.

[0088] In step 604, the encoder can determine whether or not to generate and signal a parameter presence indicator for the current parameter group (mode). Furthermore, the following parameters can be generated and signaled to the decoder in subsequent steps only if the indicator is true. If the indicator is false, the parameters of the current group (mode) will not be generated and signaled. As shown in FIG. 6, step 604 can be implemented in the form of a poll for each mode until a true indicator is found.

[0089] In step 606, if the parameter presence indicator is determined to be flagged, the encoder may first generate and flag the number of parameter types when flagging the current group of parameters.

[0090] In step 608, consequently for each parameter type, the encoder can generate and signal the detailed parameters accordingly.

[0091] In step 610, after receiving the SEI message, the decoder can decode the parameters to reconstruct the facial image. As such, facial images can be generated with the decoded parameters based on the base image.

[0092] In some modes, the base image is an image Petition 870250090349, dated 03 / 10 / 2025, page 43 / 193 36 / 147 which can be encoded using a conventional encoding method. For example, it could be with H.264, H.265, and H.266. The base image provides the texture information of the human face, and based on this image, the generative network can generate additional facial images with the help of parameters indicating the differences between the images to be generated and the base image. The base image is usually the first image in a video sequence, and the following image can be generated by the neural network. The parameters needed to generate the following images are signaled in the SEI message. An SEI message contains the parameters needed to generate one or multiple images.

[0093] FIG. 7A is a flowchart of an exemplary method. 700 to generate a facial image, according to some embodiments of the present disclosure. As shown in FIG. 7A, method 700 may include steps 702 to 710, which can be implemented by a decoder (e.g., image / video decoder 144 in FIG. 1 or device 400 in FIG. 4).

[0094] In step 702, the decoder may receive a bitstream from an encoder (e.g., picture / video encoder 124 in FIG. 1 or device 400 in FIG. 4) or content distribution operator. As appreciated, the bitstream may include encoded information from a series of images. In some embodiments, the SEI message may be signaled before a GOP (Group of Images), which may be decoded together with the image pixel content.

[0095] In step 704, the decoder can decode the encoded bitstream information to obtain a base image and an SEI message. As described above, the base image can be a reference image and can provide the texture information of the human face. The generative network can generate an actual facial image by referencing the base image and based on the parameters that indicate Petition 870250090349, dated 03 / 10 / 2025, page 44 / 193 37 / 147 is the difference between the current facial image and the baseline image. The parameters mentioned here can be transmitted in the SEI message.

[0096] In step 706, the decoder can determine whether the SEI message applies to a neural network to generate a facial image. In some embodiments, the SEI message can apply to the decoder's neural network to generate the facial image when it meets two criteria: (1) the SEI message contains information relating to the facial image intended to be reconstructed; and (2) the information within the SEI message can be used by the decoder.

[0097] In step 708, the decoder can determine a mode and a corresponding facial information parameter used to encode the facial image based on the SEI message, when it is determined that the SEI message applies the neural network to generate the facial image in step 706.

[0098] In step 710, the decoder can generate the facial image based on the base image and the facial information parameter by the neural network. FIG. 7B is a schematic diagram illustrating a neural network for generating a facial image, according to some embodiments of the present disclosure. As shown in FIG. 7B, a base image and a facial information parameter (e.g., landmarks) can be introduced into the decoder's generative neural network. The generative neural network will produce a reconstructed facial image as an output. As appreciated, the facial image is encoded as the facial information parameter relative to the base image. Compared with transmitting the complete facial image, the proposed approach of exchanging only the encoded facial information parameter is advantageous for bandwidth.

[0099] Table 1 below summarizes the facial representations for generative facial video compression algorithms. In the present di Petition 870250090349, dated 03 / 10 / 2025, page 45 / 193 38 / 147 Disclosure, algorithms can also be referred to as modes, methods, or approaches. In particular, facial images exhibit strong statistical regularities, which can be economically characterized with 2D reference points, 2D keypoints, regional matrix, 3D keypoints, compact feature matrices, or facial semantics. Such facial description strategies can result in reduced encoding bitrate and improve encoding efficiency, which is thus applicable to video conferencing and live entertainment. Table 1: Examples of facial representations for generative facial video compression algorithms (modes) Facial Representation Interpretation 2D Reference Points VSBNet is a representative model that can utilize 98 groups of 2D B2x98 facial reference points to represent the main structural information of the human face, where the total number of encoding parameters for each interframe is 196. 2D Keypoints + Affine Transformation Matrix FOMM is a representative model that adopts 10 groups of learned 2D K2x10 keypoints along with their local K2x2x10 affine transformations to characterize complex movements. The total number of encoding parameters for each interframe is 60. Regional Matrix MRAA is a representative model that extracts consistent facial regions that speak to describe location, shape, and pose, mainly represented by the K2x10 change matrix, K2x2x10 covariance matrix, and K2x2x10 affine matrix. As such, the total number of encoding parameters for each interframe is 100. Petition 870250090349, dated 03 / 10 / 2025, p. 46 / 193 39 / 147 Facial Representation Interpretation Keypoints 3D Face_vid2vid is a representative model that can estimate 12-dimensional head parameters (i.e., K3x3 rotation matrix and K3x1 translation parameters) and 15 groups of learned K3x15 3D keypoint perturbations due to facial expressions, where the total number of encoding parameters for each inter-frame is 57. Compact Feature Matrix CFTE is a representative model that can model the temporal evolution of faces for the learned compact feature representation with the K4x4 matrix, where the total number of encoding parameters for each inter-frame is 16. Facial Semantics IFVC is a representative model that adopts a collection of transmitted facial semantics to represent the facial frame, including K6 mouth parameters, K1 eye parameter, K3 rotation parameters, K3 translation parameters, and K1 location parameter. The total number of encoding parameters for each inter-frame is 14.

[0100] At the 29th meeting of the Joint Video Experts Team (JVET), a generative facial video SEI message was proposed. The generative facial video SEI message can employ a range of facial semantic information to represent head posture and facial expression. The initially proposed SEI message considered only one facial representation in generative facial compression. However, facial video can be described by variations in feature structures with strong antecedents, such as reference points, 2D keypoints, regional matrix, 3D keypoints, matrix of Petition 870250090349, dated 03 / 10 / 2025, page 47 / 193 40 / 147 compact feature, facial semantics, and other formats. These facial representations can provide greater freedom to implement the syntax design and semantic description of the SEI message for facial video compression. It is desirable for SEI messages in the VVC standard to consider the different facial representations in the facial video compression task.

[0101] Currently, facial video communication has demanded more common use cases, such as facial video redirection or animation, in addition to facial video reconstruction. For example, with the popularity of metaverse activities, real-world facial movement may need to be transferred to the virtual metaverse world and represented by another person. Furthermore, facial video reconstruction is expected to be more in line with the real-world situation and create corresponding redirection. Establishing SEI messages can be important for compressing facial videos that may incorporate various forms of data to specify video frame synchronization, characterizing encoded video properties, elucidating their potential uses or improvements. Doing so would ensure that the reconstructed facial video can be efficiently post-processed and displayed in a user-friendly way to meet the real needs of users.

[0102] To solve at least one of the above problems, a new SEI message called the generative facial video SEI message is proposed in this disclosure. The proposed SEI can work in conjunction with different facial representations, such as 2D keypoints, 2D reference points, 3D keypoints, or facial semantics, which can be used to reconstruct high-quality facial video speech at ultra-low bitrate or to manipulate facial video speech for personalized characterization. As such, the proposed generative facial video SEI message can be applicable to video conferencing, Petition 870250090349, dated 03 / 10 / 2025, page 48 / 193 41 / 147 live entertainment, facial animation and metaverse-related features.

[0103] Table 2 below shows an example syntax of the SEI message. The process of implementing the SEI message can be performed as follows. As shown in Table 2, an identification number indicator gfv_id can be generated by an encoder and signaled to a decoder. In some embodiments, gfv_id can be used to indicate whether the SEI is used to encode the facial image. In some embodiments, the traditional SEI message can be signaled and not used to encode the facial image, while the generative facial video SEI message shown in Table 2 can also be signaled. Therefore, gfv_id can be used to distinguish between a traditional SEI message and a generative facial video SEI message. Referring again to FIG. 7A, the decoder can determine whether the SEI message is used to encode the facial image based on the identification number indicator in step 706.In some modes, the identification number indicator can be used to indicate a generative facial video filter.

[0104] In further reference to Table 2, the SEI message may include a gfv_feature_mode indicator. The mode used to encode the facial image can be determined based on gfv_feature_mode in step 708 by the decoder. According to different gfv_feature_mode values ​​(gfv_feature_mode == 0, gfv_feature_mode == 1, gfv_feature_mode == 0, etc.), a parameter indicator may be included in the SEI message to transmit the facial information parameters. For example, if gfv_feature_mode == 0, meaning that 2D landmarks were chosen as the feature mode in the SEI message, then the parameter indicators (1) 2d_landmark_quantization_factor, (2) 2d_landmark_num and (3) Petition 870250090349, dated 03 / 10 / 2025, page 49 / 193 42 / 147 x[i], y[i] can be included in the SEI message and can be decoded as facial information parameters.

[0105] In summary, in some modalities, the SEI message in Table 2 can be used for the following operations. First, the generative facial video SEI message contains a gfv_id identification number that can be used to identify the generative facial video SEI message. Second, a gfv_feature_mode indicator is signaled in the SEI message to determine which mode is used. Then, according to the mode used, the corresponding parameters are signaled. Third, after receiving and decoding the SEI message, the facial video can be reconstructed into a high-quality, user-friendly form through the generative adversarial network generation capability using the parameters signaled in the SEI message and the previously decoded base image as inputs. Table 2: Exemplary syntax of a generative facial video SEI message Generative_face_video (payloadSize) { Descriptor gfv_id ue(v) gfv_feature_mode ue(v) if(gfv_feature_mode == 0){ 2d_landmark_quantization_factor ue(v) 2d_landmark_num ue(v) for(i=0; i< 2d_landmark_num;i++){ x[i] ue(v) y[i] ue(v)}} else if(gfv_feature_mode == 1){ Petition 870250090349, dated 03 / 10 / 2025, p. 50 / 193 43 / 147 2d_keypoint_quantization_factor ue(v) 2d_keypoint_num ue(v) is_affine_transformation_matrix_flag u(1) for(i=0; i< 2d_keypoint_num;i++){ x[i] ue(v) y[i] ue(v) if(is_affine_transformation_matrix_flag){ for(j=0; j< 2;j++){ for(k=0;k<2;k++){ affine_ transformation _matrix[i][j][k] ue(v)}}}}} else if(gfv_feature_mode == 2){ region_quantization_factor ue(v) region_keypoint_num ue(v) is_affine_ transformation_matrix_flag u(1) is_covariance_matrix_flag u(1) for(i=0; i< region_keypoint_num;i++){ x[i] ue(v) y[i] ue(v) if(is_affine_transmation_matrix_flag){ for(j=0; j< 2;j++){ for(k=0;k<2;k++){ affine_ transformation_matrix[i][j][k]; ue(v) Petição 870250090349, de 03 / 10 / 2025, pág. 51 / 193 44 / 147 } if( is_covariance_matrix_flag){ for(m=0;m< 2;m++){ for(n=0;n<2;n++){ covariance_matrix[i][m][n]; ue(v)}}}}} else if(gfv_feature_mode == 3){ 3d_keypoint_quantization_factor ue(v) 3d_keypoint_num ue(v) is_rotation_matrix_flag u(1) is_translation_matrix_flag u(1) for(i=0; i< 3d_keypoint_num;i++){ x[i] ue(v) y[i] ue(v) z[i] ue(v)} if(is_rotation_matrix_flag){ for(j=0; j< 3;j++){ for(k=0;k<3;k++){ rotation_matrix[j][k]; ue(v)}} if(is_translation_matrix_flag){ Petição 870250090349, de 03 / 10 / 2025, pág. 52 / 193 45 / 147 for(l=0; l< 3;l++){ translation_matrix[l]; ue(v)}}} else if(gfv_feature_mode == 4){ compact_feature_quantization_factor ue(v) matrix_channel ue(v) matrix_width ue(v) matrix_height ue(v) for(k=0; k< matrix_channel; k++){ for(m=0; m< matrix_height;m++){ for(n=0;n <matrix_width;n++){ compact_feature_matrix_element[k][m][n]; ue(v)}}}} else if(gfv_feature_mode == 5){ semantic_quantization_factor ue(v) semantic_type_num ue(v) for(l=0; l<semantic_type_num;l++){ semantic_idx[l] ue(v) semantic_width[l] ue(v) semantic_height[l] ue(v) for(y=0; y< semantic_width[l];y++){ for(z=0; z< semantic_height[l];z++){ Petition 870250090349, dated 03 / 10 / 2025, p. 53 / 193 46 / 147 semantic_element[l][y][z]; ue(v)}}}} else if(gfv_feature_mode>5 && gfv_feature_mode <128){ data_quantization_factor ue(v) data_length ue(v) for(i=0;i <data_length;i++){ data_element[i] ue(v)}}}

[0106] The semantics for the syntax above are described as follows.

[0107] In some embodiments, the SEI message may contain facial parameters for different feature representations, such as 2D keypoints, 2D reference points, 3D keypoints, or facial semantics, which can be used for generative facial compression. For these facial representations, they can be categorized into different types, and gfv_feature_mode can be flagged to determine which type of facial representation is used. Based on the base image, the facial parameters in the SEI message can be used to reconstruct the facial image, and each SEI message is used to generate a facial image.

[0108] In some modes, gfv_id contains an identification number that can be used to identify a generative facial video SEI message. The value of gfv_id can be in the range of 0 to 232-2, inclusive. Petition 870250090349, dated 03 / 10 / 2025, page 54 / 193 47 / 147

[0109] In some modes, the gfv_feature_mode value may be in the range of 0 to 5, inclusive. Values ​​from 6 to 128, inclusive, for the feature code may be reserved for future use by ITU-T | ISO / IEC and may not be present in the bitstreams. Decoders may ignore GFV SEI messages with gfv_feature_mode in the range of 6 to 128, inclusive. Feature code values ​​greater than 1023 may not be present in bitstreams and are not reserved for future use. Table 3: Definition of gfv_feature_mode Value Interpretation 0 Choose 2D facial landmarks as feature mode in GFV SEI messages. In this document, VSBNet [1] is a representative model that uses 98 groups of 2D K2x98 facial landmarks to represent the main structure information of the human face, where the total number of encoding parameters for each inter-frame is 196. 1 Choose 2D keypoints as feature mode in GFV SEI messages. In this document, FOMM [2] is a representative model that adopts 10 groups of learned 2D keypoints K2x10 along with their local affine transformations ^2x2x10 to characterize complex movements. The total number of encoding parameters for each inter-frame is 60. 2 Choose consistent regions as feature mode in GFV SEI messages.In this document, MRAA [3] is a representative model that extracts consistent facial regions that speak to describe location, shape, and pose, represented primarily by the K2x10 change matrix, K2x2x10 covariance matrix, and K2x2x10 affine matrix. As such, the total number of encoding parameters for each inter-frame is 100. Petition 870250090349, dated 03 / 10 / 2025, page 55 / 193 48 / 147 Value Interpretation 3 Choice of 3D Keypoint as Feature Mode in GFV SEI Messages. In this document, Face_vid2vid [4] is the representative model that can estimate 12-dimensional head parameters (i.e., K3x3 rotation matrix and K3x1 translation parameters) and 15 learned K3x15 3D Keypoint perturbation groups due to facial expressions, where the total number of encoding parameters for each inter-frame is 57. 4 Choice of Compact Feature as Feature Mode in GFV SEI Messages. In this document, CFTE [5] is a representative model that can model the temporal evolution of faces for the learned compact feature representation with the K4x4 matrix, where the total number of encoding parameters for each inter-frame is 16.5. Choice of facial semantics as a feature mode in GFV SEI messages In this document, IFVC [6] is a representative model that adopts a collection of transmitted compact facial semantics to represent the facial frame, including mouth parameters K6, eye parameters K1, rotation parameters K3, translation parameters K3 and location parameter K1. The total number of encoding parameters for each inter-frame is 14.

[00110] [1] Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J Sullivan, and Jens-Rainer Ohm, “Overview of the versatile video coding (vvc) standard and its applications, IEEE Transactions on Circuits and Systems for Video Technology, 2021.

[00111] [2] GJ Sullivan, JR Ohm, WJ Han, T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Trans. Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649-1668, Dec. 2012

[00112] [3] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Petição 870250090349, de 03 / 10 / 2025, pág. 56 / 193 49 / 147 Xu, David Warde Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio, “Generative adversarial nets, Advances in neural information processing systems, vol. 27, 2014.

[00113] [4] Aliaksandr Siarohin, St_ephane Lathuili_ere, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe, “First order motion model for image animation, Advances in Neural Information Processing Systems, vol. 32, pp. 7137-7147, 2019.

[00114] [5] Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu, “Oneshot free-view neural talking-head synthesis for video conferencing, in Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 10039-10049.

[00115] [6] Bolin Chen, Zhao Wang, Bin Li, Rongqun Lin, Shiqi Wang, and Yan Ye, “Beyond key-point coding: Temporal evolution inference with compact feature representation for talking face video compression,” in Proceedings of the IEEE Data Compression Conference, 2022.

[00116] With reference to the definition of gfv_feature_mode in Table 3, if there are other facial feature modes that are different from the six types given, gfv_feature_mode can be further expanded. Additionally, when distinct or future generative compression models exist that may exhibit better rate distortion performance than the representative models given above, these existing models can be replaced.

[00117] The definition and possible values ​​of the parameter indicators according to their respective modes are illustrated below.

[00118] In some embodiments, when it is decided that 2D facial reference points will be the mode used to encode the facial image, the parameter indicators indicate that a facial representation of the 2D reference points can be generated and signaled by the encoder and then received by the decoder, wherein: Petition 870250090349, dated 03 / 10 / 2025, p. 57 / 193 50 / 147

[00119] 2d_landmark_quantization_factor specifies the quantization factor for processing facial semantic parameters (i.e., x[i] and y[i]) in mode 0 (i.e., gfv_feature_mode equals 0). The parameter values ​​used for facial generation are equal to the values ​​of the corresponding syntax elements divided by 2d_landmark_quantization_factor.

[00120] 2d_landmark_num specifies the number of facial landmarks in the 2D coordinate.

[00121] x[i] specifies the quantized x-axis value for the i-th point in 2D coordinate.

[00122] y[i] specifies the quantized y-axis value for the i-th point in 2D coordinate.

[00123] In some embodiments, when it is decided that 2D keypoints will be the mode used to encode the facial image, the parameter indicators indicate that a facial representation of the 2D keypoints can be generated and signaled by the encoder and then received by the decoder, where:

[00124] 2d_keypoint_quantization_factor specifies the quantization factor for processing facial semantic parameters (i.e., x[i], y[i], and affine_transmation_matrix[i][j][k]) in mode 1 (i.e., gfv_feature_mode equals 1). The parameter values ​​used for facial generation are equal to the values ​​of the corresponding syntax elements divided by 2d_keypoint_quantization_factor.

[00125] 2d_keypoint_num specifies the number of facial keypoints in the 2D coordinate.

[00126] is_affine_transformation_matrix_flag equal to 1 indicates that the SEI message contains the affine transformation parameters. is_affine_transformation_matrix_flag equal to 0 indicates that the SEI message does not contain the affine transformation matrix parameters. Petition 870250090349, dated 03 / 10 / 2025, page 58 / 193 51 / 147

[00127] affine_transformation_matrix[i][j][k] specifies the quantized element value from the corresponding affine transformation matrix.

[00128] In some embodiments, when it is decided that consistent regions will be the mode used to encode the facial image, the parameter indicators indicate that a facial representation of the consistent regions can be generated and signaled by the encoder and then received by the decoder, where:

[00129] region_quantization_factor specifies the quantization factor for processing the facial semantic parameters (i.e., x[i], y[i], affine_transmation_matrix[i][j][k], and covariance_matrix[i][m][n]) in mode 2 (i.e., gfv_feature_mode equals 2). The parameter values ​​used for facial generation are equal to the values ​​of the corresponding syntax elements divided by region_quantization_factor.

[00130] region_keypoint_num specifies the number of facial keypoints in the 2D coordinate.

[00131] is_covariance_matrix_flag equal to 1 indicates that the SEI message contains covariance matrix parameters. is_covariance_matrix_flag equal to 0 indicates that the SEI message does not contain covariance matrix parameters.

[00132] covariance_matrix[i][m][n] specifies the value of the quantized element from the corresponding covariance matrix.

[00133] In some embodiments, when it is decided that 3D keypoints will be the mode used to encode the facial image, the parameter indicators indicate that a facial representation of the 3D keypoints can be generated and signaled by the encoder and then received by the decoder, where:

[00134] 3d_keypoint_quantization_factor specifies the quantization factor for processing facial semantic parameters () in mode 3 (i.e., gfv_feature_mode equals 3). The parameter values ​​used Petition 870250090349, dated 03 / 10 / 2025, page 59 / 193 52 / 147 for facial generation are equal to the values ​​of the corresponding syntax elements divided by 3d_keypoint_quantization_factor.

[00135] 3d_keypoint_num specifies the number of facial keypoints in the 3D coordinate.

[00136] is_rotation_matrix_flag equal to 1 indicates that the SEI message contains rotation matrix parameters. is_rotation_matrix_flag equal to 0 indicates that the SEI message does not contain rotation matrix parameters.

[00137] is_translation_matrix_flag equal to 1 indicates that the SEI message contains translation matrix parameters. is_translation_matrix_flag equal to 0 indicates that the SEI message does not contain translation matrix parameters.

[00138] z[i] specifies the quantized z-axis value for the i-th point in 3D coordinate.

[00139] rotation_matrix[j][k] specifies the quantized element value of the corresponding rotation matrix.

[00140] translation_matrix[l] specifies the quantized element value of the corresponding translation matrix.

[00141] In some embodiments, when it is decided that compact features will be the mode used to encode the facial image, the parameter indicators indicate that a facial representation of compact features can be generated and signaled by the encoder and then received by the decoder, where:

[00142] compact_feature_quantization_factor specifies the quantization factor for processing facial semantic parameters (i.e., compact_feature_matrix_element[k][m][n]) in mode 4 (i.e., gfv_feature_mode equals 4). The parameter values ​​used for facial generation are equal to the values ​​of the corresponding syntax elements divided by compact_feature_quantization_factor. Petition 870250090349, dated 03 / 10 / 2025, page 60 / 193 53 / 147

[00143] matrix_channel specifies the compact feature channel and matrix_channel can be equal to or greater than 1.

[00144] matrix_width specifies the width (number of rows) of compact features and matrix_width can be equal to or greater than 1.

[00145] matrix_height specifies the height (column number) of the compact feature and matrix_height can be equal to or greater than 1.

[00146] compact_feature_matrix_element specifies the quantized element value of the corresponding compact facial feature.

[00147] In some embodiments, when it is decided that facial semantics will be the mode used to encode the facial image, the parameter indicators indicate that a facial representation of facial semantics can be generated and signaled by the encoder and then received by the decoder, where:

[00148] semantic_quantization_factor specifies the quantization factor for processing facial semantic parameters (i.e., semantic_element[l][y][z]) in mode 5 (i.e., gfv_feature_mode equals 5). The parameter values ​​used for facial generation are equal to the values ​​of the corresponding syntax elements divided by semantic_quantization_factor.

[00149] semantic_type_num specifies the number of facial semantic types in the SEI message. The value of semantic_type_num can be in the range of 0 to 26, inclusive. In some modalities, in the SEI message, the facial semantic type can be categorized into mouth parameters, eye parameters, head rotation parameters, head translation parameters, and head location parameters.

[00150] semantic_idx contains an identifier number in which facial semantics can be used for the SEI message. Petition 870250090349, dated 03 / 10 / 2025, page 61 / 193 54 / 147 Table 4: Definition of semantic_idx Value Interpretation 0 Mouth parameters with dimension 1*6 1 Eye parameters with dimension 1*1, representing the open-close state. 2 Head rotation parameters with dimension 1*3, including yaw, rotation and tilt 3 Head translation parameters with dimension 1*3, including x-axis, y-axis and z-axis 4 Head location index with dimension 1*1 used to locate the face in the input image

[00151] With reference to the definition of semantic_idx in Table 4, if there are other semantic facial representations that are different from the given semantic types, semantic_idx can be further expanded.

[00152] semantic_width[l] specifies the width (the number of lines) of the l-th semantic type and semantic_width[l] can be equal to or greater than 1.

[00153] semantic_height[l] specifies the height (column number) of the l-th semantic type and semantic_height[l] can be equal to or greater than 1.

[00154] semantic_element[l][y][z] specifies the quantized element value of the corresponding face semantic.

[00155] data_quantization_factor specifies the quantization factor for processing facial semantic parameters (i.e., data_element[i]) in the other potential mode. The parameter values ​​used for facial generation are equal to the values ​​of the corresponding syntax elements divided by data_quantization_factor. Petition 870250090349, dated 03 / 10 / 2025, p. 62 / 193 55 / 147

[00156] data_length specifies the length of feature parameters that can be used in other gfv_feature_mode modes, except for data modes.

[00157] data_element[i] specifies the quantized value for the i-th characteristic data_length.

[00158] The syntax of a generative facial video SEI message shown in Table 2 can be encoded and signaled in a bitstream, and can be further decoded by the decoder. FIG. 8 is a schematic diagram illustrating an exemplary SEI message 800, according to some embodiments of the present disclosure. As shown in FIG. 8, the SEI message 800 can include an identification number indicator (gfv_id) 801, a mode indicator (gfv_feature_mode) 802, a parameter indicator 803, etc. The corresponding bit length of these parameters is also shown in FIG. 8 as an example. As appreciated, the bit length of these parameters can be designed in another way. The parameter indicator 803 is signaled in line with the mode indicator 802. The other parameters are not shown in FIG. 8 to avoid causing any ambiguity.

[00159] Table 5 shows an example syntax of the SEI message. As shown in Table 5, an identification number indicator gfv_id can be generated by an encoder and signaled to a decoder. In some embodiments, gfv_id can be used to indicate whether the SEI is used to encode the facial image. As described above, gfv_id can be used to distinguish between a traditional SEI message and a generative facial video SEI message. Referring again to FIG. 7A, the decoder can determine whether the SEI message is used to encode the facial image based on the identification number indicator in step 706. In some embodiments, the identification number indicator can be used to indicate a Petition 870250090349, dated 03 / 10 / 2025, page 63 / 193 56 / 147 generative facial video filter.

[00160] With further reference to Table 5, the SEI message may include at least one parameter presence indicator (e.g., coordinate_present_flag, matrix_present_flag, or semantic_present_flag). The mode used to encode the facial image can be determined based on the parameter presence indicator in step 708 by the decoder. According to different modes indicated by the parameter presence indicator, a parameter indicator may be included in the SEI message to transmit the facial information parameters. For example, if coordinate_present_flag == 1 (meanwhile, matrix_present_flag == 0 and semantic_present_flag == 0), which means that the SEI message contains the coordinate parameters, then the parameter indicators (1) coordinate_quantization_factor, (2) is_3D_coordinate_flag, (3) coordinate_point_num and (4) x[i], y[i], z[i] can be included in the SEI message and can be decoded as the facial information parameters.

[00161] In some modalities, the SEI message in Table 5 can be used for the following operations. First, the generative facial video SEI message contains a gfv_id identification number that can be used to identify the generative facial video SEI message. Second, the parameter presence indicators coordinate_present_flag, matrix_present_flag, and semantic_present_flag for different parameter types are flagged. If a parameter presence indicator shows that parameters of a current type are present, then the corresponding parameters are flagged. Otherwise, the flagging of parameters of the current type is ignored. In this example, the parameters are categorized into three types: keypoints, matrices, and semantics. The semantic type includes parameters indicating the state, movement, and location of the mouth, eyes, and head. In addition, the matrix type includes affine translation matrix, matrix Petition 870250090349, dated 03 / 10 / 2025, p. 64 / 193 57 / 147 covariance, translation matrix, rotation matrix, and compact feature matrix. Third, after receiving and decoding the SEI message, the facial video can be reconstructed into a high-quality, user-friendly form through the generative adversarial network capability with the parameters signaled in the SEI message and the previously decoded base image as input. Table 5: Exemplary syntax of a generative facial video SEI message Generative_face_video (payloadSize) { Descriptor gfv_id ue(v) coordinate_present_flag u(1) if(coordinate_present_flag){ coordinate_quantization_factor ue(v) is_3D_coordinate_flag u(1) coordinate_point_num ue(v) for(i=0; i<coordinate_point_num;i++){ x[i] ue(v) y[i] ue(v) if(is_3D_coordinate_flag){ z[i] ue(v)}}} matrix_present_flag u(1) if(matrix_present_flag){ Petition 870250090349, dated 03 / 10 / 2025, p. 65 / 193 58 / 147 Generative_face_video (payloadSize) { Descriptor matrix_quantization_factor ue(v) matrix_type_num ue(v) for(j=0; j <matrix_type_num;j++){ matrix_idx[j] ue(v) if(coordinate_present_flag){ matrix_num_equal_to_coordinate_point _flag[j] u(1) if(! coordinate_present_flag || ! ma- trix_n u m_equal_to_coord inate_flag[j]){ matrix_num[j] ue(v) MatrixNum[j]=matrix_num[j]} else{ MatrixNum[j]= coordinate_point_num}} matrix_width[j] ue(v) matrix_height[j] ue(v) for(k=0; k<MatrixNum[j]; k++){ for(m=0; m< matrix_height[j];m++){ for(n=0;n<matrix_width[j];n++){ matrix_element[j][k][m][n]; ue(v)}}} Petition 870250090349, dated 03 / 10 / 2025, p. 66 / 193 59 / 147 Generative_face_video (payloadSize) { Descriptor}} semantic_present_flag u(1) if(semantic_present_flag){ semantic_quantization_factor ue(v) semantic_type_num ue(v) for(l=0; l <semantic_type_num;l++){ semantic_idx[l] ue(v) semantic_width[l] ue(v) semantic_height[l] ue(v) for(y=0; y< semantic_width[l];y++){ for(z=0; z< semantic_height[l];z++){ semantic_element[l][y][z]; ue(v)}}}}}

[00162] The semantics for the syntax above are described as follows.

[00163] In some modalities, the SEI message may contain facial parameters for different feature representations, such as 2D keypoints, 2D reference points, 3D keypoints, or facial semantics, which can be used for generative facial compression. For these facial representations, they can be categorized Petition 870250090349, dated 03 / 10 / 2025, page 67 / 193 60 / 147 in three types, including coordinate parameters, matrix parameters, and semantic parameters. Based on the base image, the facial parameters in the SEI message can be used to reconstruct the facial image, and each SEI message is used to generate a facial image.

[00164] In some modes, gfv_id contains an identification number that can be used to identify a generative facial video SEI message. The value of gfv_id can be in the range of 0 to 232-2, inclusive.

[00165] The definition and possible values ​​of the parameter indicators according to their respective modes are illustrated below.

[00166] In some modes, coordinate_present_flag equal to 1 indicates that the SEI message contains the coordinate parameters, while coordinate_present_flag equal to 0 indicates that the SEI message does not contain the coordinate parameters. When it is decided that the coordinate information of the keypoints is used to encode the facial image, the parameter indicators indicate that the coordinate information of the keypoints can be generated and signaled by the encoder and then received by the decoder, where:

[00167] coordinate_quantization_factor specifies the quantization factor for processing facial coordinate parameters (i.e., x[i], y[i], and z[i]). The parameter values ​​used for facial generation are equal to the values ​​of the corresponding syntax elements divided by coordinate_quantization_factor.

[00168] is_3D_coordinate_flag equal to 1 indicates that the coordinate parameters belong to three-dimensional space, which can be represented by (x[i], y[i], z[i]). is_3D_coordinate_flag equal to 0 indicates that the coordinate parameters belong to two-dimensional space, which can be represented by (x[i], y[i]). Petition 870250090349, dated 03 / 10 / 2025, p. 68 / 193 61 / 147

[00169] coordinate_point_num specifies the number of facial coordinate parameters defined in the SEI message. One parameter set represents a point on the coordinate. The value of coordinate_point_num can be in the range of 0 to 210, inclusive.

[00170] x[i] specifies the quantized x-axis value for the i-th keypoint.

[00171] y[i] specifies the quantized y-axis value for the i-th keypoint.

[00172] z[i] specifies the quantized z-axis value for the i-th keypoint.

[00173] In some modes, matrix_present_flag equal to 1 indicates that the SEI message contains matrix parameters, while matrix_present_flag equal to 0 indicates that the SEI message does not contain matrix parameters. When it is decided that matrix parameters are used to encode the facial image, the parameter indicators indicate that matrix parameters can be generated and signaled by the encoder and then received by the decoder, where:

[00174] matrix_quantization_factor specifies the quantization factor for processing facial matrix parameters (i.e., matrix_element[j][k][m][n]). The parameter values ​​used for facial generation are equal to the values ​​of the corresponding syntax elements divided by matrix_quantization_factor.

[00175] matrix_type_num specifies the number of facial matrix types in the SEI message. The value of matrix_type_num can be in the range of 0 to 26, inclusive. In some embodiments, the facial matrix type can be categorized into affine translation matrix, covariance matrix, rotation matrix, translation matrix, and compact feature matrix. And these matrices can be further distinguished as to their relationship to the points in the coordinate. Petition 870250090349, dated 03 / 10 / 2025, p. 69 / 193 62 / 147

[00176] matrix_idx contains an identification number relating to which facial matrix can be used in the SEI message. Table 6: Definition of matrix_idx matrix_idx Interpretation 0 The affine translation matrix corresponding to coordinate_point_num. Each keypoint in the coordinate can have a 2*2 affine translation matrix group. 1 The covariance matrix corresponding to coordinate_point_num. Each keypoint in the coordinate can have a 2*2 covariance matrix group. 2 Rotation matrix with dimension 3*3 to represent head rotation. 3 Translation matrix with dimension 1*3 to represent head translation. 4 Compact feature matrix with dimension 4*4

[00177] With regard to the definition of matrix_idx in Table 6, if there are other facial matrix representations different from the given matrix types, matrix_idx can be expanded further.

[00178] matrix_num_equal_to_coordinate_point _flag[j] equal to 1 indicates that the number of face matrices is equal to coordinate_point_num.

[00179] matrix_num[j] specifies the number of facial matrices when coordinate_present_flag or matrix_num_equal_to_coordinate_point_flag[j] does not exist. Otherwise, if both of these flags exist, the number of facial matrices is equal to coordinate_point_num.

[00180] matrix_width[j] specifies the width (the number of rows) of the j-th face matrix, and matrix_width[j] can be equal to or greater than 1.

[00181] matrix_height[j] specifies the height (column number) of the j-th facial matrix, and matrix_height[j] can be equal to or greater than 1. Petition 870250090349, dated 03 / 10 / 2025, p. 70 / 193 63 / 147

[00182] matrix_element[j][k][m][n] specifies the value of the quantized element of the corresponding face matrix.

[00183] In some embodiments, semantic_present_flag equal to 1 indicates that the SEI message contains semantic parameters, while semantic_present_flag equal to 0 indicates that the SEI message does not contain semantic parameters. When it is decided that semantic parameters are used to encode the facial image, the parameter indicators indicate that semantic parameters can be generated and signaled by the encoder and then received by the decoder, where:

[00184] semantic_quantization_factor specifies the quantization factor for processing facial semantic parameters (i.e., semantic_element[l][y][z]). The parameter values ​​used for facial generation are equal to the values ​​of the corresponding syntax elements divided by semantic_quantization_factor.

[00185] semantic_type_num specifies the number of facial semantic types in the SEI message. The value of semantic_type_num can be in the range of 0 to 26, inclusive. In some modalities, the facial semantic type can be categorized into mouth parameters, eye parameters, head rotation parameters, head translation parameters, and head location parameters.

[00186] semantic_idx contains an identifier number in which facial semantics can be used for the SEI message. Table 7: Definition of semantic_idx semantic_idx Interpretation 0 Mouth parameters with dimension 1*6 1 Eye parameters with dimension 1*1, representing the open-close state. 2 Head rotation parameters with dimension Petition 870250090349, dated 03 / 10 / 2025, page 71 / 193 64 / 147 semantic_idx Interpretation 1*3, including yaw, rotation, and tilt 3 Head translation parameters with dimension 1*3, including x-axis, y-axis, and z-axis 4 Head location index with dimension 1*1 used to locate the face in the input image

[00187] With regard to the definition of semantic_idx in Table 7, if there are other semantic facial representations that are different from the given semantic types, semantic_idx can be expanded further.

[00188] semantic_width[l] specifies the width (the number of lines) of the l-th semantic type and semantic_width[l] can be equal to or greater than 1.

[00189] semantic_height[l] specifies the height (column number) of the l-th semantic type and semantic_height[l] can be equal to or greater than 1.

[00190] semantic_element[l][y][z] specifies the quantized element value of the corresponding face semantic.

[00191] The syntax of a generative facial video SEI message shown in Table 5 can be encoded and signaled in a bitstream, and can be further decoded by the decoder. FIG. 9 is a schematic diagram illustrating an exemplary SEI message 900, according to some embodiments of the present disclosure. As shown in FIG. 9, the SEI message 900 may include an identification number indicator (gfv_id) 901, a coordinate indicator (coordinate_present_flag) 902, a matrix indicator (matrix_present_flag) 903, a semantic indicator (semantic_present_flag) 904, a parameter indicator 905, etc. The corresponding bit length of these parameters is also shown in FIG. 9 as Petition 870250090349, dated 03 / 10 / 2025, page 72 / 193 65 / 147 example. As noted, the bit length of these parameters can be designed differently. Coordinate indicator 902, matrix indicator 903, and semantic indicator 904 are signaled collaboratively. That is, in a proper SEI message 900, only one of coordinate indicator 902, matrix indicator 903, and semantic indicator 904 can be activated. In addition, parameter indicator 905 can be generated and signaled according to the activated mode. The other parameters are not shown in FIG. 9 to avoid causing any ambiguity.

[00192] An example of the common SEI message syntax for generative facial video is shown in Table 8. As shown in Table 8, a gfv_key key indicator can be generated by an encoder and signaled to a decoder. In some embodiments, gfv_key can be used to indicate a neural network used to generate the facial information parameter on the encoder side. In some embodiments, the neural network on the decoder side may not coincide with the neural network on the encoder side. As such, gfv_key can be signaled to the decoder for coincidence. Referring again to FIG. 7A, the decoder can determine whether the SEI message matches its neural network based on the key indicator in step 706. In some embodiments, as the neural network on the encoder side is also expected to be the neural network on the decoder side to decode the bitstream, the neural network on the encoder side is also referred to as a target network.

[00193] With further reference to Table 8, the SEI message may include a gfv_pic_order_cnt order flag to indicate a display order of the facial image generated according to the SEI message.

[00194] In some modalities, the SEI message may include a parameter presence flag (e.g., coordinate_pre Petition 870250090349, dated 03 / 10 / 2025, p. 73 / 193 66 / 147 sent_flag or matrix_present_flag). The mode used to encode the facial image can be determined based on the parameter presence indicator in step 708 by the decoder. According to different modes indicated by the parameter presence indicator, a parameter indicator can be included in the SEI message to transmit the facial information parameters. For example, if coordinate_present_flag == 1 (while matrix_present_flag == 0), which means that the SEI message carries the coordinate parameters, then the parameter indicators coordinate_precision_factor_minus1, num_coordinates_minus1, coordinate_z_present_flag, etc. can be included in the SEI message and can be decoded as the facial information parameters.

[00195] In some embodiments, the SEI message in Table 8 can be used by the following operations. First, a gfv_key is signaled which can be used to determine if the current generative facial video SEI message matches the generative network. The decoder or post-processor can generate facial images with the received SEI message only if it contains a known gfv_key that is specified in the application; otherwise, the parameters in the received SEI message may be extracted with a network that does not match the generative network in the decoder or post-processor, and the decoder or post-processor must ignore the received SEI message and not generate facial images. Then, the gfv_pic_order_cnt image order counter is signaled to indicate the order of the image generated by the SEI message.

[00196] Secondly, parameter presence indicators for different parameter types are signaled. If a parameter presence indicator shows that parameters of the current type are present, then the corresponding parameters are signaled. Case Petition 870250090349, dated 03 / 10 / 2025, page 74 / 193 67 / 147 otherwise, the signaling of the parameters of the current type is ignored. In some embodiments, the parameters can be categorized into two types: keypoints and matrices. Compared to some of the embodiments described above and accompanied by Table 5, the semantic type is merged with the matrix type in this document. In some embodiments, the matrix type may include affine translation matrix, covariance matrix, mouth matrix, eye matrix, head rotation matrix, head translation matrix, head location matrix, and compact feature matrix. In some embodiments, the matrix type may include affine translation matrix, covariance matrix, compact feature matrix, and semantic matrix. The semantic matrix type may additionally include mouth parameter matrix, eye parameter matrix, head rotation parameter matrix, head translation matrix, and head location matrix.

[00197] Thirdly, after receiving and decoding the SEI message, the facial video can be reconstructed into a high-quality, user-friendly form through the generative adversarial network capability with the parameters signaled in the SEI message and the previously decoded base image as input. Table 8: Exemplary syntax of a generative facial video SEI message generative_face_video (payloadSize) { Descriptor vgfv_key u(32) gfv_pic_order_cnt u(32) coordinate_present_flag u(1) if ( coordinate_present_flag ) { coordinate_precision_factor_minus1 ue(v) num_coordinates_minus1 ue(v) coordinate_z_present_flag u(1) if ( coordinate_z_present_flag ) coordinate ate_z_m ax_va lue_minus1 ue(v) Petition 870250090349, dated 03 / 10 / 2025, p. 75 / 193 68 / 147 generative_face_video (payloadSize) { Descritor for ( i=0; i< num_coordinates_minus1; i++ ) { coordinate_x_abs[ i ] u(v) if ( coordinate_x_abs[ i ] ) coordinate_x_sign_flag[ i ] u(1) coordinate_y_abs[ i ] u(v) if ( coordinate_y_abs[ i ] ) coordinate_y_sign_flag[ i ] u(1) if ( coordinate_z_present_flag ) { coordinate_z_abs[ i ] u(v) if ( coordinate_z_abs[ i ] ) coordinate_z_sign_flag[ i ] u(1)}}} matrix_present_flag u(1) if ( matrix_present_flag ) { matrix_element_precision_factor_minus1 ue(v) num_matrix_types_minus1 ue(v) for ( i=0; i <= num_matrix_types_minus1;i++ ) { matrix_type_idx[ i ] u(6) if ( matrix_type_idx[ i ] == 0 || matrix_type_idx[ i ] == 1) { if ( coordinate_present_flag ) num_matrices_equal_to_num_coordinates_flag[ i ] u(1) if ( !coordinate_present_flag || !num_ma- trix_equal_to_num_coordinates_flag[ i ] ) num_matrices_info[ i ] ue(v)} else if ( matrix_type_idx[ i ] == 2 || matrix_type_idx[ i ] == 3 || matrix_type_idx[ i ] >= 7 ) { if ( matrix_type_idx[ i ] >= 7 ) num_matrices_minus1[ i ] u(1) matrix_width_minus1[ i ] ue(v); Petição 870250090349, de 03 / 10 / 2025, pág. 76 / 193 69 / 147 generative_face_video (payloadSize) { Descritor matrix_height_minus1[ i ] ue(v)} else if ( matrix_type_idx[ i ] >= 4 && matrix_type_idx[ i ] <= 6 && !coordinate_present_flag ) matrix_for_3D_space_flag[ i ] u(1) for ( j=0; j<= numMatrices[ i ]; j++ ) { for ( k=0; k<= matrix_height_minus1[ i ]; k++ ) { for ( l=0;l<=matrix_width_minus1[ i ]; l++ ) { matrix_element_int[ i ][ j ][ k ][ l ] ue(v) matrix_element_dec[ i ][ j ][ k ][ l ] u (v) if ( matrix_element_int[ i][ j ][ k ][ l ] || matrix_ele- ment_dec[ i ][ j ][ k ][ l ] ) matrix_element_sign_flag[ i ][ j ][ k ][ l ] u(1)}}}}}}

[00198] This SEI message specifies the syntax and semantics of a variety of facial representations that can be used by a deep generative network to generate images based on a base image. The base image is an image that provides the texture information needed by the deep generative network. The base image can be encoded as a bitstream according to Rec. ITU-T H.264, Rec. ITU-T H.265, Rec. ITU-T H.266, etc. The decoder can decode the base image from the compliant bitstream, as the first image in the sequence and subsequent images can be generated with a deep generative network using syntax elements signaled in the generative facial video SEI message. Or the decoder can periodically decode the image Petition 870250090349, dated 03 / 10 / 2025, page 77 / 193 70 / 147 base from the conforming bitstream. Based on the base image, a number of subsequent images can be generated with a generative network using syntax elements signaled in the generative facial video SEI message. The texture of the base image and the additionally generated images is expected to primarily contain the speaking face.

[00199] The use of this SEI message requires the definition of the following variables: - Width and height of the decoded and cropped output base image in luma sample units, indicated in this document by CroppedWidth and CroppedHeight, respectively. - CroppedYPic from luma sample matrix and CroppedCbPic from chroma sample matrices and CroppedCrPic from decoded and cropped output base image. - A chroma format indicator, indicated in this document by ChromaFormatId.

[00200] The variables SubWidthC and SubHeightC are derived from ChromaFormatIdc.

[00201] In some embodiments, gfv_key indicates a key that can be used to identify the parsing network used to generate the syntax elements of the current generative facial video SEI message. The value of gfv_key can be used to determine whether the parsing network in the encoder matches the generative network in the decoder.

[00202] The generative network in the decoder can use the SEI message to generate video images only if it recognizes the gfv_key value. The gfv_key value can be specified by applications that rely on the matching networks in the encoder and decoder. For such applications, if the parameters in the received SEI message are extracted with a parsing network that does not match the Petition 870250090349, dated 03 / 10 / 2025, page 78 / 193 71 / 147 generative network in the decoder, the generative network can ignore the received SEI message.

[00203] In some modes, gfv_pic_order_cnt specifies the display order counting module 1<<31 of the generated image with the current SEI message.

[00204] In some modalities, coordinate_present_flag equal to 1 indicates that the coordinate information of the keypoints is present, while coordinate_present_flag equal to 0 indicates that the coordinate information of the keypoints is not present. When it is decided that the coordinate information of the keypoints is used to encode the facial image, the parameter indicators indicate that the coordinate information of the keypoints can be generated and signaled by the encoder and then received by the decoder, where:

[00205] is a bitstream conformance requirement that, when matrix_type_idx[i] for any of i from 0 to num_matrix_types_minus1 is equal to 0 or 1, the value of coordinate_present_flag can be equal to 1.

[00206] coordinate_precision_factor_minus1 plus 1 indicates the length, in bits, of coordinate_x, coordinate_y, and coordinate_z.

[00207] In some modes, the lower limit of precision may be greater than 1. For example, the keo syntax element may be changed to coordinate_precision_factor_minusk. Coordinate_precision_factor_minusk plus k indicates the length, in bits, of coordinate_x, coordinate_y, and coordinate_z.

[00208] num_coordinates_minus1 plus 1 indicates the number of keypoint coordinates. The value of num_coordinates_minus1 can be in the range of 0 to 210-1, inclusive.

[00209] coordinate_z_present_flag equal to 1 indicates that it informs Petition 870250090349, dated 03 / 10 / 2025, p. 79 / 193 72 / 147 z-axis coordinate information for key points is present. A value of 0 indicates that z-axis coordinate information for key points is not present. When coordinate_z_present_flag is not present, the value is inferred as 0.

[00210] coordinate_z_max_value_minus1 plus 1 indicates the maximum absolute value of the z-axis coordinates of the key points.

[00211] In some modes, the lower limit of the maximum value of the z-axis coordinate may be greater than 1, for example, the flagged syntax element ke may be changed to coordinate_z_max_value_minusk. coordinate_z_max_value_minusk plus k indicates the maximum absolute value of the z-axis coordinates of the keypoints.

[00212] coordinate_x_abs[i] specifies the normalized absolute value of the x-axis coordinate of the i-th keypoint.

[00213] coordinate_x_sign_flag[i] specifies the sign of the x-axis coordinate of the i-th keypoint. When coordinate_x_sign_flag[i] is not present, the value is inferred to be 0.

[00214] coordinate_y_abs[i] specifies the normalized absolute value of the y-axis coordinate of the i-th keypoint.

[00215] coordinate_y_sign_flag[i] specifies the sign of the y-axis coordinate of the i-th keypoint. When coordinate_y_sign_flag[i] is not present, the value is inferred to be 0.

[00216] coordinate_z_abs[i] specifies the normalized absolute value of the z-axis coordinate of the i-th keypoint.

[00217] coordinate_z_sign_flag[i] specifies the z-axis coordinate sign of the i-th keypoint. When coordinate_z_sign_flag[i] is not present, the value is inferred to be 0.

[00218] The variables coordinateX[i], coordinateY[i] and coordinateZ[i] indicating the x-axis coordinate, y-axis coordinate and y-axis coordinate. Petition 870250090349, dated 03 / 10 / 2025, p. 80 / 193 73 / 147 z-axis of the i-th keypoint are derived, respectively, as follows: + vr i — (1-2 * coordinate_x_sign_flag[ i ]) * coordinate_x_abs[ i ] * CroppedWidth[]1<< (coordinate precision factor minus1 +1) .νΓ1_ (1-2 * coordinate_y_sign_flag[ i ]) * coordinate_y_abs[ i ] * CroppedHeight[]1<< (coordinate precision factor minus1 +1) coordenaZ[i] = (1-2 * coordination z-flag[ i ]) * coordinate z abs[ i ] *(coordinate z max value minus1+1) 1<< (coordinate_precision_factor_minus1 +1)

[00219] In some embodiments, matrix_present_flag equal to 1 indicates that matrix parameters are present, while matrix_present_flag equal to 0 indicates that matrix parameters are not present. When it is decided that matrix parameters are used to encode the facial image, the parameter indicators indicate that matrix parameters can be generated and signaled by the encoder and then received by the decoder, where:

[00220] matrix_element_precision_factor_minus1 plus 1 indicates the length, in bits, of matrix_element_dec[i][j][k][l].

[00221] num_matrix_types_minus1 plus 1 indicates the number of matrix types indicated in the SEI message. The value of matrix_type_num_minus1 can be in the range of 0 to 26-1, inclusive.

[00222] matrix_type_idx[i] indicates the index of the i-th matrix type as specified in Table 9. Table 9: Matrix_type_idx specification Value Specification 0 Affine translation matrix with size 2*2 or 3*3. 1 Covariance matrix with size 2*2 or 3*3. 2 Mouth matrix representing mouth movement 3 Eye matrix representing open-close state and eye level. Petition 870250090349, dated 03 / 10 / 2025, page 81 / 193 74 / 147 Value Specification 4 Head rotation parameters with a size of 2*2 or 3*3 representing the rotation of the head in 2D or 3D space. 5 Head translation matrix with a size of 1*2 or 1*3 representing the translation of the head in 2D or 3D space. 6 Head location matrix with a size of 1*2 or 1*3 representing the location of the head in 2D or 3D space. 7 Compact feature matrix with the size being specified by matrix_width_minus1[i] and matrix_height_minus1[i]. 8.31 Other undefined matrix with the size being specified by matrix_width_minus1[i] and matrix_height_minus1[i]. 32.63 Reserved

[00223] The undefined matrix type is used to represent the matrix type that is not an affine translation matrix, covariance matrix, rotation matrix, translation matrix, or compact feature matrix. It can be used by the user to extend the matrix type.

[00224] num_matrices_equal_to_num_coordinates_flag[i] equal to 1 indicates that the number of matrices of the nth matrix type is equal to num_coordinates_minus1 + 1. num_matrices_equal_to_num_coordinates_flag[i] equal to 0 indicates that the number of matrices of the ith matrix type is not equal to num_coordinates_minus1 + 1.

[00225] num_matrices_info[i] provides information for deriving the number of matrices of type i-th matrix.

[00226] matrix_width_minus1[i] plus 1 indicates the width of the matrix of Petition 870250090349, dated 03 / 10 / 2025, p. 82 / 193 75 / 147 i-th matrix type.

[00227] When matrix_width_minus1 [i] is not present, it is inferred as follows: — If matrix_type_idx[i] equals 0, 1, or 4, and coordinate_z_present_flag is 1, matrix_width_minus1[i] is inferred to be 2 — otherwise, if matrix_type_idx[i] equals 0, 1, or 4, and coordinate_z_present_flag is 0, matrix_width_minus1[i] is inferred to be 1 — otherwise, (matrix_type_idx[i] equals 5 or 6), matrix_width_minus1[i] is inferred to be 0 matrix_height_minus1[i] plus 1 indicates the height of the i-th matrix type. When matrix_height_minus1[i] is not present, it is inferred as follows: — If matrix_type_idx equals 0, 1, 4, 5, or 6, and coordinate_z_present_flag is 1, then matrix_height_minus1[i] is inferred to be equal to 2. — otherwise (matrix_type_idx is equal to 0, 1, 4, 5, or 6, and coordinate_z_present_flag is 0), matrix_height_minus1[i] is inferred to be equal to 1.

[00228] num_matrices_minus1[i] plus 1 indicates the number of matrices of the i-th matrix type.

[00229] matrix_for_3D_space_flag[i] equal to 1 indicates that the matrix of type i-th matrix is ​​a matrix for three-dimensional space. matrix_size_flag[i] equal to 0 indicates that the matrix of type i-th matrix is ​​a matrix for two-dimensional space.

[00230] The variable numMatrices[i] indicating the number of matrices of the i-th matrix type is derived as follows: if ( matrix_type_idx[i] == 0 || matrix_type_idx[i] == 1 ) { Petition 870250090349, dated 03 / 10 / 2025, p. 83 / 193 76 / 147 if ( coordinate_present_flag ) numMatrices[i] = num_matrices_equal_to_num_coordinates_flag[i] ? num_coordinates_minus1 + 1 : ( num_matrices_idc[i] < num_coordinates_minus1 ? num_matrices_idc[i] + 1 : num_matrices_idc[i] + 2 ) else numMatrices[i] = num_matrices_idc[i] + 1} else if ( matrix_type_idx[i] >= 2 && matrix_type_idx[i] < 7) numMatrices[i] = 1 else numMatrices[i] = num_matrices_minus1[i] + 1

[00231] matrix_element_int[i][j][k][l] indicates the integer part of the matrix element value at position (k, l) of the j-th matrix of the i-th matrix type.

[00232] matrix_element_dec[i][j][k][l] indicates the decimal part of the value of the matrix element at position (k, l) of the j-th matrix of the i-th matrix type.

[00233] matrix_element_sign_flag[i][j][k][l] indicates the sign of the matrix element at position (k, l) of the j-th matrix of the i-th matrix type. When matrix_element_sign_flag[i][j][k][l] is not present, it is inferred to be equal to 0.

[00234] The variable MatrixElementVal[i][j][k][l] representing the value of the matrix element at position (k, l) of the j-th matrix of the i-th matrix type is derived as follows: matrixElementVal[i][j][k][l] = (1-2* matrix_element_sign_flag[i][j][k][l]) * (matrix_element_int[i][j][k][l] + matrix_element_dec[i][j][k][l] .. 1<< (matrix_element_precision_factor_minus1 +1)

[00235] O processo DeriveInputTensors( ), para derivar o tensor de entrada inputTensorImgY, inputTensorImgCb, inputTesnsorImgCr, in Petição 870250090349, de 03 / 10 / 2025, pág. 84 / 193 77 / 147 putTensorKeyPoint, inputTensorMatrix é especificado conforme a seguir: Inicializa inputTensorImgY, inputTensorImgCb, inputTesnsorImgCr, inputTensorKeyPoint e inputTensorMatrix com 0. for ( x = 0; x< CroppedWidth; x++ ) { for ( y = 0; y< CroppedHeight; y++ ) { inputTsensorImgY[x][y] = CroppedYPic[ x ][ y ]} } for ( x=0; x< CroppedWidth / SubWidthC; x++ ) { for ( y=0; y< CroppedHeight / SubHeightC; y++ ) { inputTsensorImgCb[x][y] = CroppedCbPic[x][y] inputTsensorImgCr[x][y] = CroppedCrPic[x][y]} } if ( coordinate_present_flag ) { for ( i=0; i< =num_coordinates_minus1; i++ ) { inputTensorKeyPoint[i][0] = coordinateX[i] inputTensorKeyPoint[i][1] = coordinateY[i] if ( coordinate_z_present_flag ) inputTensorKeyPoint[i][2] = coordinateZ[i]} } if ( matrix_present_flag ) { for ( i=0; i<=num_matrix_types_minus1; i++ ) { for ( j=0; j< numMatrices[i]; j++ ) { for( k=0; k<= matrix_height_minus1[i]; k++ ) { for ( l=0;l<=matrix_width_minus1[i]; l++) { inputTensorMatrix[i][j][k][l] = matrixElementVal[i][j][k][l]} Petição 870250090349, de 03 / 10 / 2025, pág. 85 / 193 78 / 147} }} }

[00236] O processo StoreOutputTensors( ), para derivar matrizes de amostra de saída OutputYPic, OutputCbPic e OutputCrPic do tensor de saída outputTensorY, outputTensorCb e outputTensorCr é especificado a seguir: for(x=0; x< CroppedWidth; x++){ for(y=0; y< CroppedHeight; y++){ OutputYPic[x][y] = outputTensorY[x][y]}} for(x=0; x< CroppedWidth / SubWidthC; x++){ for(y=0; y< CroppedHeight / SubHeightC; y++){ OutputCbPic[x][y] = outputTensorCb[x][y] OutputCdPic[x][y] = outputTensorCr[x][y]}

[00237] The following process is used to generate a video image, where PictureGeneration() is a deep generative network-based process for sending the image sample values ​​with signaled parameters in the facial generation SEI message and the decoded base image as input. { DeriveInputTensors( ) outputTensor = PictureGeneration ( inputTensorImgY, inputTensorImgCb, inputTesnsorImgCr, inputTensorKeyPoint, inputTensorMatrix) StoreOutputTensors()}

[00238] In some modalities, the generative neural network can be Petition 870250090349, dated 03 / 10 / 2025, page 86 / 193 79 / 147 divided into two subnets (for example, one subnet and a second subnet) to solve the interoperability problem.

[00239] In some embodiments, the first subnetwork may be a stream translator network that transfers the signaled parameters in the SEI message into the stream map, while the second subnetwork may be a generative network that generates the images based on the stream map output from the first subnetwork. The stream translator network could support different types of facial representations for different algorithms and transfer all facial representation parameters into a stream map. Thus, the generative network can be fixed independently of which type of facial representation is signaled in the SEI message, since the input to the generative network is always a stream map.

[00240] In some embodiments, the first subnetwork may be a parameter translator network that translates the facial representation parameters signaled in the SEI message into a specific parameter type, while the second subnetwork may be a generative network that generates the image based on the specified type of facial representation parameters. Consequently, the generative network may also be fixed, as it supports the specified parameter type as input.

[00241] In some embodiments, the SEI message can be used to directly signal the network or provide a URI to the subnetwork to indicate which network type is signaled in the SEI message. For example, a key indicator (e.g., gfv_key as described in this document) can be signaled to indicate which network type is signaled in the SEI message.

[00242] The network flagged or indicated in the SEI message can be updated. A first SEI message in the current CLVS can be used to flag or indicate a network as a base GFV, and a subsequent SEI message can be used to update the base GFV. Petition 870250090349, dated 03 / 10 / 2025, page 87 / 193 80 / 147 signaling or indicating an updated network.

[00243] The syntax of a generative facial video SEI message shown in Table 8 can be encoded and signaled in a bitstream, and can be further decoded by the decoder. FIG. 10 is a schematic diagram illustrating an exemplary SEI message 1000, according to some embodiments of the present disclosure. As shown in FIG. 10, the SEI message 1000 can include a key indicator (gfv_key) 1001, an order indicator (gfv_pic_order_cnt) 1002, a coordinate indicator (coordinate_present_flag) 1003, a matrix indicator (matrix_present_flag) 1004, etc. The corresponding bit length of these parameters is also shown in FIG. 10 as an example. As appreciated, the bit length of these parameters can be designed in another way. Coordinate indicator 1003 and matrix indicator 1004 are collaboratively flagged.That is, in a proper SEI message 1000, only one of the coordinate indicator 1003 and matrix indicator 1004 can be activated. Additionally, the parameter indicator 1005 can be generated and signaled according to the activated mode. The other parameters are not shown in FIG. 10 to avoid any ambiguity.

[00244] An example of the common SEI message syntax for generative facial video is shown in Table 10. As shown in Table 10, a gfv_id identification number indicator can be generated by an encoder and signaled to a decoder. In some embodiments, gfv_id can be used to indicate whether SEI is used to encode the facial image.

[00245] In some modes, a key presence indicator (e.g., gfv_key_present_flag as described below) may be included in the SEI message to indicate the presence of the key indicator (e.g., gfv_key as described in this document). Referring to Petition 870250090349, dated 03 / 10 / 2025, page 88 / 193 81 / 147 again to FIG. 7A, in step 706, the decoder can determine if the SEI message matches the decoder's neural network based on the key indicator, under the premise that the key presence indicator indicates the presence of the Key Indicator. That is, the decoder can ignore the determination of correspondence between neural networks when the key presence indicator indicates the absence of the Key Indicator.

[00246] In some embodiments, the neural network indicator may be included in the SEI message to indicate a target neural network configuration. Therefore, the decoder can adjust (e.g., initialize or update) the neural network based on the neural network indicator when the key presence indicator (e.g., gfv_key_present_flag) indicates the absence of the key indicator (e.g., gfv_key). In some embodiments, the neural network indicator may include gfv_nn_mode_idc, gfv_nn_type_idc, etc., as described below. Table 10: An example of the syntax of a generative facial video compression SEI message. generative_face_video ( payloadSize ) { Descriptor gfv_id ue(v) gfv_key_present_flag if( gfv_key_present_flag ) { gfv_key u(32) else { gfv_nn_mode_idc ue(v) gfv_nn_type_idc if( gfv_nn_mode_idc = = 1 ) { / *specify the complete decoder ( )* / while( !byte_aligned( ) ) Petition 870250090349, dated 03 / 10 / 2025, page 89 / 193 82 / 147 generative_face_video ( payloadSize ) { Descritor gfv_nn_reserved_zero_bit_a u(1) gfv_nn_tag_uri st(v) gfv_nn_uri st(v)}} coordinate_present_flag u(1) if ( coordinate_present_flag ) { coordinate_precision_factor_minus1 ue(v) num_coordinates_minus1 ue(v) coordinate_z_present_flag u(1) if ( coordinate_z_present_flag ) coordinate_z_max_value_minus1 ue(v) for ( i=0; i< num_coordinates_minus1; i++ ) { coordinate_x_abs[i] u(v) if ( coordinate_x_abs[i] ) coordinate_x_sign_flag[ i ] u(1) coordinate_y_abs[ i ] u(v) if ( coordinate_y_abs[ i ] ) coordinate_y_sign_flag[ i ] u(1) if ( coordinate_z_present_flag ) { coordinate_z_abs[ i ] u(v) if ( coordinate_z_abs[ i ] ) coordinate_z_sign_flag[ i ] u(1)}} Petição 870250090349, de 03 / 10 / 2025, pág. 90 / 193 83 / 147 generative_face_video ( payloadSize ) { Descritor} matrix_present_flag u(1) if ( matrix_present_flag ) { matrix_element_precision_factor_minus1 ue(v) num_matrix_types_minus1 ue(v) for ( i=0; i <= num_matrix_types_minus1; i++ ) { matrix_type_idx[i] u(6) if ( matrix_type_idx[i] == 0 || matrix_type_idx[i] == 1) { if ( coordinate_present_flag ) num_matrices_eq ual_to_nu m_coordinates_flag[i] u(1) if ( !coordinate_present_flag || !num_ma- trix_equal_to_nu m_coordinates_flag[i] ) num_matrices_info[i] ue(v)} else if ( matrix_type_idx[i] == 2 || ma- trix_type_idx[i] == 3 || matrix_type_idx[i] >= 7 ) { matrix_width_minus1[i] ue(v) matrix_height_minus1[i] ue(v) if ( matrix_type_idx[i] >= 7 ) num_matrices_minus1[i]} for ( j=0; j<= num_matrices_minus1[i]; j++ ) { for ( k=0; k<= matrix_height_minus1[i]; k++ ) { for ( l=0;l<=matrix_width_minus1[i]; l++ ) { matrix_element_int[i][j][k][l] ue(v) Petição 870250090349, de 03 / 10 / 2025, pág. 91 / 193 84 / 147 generative_face_video ( payloadSize ) { Descriptor matrix_element_dec[i][j][k][l] u (v) if ( matrix_element_int[i][j][k][l] || matrix_element_dec[i][j][k][l] ) matrix_element_sign_flag[i][j][k][l] u(1)}}}}} if(gfv_nn_mode_idc = = 0 ) { while( !byte_aligned( ) ) gfv_nn_reserved_zero_bit_b u(1) for( i = 0; more_data_in_payload( ); i++ ) gfv_nn_payload_byte[i] b(8)}}

[00247] The Generative Facial Video (GFV) SEI message indicates the facial parameters and specifies a neural network, named Generator(), which can be used to generate new output images using the indicated facial parameters and previously decoded output images. The Generator() network is further divided into a translator subnetwork Translator() and an image generation network Decoder(). In one example, Translator() converts the facial parameters into a flowmap, and Decoder() generates images based on the flowmap output by Translator(). In another example, Translator() translates Petition 870250090349, dated 03 / 10 / 2025, pp. 92 / 193 85 / 147 the facial parameters signaled in the SEI message in a specified parameter type and Decoder() generates images based on the specified type of facial parameters sent by Translator().

[00248] Note that facial parameters can be determined from source images prior to encoding.

[00249] Note also that when the current image is not a base image, the GFV SEI message can be used to generate a new facial image based on the previously decoded base image, the facial parameters transmitted by the GFV SEI message, or the operation fusion image. Base and fusion images can be encoded according to Rec. ITU-T H.264, Rec. ITU-T H.265, Rec. ITU-T H.266, etc.

[00250] The use of this SEI message requires the definition of the following variables: • Width and height of the input image in luma sample units, indicated in this document by CroppedWidth and CroppedHeight, respectively. • Luma sample matrix baseCroppedYPic and chroma sample matrices baseCroppedCbPic and baseCroppedCrPic for a decoded output image, called BasePicture, corresponding to a source base image. • Luma sample matrix driveCroppedYPic and chroma sample matrices driveCroppedCbPic and driveCroppedCrPic for a decoded output image, called DrivePicture, corresponding to a source targeting image. • Bit depth (BitDepthY) for the luma sample array of the input images. • BitDepthC for the chroma sample matrices, if any, of the input images.

[00251] gfv_id contains an identification number that can be Petition 870250090349, dated 03 / 10 / 2025, pp. 93 / 193 86 / 147 is used to identify facial feature information and specify a neural network that can be used as a Generator(). The gfv_id value can be in the range of 0 to 232 -2, inclusive. gfv_id values ​​from 256 to 511, inclusive, and from 231 to 232 -2, inclusive, are reserved for future use by ITU-T | ISO / IEC. Decoders that encounter a GFV SEI message with gfv_id in the range of 256 to 511, inclusive, or in the range of 231 to 232 -2, inclusive, can ignore the SEI message.

[00252] Note that the different gfv_id values ​​in the different GFV SEI messages can be used to identify different faces when more than one face is present in an output image, for example.

[00253] In some modes, gfv_key_present_flag equal to 1 indicates that the gfv_key syntax element is present and the gfv_nn_mode_idc, gfv_nn_reserved_zero_bit_a, gfv_nn_tag_uri, gfv_nn_uri, gfv_nn_payload_byte[i] syntax elements are not present. gfv_key_present_flag equal to 0 indicates that the gfv_key syntax element is not present and the gfv_nn_mode_idc, gfv_nn_reserved_zero_bit_a, gfv_nn_tag_uri, gfv_nn_uri, gfv_nn_payload_byte[i] syntax elements may be flagged.

[00254] In some embodiments, gfv_key indicates a key that can be used to identify the analysis network used to generate the syntax elements of the current generative facial video SEI message.

[00255] Note that the gfv_key value can be used to determine if the parsing network in the encoder matches the generative network in the decoder. The generative network in the decoder can use the SEI message to generate the video images only if it recognizes the gfv_key value. The gfv_key value can be specified by applications that depend on the matching networks in the encoder and Petition 870250090349, dated 03 / 10 / 2025, pp. 94 / 193 87 / 147 in the decoder. For such applications, if the parameters in the received SEI message are extracted with a parsing network that does not match the generative network in the decoder, the generative network may ignore the received SEI message.

[00256] In some embodiments, a neural network mode indicator may be included in the SEI message to indicate a format of information pertaining to the neural network. For example, gfv_nn_mode_idc, when equal to 0, indicates that neural network information is contained in the GFV SEI message, and the neural network information is in the format of an ISO / IEC 15938-17 bitstream. gfv_nn_mode_idc equal to 1 indicates that the neural network information is identified by the URI indicated by gfv_nn_uri with the format identified by the URI identifier gfv_nn_tag_uri.

[00257] The value of gfv_nn_mode_idc is in the range of 0 to 255, inclusive. Values ​​from 2 to 255, inclusive, for gfv_nn_mode_idc are reserved for future use by ITU-T | ISO / IEC and are not present in ITU-T | ISO / IEC compliant bitstreams. ITU-T | ISO / IEC compliant decoders may ignore GFV SEI messages with gfv_nn_mode_idc in the range of 2 to 255, inclusive.

[00258] In some embodiments, a neural network type indicator may be included in the SEI message to indicate a type of neural network indicated in the SEI message. For example, gfv_nn_type_idc may indicate the type of network contained in or indicated by the GFV SEI message.

[00259] When gfv_nn_type_idc equals 0, the Generator() is contained or indicated in the GFV SEI message. The network is a complete image generator with a motion estimation module to convert facial parameters into a flow map and a frame generation module to reconstruct the facial image.

[00260] When gfv_nn_type_idc equals 1, the Translator() is contained Petition 870250090349, dated 03 / 10 / 2025, pp. 95 / 193 88 / 147 or as indicated in the GFV SEI message. And the Translator() is a network converting the different types of facial parameters signaled in the SEI message into a flow map for image generation.

[00261] When gfv_nn_type_idc equals 2, Translator() is contained or indicated in the GFV SEI message. And Translator() is a network converting different types of facial parameters into a specified type of facial parameters.

[00262] When gfv_nn_type_idc equals 3, Decoder() is contained or indicated in the GFV SEI message. Decoder() generates the images based on the stream map or a specified type of facial parameters as input.

[00263] gfv_nn_reserved_zero_bit_a is equal to 0.

[00264] gfv_nn_tag_uri contains a URI marker with syntax and semantics as specified in IETF RFC 4151 identifying the format and information associated with the neural network used as a base GFV or an update relative to the base GFV with the same gfv_id value specified by gfv_nn_uri.

[00265] For example, if gfv_nn_tag_uri equals “tag:iso.org, 2023:15938-17”, then it indicates that the neural network data identified by gfv_nn_uri conforms to ISO / IEC 15938-17.

[00266] In some embodiments, gfv_nn_uri contains a URI with syntax and semantics as specified in IETF Internet Standard 66 identifying the neural network used as a base GFV or an update relative to the base GFV with the same gfv_id value.

[00267] coordinate_present_flag equal to 1 indicates that keypoint coordinate information is present, while coordinate_present_flag equal to 0 indicates that keypoint coordinate information is not present. When it is decided that keypoint coordinate information is used for encoding Petition 870250090349, dated 03 / 10 / 2025, pp. 96 / 193 89 / 147 the facial image, the parameter indicators indicate that the coordinate information of the key points can be generated and signaled by the encoder and then received by the decoder, where:

[00268] coordinate_precision_factor_minus1 plus 1 indicates the length, in bits, of coordinate_x, coordinate_y and coordinate_z.

[00269] In some modes, the lower precision limit may be greater than 1. For example, the syntax element `keo` can be changed to `coordinate_precision_factor_minusk`. `coordinate_precision_factor_minusk` plus `k` indicates the length, in bits, of `coordinate_x`, `coordinate_y`, and `coordinate_z`.

[00270] num_coordinates_minus1 plus 1 indicates the number of keypoint coordinates. The value of num_coordinates_minus1 is in the range of 0 to 210-1, inclusive.

[00271] coordinate_z_present_flag equal to 1 indicates that the z-axis coordinate information for the key points is present. coordinate_z_present_flag equal to 0 indicates that the z-axis coordinate information for the key points is not present. When coordinate_z_present_flag is not present, the value is inferred as 0.

[00272] coordinate_z_max_value_minus1 plus 1 indicates the maximum absolute value of the z-axis coordinates for the key points.

[00273] In some modes, the lower limit of the maximum value of the z-axis coordinate may be greater than 1, for example, the flagged syntax element ke may be changed to coordinate_z_max_value_minusk. coordinate_z_max_value_minusk plus k indicates the maximum absolute value of the z-axis coordinates of the key points.

[00274] coordinate_x_abs[i] specifies the normalized absolute value of the x-axis coordinate of the i-th keypoint.

[00275] coordinate_x_sign_flag[i] specifies the sign of the x-axis coordinate of the i-th keypoint. When coordinate_x_sign_flag[i] does not Petition 870250090349, dated 03 / 10 / 2025, pp. 97 / 193 Since 90 / 147 is present, the value is inferred to be equal to 0.

[00276] coordinate_y_abs[i] specifies the normalized absolute value of the y-axis coordinate of the i-th keypoint.

[00277] coordinate_y_sign_flag[i] specifies the sign of the y-axis coordinate of the i-th keypoint. When coordinate_y_sign_flag[i] is not present, the value is inferred to be 0.

[00278] coordinate_z_abs[i] specifies the normalized absolute value of the z-axis coordinate of the i-th keypoint.

[00279] coordinate_z_sign_flag[i] specifies the z-axis coordinate sign of the i-th keypoint. When coordinate_z_sign_flag[i] is not present, the value is inferred to be 0.

[00280] The variables coordinateX[i], coordinateY[i] and coordinateZ[i] indicating the x-axis coordinate, y-axis coordinate and z-axis coordinate of the i-th keypoint are derived, respectively, as follows: coordinateX[i] = coordinateY[i] = (1-2 * coordinate x sign flag[i]) * coordinate x abs[i] * CroppedWidth 1<< (coordinate_precision_f actor _minus1 +1) (1-2 * coordinate y sign flag[i]) * coordinate y abs[i] * CroppedHeight 1<< (coordinate_precision_f actor _minus1 +1) coordinateZ[i] (1-2 * coordinate z sign flag[i]) * coordinate z abs[i] *(coordinate z max value minus1+1) 1<< (coordinate_precision_f actor _minus1 +1)

[00281] In some embodiments, matrix_present_flag equal to 1 indicates that matrix parameters are present, while matrix_present_flag equal to 0 indicates that matrix parameters are not present. When it is decided that matrix parameters are used to encode the facial image, the parameter indicators indicate that matrix parameters can be generated and signaled by the encoder and then received by the decoder, where:

[00282] matrix_element_precision_factor_minus1 plus 1 indicates the length, in bits, of matrix_element_dec[i][j][k][l].

[00283] num_matrix_types_minus1 plus 1 indicates the number of types Petition 870250090349, dated 03 / 10 / 2025, pp. 98 / 193 91 / 147 of matrices are flagged in the SEI message. The value of matrix_type num_minus1 is in the range of 0 to 26-1, inclusive.

[00284] matrix_type_idx indicates the matrix type index as specified in Table 11. Table 11: Matrix_type_idx specification Value Specification 0 Affine translation matrix with size 2*2 or 3*3. 1 Covariance matrix with size 2*2 or 3*3. 2 Mouth matrix representing mouth movement. 3 Eye matrix representing open-close state and eye level. 4 Head rotation parameters with size 2*2 or 3*3 representing head rotation in 2D or 3D space. 5 Head translation matrix with size 1*2 or 1*3 representing head translation in 2D or 3D space. 6 Head location matrix with size 1*2 or 1*3 to locate the face in 2D or 3D space. 7 Compact feature matrix with size specified by matrix_width_minus1[i] and matrix_height_minus1[i]. 8.31 Another undefined array with the size being specified by matrix_width_minus1[i] and matrix_height_minus1[i]. 32.63 Reserved Petition 870250090349, dated 03 / 10 / 2025, pp. 99 / 193 92 / 147

[00285] Note that the undefined matrix type is used to represent the type of matrix that is not an affine translation matrix, covariance matrix, rotation matrix, translation matrix, or compact feature matrix. It can be used by the user to extend the matrix type.

[00286] num_matrix_equal_to_num_coordinates_flag[i] equal to 1 indicates that the number of matrices of type i-th matrix is ​​equal to num_coordinates_minus1 + 1. num_matrix_equal_to_num_coordinates_flag[i] equal to 0 indicates that the number of matrices of type i-th matrix is ​​not equal to num_coordinates_minus1 + 1.

[00287] num_matrices_info[i] provides information for the number of matrices of the i-th matrix type.

[00288] matrix_width_minus1[i] plus 1 indicates the width of the i-th matrix.

[00289] When matrix_width_minus1[i] is not present, it is inferred as follows: • If matrix_type_idx[i] equals 0, 1, or 4, and coordinate_z_present_flag is 1, matrix_width_minus1[i] is inferred to be 2. • Otherwise, if matrix_type_idx[i] equals 0, 1, or 4, and coordinate_z_present_flag is 0, matrix_width_minus1[i] is inferred to be 1. • Otherwise, if (matrix_type_idx[i] equals 5 or 6), matrix_width_minus1[i] is inferred to be 0.

[00290] matrix_height_minus1[i] plus 1 indicates the height of the matrix of the i-th matrix type.

[00291] When matrix_height_minus1[i] is not present, it is inferred as follows: • If matrix_type_idx equals 0, 1, 4, 5, or 6, and coordinate_z_present_flag is 1, then matrix_height_minus1[i] is inferred to be equal to 2. Petition 870250090349, dated 03 / 10 / 2025, pp. 100 / 193 93 / 147 • otherwise, if (matrix_type_idx is equal to 0, 1, 4, 5 or 6, and coordinate_z_present_flag is 0), matrix_height_minus1[i] is inferred to be equal to 1.

[00292] num_matrices_minus1[i] plus 1 indicates the number of matrices of the i-th matrix type.

[00293] The variable numMatrices[i] indicating the number of matrices of the i-th matrix type is derived as follows: if ( matrix_type_idx[i] == 0 || matrix_type_idx[i] == 1 ) { if ( coordinate_present_flag ) numMatrice[i] = num_matrices_equal_to_num_coordinates_flag[i] ? num_coordinates_minus1 + 1 : ( num_matrices_idc[i] < num_coordinates_minus1 ? num_matrices_idc[i] + 1 : num_matrices_idc[i] + 2 ) else numMatrice[i] = num_matrices_idc[i] + 1} else if ( matrix_type_idx[i] >= 2 && matrix_type_idx[i] < 7) numMatrice[i] = 1 else numMatrice[i] = num_matrices_minus1[i] + 1

[00294] matrix_element_int[i][j][k][l] indicates the integer part of the matrix element value at position (k, l) of the j-th matrix of the i-th matrix type.

[00295] matrix_element_dec[i][j][k][l] indicates the decimal part of the value of the matrix element at position (k, l) of the j-th matrix of the i-th matrix type.

[00296] matrix_element_sign_flag[i][j][k][l] indicates the sign of the matrix element at position (k, l) of the j-th matrix of the i-th matrix type. When matrix_element_sign_flag[i][j][k][l] is not present, it is inferred to be equal to 0.

[00297] The variable MatrixElementVal[i][j][k][l] representing the value Petition 870250090349, dated 03 / 10 / 2025, pp. 101 / 193 94 / 147 of the matrix element at position (k, l) of the j-th matrix of type j-th matrix is ​​derived as follows: MatrixElementVal[i][j][k][l] = (1-2* matrix_element_sign_flag[i][j][k][l]) * (matrix_element_int[i][j][k][l] + matrix_element_dec[i][j][k][l] .. 1<< (matrix_element_precision_factor_minus1 +1)

[00298] O processo DeriveInputTensors( ), para derivar o tensor de entrada inputTensorImgY, inputTensorImgCb, inputTesnsorImgCr, inputTensorKeyPoint, inputTensorMatrix é especificado conforme a seguir: Inicializa inputTensorImgY, inputTensorImgCb, inputTesnsorImgCr, inputTensorKeyPoint e inputTensorMatrix com 0. for ( x = 0; x< CroppedWidth; x++ ) { for ( y = 0; y< CroppedHeight; y++ ) { inputTsensorImgY[x][y] = CroppedYPic[x][y]}} for ( x=0; x< CroppedWidth / SubWidthC; x++ ) { for ( y=0; y< CroppedHeight / SubHeightC; y++ ) { inputTsensorImgCb[x][y] = CroppedCbPic[x][y] inputTsensorImgCr[x][y] = CroppedCrPic[x][y]}} if ( coordinate_present_flag ) { for ( i=0; i< =num_coordinates_minus1; i++ ) { inputTensorKeyPoint[i][0] = coordinateX[i] inputTensorKeyPoint[i][1] = coordinateY[i] if ( coordinate_z_present_flag ) inputTensorKeyPoint[i][2] = coordinateZ[i]}} Petition 870250090349, dated 03 / 10 / 2025, p. 102 / 193 95 / 147 if ( matrix_present_flag ) { for ( i=0; i<=num_matrix_types_minus1; i++ ) { for ( j=0; j< numMatrice[i]; j++ ) { for( k=0; k<= matrix_height_minus1[i]; k++ ) { for ( l=0;l<=matrix_width_minus1[i]; l++) { inputTensorMatrix[i][j][k][l] = MatrixElementVal[i][j][k][l]}} }} }

[00299] When gfv_nn_type_idc equals 1, the TranslateTensors() process is used to derive the input tensor TransTensorFlow for Decoder() from the output of Translator(); when gfv_nn_type_idc equals 2, the TranslateTensors() process derives the input tensor TransTensorKeyPoint and TransTensorMatrix for Decoder() from the output of Translator(). The process can be specified as follows: if ( gfv_nn_type_idc == 1 ) { for(x=0; x< CroppedWidth / SubWidthC; x++){ for(y=0; y< CroppedHeight / SubHeightC; y++){ TransTensorFlow [x][y][0] = outputensorFlow[x][y] [0] TransTensorFlow [x][y][1] = outputensorFlow[x][y] [1]}} if ( gfv_nn_type_idc == 2 ) { if ( coordinate_present_flag ) { for ( i=0; i< =num_coordinates_minus1; i++ ) { TransTensorKeyPoint [i][0] = outputensorKeyPoint [i] [0] TransTensorKeyPoint [i][1] = outputensorKeyPoint [i] [1] if ( coordinate_z_present_flag ) Petição 870250090349, de 03 / 10 / 2025, pág. 103 / 193 96 / 147 TransTensorKeyPoint [i][2] = outputensorKeyPoint [i] [2]}} if ( matrix_present_flag ) { for ( i=0; i<=num_matrix_types_minus1; i++ ) for ( j=0; j< numMatrice[i]; j++ ) for( k=0; k<= matrix_height_minus1[i]; k++ ) for ( l=0;l<=matrix_width_minus1[i]; l++) TransTensorMatrix [i][j][k][l] = outputensorMatrix [i][j][k][l]}}

[00300] The StoreOutputTensors() process, for deriving sample values ​​in the generated output sample matrices OutputYPic, OutputCbPic, and OutputCrPic from the output tensor outputTensorY, outputTensorCb, and outputTensorCr, is specified below: for(x=0; x< CroppedWidth; x++){ for(y=0; y< CroppedHeight; y++){ OutputYPic[x][y] = outputTensorY[x][y]} } for(x=0; x< CroppedWidth / SubWidthC; x++){ for(y=0; y< CroppedHeight / SubHeightC; y++){ OutputCbPic[x][y] = outputTensorCb[x][y] OutputCdPic[x][y] = outputTensorCr[x][y]}

[00301] The following process is used to generate a video image. Generator() is used to send the sample image values ​​with parameters signaled in the GFV SEI message and the decoded base image as inputs. Translator() is used to convert the facial parameters signaled in the SEI message into a map of Petition 870250090349, dated 03 / 10 / 2025, page 104 / 193 97 / 147 fluxo ou converter os parâmetros faciais sinalizados na mensagem SEI em um tipo especificado de parâmetros faciais. if(gfv_nn_mode_idc==0) { DeriveInputTensors( ) Generator ( inputTensorImgY, inputTensorImgCb, inputTesnsorImgCr, inputTensorKeyPoint, inputTensorMatrix) StoreOutputTensors( outputTensor)} else if (gfv_nn_mode_idc == 1) { DeriveInputTensors( ) Translator (inputTensorKeyPoint, inputTensorMatrix) TranslateTensors(outputensorFlow) outputTensor = Decoder ( inputTensorImgY, inputTensorImgCb, inputTesnsorImgCr, TransTensorFlow) StoreOutputTensors( )} else if (gfv_nn_mode_idc == 2) { DeriveInputTensors( ) Translator (inputTensorKeyPoint, inputTensorMatrix) TranslateTensors(outputensorKeyPoint, outputensorMatrix) outputTensor = Decoder ( inputTensorImgY, inputTensorImgCb, inputTesnsorImgCr, TransTensorFlow) StoreOutputTensors()}

[00302] Although the disclosed facial video compression and generation method are described above in connection with the SEI message, they are not limited to the SEI message. On the contrary, they can be Petition 870250090349, dated 03 / 10 / 2025, pp. 105 / 193 98 / 147 implemented in other ways. For example, an extension of the current video encoding standard or a new standard could be defined to support the proposed generative video compression. Videos could be encoded with the proposed method in a base layer or a high layer. And all the syntax, semantics, and decoding methods described above are applicable to the extension or new standard.

[00303] In some embodiments, the first subnetwork is a parameter translator network that translates the facial representation parameters signaled in the SEI message into a fixed parameter format, and the second subnetwork is a generative network that generates the image based on the fixed parameter format.

[00304] The first subnetwork (i.e., the parameter translator network), which is called TranslatorNN(), can be signaled in the SEI message or indicated by the URI contained in the SEI message. An indicator is set to show whether the translator network is signaled or indicated by the SEI message. In addition, the information in fixed parameter format (e.g., the number of keypoints, the number of arrays, and the size of each array) provided by the parameter translator can also be signaled in the SEI message or provided by the translator itself.

[00305] In addition, for the signaling of keypoint coordinates and matrix elements, the predictive signaling method is supported. That is, the differences between the keypoint or matrix element coordinates of the current frame and those of the base image are signaled. Similarly, an indicator is provided to show whether difference values ​​or absolute values ​​are signaled.

[00306] The syntax of a generative facial video SEI message shown in Table 10 can be encoded and signaled in a bitstream, and can be further decoded by the decoder. FIG. Petition 870250090349, dated 03 / 10 / 2025, pp. 106 / 193 Figure 99 / 147 is a schematic diagram illustrating an exemplary SEI message 1100, according to some embodiments of the present disclosure. As shown in FIG. 11, the SEI message 1100 may include, when used to transmit the facial information parameter, an identification number indicator (gfv_id) 1101, a key presence indicator (gfv_key_present_flag) 1102 indicating the presence of a key indicator, a key indicator (gfv_key) 1103, a coordinate indicator (coordinate_present_flag) 1104, a matrix indicator (matrix_present_flag) 1105, etc. The coordinate indicator 1104 and the matrix indicator 1105 are collaboratively signaled. That is, in a proper SEI message 1100, only one of the coordinate indicator 1104 and matrix indicator 1105 may be activated. In addition, parameter indicator 1106 can be generated and signaled according to the activated mode.

[00307] In some embodiments, the SEI 1100 message may include, when used to transmit the configuration of a neural network, an identification number indicator (gfv_id) 1101, a key presence indicator (gfv_key_present_flag) 1102 indicating the absence of a key indicator, a neural network mode indicator (gfv_nn_mode_idc) 1107, a neural network type indicator (gfv_nn_type_idc) 1108, etc. The corresponding bit length of these parameters is also shown in FIG. 11 as an example. As appreciated, the bit length of these parameters may be designed in another way. The other parameters are not shown in FIG. 11 to avoid causing any ambiguity.

[00308] An example of the common SEI message syntax for generative facial video is shown in Table 12. As shown in Table 12, a gfv_id identification number indicator can be generated by an encoder and signaled to a decoder. In some embodiments, gfv_id can be used to indicate whether SEI is used for Petition 870250090349, dated 03 / 10 / 2025, page 107 / 193 100 / 147 encode facial image.

[00309] As described above, the facial image can be reconstructed based on a base image. Therefore, the images encoded and signaled to the decoder can be classified as either the base images or the images used to generate the facial information parameter (also referred to in this document as the targeting image). It is appreciated that the facial information parameter may not be signaled in the SEI message associated with the base image. Therefore, the SEI message associated with the base image can be used to define (initialize or update) the neural network on the decoder side.

[00310] In some embodiments, the base image indicator (e.g., gfv_base_pic_flag as described below) may be included in the SEI message to indicate that the current SEI message is associated with a base image. Referring again to FIG. 7A, in step 706, the decoder can determine that the SEI message is applied to the neural network to generate a facial image when the base image indicator shows that the current SEI message is not associated with a base image. Otherwise, the decoder can determine that the current SEI message does not contain any facial information parameters.

[00311] In some embodiments, when it is determined that the current SEI message is associated with a base image, the current SEI message can be used to define (initialize or update) the neural network on the decoder side. Additionally, the neural network presence indicator (e.g., gfv_nn_present_flag as described below) can be included in the SEI message to indicate whether the first subnetwork information is indicated by the SEI message. In some embodiments, the first subnetwork can be defined (initialized or updated) based on the neural network indicator (e.g., Petition 870250090349, dated 03 / 10 / 2025, pp. 108 / 193 101 / 147 gfv_nn_base_flag, gfv_nn_mode_idc) when the base image indicator shows that the SEI message is associated with a base image and the neural network presence indicator shows that the first subnetwork information is indicated by the SEI message. Table 12: An example of the SEI message syntax for generative facial video compression. generative_face_video ( payloadSize ) { Descriptor gfv_id ue(v) gfv_base_pic_flag / *indicates whether the current decoded output image is a base image* / u(1) if( gfv_base_pic_flag ) { / *specify TranslatorNN( )* / gfv_nn_present_flag u(1) if( gfv_nn_present_flag ) { gfv_nn_base_flag u(1) gfv_nn_mode_idc ue(v) if( gfv_nn_mode_idc = = 1 ) { while( !byte_aligned( ) ) gfv_nn_reserved_zero_bit_a u(1) gfv_nn_tag_uri st(v) gfv_nn_uri st(v)}}} else / * the output image current decoded is a drive image* / gfv_drive_pic_fusion_flag / *indicates whether DrivePicture is inserted into GenerativeNN()* / ue(v) gfv_coordinate_present_flag u(1) Petition 870250090349, dated 03 / 10 / 2025, p. 109 / 193 102 / 147 generative_face_video ( payloadSize ) { Descritor if( gfv_coordinate_present_flag ) { gfv_coordinate_precision_factor_minus1 ue(v) gfv_n um_kp_m inus 1 ue(v) gfv_kp_pred_flag u(1) gfv_coordinate_z_present_flag u(1) if(gfv_coordinate_z_present_flag ) gfv_coordinate_z_max_value_minus1 ue(v) for( i = 0; i <= num_kp_minus1; i++ ) { if(!gfv_kp_pred_flag) { gfv_coordinate_x_abs[i] u(v) if( gfv_coordinate_x_abs[i]) gfv_coordinate_x_sign_flag[i] u(1) gfv_coordinate_y_abs[i] u(v) if( gfv_coordinate_y_abs[i]) gfv_coordinate_y_sign_flag[i] u(1) if( gfv_coordinate_z_present_flag ) { gfv_coordinate_z_abs[i] u(v) if( gfv_coordinate_z_abs[i] ) gfv_coordinate_z_sign_flag[i] u(1)}} else { gfv_coordinate_dx_abs[i] u(v) if(gfv_coordinate_dx_abs[i] ) gfv_coordinate_dx_sign_flag[i] u(1) Petição 870250090349, de 03 / 10 / 2025, pág. 110 / 193 103 / 147 generative_face_video ( payloadSize ) { Descritor gfv_coordinate_dy_abs[i] u(v) if( gfv_coordinate_dy_abs[i] ) gfv_coordinate_dy_sign_flag[i] u(1) if( gfv_coordinate_z_present_flag ) { gfv_coordinate_dz_abs[i] u(v) if( gfv_coordinate_dz_abs[i] ) gfv_coordinate_dz_sign_flag[i] u(1)}}}} gfv_matrix_present_flag u(1) if(gfv_matrix_present_flag ) { gfv_matrix_element_precision_factor_minus1 ue(v) gfv_num_matrix_types_minus1 ue(v) if( !gfv_base_pic_flag ) gfv_matrix_pred_flag u(1) for( i = 0; i <= num_matrix_types_minus1; i++ ) { gfv_matrix_type_idx[i] u(6) if( gfv_matrix_type_idx[i] = =0 | | gfv_ma- trix_type_idx[i] = = 1 ) { if( gfv_coordinate_present_flag ) gfv_num_matrices_equal_to_num_kps_flag[i] u(1) Petição 870250090349, de 03 / 10 / 2025, pág. 111 / 193 104 / 147 generative_face_video ( payloadSize ) { Descritor if( !gfv_coordinate_present_flag | | !gfv_num_matrix_equal_to_num_kps_flag[ i ] ) gfv_num_matrices_info[i] ue(v)}else if( gfv_matrix_type_idx[i] = = 2 | | gfv_ma- trix_type_idx[i] = = 3 | | gfv_matrix_type_idx[i] >= 7 ){ if( gfv_matrix_type_idx[i] >= 7 ) gfv_num_matrices_minus1 [i] ue(v) gfv_matrix_width_minus1 [i] ue(v) gfv_matrix_height_minus1[i] ue(v)}else if( gfv_matrix_type_idx[i] >= 4 && gfv_ma- trix_type_idx[i] <= 6 && !gfv_coordinate_present_flag ){ gfv_matrix_for_3D_space_flag[i] u(1) for( j = 0; j < numMatrices[i]; j++ ) for( k = 0; k < matrixHeight[i]; k++ ) for( m = 0; m <matrixWidth[i]; m++ ) { if( !gfv_matrix_pred_flag ) { gfv_matrix_element_int[i][j][k][m] ue(v) gfv_matrix_element_dec[i][j][k][m] u (v) if( gfv_matrix_ele- ment_int[i][j][k][m] | | gfv_matrix_element_dec[i][j][k][m] ) gfv_matrix_element_sign_flag[i][j][k][m] u(1)} else { gfv_matrix_delta_element_int[i][j][k][m] gfv_matrix_delta_element_dec[i][j][k][m] Petição 870250090349, de 03 / 10 / 2025, pág. 112 / 193 105 / 147 generative_face_video ( payloadSize ) { Descritor if( gfv_matrix_ele- ment_int[i][j][k][m] | | gfv_matrix_element_dec[i][j][k][m] ) gfv_matrix_delta_element_sign_flag[i][j][k][m]}}}}} if( gfv_nn_present_flag ) { gfv_nn_output_info_present_flag if( gfv_nn_output_info_present_flag { gfv_nn_output_num_kps ue(v) gfv_nn_output_num_matrices ue(v) for( i=0; i< gfv_nn_output_matrix_num; i++) { gfv_nn_output_matrix_width_minus1 [i] ue(v) gfv_nn_output_matrix_height_minus1 [i] ue(v)}}} if( gfv_nn_present_flag ) if( gfv_nn_mode_idc = = 0 ) { while( !byte_aligned( ) ) gfv_nn_reserved_zero_bit_b u(1) Petição 870250090349, de 03 / 10 / 2025, pág. 113 / 193 106 / 147 generative_face_video ( payloadSize ) { Descritor for( i = 0; more_data_in_payload( ); i++ ) gfv_nn_payload_byte[i] b(8)}}

[00312] The Generative Facial Video (GFV) SEI message indicates the facial parameters and specifies a facial parameter translator network called TranslatorNN(), which can be used to convert various facial parameter formats signaled in the SEI message into a fixed parameter format, and a facial image generator neural network called GenerativeNN(), which can be used to generate output images using the fixed facial parameter format and previously decoded output images.

[00313] Note that facial parameters could be determined from source images prior to encoding. Such source images can be referred to as targeting images.

[00314] In addition, previously decoded output images introduced into GenerativeNN() can be a base image (a decoded output image that provides the reference texture from which facial images can be generated) and an optional image that can be merged by GenerativeNN() to enhance background texture and facial details. When the current image is not a base image, the GFV SEI message can be used to generate a facial image based on the previously decoded base image, the facial parameters transmitted by the GFV SEI message, and the optional current decoded image for fusion purposes.

[00315] The use of this SEI message requires the definition of the following variables: - Input image width and height in units of Petition 870250090349, dated 03 / 10 / 2025, pp. 114 / 193 107 / 147 luma samples, indicated in this document by CroppedWidth and CroppedHeight, respectively. - Luma sample matrix baseCroppedYPic and chroma sample matrices baseCroppedCbPic and baseCroppedCrPic for a decoded output image, called BasePicture, corresponding to a source base image. - Luma sample matrix driveCroppedYPic and chroma sample matrices driveCroppedCbPic and driveCroppedCrPic for a decoded output image, called DrivePicture, corresponding to a source targeting image. - Bit depth (BitDepthy) for the luma sample array of the input images. - BitDepthC for the chroma sample matrices, if any, of the input images. - A chroma format indicator, indicated in this document by ChromaFormatIdc, as described in subclause 7.3. The variables SubWidthC and SubHeightC are derived from ChromaFormatIdc.

[00316] gfv_id contains an identification number that can be used to identify facial feature information and specify a neural network that can be used as GenerativeNN(). The gfv_id value can be in the range of 0 to 232-2, inclusive. gfv_id values ​​from 256 to 511, inclusive, and from 231 to 232-2, inclusive, are reserved for future use by ITU-T | ISO / IEC. Decoders that encounter a GFV SEI message with gfv_id in the range of 256 to 511, inclusive, or in the range of 231 to 232-2, inclusive, can ignore the SEI message.

[00317] Note that the different gfv_id values ​​in the different GFV SEI messages could be used to identify different faces when more than one face is present in an output image. Petition 870250090349, dated 03 / 10 / 2025, pp. 115 / 193 108 / 147 for example.

[00318] In some modes, gfv_base_pic_flag equal to 1 indicates that the current decoded output image matches a baseline image. gfv_base_pic_flag equal to 0 indicates that the current decoded output image does not match a baseline image.

[00319] The following restrictions apply to the gfv_base_pic_flag value: - When a GFV SEI message is the first GFV SEI message, in the decoding order, that has a particular gfv_id value within the current CLVS, the gfv_base_pic_flag value can be equal to 1. - When a GFV SEI message that has a specific gfv_id value has gfv_base_pic_flag equal to 0, this SEI message belongs to the current decoded image and all subsequent decoded images of the current layer, in output order, up to the end of the current CLVS or up to, but excluding, the decoded image that follows the current decoded image in output order within the current CLVS and is associated with a subsequent GFV SEI message, in decoding order, having gfv_base_pic_flag equal to 0 and that particular gfv_id value within the current CLVS, whichever occurs first.

[00320] In some embodiments, gfv_nn_present_flag equal to 1 indicates that a neural network that can be used as TranslatorNN() is contained in or indicated by the SEI message. A value of 0 for gfv_nn_present_flag indicates that a neural network that can be used as a TranslatorNN() is not contained in or indicated by the SEI message.

[00321] gfv_nn_base_flag, gfv_nn_mode_idc, gfv_nn_reserved_zero_bit_a, gfv_nn_tag_uri, gfv_nn_uri, gfv_nn_payload_byte[i] specify a neural network that can be used as TranslatorNN( ). gfv_nn_base_flag, gfv_nn_mode_idc, gfv_nn_reserved_zero_bit_a, gfv_nn_tag_uri, gfv_nn_uri, gfv_nn_payload_byte[i] have the same Petition 870250090349, dated 03 / 10 / 2025, pp. 116 / 193 109 / 147 syntax and semantics that nnpfc_base_flag, nnpfc_mode_idc, nnpfc_reserved_zero_bit_a, nnpfc_tag_uri, nnpfc_uri, nnpfc_payload_byte[i], respectively.

[00322] In some modes, gfv_drive_pic_fusion_flag, when present, equal to 1 indicates that the current decoded image, which corresponds to a targeting image that can be used for fusion, can be introduced into GenerativeNN(). gfv_drive_pic_fusion_flag equal to 0 indicates that the current decoded image should not be introduced into GenerativeNN().

[00323] Note that a gfv_drive_pic_fusion_flag value of 1 can be used, for example, to indicate that the current decoded image can be used to enhance facial details and manipulate background changes.

[00324] Note that the fusion considers the three inputs: the base image, the characteristics of the key points or matrices contained in the GFV SEI message, and the current decoded image, and sends an image.

[00325] Note that when the current decoded image matches a targeting image, it should be marked as not intended for output purposes.

[00326] gfv_coordinate_present_flag equal to 1 indicates that keypoint coordinate information is present. gfv_coordinate_present_flag equal to 0 indicates that keypoint coordinate information is not present.

[00327] It is a bitstream conformance requirement that when gfv_matrix_type_idx[i] any value of i from 0 to gfv_num_matrix_types_minus1 is equal to 0 or 1, the value of gfv_coordinate_present_flag can be equal to 1.

[00328] gfv_coordinate_precision_factor_minus1 plus 1 indicates the length, in bits, of gfv_coordinate_x_abs[i], gfv_coordinate_y_abs[i] and gfv_coordinate_z_abs[i]. Petition 870250090349, dated 03 / 10 / 2025, p. 117 / 193 110 / 147

[00329] gfv_num_kps_minus1 plus 1 indicates the number of keypoints. The value of gfv_num_kp_minus1 can be in the range of 0 to 210-1, inclusive.

[00330] gfv_kp_pred_flag equal to 1 indicates that the syntax elements gfv_coordinate_dx_abs[i], gfv_coordinate_dy_abs[i], and gfv_coordinate_dz_abs[i] are present and the syntax elements gfv_coordinate_dx_sign_flag[i], gfv_coordinate_dy_sign_flag[i] and gfv_coordinate_dz_sign_flag[i] may be present. A value of gfv_kp_pred_flag equal to 0 indicates that gfv_coordinate_x_abs[i], gfv_coordinate_y_abs[i], and gfv_coordinate_z_abs[i] are present, and the syntax elements gfv_coordinate_x_sign_flag[i], gfv_coordinate_y_sign_flag[i], and gfv_coordinate_z_sign_flag[i] may also be present.

[00331] gfv_coordinate_z_present_flag equal to 1 indicates that the z-axis coordinate information for the key points is present. coordinate_z_present_flag equal to 0 indicates that the z-axis coordinate information for the key points is not present.

[00332] gfv_coordinate_z_max_value_minus1 plus 1 indicates the maximum absolute value of the z-axis coordinates of the key points.

[00333] gfv_coordinate_x_abs[i] indicates the normalized absolute value of the x-axis coordinate of the i-th keypoint.

[00334] gfv_coordinate_x_sign_flag[i] specifies the sign of the x-axis coordinate of the i-th keypoint. When gfv_coordinate_x_sign_flag[i] is not present, it is preferred to be equal to 0.

[00335] gfv_coordinate_y_abs[i] specifies the normalized absolute value of the y-axis coordinate of the i-th keypoint.

[00336] gfv_coordinate_y_sign_flag[i] specifies the y-axis coordinate sign of the i-th keypoint. When gfv_coordinate_y_sign_flag[i] is not present, it is inferred to be equal to 0.

[00337] gfv_coordinate_z_abs[i] specifies the normalized absolute value of the z-axis coordinate of the i-th keypoint. Petition 870250090349, dated 03 / 10 / 2025, pp. 118 / 193 111 / 147

[00338] gfv_coordinate_z_sign_flag[i] specifies the z-axis coordinate sign of the i-th keypoint. When gfv_coordinate_z_sign_flag[i] is not present, it is inferred to be equal to 0.

[00339] gfv_coordinate_dx_abs[i] indicates the absolute difference value of the normalized x-axis coordinate of the i-th keypoint.

[00340] gfv_coordinate_dx_sign_flag[i] specifies the sign of the difference value of the x-axis coordinate of the i-th keypoint. When gfv_coordinate_dx_sign_flag[i] is not present, it is inferred to be equal to 0.

[00341] gfv_coordinate_dy_abs[i] specifies the absolute difference value of the normalized y-axis coordinate of the i-th keypoint.

[00342] gfv_coordinate_dy_sign_flag[i] specifies the sign of the y-coordinate difference value of the i-th keypoint. When gfv_coordinate_yd_sign_flag[i] is not present, it is inferred to be equal to 0.

[00343] gfv_coordinate_dz_abs[i] specifies the absolute difference value of the normalized z-axis coordinate of the i-th keypoint.

[00344] gfv_coordinate_dz_sign_flag[i] specifies the sign of the z-axis coordinate difference value of the i-th keypoint. When gfv_coordinate_dz_sign_flag[i] is not present, it is inferred to be equal to 0.

[00345] The variables coordinateDeltaX[i], coordinateDeltaY[i], and coordinateDeltaZ[i], indicating the x-axis delta coordinate, y-axis delta coordinate, and z-axis delta coordinate of the i-th keypoint, respectively, are derived as follows: coordinateDeltaX[i] = (1-2 * gfv coordinate dx sign flag[i]) * gfv coordinate dx abs\i\ * CroppedWidth 1<< (gfv_coordinate_precision_f actor_minus1 +1) coordinateDeltaY[i] = (1-2 * gfv coordinate dy sign flag[i]') * gfv coordinate dy abs\i\ * CroppedHeight 1<< (gfv_coordinate_precision_f actor_minus1 +1) Petition 870250090349, dated 03 / 10 / 2025, pp. 119 / 193 112 / 147 coordinateDeltaZ[i] = (1-2 * coordinate dz sign flag[i]') *gfv coordinate dz abs[i] *'-(gfv co'jrdiriate z max vatiie mitiusl+L) 1« (gfv_coordinatej>recision_factor_minusl +1)

[00346] The variables coordinateX[i], coordinateY[i] and coordinateZ[i] indicating the x-axis coordinate, y-axis coordinate and z-axis coordinate of the i-th keypoint are derived, respectively, as follows: When gfv_kp_pred_flag is equal to 0, coordinate inateX[i] = (1-2 * gfv coordinate x sign flag\i\) *gfv coordinate x abs\i \ * CroppedWidth 1« (gfv_coordinate_precision_f actor _minusl +1) coordinate inateY[i] = (1-2 * gfv coordinate y sign flag\i\) * gfv coordinate y abs[i] * CroppedHeight 1« (gfv_coordinate_precision_f actor_minusl +1) coord inateZ[i] = (1-2 * gfv coordinate z sign flag[i]')^ / v coordinate z abs[i] *(gfv coordinate z max value minusl+l) 1« (gfv_coordínate_precísioTiJactor_minusl +1) when gfv_kp_pred_flag is equal to 1, if( gfv_base_pic_flag) { coordinateX[i] = ((i>0) ? coordinateX[i-1]: 0 ) + coordinateDeltaX[i] coordinateY[i] = ((i>0) ? coordinateY[i-1]: 0 ) + coordinateDeltaY[i] coordinateZ[ i ] = ((i>0) ? coordinateZ[i-1]: 0 ) + coordinateDeltaZ[i]} else { coordinate inateX[i] = BaseKpCoordinateX[i] + coordinateDeltaX[i] coordinate inateY[i] = BaseKpCoordinateY[i] + coordinateDeltaY[i] coordinate inateZ[i] = BaseKpCoordinateZ[i] + coordinateDeltaZ[i]} where BaseKpCoordinateX[i], BaseKpCoordinateY[i], BaseKpCoordinateZ[i] indicating the coordinates of the x-axis, y-axis and z-axis, respectively, of the i-th keypoint for the base image are derived as follows: Petition 870250090349, dated 03 / 10 / 2025, pp. 120 / 193 113 / 147 if( gfv_base_pic_flag ) { BaseKpCoordinateX[i] = coordinateX[i] BaseKpCoordinateY[i] = coordinateY[i] BaseKpCoordinateZ[i] = coordinateZ[i]}

[00347] gfv_matrix_present_flag equal to 1 indicates that the matrix parameters are present. gfv_matrix_present_flag equal to 0 indicates that the matrix parameters are not present.

[00348] gfv_matrix_element_precision_factor_minus1 plus 1 indicates the length, in bits, of gfv_matrix_element_dec[i][j][k][m].

[00349] gfv_num_matrix_types_minus1 plus 1 indicates the number of matrix types signaled in the SEI message. The value of gfv_matrix_type_num_minus1 can be in the range of 0 to 26-1, inclusive.

[00350] gfv_matrix_pred_flag equal to 1 indicates that the syntax elements gfv_matrix_element_int [i][j][k][m], gfv_matrix_element_dec [i][j][k][m], gfv_matrix_element_sign_flag [i][j][k][m] may be present. gfv_matrix_pred_flag equal to 0 indicates that gfv_matrix_delta_element_int [i][j][k][m], gfv_matrix_delta_element_dec [i][j][k][m] are present and the syntax element gfv_matrix_delta_element_sign_flag [i][j][k][m] may be present. When gfv_matrix_pred_flag is not present, the value of gfv_matrix_pred_flag is inferred to be 0.

[00351] gfv_matrix_type_idx[i] indicates the index of the i-th matrix type as specified in Table 13 below. Table 13: Specification of gfv_matrix_type_idx Value Specification 0 Affine translation matrix with size 2*2 or 3*3. 1 Covariance matrix with size 2*2 or 3*3. 2 Mouth matrix representing mouth movement Petition 870250090349, dated 03 / 10 / 2025, pp. 121 / 193 114 / 147 Value Specification 3 Eye matrix representing the open-close state and eye level. 4 Head rotation parameters with a size of 2*2 or 3*3 representing head rotation in 2D or 3D space. 5 Head translation matrix with a size of 1*2 or 1*3 representing head translation in 2D or 3D space. 6 Head location matrix with a size of 1*2 or 1*3 representing head location in 2D or 3D space. 7 Compact feature matrix with size specified by gfv_matrix_width_minus1[i] and gfv_matrix_height_minus1[i]. 8.31 Another matrix that can be used as determined by the application with size specified by gfv_matrix_width_minus1[i] and gfv_matrix_height_minus1[i]. 32.63 Reserved

[00352] Note that the undefined matrix type is used to represent the matrix type that is not an affine translation matrix, covariance matrix, rotation matrix, translation matrix, or compact feature matrix. It can be used by the user to extend the matrix type.

[00353] gfv_num_matrices_equal_to_num_kps_flag[i] equal to 1 indicates that the number of matrices of the i-th matrix type is equal to gfv_num_kps_minus1 + 1. gfv_num_matrices_equal_to_num_kps_flag[i] equal to 0 indicates that the number of matrices of the i-th matrix type is not Petition 870250090349, dated 03 / 10 / 2025, pp. 122 / 193 115 / 147 is equal to gfv_num_coordinates_minus1 + 1.

[00354] gfv_num_matrices_info[i] provides information for deriving the number of matrices of type i-th matrix.

[00355] gfv_matrix_width_minus1[i] plus 1 indicates the width of the matrix of the i-th matrix type.

[00356] gfv_matrix_height_minus1[i] plus 1 indicates the height of the matrix of the i-th matrix type.

[00357] gfv_matrix_for_3D_space_flag[i] equal to 1 indicates that the matrix of type i-th matrix is ​​a matrix defined in three-dimensional space. gfv_matrix_for_3D_space_flag[i] equal to 0 indicates that the matrix of type i-th matrix is ​​a matrix defined in two-dimensional space.

[00358] When gfv_matrix_width_minus1[i] is not present, it is inferred as follows: If gfv_matrix_type_idx[i] equals 0, 1, or 4, and one of coordinate_z_present_flag and gfv_matrix_for_3D_space_flag[i] is present and equals 1, then gfv_matrix_width_minus1[i] is inferred to be equal to 2. Otherwise, if matrix_type_idx[i] equals 0, 1, or 4, and one of coordinate_z_present_flag and gfv_matrix_for_3D_space_flag[i] is present and equals 0, then gfv_matrix_width_minus1[i] is inferred to be equal to 1. Otherwise (if matrix_type_idx[i] equals 5 or 6), gfv_matrix_width_minus1[i] is inferred to be 0. When gfv_matrix_height_minus1[i] is not present, it is inferred as follows: — If matrix_type_idx is equal to 0, 1, 4, 5, or 6, and one of gfv_coordinate_z_present_flag and gfv_matrix_for_3D_space_flag[i] is present and equal to 1, gfv_matrix_height_minus1[i] is inferred to be equal to 2. — otherwise (gfv_matrix_type_idx equals 0, 1, 4, 5 or Petition 870250090349, dated 03 / 10 / 2025, pp. 123 / 193 116 / 147 6, and one of gfv_coordinate_z_present_flag and gfv_matrix_for_3D_space_flag[i] is 0), gfv_matrix_height_minus1[i] is inferred to be equal to 1.

[00359] The variables matrixWidth[i] and matrixHeight[i], indicating the width and height of the i-th matrix, are derived as follows: matrixWidth[i] = gfv_matrix_width_minus1[i] + 1 matrixHeight[i] = gfv_matrix_height_minus1[i] + 1

[00360] gfv_num_matrices_minus1[i] plus 1 indicates the number of matrices of the i-th matrix type.

[00361] The variable numMatrices[i] indicating the number of matrices of the i-th matrix type is derived as follows: if( gfv_matrix_type_idx[i] == 0 || gfv_matrix_type_idx[i] == 1 ) { if( gfv_coordinate_present_flag ) numMatrices[i] = gfv_num_matrices_equal_to_num_kps_flag[i] ? gfv_num_kps_minus1 + 1 : ( gfv_num_matrices_info[i] < gfv_num_kp_ minus1 ? gfv_num_matrices_info [i] + 1 : gfv_num_matrices_info [i] + 2 ) else numMatrices[i] = gfv_num_matrices_info[i] + 1} else if( gfv_matrix_type_idx[i] >= 2 && gfv_matrix_type_idx[i] < 7) numMatrices[i] = 1 else numMatrices[i] = gfv_num_matrices_minus1[i] + 1

[00362] In some of the disclosed modalities, the position (k, m) represents the position in the k-th row and in the m-th column of the matrix.

[00363] gfv_matrix_element_int[I][j][k][m] indicates the integer part of the value of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type.

[00364] gfv_matrix_element_dec[I][j][k][m] indicates the decimal part of the Petition 870250090349, dated 03 / 10 / 2025, pp. 124 / 193 117 / 147 value of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type.

[00365] gfv_matrix_element_sign_flag[i][j][k][m] indicates the sign of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type. When gfv_matrix_element_sign_flag[i][j][k][m] is not present, it is inferred to be equal to 0.

[00366] gfv_matrix_element_int[i][j][k][m] indicates the integer part of the difference value of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type.

[00367] gfv_matrix_element_dec[i][j][k][m] indicates the decimal part of the difference value of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type.

[00368] gfv_matrix_element_sign_flag[i][j][k][m] indicates the sign of the difference value of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type. When gfv_matrix_element_sign_flag[i][j][k][m] is not present, it is inferred to be equal to 0.

[00369] gfv_matrix_pred_flag indicates whether the values ​​of the matrix elements are predictively flagged. If the flag is equal to 1, the difference of the matrix element values ​​is flagged. The flagged difference value can be the differences between the matrix element values ​​of the current image and the matrix element values ​​of the base image. Alternatively, the flagged difference value can be the difference between a matrix element value in an image and a previous matrix element value in the image. Similarly, the differences between matrix element values ​​in the current image and matrix element values ​​in the base image can be derived first, and then a second difference between the first matrix element difference value and the first previous matrix element difference value is derived and flagged. Petition 870250090349, dated 03 / 10 / 2025, p. 125 / 193 118 / 147

[00370] As an example, the variable matrixElementDeltaVal[i][j][k][m] representing the difference value of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type is derived as follows: matrixElementDeltaVal[i][j][k][m] = (1 — 2 * gfv_matrix_delta_element_sign_flag[i][j][k][m]') * (gfv_matrix_delta_element_int[i] [j][k] [m] gfv_matrix_delta_element_dec[i] [j] [k] [m]+1 << (gfv_matrix_element_precision_factor_minus1 + 1))

[00371] The variable matrixElementVal[i][j][k][m] representing the value of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type is derived as follows: when gfv_matrix_pred_flag equals 0: matrixElementVal[i][j][k][m] = (1—2* gfv_matrix_element_sign_flag[i][j][k][m]') * (gfv_m.atrix_element_int[i] [j] [k] [m] + gfv_matrix_element_dec[i] [j] [k] [m] .. 1 << (gfv_matrix_element_precision_factor_minus1 + 1) when gfv_matrix_pred_flag equals 1: matrixElementVal[i][j][k][m] = BaseMatrixElementVal[i][j][k][m] + matrixElementDeltaVal[i][j][k][m] + if( gfv_base_pic_flag ) BaseMatrixElementVal[i][j][k][m] = matrixElementVal[i][j][k][m]

[00372] In another example, the variable matrixElementDeltaVal[i][j][k][m] representing the difference value of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type is derived as follows: matrixElementDeltaVal[i][j][k][m] = Petition 870250090349, dated 03 / 10 / 2025, pp. 126 / 193 119 / 147 (1-2* gfv_matrix_delta_element_sign_flag[i][j][k][m]') * (gfv_matrix_delta_element_int[i] [j][k] [m] gfv_matrix_delta_element_dec[i] [j] [k] [m] + 1 << (gfv_matrix_element_precision_factor_minus1 + 1))

[00373] The variable matrixElementVal[i][j][k][m] representing the value of the matrix element at position (k, m) of the j-th matrix of the i-th matrix type is derived as follows: when gfv_matrix_pred_flag equals 0 matrixElementVal[i][j][k][m] = (1 - 2 * gfv_matrix_element_sign_f lag[i] [j] [k] [m]) * (gfv_matrix_element_int[i] [j] [k] [m] + gfv_matrix_element_dec[i] [j] [k] [m] .. 1<< (gfv_matrix_element_precision_factor_minus1 +1) when gfv_matrix_pred_flag is equal to 1 matrixElementVal[i][j][k][m] = BaseMatrixElementVal[i][j][k][m] + matrixElementDeltaVal[i][j][k][m] + if( gfv_base_pic_flag ) { for( k = 0; k < matrixHeight[i]; k++) { for(m = 0; m <matrixWidth[i]; m++ ) { tempVal[ i][ j ][ k ][ m ] = ((k==0) ? matrixElementVal[i][j][k][m] : (tempVal[i][j][k-1][m] + matrixElementVal[i][j][k][m]))} } for( k = 0; k < matrixHeight[i]; k++) { for( m = 0; m <matrixWidth[i]; m++ ) { BaseMatrixElementVal[i][j][k][m] = ((m==0) ? tempVal[i][j][k][m] : (BaseMatrixElementVal [i][j][k][m] + tempVal[i][j][k][m]))}}}

[00374] In some modes, gfv_nn_output_info_present_flag Petition 870250090349, dated 03 / 10 / 2025, p. 127 / 193 120 / 147 equal to 1 indicates that the syntax elements gfv_nn_output_num_kps, gfv_nn_output_num_matrices, gfv_nn_output_matrix_width_minus1[i], and gfv_nn_output_matrix_height_minus1[i] may be present. gfv_nn_output_info_present_flag equal to 0 indicates that the syntax elements gfv_nn_output_num_kps, gfv_nn_output_num_matrices, gfv_nn_output_matrix_width_minus1[i], and gfv_nn_output_matrix_height_minus1[i] are not present, which can be used to define the first subnet.

[00375] gfv_nn_output_num_kps indicates the number of keypoints sent by TranslatorNN(). The value of gfv_nn_output_num_kps can be in the range of 0 to 210, inclusive.

[00376] gfv_nn_output_num_matrices indicates the number of matrices sent by TranslatorNN(). The value of gfv_nn_output_num_matrices can be in the range of 0 to 210, inclusive.

[00377] gfv_nn_output_matrix_width_minus1[i] plus 1 indicates the width of the i-th matrix sent by TranslatorNN(). The value of gfv_nn_output_matrix_width_minus1[i] can be in the range of 0 to 210-1, inclusive.

[00378] gfv_nn_output_matrix_height_minus1[i] plus 1 indicates the height of the i-th matrix sent by TranslatorNN(). The value of gfv_nn_output_matrix_height_minus1[i] can be in the range of 0 to 210-1, inclusive.

[00379] The following process is used to generate a video image: DeriveSigParam() TranslatorNN (sigKeyPoint, sigMatrix) DeriveInputTensors( ) if( gfv_base_pic_flag == 0 && gfv_drive_pic_fusion_flag == 0) { if(ChromaFormatIdc == 0 ) GenerativeNN( inputBaseY, inputBaseKeyPoint, inputBaseMatrix, Petição 870250090349, de 03 / 10 / 2025, pág. 128 / 193 121 / 147 inputDriveKeyPoint, inputDriveMatrix) else GenerativeNN( inputBaseY, inputBaseCb, inputBaseCr, inputBaseKeyPoint, inputBaseMatrix, inputDriveKeyPoint, inputDriveMatrix)} else if(gfv_base_pic_flag == 0 && gfv_drive_pic_fusion_flag == 1) { if(ChromaFormatIdc == 0 ) GenerativeNN( inputBaseY, inputDriveY, inputBaseKeyPoint, inputBaseMatrix, inputDriveKeyPoint, inputDriveMatrix) else GenerativeNN( inputBaseY, inputBaseCb, inputBaseCr, inputDriveY, inputDriveCb, inputDriveCr, inputBaseKeyPoint, inputBaseMatrix,, inputDriveKeyPoint, inputDriveMatrix)} StoreOutputTensors( )

[00380] O processo DeriveSigParam () para derivar as entradas de TranslatorNN () é especificado a seguir: A matriz de coordenadas de pontos-chave sigKeyPoint e a matriz sigMatrix são derivadas a seguir: if( gfv_coordinate_present_flag ) { for ( i=0; i< = gfv_num_kps_minus1; i++ ) { sigKeyPoint[i][0] =coordinateX[i] sigKeyPoint[i][1] = coordinateY[i] if ( gfv_coordinate_z_present_flag ) sigKeyPoint[i][2] = coordinateZ[i]} } else { for ( i=0; i< =num_kps_minus1; i++ ) { sigKeyPoint [i][0] = 0 Petição 870250090349, de 03 / 10 / 2025, pág. 129 / 193 122 / 147 sigKeyPoint [i][1] = 0 if ( gfv_coordinate_z_present_flag ) sigKeyPoint [i][2] = 0} } if( gfv_matrix_present_flag ) { for ( i=0; i<= gfv_num_matrix_types_minus1; i++ ) { for ( j=0; j< numMatrices[i]; j++ ) { for ( k=0; k< matrixHeight [i]; k++ ) { for ( l=0;l< matrixWidth [i]; l++) { sigMatrix[i][j][k][l] = matrixElementVal[i][j][k][l]}}}}} else { for ( i=0; i<=gfv_num_matrix_types_minus1; i++ ) { for ( j=0; j< numMatrices[i]; j++ ) { for( k=0; k< matrixHeight [i]; k++ ) { for ( l=0;l< matrixWidth [i]; l++) { sigMatrix [i][j][k][l] = 0} }} }

[00381] As an example, TranslatorNN() is a process for translating the various formats of facial parameters contained in the SEI message into the fixed format of facial parameters to be inserted into Petition 870250090349, dated 03 / 10 / 2025, pp. 130 / 193 123 / 147 generative network to generate the output image.

[00382] The input to TranslatorNN() includes sigKeyPoint and sigMatrix. And the output of TranslatorNN() includes convKeyPoint and convMatrix.

[00383] The DeriveInputTensors() process for deriving the GenerativeNN() inputs is specified below: When gfv_base_pic_flag equals 1, the input tensors inputBaseY, inputBaseCb, and inputBaseCr of BasePicure are derived as follows: for( x = 0; x< CroppedWidth; x++ ) { for ( y = 0; y< CroppedHeight; y++ ) { inputBaseY[x][y] = InpY( baseCroppedYPic[x][y] )} } if (ChromaFormatIdc !=0) { for( x=0; x< CroppedWidth / SubWidthC; x++ ) { for ( y=0; y< CroppedHeight / SubHeightC; y++ ) { inputBaseCb[x][y] = InpC( baseCroppedCbPic[x][y] ) inputBaseCr[x][y] = InpC( baseCroppedCrPic[x][y] )} }} When gfv_drive_pic_fusion_flag equals 1, the sample matrix luma inputDriveY, inputDriveCb of DrivePicture and the DriveCr input are derived as follows: for( x = 0; x< CroppedWidth; x++ ) { for ( y = 0; y< CroppedHeight; y++ ) { inputDriveY[x][y] = InpY( driveCroppedYPic[x][y] )} } if (ChromaFormatIdc != 0) { Petition 870250090349, dated 03 / 10 / 2025, pp. 131 / 193 124 / 147 for( x=0; x< CroppedWidth / SubWidthC; x++ ) { for ( y=0; y< CroppedHeight / SubHeightC; y++ ) { InputDriveCb[x][y] = InpC( driveCroppedCbPic[ x][ y ] ) InputDriveCr[x][y] = InpC( driveCroppedCrPic[ x][ y ] )} }} When gfv_base_pic_flag equals 0, the keypoint coordinate matrix inputDriveKeyPoint and the inputDriveMatrix matrix for the current image are derived as follows: for ( i = 0; i <= gfv_nn_output_num_kps; i++ ) { inputDriveKeyPoint[i][0] = convKeyPoint[i][0] inputDriveKeyPoint [i][1] = convKeyPoint[i][0] inputDriveKeyPoint [i][2]] = convKeyPoint[i][0]} for( i=0; i <gfv_nn_output_num_matrices; i++) for( k = 0; k < gfv_nn_output_matrix_height_minus1[i]; k++ ) { for ( l = 0;l < gfv_nn_output_matrix_width_minus1[i]; l++) inputDriveMatrix[i][k][l] = convMatrix [i][k][l]} When gfv_base_pic_flag equals 1, the keypoint coordinate matrix inputBaseKeyPoint and the inputBaseMatrix matrix for the base image are derived as follows: for ( i=0; i<= gfv_nn_output_num_kps; i++ ) { inputBaseKeyPoint[i][0] = convKeyPoint[i][0] inputBaseKeyPoint[i][1] = convKeyPoint[i][0] inputBaseKeyPoint [i][2] = convKeyPoint[i][0]} for( i=0; i <gfv_nn_output_num_matrices; i++) for( k = 0; k < gfv_nn_output_matrix_height_minus1[i]; k++ ) Petition 870250090349, dated 03 / 10 / 2025, pp. 132 / 193 125 / 147 for ( l = 0;l < gfv_nn_output_matrix_width_minus1[i]; l++) inputBaseMatrix[i][k][l] = convMatrix [i][k][l]} }

[00384] The InpY() and InpC() functions are specified as follows: InpY(x)=x+((1 << BitDepthy)-1) InpC(x)=x+((1 << BitDepthc)-1)

[00385] In another example, when the translator sends the fixed format of facial parameters to be fed into the generative network, it also sends the information about the fixed format of the facial parameter being sent, such as the number of keypoints, the number of matrices, and the size of each matrix. Therefore, this information does not need to be signaled in the SEI message. Thus, the syntax elements gfv_nn_output_num_kps, gfv_nn_output_num_matrices, gfv_nn_output_matrix_width_minus1[i], and gfv_nn_output_matrix_height_minus1[i] can be ignored. In this example: The entries in TranslatorNN() are: - sigKeyPoint and sigMatrix The outputs of TranslatorNN() are: - convKeyPoint, numConvKeyPoint - convMatrix, numConvMatrix and matrix convMatrixWidth[i] and convMatrixHeight[i] (for i=0 in numConvMatrix-1 if numConvMatrix>0)

[00386] The DeriveInputTensors() process for deriving the GenerativeNN() inputs is specified below: When gfv_base_pic_flag equals 0, the inputDriveKeyPoint keypoint coordinate matrix and the inputDriveMatrix matrix for the current image are derived as follows: for ( i = 0; i <= numConvKeyPoint; i++ ) { Petition 870250090349, dated 03 / 10 / 2025, pp. 133 / 193 126 / 147 inputDriveKeyPoint[i][0] = convKeyPoint[i][0] inputDriveKeyPoint [i][1] = convKeyPoint[i][0] inputDriveKeyPoint [i][2] = convKeyPoint[i][0]} for( i=0; i< numConvMatrix; i++) for( k = 0; k < convMatrixHeight[i]; k++ ) { for ( l = 0;l < convMatrixWidth[i]; l++) inputDriveMatrix[i][k][l] = convMatrix [i][k][l]} When gfv_base_pic_flag equals 1, the keypoint coordinate matrix inputBaseKeyPoint and the inputBaseMatrix matrix for the base image are derived as follows: - for ( i=0; i< = numConvKeyPoint; i++ ) { inputBaseKeyPoint[i][0] = convKeyPoint[i][0] inputBaseKeyPoint [i][1] = convKeyPoint[i][0] inputBaseKeyPoint [i][2] = convKeyPoint[i][0]} for( i=0; i< numConvMatrix; i++) for( k = 0; k < convMatrixHeight[i]; k++ ) for ( l = 0;l < convMatrixWidth[i]; l++) inputBaseMatrix[i][k][l] = convMatrix [i][k][l]}}

[00387] GenerativeNN() is a process for generating sample values ​​from a corresponding output image for a target image. It is invoked only when gfc_base_pic_flag is equal to 0. The input value in GenerativeNN() and the output values ​​of GenerativeNN() are real numbers.

[00388] The entries in GenerativeNN() are: Petition 870250090349, dated 03 / 10 / 2025, pp. 134 / 193 127 / 147 - When gfv_base_pic_flag equals 0, gfv_drive_pic_fusion_flag equals 0, and ChromaFormatIdc equals 0: inputBaseY, inputBaseKeyPoint, inputBaseMatrix, inputDriveKeyPoint, inputDriveMatrix. - When gfv_base_pic_flag equals 0 and gfv_drive_pic_fusion_flag equals 0 and ChromaFormatIdc does not equal 0: inputBaseY, inputBaseCb, inputBaseCr, inputBaseKeyPoint, inputBaseMatrix, inputDriveKeyPoint, inputDriveMatrix. - When gfv_base_pic_flag is equal to 0 and gfv_drive_pic_fusion_flag is equal to 1 and ChromaFormatIdc is equal to 0: inputBaseY, inputDriveY, inputBaseKeyPoint, inputBaseMatrix, inputDriveKeyPoint, inputDriveMatrix. - When gfv_base_pic_flag is equal to 0 and gfv_drive_pic_fusion_flag is equal to 1 and ChromaFormatIdc is not equal to 0: inputBaseY, inputBaseCb, inputBaseCr, inputDriveY, inputDriveCb, inputDriveCr, inputBaseKeyPoint, inputBaseMatrix, inputDriveKeyPoint, inputDriveMatrix.

[00389] Outputs of GenerativeNN() are: - A sample matrix luma genY - When ChromaFormatIdc is not equal to 0, two sample matrices are created: chroma genCb and genCr.

[00390] The StoreOutputTensors() process for deriving the output is specified below: When gfv_base_pic_flag equals 0, the output sample matrices outYPic[x][y], outCbPic[x][y], and outCrPic[x][y] are derived as follows: for(x=0; x< CroppedWidth; x++){ for(y=0; y< CroppedHeight; y++){ outputYPic[x][y] = OutY( genY[x][y] )} Petition 870250090349, dated 03 / 10 / 2025, pp. 135 / 193 128 / 147 if(ChromaFormatIdc != 0) { for(x=0; x< CroppedWidth / SubWidthC; x++){ for(y=0; y< CroppedHeight / SubHeightC; y++){ outputCbPic[x][y] = OutC( genCb[x][y] ) outputCrPic[x][y] = OutC( genCr[x][y] )} }} When gfv_base_pic_flag equals 1, the output sample matrices outYPic[x][y], outCbPic[x][y], and outCrPic[x][y] are derived as follows: for(x=0; x< CroppedWidth; x++){ for(y=0; y< CroppedHeight; y++){ outputYPic[x][y] = baseCroppedYPic [x][y]} } if(ChromaFormatIdc != 0) { for(x=0; x< CroppedWidth / SubWidthC; x++){ for(y=0; y< CroppedHeight / SubHeightC; y++){ outputCbPic[x][y] = baseCroppedCbPic [x][y] outputCrPic[x][y] = baseCroppedCbPic [x][y]} }}

[00391] As funções OutY( ) e OutC( ) são especificadas conforme a seguir: OutY(x)= Clip3(0, (1 << BitDepthY)-1, x*((1 << BitDepthy)1) OutC(x)=Clip3(0, ( 1 << BitDepthC)-1, x*((1 << BitDepthC)Petição 870250090349, de 03 / 10 / 2025, pág. 136 / 193 129 / 147 1)

[00392] The syntax of a generative facial video SEI message shown in Table 12 can be encoded and signaled in a bitstream, and can be further decoded by the decoder. FIG. 12 is a schematic diagram illustrating an exemplary 1200 SEI message, according to some embodiments of the present disclosure. As shown in FIG. 12, the SEI 1200 message may include, when used to transmit a neural network configuration, an identification number indicator (gfv_id) 1201, a base image indicator (e.g., gfv_base_pic_flag) 1202 indicating that the SEI 1200 message is associated with a base image, a neural network presence indicator (e.g., gfv_nn_present_flag) 1203 indicating that the first subnetwork information is indicated by the SEI 1200 message, neural network indicators (e.g., gfv_nn_mode_idc 1204 and gfv_nn_mode_idc 1205), etc.

[00393] In some embodiments, the SEI 1100 message may include, when used to transmit the facial information parameter, an identification number indicator (gfv_id) 1201, a base image indicator (e.g., gfv_base_pic_flag) 1202 indicating that the SEI 1200 message is not associated with a base image, a neural presence indicator (e.g., gfv_nn_present_flag) 1203 indicating that the first subnetwork information is not indicated in the SEI 1200 message, gfv_drive_pic_fusion_flag 1206, a coordinate indicator (coordinate_present_flag) 1207, a matrix indicator (matrix_present_flag) 1208, etc. The coordinate indicator 1207 and matrix indicator 1208 are collaboratively flagged. That is, in a proper SEI message 1200, only one of the coordinate indicator 1207 and matrix indicator 1208 can be activated. Furthermore, the parameter indicator 1209 can be generated and signaled accordingly. Petition 870250090349, dated 03 / 10 / 2025, pp. 137 / 193 130 / 147 the activated mode. The corresponding bit length of these parameters is also shown in FIG. 12 as an example. As noted, the bit length of these parameters can be designed differently. The other parameters are not shown in FIG. 12 to avoid any ambiguity.

[00394] In some embodiments, the method of encoding a sequence of videos into a bitstream is also provided. FIG. 13 is a flowchart of an exemplary method 1300 for encoding a sequence of videos into a bitstream, according to some embodiments of the present disclosure. As shown in FIG. 13, method 1300 may include steps 1302 and 1304, which may be implemented by an encoder (e.g., picture / video encoder 124 in FIG. 1 or device 400 in FIG. 4).

[00395] In step 1302, the encoder receives a sequence of videos.

[00396] In step 1304, the encoder can encode one or more images from the video sequence to generate a bitstream. The encoder can encode a base image from one or more images and a supplementary enhancement information (SEI) message. The SEI message is used to indicate a mode and a corresponding facial information parameter used to encode a facial image. The bitstream is used to generate the facial image by a neural network in a decoder (e.g., image / video decoder 144 in FIG. 1 or device 400 in FIG. 4) based on the base image and the facial information parameter.

[00397] As noted, the SEI message can be encoded according to the description in relation to FIGS. 8-12 and the specification illustrated above in the form of tables.

[00398] In some models, media is also provided. Petition 870250090349, dated 03 / 10 / 2025, pp. 138 / 193 131 / 147 of non-transient computer-readable storage that stores a bitstream. The bitstream can be encoded and decoded according to the generative facial video supplementary enhancement information (SEI) messages described above (e.g., FIGS. 8 to 12).

[00399] In some embodiments, a non-transient computer-readable storage medium containing instructions is also provided, and the instructions can be executed by a device (such as the disclosed encoder and decoder) to perform the methods described above. Common forms of non-transient media include, for example, a floppy disk, a floppy disk, a hard disk, a solid-state drive, magnetic tape or any other magnetic data storage media, a CD-ROM, any other optical data storage media, any physical media with hole patterns, a RAM, a PROM and EPROM, a FLASH-EPROM or any other flash memory, NVRAM, a cache, a register, any other memory chip or cartridge, and network versions thereof. The device may include one or more processors (CPUs), an input / output interface, a network interface and / or memory.

[00400] The modalities can be further described using the following clauses: 1. A method for generating a facial image, including: receiving a bitstream; Decode the encoded bitstream information to obtain a baseline image and a supplementary enhancement information (SEI) message; Determine if the SEI message applies to a neural network for generating a facial image. in response to the SEI message that applies to the neural network for Petition 870250090349, dated 03 / 10 / 2025, pp. 139 / 193 132 / 147 generate the facial image, determine a mode used to encode the facial image and a corresponding facial information parameter based on the SEI message; and generate, using the neural network, the facial image based on the base image and the facial information parameter. 2. Method, according to clause 1, wherein the SEI message includes an identification number indicator to indicate whether the SEI message is used to encode the facial image; and determining whether the SEI message applies to the neural network to generate the facial image includes: determining, based on the identification number indicator, whether the SEI message is used to encode the facial image. 3. Method, according to clause 2, where the identification number indicator indicates a generative facial video filter. 4. Method, according to any of clauses 1 to 3, wherein the SEI message includes a mode indicator, and wherein determining the mode used to encode the facial image includes: determining the mode based on the mode indicator. 5. Method, according to clause 4, in which the SEI message additionally includes a parameter indicator corresponding to the mode indicator, and the facial information parameter is determined based on the parameter indicator. 6. Method, according to clause 4, where the mode indicator indicates at least one of the following as the mode: 2D facial landmarks, 2D key points, consistent regions, 3D key points, compact features, or facial semantics. 7. Method, according to clause 4, where the mode indicator includes at least one coordinate indicator to indicate whether the SEI message contains coordinate parameters for Petition 870250090349, dated 03 / 10 / 2025, pages 140 / 193 133 / 147 to encode the facial image, a matrix indicator to show whether the SEI message contains the matrix parameters to encode the facial image, or a semantic indicator to show whether the SEI message contains the semantic parameters to encode the facial image. 8. Method, according to clause 7, wherein the SEI message additionally includes a parameter indicator to transmit the coordinate parameters, in response to the coordinate indicator indicating that the SEI message contains the coordinate parameters; and wherein the coordinate parameters are determined as the facial information parameters based on the parameter indicator. 9. Method, according to clause 7, wherein the SEI message additionally includes a parameter indicator to transmit the matrix parameters, in response to the matrix indicator indicating that the SEI message contains the matrix parameters; and wherein the matrix parameters are determined as the facial information parameters based on the parameter indicator. 10. Method, according to clause 7, wherein the SEI message additionally includes a parameter indicator to transmit the semantic parameters, in response to the semantic indicator indicating that the SEI message contains the semantic parameters; and wherein the semantic parameters are determined as the facial information parameters based on the parameter indicator. 11. Method, according to clause 7, wherein the coordinate parameters include at least one of a 2D facial landmark representation, a 2D facial keypoint representation, a consistent region representation, or a Petition 870250090349, dated 03 / 10 / 2025, pp. 141 / 193 134 / 147 facial representation of 3D key points. 12. Method, according to clause 7, wherein the matrix parameters include at least one of an affine translation matrix, covariance matrix, translation matrix, rotation matrix, or compact feature matrix. 13. Method, according to clause 7, in which the semantic parameters include at least one of the following: semantic description of a mouth, semantic description of an eye; or semantic description of a head. 14. Method, according to clause 7, in which the matrix parameters include at least one of an affine translation matrix, covariance matrix, compact feature matrix, or semantic matrix. 15. Method, according to clause 14, in which the semantic matrix includes at least one of a mouth matrix, eye matrix, head rotation matrix, head translation matrix, or head location matrix. 16. A method, according to any one of clauses 1 to 15, wherein the SEI message includes a key indicator to identify a target neural network used to generate the facial information parameter; and wherein determining whether the SEI message applies to the neural network to generate the facial image includes: To determine if the neural network matches the target neural network based on the key indicator. 17. Method, according to any of clauses 1 to 16, where the SEI message includes an order indicator to indicate an order of display of the facial image, the method additionally including: Display the facial image according to the display order. Petition 870250090349, dated 03 / 10 / 2025, pp. 142 / 193 135 / 147 18. A method, according to any of clauses 1 to 17, in which the neural network includes a first subnetwork and a second subnetwork; and in which generating the facial image based on the base image and the facial information parameter by the neural network includes: The first subnetwork translates the facial information parameter into a predetermined parameter format; and the second subnetwork generates the facial image based on the predetermined parameter format and the base image. 19. Method, according to clause 18, in which the predetermined format is a flow map. 20. Method, according to clause 18, wherein the SEI message includes a key presence indicator to indicate the presence of a key indicator; and wherein determining whether the SEI message applies to the neural network to generate the facial image includes: To obtain the key indicator to identify a target neural network used to generate the facial information parameter of the SEI message, in response to the key presence indicator that indicates the presence of the key indicator; and to determine if the neural network matches the target neural network based on the key indicator. 21. Method, according to clause 20, wherein the SEI message additionally includes a neural network indicator to indicate the target neural network; and wherein the method additionally includes: Define the neural network based on the neural network indicator in response to the key presence indicator, indicating the absence of the key indicator. 22. Method according to clause 21, the network indicator Petition 870250090349, dated 03 / 10 / 2025, pages 143 / 193 136 / 147 neural includes at least one of a neural network mode indicator to indicate a target neural network format or a neural network type indicator to indicate a target neural network type. 23. Method, according to clause 18, in which the predetermined parameter format includes at least one of: a predetermined number of key points used to describe a facial feature; a predetermined number of matrices used to describe a facial feature; or a predetermined size of a matrix used to describe the facial feature. 24. Method, according to clause 18, in which translating the facial information parameter into the predetermined parameter format by the first subnetwork includes: Decode a syntax element indicating a difference between the predetermined parameter format and a parameter format associated with the base image; and determine the predetermined parameter format based on the difference and the parameter format associated with the base image. 25. Method, according to clause 18, wherein the SEI message includes a base image indicator to indicate whether the SEI message is associated with a base image and a neural network presence indicator to indicate whether the first subnetwork information is indicated by the SEI message; and wherein the method additionally includes: Define the first subnetwork based on the neural network indicator in response to the base image indicator that indicates the SEI message is associated with a base image and the neural network presence indicator that indicates that the first subnetwork information is indicated by the SEI message. Petition 870250090349, dated 03 / 10 / 2025, pp. 144 / 193 137 / 147 26. Method for encoding a sequence of videos into a bitstream, including: Receive a sequence of videos; and encode one or more images from the video sequence to generate a bitstream, including: to encode a base image from one or more images and a supplementary enhancement information (SEI) message, the SEI message indicating a mode and a corresponding facial information parameter used to encode a facial image, and wherein the bitstream is used to generate the facial image by a neural network based on the base image and the facial information parameter. 27. Method, according to clause 26, in which the SEI message includes an identification number indicator to indicate whether the SEI message is used to encode the facial image. 28. Method, according to clause 27, in which the identification number indicator indicates a generative facial video filter. 29. Method, according to any of clauses 26 to 28, where the SEI message includes a mode indicator to indicate the mode used to encode the facial image. 30. Method, according to clause 29, in which the SEI message additionally includes a parameter indicator corresponding to the mode indicator for transmitting the facial information parameter. 31. Method, according to clause 30, where the mode indicator indicates at least one of the following as the mode: 2D facial landmarks, 2D key points, consistent regions, 3D key points, compact features, or facial semantics. 32. Method, according to clause 29, wherein the mode indicator includes at least one coordinate indicator. Petition 870250090349, dated 03 / 10 / 2025, pages 145 / 193 138 / 147 to indicate whether the SEI message contains coordinate parameters for encoding the facial image, a matrix indicator to indicate whether the SEI message contains the matrix parameters for encoding the facial image, or a semantic indicator to indicate whether the SEI message contains the semantic parameters for encoding the facial image. 33. Method, according to clause 32, in which the SEI message additionally includes a parameter indicator to transmit the coordinate parameters, in response to the coordinate indicator indicating that the SEI message contains the coordinate parameters. 34. Method, according to clause 32, in which the SEI message additionally includes a parameter indicator to transmit the array parameters, in response to the array indicator indicating that the SEI message contains the array parameters. 35. Method, according to clause 32, in which the SEI message additionally includes a parameter indicator to transmit the semantic parameters, in response to the semantic indicator indicating that the SEI message contains the semantic parameters. 36. Method, according to clause 32, wherein the coordinate parameters include at least one of a 2D facial landmark representation, a 2D facial keypoint representation, a consistent region representation, or a 3D facial keypoint representation. 37. Method, according to clause 32, in which the matrix parameters include at least one of an affine translation matrix, covariance matrix, translation matrix, rotation matrix, or compact feature matrix. 38. Method, according to clause 32, in which the semantic parameters include at least one of the following: semantic description of a mouth, semantic description of an eye; or semantic description of Petition 870250090349, dated 03 / 10 / 2025, pp. 146 / 193 139 / 147 one head. 39. Method, according to clause 32, in which the matrix parameters include at least one of an affine translation matrix, covariance matrix, compact feature matrix, or semantic matrix. 40. Method, according to clause 39, in which the semantic matrix includes at least one of a mouth matrix, eye matrix, head rotation matrix, head translation matrix, or head location matrix. 41. Method, according to any of clauses 26 to 40, in which the SEI message includes a key indicator to identify the neural network. 42. Method, according to any of clauses 26 to 41, in which the SEI message includes an order indicator to indicate an order of display of the facial image. 43. Method, according to any of clauses 26 to 42, in which the neural network includes: A first subnetwork is configured to translate the facial information parameter into a predetermined parameter format; and a second subnetwork is configured to generate the facial image based on the predetermined parameter format and the base image. 44. Method, according to clause 43, in which the predetermined format is a flow map. 45. Method, according to clause 43, wherein the SEI message includes a key presence indicator to indicate the presence of a key indicator, and wherein the SEI message additionally includes the key indicator to identify the neural network, in response to the key presence indicator that indicates the presence of the key indicator. Petition 870250090349, dated 03 / 10 / 2025, pp. 147 / 193 140 / 147 46. ​​Method, according to clause 45, wherein the SEI message additionally includes a neural network indicator to indicate the neural network, in response to the key presence indicator that indicates the absence of the key indicator, and wherein the neural network indicator is used to define a neural network to decode the bitstream. 47. Method, in accordance with clause 46, the neural network indicator includes at least one of a neural network mode indicator to indicate a neural network format or a neural network type indicator to indicate a neural network type. 48. Method, according to clause 43, in which the predetermined parameter format includes at least one of: a predetermined number of key points used to describe a facial feature; a predetermined number of matrices used to describe a facial feature; or a predetermined size of a matrix used to describe the facial feature. 49. Method, according to clause 43, in which the SEI message includes a syntax element indicating the difference between the predetermined parameter format and a parameter format associated with the baseline image. 50. Method, according to clause 43, wherein the SEI message includes a baseline image indicator to indicate whether the SEI message is associated with the baseline image, and the SEI message additionally includes a neural network presence indicator to indicate whether the first subnetwork information is indicated by the SEI message, in response to the baseline image indicator that indicates that the SEI message is associated with the baseline image. Petition 870250090349, dated 03 / 10 / 2025, pp. 148 / 193 141 / 147 51. Method, according to clause 43, wherein the SEI message additionally includes a neural network indicator to indicate the neural network, in response to the neural network presence indicator which indicates that the information of the first subnetwork is indicated by the SEI message, and wherein the neural network indicator is used to define a neural network to decode the bitstream. 52. Non-transient, computer-readable storage media that stores a bitstream of a video, the bitstream including: a base image and a supplementary enhancement information (SEI) message, the SEI message indicating a mode and a corresponding facial information parameter used to encode a facial image, and wherein the bitstream is used to generate the facial image by a neural network based on the base image and the facial information parameter. 53. Non-transient, computer-readable storage media, in accordance with clause 52, wherein the SEI message includes an identification number indicator to indicate whether the SEI message is used to encode facial imagery. 54. Non-transient, computer-readable storage media, in accordance with clause 53, wherein the identification number indicator indicates a generative facial video filter. 55. Non-transient, computer-readable storage media, in accordance with clause 54, wherein the SEI message includes a mode indicator to indicate a mode used to encode the facial image. 56. Non-transient, computer-readable storage media, in accordance with clause 55, where the SEI message includes Petition 870250090349, dated 03 / 10 / 2025, pp. 149 / 193 142 / 147 additionally a parameter indicator corresponding to the mode indicator to transmit the facial information parameter. 57. Non-transient, computer-readable storage media, in accordance with clause 56, wherein the mode indicator indicates at least one of the following as the mode: 2D facial landmarks, 2D keypoints, consistent regions, 3D keypoints, compact features, or facial semantics. 58. Non-transient, computer-readable storage media, in accordance with clause 55, wherein the mode indicator includes at least one of a coordinate indicator to indicate whether the SEI message contains the coordinate parameters for encoding the facial image, a matrix indicator to indicate whether the SEI message contains the matrix parameters for encoding the facial image, or a semantic indicator to indicate whether the SEI message contains the semantic parameters for encoding the facial image. 59. Non-transient, computer-readable storage media, in accordance with clause 58, wherein the SEI message additionally includes a parameter indicator to transmit the coordinate parameters, in response to the coordinate indicator indicating that the SEI message contains the coordinate parameters. 60. Non-transient, computer-readable storage media, in accordance with clause 58, wherein the SEI message additionally includes a parameter indicator to transmit the array parameters, in response to the array indicator indicating that the SEI message contains the array parameters. 61. Non-transient, computer-readable storage media, in accordance with clause 58, wherein the SEI message additionally includes a parameter indicator to transmit the semantic parameters, in response to the semantic indicator indicating that the SEI message contains the semantic parameters. Petition 870250090349, dated 03 / 10 / 2025, pp. 150 / 193 143 / 147 62. Non-transient, computer-readable storage media, in accordance with clause 58, wherein the coordinate parameters include at least one of a 2D facial landmark representation, a 2D facial keypoint representation, a consistent region representation, or a 3D facial keypoint representation. 63. Non-transient, computer-readable storage media, according to clause 58, wherein the matrix parameters include at least one of an affine translation matrix, covariance matrix, translation matrix, rotation matrix, or compact feature matrix. 64. Non-transient, computer-readable storage media, in accordance with clause 58, wherein the semantic parameters include at least one of the following: a semantic description of a mouth, a semantic description of an eye, or a semantic description of a head. 65. Non-transient, computer-readable storage media, in accordance with clause 58, wherein the matrix parameters include at least one of an affine translation matrix, covariance matrix, compact feature matrix, or semantic matrix. 66. Non-transient, computer-readable storage media, as defined in clause 65, wherein the semantic array includes at least one of a mouth array, eye array, head rotation array, head translation array, or head location array. 67. Non-transient, computer-readable storage media, in accordance with any of clauses 52 to 66, wherein the SEI message includes a key indicator to identify the neural network. 68. Non-transient, computer-readable storage media, in accordance with any of clauses 52 to 67, wherein the SEI message includes an order indicator to indicate an order. Petition 870250090349, dated 03 / 10 / 2025, pages 151 / 193 144 / 147 facial image display. 69. Non-transient, computer-readable storage media, in accordance with any of clauses 52 to 68, wherein the neural network includes: A first subnetwork is configured to translate the facial information parameter into a predetermined parameter format; and a second subnetwork is configured to generate the facial image based on the predetermined parameter format and the base image. 70. Non-transient, computer-readable storage media, in accordance with clause 69, wherein the predetermined format is a stream map. 71. Non-transient, computer-readable storage media, as defined in clause 69, wherein the SEI message includes a key presence indicator to indicate the presence of a key indicator, and wherein the SEI message additionally includes the key indicator to identify the neural network, in response to the key presence indicator that indicates the presence of the key indicator. 72. Non-transient, computer-readable storage media, as defined in clause 71, wherein the SEI message additionally includes a neural network indicator to indicate the neural network, in response to the key presence indicator which indicates the absence of the key indicator, and wherein the neural network indicator is used to define a neural network to decode the bitstream. 73. Non-transient, computer-readable storage media, in accordance with clause 72, the neural network indicator includes at least one of a neural network mode indicator to indicate a neural network format or a neural network type indicator to indicate a neural network format. Petition 870250090349, dated 03 / 10 / 2025, pp. 152 / 193 145 / 147 indicate a type of neural network. 74. Non-transient, computer-readable storage media, in accordance with clause 69, wherein the predetermined parameter format includes at least one of: a predetermined number of key points used to describe a facial feature; a predetermined number of matrices used to describe a facial feature; or a predetermined size of a matrix used to describe the facial feature. 75. Non-transient, computer-readable storage media, as defined in clause 69, where the SEI message includes a syntax element indicating a difference between the predetermined parameter format and a parameter format associated with the base image. 76. Non-transient, computer-readable storage media, in accordance with clause 69, wherein the SEI message includes a base image indicator to indicate whether the SEI message is associated with the base image, and the SEI message additionally includes a neural network presence indicator to indicate whether the first subnetwork information is indicated by the SEI message, in response to the base image indicator indicating that the SEI message is associated with the base image. 77. Non-transient, computer-readable storage media, in accordance with clause 69, wherein the SEI message additionally includes a neural network indicator to indicate the neural network, in response to the neural network presence indicator which indicates that the information of the first subnetwork is indicated by the SEI message, and Petition 870250090349, dated 03 / 10 / 2025, pages 153 / 193 146 / 147 where the neural network indicator is used to define a neural network to decode the bitstream.

[00401] Note that relational terms in this document, such as “first” and “second,” are used only to differentiate one entity or operation from another entity or operation, and do not require or imply any actual relationship or sequence between those entities or operations. Furthermore, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and to be open-ended, so that an item or items following any of these words do not signify a comprehensive list of such item or items, or be limited only to the item or items listed.

[00402] As used in this document, except where specifically stated otherwise, the term “or” encompasses all possible combinations, except where it is impractical. For example, if it is stated that a database may include A or B, then, unless specifically stated otherwise or if it is impractical, the database may include A, or B, or A and B. As in the second example, if it is stated that a database may include A, B, or C, then, unless specifically stated otherwise or if it is impractical, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[00403] It is appreciated that the embodiments described above may be implemented by hardware, or software (program code), or a combination of hardware and software. If implemented by software, they may be stored on computer-readable media as described above. The software, when executed by the processor, may execute the disclosed methods. The computing units and other functional units described in this disclosure may be implemented by hardware, or software, or a combination of hardware. Petition 870250090349, dated 03 / 10 / 2025, pp. 154 / 193 147 / 147 and software. A subject matter expert will also understand that multiple modules / units described above can be combined as one module / unit, and each of the modules / units described above can be further divided into a plurality of submodules / subunits.

[00404] In the preceding descriptive report, embodiments were described with reference to various specific details that may vary from implementation to implementation. Certain adaptations and modifications of the described embodiments may be made. Other embodiments may be apparent to those skilled in the art from consideration of the descriptive report and practice of the invention disclosed in this document. The descriptive report and examples are intended to be considered only as illustrative, with the true scope and spirit of the invention being indicated by the following claims. It is also intended that the sequence of steps shown in the figures is for illustrative purposes only and is not limited to any particular sequence of steps. Thus, those skilled in the art may understand that these steps can be performed in a different order while implementing the same method.

[00405] In the drawings and descriptive report, illustrative embodiments have been disclosed. However, many variations and modifications can be made to these embodiments. Consequently, although specific terms are employed, they are used only in a generic and descriptive sense and not for the purpose of limitation. Petition 870250090349, dated 03 / 10 / 2025, pages 155 / 193

Claims

1 / 5 CLAIMS 1. A method for generating a facial image, characterized in that it comprises: receiving a bitstream; decoding encoded information from the bitstream to obtain a base image and a supplementary enhancement information (SEI) message; determining whether the SEI message applies to a neural network to generate a facial image; in response to the SEI message applying to the neural network to generate the facial image, determining a mode used to encode the facial image and a corresponding facial information parameter based on the SEI message; and generating, using the neural network, the facial image based on the base image and the facial information parameter.

2. A method according to claim 1, characterized in that the SEI message comprises an identification number indicator to indicate whether the SEI message is used to encode the facial image; and determining whether the SEI message applies to the neural network to generate the facial image comprises: determining, based on the identification number indicator, whether the SEI message is used to encode the facial image.

3. Method, according to claim 2, characterized in that the identification number indicator indicates a generative facial video filter.

4. Method, according to claim 1, characterized in that the SEI message comprises a mode indicator, and in which determining the mode used to encode the facial image comprises: determining the mode based on the mode indicator.

5. Method, according to claim 4, characterized in that the SEI message additionally comprises a parameter indicator corresponding to the mode indicator, and the facial information parameter is determined based on the parameter indicator.

6. Method, according to claim 4, characterized in that the mode indicator indicates at least one of the following as the mode: 2D facial landmarks, 2D key points, consistent regions, 3D key points, compact features, or facial semantics.

7. Method, according to claim 4, characterized in that the mode indicator comprises at least one of a coordinate indicator to indicate whether the SEI message contains coordinate parameters for encoding the facial image, a matrix indicator to indicate whether the SEI message contains the matrix parameters for encoding the facial image, or a semantic indicator to indicate whether the SEI message contains the semantic parameters for encoding the facial image.

8. A method according to claim 7, characterized in that the SEI message additionally comprises a parameter indicator for transmitting the coordinate parameters, in response to the coordinate indicator indicating that the SEI message carries the coordinate parameters; and wherein the coordinate parameters are determined as the facial information parameters based on the parameter indicator.

9. Method, according to claim 7, characterized in that the SEI message additionally comprises a parameter indicator to transmit the matrix parameters, in response to the matrix indicator indicating that the SEI message contains the matrix parameters; and wherein the matrix parameters are determined as the facial information parameters based on the parameter indicator.

10. A method for encoding a sequence of videos into a bitstream, characterized in that it comprises: receiving a sequence of videos; and encoding one or more images from the sequence of videos to generate a bitstream, comprising: encoding a base image from one or more images and a supplementary enhancement information (SEI) message, the SEI message indicating a mode and a corresponding facial information parameter used to encode a facial image, and wherein the bitstream is used to generate the facial image by a neural network based on the base image and the facial information parameter.

11. Method, according to claim 10, characterized in that the SEI message comprises an identification number indicator to indicate whether the SEI message is used to encode the facial image.

12. Method, according to claim 11, characterized in that the identification number indicator indicates a generative facial video filter.

13. Method, according to claim 10, characterized in that the SEI message comprises a mode indicator to indicate the mode used to encode the facial image.

14. Method, according to claim 13, characterized in that the SEI message additionally comprises a parameter indicator corresponding to the mode indicator for transmitting the facial information parameter.

15. Method, according to claim 14, characterized in that the mode indicator indicates at least one of the following as the mode: 2D facial landmarks, 2D key points, consistent regions, 3D key points, compact features, or facial semantics.

16. Method, according to claim 13, characterized in that the mode indicator comprises at least one of a coordinate indicator to indicate whether the SEI message contains coordinate parameters for encoding the facial image, a matrix indicator to indicate whether the SEI message contains the matrix parameters for encoding the facial image, or a semantic indicator to indicate whether the SEI message contains the semantic parameters for encoding the facial image.

17. Method, according to claim 16, characterized in that the SEI message additionally comprises a parameter indicator for transmitting the coordinate parameters, in response to the coordinate indicator indicating that the SEI message carries the coordinate parameters.

18. Method, according to claim 16, characterized in that the SEI message additionally comprises a parameter indicator for transmitting the array parameters, in response to the array indicator indicating that the SEI message carries the array parameters.

19. Non-transient, computer-readable storage media that stores a bitstream of a video, the bitstream characterized in that it comprises: a base image and a supplementary enhancement information (SEI) message, the SEI message indicating a mode and a corresponding facial information parameter used to encode a facial image, and wherein the bitstream is used to generate the facial image by a neural network based on the base image and the facial information parameter.

20. Non-transient, computer-readable storage media according to claim 19, characterized in that the SEI message comprises an identification number indicator to indicate whether the SEI message is used to encode facial imagery. Petition 870250090349, dated 10 / 03 / 2025, pp. 160 / 193

Images