System and method for signaling downsampling filters for chromatintra prediction modes from luma

Abstract

Various implementations described herein include methods and systems for decoding video. In one aspect, the method includes receiving a video stream having a sequence of frames. The sequence of frames includes one or more key frames. Each key frame has a respective downsampling filter type. The method includes, in accordance with determining that a current frame corresponds to a first key frame, retrieving from the video stream a syntax element associated with a first downsampling filter type associated with the first key frame. The method includes using the first downsampling filter type to obtain a downsampled frame including luma blocks of the current frame and a predefined set of frames immediately following the current frame. The method includes predicting chroma blocks of the current frame based on the downsampled frame.

Description

Technical Field

[0001] Related Applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 428,714, filed November 29, 2022, entitled "Signaling of Downsampling Filters for Chroma from Luma Intra Prediction Mode," and is a continuation of U.S. Patent Application No. 18 / 144,042, filed May 5, 2023, entitled "Systems and Methods for Signaling of Downsampling Filters for Chroma from Luma Intra Prediction Mode," and claims priority thereto, and all of them are incorporated herein by reference.

[0002]

[0002] The disclosed embodiments generally relate to video coding, including, but not limited to, systems and methods for signaling of downsampling filters for chroma from luma intra prediction mode.

Background Art

[0003]

[0003] Digital video is supported by various electronic devices such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video gaming consoles, smartphones, video teleconferencing devices, video streaming devices, etc. The electronic devices may transmit and receive or in some cases communicate digital video data across a communication network and / or store digital video data in a storage device. Due to the limited bandwidth capacity of the communication network and the limited memory resources of the storage device, video coding may be used to compress video data according to one or more video coding standards before the video data is communicated or stored.

[0004]

[0004] Multiple video codec standards have been developed. For example, video coding standards include AOMedia Video 1 (AV1), General Purpose Video Coding (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), and Moving Picture Expert Group (MPEG) coding. Video coding generally utilizes prediction methods (e.g., interpretation, intrapretation, etc.) that take advantage of the redundancy inherent in video data. Video coding aims to compress video data into a format that uses a lower bitrate while avoiding or minimizing degradation of video quality.

[0005]

[0005] HEVC, also known as H.265, is a video compression standard designed as part of the MPEG-H project. The ITU-T and ISO / IEC published the HEVC / H.265 standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). General-Purpose Video Coding (VVC), also known as H.266, is a video compression standard intended as a successor to HEVC. The ITU-T and ISO / IEC published the VVC / H.266 standard in 2020 (version 1) and 2022 (version 2). AV1 is an open video coding format designed as an alternative to HEVC. On January 8, 2019, a verified version 1.0.0 with Errata 1 of the specification was released. [Overview of the project] [Problems that the invention aims to solve]

[0006]

[0006] This disclosure describes various techniques for signaling downsampling filters used in cross-component intra-prediction mode. Luma-to-chroma (CfL) prediction is an efficient video coding tool that models chroma pixels as a linear function of concurrently reconstructed luma pixels. When there are multiple downsampling filters used in CfL mode, these filters need to be signaled in high-level syntax. Current signaling techniques may have two problems: firstly, the results may differ between sequential and parallel operation of multiple picture groups (GOPs); and secondly, there may be redundant signaling overhead for the intercoded frames. [Means for solving the problem]

[0007]

[0007] Therefore, improved methods and systems are needed for signaling downsampling filters in CfL mode. This disclosure describes various techniques for signaling downsampling filters used in cross-component intra-predictive mode. The techniques disclosed may be used by video bitstream decoders to improve the quality and / or efficiency of decoding. Video encoders may also implement these techniques during encoding (for example, to reconstruct encoded frames and / or test hypotheses).

[0008]

[0008] According to several embodiments, a method for video decoding is provided. The method includes receiving a video stream having a sequence of frames, wherein the sequence of frames includes one or more keyframes, and each keyframe has a respective downsampling filter type. The method includes, in accordance with the determination that the current frame corresponds to a first keyframe among the one or more keyframes, (i) extracting a syntax element from the video stream related to a first downsampling filter type associated with the first keyframe; (ii) downsampling the chroma block of the current frame and a predefined set of frames immediately following the current frame using the first downsampling filter type to obtain a downsampled frame; and (iii) predicting the chroma block of the current frame based on the downsampled frame.

[0009]

[0009] According to some embodiments, a computing system is provided, such as a streaming system, a server system, a personal computer system, or other electronic device. The computing system includes a control circuit and a memory for storing one or more sets of instructions. One or more sets of instructions, including instructions for performing any of the methods described herein. In some embodiments, the computing system includes encoder components and / or decoder components.

[0010]

[0010] According to some embodiments, a non-temporary computer-readable storage medium is provided. The non-temporary computer-readable storage medium stores one or more sets of instructions for execution by a computing system. One or more sets of instructions, including instructions for performing any of the methods described herein.

[0011]

[0011] Accordingly, devices and systems are disclosed along with methods for coding video. Such methods, devices, and systems may complement or replace conventional methods, devices, and systems for coding video.

[0012]

[0012] The features and advantages described herein are not necessarily all, and in particular, several additional features and advantages will become apparent to those skilled in the art in view of the drawings, specification and claims provided herein. Furthermore, it should be noted that the language used herein has been selected primarily for readability and educational purposes and is not necessarily selected to define or limit the subject matter described herein.

[0013]

[0013] A more detailed description may be given by reference to the features of various embodiments, some of which are shown in the accompanying drawings, so that the disclosure may be better understood. However, the accompanying drawings only illustrate the features relevant to the disclosure and should not necessarily be considered limiting, as such description may lead to other useful features, as will be understood by those skilled in the art by reading the disclosure. [Brief explanation of the drawing]

[0014] [Figure 1]

[0014] This block diagram shows an exemplary communication system according to several embodiments. [Figure 2A]

[0015] This is a diagram showing exemplary elements of encoder components according to several embodiments. [Figure 2B]

[0016] This is a block diagram illustrating exemplary elements of decoder components according to several embodiments. [Figure 3]

[0017] This is a diagram illustrating exemplary server systems in several embodiments. [Figure 4]

[0018] A diagram showing the nominal angle in directional intra prediction according to some embodiments. [Figure 5]

[0019] A diagram showing the top position, left position, and top - left position for the PAETH intra prediction mode for predicting a coding block according to some embodiments. [Figure 6]

[0020] A block diagram showing the chroma - from - luma (CfL) prediction process according to some embodiments. [Figure 7]

[0021] A block diagram showing the luma samples inside and outside the picture boundary according to some embodiments. [Figure 8]

[0022] A diagram showing different chroma downsampling formats according to some embodiments. [Figure 9]

[0023] A diagram showing the AVI CfL downsampling filter according to some embodiments. [Figure 10]

[0024] A diagram showing the binarization process and their corresponding codes according to some embodiments. [Figure 11]

[0025] A diagram showing the temporal layer identifier numbers for nine consecutive frames according to some embodiments. [Figure 12]

[0026] A flowchart showing an exemplary method of video decoding according to some embodiments.

Best Mode for Carrying Out the Invention

[0015]

[0027] According to convention, the various features shown in the drawings are not necessarily drawn to a fixed scale, and like reference numbers may be used throughout the specification and the figures to indicate like features.

[0016]

[0028] This disclosure describes signaling a downsampling filter type in a rumor-to-chroma (CfL) intra-prediction mode for video coding. A video stream having a sequence of frames is received. The sequence of frames includes one or more keyframes, each keyframe having its own downsampling filter type. According to the determination that the current frame corresponds to a first keyframe among the one or more keyframes, the syntax elements associated with the first downsampling filter type associated with the first keyframe are extracted from the video bitstream. To obtain a downsampled frame, the rumor block of the current frame (e.g., a block of pixels) and a predefined set of frames immediately following the current frame are downsampled using the first downsampling filter type. Based on the downsampled frame, the chroma block of the current frame (e.g., a block of pixels) is predicted.

[0017] Exemplary Systems and Devices

[0029] Figure 1 is a block diagram showing a communication system 100 according to several embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to 120-m) that are communicatively coupled to one or more networks. In some embodiments, the communication system 100 is a streaming system for use with video-enabled applications such as video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0018]

[0030] Source device 102 includes a video source 104 (e.g., a camera component or media storage) and an encoder component 106. In some embodiments, the video source 104 is a digital camera (e.g., configured to create an uncompressed video sample stream). The encoder component 106 generates one or more encoded video bitstreams from the video stream. The video stream from video source 104 may have a higher data volume compared to the encoded video bitstream 108 generated by the encoder component 106. Since the encoded video bitstream 108 has a lower data volume (less data) compared to the video stream from the video source, the encoded video bitstream 108 requires less bandwidth to transmit and less storage space to store compared to the video stream from video source 104. In some embodiments, source device 102 does not include the encoder component 106 (e.g., it is configured to transmit uncompressed video data to one or more networks 110).

[0019]

[0031] One or more networks 110 represent any number of networks that transmit information between the source device 102, the server system 112, and / or the electronic device 120, including, for example, wireline (wired) and / or wireless communication networks. One or more networks 110 may exchange data in circuit-switched channels and / or packet-switched channels. Typical networks include telecommunications networks, local area networks, wide area networks, and / or the Internet.

[0020]

[0032] One or more networks 110 include a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is or includes a streaming server (configured to store and / or deliver video content, such as an encoded video stream from a source device 102). The server system 112 includes a coder component 114 (configured to encode and / or decode video data, for example). In some embodiments, the coder component 114 includes an encoder component and / or a decoder component. In various embodiments, the coder component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the coder component 114 is configured to decode an encoded video bitstream 108 using different encoding standards and / or methodologies to produce encoded video data 116, and to re-encode the video data. In some embodiments, the server system 112 is configured to produce multiple video formats and / or encodings from the encoded video bitstream 108.

[0021]

[0033] In some embodiments, the server system 112 functions as a media-aware network element (MANE). For example, the server system 112 may be configured to prune an encoded video bitstream 108 to adjust potentially different bitstreams to one or more of the electronic devices 120. In some embodiments, the MANE is provided separately from the server system 112.

[0022]

[0034] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode encoded video data 116 to generate an outgoing video stream that can be rendered on a display or other type of rendering device. In some embodiments, one or more of the electronic devices 120 do not include a display component (for example, including media storage that is communicably coupled to an external display device). In some embodiments, the electronic device 120 is a streaming client. In some embodiments, the electronic device 120 is configured to access a server system 112 to retrieve encoded video data 116.

[0023]

[0035] The source device and / or multiple electronic devices 120 may be referred to as “terminal devices” or “user devices.” In some embodiments, one or more of the source device 102 and / or electronic devices 120 are instances of a server system, a personal computer, a portable device (e.g., a smartphone, tablet, or laptop), a wearable device, a video conferencing device, and / or other types of electronic devices.

[0024]

[0036] In an exemplary operation of the communication system 100, source device 102 transmits an encoded video bitstream 108 to server system 112. For example, source device 102 may encode a stream of pictures captured by the source device. Server system 112 receives the encoded video bitstream 108 and may decode and / or encode the encoded video bitstream 108 using coder components 114. For example, server system 112 may apply encoding to video data, which is more optimal for network transmission and / or storage. Server system 112 may transmit the encoded video data 116 (e.g., one or more encoded video bitstreams) to one or more of the electronic devices 120. Each electronic device 120 may decode the encoded video data 116 to restore a video picture and optionally display it.

[0025]

[0037] In some embodiments, the transmission described above is a unidirectional data transmission. Unidirectional data transmission may be used in media serving applications, etc. In some embodiments, the transmission described above is a bidirectional data transmission. Bidirectional data transmission may be used in video conferencing applications, etc. In some embodiments, the encoded video bitstream 108 and / or encoded video data 116 are encoded and / or decoded according to one of the video coding / compression standards described herein, such as HEVC, VVC, and / or AV1.

[0026]

[0038] Figure 2A is a block diagram showing exemplary elements of an encoder component 106 according to several embodiments. The encoder component 106 receives a source video sequence from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., a transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives a video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder component 106). The video source 104 may provide a source video sequence in the form of a digital video sample stream, which may be of any preferred bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 Y CrCB, or RGB), and any preferred sampling structure (e.g., Y CrCb 4:2:0 or Y CrCb 4:4:4). In some embodiments, the video source 104 is a storage device that stores previously captured / prepared video. In some embodiments, the video source 104 is a camera that captures local image information as a video sequence. Video data may be provided as a series of individual pictures that give movement when viewed sequentially. These pictures themselves may be organized as a spatial array of pixels, with each pixel containing one or more samples depending on the sampling structure, color space, etc., in use. Those skilled in the art will readily understand the relationship between pixels and samples. The following description will focus on samples.

[0027]

[0039] The encoder component 106 is configured to encode and / or compress pictures from a source video sequence into an encoded video sequence 216 in real time or under other time constraints required by the application. One function of the controller 204 is to execute an appropriate coding rate. In some embodiments, the controller 204 controls and is functionally coupled to other functional units, as described below. Parameters set by the controller 204 may include rate control relation parameters (e.g., picture skipping, quantizer, and / or λ value for rate-distortion optimization techniques), picture size, picture group (GOP) layout, maximum motion vector search range, etc. Other functions of the controller 204 may relate to the encoder component 106 which is optimized for a particular system design, so those skilled in the art will be able to easily identify the other functions of the controller 204.

[0028]

[0040] In some embodiments, the encoder component 106 is configured to operate within a coding loop. In a simplified example, the coding loop includes a source coder 202 (responsible for creating symbols, such as a symbol stream, based on, for example, an input picture to be coded and one or more reference pictures) and a (local) decoder 210. The decoder 210 reconstructs the symbols to create sample data in a similar manner to the (remote) decoder (when the compression between the symbols and the coded video bitstream is reversible). The reconstructed sample stream (sample data) is input to the reference picture memory 208. Since decoding the symbol stream yields a bit-exact result independent of the decoder location (local or remote), the contents in the reference picture memory 208 are also bit-exact between the local encoder and the remote encoder. In this way, the prediction portion of the encoder interprets the same sample values ​​as reference picture samples that the decoder would interpret when using predictions during decoding. This principle of reference picture simultaneity (and, if simultaneity cannot be maintained due to, for example, channel errors, the resulting drift) is known to those skilled in the art.

[0029]

[0041] The operation of decoder 210 may be the same as that of a remote decoder, such as decoder component 122, which will be described in detail below in relation to Figure 2B. However, referring briefly to Figure 2B, since symbols are available and the encoding / decoding of symbols to the coded video sequence by the entropy coder 214 and parser 254 may be reversible, the entropy decoding portion of decoder component 122, including buffer memory 252 and parser 254, may not be fully implemented in local decoder 210.

[0030]

[0042] An observation that can be made at this point is that any decoder techniques other than pars / entropy decoding present in the decoder must also be present in the corresponding encoder in a substantially equivalent functional form. For this reason, the subject matter disclosed will focus on decoder operation. The description of encoder techniques may be omitted, as encoder techniques are the inverse of the decoder techniques that are described comprehensively. Further details are required only in a few areas and are provided below.

[0031]

[0043] As part of its operation, the source coder 202 may perform motion-compensated predictive coding, predictively coding the input frame by referencing one or more previously coded frames from a video sequence designated as a reference frame. In this way, the coding engine 212 codes the difference between the pixel blocks of the input frame and the pixel blocks of one or more reference frames that may be selected as predictive references to the input frame. The controller 204 may manage the coding operation of the source coder 202, including, for example, setting parameters and subgroup parameters used to encode the video data.

[0032]

[0044] Decoder 210 decodes the coded video data of a frame that may be designated as a reference frame based on symbols created by source coder 202. The operation of coding engine 212 may, advantageously, be an irreversible process. When the coded video data is decoded by a video decoder (not shown in Figure 2A), the reconstructed video sequence may be a copy of the source video sequence with some errors. Decoder 210 replicates the decoding process that may be performed on the reference frame by a remote video decoder, which may cause the reconstructed reference frame to be stored in reference picture memory 208. In this way, encoder component 106 locally stores a copy of the reconstructed reference frame with common content as the reconstructed reference frame that will be acquired by the remote video decoder (without transmission errors).

[0033]

[0045] The predictor 206 may perform a predictive search for the coding engine 212. That is, for a new frame to be coded, the predictor 206 may search the reference picture memory 208 for sample data (as candidate reference pixel blocks) or metadata such as reference picture motion vectors and block shapes that can act as appropriate predictive references for the new picture. The predictor 206 may operate on a sample block-by-pixel-block basis to find appropriate predictive references. In some cases, the input picture may have predictive references drawn from multiple reference pictures stored in the reference picture memory 208, as determined by the search results obtained by the predictor 206.

[0034]

[0046] The outputs of all the aforementioned functional units can undergo entropy coding in the entropy coder 214. The entropy coder 214 converts the symbols generated by the various functional units into coded video sequences by reversibly compressing those symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, and / or arithmetic coding).

[0035]

[0047] In some embodiments, the output of the entropy coder 214 is coupled to a transmitter. The transmitter may be configured to buffer the coded video sequence (one or more) created by the entropy coder 214 in order to prepare it for transmission over a communication channel 218, which may be a hardware / software link to a storage device that will store the coded video data. The transmitter may be configured to merge the coded video data from the source coder 202 with other data to be transmitted, such as coded audio data and / or auxiliary data streams (source not shown). In some embodiments, the transmitter may transmit additional data along with the coded video. The source coder 202 may include such data as part of the coded video sequence. The additional data may include time / space / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, supplemental enhancement information (SEI) messages, visual usability information (VUI) parameter set fragments, and the like.

[0036]

[0048] The controller 204 can manage the operation of the encoder component 106. During coding, the controller 204 may assign a coded picture type to each coded picture, which may affect the coding technique applied to each picture. For example, a picture may be assigned as an intra-picture (I-picture), a predictive picture (P-picture), or a bidirectional predictive picture (B-picture). An intra-picture can be coded and decoded without using other frames in the sequence as a source for prediction. Some video codecs allow different types of intra-pictures, including, for example, independent decoder refresh (IDR) pictures. Those skilled in the art will be aware of their variations of I-pictures and their respective applications and characteristics, and therefore they will not be repeated here. A predictive picture can be coded and decoded using intra-prediction or inter-prediction with at most one motion vector and reference index to predict the sample value of each block. Bidirectional predictive pictures can be coded and decoded using intra-prediction or inter-prediction with at most two motion vectors and reference indices to predict the sample values ​​for each block. Similarly, multiple predictive pictures can use three or more reference pictures and associated metadata for the reconstruction of a single block.

[0037]

[0049] A source picture can typically be spatially subdivided into multiple sample blocks (for example, blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coded block by block. Blocks can be predictively coded by referencing other (already coded) blocks determined by the coding assignment applied to each picture in the block. For example, blocks in picture I can be coded unpredictably, or they can be coded predictively by referencing already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks in picture P can be coded unpredictably via spatial prediction or temporal prediction by referencing one previously coded reference picture. Blocks in picture B can be coded unpredictably via spatial prediction or temporal prediction by referencing one or two previously coded reference pictures.

[0038]

[0050] Video can be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) utilizes spatial correlations in a given picture, while inter-picture prediction utilizes (temporal or other) correlations between pictures. In one example, a specific picture to be encoded / decoded, called the current picture, is partitioned into blocks. When a block in the current picture is similar to a reference block in a previously coded and still-buffered reference picture in the video, the block in the current picture can be coded by a vector called a motion vector. The motion vector points to a reference block in the reference picture and may have a third dimension to identify the reference picture if multiple reference pictures are in use.

[0039]

[0051] The encoder component 106 may perform coding operations in accordance with a predetermined video coding technique or standard, such as any of those described herein. In these operations, the encoder component 106 may perform various compression operations, including predictive coding operations that leverage temporal and spatial redundancy in the input video sequence. Thus, the coded video data may conform to the syntax specified by the video coding technique or standard being used.

[0040]

[0052] Figure 2B is a block diagram showing exemplary elements of a decoder component 122 according to several embodiments. The decoder component 122 in Figure 2B is coupled to channel 218 and display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to a loop filter 256 and configured to transmit data to display 124 (for example, via a wired or wireless connection).

[0041]

[0053] In some embodiments, the decoder component 122 includes a receiver coupled to channel 218 and configured to receive data from channel 218 (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more coded video sequences to be decoded by the decoder component 122. In some embodiments, the decoding of each coded video sequence is independent of other coded video sequences. Each coded video sequence may be received from channel 218, which may be a hardware / software link to a storage device that stores coded video data. The receiver may receive coded video data together with other data, e.g., coded audio data and / or auxiliary data streams, which may be forwarded to their respective use entities (not shown). The receiver may isolate the coded video sequence from other data. In some embodiments, the receiver receives additional (redundant) data along with the coded video. The additional data may be included as part of one or more coded video sequences. The additional data may be used by the decoder component 122 to decode the data and / or to more accurately reconstruct the original video data. Additional data may include, for example, time, space, or SNR enhancement layers, redundant slices, redundant pictures, or forward error correction codes.

[0042]

[0054] According to some embodiments, the decoder component 122 includes a buffer memory 252, a parser 254 (sometimes called an entropy decoder), a scaler / inverse unit 258, an intra-picture prediction unit 262, a motion compensation prediction unit 260, an aggregator 268, a loop filter unit 256, a reference picture memory 266, and a current picture memory 264. In some embodiments, the decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuits. In some embodiments, the decoder component 122 is implemented at least partially in software.

[0043]

[0055] Buffer memory 252 is coupled between channel 218 and parser 254 (for example, to eliminate network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, in addition to buffer memory 252 inside decoder component 122 (configured, for example, to handle playout timing), a separate buffer memory is provided outside decoder component 122 (for example, to eliminate network jitter). When receiving data from a storage / transfer device with sufficient bandwidth and controllability, or from an isosynchronous network, buffer memory 252 may not be required or may be small. For use in best-effort packet networks such as the Internet, buffer memory 252 may be required, may be relatively large, may be advantageously adaptively sized, and may be at least partially implemented in an operating system or similar element (not shown) outside decoder component 122.

[0044]

[0056] The parser 254 is configured to reconstruct symbols 270 from the coded video sequence. These symbols may include, for example, information used to manage the operation of decoder component 122 and / or information for controlling rendering devices such as the display 124. The control information for (one or more) rendering devices may be, for example, in the form of supplemental enhancement information (SEI) messages or video usability information (VUI) parameter set fragments (not shown). The parser 254 parses (entropy decodes) the coded video sequence. The coding of the coded video sequence may follow video coding techniques or standards and may follow principles well known to those skilled in the art, including variable-length coding, Huffman coding, and arithmetic coding with or without context sensitivity. From the coded video sequence, the parser 254 may extract a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder, based on at least one parameter corresponding to that group. Subgroups can include picture groups (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transformation units (TU), and prediction units (PU). Parser 254 can also extract information such as transformation coefficients, quantizer parameter values, and motion vectors from the coded video sequence.

[0045]

[0057] The reconstruction of symbol 270 may involve multiple different units, depending on the type of coded video picture or part thereof (such as interpictures and intrapictures, interblocks and intrablocks), and other factors. Which units are involved and how they are involved may be controlled by parser 254 through subgroup control information parsed from the coded video sequence. The flow of such subgroup control information between parser 254 and the following multiple units is not illustrated for clarity.

[0046]

[0058] In addition to the functional blocks already described, the decoder component 122 can be conceptually subdivided into several functional units, as described below. In actual implementations operating under commercial constraints, many of these units may interact closely with each other and, at least partially, integrate with one another. However, for the purpose of illustrating the subject matter being disclosed, the conceptual subdivision into the following functional units is maintained.

[0047]

[0059] The scaler / inverse unit 258 receives from the parser 254, as one or more symbols 270, the quantized transformation coefficients and control information (such as which transformation to use, block size, quantization factor, and / or quantization scaling metric). The scaler / inverse unit 258 can output a block containing sample values ​​that can be input to the aggregator 268.

[0048]

[0060] In some cases, the output samples of the scaler / inverse unit 258 relate to intracoded blocks, i.e., blocks that do not use predictive information from previously reconstructed pictures but can use predictive information from parts of the picture that were previously reconstructed. Such predictive information may be provided by the intrapicture predictive unit 262. The intrapicture predictive unit 262 may generate a block of the same size and shape as the block being reconstructed, using surrounding already reconstructed information fetched from the current (partially reconstructed) picture from the picture memory 264. The aggregator 268 may, sample by sample, add the predictive information generated by the intrapicture predictive unit 262 to the output sample information provided by the scaler / inverse unit 258.

[0049]

[0061] In other cases, the output samples of the scaler / inverse unit 258 relate to an interconnected and potentially motion-compensated block. In such cases, the motion-compensated prediction unit 260 can access the reference picture memory 266 to fetch samples to be used for prediction. After motion-compensating the fetched samples according to the symbols 270 relating to the block, these samples can be added by the aggregator 268 to the output of the scaler / inverse unit 258 (called residual samples or residual signals in this case) to generate output sample information. The address in the reference picture memory 266 from which the motion-compensated prediction unit 260 fetches the predicted samples may be controlled by a motion vector. The motion vector may be available to the motion-compensated prediction unit 260 in the form of a symbol 270 which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory 266 when the exact motion vector of a subsample is in use, a motion vector prediction mechanism, etc.

[0050]

[0062] The output samples of the aggregator 268 can undergo various loop filtering techniques in the loop filter unit 256. The video compression technique may include in-loop filtering techniques, which are contained in the coded video bitstream and controlled by parameters made available to the loop filter unit 256 as symbols 270 from the parser 254, but may also respond to metadata obtained during decoding of previous portions (in the decoding order) of the coded picture or coded video sequence, as well as to previously reconstructed and loop-filtered sample values.

[0051]

[0063] The output of the loop filter unit 256 may be a sample stream that is output to a render device such as the display 124 and can also be stored in the reference picture memory 266 for use in future interpicture prediction.

[0052]

[0064] Some coded pictures, once fully reconstructed, can be used as reference pictures for future predictions. Once a coded picture is fully reconstructed and identified as a reference picture (for example, by parser 254), the current reference picture can become part of reference picture memory 266, and fresh current picture memory can be reallocated before starting the reconstruction of subsequent coded pictures.

[0053]

[0065] The decoder component 122 may perform decoding operations according to a predetermined video compression technique that may be documented in a standard, such as one of the standards described herein. The coded video sequence may comply with the syntax specified by the video compression technique or standard being used, in that it conforms to the syntax of the video compression technique or standard as specified in the video compression technique documentation or standard, and in particular in the profile documentation therein. Also, for compliance with some video compression techniques or standards, the complexity of the coded video sequence may be within limits defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (measured, for example, in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further limited through the virtual reference decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0054]

[0066] Figure 3 is a block diagram showing a server system 112 according to several embodiments. The server system 112 includes a control circuit 302, one or more network interfaces 304, memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuit 302 includes one or more processors (e.g., CPU, GPU, and / or DPU). In some embodiments, the control circuit includes one or more field-programmable gate arrays (FPGAs), hardware accelerators, and / or one or more integrated circuits (e.g., application-specific integrated circuits).

[0055]

[0067] The (one or more) network interface 304 may be configured to interface with one or more communication networks (e.g., wireless networks, wireline networks, and / or optical networks). The communication networks may be local, wide-area, metropolitan, automotive and industrial, real-time, latency-tolerant, etc. Examples of communication networks include local area networks such as Ethernet, wireless LAN, and cellular networks, including GSM, 3G, 4G, 5G, and LTE; TV wireline or wireless wide-area digital networks, including cable TV, satellite TV, and terrestrial broadcast TV; and automotive and industrial networks, including CANBus. Such communications may be unidirectional receive-only (e.g., broadcast TV), unidirectional transmit-only (e.g., CANbus to several CANbus devices), or bidirectional (e.g., with other computer systems using local or wide-area digital networks). Such communications may include communications to one or more cloud computing networks.

[0056]

[0068] The user interface 306 includes one or more output devices 308 and / or one or more input devices 310. The input devices 310 may include one or more of the following: a keyboard, mouse, trackpad, touchscreen, data glove, joystick, microphone, scanner, camera, etc. The output devices 308 may include one or more of the following: an audio output device (e.g., a speaker), a visual output device (e.g., a display or monitor), etc.

[0057]

[0069] Memory 314 may include high-speed random-access memory (such as DRAM, SRAM, DDR RAM, and / or other random-access solid-state memory devices) and / or non-volatile memory (such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state memory devices). Memory 314 optionally includes one or more storage devices located remotely from the control circuit 302. Memory 314, or alternatively, one or more non-volatile solid-state memory devices within Memory 314, includes a non-temporary computer-readable storage medium. In some embodiments, Memory 314, or the non-temporary computer-readable storage medium of Memory 314, stores the following programs, modules, instructions, and data structures, or subsets or supersets thereof: Operating System 316 includes procedures for handling various basic system services and for performing hardware-dependent tasks. • A network communication module 318 used to connect the server system 112 to other computing devices via one or more network interfaces 304 (for example, via wired and / or wireless connections). • A coding module 320 for performing various functions related to encoding and / or decoding data, such as video data. In some embodiments, the coding module 320 is an instance of the coder component 114. The coding module 320 is, but is not limited to, Regarding the decoder component 122, the decoding module 322 performs various functions related to decoding the encoded data, such as those previously described, Regarding the encoder component 106, the encoding module 340 performs various functions related to encoding data, as previously described. This includes one or more of the following, and For example, a picture memory 352 for storing pictures and picture data for use with the coding module 320. In some embodiments, the picture memory 352 includes one or more of the following: a reference picture memory 208, a buffer memory 252, a current picture memory 264, and a reference picture memory 266.

[0058]

[0070] In some embodiments, the decoding module 322 includes a parsing module 324 (configured to perform various functions previously described with respect to, for example, the parser 254), a conversion module 326 (configured to perform various functions previously described with respect to, for example, the scalar / inverse conversion unit 258), a prediction module 328 (configured to perform various functions previously described with respect to, for example, the motion compensation prediction unit 260 and / or the intrapicture prediction unit 262), and a filter module 330 (configured to perform various functions previously described with respect to, for example, the loop filter 256).

[0059]

[0071] In some embodiments, the coding module 340 includes a coding module 342 (configured to perform various functions previously described with respect to, for example, the source coder 202 and / or the coding engine 212) and a prediction module 344 (configured to perform various functions previously described with respect to, for example, the predictor 206). In some embodiments, the decoding module 322 and / or the coding module 340 includes a subset of the modules shown in Figure 3. For example, a shared prediction module is used by both the decoding module 322 and the coding module 340.

[0060]

[0072] Each of the modules identified above and stored in memory 314 corresponds to a set of instructions for performing the functions described herein. The modules identified above (e.g., sets of instructions) do not need to be implemented as separate software programs, procedures, or modules, and therefore various subsets of these modules may be combined or possibly rearranged in various embodiments. For example, the coding module 320 may optionally not include separate decoding and coding modules, but rather use the same set of modules to perform both sets of functions. In some embodiments, memory 314 stores a subset of the modules and data structures identified above. In some embodiments, memory 314 stores additional modules and data structures not described above, such as an audio processing module.

[0061]

[0073] In some embodiments, the server system 112 includes a web or hypertext transfer protocol (HTTP) server, a file transfer protocol (FTP) server, and web pages and applications implemented using Common Gateway Interface (CGI) scripts, PHP hypertext preprocessor (PHP), Active Server Pages (ASP), hypertext markup language (HTML), extensible markup language (XML), Java, JavaScript, asynchronous JavaScript and XML (AJAX), XHP, Javelin, Wireless Universal Resource File (WURFL), and the like.

[0062]

[0074] Figure 3 shows server systems 112 in several embodiments, but is intended to be a functional description of various features that may be present in one or more server systems, rather than a schematic diagram of the structures of the embodiments described herein. In practice, and as will be recognized by those skilled in the art, items shown separately may be combined, and some items may be separated. For example, some items shown separately in Figure 3 may be implemented on a single server, and a single item may be implemented by one or more servers. The actual number of servers used to implement server system 112, and how features are allocated among them, will vary depending on the implementation form and, by choice, will partially depend on the amount of data traffic that the server system will handle during peak and average usage periods.

[0063] Example intra-predictive mode

[0075] Figure 4 shows nominal angles in directional intra-prediction according to several embodiments. In some implementations of intra-prediction, the directional intra-modes can be further extended to angle sets with finer granularity to further leverage more spatial redundancy in the directional texture. For example, the VP9 coding format supports eight directional modes corresponding to angles from 45 to 207 degrees. In some embodiments, these eight directional modes are configured to provide eight nominal angles, called V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED, as shown in Figure 4.

[0064]

[0076] For each nominal angle, a predefined number (e.g., 7) finer angles may be added. Such extensions may make a larger total number (e.g., 56 in this example) of directional angles available for intra-prediction, corresponding to the same number of predefined directional intra-modes. In some embodiments, the predicted angle may be represented by the nominal intra-angle plus the angle delta. In the particular example above with seven finer angular directions for each nominal angle, the angle delta may be -3 to 3 multiplied by a step size of 3 degrees. To implement the directional prediction mode in a general manner, the directional intra-prediction mode is implemented with an integrated directional predictor that projects each pixel to a reference subpixel location and interpolates the reference pixels by a two-tap bilinear filter.

[0065]

[0077] In some embodiments, a predefined number of non-directional intra-prediction modes may also be predefined and made available. For example, five non-directional intra-prediction modes, namely DC, PAETH, SMOOTH, SMOOTH_V, and SMOOTH_H, may be specified. In some embodiments, non-directional intra-prediction modes are also known as non-directional smoothing intra-prediction modes.

[0066]

[0078] The prediction of samples for a particular block under these exemplary non-directional modes is shown in Figure 5. Figure 5 shows an exemplary 4x4 pixel block 502, which is predicted by samples from the upper and / or left adjacent lines of that block. The current pixel 510 in block 502 may correspond to sample 504 located above the current pixel 510, the upper-left sample 506 located at the intersection of the upper and left adjacent lines, and sample 508 located to the left of the current pixel 510. In DC intra-prediction mode, the average of the left adjacent sample and the upper adjacent sample is used as the predictor for the block to be predicted. In PAETH intra-prediction mode, the upper reference sample, left reference sample, and upper-left reference sample are fetched, and then the value closest to (upper + left - upper-left) is set as the predictor for the pixel to be predicted. SMOOTH, SMOOTH_V, and SMOOTH_H intra-prediction modes predict block 502 using quadratic interpolation in the vertical or horizontal direction, or by averaging in both directions. The non-directional intra-prediction modes described above are illustrative only. It will be apparent to those skilled in the art that other adjacent lines or non-directional modes may be selected and / or combined with the sample to be predicted in order to predict a particular sample within a prediction block.

[0067]

[0079] The encoder's selection of a specific intra-prediction mode from the above directional or non-directional modes at various coding levels (e.g., picture, slice, block, unit, etc.) can be signaled in the bitstream. In some embodiments, eight exemplary nominal directional modes (13 options in total) may be initially signaled along with five non-angle smoothing modes. In some embodiments, if the signaled mode is one of the eight nominal angle intra-modes, an index is further signaled to indicate the selected angle delta for the corresponding signaled nominal angle. In some embodiments, all intra-prediction modes may be indexed together for signaling (e.g., 56 directional modes + 5 non-directional modes, resulting in 61 intra-prediction modes). In some embodiments, the 56 or other number of exemplary directional intra-prediction modes may be implemented with an integrated directional predictor that projects each sample in a block to a reference subsample location and interpolates the reference sample by a two-tap bilinear filter.

[0068]

[0080] Figure 6 is a block diagram showing a lumera-to-chroma (CfL) prediction process 600 (performed by, for example, prediction module 328) according to several embodiments. CfL is an intra-prediction mode in which chroma pixels are modeled as a linear function of simultaneously reconstructed luma pixels. CfL prediction is expressed as follows: CfL(α) = α × L AC +DC (1)

[0069]

[0081] In equation (1), L AC α represents the AC contribution of the lumar component, α represents the parameter of the linear model, and DC represents the DC contribution of the chromatic component.

[0070]

[0082] Figure 6 shows that in the CfL prediction process 600, the reconstructed lumen pixels 602 are subsampled (604) (for example, downsampled) to the chroma resolution, and then averaged (606) to obtain the mean value 608. The mean value 608 is subtracted from the subsampled lumen pixels (610) to form the AC contribution 612 of the lumen component. The AC contribution 612 of the lumen component is then multiplied (614) by the scaling parameter α616 (defined in equation (1) above) to produce the scaled AC contribution 617 of the lumen component. The scaled AC contribution 617 of the lumen component is then added (618) to the DC contribution prediction (620) of the chroma component to obtain the chroma prediction sample (CfL prediction 622) of the predicted chroma block.

[0071]

[0083] In some embodiments, instead of requiring the decoder to compute a scaling parameter to approximate the chroma AC component from the AC contribution, the parameter α is determined based on the original chroma pixels (e.g., by the prediction module 328) and signaled in the bitstream. This reduces decoder complexity and results in more precise predictions. Regarding the DC contribution of the chroma component, it is computed using intra-DC mode, which is sufficient for most chroma content and has a mature, fast implementation.

[0072]

[0084] In some embodiments, when some lumens of a collated lumens block are outside the picture boundary, these lumens may be padded, and the padded lumens may be used to calculate the lumens mean (block 606). Figure 7 is a block diagram of lumens inside and outside the picture boundary according to some embodiments. The outer picture lumens sample 702 may be padded by dealing with the value of the nearest available sample in the block.

[0073]

[0085] In CfL mode, the lumern subsampling process is combined with the mean subtraction process, as shown in Figure 6. In this way, not only is the equation simplified, but the subsampling division and the corresponding rounding error are also eliminated. Equation (2) corresponds to the combination of both processes, which is simplified to equation (3). Both equations (2) and (3) use integer division. M × N is the matrix of pixels in the lumern plane.

[0074]

number

[0075]

[0086] Based on supported chroma subsampling, S x ×S y It can be shown that η ∈ {1, 2, 4} and that since both M and N are powers of 2, M × N is also a power of 2.

[0076]

[0087] For example, in the context of 4:2:0 chroma subsampling, instead of applying a box filter, the proposed method only requires summing the four reconstructed chroma pixels that match the chroma pixels. That is, 4 taps.

[0077]

number

[0078]

[0088] There can be different YUV formats depending on the chroma downsampling phase. Figure 8 shows different chroma downsampling formats according to several embodiments. Different chroma formats define different downsampling grids (phases) for different color components. In the 4:2:0 format, there are two different common downsampling formats, namely 4:2:0 MPEG1 or 4:2:0 MPEG2, as shown in Figure 8.

[0079]

[0089] In some embodiments, the current lumar downsampling filter in AV1 applies equation (4) to derive the reconstructed sample of the lumar.

[0080]

number

[0081]

[0090] The downsampling filter in AV1 may assume a chroma downsampling format corresponding to the 4:2:0 MPEG1 downsampling format, as shown in Figure 9. In some embodiments, multiple downsampling filters may be supported. In at least some of these embodiments, the filter type may be signaled, for example, by high-level syntax. In some embodiments, the multiple filters may include one or more 4-tap filters in AVI, one or more 6-tap filters, and another 4-tap filter.

[0082]

[0091] In entropy coding, syntax is binarized into 1s and 0s. Figure 10 shows various binarization schemes and their corresponding codes (e.g., binary symbols or bins) according to several embodiments. Generally, a binarization scheme defines a unique mapping of syntax element values ​​to a sequence of binary symbols (e.g., bins), which can also be interpreted in terms of a binary code tree.

[0083]

[0092] Several different binarization processes are used in HEVC, including k-th truncated Rice (TRk), k-th exponential Golomb (EGk), and fixed-length (FL) binarization. Parts of these forms of binarization, including the truncated unary (TrU) scheme as zero-th order TRk binarization, were also used in H.264 / AVC. These various methods of binarization can be described in terms of how they signal the unsigned value N. An example is also provided in Figure 10.

[0084]

[0093] Unary coding involves signaling a binstring of length N+1, where the first N bins are 1 and the last bin is 0. The decoder searches for 0 to determine when the syntax element is complete. In the TrU scheme, truncation is invoked for the largest possible value cMax1 of the syntax element being decoded.

[0085]

[0094] The k-th order truncated rice is a parameterized rice code consisting of a prefix and a suffix. The prefix is ​​a truncated unary string of value N >> k, where the largest possible value is cMax. The suffix is ​​a fixed-length binary representation of the least significant bin of N, where k is the number of the least significant bins. When k = 0, the truncated rice is equivalent to the truncated unary binarization.

[0086]

[0095] The k-th exponential Golomb code is a robust, near-optimal, prefix-free code for geometrically dispersed sources with unknown or fluctuating variance parameters. Each codeword has a length L N +1 unary prefix and length L N It consists of the suffix +k, where L N = log2((N>>k)+1).

[0087]

[0096] Fixed-length codes use a fixed-length bin string with length log2(cMax+1) and a top-level bin signaled before the lowest-level bin.

[0088]

[0097] In some embodiments, the binarization scheme (e.g., process) is selected based on the type of syntax element. In some embodiments, the binarization scheme is selected depending on the value of a previously processed syntax element (e.g., the binarization of coeff_abs_level_remaining depends on the previously decoded coefficient level) or depending on slice parameters indicating whether certain modes are enabled. For example, the binarization of partition mode, the so-called part_mode, depends on whether asymmetric motion partitioning is enabled. The majority of syntax elements use the binarization processes listed above, or some combination thereof (e.g., cu_qp_delta_abs uses TrU (prefix) + EG0 (suffix)). However, some syntax elements (e.g., part_mode and intra_chroma_pred_mode) use custom binarization processes.

[0089]

[0098] Figure 12 is a flowchart illustrating a method 1200 for video decoding according to several embodiments. The method 1200 can be implemented in a computing system (for example, a server system 112, a source device 102, or an electronic device 120) having a control circuit and a memory for storing instructions for execution by the control circuit. In some embodiments, the method 1200 is implemented by executing instructions stored in the memory of the computing system (for example, memory 314).

[0090]

[0099] The system receives a video stream having a sequence of frames (1202). The sequence of frames includes one or more keyframes. Each keyframe has a corresponding downsampling filter type. According to the determination that the current frame corresponds to a first keyframe among the one or more keyframes, the system extracts from the video bitstream a syntax element (e.g., a signal element) associated with the first downsampling filter type associated with the first keyframe (1204) (e.g., the syntax element is binarized to 1s and 0s). The system uses the first downsampling filter type to downsample the rumor block (e.g., rumor components, blocks of rumor pixels) of the current frame and a predefined set of frames immediately following the current frame (1206). In some embodiments, the predefined set of frames is a frame immediately following the current frame and immediately following a second keyframe among the one or more keyframes. The above system predicts the chroma blocks (e.g., chroma components, blocks of chroma pixels) of the current frame (e.g., a predefined set of the current frame and the frame immediately following it) based on downsampled frames (1208).

[0091]

[0100] In some embodiments, the downsampling filter type is signaled in each keyframe. In some embodiments, the downsampling filter type is signaled in instantaneous decoder refresh (IDR) frames. In some embodiments, the downsampling filter type is signaled in each intraframe. In some embodiments, the downsampling filter type is signaled in several selected keyframes. In some embodiments, the downsampling filter type is signaled in several selected intraframes.

[0092]

[0101] In some embodiments, the downsampling filter type is signaled at selected interframes. For example, the downsampling filter type is signaled at selected interframes associated with time layer identifiers (IDs) lower than a given threshold (e.g., thresholds corresponding to layer 1, layer 2, layer 3, or layer 4). Figure 11 shows an example of nine consecutive frames of a video bitstream with four time layers, according to some embodiments. Arrows indicate how those frames refer to other frames. For example, frame 4 is a frame using biprediction that refers to frames 0 and 8, and frame 3 is a frame using biprediction that refers to frames 2 and 4. In some embodiments, the video bitstream may contain thousands or tens of thousands of frames with tens or hundreds of time layers, each of which has its own time layer ID.

[0093]

[0102] According to some embodiments, the downsampling filter (or filter type) is signaled using one or more binarization methods (e.g., a binarization scheme, Figure 10).

[0094]

[0103] In some embodiments, downsampling filter types are signaled in fixed-length coding. For example, when there are N types of filters, a fixed length of M bits is used to signal the N filters. 2M is the smallest integer greater than or equal to N.

[0095]

[0104] In some embodiments, the downsampling filter type is signaled in variable-length coding. The downsampling filter type is binarized and signaled.

[0096]

[0105] In some embodiments, the first bin for signaling the downsample filter is to indicate whether the filter type is a 6-tap filter.

[0097]

[0106] In some embodiments, the semantics for the first bin of the downsample filter depend on whether the current frame is detected as screen content. For example, if the current frame is detected as screen content, the first bin for signaling the downsample filter is used to indicate whether the collated lumens sample is used directly without filtering. If the current frame is detected as non-screen content, the first bin for signaling the downsample filter is used to indicate whether the filter type is a 6-tap filter.

[0098]

[0107] In some embodiments, the downsampling filter type may be binarized using unary codewords (e.g., codes). This is shown in Figure 10.

[0099]

[0108] In some embodiments, the downsampling filter type may be binarized using a truncated unary codeword (e.g., a code). This is shown in Figure 10.

[0100]

[0109] In some embodiments, the downsampling filter type may be binarized using a truncated trice codeword (e.g., a code). This is shown in Figure 10.

[0101]

[0110] In some embodiments, the downsampling filter type can be binarized using an exponential Golomb codeword (e.g., a code), as shown in Figure 10.

[0102]

[0111] Table 1 shows an example binary table when there are three filter types.

[0103] [Table 1]

[0104]

[0112] Figure 12 shows several logical stages in a specific order, but the order-independent stages can be rearranged, and the other stages can be combined or separated. Any rearrangements or other groupings not described in detail will be apparent to those skilled in the art, and therefore the orderings and groupings presented herein are not exhaustive. Furthermore, it should be recognized that the stages can be implemented in hardware, firmware, software, or any combination thereof.

[0105]

[0113] Next, we will refer to some exemplary embodiments.

[0106]

[0114] (A1) In one embodiment, several embodiments include a method for video decoding (e.g., method 1200). In some embodiments, the method is implemented in a computing system having memory and a control circuit (e.g., server system 112). In some embodiments, the method is implemented in a coding module (e.g., coding module 320). The method includes (i) receiving a video stream having a sequence of frames, wherein the sequence of frames includes one or more keyframes, each keyframe having a respective downsampling filter type; (ii) extracting syntax elements (e.g., signal elements) from the video bitstream that relate to a first downsampling filter type associated with the first keyframe (e.g., the syntax elements are binarized to 1s and 0s) according to the determination that the current frame corresponds to a first keyframe among the one or more keyframes; (iii) downsampling the lumen block (e.g., lumen components, blocks of lumen pixels) of the current frame and a predefined set of frames immediately following the current frame using the first downsampling filter type to obtain a downsampled frame; and (iv) predicting the chroma block (e.g., chroma components, blocks of chroma pixels) of the current frame (e.g., and a predefined set of frames immediately following the current frame) based on the downsampled frame (e.g., using a prediction module 344). In some embodiments, the predefined set of frames is the frame immediately following the current frame and immediately following the second keyframe of one or more keyframes.

[0107]

[0115] (A2) In some embodiments of A1, the current frame includes multiple color components. The method further includes downsampling a first color component from the multiple color components of the current frame to obtain a first downsampled color component, and predicting the other color components from the multiple color components based on the first downsampled color component.

[0108]

[0116] (A3) In some embodiments of A1 or A2, one or more keyframes include intraframes.

[0109]

[0117] (A4) In some embodiments of A1 to A3, one or more keyframes correspond to any intraframe of the video stream.

[0110]

[0118] (A5) In some embodiments of A1 to A4, one or more keyframes include an instantaneous decoder refresh (IDR) frame. An IDR frame is a special type of I-frame. An IDR frame specifies that a frame following the IDR frame cannot reference a frame preceding it.

[0111]

[0119] (A6) In some embodiments of A1 to A5, one or more keyframes include one or more interframes, each of which is associated with a time layer ID lower than a predetermined threshold.

[0112]

[0120] (A7) In some embodiments of A1 to A6, the first downsampling filter type is a filter type corresponding to a luma-to-chroma (CfL) prediction mode for a video stream. In some embodiments, the first downsampling filter type is a filter type corresponding to a prediction mode that uses one color component to predict another color component, and downsampling is required with respect to one or more color components. In some embodiments, the first downsampling filter type is a filter type corresponding to a prediction mode in which the luma is replaced with one specific color component (e.g., R) and the chroma is replaced with another specific color component (e.g., G or B).

[0113]

[0121] (A8) In some embodiments of A1 to A7, the syntax elements have fixed-length coding.

[0114]

[0122] (A9) In some embodiments of A8, one or more keyframes are associated with N downsampling filter types, fixed-length coding corresponds to M bits, and N and M are related by N ≤ 2 M It is an integer that satisfies the following condition.

[0115]

[0123] (A10) In some embodiments of A1 to A9, the syntax element has variable-length coding.

[0116]

[0124] In some embodiments of A1 to A10, the syntax element includes a first attribute having an attribute value indicating whether the first downsampling filter type is a 6-tap filter.

[0117]

[0125] (A12) In some embodiments of A1 to A11, the syntax element includes a first attribute having an attribute value determined based on whether the current frame corresponds to detected screen content (for example, detected by the encoding module 340).

[0118]

[0126] (A13) In some embodiments of A12, when the current frame corresponds to detected screen content, the attribute value indicates whether the collated lumens sample is used without applying a downsampling filter. When the current frame does not correspond to detected screen content, the attribute value indicates whether the downsampling filter type is a 6-tap filter.

[0119]

[0127] (A14) In some embodiments of A1 to A13, the syntax elements are binarized using unary codes (see, for example, Figure 10).

[0120]

[0128] (A15) In some embodiments of A1 to A14, the syntax elements are binarized using truncated unary codes (see, for example, Figure 10).

[0121]

[0129] (A16) In some embodiments of A1 to A15, the syntax elements are binarized using truncated tricecode (see, for example, Figure 10).

[0122]

[0130] (A17) In some embodiments of A1 to A16, the syntax elements are binarized using exponential Golomb codes (see, for example, Figure 10).

[0123]

[0131] In other embodiments, some embodiments include a computing system (e.g., a server system 112) which includes a control circuit (e.g., control circuit 302) and a memory coupled to the control circuit (e.g., memory 314), the memory which stores one or more sets of instructions configured to be executed by the control circuit, the one or more sets of instructions which include instructions for performing any of the methods described herein (e.g., A1 to A16 above).

[0124]

[0132] In another embodiment, some embodiments include a non-temporary computer-readable storage medium that stores one or more sets of instructions for execution by a control circuit of a computing system, the one or more sets of instructions including instructions for performing any of the methods described herein (for example, A1 to A16 above).

[0125]

[0133] Terms such as "first," "second," etc., may be used herein to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are used merely to distinguish one element from another.

[0126]

[0134] The technical terms used herein are for the purpose of describing specific embodiments and do not limit the scope of the claims. In the descriptions of those embodiments and the appended claims, the singular forms “a,” “an,” and “the” are also to include the plural form unless the context otherwise explicitly indicates. Furthermore, the terms “and / or” as used herein should be understood to refer to and encompass one or more of any and all possible combinations of the listed items relating to the invention. In addition, the terms “comprises” and / or “comprising” as used herein specify the presence of the described features, assemblies, processes, operations, elements, and / or components, but should not be understood to exclude the presence or addition of one or more other features, assemblies, processes, operations, elements, components, and / or groups thereof.

[0127]

[0135] As used herein, the term “if” may be interpreted, depending on the context, as meaning “when” or “upon” or “in response to determining” or “in accordance with a determination” or “in response to detecting” that the stated condition precedent is true. Similarly, the phrases “if” or “if” or “when” may be interpreted, depending on the context, as meaning “upon determining” or “in response to determining” or “in accordance with a determination” or “upon detecting” or “in response to detecting” that the stated condition precedent is true.

[0128]

[0136] The above description has been provided with reference to specific embodiments for illustrative purposes. However, the above exemplary description is neither exhaustive nor does it limit the claims to the exact form disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments have been selected and described to best illustrate the principles of operation and practical applications, thereby enabling others skilled in the art.

Claims

1. A method for video decoding, which is implemented in a computing system having one or more processors and memory, wherein the method is Receiving a video stream having a sequence of frames, wherein the sequence of frames includes one or more keyframes, and each keyframe has a corresponding downsampling filter type. In accordance with the determination that the current frame corresponds to the first keyframe among the one or more keyframes, Extracting syntax elements from the video stream related to the first downsampling filter type associated with the first keyframe, To obtain a downsampled frame, the first downsampling filter type is used to downsample the rumor block of the current frame and a predefined set of frames immediately following the current frame. Based on the downsampled frame, predict the chroma block of the current frame. Methods that include...

2. The current frame includes multiple color components, and the method is To obtain a first downsampled color component, the first color component of the plurality of color components of the current frame is downsampled, Based on the first downsampled color component, predict the other color components among the plurality of color components. The method according to claim 1, further comprising:

3. The method according to claim 1, wherein one or more keyframes include an intraframe.

4. The method according to claim 1, wherein the one or more keyframes correspond to any intraframe of the video stream.

5. The method according to claim 1, wherein the one or more keyframes include an instantaneous decoder refresh (IDR) frame.

6. The method according to claim 1, wherein the one or more keyframes include one or more interframes, and each of the one or more interframes is associated with a time layer ID lower than a predetermined threshold.

7. The method according to claim 1, wherein the first downsampling filter type is a filter type corresponding to a lumen-to-chroma (CfL) prediction mode for the video stream.

8. The method according to claim 1, wherein the syntax element has a fixed-length coding.

9. The one or more keyframes are associated with N downsampling filter types, The aforementioned fixed-length coding corresponds to M bits, N and M are related N ≤ 2 Ｍ An integer that satisfies the following conditions: The method according to claim 8.

10. The method according to claim 1, wherein the syntax element has variable-length coding.

11. The method according to claim 1, wherein the syntax element includes a first attribute having an attribute value indicating whether the first downsampling filter type is a 6-tap filter.

12. The syntax element includes a first attribute having an attribute value determined based on whether the current frame corresponds to detected screen content. The method according to claim 1.

13. When the current frame corresponds to the detected screen content, the attribute value indicates whether the collated lumens sample is used without applying a downsampling filter. When the current frame does not correspond to the detected screen content, the attribute value indicates whether the downsampling filter type is a 6-tap filter. The method according to claim 12.

14. The method according to claim 1, wherein the syntax element is binarized using a unary code.

15. The method according to claim 1, wherein the syntax element is binarized using a truncated unary code.

16. The method according to claim 1, wherein the syntax element is binarized using a truncated liccode.

17. The method according to claim 1, wherein the syntax element is binarized using exponential Golomb code.

18. Control circuit and Memory and One or more sets of instructions stored in the memory and configured for execution by the control circuit, A computing system comprising, wherein one or more sets of instructions, Receiving a video stream having a sequence of frames, wherein the sequence of frames includes one or more keyframes, and each keyframe has a corresponding downsampling filter type. In accordance with the determination that the current frame corresponds to the first keyframe among the one or more keyframes, Extracting syntax elements from the video stream related to the first downsampling filter type associated with the first keyframe, To obtain a downsampled frame, the first downsampling filter type is used to downsample the rumor block of the current frame and a predefined set of frames immediately following the current frame. Based on the downsampled frame, predict the chroma block of the current frame. A computing system equipped with instructions for performing a certain task.

19. The computing system according to claim 18, wherein one or more keyframes include an intraframe.

20. A non-temporary computer-readable storage medium for storing one or more sets of instructions configured for execution by a computing device having a control circuit and memory, wherein the one or more sets of instructions are Receiving a video stream having a sequence of frames, wherein the sequence of frames includes one or more keyframes, and each keyframe has a corresponding downsampling filter type. In accordance with the determination that the current frame corresponds to the first keyframe among the one or more keyframes, Extracting syntax elements from the video stream related to the first downsampling filter type associated with the first keyframe, To obtain a downsampled frame, the first downsampling filter type is used to downsample the rumor block of the current frame and a predefined set of frames immediately following the current frame. Based on the downsampled frame, predict the chroma block of the current frame. A non-temporary computer-readable storage medium equipped with instructions for performing the following actions.

Images