Block quantization parameter adaptation system

Abstract

The present disclosure relates to a block quantization parameter adaptation system. In video compression, a quantization parameter (QP) determines the compression level and bit rate of a video. Block-level QP adaptation can improve video coding efficiency, but it is not trivial to determine a suitable QP at the block level without incurring significant overhead. To address this issue, QP adaptation can be performed on a QP unit having one or more blocks for which one or more filtered distortion values have been computed by a motion compensated temporal filter (MCTF). One or more minimum distortion values are used to compute a distortion metric, which is then used to determine a quantization parameter to be applied to the QP unit. The one or more minimum distortion values are also used to determine a size of the QP unit. Depending on whether the MCTF is on or off, different thresholds can be applied to the distortion metric.

Description

Background Technology

[0001] Video compression is a technology that makes video files smaller and easier to transmit over the internet. There are various methods and algorithms for video compression, each with different performance characteristics and advantages / disadvantages. Video compression involves encoding and decoding. Encoding is the process of converting (uncompressed) video data into a compressed format. Decoding is the process of restoring video data from the compressed format. An encoder-decoder system is called a codec. Attached Figure Description

[0002] The embodiments will be readily understood from the following detailed description and in conjunction with the accompanying drawings. For ease of description, similar reference numerals denote similar structural elements. In the accompanying figures, embodiments are illustrated by way of example rather than limitation.

[0003] Figure 1 The illustrations depict an encoding system and a plurality of decoding systems according to some embodiments of the present disclosure.

[0004] Figure 2 An exemplary encoder according to some embodiments of the present disclosure is illustrated for encoding video frames and outputting an encoded bitstream.

[0005] Figure 3 An exemplary decoder according to some embodiments of the present disclosure is illustrated for decoding an encoded bitstream and outputting a decoded video.

[0006] Figure 4 The illustration shows an exemplary encoder and preprocessing of a video frame according to some embodiments of the present disclosure.

[0007] Figure 5 The illustration shows a motion-compensated time filter and an exemplary block quantization parameter adaptive system according to some embodiments of the present disclosure.

[0008] Figure 6 The illustration shows adaptive dQP block size selection according to some embodiments of the present disclosure.

[0009] Figure 7 This is a flowchart illustrating a method for determining quantization parameters to be applied to a quantization parameter unit according to some embodiments of the present disclosure.

[0010] Figure 8 A block diagram of an exemplary computing device according to some embodiments of the present disclosure is described. Detailed Implementation

[0011] Overview

[0012] Video encoding, or video compression, is the process of compressing video data for storage, transmission, and playback. Video compression may involve acquiring large amounts of raw video data and applying one or more compression techniques to reduce the amount of data needed to represent the video while maintaining an acceptable level of visual quality. In some cases, video compression can provide efficient storage and transmission of video content over networks with limited bandwidth.

[0013] Video comprises one or more video frames or frames in a (time) sequence. A frame with a larger frame index or a frame associated with a later timestamp relative to the current frame can be considered a frame in the forward direction relative to the current frame. A frame with a smaller frame index or a frame associated with an earlier timestamp relative to the current frame can be considered a frame in the backward direction relative to the current frame. A frame can include an image or a single still image. A frame can have millions of pixels. For example, a frame of uncompressed 4K video can have a resolution of 3840 × 2160 pixels. Pixels can have luminance / luminance values ​​and chrominance / chrominance values. The terms "frame" and "picture" are used interchangeably.

[0014] There are several frame types for picture types. I-frames, or intra-frames, are the least compressible and do not depend on other frames for decoding. I-frames can include scene-changing frames. I-frames can be reference frames for one or more other frames. P-frames can depend on data from previous frames for decoding and are more compressible than I-frames. P-frames can be reference frames for one or more other frames. B-frames can depend on data from previous and forward frames for decoding and are more compressible than I-frames and P-frames. B-frames can reference two or more frames, such as a future frame and a past frame. Other frame types can include reference B-frames and non-reference B-frames. Reference B-frames can serve as a reference to another frame. Non-reference B-frames are not used as references to any frame. Reference B-frames are stored in the decoded picture buffer, while non-reference B-frames are not. P-frames and B-frames can be called inter-frames. The order or coding hierarchy of I-frames, P-frames, and B-frames can be called a group of pictures (GOP). In some cases, a frame can be an instantaneous decoder refresh (IDR) frame within a GOP. An IDR frame indicates that frames following the IDR frame should not reference any frames preceding the IDR frame. Therefore, an IDR frame signals to the decoder that it can clear the decoded image buffer. Each IDR frame can be an I-frame, but an I-frame may or may not be an IDR frame. A closed GOP can begin with an IDR frame. A slice can be a spatially distinct region of a frame that is encoded separately from any other region within the same frame.

[0015] In some cases, frames can be partitioned into one or more blocks. Blocks can be used for block-based compression. The pixel blocks resulting from partitioning can be called partitions. Blocks can have very small sizes, such as 512×512 pixels, 256×256 pixels, 128×128 pixels, 64×64 pixels, 32×32 pixels, 16×16 pixels, 8×8 pixels, 4×4 pixels, etc. Blocks can include square or rectangular areas of a frame. Different video compression techniques may use different terminology for blocks, or use different partitioning structures to create blocks. In some video compression techniques, frames can be partitioned into Coded Tree Units (CTUs) or macroblocks. The size of a CTU can be 32×32 pixels, 64×64 pixels, 128×128 pixels, or larger. The size of a macroblock can be between 8×8 pixels and 16×16 pixels. CTUs or macroblocks can be divided into Coded Units (CUs) or smaller blocks, for example, according to a tree structure (for the luma and chroma components, respectively). CUs or smaller can have sizes of 64×64 pixels, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels.

[0016] In video compression, the quantization parameter (QP) determines the compression level and bitrate of the video. Frame-level QP adaptation can assign different QP values ​​to different frames for bitrate control and compression efficiency. However, regions or blocks within a video frame can vary significantly in the spatial or temporal domains. For example, some blocks may contain finer textures and details, while others may be quite flat and smooth. Some blocks may have no visual activity for many frames, while others may have fast motion. Applying the same QP to all blocks within a frame is not optimal.

[0017] Block-level QP adjustment or adaptation is a technique for assigning different QP offsets or different incremental QPs (ΔQP) to different blocks in video coding. Block-level QP adaptation can further improve video coding efficiency to better adapt to the spatial and temporal characteristics of video content. While block-level QP adaptation can improve video coding efficiency, determining the appropriate QP at the block level without incurring significant overhead is not easy. In one block-level QP adaptation scheme, a look-ahead algorithm estimates the amount of information each macroblock contributes to the prediction of future frames and adjusts the QP of each macroblock based on its contribution. Similarly, in a similar block-level QP adaptation scheme, a look-ahead algorithm estimates the amount of information each CU contributes to the prediction of subsequent CUs. Both schemes are implemented in look-ahead coding processing, and the number of look-ahead frames is typically at least the number of frames within two seconds (e.g., 60 frames), which implies high computational complexity and long coding latency. In another scheme, an algorithm called Block Importance Mapping (BIM) estimates the likelihood that a block sample will be used as a reference in the coding of nearby pictures and adjusts the block QP accordingly. The BIM algorithm does not require look-ahead processing and utilizes metrics calculated by a GOP-based Motion Compensation Temporal Filter (MCTF), making it a lightweight and low-latency scheme. However, the BIM algorithm does not fully utilize the metrics from the MCTF, and the QP offset or ΔQP range is small, which reduces the coding quality gain. Furthermore, all these schemes use a fixed block size for all frames in the video sequence, without adjusting the block size according to the different characteristics of different frames.

[0018] To address one or more of these issues, QP adaptation on a QP unit (or dQP block) with one or more blocks can be performed using one or more filter distortion values ​​already computed by the MCTF for one or more blocks. Instead of considering only filter distortion values ​​related to the current block's time / frame distance of ±1 and ±2 (other values ​​are ignored and discarded), all available filter distortion values ​​are considered, such as those related to the current block's time / frame distance of ±1, ±2, ±3, and ±4. Considering additional filter distortion values ​​means taking more information into account when evaluating the importance of a block in the encoding process. Considering more information is beneficial because hierarchical GOP structures typically involve using more than two reference frames in both the forward and backward directions.

[0019] One or more minimum distortion values ​​are used to calculate a distortion metric, which is then used to determine the quantization parameters to be applied to the QP cell. Specifically, one or more minimum distortion values ​​are determined by considering available filtered distortion values. In some embodiments, the one or more minimum distortion values ​​include one or more of a forward minimum distortion value, a further forward minimum distortion value, a backward minimum distortion value, and a further backward minimum distortion value. The forward minimum distortion value may be the lowest filtered distortion value among one or more filtered distortion values ​​calculated for a block in the forward direction. The further forward minimum distortion value may be the second lowest filtered distortion value among one or more filtered distortion values ​​calculated for a block in the forward direction. The backward minimum distortion value may be the lowest filtered distortion value among one or more filtered distortion values ​​calculated for a block in the backward direction. The further backward minimum distortion value may be the second lowest filtered distortion value among one or more filtered distortion values ​​calculated for a block in the backward direction.

[0020] A distortion metric can be calculated using one or more subsets of minimum distortion values. Specifically, the distortion metric can be calculated using the minimum value from one or more subsets of minimum distortion values. This subset may include one or more of the forward minimum distortion value and the backward minimum distortion value. The minimum value determined for one or more blocks of a QP cell can be averaged and used as the distortion metric.

[0021] A distortion metric can be calculated using a further subset of one or more minimum distortion values. Specifically, a further distortion metric can be calculated using a further minimum of a further subset of one or more minimum distortion values. This further subset may include one or more of a further forward minimum distortion value and a further backward minimum distortion value. The further minimum values ​​determined for one or more blocks of a QP cell can be averaged and used as a further distortion metric.

[0022] The QP to be applied to a QP cell having one or more blocks can be determined based on the distortion metric and further distortion metrics, such as ΔQP or QP offset. A combined distortion metric for the QP cell can be determined based on the distortion metric and further distortion metrics. The QP for the QP cell can be determined based on the combined distortion metric.

[0023] In some implementations, a lookup table of combined distortion metrics with different ranges and corresponding QPs (e.g., ΔQP or QP offset offset from a selected / chosen QP value) can be used to determine a QP based on the combined distortion metric. The combined distortion metric calculated for a QP cell can be compared to one or more thresholds, which correspond to different QPs to be used for that QP cell.

[0024] In this paper, a QP cell (or dQP block) represents a QP-adaptive cell and may include one or more blocks, where the QP (e.g., ΔQP or QP offset) can vary from one QP cell to another (but the QP (e.g., ΔQP or QP offset) is the same for one or more blocks within the QP cell). A QP cell may include multiple 8×8 pixel blocks. As shown, a QP cell may include (but is not limited to) 64 8×8 pixel blocks (or 64×64 pixels), 16 8×8 pixel blocks (or 32×32 pixels), 4 8×8 pixel blocks (or 16×16 pixels), or 1 8×8 pixel block. Other schemes may be limited to adapting the QP to a fixed QP cell size and not to the different spatial and temporal characteristics of the pixels within the QP cell. To mitigate this problem, a QP-adaptive system may adjust the size of the QP cell based on the characteristics of the blocks within a CTU or macroblock. The characteristics of these blocks may be inferred based on filtered distortion values ​​already calculated by the MCTF. In some embodiments, one or more minimum distortion values ​​are also used to determine the size of the QP cell, such as the number of blocks or pixels in the QP cell. In some embodiments, an average value may be determined based on one or more minimum distortion values. For example, the average value may be determined based on a subset of one or more minimum distortion values, such as a forward minimum distortion value and a backward minimum distortion value. The variance of the average value determined for blocks in a CTU or macroblock may be determined. This variance may be used to determine the size of the QP cell, such as by comparing the variance to one or more thresholds corresponding to different possible sizes of the QP cell.

[0025] In some cases, applying the same lookup table with MCTF enabled or disabled is not optimal for encoder performance. MCTF is applied to frames before encoding, and QP is applied during encoding. Even if the distortion metric is calculated the same way, applying the same QP determined for the QP unit may have different effects on frames processed by MCTF and frames not processed by MCTF. To address this issue, different thresholds can be applied to the distortion metric depending on whether MCTF is enabled or disabled. In other words, the threshold can be determined based on whether MCTF is enabled or disabled. With MCTF enabled, the correlation between frames can be greater than when MCTF is disabled because the current frame is filtered along with adjacent frames. With MCTF enabled, a more aggressive QP adjustment can be applied (the threshold can be lowered to induce a larger QP adjustment) to account for the correlation.

[0026] To further improve the block-level QP adaptive system, more than four thresholds can be used to determine different QPs based on a combined distortion metric. More thresholds and corresponding ΔQPs can be used to enable a wider range of QP adaptations. In one example, eight thresholds with ΔQPs ranging from [-4, +4] are used in a lookup table to determine the QP for each QP cell.

[0027] In a series of experiments on various content types (e.g., natural, screen, dark, different resolutions, high noise levels, clean or low noise levels, high motion, video conferencing, rich textures, etc.), significant subjective and objective quality gains (ranging from 1% to 4.5%) were consistently observed using the features described herein compared to other block-level QP adaptive schemes. This scheme is codec-independent and can be applied prior to any encoder to improve encoder efficiency. The techniques for block-level QP adaptation described and illustrated herein can be applied to various codecs such as AVC (Advanced Video Coding), HEVC (High-Efficiency Video Coding), AV1 (AO Media Video 1), VVC (Various Video Coding), and VP9. AVC, also known as "ITU-TH.264," was approved in 2003, with the most recent revision dated August 22, 2021. HEVC, also known as "ITU-TH.265," was approved in 2013, with the most recent revision dated September 13, 2023. AV1 is a video coding codec designed specifically for video transmission over the Internet. The "AV1 Bitstream and Decoding Process Specification" version 1.1.1 (including errata) was last revised in 2019. VVC, also known as "ITU-TH.266", was finalized in 2020. VP9 is an open video codec, released on June 17, 2013.

[0028] Video compression

[0029] Figure 1 The illustration shows an encoding system 130 and one or more decoding systems 150 according to some embodiments of the present disclosure. 1…D .

[0030] Encoding system 130 can be implemented in Figure 8The encoding system 130 is implemented on the computing device 800 shown. The encoding system 130 can be implemented in the cloud or a data center. The encoding system 130 can be implemented on a device used to capture video. The encoding system 130 can be implemented on a stand-alone computing system. The encoding system 130 can perform the encoding process in video compression. The encoding system 130 can receive video (e.g., uncompressed video, raw video, original video, etc.) comprising a sequence of video frames 104. Video frames 104 can include image frames or images that constitute the video. The video can have a frame rate, or frames per second (FPS), which defines the number of frames per second of the video. The higher the FPS, the more realistic and smooth the video looks. Typically, FPS is greater than 24 frames per second to provide a natural and realistic viewing experience for human viewers. Examples of videos can include television series, movies, short films, short videos (e.g., less than 15 seconds long), videos capturing gaming experiences, computer screen content, video conferencing content, live event content, sports content, surveillance video, videos shot using mobile computing devices (e.g., smartphones), etc. In some cases, the video can include a mixture or combination of different types of video.

[0031] Encoding system 130 may include encoder 102, which receives video frames 104 and encodes the video frames 104 into an encoded bitstream 180. An exemplary implementation of encoder 102 is as follows... Figure 2 As shown.

[0032] The encoded bitstream 180 can be compressed, meaning its size can be smaller than video frame 104. The encoded bitstream 180 may include a sequence of bits, for example, consisting of 0s and 1s. The encoded bitstream 180 may have header information, payload information, and trailer information, which can be encoded as bits in the bitstream. The header information can provide information about one or more of the following: the format of the encoded bitstream 180, the encoding process implemented in encoder 102, the parameters of encoder 102, and metadata of the encoded bitstream 180. For example, the header information may include one or more of the following: resolution information, frame rate, aspect ratio, color space, etc. The payload information may include data representing the content of video frame 104, such as sample frames, symbols, syntax elements, etc. For example, the payload information may include bits representing the motion predictor, conversion coefficients, prediction mode, and quantization level of the encoded video frame 104. The trailer information can indicate the end of the encoded bitstream 180. The trailer information may include other information, including one or more of the following: checksum, error correction code, and signature. The format of the encoded bitstream 180 can vary according to the specifications of the encoding and decoding process (i.e., the codec).

[0033] The encoded bitstream 180 may include data packets in which encoded video data and signaling information can be packaged. An exemplary format is the Open Bitstream Unit (OBU), used in AV1 encoded bitstreams. An OBU may include a header and a payload. The header may include information about the OBU, such as information indicating the OBU type. Examples of OBU types include Sequence Header OBU, Frame Header OBU, Metadata OBU, Time Separator OBU, and Tiled-up Group OBU. The payload in the OBU may carry quantization conversion coefficients and syntax elements, which are used in the decoder to correctly decode the encoded video data to regenerate video frames.

[0034] The encoded bitstream 180 can be transmitted over network 140 to one or more decoding systems 150. 1…D Network 140 can be the Internet. Network 140 can include one or more of the following: cellular data network, wireless data network, wired data network, wired Internet network, fiber optic network, satellite Internet network, etc.

[0035] The diagram illustrates 150 D decoding systems. 1…D Decoding system 150 1…D At least one of them can be implemented in Figure 8 On computing device 800. System 150 1…D Examples can include personal computers, mobile computing devices, gaming devices, augmented reality devices, mixed reality devices, virtual reality devices, televisions, etc. Each decoding system has 150 [units / equipment / devices]. 1…D Both can perform the decoding process in video compression. Each decoding system has 150... 1…D It may include decoders (e.g., decoder 1…D 162) 1…D ) and one or more display devices (e.g., display device 1…D 164) 1…D ). Figure 3 The figure illustrates an exemplary implementation of the decoder, such as decoder 11621.

[0036] For example, decoding system 1 1501 may include decoder 1 1621 and display device 1 1641. Decoder 1 1621 may implement the decoding process for video compression. Decoder 1 1621 may receive encoded bitstream 180 and generate decoded video 1681. Decoded video 1681 may include a sequence of video frames, which may be a version or reconstructed version of video frame 104 encoded by encoding system 130. Display device 1 1641 may output decoded video 1681 to display to one or more human viewers or users of decoding system 1 1501.

[0037] For example, decoding system 2 1502 may include decoder 2 1622 and display device 2 1642. Decoder 2 1622 may implement the decoding process for video compression. Decoder 2 1622 may receive encoded bitstream 180 and generate decoded video 1682. Decoded video 1682 may include a sequence of video frames, which may be a version or reconstructed version of video frame 104 encoded by encoding system 130. Display device 2 1642 may output decoded video 1682 to be displayed to one or more viewers or users of decoding system 2 1502.

[0038] For example, the decoding system D 150 D It can include decoder D 162 D and display device D 164 D Decoder D 162 D This allows for the decoding process of video compression. Decoder D 162 D It can receive 180 encoded bitstreams and generate 168 decoded video. D Decoding video 168 D It may include a sequence of video frames, which may be a version or reconstructed version of video frame 104 encoded by encoding system 130. Display device D 164 D It can output decoded video 168 D To display to the decoding system D 150 D One or more viewers or users.

[0039] Video encoder

[0040] Figure 2 An encoder 102 according to some embodiments of the present disclosure is shown encoding a video frame 104 and outputting an encoded bitstream. The encoder 102 may include one or more of signal processing operations and data processing operations, including inter-frame and intra-frame prediction, transformation, quantization, intra-loop filtering, and entropy coding. The encoder 102 may include a reconstruction loop involving inverse quantization and inverse transformation to ensure that the decoder sees the same reference block and frame. The encoder 102 may receive the video frame 104 and encode the video frame 104 into an encoded bitstream 180. The encoder 102 may include one or more of partitioning 206, transformation and quantization 214, inverse transformation and inverse quantization 218, intra-loop filtering 228, motion estimation 234, inter-frame prediction 236, intra-frame prediction 238, and entropy coding 216.

[0041] In some embodiments, video frame 104 may be processed by preprocessing 290 before the encoding process is applied by encoder 102. Preprocessing 290 and encoder 102 can form as follows: Figure 1The encoding system 130 is shown. Preprocessing 290 can analyze video frame 104 to determine picture statistics, which can be used to inform one or more encoding processes to be performed by one or more components in encoder 102. Preprocessing 290 can determine information that can be used for QP adaptation, scene clipping detection, and frame type adaptation. Preprocessing 290 can determine a recommended frame type for each frame. Preprocessing 290 can apply MCTF to denoise video frame 104. The filtered version of video frame 104 with MCTF applied can be provided to encoder 102 as an input video frame (instead of video frame 104), for example, for partitioning 206. MCTF can include motion estimation analysis operations and bilateral filtering operations. MCTF can attenuate random picture components in a motion-aware manner to improve encoding efficiency. MCTF can operate on 8×8 pixel or 16×16 pixel blocks. MCTF can operate on luma and chroma values ​​separately. MCTF can be applied in three dimensions (e.g., spatial and temporal orientations). MCTF can generate noise estimates for individual blocks. In some embodiments, one or more operations of preprocessing 290 may be implemented as software instructions executed by a processor. In some embodiments, one or more operations of preprocessing 290 may be implemented using computational circuitry designed to perform one or more operations in hardware.

[0042] Partition 206 can divide video frame 104 (or a filtered version of video frame 104 from preprocessing 290) into pixel blocks. Different codecs can allow different variable ranges of block sizes. In one codec, a frame can be partitioned by partition 206 into blocks of 128×128 or 64×64 pixels. In some cases, a frame can be partitioned by partition 206 into blocks of 256×256 or 512×512 pixels. In some cases, a frame can be partitioned by partition 206 into blocks of 32×32 or 16×16 pixels. Larger blocks can be called superblocks, macroblocks, or CTUs. Partition 206 can further subdivide each larger block using a multi-way partition tree structure. In some cases, partitions of a superblock can be recursively further subdivided by partition 206 using a multi-way partition tree structure (e.g., to blocks / partitions of 4×4 size). In another codec, a frame can be partitioned by partition 206 into CTUs of 128×128 pixels. Partition 206 can divide the CTU into four CUs using a quadtree partitioning structure. Partition 206 can further recursively partition the CUs using a quadtree partitioning structure. Partition 206 can further subdivide the CUs using multiple types of tree structures (e.g., quadtree, binary tree, or ternary tree). The smallest CU can have a 4×4 pixel size. In this paper, a CU can be referred to as a block or partition. Partition 206 can output the original sample 208, for example, as a pixel block or partition.

[0043] In VVC, a frame in video frame 104 can be partitioned into multiple non-overlapping CTUs. CTUs have specified sizes, such as 128×128 pixels or 64×64 pixels. A CTU can be recursively divided into smaller blocks or partitions using different types of partition shapes. A CTU can be partitioned into 4 CUs using a quadtree partitioning structure. One or more CUs obtained through a quadtree partitioning structure can be recursively (e.g., up to three times) divided into smaller CUs using one of several structures, including, for example, quadtree, binary tree, or ternary tree structures, to support non-square partitions. A quadtree partitioning structure can partition a CU into 4 CUs. A binary tree partitioning structure can partition a CU into 2 CUs (e.g., horizontally or vertically). A ternary tree structure can partition a CU into 3 CUs (e.g., horizontally or vertically). The smallest CU (e.g., called a block or partition) can have a 4×4 pixel size. CUs can be larger than 4×4 pixels. It is understood that a CTU can be partitioned into CUs through multiple different feasible partitioning combinations. CTU can be partitioned in many different ways, resulting in multiple different partitioning results.

[0044] In some cases, one or more operations in partition 206 may be implemented in intra-frame prediction 238 and / or inter-frame prediction 236.

[0045] Intra-frame prediction 238 can predict samples of a block or partition based on reconstructed prediction samples of previously encoded spatially adjacent / reference blocks in the same frame. Intra-frame prediction 238 can receive reconstructed prediction samples 226 (of previously encoded spatially adjacent blocks in the same frame). Reconstructed prediction samples 226 can be generated by summer 222 based on reconstructed prediction residuals 224 and prediction samples 212. Intra-frame prediction 238 can determine a suitable predictor for the prediction samples based on the reconstructed prediction samples of previously encoded spatially adjacent / reference blocks in the same frame (thus making an intra-frame prediction decision). Intra-frame prediction 238 can generate prediction samples 212 using the suitable predictor. Intra-frame prediction 238 can output or identify adjacent / reference blocks and the predictor used to generate prediction samples 212. The identified adjacent / reference blocks and predictors can be encoded in the encoded bitstream 180 to enable the decoder to reconstruct blocks using the same adjacent / reference blocks and predictors. In a codec, intra-frame prediction 238 can support multiple different predictors, such as 56 different predictors. In a codec, intra-prediction 238 can support multiple different predictors, such as 95 different predictors. Some predictors, such as orientation predictors, can capture different spatial redundancies in the orientation texture. The pixel values ​​of a block can be predicted using the orientation predictor in intra-prediction 238 by extrapolating the pixel values ​​of adjacent / reference blocks along a specific orientation. Intra-prediction 238 in different codecs can support different sets of predictors to take advantage of different spatial patterns within the same frame. Examples of predictors can include DC, planar, Paeth, smooth, smooth vertical, smooth horizontal, recursive filtering patterns, chroma-from-luma conversion, IBC, palette or palette coding, multiple reference lines, intra-sub-partitioning, matrix-based intra-prediction (matrix coefficients can be defined by offline training using a neural network), angle prediction, wide-angle prediction, cross-component linear model, template matching, etc. IBC predicts the current block by working by copying a reference block within the same frame. Palette encoding or palette mode works by using a palette with a small number of colors (e.g., 2-8 colors) and encoding the current block using a palette index. In some cases, intra-frame prediction 238 can perform block prediction, where the predicted block can be generated using vectors based on reconstructed neighboring / reference blocks in the same frame. Optionally, a specific type of interpolation filter can be applied to the predicted block to blend the pixels of the predicted block. The pixel values ​​of the block can be predicted using the vector compensation process in intra-frame prediction 238 by generating prediction samples 212 based on vector translation (within the same frame) of neighboring / reference blocks (and optionally applying interpolation filters to the neighboring / reference blocks). Intra-frame prediction 238 can output or recognize the vectors applied to generate the prediction samples 212.In some codecs, intra-prediction 238 can encode (1) the residual vector generated based on the applied vector and the vector predictor candidates, and (2) information identifying the vector predictor candidates, rather than encoding the applied vector itself. Intra-prediction 238 can output or identify the type of interpolation filter applied to generate the prediction sample 212.

[0046] Motion estimation 234 and inter-frame prediction 236 can predict block samples based on samples from previously encoded frames (e.g., reference frames in the decoded image buffer 232). Motion estimation 234 and inter-frame prediction 236 can perform operations to make inter-frame prediction decisions or inter-frame prediction decisions. Motion estimation 234 can perform motion analysis and determine motion information for the current frame. Motion estimation 234 can determine the motion field of the current frame. The motion field can include motion vectors of blocks in the current frame. Motion estimation 234 can determine the average magnitude of the motion vectors in the current frame. Motion estimation 234 can determine motion information that indicates how much motion is present in the current frame (e.g., large motion, very dynamic motion, small / micro motion, very static).

[0047] Motion estimation 234 and inter-frame prediction 236 can perform motion compensation, which may involve identifying suitable reference blocks and suitable motion predictors (or motion vector predictors) for the blocks, and optionally identifying interpolation filters to be applied to the reference blocks. Motion estimation 234 may receive raw samples 208 from partition 206. Motion estimation 234 may receive samples (e.g., samples from previously encoded frames or reference frames) from decoded picture buffer 232. Motion estimation 234 may use multiple reference frames to determine one or more suitable motion predictors. The motion predictor may include reference blocks and motion vectors that can be applied to generate motion-compensated blocks or predicted blocks. The motion predictor may include motion vectors for capturing inter-frame block motion in the video. Motion estimation 234 may output or identify one or more reference frames and one or more suitable motion predictors. Inter-frame prediction 236 may be applied to one or more suitable motion predictors determined in motion estimation 234 and one or more reference frames to generate predicted samples 212. The identified reference frames(s) and motion predictors(s) can be encoded in the encoded bitstream 180 to enable the decoder to reconstruct blocks using the same reference frames(s) and motion predictors(s). In a codec, motion estimation 234 can implement a single reference frame prediction mode, where a single reference frame with a corresponding motion predictor is used for inter-frame prediction 236. Motion estimation 234 can implement a composite reference frame prediction mode, where two reference frames with two corresponding motion predictors are used for inter-frame prediction 236. In a codec, motion estimation 234 can implement techniques for searching and identifying good reference frames that can produce the most efficient motion predictor. Techniques in motion estimation 234 can include searching for good reference frame(s) candidates spatially (within the same frame) and temporally (in previously encoded frames). Techniques in motion estimation 234 can include searching deep spatial neighborhoods to find a spatial candidate pool. Techniques in motion estimation 234 can include utilizing a temporal motion field estimation mechanism to generate a temporal candidate pool. Technique 234 in motion estimation 234 can use a motion field estimation process. After the temporal and spatial candidates are sorted, a suitable motion predictor can be determined. In a codec, inter-frame prediction 236 can support multiple different motion predictors. Examples of predictors can include geometric motion vectors (for complex nonlinear motion), warp motion compensation (for affine transformations capturing non-translating object motion), overlapping block motion compensation, advanced composite prediction (composite wedge prediction, differential modulation mask prediction, frame-range-based composite prediction, and composite inter-frame-intra-frame prediction), dynamic spatial and temporal motion vector references, affine motion compensation (for capturing higher-order motions such as rotation, scaling, and shearing), adaptive motion vector resolution modes, geometric partitioning modes, bidirectional optical flow, prediction refinement using optical flow, weighted bidirectional prediction, extended merging prediction, and so on.Optionally, a specific type of interpolation filter can be applied to the prediction block to blend the pixels of the prediction block. The pixel values ​​of the block can be predicted using a motion predictor / vector determined during motion compensation in motion estimation 234 and inter-frame prediction 236, and the interpolation filter can be selectively applied. In some cases, inter-frame prediction 236 can perform motion compensation, where the prediction block can be generated using a motion predictor / vector based on a reconstructed reference block of a reference frame. Inter-frame prediction 236 can output or identify the motion predictor / vector applied to generate the prediction sample 212. In some codecs, inter-frame prediction 236 can encode (1) the residual vector generated based on the applied vector and the vector predictor candidate, and (2) information identifying the vector predictor candidate, instead of encoding the applied vector itself. Inter-frame prediction 236 can output or identify the type of interpolation filter applied to generate the prediction sample 212.

[0048] Mode selection 230 can be notified by components such as motion estimation 234 to determine whether inter-frame prediction 236 or intra-frame prediction 238 is more effective for the coded block (thus making a coding decision). Inter-frame prediction 236 can output prediction samples 212 for the prediction block. Inter-frame prediction 236 can output a selected predictor and a selected interpolation filter (if applicable), which can be used to generate the prediction block. Intra-frame prediction 238 can output prediction samples 212 for the prediction block. Intra-frame prediction 238 can output a selected predictor and a selected interpolation filter (if applicable), which can be used to generate the prediction block. Regardless of the mode, prediction residual 210 can be generated by subtracting prediction samples 212 from the original sample 208 using subtractor 220. In some cases, prediction residual 210 may include residual vectors from inter-frame prediction 236 and / or intra-frame prediction 238.

[0049] Transformation and quantization 214 can receive prediction residual 210. Prediction residual 210 can be output by subtracting prediction sample 212 from original sample 208 using subtractor 220. Prediction residual 210 can be referred to as the prediction error of intra-frame prediction 238 and inter-frame prediction 236 (e.g., the error between original sample and prediction sample 212). Prediction error has a smaller range of values ​​than original sample and can be encoded in coded bitstream 180 using fewer bits. Transformation and quantization 214 can include one or more of the transformation and quantization. Transformation can include transforming prediction residual 210 from the spatial domain to the frequency domain. Transformation can include applying one or more transformation kernels. Examples of transformation kernels can include horizontal and vertical forms of Discrete Cosine Transform (DCT), Asymmetric Discrete Sine Transform (ADST), Flipped ADST, and Identity Transform (IDTX), Multiple Transform Selection, Low-Frequency Inseparable Transform, Sub-Block Transform, Non-Square Transform, DCT-VIII, Discrete Sine Transform VII (DST-VII), Discrete Wavelet Transform (DWT), etc. The conversion can transform the prediction residual 210 into conversion coefficients. Quantization can, for example, reduce the precision of the conversion coefficients. Quantization can include using a quantization matrix with quantization parameters (e.g., a linear quantization matrix and a nonlinear quantization matrix). Elements in the quantization matrix can be larger for higher frequency bands and smaller for lower frequency bands, meaning that high-frequency coefficients are quantized more coarsely, while low-frequency coefficients are quantized more finely. Quantization can include dividing each conversion coefficient by the corresponding element in the quantization matrix (e.g., the quantization parameter) and rounding to the nearest integer. In practice, the quantization matrix can implement different QPs for different frequency bands and chromaticity planes, and spatial prediction can be used. A suitable quantization matrix can be selected for each frame and a signal emitted, which is then encoded into the encoded bitstream 180. The conversion and quantization 214 can output quantized conversion coefficients and syntax elements 278, which indicate the encoding mode and parameters used in the encoding process implemented in encoder 102.

[0050] In this article, QP refers to a parameter in video coding that controls the compression level by determining how much detail is preserved or discarded during the encoding process. QP ranges from 0 to 51, with lower values ​​maintaining higher quality but resulting in larger file sizes, while higher values ​​increase compression but introduce more visual artifacts. The QP value directly affects how DCT coefficients are divided and rounded during conversion and quantization. Larger QP values ​​lead to more aggressive rounding, effectively removing high-frequency details that are imperceptible to human vision. This parameter is used in the encoder's rate-distortion optimization process, allowing the encoder to balance visual quality with bandwidth constraints. Modern encoders can dynamically adjust the QP value at both the frame and macroblock levels to optimize compression based on scene complexity and motion. In some cases, QP adjustment is performed using ΔQP or QP offset relative to a base QP. ΔQP (or QP offset) is a mechanism in video coding that allows relative adjustments to the base QP value for a specific coding unit or frame type. These offsets enable conversion and quantization to apply different compression levels to different parts of the video stream, thereby optimizing the balance between quality and bitrate. For example, B-frames typically use higher QP values ​​(positive increments (+Δ)) than I-frames because they are less important to overall quality; while visually important regions may receive negative ΔQP to preserve more detail. In many encoders, ΔQP can be configured for various structural elements, such as slice type, hierarchical coding layers, or specific regions of interest within a frame. This granular control over quantization helps achieve better subjective quality by allocating more bits to visually important content while maintaining effective compression for less noticeable areas.

[0051] In some embodiments, the quantization parameters (QP) used by the transformation and quantization 214 are determined by preprocessing 290. Preprocessing 290 can generate one or more quantization parameters to be used by the transformation and quantization 214. Determining the optimal QP is not easy, as the QP directly affects the encoder's performance.

[0052] Inverse transformation and inverse quantization 218 can be applied to the inverse operations performed in transformation and quantization 214 to generate a reconstruction prediction residual 224 as part of the reconstruction path, thereby generating a decoded picture buffer 232 for encoder 102. Inverse transformation and inverse quantization 218 can receive quantized transformation coefficients and syntax elements 278. Inverse transformation and inverse quantization 218 can perform one or more inverse quantization operations, such as applying an inverse quantization matrix, to obtain unquantized / raw transformation coefficients. Inverse transformation and inverse quantization 218 can perform one or more inverse transformation operations, such as inverse transformations (e.g., inverse DCT, inverse DWT, etc.), to obtain the reconstruction prediction residual 224. A reconstruction path is provided in encoder 102 to generate reference blocks and frames, which are stored in the decoded picture buffer 232. These reference blocks and frames can be matched with blocks and frames to be generated in the decoder. Reference blocks and frames are used as reference blocks and frames by motion estimation 234, inter-frame prediction 236, and intra-frame prediction 238.

[0053] In-loop filter 228 can implement filters for smoothing artifacts introduced by the encoding process in encoder 102 (e.g., processing performed by partition 206 and transform quantization 214). In-loop filter 228 can receive reconstructed prediction samples 226 from summer 222 and output frames to decoded picture buffer 232. Examples of in-loop filters can include constrained low-pass filters, orientation de-encircling filters, edge orientation conditional replacement filters, loop recovery filters, Wiener filters, self-guided recovery filters, constrained orientation enhancement filters (CDEF), LMCS filters, sample adaptive offset (SAO) filters, adaptive loop filters (ALF), cross-component ALF, low-pass filters, deblocking filters, etc. For example, applying a deblocking filter at the boundary between two blocks can resolve blockage artifacts caused by the Gibbs phenomenon. In some embodiments, in-loop filter 228 can acquire data from a frame buffer containing reconstructed prediction samples 226 of individual blocks of a video frame. In-loop filter 228 can determine whether to apply an in-loop filter. In-loop filter 228 can determine one or more suitable filters to achieve good visual quality and / or to appropriately remove artifacts introduced by the encoding process in encoder 102. In-loop filter 228 can determine one or more intensities (e.g., filter coefficients) of the in-loop filter to be applied to the boundary between two blocks based on the reconstructed prediction samples 226. In some cases, in-loop filter 228 can take into account the desired bit rate when determining one or more suitable filters. In some cases, in-loop filter 228 can take into account a specified QP when determining one or more suitable filters. In-loop filter 228 can apply one or more (suitable) filters at the boundary between two blocks. After applying one or more (suitable) filters, in-loop filter 228 can write the (filtered) reconstructed samples to a frame buffer, such as decoded picture buffer 232.

[0054] Entropy coding 216 can receive quantized transformation coefficients and syntax elements 278 (e.g., referred to as symbols herein) and perform entropy coding. Entropy coding 216 can generate and output an encoded bitstream 180. Entropy coding 216 can utilize statistical redundancy and apply lossless algorithms to encode symbols and generate a compressed bitstream, such as encoded bitstream 180. Entropy coding 216 can implement several versions of arithmetic coding. Different versions can have different advantages and disadvantages. In one codec, entropy coding 216 can implement (symbol-to-symbol) adaptive multi-symbol arithmetic coding. In another codec, entropy coding 216 can implement a context-based adaptive binary arithmetic encoder (CABAC). Binary arithmetic coding differs from multi-symbol arithmetic coding. Binary arithmetic coding encodes only one bit at a time, for example, with a binary value of 0 or 1. Binary arithmetic coding can first convert each symbol to a binary representation (e.g., each symbol uses a fixed number of bits). Processing only binary values ​​0 or 1 simplifies computation and reduces complexity. Binary arithmetic coding assigns a probability to each binary value (e.g., the probability that the bit has a binary value of 0 and the probability that the bit has a binary value of 1). Multi-symbol arithmetic coding encodes an alphabet with at least two or three symbol values ​​and assigns a probability to each symbol value in the alphabet. Multi-symbol arithmetic coding can encode more bits at once, which can make the number of operations required to encode the same amount of data less. However, multi-symbol arithmetic coding can require more computation and storage (because the probability estimate may need to be updated for each element in the alphabet). Maintaining and updating the probability of each possible symbol value (e.g., cumulative probability estimate) in multi-symbol arithmetic coding can be more complex (e.g., the complexity increases with the size of the alphabet). Multi-symbol arithmetic coding should not be confused with binary arithmetic coding because these two different entropy coding processes are implemented differently and can result in different encoded bitstreams for the same set of quantized transformation coefficients and syntax elements 278.

[0055] Video decoder

[0056] Figure 3The diagram illustrates a decoder 1 1621 according to some embodiments of the present disclosure, used for decoding an encoded bitstream and outputting decoded video. Decoder 1 1621 may include one or more of signal processing operations and data processing operations, including entropy decoding, inverse conversion, inverse quantization, inter-frame and intra-frame prediction, intra-loop filtering, etc. Decoder 1 1621 may have signal processing and data processing operations mirroring those performed in the encoder. Decoder 1 1621 may apply signal and data processing operations notified by signals in the encoded bitstream 180 to reconstruct the video. Decoder 1 1621 may receive the encoded bitstream 180 and generate and output a decoded video 1681 having multiple video frames. Decoded video 1681 may be provided to one or more display devices for display to one or more human viewers. Decoder 1 1621 may include one or more of entropy decoding 302, inverse conversion and inverse quantization 218, intra-loop filtering 228, inter-frame prediction 236, and intra-frame prediction 238. Some functions have been previously described and used in encoders, such as... Figure 2 Encoder 102.

[0057] Entropy decoder 302 can decode encoded bitstream 180 and output the symbols encoded in encoded bitstream 180. These symbols may include quantized transformation coefficients and syntax elements 278. Entropy decoder 302 can reconstruct these symbols from encoded bitstream 180.

[0058] The inverse transform and inverse quantization 218 can receive quantized transform coefficients and syntax elements 278, and perform operations performed in the encoder. The inverse transform and inverse quantization 218 can output reconstructed prediction residuals 224. The summer 222 can receive the reconstructed prediction residuals 224 and prediction samples 212, and generate reconstructed prediction samples 226. The inverse transform and inverse quantization 218 can output syntax elements 278 with signaling information for notifying / indicating / controlling operations in the decoder 1 1621, such as mode selection 230, intra-frame prediction 238, inter-frame prediction 236, and in-loop filters 228.

[0059] Based on the prediction pattern indicated by the signal in the encoded bitstream 180 (e.g., as quantized conversion coefficients and syntax elements in syntax element 278), intra-frame prediction 238 or inter-frame prediction 236 can be applied to generate prediction samples 212.

[0060] Summer 222 sums the prediction samples 212 of the decoded reference block and the reconstructed prediction residual 224 to generate reconstructed prediction samples 226 of the reconstructed block. For intra-frame prediction 238, the decoded reference block can be located in the same frame as the block being decoded or reconstructed. For inter-frame prediction 236, the decoded reference block can be located in a different (reference) frame in the decoded image buffer 232.

[0061] Intra-frame prediction 238 can determine a reconstructed vector based on the residual vector and selected vector predictor candidates. Intra-frame prediction 238 can apply a reconstructed predictor or vector (e.g., predictor information provided by the signal) to a reconstructed block, which can be generated using a decoded reference block from the same frame. Intra-frame prediction 238 can apply an appropriate interpolation filter type (e.g., interpolation filter information provided by the signal) to the reconstructed block to generate prediction samples 212.

[0062] Inter-frame prediction 236 can determine reconstructed vectors based on residual vectors and selected vector predictor candidates. Inter-frame prediction 236 can apply the reconstructed predictor or vector (e.g., predictor information provided by the signal) to a reconstructed block, which can be generated using decoded reference blocks from different frames of the decoded image buffer 232. Inter-frame prediction 236 can apply an appropriate interpolation filter type (e.g., interpolation filter information provided by the signal) to the reconstructed block to generate predicted samples 212.

[0063] The in-loop filter 228 can receive the reconstructed prediction sample 226. The in-loop filter 228 can apply one or more filters signaled in the encoded bitstream 180 to the reconstructed prediction sample 226. The in-loop filter 228 can output the decoded video 1681.

[0064] A coding system with improved block-level QP adaptive coding

[0065] To improve other block-level QP adaptive systems Figure 4-7 The diagram illustrates an algorithm and its variants that can improve encoder performance with minimal overhead to the encoding process.

[0066] Figure 4 An encoder 102 and preprocessing 290 for a video frame according to some embodiments of the present disclosure are illustrated. In some embodiments, an MCTF 406 and a block QP adaptive system 444 may be included in the preprocessing 290 prior to the encoder 102. The encoder 102 and the block QP adaptive system 444 may form an encoding system 130.

[0067] Video frame 104 can be provided to MCTF 406. MCTF 406 can generate a filtered frame 430 and provide the filtered frame 430 to encoder 102. MCTF 406 can receive video frame 104 and analyze consecutive frames to detect and quantize motion between frames. MCTF 406 can generate motion vectors describing the movement of detected objects. Motion vectors can be generated by first estimating motion between consecutive frames using block matching or optical flow algorithms to track how objects move in the scene. MCTF 406 can align 8×8 pixel blocks from adjacent frames based on the generated motion vectors. MCTF 406 can apply temporal filtering to the aligned 8×8 pixel blocks to reduce noise while maintaining motion integrity and generate filtered frame 430. Temporal filtering can effectively average the corresponding pixels over time while taking motion into account. MCTF 406 can reduce noise and artifacts while maintaining image quality by utilizing information from multiple frames.

[0068] Video frame 104 may be provided to block QP adaptive system 444. One or more filter distortion values ​​490 may be provided to block QP adaptive system 444. Block QP adaptive system 444 may determine one or more quantization parameters (e.g., one or more ΔQP or QP offsets) to be applied to one or more blocks of the frame as part of ΔQP mapping 440. Block QP adaptive system 444 may determine one or more quantization parameters based on one or more filter distortion values ​​490. ΔQP mapping 440 may include one or more ΔQP values ​​to be applied to one or more blocks of the frame. ΔQP mapping 440 may be provided as input to encoder 102. For example, ΔQP mapping 440 may be... Figure 2 The conversion and quantization in 214 are used.

[0069] MCTF 406, for example, for random access GOP size 8 encoding, implements a GOP-based temporal filter every 8th frame, or a filter applied to lower GOP layers, to compute statistics about the motion-compensated block. These statistics can be used as an indicator to estimate the block's importance, such as for other frames where the block will be used as... Figure 2The probability of references in inter-frame prediction 236 and motion estimation 234. The QP of important blocks can be reduced to increase the quality of important blocks. The QP of unimportant blocks can be increased to reduce the quality of unimportant blocks (or reduce the number of coded bits). In some embodiments, after motion compensation for a given 8×8 pixel block is completed, the MCTF 406 can determine one or more filter distortion values ​​490 for one or more reference images of the GOP-based temporal filter for that 8×8 pixel block. The one or more filter distortion values ​​490 can include a metric measuring the accuracy or error of motion compensation for the given 8×8 pixel block. In this document, the one or more filter distortion values ​​490 may be referred to as one or more E metrics.

[0070] Video frame 104 can also be provided to encoder 102 (e.g., if MCTF 406 is turned off or bypassed).

[0071] Figure 5 The illustration shows an MCTF 406 and a block QP adaptive system 444 according to some embodiments of the present disclosure.

[0072] The MCTF 406 can process individual CTUs (or macroblocks), such as CTU 502. A CTU may include multiple 8×8 pixel blocks, such as 64 8×8 pixel blocks. In some implementations, the MCTF 406 can apply filtering at the block level (e.g., at the 8×8 pixel block level) and calculate one or more filter distortion values ​​490, or one or more E-metric values ​​for each block of the CTU (e.g., CTU 502). The block-level filter distortion value can be determined by a corresponding filter distortion value calculation, such as 8×8 block filter distortion value calculation 504. For a given block of the CTU, one or more filter distortion values ​​490E can be calculated based on the motion-compensated block and relative to each neighboring / reference frame at different distances. d，x，y “d” represents the time / frame distance from the current block. In some implementations, “d” can have the following time / frame distances: [+4,+3,+2,+1,-1,-2,-3,-4]. “x,y” represents the index of the current block in the CTU.

[0073] The filter distortion value calculated in MCTF 406 using an 8×8 block is 490 (E). d，x，y E can be used as an indicator or heuristic for evaluating the importance of a given block. After motion compensation, E can be calculated for each 8×8 luma block according to the following equation. d，x，y :

[0074]

[0075] Sum of Squared Differences (SSD) d，x，yV represents the sum of the squared differences in pixel values ​​between a given block at x, y in the current frame and a motion-compensated block at time / frame distance d in an adjacent / reference frame. x，y This is the variance of the current luminance block. s is the block size. In some implementations, the MCTF 406 block size is fixed at 8×8, and s is always equal to 8. For SSDs with low... d，x，y Value and low relative error (e.g., SSD) d，x，y / V x，y Equation 1, which is a block of SSD (or the ratio of SSD to variance), can produce a low filter distortion value E. d，x，y This indicates that the block is of high importance.

[0076] For a given block (e.g., an 8×8 pixel block), the following filter distortion value 490 (E) can be calculated. d，x，y ):E +4，x，y E +3，x，y E +2，x，y E +1，x，y E -1，x，y E -2，x，y E -3，x，y and E -4，x，y 8×8 block filter distortion value calculation 504 Based on the sum of the squared differences between the block and the motion compensation block, and the variance of the block, determine one or more filter distortion values ​​for the block 490 (E d，x，y The 8×8 block filter distortion value calculation 504 determines the filter distortion value 490 at different time / frame distances (±1, ±2, ±3, and ±4). The block filter distortion value 490 is determined by MCTF 406.

[0077] In some implementations, adaptive QP can operate at the CTU (or macroblock) level, setting different QPs for different CTUs (and using the same QP for all blocks within a given CTU). For a given CTU, the filtered distortion value of each block within the CTU can be 490 (E d，x，y Aggregates to form a combined distortion metric E at the CTU level (or macroblock level). combined，i “i” represents the index of the intra-frame CTU. Macroblock-level combined distortion metric E combined,i It can be used to determine the quantization parameters (e.g., ΔQP or QP offset) to be applied to the CTU or macroblock.

[0078] At the CTU level, considering the filtered distortion values ​​at different time / frame distances and different blocks within the CTU, the distortion metric E1 can be determined. i And further distortion measure E2 i .

[0079] Briefly referencing a scheme that may have performance below that of an ideal encoder, the distortion metric E1... iAnd further distortion measure E2 i The calculation can be performed as follows:

[0080]

[0081] N is the number of filtered distortion values ​​being summed. In other words, the distortion metric E1 i The filter distortion value E of all 8×8 blocks in the CTU at d = ±1 d，x，y Perform averaging. Further distortion measurement E2. i The filter distortion value E of all 8×8 blocks in the CTU at d = ±2 d，x，y Averaging is performed. Therefore, the distortion measure E1 i And further distortion measure E2 i The filter distortion values ​​at time / frame distance = ±1 and time / frame distance = ±2 were considered and averaged respectively. Distortion metric E1 i And further distortion measure E2 i The following distortion metric E can be formed by combining these components. combined_i :

[0082] E combined,i =max(E1) i E2 i )+3·abs(E2 i -E1 i (Equation 4)

[0083] In some cases, for a CTU located in the highest time layer of the filtered frame, the distortion metric E combined,i as follows:

[0084] E combined =0.6·E combined +0.4·C center (Equation 5)

[0085] C center It is a measure of combined distortion, E combined，i The midpoint of the range of values ​​(e.g., 45). Equation 5 is used to adjust the E values ​​of frames that are not in the base layer of the encoded GOP hierarchy. combined，i value.

[0086] Quantization parameters (e.g., ΔQP or QP offset) can be used with E combined，i Obtained through a lookup table. An example lookup table is shown below:

[0087] <![CDATA[E combined，i ]]> ΔQP 0-14 -2 15-29 -1 30-60 0 61-75 +1 >75 +2

[0088] Table 1: E in different ranges combined，i Example ΔQP lookup table associated with different ΔQP values

[0089] This non-ideal solution contains one or more defects. When E1 is calculated based on Equations 2 and 3... i and E2 i At that time, the averaging operation is performed using only the filtered distortion values ​​from neighboring compensated frames whose absolute time / frame distance values ​​are 1 and 2, respectively. In each direction (past or future; forward or backward), regardless of the number of reference frames available, the calculation of E1... i and E2 i During this period, information from only two reference frames is used for each 8×8 pixel block. Information from other reference frames where the absolute time / frame distance is greater than 2 is ignored and discarded, even if it is available. Therefore, based on E1... i and E2 i Determining the quantization parameters does not take into account all available information. Furthermore, the range of ΔQP in the lookup table of Table 1 is limited to [-2, +2].

[0090] To address at least one of the aforementioned drawbacks, the block QP adaptive system 444 includes one or more of the following aspects: (1) modified for E1 i and E2 i The calculation includes: (2) adaptive QP unit size (or dQP block size) selection; and (3) a modified lookup table for determining ΔQP. Including any one or more of these aspects can provide a block-level QP adaptive system that can better adapt to various video content and further improve coding efficiency.

[0091] Unlike Equations 2 and 3, which only consider information from reference frames at absolute time / frame distances of 1 and 2, in calculating E1... i and E2 i When calculating E1, all available information about the reference frames is considered, including information about reference frames with an absolute time / frame distance greater than 2. This information has already been calculated by MCTF 406 and does not incur additional overhead. Furthermore, the minimum distortion value is considered because, during encoding, these reference frames with the minimum distortion value at time / frame distance are most likely to be selected for motion prediction. The minimum distortion value can serve as a strong signal of block importance. Since the goal of the block-level QP adaptive scheme is to estimate the probability that a block will be used as a reference when encoding adjacent frames, this is considered when calculating E1. i and E2 i When considering only one or two adjacent frames in each direction, it is not optimal.

[0092] For simplicity, the following section focuses on calculating E1. i and E2 iThe description assumes that the QP cell size or dQP block size is the CTU size or macroblock size (e.g., the minimum block size or a base cell that may have a different ΔQP value or QP offset value than other base cells). In calculating E1... i and E2 i In one direction, for each 8×8 block, instead of using only the filtered distortion value E from the nearest reference frame (d = ±1, ±2), d,x,y Use one or more of the minimum filtered distortion values ​​from all reference frames in the same direction.

[0093] The block QP adaptive system 444 can perform processing on individual CTUs (e.g., CTU 502). For CTU 502, the block-level minimum distortion value of the block can be determined by a corresponding minimum distortion calculation, such as an 8×8 block minimum distortion value calculation 506. The 8×8 block minimum distortion value calculation 506 can determine one or more minimum distortion values ​​of the block based on one or more filtered distortion values ​​calculated for the block of a video frame (e.g., at different time / frame distances, such as d = ±1, ±2, ±3, and ±4).

[0094] In some embodiments, the minimum future calculation 510 may determine one or more forward minimum distortion values ​​for each 8×8 block, such as a forward minimum distortion value or minE_f,x,y and a further forward minimum distortion value or min′E_f,x,y. The minimum future calculation 510 may set the lowest filtered distortion value among one or more filtered distortion values ​​calculated for the block in the forward direction as the forward minimum distortion value or minE_f,x,y. The minimum future calculation 510 may also set a second lowest filtered distortion value among one or more filtered distortion values ​​calculated for the block in the forward direction as the further forward minimum distortion value or min′E_f,x,y. The minimum future calculation 510 may output the two lowest filtered distortion values ​​(forward minimum distortion value minE_f,x,y and further forward minimum distortion value min′E_f,x,y) among the filtered distortion values ​​calculated for the reference block in the forward direction (where d is positive), for example, a set of filtered distortion values: E +4，x，y E +3，x，y E +2,x,y and E +1,x,y .

[0095] In some embodiments, the least past calculation 508 may determine one or more backward minimum distortion values ​​for each 8×8 block, such as a backward minimum distortion value minE_p,x,y and a further backward minimum distortion value min′E_p,x,y. The least past calculation 508 may set the lowest filtered distortion value among one or more filtered distortion values ​​calculated for the block in the backward direction as either the backward minimum distortion value or minE_p,x,y. The least past calculation 508 may set the second lowest filtered distortion value among one or more filtered distortion values ​​calculated for the block in the backward direction as either the further backward minimum distortion value or min′E_p,x,y. The least past calculation 508 may output the two lowest filtered distortion values ​​(backward minimum distortion value minE_p,x,y and further backward minimum distortion value min′E_p,x,y) among the filtered distortion values ​​calculated for the reference block in the backward direction (where d is negative), for example, a set of filtered distortion values: E -4,x,y E -3,x,y E -2,x,y and E -1,x,y .

[0096] The minimum past calculation 508 can output minE_p,x,y and min′E_p,x,y for each 8×8 block in the backward direction. The minimum future calculation 510 can output minE_f,x,y and min′E_f,x,y for each 8×8 block in the forward direction. One or more minimum distortion values ​​(e.g., minE_f,x,y, min′E_f,x,y, minE_p,x,y, and min′E_p,x,y) generated by the minimum past calculation 508 and minimum future calculation 510 of the 8×8 block minimum distortion value calculation 506 can be used to determine the distortion metric E1. i And further distortion measure E2 i .

[0097] For CTU (or QP cell or dQP block), the distortion metric E1 i And further distortion measure E2 i The calculation can be performed as follows:

[0098]

[0099] N is the number of 8×8 blocks in the CTU (or QP unit or dQP block), or the number of minimum filtered distortion values ​​summed. In other words, the distortion metric E1 i The minimum filtered distortion value (e.g., min(minE_p,x,y,minE_f,x,y)) of all 8×8 blocks in the CTU is averaged in the forward or backward direction (e.g., d<0 or d>0). Further distortion metric E2 is then applied. iThe minimum filtered distortion value (e.g., min(min'E_p,x,y,min'E_f,x,y)) of all 8×8 blocks in the CTU is averaged in the forward or backward direction (e.g., d<0 or d>0). Therefore, the distortion metric E1 is... i And further distortion measure E2 i One or more minimum filter distortion values ​​were considered from the set of filter distortion values ​​for all time / frame distances (d = ±1, ±2, ±3, and ±4). Distortion metric E1 i And further distortion measure E2 i It can be combined according to Equation 4 or Equation 5 to form the combined distortion metric E. combined，i .

[0100] In some implementations, equations 6 and 7 can have alternative forms:

[0101]

[0102] N is the number of 8×8 blocks in the CTU (or QP unit or dQP block). 2N can be the number of minimum filtered distortion values ​​to be summed. In other words, the distortion metric E1 i The minimum filtered distortion values ​​(e.g., minE_p,x,y and minE_f,x,y) of all 8×8 blocks in the CTU are averaged in both the forward and backward directions (e.g., d<0 or d>0). Further distortion metric E2 is applied. i The minimum filtered distortion values ​​(e.g., min′E_p,x,y and min′E_f,x,y) of all 8×8 blocks in the CTU are averaged in both the forward and backward directions (e.g., d<0 or d>0). Therefore, the distortion metric E1... i And further distortion measure E2 i One or more minimum filter distortion values ​​were considered from the set of filter distortion values ​​for all distances (d = ±1, ±2, ±3, and ±4). Distortion metric E1 i And further distortion measure E2 i It can be combined according to Equation 4 or Equation 5 to form the combined distortion metric E. combined,i .

[0103] The calculations shown by equations 6 and 7 (or equations 8 and 9) can be performed by E1 calculation 522 and E2 calculation 524 respectively to generate the distortion metric E1. i And further distortion measure E2 i The calculations shown in Equations 4 and 5 can be performed by E. COMBINED 526 calculations were performed to generate the combined distortion metric E. combined，i .

[0104] In some embodiments, E1 calculation 522 can determine a distortion metric E1 for a QP cell that includes at least the block. i Distortion metric E1 i It can be based on a subset of one or more minimum distortion values. Distortion metric E1 i One or more minimum distortion values ​​determined for blocks across all blocks of a CTU (e.g., a QP cell, a dQP block) can be averaged. A subset of these one or more minimum distortion values ​​may include one or more of the forward minimum distortion value (minE_f, x, y) and the backward minimum distortion value (minE_p, x, y) determined for that block. In some embodiments, E1 calculates 522 to determine the minimum of a subset of one or more minimum distortion values ​​for that block (e.g., min(minE_p, x, y, minE_f, x, y)) and to determine the average of one or more minimum values ​​determined for one or more blocks of a CTU (or dQP block) (e.g., as shown in Equation 6). These one or more minimum values ​​include the minimum values ​​determined for that block. In some embodiments, E1 calculates 522 to determine the average of one or more subsets of one or more minimum distortion values ​​determined for one or more blocks of a quantization parameter cell (e.g., as shown in Equation 8). These one or more subsets include a subset of one or more minimum distortion values ​​for the block (e.g., minE_p, x, y and min′E_y, x, y).

[0105] In some embodiments, E1 calculates 522 by adding or summing the smaller of the forward minimum distortion value and the backward minimum distortion value (minimum distortion value in either direction) of each 8×8 block, and then averaging by dividing by N, where N is the number of blocks. In some embodiments, E1 calculates 522 by adding or summing the forward minimum distortion value and the backward minimum distortion value (minimum distortion value in both directions) of each 8×8 block, and then averaging by dividing by 2N, where N is the number of blocks.

[0106] In some embodiments, E2 calculation 524 can determine further distortion metrics E2 for QP cells including at least this block. i The further distortion metric E2 i It can be a further subset based on one or more minimum distortion values. Further distortion metric E2 iOne or more further minimum distortion values ​​determined for blocks across all blocks in a CTU (e.g., QP cell, dQP block) can be averaged. A further subset of the one or more minimum distortion values ​​may include one or more of the further forward minimum distortion values ​​min′E_f, x, y and the further backward minimum distortion values ​​min′E_p, x, y determined for that block. In some embodiments, E2 calculation 524 may determine the minimum of a further subset of the one or more minimum distortion values ​​for that block (e.g., min(min'E_p, x, y, min'E_f, x, y)) and determine the average of the one or more minimum values ​​determined for one or more blocks of the CTU (or dQP block) (e.g., as shown in Equation 7). The one or more minimum values ​​include the minimum values ​​determined for that block. In some embodiments, E2 calculation 524 may determine the average of one or more further subsets of the one or more minimum distortion values ​​determined for one or more blocks of the quantization parameter cell (e.g., as shown in Equation 9). The one or more further subsets include further subsets of one or more minimum distortion values ​​of the block (e.g., min′E_p,x,y and min′E_f,x,y).

[0107] In some embodiments, E2 calculation 524 adds or sums the smaller of the further forward minimum distortion value and the further backward minimum distortion value (the second minimum distortion value in either direction) of each 8×8 block and averages it by dividing by N, where N is the number of blocks. In some embodiments, E2 calculation 524 adds or sums the further forward minimum distortion value and the further backward minimum distortion value (the minimum distortion values ​​in both directions) of each 8×8 block and averages it by dividing by 2N, where N is the number of blocks.

[0108] In some embodiments, the distortion metric E1 can be used as a basis. i And further distortion measure E2 i Determine the QP value (or ΔQP or QP offset) to be applied to the CTU (or QP cell or dQP block).

[0109] E COMBINED Calculate 526 based on distortion metric E1 i And further distortion measure E2 i Generate the combined distortion metric E (based on equation 4 or equation 5). combined，i It can be based on the combined distortion metric E. combined，i Determine the QP value (or ΔQP or QP offset) applied to the CTU (or QP cell or dQP block). Combined distortion metric E combined，i It can be used as input to lookup table 530 to perform a lookup based on the combined distortion metric E. combined，i Determine the quantization parameters (e.g., dQP 532).

[0110] It is worth noting that equations 6, 7, 8, and 9 consider the minimum distortion value among the filtered distortion values ​​calculated for all reference frames (e.g., all d), not just the filtered distortion values ​​for the two nearest adjacent frames (e.g., d = ±1 and ±2). In many cases, the number of reference frames in any direction can be greater than 2, especially in frames in the base layer of a hierarchical GOP structure. In these cases, the number of available filtered distortion values ​​490 calculated by MCTF 406 is also greater than 2 in each direction. The filtered distortion values ​​490 for all reference frames have already been calculated in MCTF 406 and can be directly used in the block QP adaptive system 444 without any additional computation or overhead.

[0111] In one approach, the basic unit of adaptive QP is fixed at the CTU size across all frames of the video, which can be inflexible when it comes to adapting to content with different characteristics within the CTU. In some modern codecs (e.g., HEV and more), QP can be changed at the sub-CTU level or quantization group level, where the granularity at which QP modification occurs can be signaled by the encoder in the bitstream syntax (e.g., picture header). Leveraging this capability, the block QP adaptive system 444 can implement adaptive dQP block size selection to better adapt to diverse video content and help achieve better encoder performance.

[0112] Adaptive dQP block size selection 512 and CTU segmentation 514 can be implemented in the block QP adaptive system 444 to adjust the QP cell size (or dQP block size 516) and allow different QP (or ΔQP or QP offset) settings for different QP cells or dQP blocks of different sizes. Adaptive dQP block size selection 512 can use one or more filter distortion values ​​490 as input to determine the dQP block size 516. Adaptive dQP block size selection 512 can use one or more minimum distortion values ​​calculated by the 8×8 block minimum distortion value calculation 506 as input to determine the dQP block size 516.

[0113] The dQP block size 516 can specify the number of 8×8 blocks in a dQP block or QP cell, such as 64, 16, 4, and 1. The dQP block size 516 can specify the pixel size of a dQP block or QP cell, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels.

[0114] CTU partitioning 514 can partition a CTU (e.g., CTU 502) into one or more QP cells or dQP blocks (described as dQP blocks, including dQP block 520) based on dQP block size 516. CTU partitioning 514 can generate one or more QP cells or dQP blocks (described as dQP blocks, including dQP block 520) having dQP block size 516.

[0115] Because CTU is divided into 514, E1 calculates 522, E2 calculates 524, and E... COMBINED Calculation 526 can operate at the dQP block level instead of the CTU (or macroblock) level (e.g., perform a calculation on dQP block 520 instead of CTU 502). E1 calculation 522 can average multiple blocks in a dQP block instead of averaging multiple blocks in a CTU (or macroblock). E2 calculation 524 can average multiple blocks in a dQP block instead of averaging multiple blocks in a CTU (or macroblock).

[0116] Figure 6 An adaptive dQP block size selection 512 is illustrated according to some embodiments of the present disclosure. The adaptive dQP block size selection 512 can determine a dQP block size 516 for an individual CTU (e.g., CTU 502). For a given CTU, a block-level average minimum distortion value (minE, x, y) for each individual block can be determined by a corresponding 8×8 averaging calculation (such as 8×8 averaging calculation 602). In some embodiments, 8×8 averaging calculation 602 can determine the average minimum distortion metric of the block based on one or more minimum distortion values, such as a subset of one or more minimum distortion values ​​of the block (e.g., minE_p, x, y and minE_f, x, y). In some embodiments, the 8×8 averaging calculation 602 may determine the average value of the block (e.g., the average minimum distortion metric minE,x,y) based on one or more minimum distortion values, such as including a subset of one or more minimum distortion values ​​(e.g., minE_p,x,y and minE_f,x,y) and a further subset of one or more minimum distortion values ​​(e.g., min′E_p,x,y and min′E_f,x,y).

[0117] Variance 604 can be determined as the variance of one or more averages (e.g., one or more average minimum distortion metrics) determined for one or more blocks of a CTU or macroblock. These one or more averages include the averages determined for that block. This variance is fed as input to dQP block size decision 606, which determines a dQP block size 516 suitable for that CTU or macroblock. The variances of all CTUs or macroblocks of a frame can be fed as input to dQP block size decision 606, which determines a dQP block size 516 suitable for that frame.

[0118] The dQP block size decision 606 can determine the size of a QP unit (e.g., dQP block size 516) based on the variance of a frame's CTU / macroblock, or the variance of one or more CTUs / macroblocks in a frame. The dQP block size decision 606 can apply logic or one or more rules to the variance of one or more CTUs / macroblocks or one or more variances to determine the dQP block size 516. Exemplary logic or rules may include setting the dQP block size 516 to a specific value if a certain proportion of the variance exceeds a variance threshold. The dQP block size decision 606 can compare the variance or one or more variances to one or more thresholds corresponding to one or more different sizes of QP units or different values ​​of the dQP block size 516, and determine the value of the dQP block size 516 accordingly.

[0119] Adaptive dQP block size selection 512 can determine the dQP block size 516 by considering all available filtered distortion values ​​of all reference frames and by utilizing the minimum distortion value among the available filtered distortion values ​​of all reference frames. For example, adaptive dQP block size selection 512 can use the block-level forward minimum distortion value minE_p,x,y and the block-level backward minimum distortion value minE_f,x,y. The minimum distortion values ​​in both directions are considered and averaged in an 8×8 averaging calculation 602 to obtain a block-level average minimum distortion metric minE,x,y. The two-dimensional variance is calculated by variance 604 over the average minimum distortion metric minE,x,y calculated for one or more blocks of CTU 502, thus obtaining the variance of CTU 502.

[0120] Higher variance can mean that the dQP block size 516 can be set smaller through adaptive dQP block size selection 512 to account for different characteristics in the CTU or frame. A smaller dQP block size 516 allows QP to change for smaller dQP blocks, thus enabling the block QP adaptive system 444 to better adapt to different content. Lower variance can mean that the dQP block size 516 can be set larger through adaptive dQP block size selection 512 to account for consistency characteristics in the CTU or frame. A larger dQP block size 516 allows QP to change for larger dQP blocks, thus preventing unnecessary QP changes when quantizing and smoothing content.

[0121] The block QP adaptive system 444 (including adaptive dQP block size selection 512 and CTU segmentation 514) has the ability to switch QPs at the CTU or sub-CTU level (or macroblock or sub-macroblock level) based on the video content’s spatial and temporal characteristics, such as for a given CTU or macroblock, or for a given frame.

[0122] Reference Figure 5In some cases, MCTF 406 can be disabled, not supported, or not applied to the generation of filtered video frames. The block QP adaptive system 444 can perform calculations according to Equation 1 to obtain the individual block's filter distortion value 490. In schemes using Table 1 to set quantization parameters, based on E... COMBINED The determined maximum QP adjustment (e.g., ΔQP range) is ±2. Furthermore, Table 1 is independent of whether MCTF 406 is on or off (e.g., Table 1 is used as a lookup table regardless of whether MCTF 406 is on or off). This method may have two potential problems. For video content with static regions or regions as references in many adjacent frames, the ΔQP range is too small to significantly improve quality; a typical example is screen content with some text static in many adjacent frames. After applying filtering through MCTF 406, the video content changes, especially when the video has a certain level of noise. Applying the same ΔQP to frames using and not using MCTF filters (e.g., MCTF 406 is on or off) is not optimal for encoder performance.

[0123] To address at least one of the potential problems, the lookup table 530 to be applied can be selected or determined based on whether the MCTF 560 is on (i.e., a signal indicating whether the MCTF 406 is on or off). Furthermore, the lookup table 530 can have four or more rows to store four or more different ranges of E. combined,i Associated with different ΔQP values. Below are two illustrative lookup tables:

[0124]

[0125]

[0126] Table 2: Exemplary ΔQP lookup table to be used when MCTF is enabled, which will handle different ranges of E. combined,i Associated with different ΔQP values.

[0127] <![CDATA[E combined，i ]]> ΔQP 0-2 -4 3-7 -3 8-19 -2 20-49 -1 50-99 0 80-199 +1 200-499 +2 500-999 +3 >999 +4

[0128] Table 3: Exemplary ΔQP lookup table to be used when MCTF is closed, which will handle different ranges of E. combined，i Associated with different ΔQP values.

[0129] In contrast, applying the lookup table shown in Table 1 will... combined，i The ΔQP value is obtained by comparing it with four thresholds. Therefore, the lookup table has five rows to represent the five different ranges of E. combined，i Associated with different ΔQP values ​​(where ΔQP ranges from [-2, +2]). Apply the lookup tables shown in Tables 2 and 3 to E combined，iThe ΔQP value is obtained by comparing it with eight thresholds. Therefore, the lookup table has nine rows to represent nine different ranges of E. combined，i It is associated with different ΔQP values ​​(where ΔQP ranges from [-4, +4]).

[0130] In some embodiments, applying lookup table 530 to determine dQP 532 may include combining the distortion metric E combined，i The dQP 532 is set by comparing it with four or more thresholds corresponding to different values ​​of dQP 532. In some embodiments, applying a lookup table 530 to determine dQP 532 may include combining the distortion metric E. combined,i The dQP 532 is set based on comparisons with more than four thresholds corresponding to different values ​​of dQP 532.

[0131] The MCTF enable signal 560 can cause different lookup tables (e.g., different threshold sets) to be applied as lookup table 530. When determining the value for dQP 532, the MCTF enable signal 560 can select one of the available lookup tables (such as one of Tables 2 and 3) as lookup table 530. Table 2 reflects a more aggressive QP adjustment, while Table 3 reflects a more conservative QP adjustment. Applying different QP adjustments based on whether MCTF is enabled or disabled can allow the block QP adaptive system 444 to perform better. The application of MCTF changes what is being encoded, and how QP is adjusted can vary depending on whether MCTF is applied (e.g., how QP is adjusted in response to upstream MCTF processing). Although the combined distortion metric E... combined,i The calculations are identical and based on the original video, but the QP adjustment according to dQP 532 occurs within the encoder after MCTF 406 and the block QP adaptive system 444. The same combined distortion metric E is used. combined，i With MCTF enabled, the correlation between frames is higher than when MCTF is disabled because the current frame is filtered along with adjacent frames, and the importance of the same block is also higher. With MCTF enabled, more aggressive ΔQP adjustment can be performed. More aggressive ΔQP adjustment means using a higher absolute value of dQP 532 to improve the combined distortion metric E. combined，i The lower end range is larger and makes the combined distortion metric E combined，i The high-end range is also larger. Less aggressive ΔQP adjustments can be made when MCTF is off. Less aggressive ΔQP adjustments mean using a lower absolute value of dQP 532 to reduce the combined distortion metric E. combined,i The lower end range is smaller and makes the combined distortion metric E combined,i The high-end range is also smaller. In the distortion metric Ecombined,i When the values ​​are very low or very high, use the absolute value of the highest dQP 532.

[0132] Figure 7 This is a flowchart illustrating a method 700 for determining quantization parameters to be applied to a quantization parameter unit according to some embodiments of the present disclosure. Method 700 can be... Figure 4 The block QP adaptive system 444 in the middle is executed.

[0133] In 702, one or more minimum distortion values ​​are determined based on one or more filtered distortion values ​​calculated for a block of a video frame.

[0134] In 704, a distortion metric is determined for the quantization parameter cells that include at least the block. This distortion metric is based on a subset of one or more minimum distortion values ​​for the block.

[0135] In 706, a further distortion metric is determined for the quantization parameter cell. This further distortion metric is a further subset based on one or more minimum distortion values ​​of the block.

[0136] In 708, the quantization parameters to be applied to the quantization parameter cell are determined based on the distortion metric and further distortion metric.

[0137] In some embodiments, one or more minimum distortion values ​​include one or more of a forward minimum distortion value, a further forward minimum distortion value, a backward minimum distortion value, and a further backward minimum distortion value.

[0138] In some embodiments, a subset of one or more minimum distortion values ​​includes one or more of forward minimum distortion values ​​and backward minimum distortion values.

[0139] In some embodiments, a further subset of one or more minimum distortion values ​​includes one or more of a further forward minimum distortion value and a further backward minimum distortion value.

[0140] Exemplary computing device

[0141] Figure 8 This is a block diagram of an apparatus or system (e.g., an exemplary computing device 800) according to some embodiments of the present disclosure. One or more computing devices 800 may be used to implement the functions described herein. Figure 8The various components shown may be included in computing device 800, but any one or more of these components may be omitted or repeated to suit the application. In some embodiments, some or all of the components included in computing device 800 may be attached to one or more motherboards. In some embodiments, some or all of these components are fabricated on a single system-on-a-chip (SoC) die. Furthermore, in various embodiments, computing device 800 may not include... Figure 8 The computing device 800 may include one or more of the components shown, but may include interface circuitry for coupling to said one or more components. For example, the computing device 800 may not include display device 806, but may include display device interface circuitry (e.g., connectors and driver circuitry) to which display device 806 may be coupled. In another set of examples, the computing device 800 may not include audio input device 818 or audio output device 808, but may include audio input or output device interface circuitry (e.g., connectors and support circuitry) to which audio input device 818 or audio output device 808 may be coupled.

[0142] Computing device 800 may include processing device 802 (e.g., one or more processing devices, one or more processing devices of the same type, or one or more processing devices of different types). Processing device 802 may include processing circuitry or electronic circuitry that processes electronic data from data storage elements (e.g., registers, memories, resistors, capacitors, qubit units) to convert that electronic data into other electronic data that can be stored in registers and / or memories. Examples of processing device 802 may include a central processing unit (CPU), graphics processing unit (GPU), quantum processor, machine learning processor, artificial intelligence processor, neural network processor, artificial intelligence accelerator, application-specific integrated circuit (ASIC), analog signal processor, analog computer, microprocessor, digital signal processor, field-programmable gate array (FPGA), tensor processing unit (TPU), data processing unit (DPU), etc.

[0143] Computing device 800 may include memory 804, which itself may include one or more storage devices, such as volatile memory (e.g., DRAM), non-volatile memory (e.g., read-only memory (ROM)), high-bandwidth memory (HBM), flash memory, solid-state memory, and / or hard disk drives. Memory 804 includes one or more non-transitory computer-readable storage media. In some embodiments, memory 804 may include memory that shares a die with processing device 802.

[0144] In some embodiments, memory 804 includes one or more non-transitory computer-readable media storing executable instructions to perform the operations described herein, such as Figure 1-6 The operations and methods 700 are illustrated. In some embodiments, memory 804 includes one or more non-transitory computer-readable media storing executable instructions to perform one or more operations of encoder 102. In some embodiments, memory 804 includes one or more non-transitory computer-readable media storing executable instructions to perform one or more operations of preprocessing 290. In some embodiments, memory 804 includes one or more non-transitory computer-readable media storing executable instructions to perform one or more operations of block QP adaptive system 444. The instructions stored in memory 804 can be executed by processing device 802.

[0145] In some embodiments, memory 804 may store data as described in the accompanying drawings and herein, such as data structures, binary data, bits, metadata, files, large objects (blobs), etc. Memory 804 may include one or more non-transitory computer-readable media storing one or more of the following: input frames of the encoder (e.g., video frame 104), intermediate data structures computed by the encoder, bitstreams generated by the encoder (encoded bitstream 180), bitstreams received by the decoder (encoded bitstream 180), intermediate data structures computed by the decoder, and reconstructed frames generated by the decoder. Memory 804 may include one or more non-transitory computer-readable media storing one or more of the data received and / or generated by preprocessing 290. Memory 804 may include one or more non-transitory computer-readable media storing one or more of the data received and / or generated by the block QP adaptive system 444. Memory 804 may include one or more non-transitory computer-readable media storing data as described in the accompanying drawings and herein. Figure 7 Method 700 refers to one or more of the data received and / or generated.

[0146] In some embodiments, computing device 800 may include communication device 812 (e.g., one or more communication devices). For example, communication device 812 may be configured to manage wired and / or wireless communications to transmit data to or from computing device 800. The term “wireless” and its derivatives can be used to describe circuits, devices, systems, methods, technologies, communication channels, etc., which can transmit data in a non-solid medium by using modulated electromagnetic radiation. This term does not imply that the associated devices do not contain any wires, although they may be wire-free in some embodiments. Communication device 812 may implement any of a variety of wireless standards or protocols, including but not limited to Institute of Electrical and Electronics Engineers (IEEE) standards, including Wi-Fi (IEEE 802.10 series), IEEE 802.16 standards (e.g., IEEE 802.16-2005 revision), Long Term Evolution (LTE) projects, and any revisions, updates, and / or modifications (e.g., Advanced LTE projects, Ultra Mobile Broadband (UMB) projects (also known as “3GPP2”), etc.). Broadband wireless access (BWA) networks compliant with the IEEE 802.16 standard are commonly referred to as WiMAX networks. WiMAX is an abbreviation for Global Microwave Access Interoperability, a certification mark for products that have passed conformance and interoperability testing of the IEEE 802.16 standard. Communication device 812 can operate according to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE networks. Communication device 812 can operate according to Enhanced GSM Evolved Data (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). Communication device 812 can operate according to Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Communication (DECT), Evolved Data Optimized (EV-DO) and its derivatives, as well as any other wireless protocol designated as 3G, 4G, 5G, and higher versions. In other embodiments, communication device 812 can operate according to other wireless protocols. Computing device 800 may include antenna 822 to facilitate wireless communication and / or receiving other wireless communications (such as radio frequency transmissions). Computing device 800 may include receiver circuitry and / or transmitter circuitry. In some embodiments, communication device 812 may manage wired communication, such as electrical communication, optical communication, or any other suitable communication protocol (e.g., Ethernet). As described above, communication device 812 may include multiple communication chips.For example, the first communication device 812 may be dedicated to short-range wireless communication, such as Wi-Fi or Bluetooth, while the second communication device 812 may be dedicated to long-range wireless communication, such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, etc. In some embodiments, the first communication device 812 may be dedicated to wireless communication, and the second communication device 812 may be dedicated to wired communication.

[0147] The computing device 800 may include a power supply / power circuit 814. The power supply / power circuit 814 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of the computing device 800 to a power source (e.g., DC power, AC power, etc.) that is separate from the computing device 800.

[0148] The computing device 800 may include a display device 806 (or a corresponding interface circuit as described above). The display device 806 may include any visual indicator, such as, for example, a head-up display, a computer monitor, a projector, a touch screen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat panel display.

[0149] The computing device 800 may include an audio output device 808 (or a corresponding interface circuit as described above). The audio output device 808 may include any device that generates sound indications, such as, for example, a speaker, headphones, or earphones.

[0150] The computing device 800 may include an audio input device 818 (or a corresponding interface circuit as described above). The audio input device 818 may include any device that generates a signal representing sound, such as a microphone, microphone array, or digital musical instrument (e.g., a musical instrument with a Musical Instrument Digital Interface (MIDI) output).

[0151] The computing device 800 may include a GPS device 816 (or a corresponding interface circuit as described above). The GPS device 816 can communicate with a satellite-based system and can receive the location of the computing device 800 as known in the art.

[0152] The computing device 800 may include a sensor 830 (or one or more sensors). The computing device 800 may include corresponding interface circuitry (as described above). The sensor 830 can sense physical phenomena and convert them into electrical signals that can be processed by, for example, the processing device 802. Examples of sensors 830 may include: capacitive sensors, inductive sensors, resistive sensors, electromagnetic field sensors, light sensors, cameras, imagers, microphones, pressure sensors, temperature sensors, vibration sensors, accelerometers, gyroscopes, strain sensors, moisture sensors, humidity sensors, distance sensors, ranging sensors, time-of-flight sensors, pH sensors, particulate matter sensors, air quality sensors, chemical sensors, gas sensors, biosensors, ultrasonic sensors, scanners, etc.

[0153] The computing device 800 may include another output device 810 (or a corresponding interface circuit as described above). Examples of other output devices 810 may include audio codecs, video codecs, printers, wired or wireless transmitters for providing information to other devices, haptic output devices, gas output devices, vibration output devices, light output devices, home automation controllers, or additional storage devices.

[0154] The computing device 800 may include another input device 820 (or a corresponding interface circuit as described above). Examples of other input devices 820 may include an accelerometer, a gyroscope, a compass, an image capture device, a keyboard, a cursor control device (such as a mouse), a stylus, a touchpad, a barcode reader, a quick-response (QR) code reader, any sensor, or a radio frequency identification (RFID) reader.

[0155] The computing device 800 can have any desired form factor, such as a handheld or mobile computer system (e.g., mobile phone, smartphone, mobile internet device, music player, tablet, laptop, netbook, personal digital assistant (PDA), ultra-mobile personal computer, remote control, wearable device, head-mounted device, glasses, footwear, electronic clothing, etc.), desktop computer system, server or other networked computing component, printer, scanner, monitor, set-top box, entertainment control unit, vehicle control unit, digital camera, digital video recorder, Internet of Things device, or wearable computer system. In some embodiments, the computing device 800 can be any other electronic device that processes data.

[0156] Select Example

[0157] Example 1 provides a method comprising: determining one or more minimum distortion values ​​based on one or more filtered distortion values ​​computed for a block of a video frame; determining a distortion metric for a quantization parameter unit comprising at least the block, the distortion metric being based on a subset of one or more minimum distortion values ​​of the block; determining a further distortion metric for the quantization parameter unit, the further distortion metric being based on a further subset of one or more minimum distortion values ​​of the block; and determining quantization parameters to be applied to the quantization parameter unit based on the distortion metric and the further distortion metric.

[0158] Example 2 provides the method of Example 1, wherein one or more minimum distortion values ​​include one or more of the following: forward minimum distortion value, further forward minimum distortion value, backward minimum distortion value, and further backward minimum distortion value.

[0159] Example 3 provides the method of Example 2, wherein one or more subsets of minimum distortion values ​​include one or more of the forward minimum distortion value and the backward minimum distortion value.

[0160] Example 4 provides a method from Example 2 or 3, wherein a further subset of one or more minimum distortion values ​​includes one or more of a further forward minimum distortion value and a further backward minimum distortion value.

[0161] Example 5 provides a method for any of the examples in Examples 2-4, wherein: the forward minimum distortion value is the lowest of one or more filtered distortion values ​​calculated for the block in the forward direction; and the further forward minimum distortion value is the second lowest of one or more filtered distortion values ​​calculated for the block in the forward direction.

[0162] Example 6 provides a method for any of the examples in Examples 2-5, wherein: the backward minimum distortion value is the lowest of one or more filtered distortion values ​​calculated for the block in the backward direction; and the further backward minimum distortion value is the second lowest of one or more filtered distortion values ​​calculated for the block in the backward direction.

[0163] Example 7 provides a method from any of Examples 1-6, further including determining one or more filter distortion values ​​for the block based on the sum of squared differences between the block and the motion compensation block, and the variance of the block.

[0164] Example 8 provides a method for any of the examples in Examples 1-7, where one or more filtered distortion values ​​are calculated for the block using a motion-compensated time filter.

[0165] Example 9 provides a method for any of the examples in Examples 1-8, wherein determining a distortion metric for a quantization parameter unit includes determining a minimum value in a subset of one or more minimum distortion values ​​for that block; and determining an average of one or more minimum values ​​determined for one or more blocks of the quantization parameter unit, the one or more minimum values ​​including the minimum value determined for that block.

[0166] Example 10 provides a method for any of the examples in Examples 1-9, wherein determining a further distortion metric for a quantization parameter unit includes determining a further minimum of a further subset of one or more minimum distortion values; and determining a further average of one or more further minimums determined for one or more blocks of the quantization parameter unit, the one or more further minimums including a further minimum determined for that block.

[0167] Example 11 provides a method for any of the examples in Examples 1-8, wherein determining a distortion metric for a quantization parameter cell includes determining the average of one or more subsets of one or more minimum distortion values ​​determined for one or more blocks of the quantization parameter cell, the one or more subsets comprising a subset of one or more minimum distortion values ​​of the block.

[0168] Example 12 provides a method for any of the examples in Examples 1-9, wherein determining a further distortion metric for a quantization parameter cell includes determining the average of one or more further subsets of one or more minimum distortion values ​​determined for one or more blocks of the quantization parameter cell, the one or more further subsets comprising further subsets of one or more minimum distortion values ​​of the block.

[0169] Example 13 provides a method for any of the examples in Examples 1-12, further including determining a combined distortion metric for the quantization parameter unit based on a distortion metric and a further distortion metric; wherein the quantization parameter is determined based on the combined distortion metric.

[0170] Example 14 provides a method for any of the examples in Examples 1-13, further comprising determining an average value based on at least a subset of one or more minimum distortion values; determining the variance of one or more average values ​​determined for one or more blocks of a macroblock, the one or more average values ​​including the average value determined for that block; and determining the size of a quantization parameter cell based on the variance.

[0171] Example 15 provides the method of Example 14, wherein determining the size of a quantization parameter cell includes comparing the variance with one or more thresholds corresponding to one or more sizes of the quantization parameter cell.

[0172] Example 16 provides a method for any of the examples in Examples 1-15, wherein determining the quantization parameter includes determining a combined distortion metric for the quantization parameter unit based on a distortion metric and a further distortion metric; and comparing the combined distortion metric with more than four thresholds.

[0173] Example 17 provides the method of Example 16, further including determining the more than four thresholds based on whether motion-compensated time filtering is on or off.

[0174] Example 18 provides a method for any of the examples in Examples 1-15, wherein determining the quantization parameters includes determining a combined distortion metric for the quantization parameter unit based on a distortion metric and a further distortion metric; and comparing the combined distortion metric with four or more thresholds.

[0175] Example 19 provides the method of Example 18, further including determining the four or more thresholds based on whether motion-compensated time filtering is on or off.

[0176] Example 20 provides one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to: determine one or more minimum distortion values ​​based on one or more filtered distortion values ​​calculated for a block of a video frame; determine a distortion metric for a quantization parameter unit including at least the block, the distortion metric being based on a subset of the one or more minimum distortion values ​​of the block; determine a further distortion metric for the quantization parameter unit, the further distortion metric being based on a further subset of the one or more minimum distortion values ​​of the block; and determine quantization parameters to be applied to the quantization parameter unit based on the distortion metric and the further distortion metric.

[0177] Example 21 provides one or more non-transitory computer-readable media of Example 20, wherein one or more minimum distortion values ​​include one or more of a forward minimum distortion value, a further forward minimum distortion value, a backward minimum distortion value, and a further backward minimum distortion value.

[0178] Example 22 provides one or more non-transitory computer-readable media of Example 21, wherein one or more subsets of minimum distortion values ​​include one or more of forward minimum distortion values ​​and backward minimum distortion values.

[0179] Example 23 provides one or more non-transitory computer-readable media of Example 21 or 22, wherein a further subset of one or more minimum distortion values ​​includes one or more of a further forward minimum distortion value and a further backward minimum distortion value.

[0180] Example 24 provides one or more non-transitory computer-readable media of any of Examples 21-23, wherein: the forward minimum distortion value is the lowest of one or more filtered distortion values ​​calculated for the block in the forward direction; and the further forward minimum distortion value is a second lowest of one or more filtered distortion values ​​calculated for the block in the forward direction.

[0181] Example 25 provides one or more non-transitory computer-readable media for any of the examples 21-23, wherein: the backward minimum distortion value is the lowest of one or more filtered distortion values ​​calculated for the block in the backward direction; and the further backward minimum distortion value is a second lowest of one or more filtered distortion values ​​calculated for the block in the backward direction.

[0182] Example 26 provides one or more non-transitory computer-readable media of any of the examples 20-25, wherein the instructions further cause one or more processors to: determine one or more filter distortion values ​​for the block based on the sum of squared differences between the block and the motion compensation block and the variance of the block.

[0183] Example 27 provides one or more non-transitory computer-readable media of any of the examples 20-26, wherein one or more filtered distortion values ​​are calculated for the block by a motion-compensated time filter.

[0184] Example 28 provides one or more non-transitory computer-readable media for any of the examples 20-27, wherein determining a distortion metric for a quantization parameter unit includes determining a minimum value in a subset of one or more minimum distortion values ​​for that block; and determining an average of one or more minimum values ​​determined for one or more blocks of the quantization parameter unit, the one or more minimum values ​​including the minimum value determined for that block.

[0185] Example 29 provides one or more non-transitory computer-readable media for any of the examples 20-28, wherein determining a further distortion metric for a quantization parameter unit includes determining a further minimum of a further subset of one or more minimum distortion values; and determining a further average of one or more further minimums determined for one or more blocks of the quantization parameter unit, the one or more further minimums including a further minimum determined for that block.

[0186] Example 30 provides one or more non-transitory computer-readable media for any of the examples 20-27, wherein determining a distortion metric for a quantization parameter cell includes determining the average of one or more subsets of one or more minimum distortion values ​​determined for one or more blocks of the quantization parameter cell, the one or more subsets including a subset of one or more minimum distortion values ​​of the block.

[0187] Example 31 provides one or more non-transitory computer-readable media of any of the examples 20-28, wherein determining a further distortion metric for a quantization parameter cell includes determining the average of one or more further subsets of one or more minimum distortion values ​​determined for one or more blocks of the quantization parameter cell, the one or more further subsets including further subsets of one or more minimum distortion values ​​of the block.

[0188] Example 32 provides one or more non-transitory computer-readable media for any of the examples in Examples 20-31, wherein the instructions further cause one or more processors to: determine a combined distortion metric for the quantization parameter unit based on a distortion metric and a further distortion metric; wherein the quantization parameter is determined based on the combined distortion metric.

[0189] Example 33 provides one or more non-transitory computer-readable media for any of the examples in Examples 20-32, wherein the instructions further cause one or more processors to: determine an average value based on at least a subset of one or more minimum distortion values; determine the variance of one or more block-determined average values ​​for one or more macroblocks, the one or more average values ​​including the average value determined for the block; and determine the size of a quantization parameter cell based on the variance.

[0190] Example 34 provides one or more non-transitory computer-readable media of Example 33, wherein determining the size of a quantization parameter cell includes comparing the variance with one or more thresholds corresponding to one or more sizes of the quantization parameter cell.

[0191] Example 35 provides one or more non-transitory computer-readable media for any of the examples 20-34, wherein determining quantization parameters includes determining a combined distortion metric for the quantization parameter unit based on a distortion metric and a further distortion metric; and comparing the combined distortion metric with more than four thresholds.

[0192] Example 36 provides one or more non-transitory computer-readable media of Example 35, wherein the instructions further enable one or more processors to determine more than four thresholds based on whether motion-compensated time filtering is on or off.

[0193] Example 37 provides one or more non-transitory computer-readable media for any of the examples 20-34, wherein determining quantization parameters includes determining a combined distortion metric for a quantization parameter unit based on a distortion metric and further distortion metrics; and comparing the combined distortion metric with four or more thresholds.

[0194] Example 38 provides one or more non-transitory computer-readable media of Example 37, wherein the instructions further cause one or more processors to determine the four or more thresholds based on whether motion-compensated time filtering is on or off.

[0195] Example 39 provides an apparatus including one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to: determine one or more minimum distortion values ​​based on one or more filter distortion values ​​calculated for a block of a video frame; determine a distortion metric for a quantization parameter unit including at least the block, the distortion metric being based on a subset of the one or more minimum distortion values ​​of the block; determine a further distortion metric for the quantization parameter unit, the further distortion metric being based on a further subset of the one or more minimum distortion values ​​of the block; and determine quantization parameters to be applied to the quantization parameter unit based on the distortion metric and the further distortion metric.

[0196] Example 40 provides the apparatus of Example 39, wherein one or more minimum distortion values ​​include one or more of a forward minimum distortion value, a further forward minimum distortion value, a backward minimum distortion value, and a further backward minimum distortion value.

[0197] Example 41 provides the apparatus of Example 40, wherein a subset of one or more minimum distortion values ​​includes one or more of a forward minimum distortion value and a backward minimum distortion value.

[0198] Example 42 provides an apparatus of Example 40 or 41, wherein a further subset of one or more minimum distortion values ​​includes one or more of a further forward minimum distortion value and a further backward minimum distortion value.

[0199] Example 43 provides an apparatus for any of the examples 40-42, wherein: the forward minimum distortion value is the lowest of one or more filtered distortion values ​​calculated for the block in the forward direction; and the further forward minimum distortion value is a second lowest of one or more filtered distortion values ​​calculated for the block in the forward direction.

[0200] Example 44 provides an apparatus for any of the examples 40-43, wherein: the backward minimum distortion value is the lowest of one or more filtered distortion values ​​calculated for the block in the backward direction; and the further backward minimum distortion value is a second lowest of one or more filtered distortion values ​​calculated for the block in the backward direction.

[0201] Example 45 provides an apparatus for any of the examples in Examples 39-44, wherein the instructions further cause one or more processors to determine one or more filter distortion values ​​for the block based on the sum of the squared differences between the block and the motion compensation block and the variance of the block.

[0202] Example 46 provides an apparatus for any of the examples 39-45, wherein one or more filtered distortion values ​​are calculated for the block using a motion-compensated time filter.

[0203] Example 47 provides an apparatus for any of the examples 39-46, wherein determining a distortion metric for a quantization parameter unit includes determining a minimum value in a subset of one or more minimum distortion values ​​for that block; and determining an average of one or more minimum values ​​determined for one or more blocks of the quantization parameter unit, the one or more minimum values ​​including the minimum value determined for that block.

[0204] Example 48 provides an apparatus for any of the examples in Examples 39-47, wherein determining a further distortion metric for a quantization parameter unit includes determining a further minimum of a further subset of one or more minimum distortion values; and determining a further average of one or more further minimums determined for one or more blocks of the quantization parameter unit, the one or more further minimums including a further minimum determined for that block.

[0205] Example 49 provides an apparatus for any of the examples in Examples 39-46, wherein determining a distortion metric for a quantization parameter cell includes determining the average of one or more subsets of one or more minimum distortion values ​​determined for one or more blocks of the quantization parameter cell, the one or more subsets including a subset of one or more minimum distortion values ​​of the block.

[0206] Example 50 provides an apparatus for any of the examples in Examples 39-47, wherein determining a further distortion metric for a quantization parameter cell includes determining the average of one or more further subsets of one or more minimum distortion values ​​determined for one or more blocks of the quantization parameter cell, the one or more further subsets including further subsets of one or more minimum distortion values ​​of the block.

[0207] Example 51 provides an apparatus for any of the examples in Examples 39-50, wherein the instructions further cause one or more processors to: determine a combined distortion metric for the quantization parameter unit based on a distortion metric and a further distortion metric; wherein the quantization parameter is determined based on the combined distortion metric.

[0208] Example 52 provides an apparatus for any of the examples in Examples 39-51, wherein the instructions further cause one or more processors to: determine an average value based on at least a subset of one or more minimum distortion values; determine the variance of one or more block-determined average values ​​for one or more macroblocks, the one or more average values ​​including the average value determined for the block; and determine the size of a quantization parameter cell based on the variance.

[0209] Example 53 provides the apparatus of Example 52, wherein determining the size of a quantization parameter cell includes comparing the variance with one or more thresholds corresponding to one or more sizes of the quantization parameter cell.

[0210] Example 54 provides an apparatus for any of the examples 39-53, wherein determining quantization parameters includes determining a combined distortion metric for the quantization parameter unit based on a distortion metric and a further distortion metric; and comparing the combined distortion metric with more than four thresholds.

[0211] Example 55 provides the apparatus of Example 54, wherein the instructions further enable one or more processors to determine more than four thresholds based on whether motion compensation time filtering is on or off.

[0212] Example 56 provides an apparatus for any of the examples 39-53, wherein determining quantization parameters includes determining a combined distortion metric for a quantization parameter unit based on a distortion metric and a further distortion metric; and comparing the combined distortion metric with four or more thresholds.

[0213] Example 57 provides the apparatus of Example 56, wherein the instructions further cause one or more processors to determine the four or more thresholds based on whether motion compensation time filtering is on or off.

[0214] Example A provides a computer program product that includes instructions that, when executed by a processor, cause the processor to perform the method of any of the examples 1-19.

[0215] Example B provides an apparatus that includes means for performing the methods of any of the examples in Examples 1-19.

[0216] Example C provides a block QP adaptive system as described and illustrated herein.

[0217] Example D provides a preprocessing method as described and illustrated herein.

[0218] Example E provides a system with an encoder and preprocessing as described and illustrated herein.

[0219] Example F provides an apparatus that includes computational circuitry for performing the methods of any of the examples in Examples 1-19.

[0220] Variations and other notes

[0221] Although reference Figure 1-7 The operations of the example methods shown and described are illustrated as being executed once each time in a specific order; however, it should be recognized that some operations can be executed in any suitable order and repeated as needed. Furthermore, one or more operations can be executed in parallel. Figure 1-7 The operations shown can be combined, or may include more or less detail than described.

[0222] The foregoing description of the illustrated embodiments of this disclosure, including the content described in the abstract, is not intended to be exhaustive or to limit this disclosure to the precise forms disclosed. While specific embodiments and examples of this disclosure have been described herein for ease of explanation, those skilled in the art will understand that various equivalent modifications can be made within the scope of this disclosure. These modifications can be made to this disclosure based on the above detailed description.

[0223] For ease of explanation, specific figures, materials, and configurations are listed to provide a thorough understanding of the exemplary embodiments. However, those skilled in the art will understand that this disclosure can be implemented without specific details, and / or may be implemented using only some of the described aspects. In other instances, well-known features have been omitted or simplified to avoid obscuring the exemplary embodiments.

[0224] Furthermore, reference has been made to the accompanying drawings, which form part of this document, in which illustrative embodiments that may be implemented are shown. It should be understood that other embodiments may be employed, and structural or logical changes may be made without departing from the scope of this disclosure. Therefore, the following detailed description should not be considered limiting.

[0225] Various operations can be described as a series of discrete actions or operations performed sequentially in a manner most conducive to understanding the disclosed subject matter. However, the order in which they are described should not be construed as implying that these operations are necessarily sequential. Specifically, these operations may not be performed in the order presented. The described operations may be performed in a different order than the described embodiments. In other embodiments, various other operations may be performed, or the described operations may be omitted.

[0226] For the purposes of this disclosure, the phrase "A or B" or the phrase "A and / or B" refers to (A), (B), or (A and B). For the purposes of this disclosure, "A, B, or C" or the phrase "A, B, and / or C" refers to (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C). When used to describe a measurement range, the term "between" includes both ends of the measurement range.

[0227] For the purposes of this disclosure, "A is less than or equal to a first threshold" is equivalent to "A is less than a second threshold," provided that the first and second thresholds are set in a manner that produces the same logical result for any value of A. For the purposes of this disclosure, "B is greater than a first threshold" is equivalent to "B is greater than or equal to a second threshold," provided that the first and second thresholds are set in a manner that produces the same logical result for any value of B.

[0228] The specification uses the phrases "in one embodiment" or "in multiple embodiments," which can refer to one or more of the same or different embodiments. The terms "comprising," "including," "having," etc., used in the embodiments of this disclosure are synonyms. This disclosure may use view-based descriptions such as "above," "below," "top," "bottom," and "side" to explain various features of the drawings, but these terms are merely for ease of discussion and do not imply desired or required orientation. The drawings are not necessarily drawn to scale. Unless otherwise stated, the ordinal numbers "first," "second," "third," etc., are used to describe the same object only to indicate different instances of the same kind of object and are not intended to imply that the described objects must be arranged in a given order in time, space, hierarchy, or any other respect.

[0229] In the following detailed description, terms commonly used by those skilled in the art will be used to describe various aspects of the exemplary embodiments in order to convey the substance of the work to others skilled in the art.

[0230] The terms “substantially,” “near,” “approximately,” “around,” and “about” generally refer to a target value within ±20% of the value described herein or known in the art. Similarly, terms indicating the orientation of various elements, such as “coplanar,” “perpendicular,” “orthogonal,” “parallel,” or any other angle between elements, generally refer to a target value within ±5–20% of the value described herein or known in the art.

[0231] Furthermore, the terms “comprising,” “including,” “containing,” “having,” “having,” or any other variations thereof are intended to cover non-exclusive inclusion. For example, a method, process, or apparatus that includes a list of elements is not necessarily limited to those elements and may include other elements not expressly listed or inherent to such a method, process, or apparatus. Additionally, the term “or” refers to an inclusive “or,” not an exclusive “or.”

[0232] The systems, methods, and apparatus disclosed herein are innovative in several ways, but no single aspect can achieve all the desirable characteristics disclosed herein. The specification and accompanying drawings illustrate details of one or more specific embodiments of the subject matter described herein.

Claims

1. A method comprising: Based on one or more filtered distortion values ​​calculated for a block of video frame, determine one or more minimum distortion values; A distortion metric is determined for a quantization parameter unit that includes at least the block, the distortion metric being based on a subset of the one or more minimum distortion values ​​of the block; Further distortion metrics are determined for the quantization parameter unit, the further distortion metrics being a further subset based on the one or more minimum distortion values ​​of the block; as well as Based on the distortion metric and the further distortion metric, the quantization parameters to be applied to the quantization parameter unit are determined.

2. The method of claim 1, wherein the one or more minimum distortion values ​​include one or more of a forward minimum distortion value, a further forward minimum distortion value, a backward minimum distortion value, and a further backward minimum distortion value.

3. The method of claim 2, wherein the subset of the one or more minimum distortion values ​​includes one or more of the forward minimum distortion value and the backward minimum distortion value.

4. The method of claim 2 or 3, wherein the further subset of the one or more minimum distortion values ​​includes one or more of the further forward minimum distortion value and the further backward minimum distortion value.

5. The method according to claim 2 or 3, wherein: The forward minimum distortion value is the lowest filtered distortion value among the one or more filtered distortion values ​​calculated for the block in the forward direction; as well as The further forward minimum distortion value is the second lowest filtered distortion value among the one or more filtered distortion values ​​calculated for the block in the forward direction.

6. The method according to claim 2 or 3, wherein: The backward minimum distortion value is the lowest filtered distortion value among the one or more filtered distortion values ​​calculated for the block in the backward direction; as well as The further backward minimum distortion value is the second lowest filtered distortion value among the one or more filtered distortion values ​​calculated for the block in the backward direction.

7. The method according to any one of claims 1-3, further comprising: The one or more filter distortion values ​​are determined for the block based on the sum of the squared differences between the block and the motion compensation block, and the variance of the block.

8. The method according to any one of claims 1-3, wherein the one or more filtered distortion values ​​are calculated for the block using a motion-compensated time filter.

9. The method according to any one of claims 1-3, wherein determining the distortion metric for the quantization parameter unit comprises: Determine the minimum value in the subset of the one or more minimum distortion values ​​for the block; as well as The average of one or more minimum values ​​determined by the one or more blocks of the quantization parameter unit is determined, the one or more minimum values ​​including the minimum value determined by the block.

10. The method according to any one of claims 1-3, wherein determining the further distortion metric for the quantization parameter unit comprises: Determine a further minimum value for a further subset of the one or more minimum distortion values; as well as A further average of one or more further minimum values ​​determined for one or more blocks of the quantization parameter unit, the one or more further minimum values ​​including the further minimum values ​​determined for the block.

11. The method according to any one of claims 1-3, wherein determining the distortion metric for the quantization parameter unit comprises: The average of one or more subsets of the one or more minimum distortion values ​​determined for one or more blocks of the quantization parameter unit, the one or more subsets including the subset of the one or more minimum distortion values ​​of the block.

12. The method according to any one of claims 1-3, wherein determining the further distortion metric for the quantization parameter unit comprises: The average of one or more further subsets of the one or more minimum distortion values ​​determined for one or more blocks of the quantization parameter unit, the one or more further subsets including the further subsets of the one or more minimum distortion values ​​of the block.

13. The method according to any one of claims 1-3, further comprising: Based on the distortion metric and the further distortion metric, a combined distortion metric is determined for the quantization parameter unit; The quantization parameters are determined based on the combined distortion metric.

14. The method according to any one of claims 1-3, further comprising: The average value is determined based on at least the subset of the one or more minimum distortion values; The variance of one or more averages determined for one or more blocks of a macroblock, the one or more averages including the averages determined for the block; and The size of the quantization parameter unit is determined based on the variance.

15. The method of claim 14, wherein determining the size of the quantization parameter unit comprises: The variance is compared with one or more thresholds corresponding to one or more sizes of the quantization parameter unit.

16. The method according to any one of claims 1-3, wherein determining the quantization parameter comprises: Based on the distortion metric and the further distortion metric, a combined distortion metric is determined for the quantization parameter unit; as well as The combined distortion metric is compared with more than four thresholds.

17. The method of claim 16, further comprising: The more than four thresholds are determined based on whether motion compensation time filtering is on or off.

18. The method according to any one of claims 1-3, wherein determining the quantization parameter comprises: Based on the distortion metric and the further distortion metric, a combined distortion metric is determined for the quantization parameter unit; as well as The combined distortion metric is compared with four or more thresholds.

19. The method of claim 18, further comprising: The four or more thresholds are determined based on whether motion compensation time filtering is on or off.

20. A computer program product comprising instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1-19.

21. An apparatus comprising means for performing the method of any one of claims 1-19.

22. An apparatus comprising: One or more processors; as well as One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1-19.

Images