Encoding / decoding video picture data

Abstract

This application relates to a method for encoding raw video frames, comprising: presampled adaptive offset vertical filtering using a first PSAO parameter combination and presampled adaptive offset horizontal filtering using a second PSAO parameter combination, wherein the first PSAO parameter combination and the second PSAO parameter combination are intended to be applied to the video frame data, wherein the incremental cost (ΔJ) is increased by... i,j Minimize the incremental cost (ΔJ) to jointly select the first and second PSAO parameter combinations. i,j The incremental distortion is defined as a linear combination of the incremental distortion and the incremental rate calculated for a pair of first candidate PSAO parameter combinations and second candidate PSAO parameter combinations. The incremental distortion is equal to the difference between the first distortion calculated by considering the presampled adaptive offset vertical filter and the presampled adaptive offset horizontal filter and the second distortion calculated without applying the presampled adaptive offset vertical filter and the presampled adaptive offset horizontal filter.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to European Patent Application No. 23305571.4, filed on 14 April 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application generally relates to video image encoding and decoding. In particular, but not limited thereto, the technical field of this application relates to cyclic filtering of video image blocks. Background Technology

[0004] This section is intended to introduce the reader to various aspects of the art that may relate to aspects of at least one exemplary embodiment of the present application described below and / or claimed. This discussion is intended to help provide the reader with background information to facilitate a better understanding of the various aspects of the present application. Therefore, these statements should be understood to be read in this context and not as an admission of prior art.

[0005] In state-of-the-art video compression systems such as HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en) or VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en), low-level and high-level picture partitioning is provided to divide the video frame into picture blocks, called codec tree units (CTUs). Their size is typically between 16x16 and 64x64 pixels for HEVC, and between 32x32, 64x64, or 128x128 pixels for VVC.

[0006] The CTU (Central Unit) division of a video frame forms a grid of fixed-size CTUs, known as the CTU grid. Its upper and left boundaries coincide spatially with the top and left boundaries of the video frame. The CTU grid represents the spatial partitioning of the video frame.

[0007] In VVC and HEVC, the CTU size (CTU width and CTU height) of all CTUs in the CTU mesh is equal to the same default CTU size (default CTU width CTU DW and default CTU height CTU DH). For example, the default CTU size (default CTU height, default CTU width) can be equal to 128 (CTU DW = CTU DH = 128). The default CTU size (height, width) is encoded into the bitstream, for example, at the sequence level in the Sequence Parameter Set (SPS).

[0008] The spatial position of a CTU within the CTU mesh is determined by its address ctuAddr, which defines the spatial position of the CTU's top-left corner relative to the origin. For example... Figure 1 As illustrated in the diagram, the CTU address can define a spatial location starting from the top left corner of the higher-level spatial structure S containing the CTU.

[0009] Each CTU is associated with a codec tree to determine the tree partitioning of the CTU.

[0010] like Figure 1 As illustrated above, in HEVC, the codec tree is a quadtree partition of the CTU, where each leaf is called a Codec Unit (CU). The spatial location of a CU in a video frame is defined by the CU index cuIdx, which indicates the spatial location starting from the top left corner of the CTU. CUs are spatially partitioned into one or more Prediction Units (PUs). The spatial location of a PU in a video frame (VP) is defined by the PU index puIdx, which defines the spatial location starting from the top left corner of the CTU. The spatial location of the elements within a partitioned PU is defined by the PU partition index puPartIdx, which defines the spatial location starting from the top left corner of the PU. Each PU is assigned some intra-frame or inter-frame prediction data.

[0011] The intra-frame or inter-frame encoding / decoding mode is assigned at the CU level. This means that although the prediction parameters vary from PU to CU, each PU is assigned the same intra-frame / inter-frame encoding / decoding mode.

[0012] According to a quadtree called a transform tree, a CU can also be spatially partitioned into one or more transform units (TUs). A transform unit is a leaf of the transform tree. The spatial location of a TU in a video frame is defined by a TU index tuIdx, which defines the spatial location starting from the top-left corner of the CU. Each TU is assigned a number of transform parameters. The transform type is assigned at the TU level, and individual 2D transforms are performed at the TU level during the encoding / decoding or decoding of a frame block.

[0013] Figure 2The diagram above illustrates the existing PU partition types in HEVC. These include square partitions (2N×2N and N×N), which are the only partitions used in both intra-frame and inter-frame prediction CUs; ​​symmetric non-square partitions (2N×N, N×2N, used only in inter-frame prediction CUs); and asymmetric partitions (used only in inter-frame prediction CUs). For example, PU type 2NxnU represents an asymmetric horizontal partition of the PU, where the smaller partition is located at the top of the PU. According to another example, PU type 2NxnL represents an asymmetric horizontal partition of the PU, where the smaller partition is located at the top of the PU.

[0014] like Figure 3 As shown above, in VVC, the codec tree starts from the root node (i.e., CTU). Next, a quadtree (or quaternion tree) split divides the root node into four nodes, corresponding to four child blocks of equal size (solid lines). Then, the quadtree (or quaternion tree) leaves can be further partitioned using a so-called multi-type tree, which involves... Figure 4 The above shows one of the four splitting patterns: a binary or ternary split. These splitting types are vertical and horizontal binary splitting patterns (denoted as SBTV and SBTH) and vertical and horizontal ternary splitting patterns SPTTV and STTH.

[0015] In the case of a joint codec tree where luminance and chrominance components share a common codec tree, the leaf of the CTU's codec tree is the CU.

[0016] In contrast to HEVC, in VVC, CU, PU, ​​and TU have the same size in most cases, which means that, except in some specific codec modes, codec units are generally not partitioned into PU or TU.

[0017] Figure 5 and Figure 6 It provides an overview of video encoding / decoding methods used in current video standard compression systems such as HEVC or VVC.

[0018] Figure 5 A schematic block diagram illustrating the steps of a method 100 for encoding a video frame VP according to the prior art is shown.

[0019] In step 110, the video frame VP is partitioned into sampling blocks, and the partition information data is signaled and transmitted to the bitstream. Each block includes a sample of one component of the video frame VP. Therefore, these blocks include samples that define each component of the video frame VP.

[0020] For example, in HEVC, the frame is divided into codec tree units (CTUs). Each CTU can be further subdivided using a quadtree partition, where each leaf of the quadtree is represented as a codec unit (CU). The partition information data can then include data describing the CTU and the quadtree subdivision of each CTU.

[0021] Therefore, each sampling block (or simply block) can be a CU (if the CU includes a single PU) or a PU of the CU.

[0022] Using intra-frame or inter-frame prediction modes, each block is encoded along the coding cycle (also known as "in the cycle").

[0023] Intra-frame prediction (step 120) uses intra-frame prediction data. Intra-frame prediction involves predicting the current block using blocks of intra-frame prediction based on encoded, decoded, and reconstructed samples located around the current block, typically at the top and left of the current block. Intra-frame prediction is performed in the spatial domain.

[0024] In inter-frame prediction mode, motion estimation (step 130) and motion compensation (135) are performed. Motion estimation searches for candidate reference blocks in one or more reference video frames used for predictively encoding the current video frame as good predictors for the current block. For example, a good predictor for the current block is a predictor similar to the current block. The output of motion estimation step 130 is inter-frame prediction data, which includes motion information associated with the current block (typically one or more motion vectors and one or more reference video frame indices) and other information for obtaining the same prediction block on the encoding / decoding side. Next, motion compensation (step 135) obtains the prediction block using the motion vector(s) and reference video frame(s) determined by motion estimation step 130. Essentially, a block belonging to a selected reference video frame and pointed to by a motion vector can be used as the prediction block for the current block. Furthermore, since motion vectors are expressed as fractions of integer pixel positions (this is called sub-pixel precision motion vector representation), motion compensation generally involves spatial interpolation of some reconstructed samples of the reference video frames to compute the prediction block.

[0025] The prediction information data is signaled and transmitted into the bitstream. The prediction information may include the prediction mode (intra-frame, frame reduction, or skip), intra-frame / inter-frame prediction data, and any other information used to obtain the same prediction CU on the decoding side.

[0026] Taking into account the encoding of the calculated prediction residual block (e.g., calculated by subtracting the candidate prediction block from the current block and signaling the prediction information data required to determine the candidate prediction block on the decoding side), method 100 selects a prediction mode (intra-frame or inter-frame prediction mode) by optimizing the rate-distortion tradeoff.

[0027] Typically, the best prediction mode is given as the prediction mode of the best encoding / decoding mode p* for the current block, as given by the following formula:

[0028]

[0029] Where P is the set of all candidate codec modes for the current block, p represents the candidate codec mode in this set, and RD cost (p) is the rate-distortion cost of the candidate encoding / decoding mode p, usually expressed as:

[0030] RD cost(p) =D(p)+λ.R(p)

[0031] D(p) is the distortion between the current block and the reconstructed block obtained after encoding / decoding the current block with candidate encoding / decoding mode p, R(p) is the rate cost associated with encoding / decoding the current block with encoding / decoding mode p, and λ is the Lagrangian parameter representing the rate constraint for encoding / decoding the current block and is usually calculated based on the quantization parameter Qp used to encode the current block.

[0032] The current block is typically encoded from the predicted residual block PR. More precisely, for example, the predicted residual block PR is calculated by subtracting the best predicted block from the current block. The predicted residual block PR is then transformed (step 140) using a transform of type DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform) or any other suitable transform, and the resulting transformed coefficient block is quantized (step 150).

[0033] In a variant, method 100 can also skip transform step 140 according to a so-called transform-skip encoding / decoding mode and directly apply quantization to the predicted residual block PR (step 150).

[0034] Encode the quantized transform coefficient block (or quantized prediction residual block) into the bit stream (step 160).

[0035] Next, as part of the encoding loop, the quantized transform coefficient block (or quantized residual block) is dequantized (step 170) and inverse transformed (180) (or not) to obtain the decoded prediction residual block. Then, the decoded prediction residual block and the prediction block are combined (usually summed) to provide the reconstructed block.

[0036] In step 160, entropy encoding can also be performed on other information data to encode the current block of the video frame VP.

[0037] A loop filter (step 190) can be applied to the reconstructed frame (including reconstructed blocks) to reduce compression artifacts. Loop filters can be applied after all frame blocks have been reconstructed. Examples include deblocking filters, sample adaptive offset (SAO) filters, or adaptive loop filters.

[0038] The reconstructed block or the filtered reconstructed block forms a reference frame, which can be stored in the decoded frame buffer (DPB) so that it can be used as a reference frame for encoding the next current block of the video frame VP or as a reference frame for encoding the next video frame to be encoded.

[0039] Figure 6 A schematic block diagram illustrating the steps of a method 200 for decoding a video frame VP according to the prior art is shown.

[0040] In step 210, partition information data, prediction information data, and quantized transform coefficient blocks (or quantized residual blocks) are obtained by entropy decoding of the bitstream of encoded video frame data. For example, this bitstream has been generated according to method 100.

[0041] It can also perform entropy decoding on other information data to decode the current block of the video frame VP from the bitstream.

[0042] In step 220, the reconstructed frame is divided into current blocks based on the partitioning information. Each current block is entropy decoded from the bitstream along the decoding loop (also known as "in loop"). Each decoded current block is either a quantized transform coefficient block or a quantized prediction residual block.

[0043] In step 230, the current block is dequantized and possibly inverse transformed (step 240) to obtain the decoded prediction residual block.

[0044] On the other hand, the current block is predicted using prediction information data. The predicted block is obtained through its intra-frame prediction (step 250) or its motion-compensated temporal prediction (step 260). The prediction process performed on the decoding side is exactly the same as the prediction process on the encoding side.

[0045] Next, the decoded prediction residual block and prediction block are combined (usually by summation), which provides the reconstructed block.

[0046] In step 270, the loop filter can be applied to the reconstructed frame (including the reconstructed block), and the reconstructed block or the filtered reconstructed block forms a reference frame, which can be stored in the decoded frame buffer (DPB), as discussed above. Figure 5 ).

[0047] exist Figure 5In step 130 / 135 or Figure 6 In step 260, the inter-frame prediction block is defined based on inter-frame prediction data associated with the current block (CU or PU of the CU) of the video frame. This inter-frame prediction data includes motion information that can be represented (encoded) according to either a so-called block-based AMVP (Adaptive Motion Vector Prediction) mode or a so-called block-based merging mode.

[0048] Recently, the presampled adaptive offset (Pre-SAO) filter, as defined in JVET-Q0434-v5 (Wang-Q Lim, Christian Helmrich, Johannes Erfurt, Heiko Schwarz, Detlev Marpe and Thomas Wiegand, “Presampled Adaptive Offset Filter”, ITU-T SG16WP 3 and ISO / IEC JTC 1 / SC 29, 17th meeting, Brussels, January 7-17, 2020, document: JVET-Q0434-v5), has been introduced as a loop filter.

[0049] Essentially, in JVET-Q0434-v5, Pre-SAO (PSAO) filtering operates after deblocking filtering to further improve the encoding / decoding efficiency of filters in the loop. Pre-SAO filtering adds an offset to pixels that are first classified into one of the possible categories. For each category, there is an associated offset. For example... Figure 7 As shown in the diagram above, the Pre-SAO filter is applying two SAO-like filtering operations: one is Pre-SAO Vertical (PSAOV) filtering after Deblocking Vertical (DBLV) filtering; and the other is Pre-SAO Horizontal (PSAOH) filtering after Deblocking Horizontal (DBLH) filtering.

[0050] The PSAOV filter operates on the pixel difference calculated between pixel Y1(p) of the input video frame Y1 at the input of the DBLV filter and pixel Y2(p) of the second video frame Y2 at the output of the DBLV filter.

[0051] Mathematically, the pixel Y3(p) of the third video frame Y3 at the output of the PSAOV filter is given by the following formula:

[0052] Y3(p)=Clip1(Y2(p)+d1·(diff1(p)>T?1:0)-d2·(diff1(p)<-T?1:0)) (2)

[0053] Where T is the threshold (a predetermined normal value) and d1 and d2 are offset coefficients (predetermined normal values) based on the inter-sample difference diff1(p) between Y1(p) and Y2(p) as given by the following formula:

[0054] diff1(p) = Y1(p) - Y2(p).

[0055] The threshold T and offset coefficients d1 and d2 are PSAO parameters used for PSAOV filtering.

[0056] The first category, applying offset d1, is given by taking all pixel positions p such that diff1(p) > T, while the second category, applying offset d2, is given by diff1(p) < - T. Offset coefficients d1 and d2 of PSAOV are calculated at the encoder to minimize the mean square error between the third video frame Y3 and the reference video frame X (the original video frame VP).

[0057] The PSAOH filter operates on the pixel difference calculated between pixel Y3(p) of the third video frame Y3 at the input of the DBLH filter and pixel Y4(p) of the fourth video frame Y4 at the output of the DBLH filter.

[0058] Mathematically, the pixel Y5(p) of the fifth video frame Y5 at the output of the PSAOH filter is given by the following formula:

[0059] Y5(p)=Clip1(Y4(p)+d1·(diff2(p)>T?1:0)-d2·(diff2(p)<-T?1:0)) (3)

[0060] Where T is the threshold (a predetermined normal value) and d1 and d2 are offset coefficients (predetermined normal values) based on the inter-sample difference diff2(p) between Y3(p) and Y4(p) as given by the following formula, associated with the two classes:

[0061] diff2(p) = Y3(p) - Y4(p).

[0062] The threshold T and offset coefficients d1 and d2 are PSAO parameters used for PSAOH filtering.

[0063] The first category of applying offset d1 is given by taking all pixel positions p such that diff2(p) > T, while the second category of applying offset d2 is given by diff2(p) < - T. Offset coefficients d1 and d2 of PSAOV are calculated at the encoder to minimize the mean square error between the fifth video frame Y5 and the reference video frame X.

[0064] The two predetermined offset coefficients d1 and d2 for each of the PSAOH and PSAOV filters, as well as the predetermined threshold T and the activation (enable) flag, are separately signaled to the decoder (the respective PSAO parameters for the PSAOH and PSAOV filters).

[0065] PSAOH and PSAOV are applied independently to the luminance component and the two chrominance components (however, the two chrominance components share the same Pre-SAO syntax; for example, the same parameters are applied to both the Cb and Cr components of the video frame).

[0066] Figure 8 The three categories, as defined in JVET-Q0434-v5, are illustrated schematically.

[0067] Pixels unaffected by the deblocking filter will always have a difference of 0 before and after the deblocking filter; for example, the diff1 and diff2 of those pixels will always be 0, and therefore they will always fall into category 0. This means they will never be affected by the PSAO filter (no offset is added to those pixels). In other words, as mentioned earlier, the PSAOV and PSAOH filters only operate on pixels of the video frame modified by the corresponding DBLV and DBLH deblocking filters.

[0068] For category 1, add offset d1 to the pixel, and for category 2, subtract offset d2 from the pixel.

[0069] At the encoder, the PSAO parameters for PSAOV and PSAOH filtering are selected separately using an incremental cost RDO process.

[0070] Figure 9 The illustration shows the selection of PSAO parameters according to existing technology.

[0071] Basically, PSAO parameters are selected from the RDO process based on incremental cost ΔJ, which is defined as a linear combination of incremental distortion ΔD and incremental rate ΔR. Incremental distortion ΔD represents the distortion change when PSAO filtering is applied or not, and incremental rate ΔR represents the number of bits required to encode the syntax elements related to PSAO filtering using the selected PSAO parameters.

[0072] ΔJ=ΔD+αΔR

[0073] Where α is the Lagrange multiplier.

[0074] In the current VVC reference implementation, deblocking parameters can be defined a priori in the configuration file, for example, or deblocking parameters can be adaptively selected based on the RDO at the encoder.

[0075] The RDO process selects the set of PSAO parameters that minimize ΔJ, including the case where PSAO is not applied (ΔJ = 0).

[0076] Clearly, in order to improve compression performance by applying PSAO filtering, ΔJ must be less than 0.

[0077] More precisely, in the first step, by making the incremental cost ΔJV i Minimize, select the first PSAO parameter combination as the PSAO parameters for PSAOV filtering, incremental cost ΔJV i Defined as incremental distortion ΔD i With incremental rate ΔR i Linear combination, incremental distortion ΔD i The increment rate ΔR represents the change in distortion when PSAOV filtering is applied or not (using the i-th candidate PSAO parameter combination). i This represents the number of bits required to encode the syntax elements related to the PSAOV filter using the i-th candidate PSAO parameter combination.

[0078] In the second step, by making the incremental cost ΔJH j To minimize, the second PSAO parameter combination is selected as the PSAO parameter for PSAOH filtering, with an incremental cost ΔJH. j Defined as incremental distortion ΔD j With incremental rate ΔR j Linear combination, incremental distortion ΔD j The increment rate ΔR represents the change in distortion when PSAOH filtering is applied or not (using the j-th candidate PSAO parameter combination). j This represents the number of bits required to encode the syntax elements related to PSAOH filtering using the j-th candidate PSAO parameter combination.

[0079] Incremental distortion ΔD i Defined as incremental distortion ΔD 1,i and incremental distortion ΔD 2,i The sum of the incremental distortion ΔD 1,i Defined as distortion D after1,i With distortion D before The difference between them, and the incremental distortion ΔD 2,i Defined as distortion D after2,i With distortion D before The difference between them.

[0080] In mathematics,

[0081]

[0082] Where N1 is a subset of Diff1(p) > T.i The number of pixels (in other words, during PSAO pixel classification) Figure 8 The number of pixels p that fall into category 1, where ΔD is calculated only for those pixels p that fall into category 1. 1,i ).

[0083]

[0084] Where N2 is a subset satisfying diff1(p) <- T i The number of pixels (in other words, the number of pixels p that fall into category 2 during PSAO pixel classification, where ΔD is calculated only for those pixels p that fall into category 2). 2,i ).

[0085] Calculate the distortion D between the reference video frame X and the output video frame. after1,i The pixel value is the pixel value of the second video frame Y2 and includes d. 1,i The offset d of the i-th PSAO parameter combination 1,i sum:

[0086] D after1,i =∑(X-(Y2+d) 1,i )) 2 .

[0087] The distortion D is calculated between the reference video frame X and the second video frame Y2. before :

[0088] D before =∑(X-Y2) 2 .

[0089] Calculate the distortion D between the reference video frame X and the output video frame. after2,i The pixel value is the offset d of the pixel value of the second video frame Y2 and the i-th PSAO parameter combination. 2,i The difference between them:

[0090] D after2,i =∑(X-(Y2-d) 2,i )) 2 .

[0091] For category 0, the incremental distortion is ΔD = 0 because no change in pixel values ​​was observed and no offset was added.

[0092] Incremental rate ΔR i This represents the variation in the number of bits required to encode the PSAO syntax associated with the i-th PSAO parameter combination.

[0093] Incremental distortion ΔD j Defined as incremental distortion ΔD1,j With incremental distortion ΔD 2,j The sum of the incremental distortion ΔD 1,j Defined as distortion D after1,j With distortion D before The difference between them, and the incremental distortion ΔD 2,j Defined as distortion D after2,j With distortion D before The difference between them.

[0094] In mathematics,

[0095]

[0096] Where N1 is a subset of Diff1(p) > T. j The number of pixels (in other words, during PSAO pixel classification) Figure 8 The number of pixels p that fall into category 1, where ΔD is calculated only for those pixels p that fall into category 1. 1,j ).

[0097]

[0098] Where N2 is a subset satisfying diff1(p) <- T j The number of pixels p (in other words, the number of pixels p that fall into category 2 during PSAO pixel classification, where ΔD is calculated only for those pixels p that fall into category 2). 2,j ).

[0099] Calculate the distortion D between the reference video frame X and the output video frame. after1,j The pixel value is the pixel value of the fourth frame Y4 at the output of the DBLH filter, and includes d 1,j Offset d of the j-th PSAO parameter combination 1,j sum:

[0100] D after1,j =∑(X-(Y4+d) 1,j )) 2 .

[0101] Calculate the distortion D between the reference video frame X and the output video frame. after2,j The pixel value is the offset d of the pixel value of the fourth frame Y4 and the j-th PSAO parameter combination. 2,j The difference between them:

[0102] D after2,j =∑(X-(Y4-d) 2,j )) 2 .

[0103] Incremental rate ΔR jThis represents the change in the number of bits required to encode the PSAO syntax associated with the j-th PSAO parameter combination.

[0104] The incremental rate ΔR associated with the i-th or j-th PSAO parameter combination is defined as representing the rate of change with distortion D. before The first rate R of the number of bits required to encode the associated syntax before The expression represents the difference between and distortion D. after1,i and D after2,i Or D after1,j and D after2,j The rate R of the number of bits required to encode the associated syntax after The difference between them:

[0105] ΔR=R after -R before .

[0106] according to Figure 9 One drawback of selecting PSAO parameters is that it may miss selecting the optimal possible combination of PSAO parameters because it locally optimizes the PSAO parameters for the PSAOV filter and then optimizes the PSAO parameters for the PSAOH filter. Furthermore, since this is a sequential process, it cannot be performed in parallel. In other words, the selection of PSAO parameters for the PSAOH filter can only be completed after the PSAO parameters for the PSAOV filter have been selected.

[0107] At least one exemplary embodiment of this application is designed with the above circumstances in mind. Summary of the Invention

[0108] The following sections provide a brief overview of at least one exemplary embodiment to provide a basic understanding of some aspects of this application. This overview is not an exhaustive summary of the exemplary embodiments. Its purpose is not to identify key or core elements of the exemplary embodiments. The following overview presents only some aspects of at least one exemplary embodiment in a simplified form as a prelude to a more detailed description provided elsewhere in this document.

[0109] According to a first aspect of this application, a method for encoding raw video frames is provided, comprising: presampled adaptive offset vertical filtering using a first PSAO parameter combination and presampled adaptive offset horizontal filtering using a second PSAO parameter combination, wherein the first PSAO parameter combination and the second PSAO parameter combination are intended to be applied to video frame data.

[0110] The first and second PSAO parameter combinations are jointly selected by minimizing the incremental cost, which is defined as a linear combination of incremental distortion and incremental rate calculated for each pair of first and second candidate PSAO parameter combinations.

[0111] The incremental distortion is equal to the difference between the first distortion and the second distortion.

[0112] The first distortion is defined as the cumulative difference between the pixels of the reference video frame derived from the original video frame and the pixels of the output video frame, which is obtained by applying deblocking vertical filtering to the input video frame, followed by presampling adaptive offset vertical filtering using the first candidate PSAO parameter combination, then deblocking horizontal filtering, and finally presampling adaptive offset horizontal filtering using the second candidate PSAO parameter combination.

[0113] The second distortion is defined as the cumulative difference between the pixels of the reference video frame and the pixels of the output video frame, which is obtained by applying vertical and horizontal deblocking filters to the input video frame without applying presampled adaptive offset vertical and presampled adaptive offset horizontal filters.

[0114] and

[0115] The increment rate represents the number of bits required to encode the following related syntax elements: presampled adaptive offset vertical filtering using the first candidate PSAO parameter combination and presampled adaptive offset horizontal filtering using the second candidate PSAO parameter combination.

[0116] In one exemplary embodiment, the PSAO parameter includes two offset coefficients, wherein the offset range of the two offset coefficients allows for negative offset coefficients.

[0117] In one exemplary embodiment, the deblocking parameters used for deblocking vertical filtering or deblocking horizontal filtering are selected by minimizing the cost, which is defined as the distortion between the reference video frame (X) and the output video frame of the deblocking vertical filter or deblocking horizontal filter using the candidate deblocking parameters.

[0118] In one exemplary embodiment, the deblocking parameters used for either the deblocking vertical filter or the deblocking horizontal filter are selected without considering the presampled adaptive offset vertical filter and the presampled adaptive offset horizontal filter.

[0119] In one exemplary embodiment, the deblocking parameters used for deblocking vertical filtering or deblocking horizontal filtering are jointly selected in combination with the first PSAO parameter combination and the second PSAO parameter combination.

[0120] In one exemplary embodiment, the input video frame includes a luminance component and two chrominance components. A first joint selection of the deblocking parameters used in vertical or horizontal deblocking filtering, combined with a first PSAO parameter combination and a second PSAO parameter combination, is used for the luminance component of the input video frame. A second joint selection of the deblocking parameters used in vertical or horizontal deblocking filtering, combined with a first PSAO parameter combination and a second PSAO parameter combination, is used for the chrominance component of the input video frame.

[0121] In one exemplary embodiment, the deblocking parameters used in the vertical deblocking filter are the same as those used in the horizontal deblocking filter.

[0122] In one exemplary embodiment, the reference video frame is the original video frame.

[0123] In one exemplary embodiment, a reference video frame is obtained by motion-compensated time filtering.

[0124] According to a second aspect of this application, an apparatus is provided, comprising components for performing one of the methods according to a first aspect of this application.

[0125] According to a third aspect of this application, a computer program product is provided, including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to the first and / or second aspects of this application.

[0126] According to a fourth aspect of this application, a non-transitory storage medium is provided, which carries instructions for performing program code for executing the methods according to the first and / or second aspects of this application.

[0127] The specific nature of at least one exemplary embodiment in the exemplary embodiments, as well as other objects, advantages, features, and uses of at least one exemplary embodiment in the exemplary embodiments, will become apparent from the following description of the examples taken in conjunction with the accompanying drawings. Attached Figure Description

[0128] Reference will now be made to the accompanying drawings, which illustrate exemplary embodiments of this application, wherein:

[0129] Figure 1 An example of a codec tree unit based on HEVC is shown;

[0130] Figure 2 An example of partitioning codec units into prediction units according to HEVC is shown;

[0131] Figure 3 An example of CTU partitioning based on VVC is shown;

[0132] Figure 4An example of the splitting patterns supported in VVC's multi-type tree partitioning is shown;

[0133] Figure 5 A schematic block diagram illustrating the steps of a method 100 for encoding a video frame VP according to the prior art is shown;

[0134] Figure 6 A schematic block diagram of the steps of a method 200 for decoding a video frame VP according to the prior art is shown;

[0135] Figure 7 A pre-SAO filter combined with deblocking filtering according to the prior art is schematically illustrated;

[0136] Figure 8 The three categories, as defined in JVET-Q0434-v5, are illustrated schematically.

[0137] Figure 9 The diagram illustrates the selection of PSAO parameters according to existing technology;

[0138] Figure 10 The selection of PSAO parameters according to an exemplary embodiment of the present invention is illustrated schematically;

[0139] Figure 11 The combined selection of deblocking parameters and PSAO parameters according to an exemplary embodiment of the present invention is illustrated schematically;

[0140] Figure 12 A schematic block diagram illustrating an example of a system in which various aspects and exemplary embodiments are implemented is shown.

[0141] Similar or identical elements are referenced using the same figure labels. Detailed Implementation

[0142] At least one of the exemplary embodiments will be described more fully below with reference to the accompanying drawings, which depict examples of at least one of the exemplary embodiments. However, the exemplary embodiments may be implemented in various alternative forms and should not be construed as limited to the examples set forth herein. Accordingly, it should be understood that the present invention is not intended to limit the exemplary embodiments to the specific forms disclosed. Rather, this application is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this application.

[0143] At least one of these aspects generally relates to video image encoding and decoding, another generally relates to the transmission of a bit stream provided or encoded, and yet another relates to the reception / access of a decoded bit stream.

[0144] At least one of the exemplary embodiments is described in relation to encoding / decoding a single video frame, but this is extended to encoding / decoding multiple video frames (a sequence of frames), as each video frame is encoded / decoded sequentially as described below.

[0145] Furthermore, at least one exemplary embodiment is not limited to MPEG standards, such as AVC (ISO / IEC 14496-10 Advanced Video Coding for generic audio-visual services, ITU-T Recommendation H.264). https: / / www.itu.int / rec / T-REC-H.264-202108-P / en This doesn't mean it can be applied to other standards and recommendations, such as AV1 (AOMedia Video 1). The specific standards mentioned include EVC (ISO / IEC 23094-1 Essential Video Coding), HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en), and VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en). http: / / aomedia.org / av1 / specification / At least one exemplary embodiment can be applied to any existing or future developments and extensions of such standards and recommendations. Unless otherwise stated or technically excluded, the aspects described in this application may be used alone or in combination.

[0146] A pixel corresponds to the smallest display unit on the screen, and it can consist of one or more light sources (one for a monochrome screen and three or more for a color screen).

[0147] A video frame, also known as a frame or video frame, includes at least one component (also known as a video component or channel) defined by a specific video / video format that specifies all information related to pixel values ​​and all information that can be used by a display unit and / or any other device to display and / or decode video frame data associated with the video frame.

[0148] The video frame includes at least one component, which is typically represented in the form of a sample array.

[0149] Monochrome video footage consists of a single component, while color video footage can consist of three components.

[0150] For example, when the picture / video format is a well-known (Y,Cb,Cr) format, a color video picture may include one luminance (or brightness) component and two chrominance components, or when the picture / video format is a well-known (R,G,B) format, a color video picture may include three color components (one for red, one for green, and one for blue).

[0151] Each component of a video frame may include a number of samples relative to the number of pixels on the screen on which the video frame is to be displayed. In variations, the number of samples included in a component may be a multiple (or fraction) of the number of samples included in another component of the same video frame.

[0152] For example, in a video format that includes one luminance component and two chrominance components (such as the (Y,Cb,Cr) format), depending on the color format under consideration, the chrominance component may contain half the number of samples in width and / or height relative to the luminance component.

[0153] At least one exemplary embodiment is not limited to a specific image / video format.

[0154] A sample is the smallest unit of visual information that makes up a video frame. Sample values ​​can be, for example, luminance or chrominance values, or color values ​​in (R,G,B) format.

[0155] A pixel value is the value of a pixel on the screen. For monochrome video, a pixel value can be represented by a single sample, while for color video, it can be represented by multiple samples at co-locations. A co-location sample associated with a pixel refers to the sample corresponding to the pixel's position on the screen.

[0156] Video frames are typically viewed as a set of pixel values, with each pixel represented by at least one sample.

[0157] Figure 10 The selection of PSAO parameters according to an exemplary embodiment of the present invention is illustrated schematically.

[0158] According to the present invention, by making the incremental cost ΔJ i,j Minimize the incremental cost ΔJ for jointly selecting the first PSAO parameter combination for the PSAOV filter and the second PSAO parameter combination for the PSAOH filter. i,j Defined as the incremental distortion ΔD calculated for the pair of candidate PSAO parameter combinations of the i-th and j-th candidates. i,j and incremental rate ΔR i,j A linear combination of .

[0159] Incremental rate ΔR i,jThis indicates the number of bits required to encode the following syntax elements: PSAOV filtering using the i-th candidate PSAO parameter combination and PSAOH filtering using the j-th candidate PSAO parameter combination.

[0160] The joint selection of the first and second PSAO parameter combinations overcomes the shortcomings of selecting separate combinations of PSAO parameters for the PSAOV and PSAOH filters, and leads to more accurate PSAO parameter adaptation.

[0161] Distortion ΔD i,j Given from the following:

[0162] ΔD i,j =D i,j -D 0,0

[0163] Where D i,j =∑(X-Y5) 2 It is the cumulative difference between the pixels of the reference video frame X and the pixels of the output frame Y5.

[0164] The output image Y5 is obtained by applying DBLV filtering to the input video image Y1, followed by PSAOV filtering with the first candidate (i) PSAO parameter combination, then DBLH filtering, and finally PSAOH filtering with the second candidate (j) PSAO parameter combination.

[0165] The input video frame Y1 is the video frame on which loop deblocking and PSAO filtering are applied.

[0166] Distortion D 0,0 The output video frame corresponds to the cumulative difference between the pixels of the reference frame X and the pixels of the output video frame, which is obtained after applying vertical and horizontal deblocking filters to the input video frame Y1. Neither the PSAOV nor the PSAOH filter was applied.

[0167] Indices i and j can take values ​​from 0 to N, where N is the total number of possible PSAO parameter combinations.

[0168] Obviously, ΔD i,j <0 indicates improved distortion through the application of PSAO filtering.

[0169] In one exemplary embodiment of this application, the PSAO parameter includes two offset coefficients d1 and d2, and the offset range allows the PSAO parameters d1 and d2 to be negative.

[0170] In one variant of the exemplary embodiment, the offset range of the offset coefficients of the PSAO parameter is symmetrical, i.e., d1,d2∈{-Dmax,…,Dmax}.

[0171] In one variant of the exemplary embodiment, the offset range is asymmetric, i.e., d1,d2∈{-D1max,…,D2max}, where -D1max represents the minimum value and D2max represents the maximum allowed / supported value, and D1max≠D2max.

[0172] In one variant, Figure 9 The distortion calculation for the RDO process is as follows, so that incremental distortion calculation is performed when negative offset coefficients are allowed:

[0173] D after2,i =∑(X-(Y2+d) 2,i )) 2 ,

[0174]

[0175] D after2,j =∑(X-(Y4+d) 2,j )) 2 ,

[0176]

[0177] In one exemplary embodiment, the deblocking parameters used by DBLV or DBLH filtering are selected from a simplified RDO procedure, which makes the distortion D... m,n Cost J m,n Minimize. In the simplified RDO process, the rate is considered to be 0. In such a process, only distortion is effectively minimized. The candidate deblocking parameter DBLV is used for DBLV filtering. m and candidate deblocking parameters DBLH used for DBLH filtering n Distortion D is defined between the reference video frame X and the output video frames of the DBLV and DBLH deblocking filters. m,n .

[0178] In one variant, the deblocking parameters for DBLV and DBLH filtering are selected, without considering PSAOV and PSAOH filtering.

[0179] PSAO filtering is not applied during deblocking parameter selection, but once the deblocking parameters are selected, PSAO filtering can be applied (and optimized) later.

[0180] In one variant, the deblocking parameters are jointly selected by the RDO process in combination with the first PSAO parameter combination and the second PSAO parameter combination.

[0181] According to this last variant, the RDO process iterates over all possible combinations of DBL and PSAO parameters and selects those parameters that minimize the overall cost.

[0182] exist Figure 11 The non-limiting example embodiments provide an implementation method for the joint optimization of DBL and PSAO parameters.

[0183] exist Figure 11 In the optimization process, all possible combinations of deblocking parameters m and n are considered, which are represented here as DBLV-m for DBLV filter and DBLH-n for DBLH filter.

[0184] For each pair of deblocking parameter combinations DBLV-m and DBLH-n, by using the respective selections as defined in the prior art ( Figure 9 ) or by using a combined selection according to exemplary embodiments of this application ( Figure 10 ) to select the i-th PSAO parameter combination and the j-th PSAO parameter combination.

[0185] For the joint selection of exemplary embodiments according to this application ( Figure 10 Once the optimal i-th PSAO parameter combination and the optimal j-th PSAO parameter combination are selected for the paired deblocking parameter combinations DBLV-m and DBLH-n, then, as previously explained, based on distortion D... m,n To calculate cost J m,n .

[0186] The optimal paired combination of deblocking parameters DBLV-m and DBLH-n corresponds to the minimum cost J. m,n (or equivalent to minimum distortion D) m,n Since only distortion is effectively optimized, and it includes the minimum incremental cost (e.g., ΔJV if separate PSAO optimization is used). i and ΔJH j Alternatively, if joint PSAO optimization is used, the result is ΔJ. i,j The optimal pairwise iPSAO parameter combination and jPSAO parameter combination.

[0187] In one variant, an RDO process is required for each iteration because the PSAO parameters vary depending on the deblocking parameters being used. Then, only the distortion D is adjusted. m,n Choose the optimal deblocking parameters in a minimal way, for example, by selecting m and n, so that the distortion D m,n Minimize (m,n)|min{D m,n}

[0188] The optimal PSAO parameters are implicitly selected by choosing the best paired combination of deblocking parameters DBLV-m and DBLH-n, because the optimal PSAO parameters are obtained during the RDO process for each pair of deblocking parameters.

[0189] In one variant, the input video frame Y1 includes a luminance component and two chrominance components. A first joint selection of deblocking parameters and PSAO parameters is used for the luminance component of the input video frame Y1, and a second joint selection of deblocking parameters and PSAO parameters is used for the chrominance component of the input video frame Y1.

[0190] In one variant, the deblocking parameter DBLV-m combination is the same as the deblocking parameter DBLH-n combination.

[0191] In one variant, the reference video frame X is the video frame VP to be encoded.

[0192] In one variant, reference frame X is obtained via motion-compensated temporal filtering (MCTF), as defined in (W. Per, H. Christopher and J. Strom, “Gop-based temporal filter improvements”, Joint Video Experts Group (JVET) of ITU-TSG 16WP 3 and ISO / IEC JTC 1 / SC 29, document JVET-V0056, teleconference, April 2021).

[0193] Figure 12 A schematic block diagram illustrating an example of a system 600 in which various aspects and exemplary embodiments are implemented is shown.

[0194] System 600 can be embedded as one or more devices, including the various components described below. In various exemplary embodiments, system 600 can be configured to implement one or more aspects described in this application.

[0195] Examples of equipment that may constitute all or part of System 600 include personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMDs, see-through glasses), projectors, "cave" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process the output from the video decoder, pre-processors that provide input to the video encoder, web servers, video servers (e.g., broadcast servers, video-on-demand servers, or web servers), still or video cameras, encoding or decoding chips, or any other communication devices. The elements of System 600 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one exemplary embodiment, the processing and encoder / decoder elements of System 600 may be distributed across multiple ICs and / or discrete components. In various exemplary embodiments, System 600 may be communicatively connected to other similar systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0196] System 600 may include at least one processor 610 configured to execute instructions loaded therein for implementing various aspects, such as those described in this application. Processor 610 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 600 may include at least one memory 620 (e.g., a volatile memory device and / or a non-volatile memory device). System 600 may include a storage device 640, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 640 may include internal storage devices, attached storage devices, and / or network-accessible storage devices.

[0197] System 600 may include an encoder / decoder module 630 configured to, for example, process data to provide encoded / decoded video image data, and the encoder / decoder module 630 may include its own processor and memory. The encoder / decoder module 630 may represent one or more modules that can be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Furthermore, the encoder / decoder module 630 may be implemented as a separate element of system 600, or may be incorporated into processor 610 as a combination of hardware and software known to those skilled in the art.

[0198] Program code to be loaded into processor 610 or encoder / decoder 630 to execute the various aspects described in this application may be stored in storage device 640 and subsequently loaded into memory 620 for execution by processor 610. According to various exemplary embodiments, during the execution of the processes described in this application, one or more of processor 610, memory 620, storage device 640, and encoder / decoder module 630 may store one or more of various items. Such stored items may include, but are not limited to, video frame data, information data for encoding / decoding video frame data, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and arithmetic logic processing.

[0199] In several exemplary embodiments, the memory within the processor 610 and / or encoder / decoder module 630 may be used to store instructions and provide working memory for processes that can be performed during encoding or decoding.

[0200] However, in other exemplary embodiments, external memory (e.g., the processing device may be processor 610 or encoder / decoder module 630) is used for one or more of these functions. External memory may be memory 620 and / or storage device 640, such as dynamic volatile memory and / or non-volatile flash memory. In several exemplary embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one exemplary embodiment, fast external dynamic volatile memory, such as RAM, may be used as working memory for video encoding / decoding operations, for example, for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 video), AVC, HEVC, EVC, VVC, AVI, etc.

[0201] As indicated in box 690, input to the components of system 600 can be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section capable of receiving, for example, RF signals transmitted over the air by a broadcasting device, (ii) a composite input terminal, (iii) a USB input terminal, (iv) an HDMI input terminal, and (v) a bus, such as CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data Rate), FlexRay (ISO 17458), or Ethernet (ISO / IEC 802-3) bus, when the invention is implemented in the automotive field.

[0202] In various exemplary embodiments, the input device of block 690 has associated corresponding input processing elements, as known in the art. For example, the RF section may be associated with elements necessary for: (i) selecting a desired frequency (also known as selecting a signal, or limiting the signal band to a specific band), (ii) down-converting the selected signal, (iii) further limiting the band to a narrower band to select, for example, the signal band that may be referred to as a channel in some exemplary embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF section of various exemplary embodiments may include one or more elements performing these functions, such as a frequency selector, signal selector, band limiter, channel selector, filter, downconverter, demodulator, error corrector, and demultiplexer. The RF section may include tuners performing various functions among these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband.

[0203] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band.

[0204] Various exemplary embodiments may rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0205] Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various exemplary embodiments, the RF portion may include an antenna.

[0206] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 600 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within processor 610, as needed. Similarly, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 610, as needed. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, processor 610 and encoder / decoder 630, which operate in conjunction with memory and storage elements to process the data streams for presentation on an output device as needed.

[0207] Various components of system 600 can be provided within an integrated housing. Within the integrated housing, suitable connection arrangements 690, such as internal buses (including I2C buses), wiring, and printed circuit boards known in the art, can be used to interconnect various components and transfer data between them.

[0208] System 600 may include a communication interface 650 that enables communication with other devices via a communication channel 651. The communication interface 650 may include, but is not limited to, a transceiver configured to send and receive data on the communication channel 651. The communication interface 650 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 651 may be implemented, for example, within a wired and / or wireless medium.

[0209] In various exemplary embodiments, a Wi-Fi network such as IEEE 802.11 can be used to stream data to system 600. The Wi-Fi signals of these exemplary embodiments can be received via a communication channel 651 and a communication interface 650 suitable for Wi-Fi communication. The communication channel 651 of these exemplary embodiments can typically be connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top cloud communications.

[0210] Other exemplary embodiments may use a set-top box to provide streaming data to system 600, the set-top box delivering data via an HDMI connection in input block 690.

[0211] Other exemplary embodiments may use the RF connection of input block 690 to provide streaming data to system 600.

[0212] Streamed data can be used as a means of transmitting signaling information by System 600. This signaling information may include bitstream B and / or information such as the number of pixels in a video frame and / or any encoding / decoding settings.

[0213] It should be recognized that signal transmission can be implemented in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc., can be used to transmit signaling information to the corresponding decoder.

[0214] System 600 can provide output signals to various output devices, including a display 661, a speaker 671, and other peripheral devices 681. In various examples of exemplary embodiments, other peripheral devices 681 may include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 600.

[0215] In various exemplary embodiments, control signals may be communicated between system 600 and display 661, speaker 671 or other peripheral devices 681 using signaling such as AV.Link (audio / video link), CEC (consumer electronics control), or other communication protocols that enable device-to-device control with or without user intervention.

[0216] Output devices can be connected to system 600 via dedicated connections through the corresponding interfaces 660, 670 and 680.

[0217] Alternatively, the output device can be connected to the system 600 via communication interface 650 using communication channel 651. The display 661 and speaker 671 can be integrated with other components of the system 600 into a single unit in an electronic device, such as a television set.

[0218] In various exemplary embodiments, the display interface 660 may include a display driver, such as, for example, a timing controller (T Con) chip.

[0219] For example, if the RF portion of input 690 is part of a separate set-top box, then display 661 and speaker 671 may optionally be separate from one or more other components. In various exemplary embodiments where display 661 and speaker 671 can be external components, output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0220] exist Figure 1-12This document describes various methods, each comprising one or more steps or actions to implement the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined.

[0221] Examples of block diagrams and / or operation flowcharts are described. Each block represents a portion of circuitry, a module, or code, which includes one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in other implementations, the functions (one or more) marked in the blocks may occur out of order. For example, depending on the functions involved, two blocks shown sequentially may actually execute substantially concurrently, or sometimes these blocks may be executed in reverse order.

[0222] The embodiments and aspects described herein may be implemented in, for example, methods or processes, apparatus, computer programs, data streams, bit streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), implementations of the discussed features may be implemented in other forms (e.g., apparatus or computer programs).

[0223] The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices.

[0224] Furthermore, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having computer-executable computer-readable program code implemented thereon. Considering the inherent ability to store information therein and the inherent ability to retrieve information provided therefrom, the computer-readable storage medium as used herein can be considered a non-transitory storage medium. The computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. It should be understood that while more specific examples of computer-readable storage media to which this exemplary embodiment can be applied are provided below, they are merely illustrative and not exhaustive, as will be readily recognized by those skilled in the art: portable computer floppy disks; hard disks; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable optical disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination of the foregoing.

[0225] Instructions can form applications that are tangibly implemented on processor-readable media.

[0226] For example, instructions can be found in hardware, firmware, software, or a combination thereof. Instructions can be found, for example, in an operating system, a standalone application, or a combination of both. Therefore, a processor can be characterized as, for example, a device configured to execute a process and a device including a processor-readable medium (such as a storage device) having instructions for executing the process. Additionally, in addition to or instead of instructions, the processor-readable medium can store data values ​​generated by the implementation.

[0227] The device can be implemented, for example, in appropriate hardware, software, and firmware. Examples of such devices include personal computers, laptops, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, head-mounted display devices (HMDs, see-through glasses), projectors, "caves" (systems comprising multiple displays), servers, video encoders, video decoders, post-processors that process the output from the video decoder, pre-processors that provide input to the video encoder, web servers, set-top boxes, and any other devices or communication equipment used to process video images. It should be clear that the equipment can be mobile and even mounted in mobile vehicles.

[0228] The computer software may be implemented by the processor 610, by hardware, or by a combination of hardware and software. As a non-limiting example, exemplary embodiments may also be implemented by one or more integrated circuits. The memory 620 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 610 may be of any type suitable for the technical environment and may encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures, as non-limiting examples.

[0229] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information, such as information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, the signal may be formatted to carry a bit stream of the described exemplary embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0230] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, the singular forms “an,” “a,” and “the” may also be intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “include / comprise” and / or “including / comprising” may specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Furthermore, when an element is referred to as “in response to,” “connected to,” or “associated with,” another element, it may be directly responsive to, connected to, or associated with another element, or there may be intermediate elements. In contrast, when an element is referred to as “directly responsive to,” “directly connected to,” or “directly associated with,” another element, there are no intermediate elements.

[0231] It should be recognized that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the symbols / terms “ / ,” “and / or,” and “at least one of” can be intended to cover the selection of only the first listed option (A), or only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to cover the selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or the selection of all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many items as are listed.

[0232] Various numerical values ​​may be used in this application. Specific values ​​may be used for illustrative purposes and the aspects described are not limited to these specific values.

[0233] It will be understood that while the terms first, second, etc., may be used herein to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the teachings of this application, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. There is no implied order between the first element and the second element.

[0234] References to “an exemplary embodiment” or “an exemplary embodiment” or “an implementation” or “implementation” and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with an exemplary embodiment / implementation) is included in at least one exemplary embodiment / implementation. Therefore, the phrases “in an exemplary embodiment” or “in an exemplary embodiment” or “in one implementation” or “in one implementation” appearing throughout this application, as well as any other variations, do not necessarily refer to the same exemplary embodiment.

[0235] Similarly, the references to "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" and their variations are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an exemplary embodiment / example / implementation) may be included in at least one exemplary embodiment / example / implementation. Therefore, the expressions "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" appearing throughout this application do not necessarily refer to the same exemplary embodiment / example / implementation, nor are individual or alternative exemplary embodiments / examples / implementations necessarily mutually exclusive with other exemplary embodiments / examples / implementations.

[0236] The reference numerals appearing in the claims are for illustrative purposes only and do not limit the scope of the claims. Although not explicitly described, these exemplary embodiments / examples and variations may be employed in any combination or subcombination.

[0237] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0238] While some diagrams include arrows along the communication path to indicate the main direction of communication, it should be understood that communication can occur in the opposite direction to the arrows depicted.

[0239] Various implementations involve decoding. As used herein, "decoding" can encompass all or part of a process performed, for example, on a received video frame (which may include a received bitstream encoded with one or more video frames) to produce a final output suitable for display or further processing in a reconstructed video domain. In various exemplary embodiments, such a process includes one or more processes typically performed by a decoder. In various exemplary embodiments, such a process, for example, also includes, or optionally includes, a process performed by a decoder of the various embodiments described herein.

[0240] As a further example, in one exemplary embodiment, "decoding" may refer only to dequantization; in another exemplary embodiment, "decoding" may refer to entropy decoding; in yet another exemplary embodiment, "decoding" may refer only to differential decoding; and in yet another exemplary embodiment, "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. It will be clear, and believed to be well understood by those skilled in the art, whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process, depending on the context of the specific description.

[0241] Various implementations involve encoding. In a manner similar to the above discussion of "decoding," the term "encoding" as used herein can encompass all or part of a process performed on an input video frame to generate an output bitstream. In various exemplary embodiments, such a process includes one or more processes typically performed by an encoder. In various exemplary embodiments, such a process also includes, or optionally includes, a process performed by an encoder of the various embodiments described herein.

[0242] As a further example, in one exemplary embodiment, "encoding" may refer only to quantization; in another exemplary embodiment, "encoding" may refer only to entropy encoding; in yet another exemplary embodiment, "encoding" may refer only to differential encoding; and in still another exemplary embodiment, "encoding" may refer to a combination of quantization, differential encoding, and entropy encoding. It will be clear, and believed to be well understood, by those skilled in the art, whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process, depending on the context of the particular description.

[0243] Furthermore, this application may refer to "obtaining" various types of information. Obtaining information may include one or more of the following: estimated information, calculated information, predicted information, or information retrieved from memory, processed information, moved information, copied information, erased information, calculated information, determined information, predicted information, or estimated information.

[0244] Furthermore, this application may refer to "receiving" various messages. Receiving messages may include one or more of the following, such as access information or receiving information from a communication network.

[0245] Moreover, as used herein, the term "signal transmission" specifically refers to instructing the corresponding decoder to do something. For example, in some exemplary embodiments, the encoder signals specific information, such as encoding / decoding parameters or encoded video frame data. In this way, in exemplary embodiments, the same parameter can be used on both the encoder and decoder sides. Thus, for example, the encoder can transmit (explicit signal transmission) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, then signaling can be used without transmission (implicit signal transmission) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various exemplary embodiments by avoiding the transmission of any actual functionality. It should be recognized that signaling can be accomplished in various ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the term "signal transmission" has been mentioned above, the term "signal transmission" can also be used as a noun herein.

[0246] Several implementations have been described. However, it should be understood that various modifications can be made. For example, elements of different implementations can be combined, supplemented, modified, or removed to produce other implementations. Furthermore, those skilled in the art will understand that other structures and processes can replace the disclosed structures and processes, and the resulting implementations will perform at least substantially the same functions in at least substantially the same manner to achieve at least substantially the same results as the disclosed implementations. Therefore, these and other implementations are contemplated in this application.

Claims

1. A method for encoding raw video frames, comprising: A presampled adaptive offset vertical filter using a first PSAO parameter combination and a presampled adaptive offset horizontal filter using a second PSAO parameter combination, the first PSAO parameter combination and the second PSAO parameter combination being intended to be applied to video frame data. This is achieved by making the incremental cost (ΔJ) i,j Minimize the incremental cost (ΔJ) to jointly select the first PSAO parameter combination and the second PSAO parameter combination. i,j The incremental distortion (ΔD) is defined as the calculated value for a pair of first candidate PSAO parameter combinations and second candidate PSAO parameter combinations. i,j ) and incremental rate (ΔR) i,j A linear combination of ) The incremental distortion (ΔD) i,j ) equals first distortion (D i,j ) and second distortion (D 0,0 The difference between ) The first distortion (D) i,j Y1 is defined as the cumulative difference between the pixels of the reference video frame (X) derived from the original video frame and the pixels of the output video frame (Y5), which is obtained by applying deblocking vertical filtering to the input video frame (Y1), followed by presampling adaptive offset vertical filtering using the first candidate (i) PSAO parameter combination, then deblocking horizontal filtering, and finally presampling adaptive offset horizontal filtering using the second candidate (j) PSAO parameter combination. The second distortion (D) 0,0 ) is defined as the cumulative difference between the pixels of the reference video frame (X) and the pixels of the output video frame, which is obtained after applying the vertical deblocking filter and the horizontal deblocking filter to the input video frame (Y1) without applying the presampled adaptive offset vertical filter and the presampled adaptive offset horizontal filter. and The incremental rate (ΔR) i,j This indicates the number of bits required to encode the following related syntax elements: Presampled adaptive offset vertical filtering using the first candidate PSAO parameter combination; and Presampled adaptive offset level filtering using the second candidate PSAO parameter combination.

2. The method of claim 1, wherein the PSAO parameter includes two offset coefficients, wherein the offset range of the two offset coefficients allows for negative offset coefficients.

3. The method according to any one of claims 1 to 2, wherein the deblocking parameters used for deblocking vertical filtering or deblocking horizontal filtering are selected by minimizing cost, the cost being defined as the distortion between the reference video frame (X) and the output video frame of the deblocking vertical filter or deblocking horizontal filter using the candidate deblocking parameters.

4. The method of claim 3, wherein the deblocking parameters used for deblocking vertical filtering or deblocking horizontal filtering are selected without considering presampling adaptive offset vertical filtering and presampling adaptive offset horizontal filtering.

5. The method of claim 3, wherein the deblocking parameters used to jointly select the deblocking vertical filter or the deblocking horizontal filter are combined with the first PSAO parameter combination and the second PSAO parameter combination.

6. The method according to claim 5, wherein the input video frame (Y1) includes a luminance component and two chrominance components, and a first joint selection of the deblocking parameters used for deblocking vertical filtering or deblocking horizontal filtering with the first PSAO parameter combination and the second PSAO parameter combination is used for the luminance component of the input video frame (Y1), and a second joint selection of the deblocking parameters used for deblocking vertical filtering or deblocking horizontal filtering with the first PSAO parameter combination and the second PSAO parameter combination is used for the chrominance component of the input video frame (Y1).

7. The method according to any one of claims 3 to 6, wherein the deblocking parameters used for vertical deblocking filtering are the same as those used for horizontal deblocking filtering.

8. The method according to any one of claims 1 to 7, wherein the reference video frame (X) is the original video frame.

9. The method according to any one of claims 1 to 8, wherein the reference video frame (X) is obtained by motion-compensated time filtering.

10. An apparatus comprising components for performing any one of the methods as claimed in any one of claims 1 to 9.

11. A computer program product comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method as described in any one of claims 1 to 9.

12. A non-transitory storage medium carrying instructions for performing program code as described in any one of claims 1 to 9.

Images