Method and apparatus for adaptive loop filter with additional modes and taps related to cross-component chroma model and fixed filter in video coding

Abstract

A method and apparatus for loop filtering of reconstructed pictures are disclosed. According to the method, a reconstructed pixel is received, wherein the reconstructed pixel includes a current block, and the current block includes a first color block and a second color block. A target fixed filter set is selected from a plurality of fixed filter sets for filtering the current block using a target filter type of a filter type group based on a filter selection indication at a coding tree block (CTB) level, the filter type group including a chroma adaptive loop filter (ALF), a cross-component ALF (CCALF), or both. The target fixed filter set is applied to the current block to generate a filtered current block. The filtered current block is provided.

Description

[0001] Cross-referencing This invention is a non-provisional application and claims priority to U.S. Provisional Patent Application No. 63 / 621,199, filed January 16, 2024. The entire contents of this U.S. Provisional Patent Application are incorporated herein by reference. Technical Field

[0002] This invention relates to a video encoding / decoding system using Adaptive Loop Filters (ALFs). Specifically, the invention discloses a method and apparatus for improving performance by selecting a fixed set of filters for chroma ALFs and / or cross-component ALFs (CCALFs) based on instructions at the Coding Tree Block (CTB) level. Background Technology

[0003] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET), comprised of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). This standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology – Encoding representation of immersive media – Part 3: Versatile Video Coding, published in February 2021. VVC is developed based on its predecessor, High Efficiency Video Coding (HEVC), by adding more coding and decoding tools to improve coding and decoding efficiency and handle various types of video sources, including 3D video signals.

[0004] Figure 1AAn exemplary adaptive inter-frame / intra-frame movie coding system incorporating loop processing is described. For intra-frame prediction, the prediction data is derived from previously encoded movie data in the current frame. For inter-frame prediction 112, motion estimation (ME) is performed at the encoder, and motion compensation (MC) is performed based on the result of ME to provide prediction data derived from other frames and motion data. Switch 114 selects either intra-frame prediction 110 or inter-frame prediction 112 and provides the selected prediction data to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118 and then quantized (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 and included in the movie bitstream corresponding to the compressed movie data. The bitstream associated with the transform coefficients is then packaged with additional information, such as motion and encoding / decoding modes associated with intra-frame and inter-frame prediction, and parameters associated with loop filters applied to the base image regions. Additional information related to intra-frame prediction 110, inter-frame prediction 112, and loop filter 130, such as Figure 1A As shown, this is provided to the entropy encoder 122. When using inter-frame prediction mode, the reference picture or multiple reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residuals are processed by inverse quantization (IQ) 124 and inverse transformation (IT) 126 to recover the residuals. The residuals are then added to the predicted data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in the reference picture buffer (Ref. Pic. Buffer) 134 and used for prediction of other frames.

[0005] like Figure 1AAs shown, the input video data undergoes a series of processes in the encoding system. The reconstructed video data from REC 128 may be subject to various degradations due to these processes. Therefore, an in-loop filter module (ILPF) 130 is typically applied to the reconstructed video data before storing it in the reference picture buffer 134 to improve video quality. For example, a deblocking filter (DF), Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF) can be used. Loop filter information may need to be incorporated into the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to the entropy encoder 122 for inclusion in the bitstream. Figure 1A In the process, the loop filter 130 is applied to reconstruct the video, and then the reconstructed samples are stored in the reference image buffer 134. Figure 1A The system described herein is intended to illustrate an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Codec (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0006] like Figure 1B As shown, the decoder can use the same or partially the same functional modules as the encoder, except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. The decoder uses an entropy decoder (ED) 140 to decode the movie bitstream into quantized transform coefficients and the required encoding / decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 at the decoder end does not need to perform mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from the entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from the entropy decoder 140, without performing motion estimation.

[0007] According to VVC, the input image is divided into non-overlapping square block regions called Coding Tree Units (CTUs), similar to HEVC. Each CTU can be further divided into one or more smaller Coding Units (CUs). The resulting CU partitions can be square or rectangular in shape. Furthermore, VVC divides the CTUs into Prediction Units (PUs) as units for applying prediction processes such as inter-frame prediction, intra-frame prediction, etc.

[0008] This invention discloses a method and apparatus for improving the performance of an adaptive loop filter (ALF). Summary of the Invention

[0009] A method and apparatus for loop filtering for reconstructing a video are disclosed. According to the method, reconstructed pixels are received, wherein the reconstructed pixels include a current block, and the current block includes a first color block and a second color block. Based on a filter selection instruction at the Coding Tree Block (CTB) level, a target fixed filter set is selected from a plurality of fixed filter sets for filtering the current block using a target filter type from a filter type group, which includes an Adaptive Loop Filter (ALF), a Cross-Component ALF (CCALF), or both. The target fixed filter set is applied to the current block to generate a filtered current block. The filtered current block is then provided.

[0010] In one embodiment, samples generated from a cross-component model and / or samples generated from a cross-component model with fixed filtering are selected at the CTB level for the luma ALF, the chroma ALF, the CCALF, or a combination thereof.

[0011] In one embodiment, multiple Adaptation Parameter Sets (APSs) are selected at the image level, tile level, slice level, or a combination thereof, and a target APS is selected from the multiple APSs at the CTB level for the step of selecting the target fixed filter set.

[0012] In one embodiment, a flag is signaled or resolved at the image layer, tile layer, slice layer, or a combination thereof to control the use of fixed filtering at that CTB layer, and the use of fixed filtering is for a filter type group, which further includes a luma ALF. In one embodiment, the flag controls whether the selection of a fixed filter set is enabled at the CTB layer. In another embodiment, the flag indicates that only one fixed filter set is selected. In another embodiment, the flag enables the selection of an Adaptation Parameter Set (APS) ID and an APS filter set at the CTB layer. In yet another embodiment, the flag indicates that only one APS ID and an APS filter set are selected.

[0013] In one embodiment, a cross-component model is selected as the target filter type. In another embodiment, samples generated by the cross-component model are directly used as the current block after filtering. Attached Figure Description

[0014] Figure 1A An exemplary adaptive Inter / Intra video codec system incorporating loop processing is demonstrated.

[0015] Figure 1B Showing Figure 1A The corresponding decoder for the encoder.

[0016] Figure 2 The ALF filter shapes for the chroma (left) and luminance (right) components are shown.

[0017] Figures 3A-3D The subsampled Laplace calculations for gv (3A), gh (3B), gd1 (3C), and gd2 (3D) are shown.

[0018] Figure 4A This shows the position of CC-ALF relative to other loop filters.

[0019] Figure 4B A diamond filter for chromaticity samples is shown.

[0020] Figure 5 An example of the spatial portion of a convolutional filter is shown.

[0021] Figure 6 A reference region for deriving filter coefficients is shown, which uses padding.

[0022] Figure 7 A flowchart of an exemplary video encoding / decoding system is shown, which selects a fixed set of filters for chroma ALF and / or CCALF according to an instruction at the CTB level, disclosed according to an embodiment of the present invention. Detailed Implementation

[0023] The components of this invention, as generally described herein and illustrated in the figures, can be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of this invention is merely a representative illustration of selected embodiments of the invention and is not intended to limit the scope of the claimed invention.

[0024] The terms "a particular embodiment," "an embodiment," or similar language used in this specification indicate that a specific feature, structure, or characteristic associated with that embodiment may be included in at least one embodiment of the invention. Therefore, when the phrases "in a particular embodiment" or "in an embodiment" appear in different places in this specification, they do not necessarily refer to the same embodiment.

[0025] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the invention can be practiced without including one or more specific details, or other methods, components, etc., can be used. In some cases, well-known structures or operations are not shown or described in detail to avoid obscuring certain aspects of the invention. The illustrative embodiments of the invention will be best understood by referring to the accompanying drawings, wherein like parts are labeled with the same numerals throughout the drawings. The following description is merely illustrative and illustrates only certain selected apparatus and method embodiments consistent with the invention claimed herein.

[0026] Adaptive Loop Filter (ALF) in VVC In VVC, the Adaptive Loop Filter (ALF) is applied using a block-based filter adaptation technique. For the luminance component, a filter is selected from 25 filters for each 4×4 block based on the direction and activity of the local gradient.

[0027] 1. Filter shape Two diamond filter shapes were used (e.g.) Figure 2 (As shown). A 7×7 rhombus shape 220 is used for the luminance component, while a 5×5 rhombus shape 210 is used for the chrominance component.

[0028] 2. Block Classification For the luminance component, each 4×4 block is classified into one of 25 classes. The classification index C is a quantized value based on its directionality D and activity. Derivatives are shown below: To calculate D and First, the gradients in the horizontal, vertical, and two diagonal directions are calculated using the one-dimensional Laplacian operator: Here, indices i and j refer to the coordinates of the top-left sample within the 4×4 block, and R(i,j) represents the reconstructed sample at coordinates (i,j).

[0029] To reduce the complexity of block classification, the vertical direction ( Figure 3A ) and horizontal direction ( Figure 3B The one-dimensional Laplace calculation is applied using subsampling. For example... Figure 3C As shown in -D, the same subsampling position is used for gradient calculation in all directions. Figure 3C In and Figure 3D In ).

[0030] Then, the maximum and minimum values ​​of the gradients in the horizontal and vertical directions of D are set as follows: , The maximum and minimum values ​​of the gradients in the two diagonal directions are set as follows: , .

[0031] In order to derive the directional value These values ​​are compared with each other and with two thresholds. and Comparison: Step 1. If and If all are true, then Set as .

[0032] Step 2. If If yes, continue to step 3; otherwise, continue to step 4.

[0033] Step 3. If ,but Set as ;otherwise Set as .

[0034] Step 4. If ,but Set as ;otherwise Set as .

[0035] Activity Value The calculation is as follows: Further quantization is performed to the range of 0 to 4 (inclusive), and the quantized value is represented as... .

[0036] The chromaticity components in the image are not classified.

[0037] 3. Geometric transformation of filter coefficients and clipping values Before filtering each 4×4 luminance block, geometric transformations, such as rotation or diagonal and vertical flipping, are applied to the filter coefficients f(k,l) and the corresponding filter clipping values ​​c(k,l) based on the gradient values ​​calculated for that block. This is equivalent to applying these transformations to samples within the filter support region. The aim is to make the different blocks to which ALF is applied more similar by aligning their orientations.

[0038] Three geometric transformations are introduced: diagonal flip, vertical flip, and rotation. Diagonal flip: Vertical flip: , , Rotation: , , in It is the size of the filter. These are coefficient coordinates, which make the position... Located in the top left corner, position Located in the bottom right corner. Based on the gradient values ​​calculated for this block, the transformation is applied to the filter coefficients f(k, l) and the clipping value. The relationship between the transformation and the four gradients in the four directions is summarized in the following table.

[0039] Table 1. Mapping of gradients and transformations computed for a block 4. Filtering process On the decoder side, when ALF is enabled in CTB, each sample in CU All will be filtered, resulting in sample values. As shown below: in Indicates the filter coefficients used for decoding. It is a clipping function. This represents the pruning parameters for decoding. The variables k and l range from -L / 2 to L / 2, where L represents the filter length. Pruning function. Corresponding to function The pruning operation introduces non-linearity, making ALF more efficient by reducing the influence of neighboring sample values ​​that differ too much from the current sample value.

[0040] 5. Cross-component adaptive loop filter CC-ALF refines each chroma component using luminance sample values ​​by applying an adaptive linear filter to the luminance channel and then using the output of that filter operation for chroma refinement. Figure 4A A system hierarchy diagram of the CC-ALF process relative to the SAO, luminance ALF, and chrominance ALF processes is provided. Figure 4A As shown, each color component (i.e., Y, Cb, and Cr) is processed by its corresponding SAO (i.e., SAO Luma 410, SAO Cb 412, and SAO Cr 414). After the SAO, ALF Luma 420 is applied to the luminance of the SAO-processed luminance, while ALF Chroma 430 is applied to the Cb and Cr of the SAO-processed chrominance. However, there are cross-component terms from luminance to chrominance components (i.e., CC-ALF Cb 422 and CC-ALF Cr 424). The outputs of the cross-component ALFs are added to the output of ALF Chroma 430 via adders 432 and 434, respectively.

[0041] Filtering in CC-ALF is achieved by applying a linear, diamond-shaped filter (e.g., ...) to the luminance channel. Figure 4B This is accomplished using filters 440 and 442. Figure 4B In the diagram, blank circles represent luminance samples, and dot-filled circles represent chrominance samples. Each chrominance channel uses one filter, and the operation is expressed as follows: in This is the position of the chroma component i being refined. Based on The brightness position, It is the filter support area in the brightness component. This represents the filter coefficients.

[0042] like Figure 4B As shown, the luminance filter support region is the region co-located with the current chrominance sample, taking into account the spatial scaling factor between the luminance and chrominance planes. Figure 4BIn the diagram, circles represent luminance samples, while dot-filled circles represent chrominance samples being refined.

[0043] In the VVC reference software, CC-ALF filter coefficients are calculated by minimizing the mean squared error of each chroma channel relative to the original chroma content. To achieve this, the VVC Test Model (VTM) algorithm uses a coefficient derivation process similar to that of chroma ALF. Specifically, a correlation matrix is ​​derived, and the coefficients are calculated using a Cholesky decomposition solver, attempting to minimize the mean squared error metric. A maximum of eight CC-ALF filters can be designed and transmitted per image during filter design. The resulting filters are then indicated for two chroma channels based on the CTU.

[0044] Additional features of CC-ALF include: The design uses a 3x4 diamond shape with 8 points.

[0045] Seven filter coefficients are transmitted in the APS.

[0046] Each transmitted coefficient has a 6-bit dynamic range and is limited to powers of 2.

[0047] The eighth filter coefficient is derived in the decoder, making the sum of the filter coefficients equal to 0.

[0048] APS can be referenced in the slice header.

[0049] The CC-ALF filter selection is controlled at the CTU level for each chromaticity component.

[0050] The horizontal virtual boundary padding uses the same memory access mode as the brightness ALF.

[0051] As an additional feature, the reference encoder can be configured to enable some basic subjective adjustments via a configuration file. When enabled, the VTM reduces the application of CC-ALF in high-QP encoded regions, which are either close to mid-gray or contain a large number of high-frequency luminance areas. Algorithmically, this is achieved by disabling the application of CC-ALF in the CTU if any of the following conditions are met: The slice QP value minus 1 is less than or equal to the base QP value.

[0052] The number of chroma samples with a local contrast greater than (1 << (bitDepth – 2)) – 1 exceeds the CTU height, where local contrast is the difference between the maximum and minimum luminance sample values ​​within the filter's supported region.

[0053] More than a quarter of the chroma samples fall within the range of (1<<(bitDepth – 1)) – 16 and (1<<(bitDepth – 1)) + 16. The motivation behind this feature is to provide assurance that CC-ALF will not amplify early artifacts introduced into the decoding path (primarily because VTM does not currently explicitly optimize for subjective chroma quality). It is anticipated that alternative encoder implementations may not use this feature or will incorporate alternative strategies suited to their encoding characteristics.

[0054] 6. Filter parameter signaling ALF filter parameters are signalalled in the Adaptation Parameter Set (APS). A maximum of 25 sets of luma filter coefficients and clipping value indices, and a maximum of eight sets of chroma filter coefficients and clipping value indices can be signalalled in an APS. To reduce bit overhead, filter coefficients from different categories of the luma component can be merged. In the slice header, the signaling is used for indexing the APS of the current slice.

[0055] The clipping value index decoded from the APS allows for the determination of clipping values ​​for the luma and chroma components using a clipping value table. These clipping values ​​depend on the internal bit depth. More precisely, the clipping values ​​are obtained using the following formula: AlfClip Where B equals the internal bit depth, α is a predefined constant equal to 2.35, and N equals 4, which is the number of clipping values ​​allowed in VVC. Then AlfClip is rounded to the nearest power of 2.

[0056] In the slice header, up to seven APS indices can be signaled to specify the luma filter set used for the current slice. The filtering process can be further controlled at the CTB level. A flag is always signaled to indicate whether ALF is applied to the luma CTB. The luma CTB can select a filter set between 16 fixed filter sets and filter sets in the APS. A filter set index is signaled for the luma CTB to indicate which filter set to apply. The 16 fixed filter sets are predefined and hard-coded in the encoder and decoder.

[0057] For the chroma component, an APS index is signaled in the slice header to indicate the chroma filter set used for the current slice. At the CTB level, if there are multiple chroma filter sets in the APS, a filter index is signaled for each chroma CTB.

[0058] The filter coefficients are quantized using a 128-dimensional standard. To limit multiplication complexity, bitstream consistency is applied, ensuring that coefficients at non-center positions are within -2... 7 to 2 7 The range is -1, including. The coefficient at the center position is not signaled in the bitstream and is considered equal to 128.

[0059] Adaptive loop filters in ECM In ECM8 (Muhammed Coban et al., “Algorithmic Description of Enhanced Compression Model 8 (ECM 8),” Joint Film Experts Group (JVET) of the International Telecommunication Union ITU-T SG 16 WP 3 and the International Organization for Standardization ISO / IEC JTC 1 / SC 29, 29th Meeting, via teleconference, January 11-20, 2023, document: JVET-AC2025), some changes from VVC ALF were disclosed. A brief overview follows.

[0060] 1. Simplified removal of ALF ALF gradient subsampling and ALF virtual boundary processing have been removed. The block size for classification has been reduced from 4x4 to 2x2. The filter sizes for signaling luminance and chrominance have been increased to 9x9.

[0061] 2. Fixed Filter ALF To filter brightness samples, three different classifiers (C0, C1, and C2) and three different filter sets (F0, F1, and F2) were used. Sets F0 and F1 contain fixed filters whose coefficients are trained on classifiers C0 and C1. The coefficients of the filters in F2 are signaled. For a given sample, the classifiers are used... Assigned categories Decide from set F i Which filter should be used?

[0062] 3. Filtering First, two fixed filters, F0 and F1, in 13x13 diamond shape are applied to derive two intermediate samples. and Then, apply F2. , And neighboring samples to derive the filtered samples, as shown below: in It is the neighboring samples and the current sample The difference in cutting between them, yes The cropping difference between the current sample and the current sample. Filter coefficients. It's a signaling signal.

[0063] 4. Classification Based on directionality and activity Assign a category to each 2x2 block. : , in Indicates directionality The total number of gradients. As in VVC, a one-dimensional Laplacian operator (1-DLaplacian) is used to compute the horizontal, vertical, and two diagonal gradient values ​​for each sample. The sum of gradients of samples within a 4×4 window covered by the target 2x2 block is used for classifier C0, while the sum of gradients of samples within a 12×12 window is used for classifiers C1 and C2. The sums of the horizontal, vertical, and two diagonal gradients are expressed as follows: , , and Directionality This was determined by comparing the following: Use a set of thresholds. Directionality This is derived from the thresholds 2 and 4.5 in VVC. For and First, calculate the horizontal / vertical edge strength. and diagonal edge strength Using thresholds .if Then the edge strength =0; otherwise, Is to make The largest integer that is true. If Then the edge strength =0; otherwise, Is to make The largest integer that is true. When When horizontal / vertical edges dominate, use Table 2A for derivation. Otherwise, the diagonal edges dominate, and the derivation can be done using Table 2B. .

[0064] Table 2A. and arrive mapping Table 2B. and arrive mapping In order to obtain Sum of vertical and horizontal gradients Mapping to 0 The range, of which for The value is 4, for and It is 15.

[0065] In ALF_APS, there can be a maximum of 4 sets of luminance filters, and each set can contain up to 25 filters.

[0066] Convolutional Cross-Component Model (CCCM) The convolutional 7-tap filter consists of a 5-tap plus-shaped spatial component, a nonlinear term, and a bias term. The input to the filter's spatial 5-tap component includes the center (C) luminance sample co-located with the chrominance sample to be predicted, and its above / north (N), below / south (S), left / west (W), and right / east (E) neighbors, such as... Figure 5 As shown.

[0067] The non-linear term (denoted as P) is represented as the square of the center brightness sample C, scaled to the range of sample values ​​for the content: P = ( C*C + midVal )>>bitDepth. In other words, for 10-bit content, the calculation method is as follows: P = (C*C + 512) >> 10 The bias term (denoted as B) represents the scalar offset between the input and output (similar to the offset term in CCLM) and is set to an intermediate chroma value (512 for 10-bit content).

[0068] The filter output is obtained through the filter coefficients c. i The result is obtained by convolving with the input value and then cropping it to the range of valid chromaticity samples: predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B 1. Calculation of filter coefficients The filter coefficients ci are calculated by minimizing the mean square error (MSE) between the predicted chromaticity sample and the reconstructed chromaticity sample within the reference region. Figure 6 The reference area is illustrated, consisting of six rows of chroma samples above and to the left of the prediction unit (PU). The reference area extends one PU width to the right and one PU height downwards, beyond the PU boundary. This area is adjusted to include only usable samples. The area expansion shown in blue is to support the "side samples" of the "plus-shaped spatial filter" and to fill in unusable areas.

[0069] MSE minimization is performed by calculating the autocorrelation matrix of the luminance input and the cross-correlation vector between the luminance input and the chrominance output. The autocorrelation matrix is ​​decomposed using LDL, and the final filter coefficients are calculated by back-substitution. This process roughly follows the calculation of the ALF filter coefficients in ECM, but LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations.

[0070] The autocorrelation matrix is ​​calculated using reconstructed values ​​from luma and chromaticity samples. These samples are full-range (e.g., between 0 and 1023 for 10-bit content), resulting in relatively large values ​​in the autocorrelation matrix. This necessitates high-bit-depth operations during model parameter calculations. It is recommended to remove fixed offsets from the luma and chromaticity samples in each PU of each model. This reduces the magnitude of values ​​used in model creation and allows for a reduction in the precision required for fixed-point operations. Therefore, 16-bit decimal precision is recommended instead of the 22-bit precision used in the original CCCM implementation.

[0071] Reference sample values ​​outside the top-left corner of the PU are used as offsets (offsetLuma, offsetCb, and offsetCr) to simplify operations. The sample values ​​used in model creation and final prediction (i.e., luminance and chrominance in the reference region, and luminance in the current PU) are subtracted from these fixed values ​​by: C' = C – offsetLuma N' = N – offsetLuma S' = S – offsetLuma E' = E – offsetLuma W' = W – offsetLuma P' = nonLinear(C') B = midValue = 1<<(bitDepth - 1) Chromaticity values ​​are predicted using the following formula, where offsetChroma is equal to the offsetCr and offsetCb of the Cr and Cb components, respectively: predChromaVal = c0C' + c1N' + c2S' + c3E' + c4W' + c5P' + c6B +offsetChroma. To avoid any additional sample-level operations, the luminance offset is removed during luminance reference sample interpolation. For example, this can be done by replacing the rounding term used in luminance reference sample interpolation with an updated offset that includes rounding terms and offsetLuma. The chrominance offset can be removed by directly subtracting it from the reference chrominance sample. Alternatively, the effect of the chrominance offset can be removed from the cross-component vector, achieving the same result. To add the chrominance offset back to the output of the convolutional prediction operation, it is added to the bias term of the convolutional model.

[0072] The CCCM model parameter calculation process requires division operations. Division operations are not always considered implementation-friendly. Therefore, division operations are replaced with multiplication (with a scaling factor) and shift operations, where the scaling factor and the number of shifts are calculated based on the denominator, similar to the method used in CCLM parameter calculations.

[0073] Recommended methods Filter taps associated with cross-component models and fixed filters In one embodiment, to perform a chromatic ALF filtering process with signaling coefficients using the fixed filter results and the chromatic samples generated by applying a cross-component model to the luminance samples, the following steps can be applied (i.e., applying a cross-component model to the fixed filter results): - Classification using a fixed-filter classifier - Filtering is performed using a fixed set of filter coefficients, where the fixed set of filter coefficients is not signaled in the bitstream. - Derive the luminance-to-chrominance cross-component model coefficients using the luminance-fixed filter results from the neighboring reference region and / or the current block and the chrominance-fixed filter results. - By applying a cross-component model from luminance to chrominance to the luminance-fixed filter results of the current block, the chrominance samples of the current block are derived. - A chromatic filtering process is performed using signaling coefficients, combined with chromaticity samples and chromaticity samples from the cross-component model.

[0074] In the above embodiments, the order of steps can be changed (e.g., applying a fixed filter to samples from a cross-component model): - Derive the cross-component model coefficients for luminance to chrominance using reconstructed luminance and chrominance samples from neighboring reference regions and / or the current block. - By applying a cross-component model from luminance to chrominance to the luminance sample of the current block, the chrominance sample of the current block is derived. - Classify chromaticity samples from the cross-component model using a fixed-filter classifier. - Filtering of chroma samples from a cross-component model using a fixed set of filter coefficients, where the fixed set of filter coefficients is not signaled in the bitstream. - A chromatic filtering process is performed using signaling coefficients, combined with chromaticity samples and fixed-filter chromaticity samples from a cross-component model.

[0075] In the above embodiments, the step "classification using a fixed filter classifier" can be one of the following or a combination thereof: - Applying a fixed-filter classifier to chromaticity samples - Applying a fixed-filter classifier to brightness samples The fixed filter classifier is the same as the brightness fixed filter classifier. The fixed filter classifier is a subset of the luminance fixed filter classifier, where "subset of luminance fixed filter classifier" refers to a classifier with fewer categories, less directionality, and / or less activity compared to the luminance fixed filter classifier.

[0076] The fixed filter classifier is different from the brightness fixed filter classifier.

[0077] In the above embodiments, in the step of "filtering using a fixed set of filter coefficients, wherein the fixed set of filter coefficients is not signaled in the bitstream", the fixed set of filter coefficients can be the following or a combination of the following: The set of fixed filter coefficients is the same as the set of fixed brightness filter coefficients.

[0078] The set of fixed filter coefficients is a subset of the set of fixed brightness filter coefficients.

[0079] The set of fixed filter coefficients is different from the set of fixed brightness filter coefficients.

[0080] When filtering chroma samples using a fixed set of filter coefficients, the filter selection is determined by one or a combination of the following: Classification results of chromaticity fixed filter.

[0081] The corresponding classification results for fixed brightness filters.

[0082] In the above embodiments, a chroma-to-luminance cross-component model can also be applied. This means that by using luminance samples, the cross-component model can be applied to chroma samples for the filtering process of the Adaptive Loop Filter (ALF) or Cross-Component ALF (CCALF), and using the signaling coefficients, the following steps can be performed: Classification is performed using a fixed-filter classifier.

[0083] Filtering is performed using a fixed set of filter coefficients, where the fixed set of filter coefficients is not signaled in the bitstream.

[0084] The coefficients of the cross-component model from chroma to luminance are derived using the chroma-fixed filter results and luminance-fixed filter results from the neighboring reference region and / or the current block.

[0085] The luminance sample of the current block is derived by applying the chroma-fixed filter result to the cross-component model of chroma to luminance for the current block.

[0086] Using the coefficients of the signaling, a luminance ALF or CCALF filtering process is performed using luminance samples and luminance samples from the cross-component model.

[0087] In the above embodiments, the reconstructed luminance and chromaticity samples of the neighboring reference region used to derive the cross-component model coefficients can be samples before or after the application of the ALF filtering process.

[0088] In the above embodiments, the reconstructed luminance or chromaticity sample of the current block used to derive the cross-component model sample can be a sample before or after the application of the ALF filtering process.

[0089] In the above embodiments, the step of "deriving the coefficients of the cross-component model from luminance to chrominance using the luminance fixed filter results and chrominance fixed filter results of the neighboring reference region and / or the current block" can be the following or a combination of the following: The coefficients of the cross-component model of luminance to chrominance are derived using the luminance-fixed filter results from the neighboring reference region and / or the current block, and the reconstructed chrominance samples.

[0090] The coefficients of the cross-component model for luminance to chrominance are derived using reconstructed luminance samples and chrominance fixed filter results from neighboring reference regions and / or the current block.

[0091] In the above embodiments, the step of "deriving the chromaticity sample of the current block by applying the luminance-fixed filter result to the cross-component model of luminance to chromaticity of the current block" can be the following or a combination of the following: The chromaticity sample of the current block is derived by applying a cross-component model from luminance to chrominance to the reconstructed luminance sample of the current block.

[0092] ALF Additional Mode In ECM ALF, the syntactic design for slices and CTB levels is shown below. Several new syntactic designs for slices and CTB levels are illustrated in the following sections: Luminance ALF: Slice level: On / Off, number of APS or APS IDs (up to 7).

[0093] CTB level: Off, fixed filter set (choose 1 from 2) or APS ID and APS filter set.

[0094] Chromaticity ALF: Slice level: On / Off (Cb and Cr are separate), 1 APS ID (Cb and Cr are shared).

[0095] CTB level: Off, APS filter set (maximum 1) (Cb and Cr are separate).

[0096] CCALF: Slice level: On / Off (Cb and Cr are separated), 2 APS IDs (Cb and Cr are separated).

[0097] CTB level: Off, APS filter set (maximum 1) (Cb and Cr are separate).

[0098] In one embodiment, a fixed set of filters can be selected for the chroma ALF and / or CCALF at the CTB level.

[0099] In one embodiment, multiple APSs can be selected at the image / tile / slice level, and one APS can be selected from these APSs at the CTB level for chroma ALF and / or CCALF.

[0100] In one embodiment, samples from a cross-component model and / or samples from a cross-component model with fixed filtering can be selected at the CTB level for the luma ALF, chroma ALF, and / or CCALF.

[0101] In one embodiment, for the luma ALF, chroma ALF, and / or CCALF, a flag can be signaled at the picture / tile / slice level to control whether a fixed filter set can be selected at the CTB level. Therefore, if the flag indicates that a fixed filter set can be selected at the CTB level, both the fixed filter set and the APS filter set can be selected at the CTB level. Otherwise, only the APS filter set can be selected at the CTB level.

[0102] In one embodiment, for the luma ALF, chroma ALF, and / or CCALF, a flag can be signaled at the picture / tile / slice level to indicate that a fixed filter set can be selected only at the CTB level. Therefore, if the flag indicates that a fixed filter set can be selected only at the CTB level, then a fixed filter set can only be selected at the CTB level. Otherwise, both the fixed filter set and the APS filter set can be selected at the CTB level.

[0103] In one embodiment, for the luma ALF, chroma ALF, and / or CCALF, a flag can be signaled at the picture / tile / slice level to control whether the APS ID and APS filter set can be selected at the CTB level. Therefore, if the flag indicates that the APS ID and APS filter set can be selected at the CTB level, then a fixed filter set and an APS filter set can be selected at the CTB level. Otherwise, only the fixed filter set can be selected at the CTB level.

[0104] In one embodiment, for the luma adaptive loop filter (luma ALF), chroma adaptive loop filter (chroma ALF), and / or cross-component adaptive loop filter (CCALF), a flag can be signaled at the picture level, tile level, or slice level to allow selection of the Adaptation Parameter Set (APS) ID and APS filter set only at the Coding Tree Block (CTB) level. Therefore, if the flag indicates that the APS ID and APS filter set can only be selected at the CTB level, then the APS ID and APS filter set can only be selected at the CTB level. Otherwise, both the fixed filter set and the APS filter set can be selected at the CTB level.

[0105] In one embodiment, for the luma ALF, chroma ALF, and / or CCALF, a set of APS filters can be used at the picture level, tile level, or slice level to signal from different APSs, so that these APS filter sets can only be selected at the CTB level.

[0106] Example: There are two APSs, APS0 and APS1. For APS0, there are three filter sets. For APS1, there are two filter sets, APS0 = {0, 1, 2} and APS1 = {0, 1}. The following shows an example of two slice selections: Example 1: Slice selection selects only filter sets 0 and 2 in APS0, and filter set 0 in APS1. Then, at the CTB level, only these filter sets can be used (i.e., APS0' = {0, 2}, APS1' = {0}).

[0107] Example 2: Slice selection selects only filter sets 0 and 2 in APS0, and filter set 0 in APS1. A list containing all filter sets is formed. Then, at the CTB level, only these filter sets can be used (i.e., APS' = {APS0-0, APS1-0, APS0-2}).

[0108] In one embodiment, the cross-component model described in the foregoing sections can be selected at the CTB level for chroma ALF and / or CCALF. That is, samples generated by the cross-component model can be directly used as the output of ALF.

[0109] The proposed methods described above can be implemented in the encoder and / or decoder. For example, the proposed methods can be implemented in the loop filtering module of the encoder, and / or in the loop filtering module of the decoder.

[0110] Any of the methods described above can be implemented in the encoder and / or decoder. Furthermore, any method of the aforementioned unified classification process can also be implemented in the encoder and / or decoder. For example, any proposed method can be implemented in the loop filter module of the encoder or decoder (e.g., Figure 1A and Figure 1BThis can be implemented in ILPF 130. Alternatively, any proposed approach can be implemented as a circuit coupled to an internal module of the codec module, and / or a motion compensation module, or a merge candidate derivation module of the decoder. A simplified ALF approach can also be implemented using executable software or firmware code stored on media (e.g., hard disk or flash memory) for use by a central processing unit (CPU) or a programmable device (e.g., a digital signal processor (DSP) or field programmable gate array (FPGA)).

[0111] Figure 7 A flowchart of an exemplary video codec system is shown, which selects a fixed set of filters for chroma ALF and / or CCALF according to an instruction at the CTB (Codec Tree Block) level, disclosed according to an embodiment of the invention. The steps shown in the flowchart can be executed as program code at the encoder end of one or more processors (e.g., one or more CPUs). The steps shown in the flowchart can also be implemented in hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 710, reconstructed pixels are received, wherein the reconstructed pixels include a current block, and the current block includes a first color block and a second color block. In step 720, a target fixed set of filters is selected from a plurality of fixed filter sets according to a filter selection instruction at the CTB (Codec Tree Block) level, for filtering the current block using a target filter type from a filter type group, which includes chroma ALF (Adaptive Loop Filter), CCALF (Cross-Component ALF), or both. In step 730, the target fixed set of filters is applied to the current block to generate a filtered current block. In step 740, the filtered current block is provided.

[0112] The flowchart shown is intended to illustrate an example of video encoding / decoding according to the present invention. Those skilled in the art can modify each step, rearrange the steps, split the steps, or merge the steps to implement the invention without departing from its spirit. Specific syntax and semantics are used in this disclosure to illustrate examples of implementing embodiments of the invention. Those skilled in the art can implement the invention by substituting equivalent syntax and semantics without departing from its spirit. The foregoing description is intended to enable those skilled in the art to practice the invention according to specific applications and their needs. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the invention is not intended to be limited to the specific embodiments shown and described, but is to be given the broadest scope in accordance with the principles and novel features disclosed herein. In the foregoing detailed description, various specific details are shown to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention can be practiced without departing from its spirit.

[0113] As described above, embodiments of the present invention can be implemented in various hardware, software code, or combinations thereof. For example, embodiments of the present invention may involve integrating one or more circuits into a video compression chip, or integrating program code into video compression software to perform the processes described herein. Embodiments of the present invention may also be program code executed on a digital signal processor (DSP) to perform the processes described herein. The present invention may also relate to multiple functions executed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors can be configured to execute machine-readable software code or firmware code to define the specific methods embodied in the present invention. This software code or firmware code can be developed using different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, different code formats, styles, and languages ​​for performing tasks according to the present invention, as well as other ways of configuring the code, do not depart from the spirit and scope of the present invention.

[0114] This invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The examples described are, in all respects, illustrative only and not limiting. Therefore, the scope of this invention is indicated by the appended claims, and not by the foregoing description. All modifications within the meaning and equivalence of the claims are included within its scope.

Claims

1. A method for processing color images or videos, characterized in that, The method includes: Receive reconstructed pixels, wherein the reconstructed pixels include the current block, and the current block includes a first color block and a second color block; Based on the filter selection instruction at the Coding Tree Block (CTB) level, a target fixed filter set is selected from multiple fixed filter sets for filtering the current block using a target filter type in a filter type group, which includes an Adaptive Loop Filter (ALF), a Cross-Component ALF (CCALF), or both. Apply the target fixed filter set to the current block to generate the filtered current block; and Provides the current block after the filtering.

2. The method as described in claim 1, characterized in that, The CTB level selects samples generated from the cross-component model, and / or samples generated from the cross-component model with fixed filtering, for the luma ALF, the chroma ALF, the CCALF, or a combination thereof.

3. The method as described in claim 1, characterized in that, The step of selecting a target fixed filter set involves selecting multiple APSs at the image level, tile level, slice level, or a combination thereof, and selecting a target APS from these multiple APSs at the CTB level.

4. The method as described in claim 1, characterized in that, The system signals or parses a flag at the image level, tile level, slice level, or a combination thereof to control the use of a fixed filter at that CTB level, and the use of the fixed filter is for that filter type group, which further includes the luma ALF.

5. The method as described in claim 4, characterized in that, This flag controls whether the selection of a fixed filter set is enabled at the CTB level.

6. The method as described in claim 4, characterized in that, This flag indicates whether only one or more fixed filter sets are allowed to be selected at the CTB level.

7. The method as described in claim 4, characterized in that, This flag is used to enable the selection of the Adaptation Parameter Set (APS) ID and APS filter set at the CTB level.

8. The method as described in claim 4, characterized in that, This flag indicates whether only one or more APS (Adaptation Parameter Set) IDs and APS filter sets are allowed to be selected at the CTB level.

9. The method as described in claim 1, characterized in that, In this context, a set of Adaptation Parameter Set (APS) filters is signaled or parsed at the image level, tile level, slice level, or a combination thereof, and this APS filter set comes from different APSs, and the APS filter set can only be selected at the CTB level.

10. The method as described in claim 1, characterized in that, The cross-component model was selected as the target filter type.

11. The method as described in claim 1, characterized in that, The samples generated by this cross-component model are directly used as the current block after filtering.

12. An apparatus for color images or videos, the apparatus comprising one or more electronic devices or processors, characterized in that, Configured as follows: Receive reconstructed pixels, wherein the reconstructed pixels include the current block, and the current block includes a first color block and a second color block; Based on the filter selection instruction at the Coding Tree Block (CTB) level, a target fixed filter set is selected from multiple fixed filter sets for filtering the current block using a target filter type in a filter type group, which includes an Adaptive Loop Filter (ALF), a Cross-Component ALF (CCALF), or both. Apply the target fixed filter set to the current block to generate the filtered current block; and Provides the current block after the filtering.

Images