Unified classification method and device in loop filtering

Abstract

The invention discloses a loop filtering method and device for reconstructing a video. The method receives input data of a current block, wherein the input data includes reconstructed samples of the current block. At least two loop filters are applied to a current block, where the at least two loop filters belong to a loop filter bank comprising a bilateral filter (BIF) and an adaptive loop filter (ALF), and the classification processes of the at least two loop filters share one input source, one or more classification rules, one or more processing units or a combination thereof. A filtered output generated by applying the at least two loop filters to the current block is provided.

Description

[0001] Cross-referencing This invention is a non-provisional application of U.S. Provisional Patent Application No. 63 / 512,916 (filed July 11, 2023), and claims priority thereto. The aforementioned U.S. Provisional Patent Application is incorporated herein by reference in its entirety. [Technical Field] This invention relates to a video encoding and decoding system employing multiple loop filters (including bilateral filters (BIF) and adaptive loop filters (ALF)). Specifically, this invention relates to unifying the classification of multiple loop filters to reduce system complexity. [Background Technology] Universal Video Coding (VVC) is the latest international video coding standard (MPEG) developed by the Joint Video Experts Group (JVET) of the ITU-T Video Coding and Decoding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group. This standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology - Coded representation of immersive media - Part 3: Versatile video coding, published in February 2021. VVC was developed by adding more coding and decoding tools to improve coding and decoding efficiency and handle various types of video sources, including 3D video signals, based on its predecessor, HEVC (High Efficiency Video Coding).

[0004] Figure 1AAn exemplary adaptive inter / intra video codec system incorporating loop processing is described. For intra-frame prediction, prediction data is derived from previously encoded video data in the current frame (hereinafter referred to as a picture). For inter-frame prediction 112, motion estimation (ME) is performed on the encoder side, and motion compensation (MC) is performed based on the result of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects either intra-frame prediction 110 or inter-frame prediction 112, and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118 and subsequent quantization (Q) 120. The residual from transform and quantization is then encoded by entropy encoder 122 to be included in the video bitstream corresponding to compressed video data. Then, the bitstream associated with the transform coefficients is packaged together with side information (e.g., motion and decoding modes associated with intra-frame and inter-frame prediction) and other information (e.g., parameters associated with the loop filter applied to the underlying image area). The side information associated with intra-frame prediction 110, inter-frame prediction 112, and loop filter 130 is provided to the entropy encoder 122, such as... Figure 1A As shown. When using inter-frame prediction mode, one or more reference images must also be reconstructed at the encoder. Therefore, the transformed and quantized residuals are processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residuals. Then, at reconstruction (REC) 128, the residuals are added back to the prediction data 136 to reconstruct the video data. The reconstructed video data can be stored in the reference image buffer 134 and used to predict other frames.

[0005] like Figure 1AAs shown, the input video data undergoes a series of processing steps in the encoding system. Due to these processing steps, the reconstructed video data from REC 128 may suffer various forms of degradation. Therefore, loop filters 130 are frequently applied to the reconstructed video data before it is stored in the reference picture buffer 134 to improve video quality. For example, deblocking filters (DF), Sample Adaptive Offset (SAO), and Adaptive Loop Filters (ALF) may be used. It may be necessary to incorporate loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to entropy encoder 122 for incorporation into the bitstream. Figure 1A In the process, the loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in the reference image buffer 134. Figure 1A The system described herein is intended to illustrate an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Codec (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0006] like Figure 1B As shown, apart from transform 118 and quantization 120, the decoder can use similar or identical function blocks to the encoder, since the decoder only needs inverse quantization 124 and inverse transform 126. Instead of entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bitstream into quantized transform coefficients and the necessary encoding / decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from entropy decoder 140 without motion estimation.

[0007] Similar to HEVC, VVC partitions the input image into non-overlapping square block regions called CTUs (Code-Decoder Tree Units). Each CTU can be further divided into one or more smaller Code-Decoder Units (CUs). The resulting CU partitions can be squares or rectangles. Furthermore, VVC divides the CTUs into Prediction Units (PUs), which serve as units for applying prediction processing, such as inter-frame prediction, intra-frame prediction, etc.

[0008] Adaptive loop filter in VVC In VVC, an adaptive loop filter (ALF) with block-based filter adaptability is applied. For the luminance component, a filter is selected from 25 filters in each 4×4 block based on the direction and activity of the local gradient.

[0009] 1. Filter shape Use two diamond filter shapes (such as...) Figure 2 (As shown). A 7×7 rhombus 220 is applied to the luminance component, and a 5×5 rhombus 210 is applied to the chrominance component.

[0010] 2. Block Classification For the luminance component, each 4×4 block is divided into one of 25 categories. The classification index C is derived based on the quantized values ​​of its directionality D and activity Â, as follows: To calculate D and Â, we first use a one-dimensional Laplacian to calculate the gradients in the horizontal, vertical, and two diagonal directions: Where index and express The coordinates of the top-left sample within the block, and Representing coordinates Sampling for reconstruction.

[0011] To reduce the complexity of block classification, a one-dimensional Laplace calculation for subsampling is applied in the vertical direction ( Figure 3A ) and horizontal direction ( Figure 3B ).like Figure 3C As shown in -D, the same subsampling position is used for gradient calculation in all directions. Figure 3C In and Figure 3D In ).

[0012] The maximum and minimum values ​​of the gradients in the horizontal and vertical directions are then set as follows: The maximum and minimum values ​​of the gradients in the two diagonal directions are set as follows: To derive the values ​​of the directionality D, these values ​​are compared with each other and with two thresholds t1 and t2: Step 1. If If all are true, then Set as .

[0013] Step 2. If If yes, proceed to step 3; otherwise, proceed to step 4.

[0014] Step 3. If ,but Set to 2, otherwise Set as .

[0015] Step 4. If , but Set as Otherwise Set as .

[0016] Activity value A is calculated as follows: A is further quantized to the range of 0 to 4 (inclusive), and the quantized value is represented as .

[0017] No classification should be applied to the chromaticity components in the image.

[0018] 3. Geometric transformation of filter coefficients and clipped value Before filtering each 4×4 luminance block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients f(k,l) and the corresponding filter thresholding values ​​c(k,l), based on the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter's support region. The idea is to make different blocks to which ALF is applied more similar by aligning their orientations.

[0019] Three geometric transformations are introduced: diagonal, vertical flip, and rotation. Where K is the size of the filter. These are the coefficient coordinates, with position (0,0) located at the top left corner and position (K-1,K-1) located at the bottom right corner. Based on the gradient values ​​calculated for this block, the transform is applied to the filter coefficients f(k,l) and the limiting value c(k,l). The relationship between the transform and the four gradients in the four directions is summarized in the table below.

[0020] Table 1. Mapping of gradients and transformations computed for a block 4. Filtering Process On the decoder side, when ALF is enabled for CTB, every sample within the CU Filtered to generate sampled values , Where f(k,l) represents the decoded filter coefficients, K(x,y) is the clipping function, and c(k,l) represents the clipping parameters after decoding. The variables k and l vary between –L / 2 and L / 2, where L represents the filter length. Clipping function Corresponding to function The pruning operation introduces non-linearity, making ALF more efficient by reducing the influence of neighboring samples that differ too much from the current sample value.

[0021] 5. Cross Component Adaptive Loop Filter (CCALF or CC-ALF) CC-ALF uses luminance sample values ​​to refine each chrominance component by applying an adaptive linear filter to the luminance channel and then using the output of this filter operation for chrominance refinement. Figure 4A A system-level diagram of the CC-ALF process relative to the SAO, luminance ALF, and chrominance ALF processes is provided. For example... Figure 4A As shown, each color component (i.e., Y, Cb, and Cr) is processed by its respective SAO (i.e., SAO luminance 410, SAO Cb 412, and SAO Cr 414). After the SAO, ALF luminance 420 is applied to the luminance of the SAO-processed component, while ALF chrominance 430 is applied to the Cb and Cr of the SAO-processed component. However, there are cross-component terms from the luminance to the chrominance components (i.e., CC-ALF Cb 422 and CC-ALF Cr 424). The outputs from the cross-component ALF are added (using adders 432 and 434, respectively) to the output from the ALF chrominance 430.

[0022] Filtering in CC-ALF is achieved by using linear, diamond-shaped filters (e.g.) Figure 4B The filters 440 and 442 in the image are applied to the luminance channel. Figure 4B In the diagram, hollow circles represent luminance sampling, while dot-filled circles represent chrominance sampling. Each chrominance channel uses a filter, the operation of which is expressed as: in It is the chromaticity component i whose position has been refined. Based on The brightness position, It is the filter support region in the luminance component, and This represents the filter coefficients.

[0023] like Figure 4B As shown, the luma filter support is the region juxtaposed with the current chroma sample after considering the spatial scaling factor between the luma and chroma planes.

[0024] In the VVC reference software, CC-ALF filter coefficients are calculated by minimizing the mean squared error of each chroma channel relative to the original chroma content. To achieve this, the VTM (VVC Test Model) algorithm uses a coefficient derivation process similar to that used for chroma ALF. Specifically, the correlation matrix is ​​derived, and the coefficients are computed using a Cholesky decomposition solver to attempt to minimize the mean squared error metric. Up to eight CC-ALF filters can be designed and transmitted per image during filter design. The generated filters are then indicated for each of the two chroma channels based on the CTU.

[0025] Additional features of CC-ALF include: The design uses a 3x4 diamond shape with 8 taps.

[0026] Seven filter coefficients are transmitted in the APS.

[0027] Each transmitted coefficient has a 6-bit dynamic range and is limited to powers of 2.

[0028] The eighth filter coefficient is derived at the decoder, such that the sum of the filter coefficients equals 0.

[0029] APS can be referenced in the slice header.

[0030] CC-ALF controls filter selection at the CTU level for each chromaticity component.

[0031] The boundary padding of the horizontal virtual boundary uses the same memory access pattern as the brightness ALF.

[0032] As an additional feature, the reference encoder can be configured to enable some basic subjective tuning via a configuration file. When enabled, the VTM reduces the application of CC-ALF in areas using high QP encoding / decoding and close to mid-gray or containing a large number of high-frequency luminance values. Algorithmically, this is achieved by disabling CC-ALF in the CTU when any of the following conditions are true: The slice QP value minus 1 is less than or equal to the base QP value.

[0033] The number of chroma samples with a local contrast greater than (1 << (bitDepth – 2)) – 1 exceeds the CTU height, where the local contrast is the difference between the maximum and minimum luminance sample values ​​within the filter's supported region.

[0034] More than a quarter of the chroma samples lie in the range between (1<<(bitDepth – 1)) – 16 and (1<<(bitDepth – 1)) + 16. The motivation behind this feature is to provide a guarantee that CC-ALF will not amplify artifacts introduced earlier in the decoding path (primarily due to the fact that VTM currently does not explicitly optimize for subjective chroma quality). It is anticipated that alternative encoder implementations may not use this feature or may incorporate alternative strategies suited to their encoding characteristics.

[0035] 6. Filter parameter signaling ALF filter parameters are signaled in the Adaptation Parameter Set (APS). An APS can signal up to 25 sets of luma filter coefficients and limiting indexes, and up to 8 sets of chroma filter coefficients and limiting indexes. To reduce bit overhead, filter coefficients from different categories of luma components can be combined. In the slice header, the signaling is used for the index of the APS for the current slice.

[0036] The limiting index decoded from APS allows the use of limiting tables for both the luminance and chrominance components to determine the limiting value. These limiting values ​​depend on the internal bit depth. More precisely, the limiting value is obtained using the following formula: Where B equals the internal bit depth, α is a predefined constant equal to 2.35, and N equals 4, which is the number of clipping values ​​allowed in VVC. AlfClip then rounds the value to the nearest power of 2.

[0037] In the slice header, up to seven APS indices can be signaled to specify the luma filter bank used for the current slice. The filtering process can be further controlled at the CTB level. Always signaling flags indicate whether the ALF is applied to the luma CTB. The luma CTB can select a filter bank from 16 fixed filter banks and filter banks from the APS. Filter bank indices are signaled for the luma CTB to indicate which filter bank is applied. The 16 fixed filter banks are predefined and hard-coded in the encoder and decoder.

[0038] For chroma components, the signaling APS index is used in the slice header to indicate the chroma filter bank used for the current slice. At the CTB level, if there is more than one chroma filter bank in the APS, a signaling filter index is used for each chroma CTB.

[0039] Filter coefficients are quantized with a norm equal to 128. To limit multiplication complexity, bitstream consistency is applied such that coefficient values ​​at non-center positions should be in the range of -27 to 27–1 (inclusive). Coefficients at signaling center positions not in the bitstream are considered equal to 128.

[0040] Adaptive Loop Filters in ECM At the 29th meeting of ECM8 (Muhammed Coban et al., “Algorithm description of Enhanced Compression Model 8 (ECM 8)”, Joint Video Expert Group (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, held via teleconference, January 11-20, 2023, document: JVET-AC2025), some changes to the VVC ALF were disclosed. A brief summary is described below.

[0041] 1. Simplified removal of ALF Remove ALF gradient subsampling and ALF virtual boundary processing. Reduce the block size for classification from 4x4 to 2x2. Increase the filter size (signaling as ALF coefficients) for luminance and chrominance to 9x9.

[0042] 2. ALF with a fixed filter To filter luminance samples, three different classifiers (C0, C1, and C2) and three different filter sets (F0, F1, and F2) are used. Sets F0 and F1 contain fixed filters whose coefficients are trained on classifiers C0 and C1. The filter coefficients in F2 are signaled. Which filter from set F1 is used for a given sample is assigned to that sample by the classifier Ci. Decide.

[0043] 3. Filtering First, two 13x13 rhombus fixed filters, F0 and F1, are used to derive two intermediate samples. and Then, apply F2. , And neighboring samples to derive the filtered samples: in It is neighbor sampling and current sampling The difference in the limit between them yes The difference in amplitude limiting between current samples. Signaling filter coefficients. .

[0044] 4. Classification Based on directionality and activities , classify Assign to each 2x2 block: in Indicates directionality The total number.

[0045] Similar to VVC, a one-dimensional Laplacian operator is used to compute the horizontal, vertical, and two diagonal gradient values ​​for each sample. The sum of the sampled gradients within a 4×4 window covering a 2×2 block of the target is used for classifier C0, while the sum of the sampled gradients within a 12×12 window is used for classifiers C1 and C2. The sums of the horizontal, vertical, and two diagonal gradients are expressed as follows: Directionality By comparison Determined by a set of thresholds. Directionality It uses thresholds 2 and 4.5, as exported in VVC. For and First, calculate the horizontal / vertical edge strength. and diagonal edge strength Using thresholds .like Then the edge strength =0; otherwise, It is to satisfy The largest integer. If Then the edge strength =0; otherwise, It is to satisfy The largest integer. When When horizontal / vertical edges dominate, use Table 2A to derive... Otherwise, the diagonal edges dominate, as derived using Table 2B. .

[0046] Table 2A. [The following is a list of items] and Mapped to Table 2B. [The following is a list of items / items] and Mapped to To obtain The sum of vertical and horizontal gradients Mapping to 0 The range, of which for , Equals 4, for and , It equals 15.

[0047] In ALF_APS, the signaling can have up to 4 sets of luminance filters, and each set can have up to 25 filters.

[0048] 5. Replacement for 2x2 ALF classifier Classification in ALF is extended with an additional alternative classifier. For the signaling luminance filter set, a signaling flag indicates whether an alternative classifier is applied. Geometric transformations are not applied to the alternative band classifier. When applying a band-based classifier, the sum of sample values ​​for the 2x2 luminance block is first calculated. The category index is then calculated as follows: class_index = (sum * 25)>>(sample bit depth + 2). 6. Residual-based classifiers The third classifier is based on the luminance residual sample values. For each 2x2 luminance block, the sum of the absolute values ​​of the residual samples in the neighboring 8x8 viewports is calculated, and the class index is derived: classIdx = sum>>(sample bit depth – 4). Similar to ECM-8.0, the value of classIdx ranges from 0 to 24. For each luminance filter set in the APS, the signaling classifier usage is specified.

[0049] 7. CCALF with long tap filter The CCALF process uses a linear filter to filter luminance sample values ​​and generate residual corrections for chrominance samples. A large 25-tap filter is used in the CCALF process, such as... Figure 5 As shown. In Figure 5In the image, the taps of the luminance samples are displayed as gray dots, and the corresponding positions of the chrominance samples are displayed as small dashed circles. For a given slice, the encoder can collect and analyze the statistical data of that slice, and can signal to up to 16 filters via the APS.

[0050] 8. Adaptive filter shape switching / Using samples for ALF before deblocking filter Two candidate filter shapes: such as Figure 6 The rhombus shape shown and as Figure 7 The new cross shape shown can be adaptively selected by the luminance filter shape in ALF. For both filter shapes, the number of luminance filter coefficients is 22. Note that these 22 taps are derived from the 20 space taps in both shapes (in...). Figure 6 and Figure 7 The two taps are 610 and 710 respectively and two taps based on fixed filters (in...). Figure 6 and Figure 7 The components are composed of 620 and 720 respectively.

[0051] In each Adaptation Parameter Set (APS), the derived shape indexes of the luma filters are passed to the decoder. Each APS contains luma filters associated with their respective filter shape indices.

[0052] For each CTB, the signaling APS index indicates which luminance filter shape to use for filtering the current CTB. When filtering luminance samples, the coefficients and clip indices are also rearranged according to the corresponding filter shape.

[0053] The diamond-shaped brightness ALF is replaced by a longer filter, such as... Figure 7 As shown.

[0054] The samples before the deblocking filter are used as additional input to the ALF. The final ALF samples are derived by weighting the results of the regular ALF and the filters applied to the samples before the deblocking filter. Specifically, the filtered samples are derived as follows: , in It is the clipped difference between neighboring samples and the current sample R(x,y). It represents the difference in shearing between the intermediate samples and the current sample R(x,y). This represents the shearing difference between neighboring samples before DBF and the current sample R(x,y). Signaling filter coefficients. For example, a 3x3 diamond shape is applied to the samples before the deblocking filter. In APS, a signaling flag is used to indicate whether samples previously used in the DBF were used in the ALF; this flag is always set to true at the encoder.

[0055] 9. Taps for ALF based on extended fixed-filter-output In ALF, the filters trained online consist of four types of filter taps: spatial taps (810), reconstruction-before-DBF based taps (840), residual-based taps (850), and taps based on fixed filter outputs (820 and 830), as shown below. Figure 8 As shown.

[0056] 10. ALF with residual samples The residual samples are used as additional input to ALF. The derivation of the filtered samples is as follows: , in These are the neighboring residual sample values ​​after shearing. These are the cropped residual samples after filtering with a fixed filter. For the residual samples, the offline fixed filter is reused; this offline fixed filter is trained for reconstruction after SAO.

[0057] 11. Additional fixed filters for ALF An additional fixed filter with a 7x7 rhombus shape was introduced, and its parameters are stored in the encoder and decoder. The newly added fixed filter is not categorized.

[0058] The online filter or online trained filter of the proposed method, such as Figure 9As shown, spatial taps 910 (i.e., taps #0~#19), taps 940 based on prior DBF reconstruction (i.e., taps #26, #27, #36), residual-based taps 950 (i.e., #37~#38), and taps 920 and 930 based on fixed filter outputs (i.e., taps #20~#25, #34, #35) are identical to those in ECM-8.0, and several extended taps 960 (i.e., taps #28~#33, #39) are introduced into the lumen in-line training filter. The prior DBF reconstruction is fed into additional fixed filters to produce filter outputs, which are then used as inputs to the newly extended taps. In-line filters, or in-line trained filters, refer to filters specified in the APS (Adaptive Parameter Set), where the filters are trained at the encoder and signaled to the decoder. In-line filters, or in-line trained filters, are the opposite of fixed filters, which are trained offline and predefined in the standard.

[0059] This filter is always enabled and requires no filter shape switching.

[0060] 12. Improved ALF Fixed Filter (JVET-AE0139) An improved fixed filter for ALF is disclosed in JVET-AE0139 (Marta Karczewicz et al., “EE2-5.2: EE2-5.2: Improved fixed filters for ALF”, ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 Joint Video Experts Group (JVET), 31st Meeting, Geneva, Switzerland, July 11-19, 2023, Document: JVET-AE0139). According to JVET-AE0139, two fixed filters are applied to samples prior to the DBF and ALF inputs, with an additional 9×9 diamond filter used for the samples prior to the DBF. The shape of the first fixed filter applied to the ALF input samples is reduced from 13×13 to 9×9, while the shape of the second fixed filter applied to the ALF input (13×13) remains unchanged, as shown in Table 3.

[0061] Table 3. Comparison of ECM-9.0 and JVET-AE0139 fixed filters A classifier with a fixed filter is extended. For each 2x2 block, the mean of the surrounding window is calculated. Then, for each sample in that window, the difference between the sample value and the mean is calculated. A scaling factor is determined based on the activity value derived from the Laplacian classifier. The square root of the sum of squared differences is further quantized using the scaling factor. . The value is an integer between 0 and 7, inclusive. In ECM-9.0, when i=0, 1, let Let represent the classifier from the classifier with the i-th fixed filter. Then, the suggested class index is... Export as .

[0062] In the JVET-AE0139 proposal, the total number of fixed filters remains unchanged. Applied to The output (instead of the ALF input) and the samples before DBF.

[0063] Bilateral filters in ECM As shown in Figure 10, filtering is performed in the sample adaptive offset (SAO) loop filter stage. Both the bilateral filter (BIF) 1010 and the SAO 1020 use the deblocked samples as input. Each filter generates an offset for each sample, which is added to the input sample via adder 1030, then clipped, and then proceeds to the ALF stage.

[0064] Specifically, output sample I OUT The calculation formula is: in This is the input sample after deblocking. It is the offset of the bilateral filter. It is the offset of SAO.

[0065] This implementation allows the encoder to enable or disable filtering at both the CTU and slice levels. The encoder makes this decision by evaluating the cost of the RDO.

[0066] For a filtered CTU, the filtering process is as follows.

[0067] At image boundaries, since samples are unavailable, the bilateral filter uses expansion (sample repetition) to fill in the unavailable samples. For virtual boundaries, it behaves the same as SAO, i.e., no filtering is performed. When crossing horizontal CTU boundaries, the bilateral filter can access the same samples as SAO. For example, as Figure 11 As shown, if the center sampling point Located at the top row of CTU, then , and It will be read from the CTU above, just like SAO, but It will be padded, so no additional line buffer is required compared to JVET-P0073 (Jacob Ström et al., “CE5-3.1 Combination of bilateral filter and SAO”, ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC29 Joint Video Experts Group (JVET), 16th meeting: Geneva, Switzerland, October 1-11, 2019, document: JVET-P0073).

[0068] According to Figure 11, around the central sample The samples are labeled, where A, B, L, and R represent above, below, left, and right, respectively; NW, NE, SW, and SE represent northwest, and so on. Similarly, AA represents above-above, BB represents below-below, and so on. This rhomboid shape differs from JVET-P0073, which uses square filters for support instead of... , , , or .

[0069] Each surrounding sample , Each of these will contribute a corresponding modifier value. , These modifier values ​​are calculated as follows: from the samples on the right. Starting with the contribution, calculate the difference: Wherein represents Absolute value.

[0070] For data that is not 10 digits, we use Instead, where n = 8 represents 8 bits of data, and so on. The resulting value is clipped to be less than 16: The modifier value is now calculated as follows: in It is an array containing 16 values, which are determined by the value of qpb, qpb = clip(0, 25, QP + bilateral_filter_qp_offset-17): Given the expression { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,}, if qpb = 0 Given the array { 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,}, if qpb = 1 Given the array { 0, 2, 2, 2, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0,}, if qpb = 2 Given the array { 0, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, -1,}, if qpb = 3 Given the given array { 0, 3, 3, 3, 2, 2, 1, 2, 1, 1, 1, 1, 0, 1, 1, -1,}, if qpb = 4 Given the given array { 0, 4, 4, 4, 3, 2, 1, 2, 1, 1, 1, 1, 0, 1, 1, -1,}, if qpb = 5 Given the array { 0, 5, 5, 5, 4, 3, 2, 2, 2, 2, 2, 1, 0, 1, 1, -1,}, if qpb = 6 Given the given array { 0, 6, 7, 7, 5, 3, 3, 3, 3, 2, 2, 1, 1, 1, 1, -1,}, if qpb = 7 Given the given set { 0, 6, 8, 8, 5, 4, 3, 3, 3, 3, 3, 2, 1, 2, 2, -2,}, if qpb = 8 Given the given values ​​{ 0, 7, 10, 10, 6, 4, 4, 4, 4, 3, 3, 2, 2, 2, 2, -2,}, if qpb = 9 Given the given values ​​{ 0, 8, 11, 11, 7, 5, 5, 4, 5, 4, 4, 2, 2, 2, 2, -2,}, if qpb = 10 Given the given values ​​{ 0, 8, 12, 13, 10, 8, 8, 6, 6, 6, 5, 3, 3, 3, 3, -2,}, if qpb = 11 Given the given values ​​{ 0, 8, 13, 14, 13, 12, 11, 8, 8, 7, 7, 5, 5, 4, 4, -2,}, if qpb = 12 Given the following set of integers: { 0, 9, 14, 16, 16, 15, 14, 11, 9, 9, 8, 6, 6, 5, 6, -3,}, if qpb = 13 Given the set {0, 9, 15, 17, 19, 19, 17, 13, 11, 10, 10, 8, 8, 6, 7, -3,}, if qpb = 14 Given the set { 0, 9, 16, 19, 22, 22, 20, 15, 12, 12, 11, 9, 9, 7, 8, -3,}, if qpb = 15 Given the set {0, 10, 17, 21, 24, 25, 24, 20, 18, 17, 15, 12, 11, 9, 9, -3,}, if qpb = 16 Given the set { 0, 10, 18, 23, 26, 28, 28, 25, 23, 22, 18, 14, 13, 11, 11, -3,}, if qpb = 17 Given the set { 0, 11, 19, 24, 29, 30, 32, 30, 29, 26, 22, 17, 15, 13, 12, -3,}, if qpb = 18 Given the set { 0, 11, 20, 26, 31, 33, 36, 35, 34, 31, 25, 19, 17, 15, 14, -3,}, if qpb = 19 Given the set { 0, 12, 21, 28, 33, 36, 40, 40, 40, 36, 29, 22, 19, 17, 15, -3,}, if qpb = 20 Given the set { 0, 13, 21, 29, 34, 37, 41, 41, 41, 38, 32, 23, 20, 17, 15, -3,}, if qpb = 21 Given the set { 0, 14, 22, 30, 35, 38, 42, 42, 42, 39, 34, 24, 20, 17, 15, -3,}, if qpb = 22 Given the set { 0, 15, 22, 31, 35, 39, 42, 42, 43, 41, 37, 25, 21, 17, 15, -3,}, if qpb = 23 Given the set { 0, 16, 23, 32, 36, 40, 43, 43, 44, 42, 39, 26, 21, 17, 15, -3,}, if qpb = 24 Given the set { 0, 17, 23, 33, 37, 41, 44, 44, 45, 44, 42, 27, 22, 17, 15, -3,}, if qpb = 25 This differs from JVET-P0073, which uses five such tables and the same table is reused for multiple qp values.

[0071] As described in section 3.1.3 of JVET-N0493 (Jacob Ström et al., “CE1-related: Multiplication-free bilateral loop filter”, ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 Joint Video Experts Group (JVET), 14th meeting: Geneva, Switzerland, 19-27 March 2019, document: JVET-N0493), these values ​​can be stored using 6 bits per entry, thus occupying 26*16*6 / 8=312 bytes; if the first row of all zeros is excluded, it occupies 300 bytes.

[0072] , and The modifier value is determined in the same way by , and Calculated. For diagonal samples , , , , and samples separated by two steps. , , and The calculations also follow formulas 2 and 3, but use a value with an offset of 1. (Diagonal samples) For example, we get Other diagonal samples and samples two steps away are calculated in the same way. Add the modifier values: Note that for the previous example, equal Similarly, for the example above, equal Similar symmetry applies to the diagonal and two-step modifier values. This means that in the hardware implementation, only six values ​​need to be computed. , , , , and The remaining six values ​​can be obtained from the previously calculated values.

[0073] Now The value is multiplied by c = 1, 2, or 3, which can be achieved using a single adder and a logical AND gate, as follows: Where & represents logical AND, It is the most significant bit of the multiplier c. It is the least significant bit. The value to be multiplied is the minimum block size. The results are shown in the table below.

[0074] Table 4. Obtain the c parameter from the minimum size of the block D = min(width, height).

[0075] Finally, calculate the bilateral filter offset. For full-strength filtering, use For half-intensity filtering, instead... The general formula for n-digit data is to use... The bilateral_filter_strength parameter can be 0 or 1 and is signaled in the PPS (Image Parameter Set).

[0076] Chromaticity bilateral loop filter Similar to BIF-luma, BIF-chroma 1210 also executes in parallel with the SAO and CCSAO processes, such as... Figure 12 As shown, BIF-chroma 1210, CCSAO 1230, and SAO 1220 use the same chroma samples generated by the deblocking filter as input and generate three offsets for each chroma sample in parallel. Then, adder 1240 adds these three offsets to the input chroma samples to obtain a sum, followed by cropping (or trimming) to form the final output chroma sample value. BIF-chroma provides on / off control mechanisms at the CTU level and slice level.

[0077] The filtering process in BIF-chroma is similar to that in BIF-luma. For chroma samples, a 5×5 diamond filter is used to generate the filter offset. First, the difference between the center sample and each surrounding sample is calculated. The coefficients for each reference sample are extracted directly from a predefined lookup table based on the calculated differences. The coefficients of the chroma components are retrained, unlike those in BIF-luma. In the BIF-luma design, the block-level filter intensity parameter c is determined based on the luma TU size and CU mode. In the BIF-chroma design, when the current slice has dual-tree partitioning enabled, the chroma component parameters are determined based on the chroma TU size and mode; when dual-tree partitioning is disabled, they are determined based on the corresponding luma TU size and mode.

[0078] Dynamic Scaling of Bilateral Filters (JVET-AE0044) As described in JVET-AE0044 (V. Shchukin et al., “AHG12: Dynamic Scaling of Bilateral Filter (BIF)”, ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 Joint Video Experts Group (JVET), 31st Meeting, Geneva, Switzerland, July 11-19, 2023, Document: JVET-AE0044), the size of the basic lookup table (LUT) remains unchanged, but the content of the basic LUT changes. The calculation method will also change. The proposed method uses three scaling factors ( To pre-compute three LUTs, corresponding to three different neighborhood distances (1, √2, and 2), i.e.: The least significant bit value of the truncated portion is averaged linearly interpolated by half, i.e.: in and yes The consecutive entries. For chroma, the cutoff bit is reduced from 3 bits to 2 bits.

[0079] In JVET-AE0044, the cutoff bit length is as shown in the following formula: Increased from 5 to 8, and in, Based on the shape and size of TU, The mean absolute difference (MAD) is based on TU. and All calculations are performed using lookup tables (LUTs). More precisely, let... Given a two-dimensional 8×8 lookup table whose values ​​are non-negative 8-bit integers; let... Given a one-dimensional 16-item lookup table where each value is a non-negative 8-digit integer, these scaling factors are defined as follows: For channel samples ( The MAD of a TU of size (h×w) is defined as follows: A total of four 64-byte lookup tables were introduced. and four 16-byte lookup tables (Used for luma / chroma components and intra / inter-frame prediction respectively). Note that for all samples in the same channel within the same TU, It is a constant.

[0080] Variance-based classification of loop filters (JVET-AE0131) In JVET-AE0131 (W. Yin1 et al., “Non-EE2: Variance based Classification for In-loop Filtering”, ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 Joint Video Experts Group (JVET), 31st Meeting, Geneva, Switzerland, July 11-19, 2023, Document: JVET-AE0131), a variance-based BIF and ALF classification method is proposed.

[0081] For BIF, the variance of TU is used to identify texture intensity. BIF introduces more filtering intensity. The filtering intensity is determined by the variance.

[0082] For ALF, each classification unit is jointly divided into two texture intensity levels based on variance and boundary location. These texture intensity levels are then further combined with the existing classifier to output the final classification result.

[0083] This invention discloses a method and apparatus for simplifying the multi-loop filtering process. [Summary of the Invention] This invention discloses a loop filtering method and apparatus for reconstructing video. According to the method, input data for the current block is received, the input data including reconstructed samples of the current block. At least two loop filters are applied to the current block, the at least two loop filters belonging to a loop filter group including bilateral filters (BIF) and adaptive loop filters (ALF), and the classification processes of the at least two loop filters share an input source, one or more classification rules, one or more processing units, or a combination thereof. A filtered output generated by applying the at least two loop filters to the current block is provided.

[0085] In one embodiment, the input source shared by BIF and ALF corresponds to samples before or after the deblocking filter, and residual samples in the original or reshaped domain used for the classification process of the at least two loop filters. In one embodiment, BIF and ALF are executed in parallel.

[0086] In one embodiment, BIF and ALF share one or more classification rules. In one embodiment, the one or more classification rules include classification determinations computed within a region surrounding each processing unit, and the classification determinations are based on mean, variance, gradient mean, gradient variance, or a combination thereof. In one embodiment, BIF and ALF share one or more operational modules for determining the one or more classification rules. In one embodiment, BIF and ALF are executed based on the same processing unit.

[0087] In one embodiment, one or more high-level flags are signaled or parsed to indicate whether the classification processes of the at least two loop filters share an input source, the one or more classification rules, the one or more processing units, or a combination thereof.

[0088] In one embodiment, BIF uses an input source corresponding to the sample or residual sample before the deblocking filter (pre-DBF).

[0089] In one embodiment, the BIF classification process calculates the variance within a classification unit. In another embodiment, a classification unit corresponds to a block of fixed size.

[0090] In one embodiment, the modifier value of BIF is derived from a lookup table. In another embodiment, the modifier value is selected from the lookup table based on sample difference, sample distance, QP value, TU size, CU encoding information, or a combination thereof. In yet another embodiment, one or more adjacent or parsed flags are used to indicate which of the following—sample difference, sample distance, QP value, TU size, and CU encoding information—is used to select the modifier value.

[0091] In one embodiment, two or more modifier values ​​are selected from a lookup table and then mixed to generate a final modifier value. In another embodiment, one or more flags are signaled or parsed to indicate whether mixing the two or more modifier values ​​is enabled. In yet another embodiment, one or more flags are signaled or parsed to indicate how the two or more modifier values ​​are mixed. [Attached Image Description] Figure 1A An exemplary adaptive inter-frame / intra-frame video coding system incorporating loop processing is shown.

[0093] Figure 1B It shows Figure 1A The encoder and its corresponding decoder.

[0094] Figure 2 The ALF filter shapes for the chroma (left) and luminance (right) components are shown.

[0095] Figure 3A -D indicates the use of g v (3A), g h (3B), g d1 (3C) and g d2 Laplace calculation of (3D) subsampling.

[0096] Figure 4A The placement of the CC-ALF relative to other loop filters is shown.

[0097] Figure 4B A diamond filter used for chroma sampling is shown.

[0098] Figure 5 The 25-tap large filter used in the CCALF process is shown.

[0099] Figure 6 The diamond-shaped ALF in ECM-5.0 is displayed.

[0100] Figure 7 Shown as Figure 6 The alternative to the medium-diamond ALF is a longer ALF.

[0101] Figure 8 The filter shape of ALF in ECM-7.0 is shown.

[0102] Figure 9 An example of an ALF with an additional fixed filter is shown.

[0103] Figure 10 shows an example of a bilateral filter (BIF) with sample adaptive offset (SAO).

[0104] Figure 11 shows the area around the central sample (I) C Naming rules for samples.

[0105] Figure 12 shows an example of a bilateral loop filter that combines chroma component sample adaptive offset (SAO) and cross component sample adaptive offset (CCSAO).

[0106] Figure 13 shows a flowchart of an exemplary video encoding and decoding system according to an embodiment of the present invention, which applies a simplified classification process to multiple loop filters.

Detailed Implementation Methods

[0108] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the invention can be practiced without one or more specific details or using other methods, components, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the invention. The embodiments shown in the invention will be better understood with reference to the accompanying drawings, wherein the same components are always indicated by the same reference numerals. The following description is intended only by way of example and to simply illustrate certain selected embodiments of apparatus and methods consistent with the invention claimed herein.

[0109] The proposed method Unified classification in loop filtering In ECM, classification is performed using bilateral filtering (BIF), sample adaptive offset (SAO), cross-component SAO (CCSAO), and adaptive loop filter (ALF). However, the input sources, rules, and units differ for each classification method. This invention discloses several unified classification designs for loop filtering to reduce the complexity of the loop filtering process.

[0110] In the following description, BIF classification refers to the filter multiplier (as described in the background section, denoted as c or ...). The derivation of ).

[0111] In one embodiment, classification across different loop filtering stages (i.e., at least two of BIF, SAO, CCSAO, and ALF) shares input sources, classification rules, and / or processing units. Specifically, the input source refers to the information involved in the classification process. For example, in ECM BIF (JVET-AE0044), the input source is the output of the deblocking filter (post-DBF) samples; in ECM ALF, there are two input sources: the output of the SAO (post-SAO) samples and the residual samples. The classification rule refers to the formulas and lookup tables used in the classification process. The processing unit refers to the basic unit of classification. For example, in ECM BIF, the processing unit is a TU; in ECM ALF, the processing unit is a 2x2 block.

[0112] Example 1: BIF and ALF categories share the same input source.

[0113] BIF and ALF classification utilize samples before DBF, samples after DBF, residual samples in the original domain, and / or residual samples in the reshaped domain. This design reduces latency because ALF classification can be performed in parallel with BIF classification.

[0114] Example 2: BIF and ALF share classification rules.

[0115] The mean, variance, gradient mean, and / or gradient variance of the region surrounding each processing unit are calculated for both BIF and ALF classification. This design reduces implementation costs because ALF classification can reuse the same operational modules from BIF classification.

[0116] Example 3: BIF and ALF share the same processing unit.

[0117] The processing unit is a 2x2 block used for BIF and ALF sorting. This design is more hardware-friendly because the pipelines for BIF and ALF sorting can be easily aligned.

[0118] The examples above can be combined to form a more unified design, thereby bringing more implementation advantages.

[0119] Combining Examples 1 and 2, BIF classification and ALF classification share the same input source and the same rules. For example, the variance of each processing unit in BIF and ALF can be derived using samples post-DBF processed, and these variances can be further used for BIF and ALF classification.

[0120] Combining Examples 1 and 3, BIF classification and ALF classification share the same input source and processing unit. For example, samples processed by DBF are used to derive different statistics for each 2x2 block, and these statistics are further used for BIF and ALF classification.

[0121] Combining Examples 1, 2, and 3, the classification in BIF and ALF share the same input source, classification rules, and processing units. For example, the samples processed by DBF are used to derive the variance of each 2x2 block, and this variance is further used for classification in both BIF and ALF.

[0122] Note that the proposed embodiments emphasize some shared classification sources and / or operations between the loop filtering stages, but differences are still allowed. For example, in BIF, variance is calculated for classification; in ALF, variance and gradient are calculated for classification. As long as the variance calculation operation is the same, this design falls within the scope of the proposed embodiments. As another example, in BIF, samples after DBF are used for classification; in ALF, given multiple classifiers (e.g., two classifiers for a fixed filter in ECM, three classifiers for an APS filter), some of these classifiers use samples after DBF for classification. This design also falls within the scope of the proposed embodiments.

[0123] For the embodiments described above, one or more advanced flags can be set to indicate whether a more uniform design is used. The shared portions of the classification can be explicitly communicated or implicitly determined. For example, in an implicit method, the classification rules are also implicitly determined when the shared source is identified. The classification rules can be, for example, classification rules specific to a single shared source.

[0124] Improvements and simplifications of BIF In one embodiment, BIF classification uses samples or residual samples before the deblocking filter (pre-DBF) instead of the output of the deblocking filter (post-DBF). In this embodiment, one or more advanced flags can be set to select the classification input source.

[0125] In another embodiment, BIF classification calculates the variance within classification units, rather than the mean absolute difference. In this embodiment, one or more advanced flags can be set to select the classification rule.

[0126] In another embodiment, the BIF classification unit is a fixed-size block, rather than a TU. Specifically, some statistics about the area surrounding the block are first derived, and these statistics are used to classify the block. In this embodiment, one or more advanced flags can be set to select the size of the classification unit.

[0127] In another embodiment, the modifier value (as described in the background section, denoted as...) The value is derived from a lookup table and is determined by the sample difference, sample distance, QP value, TU size, and / or CU encoding / decoding information (in JVET-AE0044, this value is selected from the sample difference, sample distance, and QP value). In other words, the influence of TU size and CU encoding / decoding information (e.g., skip / predict mode) is shifted from the multiplier decision (i.e., classification) to the modifier value selection, making the multiplier decision depend only on block statistics. In this embodiment, one or more high-level flags can be set to select the information used to look up the modifier value.

[0128] Example 4. A lookup table containing five dimensions, including sample difference, sample distance, QP value, TU size, and CU pattern.

[0129] Specifically, the design of the first three dimensions is similar to JVET-AE0044. TU size can refer to a combination of (TU width, TU height), or the minimum / maximum value of TU width and TU height. CU mode refers to skip mode and / or predict intra / inter-frame mode.

[0130] In another embodiment, based on the above embodiments, a modifier value can be derived by mixing multiple values ​​from a lookup table, wherein the additional value is selected by modifying one or more dimensions of the lookup table (i.e., sample difference, sample distance, QP value, TU size, and / or CU encoding / decoding information). In this embodiment, one or more advanced flags can be provided to select whether to enable the mixing mechanism and / or how to mix multiple values.

[0131] Example 5. The QP of the currently processed sample is q.

[0132] The modifier value is obtained by averaging the two values, which correspond to q and q+1 in the lookup table, respectively.

[0133] Example 6. The QP of the currently processed sample is q.

[0134] If the sample belongs to a skipped pattern block, the modifier value is obtained by averaging two values, corresponding to q and q–2 respectively. If the sample belongs to a non-skipped pattern block, the modifier value is obtained directly from the lookup table.

[0135] Example 7. At the boundary of a TU, if the currently processed sample and its neighboring sample belong to different TUs, the modifier value is obtained by averaging the two values.

[0136] In this example, these two values ​​correspond to the (QP, TU size, CU mode) triplet of the currently processed sample and the triplet of the neighboring sample, respectively.

[0137] The aforementioned method for processing multiple loop filters using a unified classification process can be implemented in the encoder and / or decoder. For example, the proposed method can be implemented in the loop filtering module of the encoder and / or the loop filtering module of the decoder.

[0138] Any of the above-described unified classification processing methods can be implemented in the encoder and / or decoder. Furthermore, any of the above-described unified classification processing methods can be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the loop filter module of the encoder or decoder (e.g., Figure 1A and Figure 1B The proposed methods can be implemented in ILPF 130. Alternatively, any of the proposed methods can be implemented as a motion compensation module coupled to the encoder and / or the decoder, or as a circuit for merging candidate derived modules. The simplified ALF method can also be implemented using executable software or firmware code stored on media (e.g., hard disk or flash memory) for a CPU (central processing unit) or programmable device (e.g., DSP (digital signal processor) or FPGA (field programmable gate array)).

[0139] Figure 13 A flowchart of an exemplary video encoding / decoding system according to an embodiment of the present invention is shown, which applies a unified classification process to multiple loop filters. The steps shown in the flowchart can be implemented as executable code on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 1310, input data for the current block is received, wherein the input data includes reconstructed samples of the current block. In step 1320, at least two loop filters are applied to the current block, wherein the at least two loop filters belong to a loop filter group comprising BIF (Bilateral Filter) and ALF (Adaptive Loop Filter), and the classification process of the at least two loop filters shares an input source, one or more classification rules, one or more processing units, or a combination thereof. In step 1330, a filtered output generated by applying the at least two loop filters to the current block is provided.

[0140] The flowchart shown is intended to illustrate an example of video encoding and decoding according to the present invention. Those skilled in the art can modify, rearrange, split, or combine each step to implement the invention without departing from its spirit. Specific syntax and semantics have been used in this disclosure to illustrate examples of implementing embodiments of the invention. Those skilled in the art can practice the invention by substituting equivalent syntax and semantics without departing from its spirit.

[0141] The above description is given to enable those skilled in the art to practice the invention provided in the context of a particular application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the invention is not intended to be limited to the specific embodiments shown and described, but should be given the widest scope consistent with the principles and novel features disclosed herein. In the detailed description above, various specific details are shown to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention can be practiced.

[0142] The embodiments of the invention described above can be implemented in various hardware, software code, or a combination of both. For example, embodiments of the invention may be one or more electronic circuits integrated into a video compression chip or program code integrated into video compression software to perform the processes described herein. Embodiments of the invention may also be program code executed on a digital signal processor (DSP) to perform the processes described herein. The invention may also relate to various functions performed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors may be configured to perform specific tasks according to the invention by executing machine-readable software code or firmware code that defines the specific methods embodied in the invention. The software code or firmware code may be developed in different programming languages ​​and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, software code styles and languages, and other ways of configuring the code to perform the tasks according to the invention will not depart from the spirit and scope of the invention.

[0143] This invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples should be considered in all respects as illustrative rather than restrictive. Therefore, the scope of the invention is indicated by the appended claims rather than the foregoing description. All modifications to the meaning and scope of equivalents falling within the claims are included within their scope.

Claims

1. A loop filtering method for reconstructing video, the method comprising: Receive input data for the current block, wherein the input data includes the reconstructed sample of the current block; At least two loop filters are applied to the current block, wherein the at least two loop filters belong to a loop filter group including BIF (Bilateral Filter) and ALF (Adaptive Loop Filter), and the classification process of the at least two loop filters shares an input source, one or more classification rules, one or more processing units, or a combination thereof; as well as Provides a filtered output generated by applying at least two loop filters to the current block.

2. The method of claim 1, wherein the BIF and the ALF share the input source, which corresponds to the samples before or after the deblocking filter in the classification process of the at least two loop filters, or the residual samples in the original or reshaped domain.

3. The method of claim 1, wherein the BIF and the ALF are executed in parallel.

4. The method of claim 1, wherein the BIF and the shared one or more classification rules.

5. The method of claim 4, wherein the one or more classification rules include a classification determination calculated in the area surrounding each processing unit, and the classification determination is based on mean, variance, gradient mean, gradient variance, or a combination thereof.

6. The method of claim 5, wherein the BIF and the ALF share one or more operation modules for determining the one or more classification rules.

7. The method of claim 1, wherein the BIF and the method are executed based on the same processing unit.

8. The method of claim 1, wherein one or more advanced flags are signaled or parsed to indicate whether the classification process of the at least two loop filters shares the input source, the one or more classification rules, the one or more processing units, or a combination thereof.

9. The method of claim 1, wherein the BIF uses the input source corresponding to the sample or residual sample before the deblocking filter (pre-DBF).

10. The method of claim 1, wherein the BIF classification process calculates the variance within the classification unit.

11. The method of claim 10, wherein the classification unit corresponds to a block of fixed size.

12. The method of claim 1, wherein the modifier value of the BIF is derived from a lookup table.

13. The method of claim 12, wherein the modifier value is selected from a lookup table based on sample difference, sample distance, QP value, TU size, CU encoding information, or a combination thereof.

14. The method of claim 13, wherein one or more flags are signaled or parsed to indicate which of the following are used to select the modifier value: the sample difference, the sample distance, the QP value, the TU size, and the CU encoding information.

15. The method of claim 13, wherein two or more modifier values ​​are selected from the lookup table, and the two or more modifier values ​​are mixed to generate a final modifier value.

16. The method of claim 15, wherein one or more flags are signaled or parsed to indicate whether the mixing of the two or more modifier values ​​is enabled.

17. The method of claim 15, wherein one or more flags are signaled or parsed to indicate how the two or more modifier values ​​are mixed.

18. A video encoding / decoding apparatus, the apparatus comprising one or more electronic circuits or processors arranged as follows: Receive input data for the current block, wherein the input data includes reconstructed samples of the current block; At least two loop filters are applied to the current block, wherein the at least two loop filters belong to a loop filter group including BIF (Bilateral Filter) and ALF (Adaptive Loop Filter), and the classification process of the at least two loop filters shares an input source, one or more classification rules, one or more processing units, or a combination thereof; and Provides a filtered output generated by applying at least two loop filters to the current block.

Images