Method and apparatus for adaptive loop filter with alternative luminance classifier for video coding

Abstract

A method and apparatus for video coding using an ALF (Adaptive Loop Filter) with an alternative band classifier or a new classifier derived from two or more classifiers. According to one method, a block value of an ALF classification block of a current block is determined. The block values are corresponding to a target block classification category using a lookup table. A filter output is derived by applying a target filter to a current reconstructed pixel in the ALF classification block, wherein the target filter is selected from a set of ALFs according to a target block classification category. A filtered reconstructed pixel is provided, wherein the filtered reconstructed pixel includes a filtered output. According to another method, a new classifier is generated from two or more different classifiers. A target filter is determined from a set of ALFs of the ALF classification block using a new classifier.

Description

[0001] Cross-references

[0002] The present invention is a non-provisional application of and claims priority to U.S. Provisional Patent Application No. 63 / 368,901, filed on July 20, 2022. The entire contents of the U.S. Provisional Patent Application are incorporated herein by reference. Technical Field

[0003] The present invention relates to a video coding system using an adaptive loop filter (ALF). In particular, the present invention relates to an ALF with an alternative brightness classifier. Background Art

[0004] Versatile Video Coding (VVC) is the latest international video coding standard jointly developed by the Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). The standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology - Coded representation of immersive media - Part 3: Versatile video coding, released in February 2021. VVC is an improvement on its predecessor HEVC (High Efficiency Video Coding) by adding more coding tools to improve coding efficiency, and can also handle various types of video sources, including 3D video signals.

[0005] Figure 1A An exemplary adaptive inter / intra video coding system including a loop process is described. For intra prediction, prediction data is derived from previously encoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed on the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112 and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118 and subsequent quantization (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 to be included in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information (such as motion and coding modes associated with intra prediction and inter prediction) and other information (such as parameters associated with loop filters applied to underlying image areas). Auxiliary information associated with intra prediction 110, inter prediction 112, and in-loop filter 130 is provided to entropy encoder 122, such as Figure 1AAs shown. When using inter-frame prediction mode, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added back to the prediction material 136 at reconstruction (REC) 128 to reconstruct the video material. The reconstructed video material can be stored in a reference picture buffer 134 and used to predict other frames.

[0006] like Figure 1A As shown, the input video data undergoes a series of processes in the coding system. Due to a series of processes, the reconstructed video data from REC128 may be subject to various impairments. Therefore, loop filter 130 is often applied to reconstructed video data before the reconstructed video data is stored in reference picture buffer 134 to improve video quality. For example, deblocking filter (DF), sample adaptive offset (SAO) and adaptive loop filter (ALF) can be used. It may be necessary to merge loop filter information into the bitstream so that the decoder can correctly restore the required information. Therefore, loop filter information is also provided to entropy encoder 122 to be merged into the bitstream. Figure 1A In , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to the High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B As shown, the decoder can use similar or identical functional blocks as the encoder, except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. Instead of entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bitstream into quantized transform coefficients and required encoding information (such as ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not need to perform mode search. Instead, the decoder only needs to generate intra-frame prediction based on the intra-frame prediction information received from entropy decoder 140. In addition, for inter-frame prediction, the decoder only needs to perform motion compensation (MC152) based on the inter-frame prediction information received from entropy decoder 140 without motion estimation.

[0008] According to VVC, similar to HEVC, the input picture is divided into non-overlapping square block areas called CTUs (Coding Tree Units). Each CTU can be divided into one or more coding units (CUs) of smaller size. The generated CU partitions can be square or rectangular. In addition, VVC divides CTUs into prediction units (PUs) as units for applying prediction processes such as inter-frame prediction, intra-frame prediction, etc.

[0009] In the present invention, an adaptive loop filter (ALF) with an alternative brightness classifier is disclosed for emerging video codec developments other than VVC. Summary of the invention

[0010] A method and apparatus for video encoding and decoding using an ALF (adaptive loop filter) is disclosed. According to the method, a reconstructed pixel is received, wherein the reconstructed pixel includes a current reconstructed pixel in a current block, and the current block corresponds to a luminance block. A block value is determined for an ALF classification block of the current block. The block value is mapped to a target block classification category using a lookup table. A filter output is derived by applying a target filter to the current reconstructed pixel in the ALF classification block, wherein the target filter is selected from a set of ALFs according to the target block classification category. A filtered reconstructed pixel is provided, wherein the filtered reconstructed pixel includes a filtered output.

[0011] In one embodiment, the block value corresponds to the sum of the current reconstructed pixels in the ALF classification block. In another embodiment, the block value corresponds to the median of the current reconstructed pixels in the ALF classification block. In another embodiment, the block value corresponds to a selected sample value of the current reconstructed pixels in the ALF classification block. In another embodiment, the block value of the ALF classification block is determined using a larger block that contains the ALF classification block.

[0012] In one embodiment, the correspondence between the block value and the lookup table is predefined. In another embodiment, the correspondence between the block value and the lookup table is adaptively determined. In another embodiment, the lookup table includes a number of entries that is 2 to the power of N. In another embodiment, a picture containing the current block is analyzed, and the correspondence between the block value and the lookup table is determined based on the results of the analysis. In another embodiment, the correspondence between the block value and the lookup table is non-uniform.

[0013] According to another method, a new classifier is generated from two or more different classifiers. A target filter is determined from a set of ALFs in an ALF classification block using the new classifier. A filtered output is derived by applying the target filter to a current reconstructed pixel in the ALF classification block. A filtered reconstructed pixel is provided, wherein the filtered reconstructed pixel includes the filtered output.

[0014] In one embodiment, the two or more different classifiers include a gradient classifier and a frequency band classifier. In one embodiment, a pre-merged gradient class is generated from the gradient classifier by merging at least two gradient classes among all gradient classes associated with the gradient classifier, and a pre-merged frequency band class is generated from the frequency band classifier by merging at least two frequency band classes among all frequency band classes associated with the frequency band classifier. In another embodiment, a new class is generated for a new classifier by combining the pre-merged gradient class and the pre-merged frequency band class.

[0015] In one embodiment, the number of all gradient categories corresponds to 25, and the number of pre-merged gradient categories is reduced to 5 by applying a divide-by-5 ​​operation or a modulo-5 operation for all gradient categories. In another embodiment, the number of pre-merged gradient categories is reduced to 5 according to the directionality or activity value of the ALF classification block.

[0016] In one embodiment, the number of all band classes corresponds to 25, and the number of pre-combined band classes is reduced to 5 by applying a divide-by-5 ​​operation or a modulo-5 operation for all band classes. In another embodiment, the number of pre-combined band classes is reduced to 5 by using a smaller multiplier.

[0017] In one embodiment, the two or more different classifiers include two or more gradient classifiers. In another embodiment, the two or more different classifiers include two or more frequency band classifiers. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1A An exemplary adaptive inter / intra video coding system incorporating loop processing is shown.

[0019] Figure 1B Shows Figure 1A The corresponding decoder of the encoder in .

[0020] Figure 2 The ALF filter shapes for chroma (left) and luma (right) components are shown.

[0021] Figure 3A -D shows the subsampled Laplace calculations of gv(3A), gh(3B), gd1(3C), and gd2(3D).

[0022] Figure 4A Shown is the placement of the CC-ALF relative to the other loop filters.

[0023] Figure 4B The diamond filters used for the chroma samples are indicated.

[0024] Figure 5 An example of a block-based gradient classifier for gradient classification according to an embodiment of the present invention is shown.

[0025] Figure 6 A flow chart of an exemplary video encoding and decoding system using alternative luminance band classification according to an embodiment of the present invention is shown.

[0026] Figure 7A flow chart of an exemplary video coding system using a new classifier derived based on two or more different classifiers according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] It will be readily understood that the components of the present invention as generally described and illustrated in the accompanying drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of the embodiments of the system and method of the present invention as shown in the figures is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. References throughout this specification to "an embodiment", "one embodiment" or similar language mean that a particular feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in one embodiment" appearing throughout this specification do not necessarily all refer to the same embodiment.

[0028] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. However, those skilled in the relevant art will recognize that the present invention may be practiced without one or more specific details, or using other methods, components, etc. In other cases, well-known structures or detailed descriptions of operations are not shown or are not shown to avoid obscuring aspects of the present invention. The illustrated embodiments of the present invention will be best understood with reference to the accompanying drawings, in which like parts are represented by like numbers throughout. The following description is intended to be exemplary only and simply illustrates certain selected embodiments of the apparatus and methods consistent with the present invention as claimed herein.

[0029] Adaptive loop filter in VVC

[0030] In VVC, an adaptive loop filter (ALF) with block-based filter adaptation is applied. For the luminance component, a filter is selected from 25 filters for each 4×4 block based on the direction and activity of the local gradient.

[0031] Filter shape

[0032] Using two diamond filter shapes (such as Figure 2 ). The 7×7 diamond 220 is applied to the luma component and the 5×5 diamond 210 is applied to the chroma components.

[0033] Block Classification

[0034] For the luminance component, each 4×4 block is classified into one of 25 categories. The classification metric C is based on its directionality D and activity value The quantitative value of is derived as follows:

[0035]

[0036] To calculate D and First, use the one-dimensional Laplacian to calculate the gradients in the horizontal, vertical, and two diagonal directions:

[0037]

[0038]

[0039] Where the indices i and j refer to the coordinates of the upper left sample in the 4×4 block, and R(i,j) represents the reconstructed sample at coordinate (i,j).

[0040] To reduce the complexity of block classification, the subsampled 1D Laplacian calculation is applied in the vertical direction ( Figure 3A ) and horizontal direction ( Figure 3B ). As shown in the figure. Figure 3C -D, the same subsampling position is used for gradient calculation in all directions ( Figure 3C g_d1 and Figure 3D g_d2 in ).

[0041] Then the maximum and minimum values ​​of the horizontal and vertical gradients D are set as:

[0042]

[0043] Set the maximum and minimum values ​​of the gradients in the two diagonal directions to:

[0044]

[0045] To derive the value of the directionality D, these values ​​are compared with each other and with two thresholds t1 and t2:

[0046] Step 1. If and For true,

[0047] D is set to 0.

[0048] Step 2. If Proceed to step 3; otherwise, proceed to step 4.

[0049] Step 3. If D is set to 2; otherwise D is set to 1.

[0050] Step 4. If D is set to 4; otherwise D is set to 3.

[0051] The activity value A is calculated as follows:

[0052]

[0053] A is further quantized to a range of 0 to 4 (inclusive), and the quantized value is represented as

[0054] No classification is applied to the chrominance components of the image.

[0055] Geometric transformation of filter coefficients and clipping values

[0056] Before filtering each 4×4 luma block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients f(k,l) and the corresponding filter clipping values ​​c(k,l) according to the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks where the ALF is applied more similar by aligning the directionality.

[0057] Introduced three geometric transformations: diagonal, vertical flip and rotation:

[0058] Diagonal:f D (k,l)=f(l,k),c D (k,l)=c(l,k),

[0059] Flip vertically:f V (k,l)=f(k,Kl-1),c V (k,l)=c(k,Kl-1),

[0060] Rotation:f R (k,l)=f(Kl-1,k),c R (k,l)=c(Kl-1,k),

[0061] Where K is the size of the filter and 0≤k,l≤K-1 are the coefficient coordinates, so position (0,0) is at the top left corner and position (K-1,K-1) is at the bottom right corner. The transform is applied to the filter coefficients f(k,l) and the clipping values ​​c(k,l) according to the gradient values ​​calculated for this block. The relationship between the transform and the four gradients in the four directions is summarized in the following table.

[0062] Table 1. Mapping of gradients and transformations computed for a block

[0063] Gradient Value Transform <![CDATA[g d2 <g d1 and h <g v ]]> No transformation <![CDATA[g d2 <g d1 and v <g h ]]> diagonal <![CDATA[g d1 <g d2 and h <g v ]]> Flip Vertically <![CDATA[g d1 <g d2 and v <g h ]]> Rotation

[0064] Filtration process

[0065] On the decoder side, when ALF is enabled in CTB, each sample R(i,j) in the CU is filtered to obtain the sample value R'(i,j), as shown below,

[0066]

[0067] Where f(k,l) represents the decoded filter coefficients, K(x,y) is the clipping function, and c(k,l) represents the decoded clipping parameters. The variables k and l vary between –L / 2 and L / 2, where L represents the filter length. The clipping function K(x,y)=min(y,max(-y,x)) corresponds to the function Clip3(-y,y,x). The clipping operation introduces nonlinearity, making the ALF more efficient by reducing the impact of neighboring sample values ​​that differ too much from the current sample value.

[0068] Cross-component adaptive loop filter

[0069] CC-ALF uses luma sample values ​​to refine each chroma component by applying an adaptive linear filter to the luma channel and then using the output of this filtering operation for chroma refinement. Figure 4A A system-level diagram of the CC-ALF process relative to the SAO, Luma ALF, and Chroma ALF processes is provided. Figure 4A As shown, each color component (i.e., Y, Cb, and Cr) is processed by its respective SAO (i.e., SAO Luma 410, SAO Cb 412, and SAO Cr 414). After SAO, ALF Luma 420 is applied to the SAO processed Luma and ALF Chroma 430 is applied to the SAO processed Cb and Cr. However, there are cross-component terms from Luma to Chroma components (i.e., CC-ALF Cb 422 and CC-ALF Cr 424). The outputs from the cross-component ALFs are added (using adders 432 and 434, respectively) to the outputs from ALF Chroma 430.

[0070] The filtering in CC-ALF is achieved by combining linear and diamond filters (e.g. Figure 4B This is accomplished by applying filters 440 and 442 in FIG. 4 to the luminance channel. Figure 4B In , the empty circles represent luma samples and the dot-filled circles represent chroma samples. One filter is used for each chroma channel, and its operation is expressed as:

[0071]

[0072] where (x, y) is the position of the chrominance component i being refined, (x Y ,y Y ) is the brightness position based on (x,y), S i is the filter support area in the luminance component, c i(x0, y0) represents the filter coefficient. In the above formula, ci can correspond to Cb or Crb.

[0073] like Figure 4B As shown, the luma filter support is the region concatenated with the current chroma sample after taking into account the spatial scaling factor between the luma and chroma planes.

[0074] In the VVC reference software, the CC-ALF filter coefficients are calculated by minimizing the mean squared error of each chroma channel relative to the original chroma content. To achieve this goal, the VTM (VVC Test Model) algorithm uses a coefficient derivation process similar to the chroma ALF. Specifically, a correlation matrix is ​​derived and the coefficients are calculated using a Cholesky decomposition solver to try to minimize the mean squared error metric. When designing the filters, up to 8 CC-ALF filters can be designed and transmitted per picture. The resulting filters are then indicated for each of the two chroma channels based on the CTU.

[0075] Other features of CC-ALF include:

[0076] ●The design adopts 3x4 diamond shape and comes with 8 filters.

[0077] ● Seven filter coefficients are transmitted in APS.

[0078] ●Each transmitted coefficient has a 6-bit dynamic range and is limited to power-of-2 values.

[0079] • Deriving the eighth filter coefficients at the decoder such that the sum of the filter coefficients is equal to zero.

[0080] ●APS can be referenced in the title.

[0081] ●CC-ALF filter selection is controlled at the CTU level for each chroma component.

[0082] ●Border filling of horizontal virtual borders uses the same memory access mode as luma ALF.

[0083] As an additional feature, the reference encoder can be configured to enable some basic subjective adjustments via a profile. When enabled, VTM will weaken the application of CC-ALF in areas encoded with high QP and close to mid-grey or containing a lot of luminance high frequencies. Algorithmically, this is achieved by disabling CC-ALF in CTUs that meet any of the following conditions:

[0084] ●The slice QP value minus 1 is less than or equal to the base QP value.

[0085] The number of chroma samples with local contrast greater than (1<<(bitDepth–2))–1 exceeds the CTU height, where local contrast is the difference between the maximum and minimum luma sample values ​​within the filter support area.

[0086] More than a quarter of the chroma samples are between (1<<(bitDepth–1))–16 and (1<<(bitDepth–1))+16

[0087] The motivation for this feature is to provide some assurance that CC-ALF does not amplify artifacts introduced earlier in the decoding path (this is primarily due to the fact that VTM does not currently explicitly optimize for chroma subjective quality). It is expected that alternative encoder implementations may not use this feature or include alternative strategies appropriate to their coding characteristics.

[0088] Filter parameter signal

[0089] The ALF filter parameters are signaled in an Adaptation Parameter Set (APS). In one APS, up to 25 groups of luma filter coefficients and clipping value indices and up to 8 groups of chroma filter coefficients and clipping value indices can be signaled. To reduce bit overhead, filter coefficients of different classifications of luma components can be merged. In the slice header, the index of the APS for the current slice is signaled.

[0090] The clipping value index decoded from the APS allows the determination of the clipping values ​​using the clipping value tables for both luma and chroma components. These clipping values ​​depend on the internal bit depth. More precisely, the clipping values ​​are obtained by the following formula:

[0091] AlfClip = {round(2 B-α*n )forn∈[0..N-1]}

[0092] Where B is equal to the internal bit depth, α is a predefined constant value equal to 2.35, and N is equal to 4, which is the number of clipping values ​​allowed in VVC. AlfClip is then rounded to the nearest value in the power of 2 format.

[0093] In the slice header, up to 7 APS indices can be signaled to specify the luma filter bank used for the current slice. The filtering process can be further controlled at the CTB level. A flag is always signaled to indicate whether the ALF is applied to the luma CTB. The luma CTB can select a filter bank among 16 fixed filter banks and a filter bank from the APS. A filter bank index is signaled for the luma CTB to indicate which filter bank is applied. The 16 fixed filter banks are predefined and hard-coded in the encoder and decoder.

[0094] For chroma components, the APS index is signaled in the slice header to indicate the chroma filter set used for the current slice. At the CTB level, if there are multiple chroma filter sets in the APS, the filter index is sent for each chroma CTB.

[0095] The filter coefficients are quantized with a norm equal to 128. To limit the multiplication complexity, bitstream consistency is applied so that the coefficient values ​​in non-center positions should be in the range -27 to 27-1 (inclusive). The center position coefficients are not signaled in the bitstream and are considered equal to 128.

[0096] Adaptive loop filter in ECM

[0097] ALF Simplification

[0098] ALF gradient subsampling and ALF virtual boundary processing were removed. The block size for classification was reduced from 4x4 to 2x2. The filter size for signaling luma and chroma ALF coefficients was increased to 9x9.

[0099] ALF with fixed filter

[0100] To filter the luminance samples, three different classifiers (C0, C1, and C2) and three different sets of filters (F0, F1, and F2) are used. Sets F0 and F1 contain fixed filters whose coefficients are trained for classifiers C0 and C1. The filter coefficients in F2 are represented by the signal. i Which filter in is used by the classifier C i The category C assigned to this sample i Decide.

[0101] Filtering

[0102] First, two 13x13 diamond fixed filters F0 and F1 are applied to derive two intermediate samples R0(x,y) and R1(x,y). Then, F2 is applied to R0(x,y), R1(x,y) and adjacent samples to derive filtered samples:

[0103]

[0104] where f i,j is the crop difference between the adjacent sample and the current sample R(x,y), g i YesR i-20 The cropped difference between (x,y) and the current sample. Signaling the filter coefficients c i ,i=0,…21.

[0105] Classification

[0106] Based on the directionality Di and activity value Category C i Assign to each 2x2 block:

[0107]

[0108] Among them, M D,i Indicates directionality D i The total number of .

[0109] As in VVC, the horizontal, vertical, and two diagonal gradient values ​​of each sample are calculated using a one-dimensional Laplacian operator. The sum of the sample gradients within a 4×4 window covering the target 2×2 block is used for classifier C0, while the sum of the sample gradients within a 12×12 window is used for classifiers C1 and C2. The sum of the horizontal, vertical, and two diagonal gradients is expressed as and Directionality D i Determine by comparing the following formula and a set of thresholds

[0110]

[0111] Directivity D2 is derived in VVC using thresholds 2 and 4.5. For D0 and D1, first calculate the horizontal / vertical edge strength and diagonal edge strength Use threshold Th = [1.25, 1.5, 2, 3, 4.5, 8]. If The edge strength is 0; otherwise, is satisfied The largest integer. The edge strength is 0; otherwise, is satisfied The maximum integer of . That is, when horizontal / vertical edges dominate, use Table 2A to derive D i ; Otherwise, the diagonal edges dominate, D i Derived using Table 2B.

[0112] Table 2A: and To D i Mapping

[0113]

[0114] Table 2B: and To D i Mapping

[0115]

[0116] In order to get Vertical and horizontal gradient A i The sum of is mapped to the range from 0 to n, where for For example, n is equal to 4. and For example, n is equal to 15.

[0117] In ALF_APS, up to 4 luma filter groups can be signaled, each with up to 25 filters.

[0118] In the present invention, a technique for improving the ALF performance is disclosed as follows.

[0119] ALF with alternative brightness classifier

[0120] In the ECM (Muhammed Coban et al., "Algorithmic Description of Enhanced Compression Model 5 (ECM 5)", Joint Video Experts Group (JVET) of ITU-T SG16WP 3 and ISO / IEC JTC 1 / SC 29, 26th Meeting, by teleconference, April 20-29, 2022, document: JVET-Z2025) ALF, the filter bank can choose between a VVC-like gradient-based classifier and a new band classifier. Different classifiers provide different classification results for blocks in a frame, and different classification results lead to different derived filter banks and corresponding optimized sum-of-squared distortion (SSD) values. Therefore, adding more classifiers enables the ALF to explore more ways to assign filters to blocks. In this proposal, several classification schemes and methods are elaborated.

[0121] ALF Band Classifier

[0122] In the ECM ALF band classifier, there are 25 band classes for classification. Each 2x2 sample sum will be multiplied by 25 and right shifted by a number of bits to determine the band class for each 2x2 block as follows:

[0123] Band Class C = (2x2 sample value sum * 25) >> (InputBitDepth + 2)

[0124] The 2x2 blocks for band classification are referred to as ALF classification blocks in this disclosure. In one embodiment, each 2x2 sample value and is first mapped to a lookup table, and then the band class of each 2x2 block is determined from the lookup table, as follows:

[0125] C'=(2x2 sample value sum*K)>>(InputBitDepth+2),

[0126] Band class C = LUT[C'],

[0127] Where K is a value.

[0128] The frequency band distribution in the frequency band classifier can be predefined or adaptively changed.

[0129] The 2x2 sample values ​​and of the above-mentioned band classification are used as an example for explanation. The disclosed alternative band classification should not be interpreted as limiting the present invention. As shown in the following part of this disclosure, in addition to the sample values ​​and, other representative block values ​​can also be used for band classification. In addition, the present invention is not limited to the ALF classification block size of 2x2. Other ALF classification block sizes can also be used to practice the present invention.

[0130] In another embodiment, the entries of the lookup table may be distributed non-uniformly (or unevenly). For example, some frequency band categories appear less frequently in the lookup table. An example of a lookup table design is as follows:

[0131] LUT

[50] ={0,0,0,0,1,2,3,4,5,6,7,8,8,9,9,10,10,11,11,12,12,

[0132] 13,13,14,14,15,15,16,16,17,17,18,18,19,19,20,20,21,21,21,

[0133] 22,22,22,23,23,23,23,23,23,23,23,24,24,24}.

[0134] In another embodiment, before performing frequency band classification, the entire image is first analyzed, and the frequency band distribution is adaptively determined according to some rules, such as following the sample and distribution in the image.

[0135] In another embodiment, the number of entries in the lookup table corresponds to 2 to the power of N, and N is a positive integer. In other words, the lookup table includes 2N. Therefore, the frequency band class is calculated as follows:

[0136] C'=(2x2 sample value sum)>>(InputBitDepth-M),

[0137] Band class C = LUT[C'],

[0138] Where M is a value.

[0139] In another embodiment, for each 2x2 block, the median sample value within each 2x2 block is calculated and used to replace the 2x2 sample sum.

[0140] In another embodiment, for each 2x2 block, instead of the 2x2 sample sum, one of the four samples is used to derive the band class.

[0141] In another embodiment, for each 2x2 block, a larger window is used to derive the band class. For example, a 4x4 sample sum is calculated and used to derive the band class for the center 2x2 block.

[0142] ALF Gradient-Band Classifier

[0143] In ECM ALF classifier, there are two different classifiers used for classification: gradient classifier and band classifier. Each 2x2 block will be assigned a class by either gradient classifier or band classifier.

[0144] In one embodiment, a new classifier called gradient-band classifier is added by utilizing the classifications from gradient classifier and band classifier.

[0145] In the above embodiment, the classifications from the gradient classifier and the band classifier are first pre-merged into a smaller number of classes, and new classifications are formed by combining the two pre-merged classes. For example, 25 classes from the gradient classifier and the band classifier are pre-merged into 5 classes through some calculations (such as dividing by 5 or modulo 5), and then the pre-merged classes are combined to form 25 new classes. Some exemplary equations are shown below:

[0146] 1: Gradient band category C = (gradient category C / 5)*5+(band category C / 5)

[0147] 2: Gradient band category C = (gradient category C / 5)*5+(band category C%5)

[0148] 3: Gradient band category C = (gradient category C% 5) * 5 + (band category C / 5)

[0149] 4: Gradient band category C = (gradient category C% 5) * 5 + (band category C% 5)

[0150] 5: Gradient band category C = (band category C / 5)*5+(gradient category C / 5)

[0151] 6: Gradient band category C = (band category C% 5) * 5 + (gradient category C / 5)

[0152] In the above embodiment, the classification of the gradient-band classifier is obtained by combining the directionality (D) or activity value (A) from the gradient classifier with the classification of the band classifier with a smaller multiplier. For example, the directionality (D) and the band classifier are combined with a multiplier of 5 to form the classification of the gradient-band classifier. An exemplary equation is shown below:

[0153] Example 1:

[0154] Band Class C = (2x2 sample value sum * 5) >> (InputBitDepth + 2),

[0155] Gradient band class C = (gradient directionality D)*5+(band class C).

[0156] Example 2:

[0157] Band Class C = (2x2 sample value sum * 5) >> (InputBitDepth + 2),

[0158] Gradient band category C = (band category C)*5+(gradient activity value A).

[0159] In the above embodiments, the geometric transformation of the gradient-band classifier may be inherited from the gradient classifier or inherited from the band classifier or set as a predefined transformation.

[0160] ALF Ensemble Classifier

[0161] In one embodiment, a new classifier called an integrated classifier is added by utilizing classifications of two or more different classifiers.

[0162] For example:

[0163] two or more gradient classifiers,

[0164] two or more band classifiers,

[0165] One or more gradient classifiers and one or more frequency band classifiers.

[0166] In the above embodiment, the classifications from two or more different classifiers are first pre-merged into a smaller number of classifications, and a new classification is formed by combining the two or more pre-merged classifications. For example, the classifications from two gradient classifiers are first pre-merged into smaller classifications, and a new classification is formed by combining the two or more pre-merged classifications. An exemplary equation is shown below:

[0167] Fixed filter gradient classifier 0: C0 = A0*56 + D0 (56*16 = 896 categories).

[0168] Fixed filter gradient classifier 1: C1 = A1*56+D1 (56*16=896 categories).

[0169] Example 1: Integration category C = A0 / 4*7+D1 / 8=> 4 activity values ​​from C0 * 7 directionality from C1 = 28 categories,

[0170] Example 2: Integration category C = A1 / 4*7+D0 / 8=>28 categories.

[0171] In the above embodiment, the classification of the integrated classifier is derived by combining the directionality (D) from two or more gradient classifiers and the activity values ​​(A) from two or more gradient classifiers. An example equation is shown below:

[0172] Fixed filter gradient classifier 0: C0 = A0*56+D0 (56*16 = 896 categories)

[0173] Fixed filter gradient classifier 1: C1 = A1*56 + D1 (56*16 = 896 categories)

[0174] Example 1: Integrated category C = ((A0+A1) / 2) / 4+((D0+D1) / 2) / 8=>28 categories.

[0175] ALF Gradient Classifier with Block-Based Gradients

[0176] In one embodiment, the gradient calculation is performed on a block basis rather than a sample basis. Figure 5 As shown, the sample average or sum of each 2x2 block is first calculated, and then the gradient of each block is calculated using the average or sum of the current block and its neighboring blocks. Figure 5 , a 2x2 block 510 is shown on the left, where each small square corresponds to a sample and the 2x2 block boundary is shown as a thicker line. A block-based gradient scheme 520 is shown on the right, where each square corresponds to a block, where block values ​​(e.g., sample sums or sample averages) are derived from corresponding samples in the 2x2 block. For example, the block value of the center block indicated by the gray area in the block 520 is derived from 4 samples of the corresponding 2x2 block within the 2x2 block 510.

[0177] The aforementioned proposed method may be implemented in an encoder and / or a decoder. For example, the proposed method may be implemented in a loop filter module of an encoder and / or a loop filter module of a decoder.

[0178] Any of the above ALF methods can be implemented in an encoder and / or decoder. For example, any of the proposed methods can be implemented in a loop filter module (e.g. Figure 1A and Figure 1B Alternatively, any of the proposed methods may be implemented as a circuit coupled to an inter-frame coding module and / or a motion compensation module of an encoder, or a merge candidate derivation module of a decoder. The ALF method may also be implemented using executable software or firmware program code stored on a medium (e.g., a hard disk or flash memory) for use in a CPU (central processing unit) or a programmable device (e.g., a DSP (digital signal processor) or an FPGA (field programmable) gate array).

[0179] Figure 6 A flow chart of an exemplary video encoding and decoding system using alternative luma band classification according to an embodiment of the present invention is shown. The steps shown in the flow chart can be implemented as executable program code on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flow chart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flow chart. According to the method, a reconstructed pixel is received in step 610, wherein the reconstructed pixel includes a current reconstructed pixel in a current block, and the current block corresponds to a luma block. In step 620, a block value of an ALF classification block of the current block is determined. In step 630, a lookup table is used to correspond the block value to a target block classification category. In step 640, a filter output is derived by applying a target filter to the current reconstructed pixel in the ALF classification block, wherein the target filter is selected from a set of ALFs according to the target block classification category. A filtered reconstructed pixel is provided in step 650, wherein the filtered reconstructed pixel includes a filtered output.

[0180] Figure 7 A flow chart of an exemplary video codec system using a new classifier derived based on two or more different classifiers according to an embodiment of the present invention is shown. According to the method, a reconstructed pixel is received in step 710, wherein the reconstructed pixel includes a current reconstructed pixel in a current block, and the current block corresponds to a luminance block. In step 720, a new classifier is generated from two or more different classifiers. In step 730, a target filter is determined from a set of ALFs of the ALF classification block using the new classifier. In step 740, a filter output is obtained by applying the target filter to the current reconstructed pixel in the ALF classification block. In step 750, a filtered reconstructed pixel is provided, wherein the filtered reconstructed pixel includes the filter output.

[0181] The flowchart shown is intended to illustrate an example of video encoding according to the present invention. Without departing from the spirit of the present invention, those skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to implement the present invention. In the present disclosure, specific syntax and semantics have been used to illustrate examples to implement embodiments of the present invention. Without departing from the spirit of the present invention, those skilled in the art may implement the present invention by replacing syntax and semantics with equivalent syntax and semantics.

[0182] The above description is provided to enable those of ordinary skill in the art to practice the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but to conform to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are illustrated to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be implemented.

[0183] The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, one embodiment of the present invention can be one or more circuits integrated into a video compression chip or a program code integrated into the video compression software to perform the processing described herein. The embodiments of the present invention can also be a program code to be executed on a digital signal processor (DSP) to perform the processing described herein. The present invention can also be related to many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software codes or firmware codes that define the specific methods embodied by the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, the different code formats, styles and languages ​​of the software code and other ways of configuring the code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0184] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples should be considered in all respects as illustrative and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes that fall within the meaning and scope of equivalents to the scope of the claims should be included within their scope.

Claims

1. A method for adaptive loop filter (ALF) processing for reconstructing video, the method comprising: receiving a reconstructed pixel, wherein the reconstructed pixel comprises a current reconstructed pixel in a current block, the current block corresponding to a luma block; determining the block value of the ALF classification block of the current block; Using a lookup table to map the block value to a target block classification category; deriving a filter output by applying a target filter to a current reconstructed pixel in an ALF classified block, wherein the target filter is selected from an ALF set according to the target block classification category; and A filtered reconstructed pixel is provided, wherein the filtered reconstructed pixel comprises the filtered output.

2. The method according to claim 1, characterized in that The block value corresponds to the sum of the currently reconstructed pixels in the ALF classification block, or the block value corresponds to the median of the currently reconstructed pixels in the ALF classification block.

3. The method according to claim 1, characterized in that The block value corresponds to a selected sample value of a current reconstructed pixel in the ALF classification block.

4. The method according to claim 1, characterized in that: The number of entries of the lookup table corresponds to 2 to the power of N, where N is a positive integer.

5. The method according to claim 1, characterized in that The block value of the ALF classification block is determined using a larger block that includes the ALF classification block.

6. The method according to claim 1, characterized in that The correspondence between the block values ​​and the lookup table is predefined.

7. The method according to claim 1, characterized in that A correspondence between the block value and the lookup table is adaptively determined.

8. The method according to claim 7, characterized in that An analysis is applied to the picture including the current block, and a correspondence between the block value and the lookup table is determined based on a result of the analysis.

9. The method according to claim 1, characterized in that: The mapping between the block values ​​and the lookup table is non-uniform.

10. An apparatus for adaptive loop filter (ALF) processing of reconstructed video, the apparatus comprising one or more electronic circuits or processors arranged to: Receive reconstructed pixels, where The reconstructed pixels include current reconstructed pixels in a current block, the current block corresponding to a luminance block; Determining a block value of an ALF classification block of the current block; Using a lookup table to map the block value to a target block classification category; deriving a filter output by applying a target filter to a current reconstructed pixel in the ALF classified block, wherein the target filter is selected from an ALF set according to the target block classification category; and A filtered reconstructed pixel is provided, wherein the filtered reconstructed pixel comprises the filtered output.

11. A method for adaptive loop filter (ALF) processing for reconstructing video, the method comprising: receiving a reconstructed pixel, wherein the reconstructed pixel comprises a current reconstructed pixel in a current block, the current block corresponding to a luma block; Generate a new classifier from two or more different classifiers; Determine a target filter from a set of ALFs for an ALF classification block using the new classifier; deriving a filtered output by applying the target filter to a current reconstructed pixel in the ALF classification block; and A filtered reconstructed pixel is provided, wherein the filtered reconstructed pixel comprises the filtered output.

12. The method according to claim 11, characterized in that The two or more different classifiers include a gradient classifier and a frequency band classifier.

13. The method according to claim 12, characterized in that A pre-merged gradient class is generated from the gradient classifier by merging at least two gradient classes out of all gradient classes associated with the gradient classifier, and a pre-merged band class is generated from the band classifier by merging at least two band classes out of all band classes associated with the band classifier.

14. The method according to claim 13, characterized in that A new class is generated for the new classifier by combining the pre-merged gradient class and the pre-merged band class.

15. The method according to claim 13, characterized in that The number of all gradient categories corresponds to 25, and the number of pre-merged gradient categories is reduced to 5 by applying a divide-by-5 ​​operation or a modulo-5 operation to all gradient categories.

16. The method according to claim 13, characterized in that The number of all band classes corresponds to 25, and the number of pre-merged band classes is reduced to 5 by applying a divide-by-5 ​​operation or a modulo-5 operation to all band classes.

17. The method according to claim 13, characterized in that The number of all gradient categories corresponds to 25, and the number of pre-merged gradient categories is reduced to 5 according to the directionality or activity value of the ALF classification block.

18. The method according to claim 13, characterized in that The total number of all band classes corresponds to 25, and the number of pre-merged band classes is reduced to 5 by using a smaller multiplier.

19. The method according to claim 11, characterized in that The two or more different classifiers include two or more gradient classifiers.

20. The method according to claim 11, characterized in that The two or more different classifiers include two or more frequency band classifiers.

Images