Adaptive loop filter with adaptive filtering strength

Abstract

A method for performing an adaptive loop filter in a video system is provided. A video codec receives data for a block of pixels to be encoded or decoded as a current block of a current picture of a video. A video codec receives a set of samples of a current block. A video codec classifies a set of samples into a plurality of subsets of samples. The video codec filters the received sample set to generate a set of correction values. The video codec applies a set of filtering strengths to weight the resulting set of correction values. The video codec adds the weighted set of correction values to the received set of samples as filtered samples for the current block.

Description

[0001] Cross-references

[0002] This application is part of a non-provisional application claiming priority to U.S. Provisional Patent Application No. 63 / 368,899 filed on July 20, 2022, and U.S. Provisional Patent Application No. 63 / 368,905 filed on July 20, 2022. The above applications are hereby incorporated by reference in their entirety. Technical Field

[0003] The present invention generally relates to video coding and decoding, and more particularly to a method for coding and decoding video images using an adaptive loop filter (ALF). Background Art

[0004] Unless otherwise stated herein, the methods described in this section are not background art for the claims and are not admitted to be background art by inclusion of this section.

[0005] High-Efficiency Video Coding (HEVC) is an international video codec standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion compensated DCT-like transform codec architecture. The basic unit of compression is called a coding unit (CU), which is a 2N×2N square pixel block. Each CU can be recursively divided into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile Video Coding (VVC) is the latest international video codec standard developed by ITU-T SG16WP3 and ISO / IEC JTC1 / SC29 / WG11 Joint Video Expert Team (JVET). The input video signal is predicted from the reconstructed signal, which is derived from the coded and decoded image region. The prediction residual signal is processed by block transform. The transform coefficients are quantized and entropy encoded and decoded together with other side information in the bitstream. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal after inverse transforming the dequantized transform coefficients. The reconstructed signal is also processed by loop filtering to remove coding artifacts. The decoded image is stored in the frame buffer and used to predict future images in the input video signal.

[0007] In VVC, the coded image is divided into non-overlapping square block areas represented by related coding tree units (CTUs). The leaf nodes of the coding tree correspond to coding units (CUs). The coded image can be represented by a set of slices, each slice including an integer number of CTUs. Each CTU in a slice is processed in raster scan order. A bi-predicted (B) slice can be decoded to predict the sample values ​​of each block using intra-picture prediction or inter-picture prediction with up to two motion vectors and reference indices. A predicted (P) slice is decoded to predict the sample values ​​of each block using intra-picture prediction or inter-picture prediction with up to one motion vector and reference index. Intra-picture (I) slices are decoded using only intra-picture prediction.

[0008] Using a quadtree (QT) with a nested multi-type-tree (MTT) structure, a CTU can be split into one or more non-overlapping codec units (CUs) to accommodate various local motion and texture features. The CU is further divided into smaller CUs using one of five partition types: quadtree partition, vertical binary tree partition, horizontal binary tree partition, vertical center-side ternary tree partition, and horizontal center-side ternary tree partition.

[0009] Each CU contains one or more prediction units (PUs). The prediction unit, together with the associated CU syntax, serves as a basic unit for indicating prediction sub-information. The values ​​of the related pixel samples within the PU are predicted using a specified prediction process. Each CU may contain one or more transform units (TUs) representing prediction residual blocks. The transform unit (TU) includes a transform block (TB) of luma samples and two corresponding transform blocks of chroma samples, each TB corresponding to a residual block sample of a color component. An integer transform is applied to the transform block. The layer values ​​of the quantized coefficients are entropy encoded and decoded in the bitstream along with other side information. The terms Coding Tree Block (CTB), Coding Block (CB), Prediction Block (PB), and Transform Block (TB) are defined to specify a 2D sample array of a color component associated with a CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU includes a luma CTB, two chroma CTBs, and related syntax elements. Similar relationships apply to CU, PU, ​​and TU.

[0010] For each inter-predicted CU, motion parameters including motion vector, reference picture index and reference picture list usage index and additional information are used to generate inter-predicted samples. Motion parameters can be indicated in an explicit or implicit way. When the CU is encoded and decoded using skip mode, the CU is associated with one PU and has no valid residual coefficients, no coded motion vector delta or reference picture index. A merge mode is specified, in which the motion parameters of the current CU are obtained from neighboring CUs, including spatial candidates and temporal candidates, as well as the additional schedule introduced in VVC. Merge mode can be applied to any inter-predicted CU. An alternative to merge mode is the explicit transmission of motion parameters, in which the motion vector, the corresponding reference picture index and reference picture list usage flag for each reference picture list and other required information are explicitly indicated for each CU. Summary of the invention

[0011] The following summary of the invention is provided for illustration only and is not intended to be limiting in any way. That is, the following summary of the invention is provided to introduce the concepts, highlights, benefits and advantages of the novel and non-obvious technologies described herein. Selected embodiments will be further described in the following specific examples. Therefore, the following summary of the invention is not intended to identify the essential characteristics of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.

[0012] Some embodiments of the present disclosure provide methods for performing an adaptive loop filter in a video system. A video decoder receives data of a pixel block of a current block of a current image to be encoded or decoded as a video. A video encoder receives a set of samples of the current block. The video encoder classifies the sample set into multiple subsets of samples. The video encoder filters the received sample set to generate a set of correction values. The video encoder applies a set of filter strengths to weight the generated correction value set. The video decoder adds the weighted correction value set to the received sample set as filtered samples of the current block.

[0013] In some embodiments, each sample in the set of samples can be classified into one of a plurality of sample subsets based on a relationship of the sample to its neighbors and based on an edge offset pattern of a current block. In some embodiments, the set of samples is classified into a plurality of subsets based on a predetermined pattern. The predetermined pattern can be selected from a plurality of predetermined patterns (e.g., one of 15 on / off 2x2 patterns). In some embodiments, the set of samples is classified into a plurality of subsets by a model signaled in a bitstream of the encoded video, or trained by online or offline data.

[0014] In some embodiments, the filter is an adaptive loop filter (ALF) of a video coding solution system. This filtering can be based on a set of filter taps that receive input, the input including (i) samples within the current block, (ii) samples adjacent to the current block, or (iii) residual samples generated based on a prediction of the current block. The filtering can be based on a set of filter taps that receive input, the input including (i) samples generated by a deblocking filter (DBF) or a sample adaptive offset (SAO) filter or (ii) reconstructed samples of the current block without deblocking filtering.

[0015] The encoder (at block 840) applies a set of filter strengths to weight the generated set of correction values. In some embodiments, filtering of each subset (or category) of samples may be turned on or off individually. In some embodiments, filtering of each subset of samples may be weighted by a corresponding filter strength. In some embodiments, the encoder may turn off filtering of a subset of samples by setting the corresponding filter strength to zero (or turn on filtering of a subset by setting the corresponding filter strength to a non-zero value). In some embodiments, the set of filter strengths is indicated at a first, higher level of the video (e.g., a slice level), and whether to apply the filter strength is determined at a second, lower level of the video (e.g., a CTB level). In some embodiments, the set of filter strengths is determined by applying a filter strength model to the set of samples, wherein the filter strength model is signaled in a bitstream of the encoded video. In some embodiments, filtering is based on a set of filter taps receiving an input comprising (i) samples within a current block, (ii) samples adjacent to the current block, or (iii) residual samples generated based on a prediction of the current block. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings are included to provide a further understanding of the present invention and are incorporated into and constitute a part of the present invention. The drawings illustrate embodiments of the present invention and together with the description are used to explain the principles of the present invention. It should be noted that the drawings are not necessarily drawn to scale, as some components may be shown out of proportion to the size in the actual embodiment in order to clearly illustrate the concepts of the present invention.

[0017] Figure 1A -B shows two diamond filter shapes for the adaptive loop filter (ALF).

[0018] Figure 2 A system level schematic diagram of a loop filter is shown, wherein reconstructed or decoded samples are filtered or processed by a deblock filter (DBF), a sample adaptive offset (SAO), and an adaptive filter (ALF).

[0019] Figure 3 Filtering in cross-component-ALF (CC-ALF) is shown.

[0020] Figure 4A -B shows the switch selection pattern.

[0021] Figure 5A -B conceptually illustrates the classification of ALF samples for applying filtering strength.

[0022] FIG. 6 shows an example video encoder implementing a loop filter.

[0023] Figure 7 Portions of a video encoder implementing ALF with adaptive filtering strength are shown.

[0024] Figure 8 The process of applying adaptive filtering strength in an adaptive loop filter (ALF) is conceptually illustrated.

[0025] Fig. 9 An example video decoder that may implement an adaptive loop filter (ALF) is shown.

[0026] Fig.10 Part of a video decoder implementing ALF with adaptive filtering strength is shown.

[0027] Fig.11A process for applying adaptive filtering strength in an adaptive loop filter (ALF) is conceptually illustrated.

[0028] Fig.12 An electronic system for implementing some embodiments of the present application is schematically shown. DETAILED DESCRIPTION

[0029] In the detailed description below, many specific details are set forth by way of example to provide a comprehensive understanding of the relevant teachings. Any variants, derivatives and / or extensions based on the teachings of the present application are within the scope of protection of the present application. In some cases, in order to avoid unnecessary confusion of various aspects of the teachings of the present application, well-known methods, processes, components and / or circuits related to one or more example embodiments disclosed herein may be described at a relatively high level without details.

[0030] I. Adaptive Loop Filter (ALF)

[0031] A. Filter Shape

[0032] ALF is a loop filtering technique used in video codec standards such as VVC. It is a block-based filter that minimizes the mean square error between the original and reconstructed samples. For the luminance component, one of 25 filters is selected for each 4×4 block based on the direction and activity of the local gradient. Figure 1A -B shows two diamond filter shapes used for ALF. Figure 1A A 7x7 diamond is shown, applied to the luma component. Figure 1B A 5x5 diamond is shown, applied to the chroma components.

[0033] B. Block Classification

[0034] For the luminance component, each 4×4 block is classified into one of 25 classes. The class index C is based on its directionality D and activity The quantitative value of is derived as follows:

[0035]

[0036] In order to calculate D and First, use the one-dimensional Laplace to calculate the gradient in the horizontal, vertical, and two diagonal directions:

[0037]

[0038]

[0039] Where the indices i and j refer to the coordinates of the top left sample within the 4×4 block, and R(i,j) refers to the reconstructed sample at coordinate (i,j). In order to reduce the complexity of block classification, a sub-sampled one-dimensional Laplacian calculation is used. The same sub-sampling position can be used for gradient calculation in all directions (the sub-sampling position can be used for vertical gradient, horizontal gradient or diagonal gradient). The maximum and minimum values ​​of D for the horizontal and vertical gradients are set to:

[0040]

[0041] The maximum and minimum values ​​of the gradients in the two diagonal directions are set to:

[0042]

[0043] To derive the value of the directivity D, these values ​​are compared with each other and with two thresholds t 1 and t 2 For comparison:

[0044] Step 1. If and is true, then D is set to 0.

[0045] Step 2. If Then continue from step 3; otherwise continue from step 4.

[0046] Step 3. If If yes, set D to 2; otherwise, set D to 1.

[0047] Step 4. If Then set D to 4; otherwise, set D to 3.

[0048] The activity value A is calculated as:

[0049]

[0050] A is further quantized to the range of 0 to 4, inclusive, and the quantized values ​​are expressed as For the chrominance components in the image, no classification method is applied.

[0051] C. Geometric transformation of filter coefficients and clipping values

[0052] Before filtering each 4×4 luma block, geometric transformations such as rotation, or diagonal and vertical flipping are applied to the filter coefficients f(k,l) and the corresponding filter clipping values ​​c(k,l) according to the gradient values ​​calculated for the block. This is equivalent to applying these transformations to the samples in the filter support region. The scheme is to make different blocks to which the ALF is applied more similar by aligning the directionality of different blocks. There are three geometric transformations, including diagonal flipping, vertical flipping, and rotation, which are described as:

[0053] Diagonal: f D (k,l)=f(l,k),c D (k,l)=c(l,k),

[0054] Flip vertically: f V (k,l)=f(k,Kl-1),c V (k,l)=c(k,Kl-1)

[0055] Rotation: f R (k,l)=f(Kl-1,k),c R (k,l)=c(Kl-1,k)

[0056] Where K is the size of the filter, 0≤k,l≤K-1 are the coefficient coordinates, so that the position (0,0) is in the upper left corner and the position (K-1,K-1) is in the lower right corner. The transform is applied to the filter coefficients f(k,l) and the clipping value c(k,l) according to the gradient value calculated for the block. The relationship between the transform and the four gradients in four directions is summarized in Table 1 below, which shows the mapping of the gradient calculated for a block and the transform.

[0057] Table 1:

[0058]

[0059]

[0060] D. Filtering process

[0061] On the decoder side, when ALF is enabled for CTB, each sample R'(i,j) in the CU is filtered to obtain the sample value R'(i,j) as shown below:

[0062]

[0063] where f(k,l) represents the decoded filter coefficients, K(x,y) is the clipping function, and c(k,l) represents the decoded clipping parameters. The variables k and l vary between -L / 2 and L / 2, where | represents the filter length. The clipping function K(x,y) = min(y,max(-y,x)), which corresponds to the function Clip3(-y,y,x). The clipping operation introduces nonlinearity to make the ALF more efficient by reducing the impact of neighboring sample values ​​that differ too much from the current sample value.

[0064] E.CC-ALF

[0065] CC-ALF can use luma sample values ​​to refine each chroma component by applying an adaptive linear filter to the luma channel and then using the output of this filtering operation for chroma refinement. Figure 2 A system level diagram of a loop filter is shown, wherein reconstructed or decoded samples 210 are filtered or processed by DBF, SAO and ALF. The reconstructed or decoded samples 210 may be generated from a prediction signal and a residual signal of a current block.

[0066] The figure shows the displacement of CC-ALF relative to other loop filters. Specifically, the luma component of the SAO output is processed by a luma ALF process (ALF Y) and a pair of cross-component ALF processes (CC-ALF Cb and CC-ALF Cr). The two cross-component ALF processes generate a Cb component and a cross-component offset of the Cb component to add it to the output of the chroma ALF process (ALF Chroma) to generate the ALF output of the chroma component. The luma component and chroma component of the ALF output are then stored in the reconstructed or decoded image buffer 290 for predictive coding and decoding of subsequent blocks.

[0067] Figure 3 Filtering in CC-ALF is shown, which is accomplished by applying a linear diamond filter 310 to the luma channel. One filter is used per chroma channel, and the operation is represented as:

[0068]

[0069] where (x, y) is the position of the chrominance component i being refined, (x Y ,y Y ) is the brightness position based on (x,y), S i is the filter support area in the luminance component, c i (x 0 ,y 0 ) represents the filter coefficient. Figure 3As shown, after taking into account the spatial scaling factor between the luma plane and the chroma plane, the luma filter support area is the area that is collocated with the current chroma sample.

[0070] The CC-ALF filter coefficients can be calculated by minimizing the mean square error of each chroma channel with respect to the original chroma content. To implement this scheme, the algorithm can use a coefficient derivation process similar to that used for chroma ALF. Specifically, a correlation matrix is ​​derived and the coefficients are calculated using a Cholesky decomposition solver in an attempt to minimize the mean square error metric. When designing the filters, up to 8 CC-ALF filters can be designed and transmitted per image. The resulting filters are then indicated for each of the two chroma channels on a CTU basis.

[0071] CC-ALF filtering can use a 3x4 diamond with 8 filter taps and 7 filter coefficients transmitted in an Adaptation Parameter Set (APS) (which can be referenced in the slice header). Each transmitted coefficient has a 6-bit dynamic range and is limited to a power of 2 value. The 8th filter coefficient is derived at the decoder so that the sum of the filter coefficients is equal to 0. For each chroma component, CC-ALF filter selection can be controlled at the CTU level. Boundary padding for horizontal virtual boundaries can be the same memory access mode as luma ALF.

[0072] As an additional feature, the reference encoder can be used to enable some basic subjective tuning via a configuration file. When enabled, VTM attenuates the application of CC-ALF in areas that are encoded with a high QP and are close to mid-grey or contain a lot of luminance high frequencies. Algorithmically, this is done by disabling the application of CC-ALF in CTUs where any of the following conditions are true:

[0073] (i) The slice QP value minus 1 is less than or equal to the base QP value;

[0074] (ii) the number of chroma samples with a local contrast greater than (1<<(bitDepth–2))–1 exceeds the CTU height, where the local contrast is the difference between the maximum and minimum luminance sample values ​​within the filter support area;

[0075] (iii) More than one quarter of the chroma samples are in the range between (1<<(bitDepth–1))–16and(1<<(bitDepth–1))+16.

[0076] This is to ensure that CC-ALF does not amplify artifacts introduced earlier in the decoding path.

[0077] F. Filter parameter marking

[0078] ALF filter parameters are indicated in the APS. In one APS, up to 25 sets of luma filter coefficients and clipping value indices and up to 8 sets of chroma filter coefficients and clipping value indices can be indicated. In order to reduce bit overhead, filter coefficients of different classifications of luma components can be merged. In the slice header, the index of the APS for the current slice is indicated.

[0079] The clipping value index decoded from the APS allows to determine the clipping value using a table of clipping values ​​for luma and chroma components. These clipping values ​​depend on the internal bit depth. More precisely, the clipping value is obtained using the following formula:

[0080] ALFClip={round(2 B-α*n )for n∈[0..N-1]}

[0081] Where B is equal to the internal bit depth, α is a predefined constant value equal to 2.35, and N is equal to 4, which is the number of crop values ​​allowed in VVC. ALFClip is then rounded to the nearest value in power of 2 format.

[0082] In the slice header, up to 7 APS indices can be indicated to specify the luma filter set for the current slice. The filtering process can be further controlled at the CTB level. A flag is always indicated to indicate whether the ALF is applied to the luma CTB. The luma CTB can select a filter set from 16 fixed filter sets and a filter set from the APS. The filter set index of the luma CTB is indicated to indicate which filter set is applied. The 16 fixed filter sets are predefined and hard-coded in the encoder and decoder.

[0083] For chroma components, the APS index can be signaled in the slice header to indicate the chroma filter set being used for the current slice. At the CTB level, if there is more than one chroma filter in the APS, the filter index is signaled for each chroma CTB. The filter coefficients are quantized with a normalization equal to 128. To limit the multiplication complexity, bitstream consistency is applied so that coefficient values ​​in non-central positions will be around -2 7 To 2 7-1 In the range, including -2 7 and 2 7-1 The center position coefficient is not signaled in the bitstream and is assumed to be equal to 128.

[0084] G. Simplified ALF filtering using fixed filters

[0085] In some embodiments, ALF gradient subsampling and ALF virtual boundary processing are removed. The block size of the classification is reduced from 4x4 to 2x2. The filter size of the ALF coefficients is increased to 9x9 for luma and chroma.

[0086] To filter the brightness samples, three different classifiers can be used (C 0 , C 1 and C 2 ) and three different filter sets (F 0 、F 1 and F 2 ). Set F 0 and set F 1 Includes fixed filters with a 0 and C 1 The training coefficient. F 2 The coefficients of the filters in are labeled. Which filter in the set Fi is used for a given sample is determined by using the classifier C i The category C assigned to this sample i First, two 13x13 diamond fixed filters F are applied 0 and F 1 , to derive two intermediate samples R 0 (x,y) and R 1 (x,y). Then, F 2 Application to R 0 (x,y) and R 1 (x, y) and adjacent samples to derive the filtered samples as:

[0087]

[0088] Among them, f i,j is the clipping difference between the adjacent sample and the current sample R(x,y), g i YesR i-20 The clipping difference between (x,y) and the current sample. Filter coefficient c i , i=0,…21 are marked.

[0089] Based on directionality D i and activity Category C i Assigned to each 2x2 block.

[0090]

[0091] Among them, M D,i Indicates directionality D iThe total number of . The horizontal gradient, vertical gradient, and two diagonal gradients can be calculated for each sample using a one-dimensional Laplacian. The sum of the sampled gradients within a 4×4 window covering the target 2×2 block is used for the classifier C 0 , the sum of the sampled gradients within the 12×12 window is used for the classifier C 1 and C 2 The horizontal gradient, vertical gradient and the sum of the two diagonal gradients are expressed as and Directionality D i Determined by comparing the following to a set of thresholds:

[0092]

[0093] Directionality D 2 is derived using thresholds of 2 and 4.5. 0 and D 1 , first calculate the horizontal / vertical edge strength and diagonal edge strength Threshold Th = [1.25, 1.5, 2, 3, 4.5, 8] is used. If The edge strength value is 0, otherwise, is the largest integer, so if The edge strength is 0, otherwise, is the largest integer, so Table 2(a) and Table 2(b) below show the E i D and E i HV The mapping with Di. That is, horizontal edges / vertical edges dominate, use Table 2(a) below to derive Di. Otherwise, diagonal edges dominate, use Table 2(b) to derive D i .

[0094] Table 2(a):

[0095]

[0096] Table 2(b):

[0097]

[0098]

[0099] To gain activity Vertical gradient and horizontal gradient A i The sum of is mapped to the range from 0 to n, where for n is equal to 4, for and n is equal to 15. In ALF_APS, up to 4 luma filter sets are indicated, each set may have up to 25 filters.

[0100] II. Adaptive Filter Controls

[0101] A. ALF with on / off control

[0102] The ALF filter is derived by optimizing the frame-level sum of squares and distortion (SSD). However, due to the limited number of coefficients of the filter, although the filter is adaptively selected for each CTB, the filter may not benefit all samples of the CTB. Some samples are not suitable for ALF filtering and even become worse after ALF filtering. Some embodiments of the present disclosure provide an ALF on / off control mechanism at a level lower than the CTB level.

[0103] In some embodiments, sample-level on / off control is implemented by using a Sample-Adaptive-Offset-Edge-Offset-like (SAO-EO-like) classifier. In some embodiments, for each CTB, given an edge offset (EO) pattern (the direction of the edge), samples are classified into more than one category, and each category can have its own on / off selection. (Each sample is classified based on a direction determined by the relationship or difference between the sample and its neighbors.)

[0104] In some embodiments, the EO mode and the on / off selection for each class are signaled at the CTB level. In another embodiment, the EO mode is signaled at the CTB level, and the on / off selection for each class is signaled at a higher level. In another embodiment, both the EO mode and the on / off selection for each class are signaled at a higher level.

[0105] In some embodiments, the sampling level on / off control follows a specific pattern of CTB selections. In some embodiments, each CTB selects one of a plurality of 2x2 on / off patterns, and the on / off selection of each 2x2 block is determined accordingly. Figure 4A -B marks the switch to select mode. Figure 4A Fifteen possible switching patterns for a 2x2 block are shown. Figure 4B Switch selection in one CTB 410 is shown, where one of 15 2x2 patterns is selected and applied to the entire CTB.

[0106] In some embodiments, a lower level on / off control is implemented by considering the ALF block class. Specifically, each block class derived by the ALF classifier can have its own ALF on / off selection. In some embodiments, each class on / off selection is signaled at the filter set level. In some embodiments, each class on / off selection is signaled at the slice level.

[0107] In some embodiments, a model is used to determine whether to apply ALF to each sample or each block. In some embodiments, the model is a classifier trained offline. In some embodiments, the model is a classifier trained online. The classifier can be a probability-based model, a decision tree-based model, a support vector machine (SVM) or a neural network-based model.

[0108] B. ALF with Adaptive Filter Strength

[0109] Some embodiments of the present disclosure provide ALF based on adaptive filtering strength. In general, the ALF reconstruction process can be expressed as:

[0110]

[0111] Where R(x,y) is the sample value before ALF filtering, is the sample value after ALF filtering, c i is the i-th filter coefficient, n i is the i-th filter tap input, and R ALF is the correction value or offset of ALF. Filter tap input n i It can be a clipped adjacent difference, a correction value from another filter (e.g., a deblocking filter or an ALF fixed filter), or a correction value from another loop filter stage. i It can also be generated based on pre-ALF current and adjacent samples, fixed filtered current and adjacent samples, pre-DBF current and adjacent samples, residual current samples and fixed filtered residual samples, etc.

[0112] When the filter strength s is applied, the reconstruction process is modified as follows:

[0113]

[0114] In some embodiments, the above-mentioned ALF on / off control may be implemented as a filter strength s, that is, a filter strength s=0 indicates that the ALF is off, and a filter strength s=1 or s>0 indicates that the ALF is on.

[0115] In some embodiments, one or more filter strength values ​​are indicated at a higher level and whether to apply filter strength is determined at a lower level. In some embodiments, one filter strength value is indicated at the slice level. For each ALF-on CTB, an additional flag may be signaled to indicate whether filter strength is to be applied. In some embodiments, one filter strength value is signaled at the slice level for each APS or filter set. For each ALF-on CTB, the filter strength applied is determined according to the APS or filter set selection.

[0116] In some embodiments, one or more filter strength models are signaled at a higher level and whether to apply filter strength and which filter strength model to use can be adaptively determined at a lower level. In one embodiment, the filter strength models are signaled at the slice level. For each ALF-on CTB, the filter strength is always applied using the signaled model. In some embodiments, one filter strength model is signaled at the slice level. For each ALF-on CTB, an additional flag is signaled to indicate whether to apply the filter strength derived from the signaled model. In another embodiment, several filter strength models are signaled at the slice level. For each ALF-on CTB, an additional flag is signaled for filter strength model selection.

[0117] In some embodiments, for an ALF-on CTB, the video codec may apply different filter strengths to different subsets of samples in the CTB. In some embodiments, the samples are divided into subsets based on a predetermined pattern. In another embodiment, the samples are divided into subsets based on one or more sample adaptive offset edge offset (SAO-EO-like) classifiers. In some embodiments, the samples are divided into subsets or categories based on a neural network-based model.

[0118] Figure 5A -B conceptually illustrates the classification of ALF samples for applying filtering strength. Figure 5A A set of samples 500 for ALF operation for a particular CTB is shown. The sample set 500 is classified into 12 different subsets (or categories) 501-512 of samples. The classification can be based on any of the classifications described in Sections II-A and II-B above, such as classifying each sample based on its relationship to its neighbors according to the edge offset pattern of the CTB, or dividing the samples of the CTB based on a predetermined pattern selected from a plurality of different predetermined patterns, or classifying each sample according to a model specified in the bitstream of the encoded video. In some embodiments, ALF filtering can be turned on or off separately for each subset. In some embodiments, the ALF filtering strength can be determined separately for each subset. Figure 5BFilter strengths S that can be applied to sample subsets 501-512, respectively, are shown. 1 -S 12 . Will refer to Figure 7 and Fig.10 A further example of an ALF with adaptive filtering strength is described below.

[0119] The above-mentioned proposed method can be implemented in an encoder and / or a decoder. For example, the proposed method can be implemented in a loop filter module of an encoder and / or a loop filter module of a decoder.

[0120] III. Example Video Encoder

[0121] Figure 6 An exemplary video encoder 600 implementing a loop filter is shown. As shown, the video encoder 600 receives an input video signal from a video source 605 and encodes the signal into a bitstream 695. The video encoder 600 has several components or modules for encoding the signal from the video source 605, including at least some components selected from a transform module 610, a quantization module 611, an inverse quantization module 614, an inverse transform module 615, an intra-picture estimation module 620, an intra-picture prediction module 625, a motion compensation module 630, a motion estimation module 635, a loop filter 645, a reconstructed image buffer 650, a motion vector (MV) buffer 665, a motion vector prediction module 675, and an entropy encoder 690. The motion compensation module 630 and the motion estimation module 635 are part of the inter-picture prediction module 640.

[0122] In some embodiments, modules 610-690 are modules of software instructions being executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 610-690 are modules of hardware circuits implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 610-690 are shown as separate modules, some of these modules can be combined into a single module.

[0123] The video source 605 provides a raw video signal, which represents pixel data for each video frame without compression. A subtractor 608 calculates the difference between the raw video pixel data of the video source 605 and the predicted pixel data 613 from the motion compensation module 630 or the intra-image prediction module 625 as a prediction residual 609. The transform module 610 transforms the difference (or residual pixel data or residual signal 608) into transform coefficients (e.g., by performing a discrete cosine transform or DCT). The quantization module 611 quantizes the transform coefficients into quantized data (or quantized coefficients) 612, which are encoded by the entropy encoder 690 into a bitstream 695.

[0124] The inverse quantization module 614 dequantizes the quantized data (or quantized coefficients) 612 to obtain transform coefficients, and the inverse transform module 615 inversely transforms the transform coefficients to generate a reconstructed residual 619. The reconstructed residual 619 is added to the predicted pixel data 613 to generate reconstructed pixel data 617. In some embodiments, the reconstructed pixel data 617 is temporarily stored in a line buffer (not shown) for intra-picture prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 645 and stored in the reconstructed image buffer 650. In some embodiments, the reconstructed image buffer 650 is a storage external to the video codec 600. In some embodiments, the reconstructed image buffer 650 is a storage internal to the video encoder 600.

[0125] The image intra-picture estimation module 620 performs intra-picture prediction based on the reconstructed pixel data 617 to generate intra-picture prediction data. The intra-picture prediction data is provided to the entropy encoder 690 to be encoded into a bitstream 695. The intra-picture prediction data is also used by the intra-picture prediction module 625 to generate predicted pixel data 613.

[0126] The motion estimation module 635 performs inter-picture prediction by generating motion vectors to reference pixel data of previously decoded frames stored in the reconstructed image buffer 650. These motion vectors are provided to the motion compensation module 630 to generate predicted pixel data.

[0127] The video codec 600 uses MV prediction to generate a predicted MV instead of encoding the complete actual MV in the bitstream. The difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 695.

[0128] The motion vector prediction module 675 generates a predicted motion vector, i.e., a motion compensated motion vector used to perform motion compensation, based on the reference motion vector generated for encoding the previous video frame. The motion vector prediction module 675 retrieves the reference motion vector from the previous video frame from the motion vector buffer 665. The video encoder 600 stores these motion vectors generated for the current video frame in the motion vector buffer 665 as reference motion vectors for generating the predicted motion vector.

[0129] The motion vector prediction module 675 uses the reference motion vector to create a predicted motion vector. The predicted motion vector can be calculated by spatial motion vector prediction or temporal motion vector prediction. The difference (residual motion data) between the predicted motion vector and the motion compensation motion vector (motion compensation MV, MC MV) of the current frame is encoded into a bit stream 695 by the entropy encoder 690.

[0130] Using entropy coding techniques, such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding, the entropy encoder 690 encodes various parameters and data into a bitstream 695. The entropy encoder 690 encodes various header elements, flags, and quantized transform coefficients 612 and residual motion data as syntax elements into the bitstream 695. In turn, the bitstream 695 is stored in a storage device or transmitted to a decoder via a communication medium such as a network.

[0131] The loop filter 645 performs filtering or smoothing operations on the reconstructed pixel data 617 to reduce coding artifacts, especially artifacts at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the loop filter 645 include DBF, SAO and / or ALF.

[0132] Figure 7 6 shows a portion of a video encoder 600 that implements ALF with virtual boundaries based on samples from multiple sources. Specifically, the figure illustrates components of a loop filter 645 of the video encoder 600. As shown, the loop filter 645 receives reconstructed pixel data 617 of a current block (e.g., a current CTB) and generates a filtered output to be stored in a reconstructed image buffer 650. The input pixel data is processed by a DBF module 702 and a SAO module 704 in the loop filter 645. The processed samples generated by the DBF and SAO are provided to the ALF module 706. Figure 2 An example of the loop filter 200 having DBF, SAO, and ALF is described.

[0133] The ALF module 706 includes a classifier 710. The classifier 710 classifies each input sample (from SAO and DBF) into one of several different subsets (or categories). The classifier 610 can perform classification of samples according to predefined patterns. The selection of predefined patterns can be signaled into the bitstream by the entropy encoder 690. Classification can also be performed according to a model. Classification by model and / or pattern is described above in Sections II-A and II-B.

[0134] Each sample classified as a particular subset of samples is used to generate a correction value to be added to the sample. The correction value is generated by applying a filter 720 to the sample. The filter coefficients of the filter 720 can be signaled in the bitstream by the entropy encoder 690. The filter taps of the filter 720 can include samples of the current block (e.g., the current CTB), neighboring blocks of the current block (provided by the reconstructed image buffer 650), or the reconstructed residual 619 of the current block. The generated correction value is weighted by a filter strength specific to the particular subset to which the sample belongs. In some embodiments, filtering for a particular subset of samples can be turned off or on by, for example, setting the filter strength of the subset to zero or a non-zero value. The filter strengths of different subsets can be signaled in the bitstream by the entropy encoder 690.

[0135] Therefore, the input samples of the ALF module 706 are combined with their corresponding correction values ​​to generate the output of the ALF module 706, which is also the output of the loop filter 645. The output of the loop filter 645 is stored in the reconstructed image buffer 650 for encoding subsequent blocks.

[0136] Figure 8 Schematically illustrates a process 800 for applying adaptive filtering strength in an adaptive loop filter. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing encoder 600 perform process 800 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing encoder 600 performs process 800.

[0137] The encoder receives (at block 810) data for a current pixel block in a current picture to be encoded as a video. The current block may be a codec tree block. The encoder receives (at block 820) a set of samples for the current block.

[0138] The encoder classifies (at block 825) the set of samples into a plurality of subsets (or classes) of samples. In some embodiments, each sample in the set of samples may be classified into one of a plurality of subsets of samples based on the relationship of the sample to its neighbors and based on an edge offset pattern of a current block. In some embodiments, the set of samples is classified into a plurality of subsets based on a predetermined pattern. The predetermined pattern may be selected from a plurality of predetermined patterns (e.g., one of 15 on / off 2x2 patterns). In some embodiments, the set of samples is classified into a plurality of subsets by a model signaled in a bitstream of the encoded video, or trained by online or offline data.

[0139] The encoder (at block 830) filters the received set of samples to produce a set of correction values. In some embodiments, the filter is an adaptive loop filter (ALF) of a video codec system. This filtering can be based on a set of filter taps that receive inputs, the inputs including (i) samples within the current block, (ii) samples adjacent to the current block, or (iii) residual samples generated based on a prediction of the current block. The filtering can be based on a set of filter taps that receive inputs, the inputs including (i) samples generated by a deblocking filter (DBF) or a sample adaptive offset (SAO) filter, or (ii) reconstructed samples of the current block without deblocking filtering.

[0140] The encoder (at block 840) applies a set of filter strengths to weight the generated set of correction values. In some embodiments, filtering for each subset (or category) of samples may be turned on or off individually. In some embodiments, filtering for each subset of samples may be weighted by a corresponding filter strength. In some embodiments, the encoder may turn off filtering for a subset of samples by setting the corresponding filter strength to zero (or turn on filtering for a subset by setting the corresponding filter strength to a non-zero value). In some embodiments, the set of filter strengths is indicated at a first, higher level of the video (e.g., a slice level), and whether to apply the filter strength is determined at a second, lower level of the video (e.g., a CTB level). In some embodiments, the set of filter strengths is determined by applying a filter strength model to the set of samples, wherein the filter strength model is signaled in a bitstream of the encoded video. In some embodiments, filtering is based on a set of filter taps that receive inputs that include (i) samples within a current block, (ii) samples adjacent to the current block, or (iii) residual samples generated based on a prediction of the current block.

[0141] The encoder (at block 850) adds the weighted correction value set to the received sample set as the filtered samples of the current block. The encoder (at block 860) provides the filtered samples of the current block for encoding subsequent blocks of video (e.g., stored in the reconstructed image buffer 650).

[0142] IV. Example Video Decoder

[0143] In some embodiments, the encoder may indicate (or generate) one or more syntax elements in the bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0144] Fig. 9 An exemplary video decoder 900 that can implement an adaptive loop filter is shown. As shown, the video decoder 900 is an image decoding or video decoding circuit that receives a bitstream 995 and decodes the content of the bitstream into pixel data of a video frame for display. The video decoder 900 has several components or modules for decoding the bitstream 995, including some components selected from an inverse quantization module 911, an inverse transform module 910, an intra-picture prediction module 925, a motion compensation module 930, a loop filter 945, a decoded image buffer 950, a motion vector buffer 965, a motion vector prediction module 975, and a parser 990. The motion compensation module 930 is part of the inter-picture prediction module 940.

[0145] In some embodiments, modules 910-990 are modules of software instructions executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 910-990 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 910-990 are illustrated as separate modules, some of these modules may be combined into a single module.

[0146] The parser 990 (or entropy decoder) receives the bitstream 995 and performs initial parsing according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantized data (or quantized coefficients) 912. The parser 990 parses out the various syntax elements by using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0147] The inverse quantization module 911 dequantizes the quantized data (or quantized coefficients) 912 to obtain transform coefficients, and the inverse transform module 910 inversely transforms the transform coefficients 916 to generate reconstructed residuals 919. The reconstructed residuals 919 are added to the predicted pixel data 913 from the intra-picture prediction module 925 or the motion compensation module 930 to generate decoded pixel data 917. The decoded pixel data is filtered by the loop filter 945 and stored in the decoded image buffer 950. In some embodiments, the decoded image buffer 950 is a storage outside the video decoder 900. In some embodiments, the decoded image buffer 950 is a storage inside the video decoder 900.

[0148] The intra prediction module 925 receives intra prediction data from the bitstream 995 and generates predicted pixel data 913 based on the intra prediction data from the decoded pixel data 917 stored in the decoded image buffer 950. In some embodiments, the decoded pixel data 917 is also stored in a line buffer (not shown) for intra picture prediction and spatial MV prediction.

[0149] In some embodiments, the contents of the decoded image buffer 950 are used for display. The display device 955 retrieves the contents of the decoded image buffer 950 for direct display, or retrieves the contents of the decoded image buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 950 via pixel transfer.

[0150] Based on motion compensated MV (MC MV), the motion compensation module 930 generates predicted pixel data 913 from the decoded pixel data 917 stored in the decoded picture buffer 950. These motion compensated MVs are decoded by adding the residual motion data received from the bitstream 995 to the predicted MV received from the motion vector prediction module 975.

[0151] The motion vector prediction module 975 generates a predicted MV, for example, a motion compensated MV for performing motion compensation, based on a reference MV generated for decoding a previous video frame. The motion vector prediction module 975 retrieves the reference motion vector of the previous video frame from the motion vector buffer 965. The video decoder 900 also stores the motion compensated motion vector generated for decoding the current video frame in the motion vector buffer 965 as a reference motion vector for generating a predicted motion vector.

[0152] The loop filter 945 performs filtering or smoothing operations on the decoded pixel data to reduce encoding and decoding artifacts, especially artifacts located at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the loop filter 945 include a deblocking filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF).

[0153] Fig.101 shows a portion of a video decoder 900 that implements an ALF with adaptive filtering strength. Specifically, the figure shows components of a loop filter 945 of the video decoder 900. As shown, the loop filter 945 receives decoded pixel data 917 of a current block (e.g., a current CTB) and generates a filtered output to be stored in a decoded image buffer 950. The input pixel data is processed in the loop filter 945 by a DBF module 1002 and a SAO module 1004. The processed samples generated by the DBF and SAO are provided to an ALF module 1006. Figure 2 An example of the loop filter 200 having DBF, SAO, and ALF is described.

[0154] The ALF module 1006 includes a classifier 1010. The classifier 1010 classifies each input sample (from SAO and DBF) into one of several different subsets (or categories). The classifier 910 can perform classification of samples according to predefined patterns. The selection of predefined patterns can be parsed from the bitstream by the entropy decoder 990. Classification can also be performed according to a model. Classification by model and / or pattern is described in Sections II-A and II-B above.

[0155] Each sample classified into a particular subset of samples is used to generate a correction value to be added to the sample. The correction value is generated by applying a filter 1020 to the sample. The filter coefficients of the filter 1020 can be received from the bitstream by the entropy decoder 990. The filter taps of the filter 1020 can include samples of the current block (e.g., the current CTB), neighboring blocks of the current block (provided by the decoded image buffer 950), or the reconstructed residual 919 of the current block. The generated correction value is weighted by a filter strength specific to the particular subset to which the sample belongs. In some embodiments, filtering for a particular subset of samples can be turned off or on by, for example, setting the filter strength of the subset to zero or a non-zero value. The entropy decoder 990 can receive filter strengths for different subsets from the bitstream.

[0156] Therefore, the input samples of the ALF module 1006 are combined with their corresponding correction values ​​to generate the output of the ALF module 1006, which is also the output of the loop filter 945. The output of the loop filter 945 is stored in the decoded image buffer 950 for decoding and reconstruction of subsequent blocks.

[0157] Fig.11 Schematically illustrates a process 1100 for applying adaptive filtering strength in an adaptive filter. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 900 perform process 1100 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 900 performs process 1100.

[0158] The decoder receives (at block 1110) data for a current block to be decoded as a current picture of a video. The current block may be a codec tree block (CTB). The decoder receives (at block 1120) a set of samples for the current block.

[0159] The decoder (at block 1125) classifies the sample set into a plurality of subsets (or classes) of samples. In some embodiments, each sample in the sample set may be classified into one of a plurality of sample subsets based on the sample's relationship to its neighbors and based on an edge offset pattern of a current block. In some embodiments, the sample set is classified into a plurality of subsets based on a predetermined pattern. The predetermined pattern may be selected from a plurality of predetermined patterns (e.g., one of 15 on / off 2x2 patterns). In some embodiments, the sample set is classified into a plurality of subsets by a model signaled in a bitstream of the encoded video, or trained by online or offline data.

[0160] The decoder (at block 1130) filters the received set of samples to produce a set of correction values. In some embodiments, the filter is an adaptive loop filter (ALF) of a video codec system. This filtering can be based on a set of filter taps that receive inputs that include (i) samples within the current block, (ii) samples adjacent to the current block, or (iii) residual samples generated based on a prediction of the current block. The filtering can be based on a set of filter taps that receive inputs that include (i) samples generated by a deblocking filter (DBF) or a sample adaptive offset (SAO) filter, or (ii) reconstructed samples of the current block without deblocking filtering.

[0161] The decoder (at block 1140) applies a set of filter strengths to weight the generated set of correction values. In some embodiments, filtering of each subset (or category) of samples may be turned on or off individually. In some embodiments, filtering of each subset of samples may be weighted by a corresponding filter strength. In some embodiments, the decoder may turn off filtering of a subset of samples by setting the corresponding filter strength to zero (or turn on filtering of a subset by setting the corresponding filter strength to a non-zero value). In some embodiments, the set of filter strengths is indicated at a first, higher level of the video (e.g., a slice level), and whether to apply the filter strength is determined at a second, lower level of the video (e.g., a CTB level). In some embodiments, the set of filter strengths is determined by applying a filter strength model to the set of samples, wherein the filter strength model is signaled in a bitstream of the encoded video. In some embodiments, filtering is based on a set of filter taps receiving an input comprising (i) samples within a current block, (ii) samples adjacent to the current block, or (iii) residual samples generated based on a prediction of the current block.

[0162] The decoder (at block 1150) adds the weighted correction value set to the received set of samples as filtered samples for the current block. The decoder (at block 1160) provides the filtered samples for the current block for use in decoding subsequent blocks of video (e.g., for storage in reconstructed image buffer 950) or for display as part of a reconstructed current image.

[0163] IV. Example Electronic Systems

[0164] Many of the above features and applications can be implemented as software processes, which are specified as sets of instructions recorded on a computer readable storage medium (also referred to as a computer readable medium). When these instructions are executed by one or more computing units or processing units (e.g., one or more processors, processor cores, or other processing units), the instructions cause the processing units to perform the actions represented by the instructions. Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard disks, erasable programmable read only memory (EPROM), electrically erasable programmable read-only memory (EEPRO M), etc. Computer readable media do not include carriers and telecommunications signals connected wirelessly or wired.

[0165] In this specification, the term "software" means firmware in a read-only memory or an application stored in a magnetic storage device, which can be read into the memory for processing by the processor. At the same time, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program, while retaining different software inventions. In some embodiments, multiple software inventions can be implemented as independent programs. Finally, any combination of independent programs that implement the software inventions described herein together is within the scope of the present invention. In some embodiments, when installed to operate on one or more electronic systems, the software program defines one or more specific machine implementations that execute and implement the operations of the software program.

[0166] Fig.12The electronic system 1200 implemented in some embodiments of the present invention is conceptually shown. The electronic system 1200 can be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or other types of electronic devices. This electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1200 includes a bus 1205, a processing unit 1210, a graphics-processing unit (GPU) 1215, a system memory 1220, a network 1225, a read-only memory (ROM) 1230, a permanent storage device 1235, an input device 1240, and an output device 1245.

[0167] The bus 1205 collectively represents all system buses, peripheral buses, and chipset buses that are communicatively coupled to a number of internal devices of the electronic system 1200. For example, the bus 1205 is communicatively coupled to the processing unit 1210 via the GPU 1215, the ROM 1230, the system memory 1220, and the permanent storage device 1235.

[0168] From these various memory units, the processing unit 1210 retrieves instructions to execute and data to process in order to perform the processes of the present invention. In various embodiments, the processing unit may be a single processor or a multi-core processor. Certain instructions are transmitted to and executed by the GPU 1215. The GPU 1215 may offload various calculations or supplement the image processing provided by the processing unit 1210.

[0169] ROM 1230 stores static data and instructions required by processing unit 1210 or other modules of the electronic system. On the other hand, permanent storage device 1235 is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when electronic system 1200 is turned off. Some embodiments of the present invention use a large-capacity storage device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as permanent storage device 1235.

[0170] Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Like permanent storage device 1235, system memory 1220 is a read-write memory device. However, unlike storage device 1235, system memory 1220 is a volatile read-write memory, such as random access memory. System memory 1220 stores instructions and data that some processors need when running. In some embodiments, processing according to the present invention is stored in system memory 1220, permanent storage device 1235 and / or read-only memory 1230. For example, various memory units include instructions for processing multimedia clips according to some embodiments. For these various memory units, processing unit 1210 retrieves executed instructions and processed data in order to perform processing of certain embodiments.

[0171] The bus 1205 is also connected to input devices 1240 and output devices 1245. The input devices 1240 enable a user to communicate information and select commands to the electronic system. The input devices 1240 include alphanumeric keyboards and pointing devices (also known as "cursor control devices"), cameras (such as webcams), microphones or similar devices for receiving voice commands, etc. The output devices 1245 display images generated by the electronic system or data output in other ways. The output devices 1245 include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD), and speakers or similar audio output devices. Some embodiments include devices such as touch screens that function as both input devices and output devices.

[0172] Finally, if Fig.12 As shown, bus 1205 also couples electronic system 1200 to network 1225 via a network adapter (not shown). In this manner, the computer may be part of a computer network (e.g., a local area network (LAN), a wide area network (WAN), or an intranet) or a network of networks (e.g., the Internet). Any or all components of electronic system 1200 may be used in conjunction with the present invention.

[0173] Some embodiments include electronic components, such as microprocessors, storage devices, and memory, which store computer program instructions to machine-readable media or computer-readable media (optionally referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable Computer readable media may store computer programs executed by at least one processing unit and include sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as that produced by a compiler, and files containing high-level code executed by a computer, electronic component, or microprocessor using an interpreter.

[0174] While the above discussion refers primarily to microprocessors or multi-core processors that execute software, many of the above functions and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0175] As used in the specification and any claims of the present invention, the terms "computer", "server", "processor" and "memory" refer to electronic devices or other technical devices. These terms do not include people or groups. For the purpose of this specification, the term display or display device refers to displaying on an electronic device. As used in the specification and any claims of the present invention, the terms "computer-readable medium", "computer-readable medium" and "machine-readable medium" are entirely limited to tangible, physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals and any other transient signals.

[0176] While the present invention has been described in conjunction with many specific details, those skilled in the art will recognize that the present invention may be implemented in other specific forms without departing from the spirit of the present invention. Figure 8 and Fig.11 ) conceptually illustrates the processes. The specific operations of these processes may not be performed in the exact order shown and described. These specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the processes are implemented by using several sub-processes, or as part of a larger macro process. Therefore, it will be understood by those skilled in the art that the present invention is not limited by the foregoing illustrative details, but is defined by the scope of the claims.

[0177] Additional Notes

[0178] The subject matter described herein sometimes represents different components, which are contained in or connected to other different components. It is understood that the described structure is only an example and can actually be implemented by many other structures to implement the same function. Conceptually, any arrangement of components that implement the same function is actually "associated" in order to implement the desired function. Therefore, regardless of the structure or intermediate components, any two components combined to implement a specific function are considered to be "interrelated" to implement the desired function. Similarly, any two associated components are considered to be "operably connected" or "operably coupled" to each other to implement a specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to each other to implement a specific function. Specific examples of operable connections include, but are not limited to, physically pairable and / or physically interacting components, and / or wirelessly interactive and / or wirelessly interactive components, and / or logically interactive and / or logically interactive components.

[0179] Furthermore, with respect to the use of substantially any plural and / or singular terms, those of ordinary skill in the art can translate from the plural to the singular and / or from the singular to the plural as appropriate to the context and / or application. For clarity, the various singular / plural permutations are expressly provided herein.

[0180] In addition, it will be understood by those skilled in the art that, in general, the terms used in the present invention, especially in the claims, such as the subject matter of the claims, are generally used as "open" terms, for example, "including" should be interpreted as "including but not limited to", "having" should be interpreted as "at least having", "including" should be interpreted as "including but not limited to", etc. It will be further understood by those skilled in the art that if a specific number of claimed items are intended to be introduced, it will be explicitly indicated in the claims, and will not be displayed when such content is absent. For example, to aid understanding, the claims may include the phrases "at least one" and "one or more" to introduce the claimed items. However, the use of these phrases should not be construed as implying the use of the indefinite article "a" or "an" to introduce the claimed items, thereby limiting any particular patent scope. Even when the same claim includes the introductory phrases "one or more" or "at least one", the indefinite article, such as "a" or "an", should be interpreted to mean at least one or more, and the same is true for the use of explicit descriptions used to introduce claims. Furthermore, even when an introductory content explicitly refers to a specific number, one of ordinary skill in the art will recognize that such content should be interpreted as indicating the number of references, for example, "two references" without other modifications means at least two references, or two or more references. Furthermore, where a statement similar to "at least one of A, B, and C" is used, it is generally stated so that one of ordinary skill in the art can understand the statement, for example, "a system includes at least one of A, B, and C" will include but is not limited to a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc. One of ordinary skill in the art will further understand that any separated words and / or phrases represented by two or more alternative terms, whether in the specification, claims, or drawings, should be understood to include the possibility of one of these terms, one of them, or both of these terms. For example, "A or B" should be understood as the possibility of "A", or "B", or "A and B".

[0181] As can be seen from the foregoing, various embodiments have been described herein for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the invention. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the scope of the claims represents the true scope and spirit.

Claims

1. A video encoding and decoding method, comprising: receiving data of a pixel block to be encoded or decoded as a current block of a current image of a video; Receiving a sampling set of the current block; Filtering the received sample set to generate a correction value set; applying a set of filter strengths to weight the generated set of correction values; and The weighted correction value set is added to the received sample set as filtered samples of the current block.

2. The video encoding and decoding method according to claim 1, characterized in that: The filter is an adaptive loop filter of a video codec system, wherein the filtered samples of the current block are provided for encoding or decoding subsequent blocks of the current image.

3. The video encoding and decoding method according to claim 1, characterized in that: The sampling set is classified into a plurality of sampling subsets.

4. The video encoding and decoding method according to claim 3, characterized in that: Turn filtering on or off for each sample subset individually.

5. The video encoding and decoding method according to claim 3, characterized in that: The filtering of each sample subset is weighted by the corresponding filter strength.

6. The video encoding and decoding method according to claim 3, characterized in that: Disable filtering for a subset of samples by setting the corresponding filter strength to zero.

7. The video encoding and decoding method according to claim 3, characterized in that: Each sample in the sample set is classified into one of the plurality of sample subsets based on the relationship of the sample to its neighbors.

8. The video encoding and decoding method according to claim 3, characterized in that: The sample set is classified into the plurality of subsets based on a predetermined pattern.

9. The video encoding and decoding method according to claim 3, characterized in that: The sample set is classified into the plurality of subsets through a model.

10. The video encoding and decoding method according to claim 1, characterized in that: The filter strength set is indicated at a first level of the video, wherein whether to apply the filter strength is determined at a second level of the video, wherein the first level is a higher video level than the second level.

11. The video encoding and decoding method according to claim 10, characterized in that: The first level is the slice level, and the second level is the codec treeblock.

12. The video encoding and decoding method according to claim 1, characterized in that: The filter strength set is determined by applying a filter strength model to the set of samples, wherein the filter strength model is signaled in a bitstream of the encoded video.

13. The video encoding and decoding method according to claim 1, characterized in that: The filtering is based on a set of filter taps that receive inputs including (i) samples within the current block, (ii) samples adjacent to the current block, or (iii) residual samples generated based on prediction of the current block.

14. The video encoding and decoding method according to claim 1, characterized in that: The filtering is based on a set of filter taps whose inputs include (i) samples generated by a deblocking filter or a sample adaptive offset filter, or (ii) reconstructed samples of the current block that have not been deblocked.

15. An electronic device comprising: The video codec circuit is configured to perform operations including: receiving data of a pixel block to be encoded or decoded as a current block of a current image of a video; Receiving a sampling set of the current block; Filtering the received sample set to generate a correction value set; Applying a set of filter strengths to weight the generated set of correction values; as well as The weighted correction value set is added to the received sample set as filtered samples of the current block.

Images