Video decoder, video encoder, method for encoding and decoding video signals and computer program for adjusting one or more denoising operations

Abstract

The invention creates a video decoder and a video encoder including a noise reduction operation, the noise reduction operation including at least two denoising operations, and adapting the second denoising operation depending on the difference before and after the first denoising operation, thereby correcting the quality reduction that occurs due to the application of the first denoising operation. A corresponding method for encoding and decoding a video signal is also provided.

Description

Technical Field

[0001] The present application relates to decoding and encoding video signals, particularly using modulation of one or more denoising operations.

[0002] An embodiment according to the present invention relates to a video decoder.

[0003] Other embodiments according to the present invention relate to a video encoder.

[0004] Further embodiments according to the invention relate to a method for providing decoded video information based on encoded video information.

[0005] Further embodiments according to the invention relate to a method for providing encoded video information based on input video information.

[0006] Further embodiments according to the invention relate to a computer program for performing the method of encoding and decoding a video signal.

[0007] Other embodiments according to the invention relate to encoded video representations representing video content. Background Art

[0008] Many devices and methods for decoding and encoding video signals are currently known. It is known to use a variety of different filtering techniques to improve the compression performance and visual quality of decompressed video.

[0009] In video coding, due to its significant improvement in compression performance, the loop filter has emerged as a key module. For example, in P. List, A. Joch, J. Lainema, G. Bjontegaard, and M. Karczewicz, Adaptive deblocking filter, IEEE Trans. Circuits Syst. Video Technol. 13 (2003), No. 7, the loop filter used in the video coding standard is a deblocking filter (DBF), which is mainly used to reduce the block effect. However, although the deblocking filter (DBF) effectively removes the block effect at the smooth area, it also removes the image sharpness at the complex area. In addition, since the application of DBF makes the corner contour bright, the subjective quality of the reconstructed image is reduced.

[0010] As shown in, for example, C.-M. Fu eTal., Sample adaptive offset in the HEVC standard, IEEE Trans. Circuits Syst. Video Technol. 22 (2012) No. 12, another nonlinear loop filter, the sample adaptive offset (SAO) filter, is used in high-efficiency video coding (HEVC) to further reduce the coding artifacts after DBF. However, the visual quality is still lost, especially in the case of high-resolution video. In addition, the additional filtering component may lead to an increase in encoding latency. Therefore, further improvements in this filtering technology are important, especially for high-resolution video.

[0011] In view of the above, it is desirable to create a video coding concept that provides an improved trade-off between compression performance and visual quality as well as low encoding latency, which leads to improved coding efficiency. Summary of the invention

[0012] According to an embodiment of the present invention, a video decoder is created for providing decoded video information based on encoded video information, such as an encoded video input or a video input. The video decoder is configured to obtain first sample information, such as noisy first sample information, based on the encoded video information, such as using entropy decoding, inverse quantization and inverse transformation of a bitstream, and optionally using prediction. The first sample information may include, for example, a value represented by a YUV image, such as a pixel value, such as y1. The video decoder is also configured to apply a first denoising operation, such as a noise reduction operation, such as a combination of a DBF or deblocking filter (DBF) operation, a SAO or sample adaptive offset (SAO) filter operation, an ALF or adaptive loop filter (ALF) operation, or a combination of a DBF or DBF operation and a SAO or SAO operation, or one of the denoising operations, or a combination of a DBFV or deblocking filter (DBFV) operation for a vertical edge and a DBFH or deblocking filter (DBFH) operation for a horizontal edge, or, for example, only a DBFV, or a DBFV operation, so as to obtain first denoised sample information based on the first sample information. The first denoised sample information is, for example, SAO, or the output of a SAO operation, or ALF, or the output of an ALF operation, or DBFV and DBFH, or the output of a combination of DBFV and DBFH operations, or DBFV, or the output of a DBFV operation, such as y2. The deblocking filter (DBF) includes, for example, one or two filtering operations, namely, a deblocking filter (DBFV) for a vertical edge or / and a deblocking filter (DBFH) for a horizontal edge, which means, for example, DBF=DBFV or DBFH or DBFV+DBFH, and the deblocking filter can be, for example, a conventional deblocking filter. The video decoder is also configured to apply a second denoising operation, such as PSAO, or a post-sample adaptive offset (PSAO) filter operation, or SAO, or a SAO filter operation, or SAOV, or a SAO filter for vertical edges (SAOV) operation, based on a result of the first denoising operation, such as based on sample information of the first denoising, or, for example, based on SAO, or a SAO operation, or ALF, or an output of an ALF operation, or, for example, based on DBFV and DBFH, or an output Y of a combination of DBFV and DBFH operations, or, for example, based on DBFV, or an output Y2 of the DBFV operation. The video decoder is configured to adjust, e.g., adapt, a second denoising operation, e.g., SAO, or SAO operation, or SAOV, or SAOV operation, depending on a difference, e.g., y2(i)-y1(i) or Y-Y1 or Y2-Y1, between samples before a first denoising operation (e.g., a combination of DBFV and DBFH, or a combination of DBFV operation and DBFH operation) and samples after the first denoising operation (e.g., a combination of DBFV and DBFH, or a combination of DBFV operation and DBFH operation, or DBFV, or DBFV operation) so as to obtain decoded video information.SAOV and SAOH (SAO filtering for horizontal edges) may, for example, include classification of SAOV, or a SAOV operation and SAOH, or a SAOH operation, the classification being based on, for example, Y2-Y1 or Y-Y1, such as the output of a DBFV, or a DBFV operation, and the difference between samples before the DBFV, or a DBFV operation, or the difference between samples before a DBFH, or a DBFH operation and after a DBFH, or a DBFH operation, or. For example, DBFH, or the output of a DBFH operation and SAOV, or the difference between the outputs of a SAOV operation, respectively.

[0013] This embodiment is based on the following discovery: adapting the second denoising operation depending on the difference before and after the first denoising operation can correct the quality reduction that occurs due to the application of the first denoising operation. Therefore, this embodiment provides effective noise filtering that maintains video quality. For example, the difference between the samples before the first denoising operation and after the second denoising operation indicates in which areas the first denoising operation has a particularly large impact, and therefore when performing the subsequent second denoising operation, the knowledge of in which areas the first denoising operation has a particularly large impact can be taken into account. For example, one or more parameters of the second denoising operation can be adjusted (e.g., in a locally variable manner) depending on the (local) difference between the sample (or corresponding multiple samples) before and after the first denoising operation. In other words, local changes in the difference can be used to locally adapt the second denoising operation. For example, it can be considered that the first type of second denoising operation or the first parameter or parameter set of the second denoising operation can provide the best results (and therefore should be applied) in the area where the difference is relatively small, while the second type of second denoising operation or the second parameter or parameter set of the second denoising operation can provide the best results (and therefore should be used) in the area where the difference is relatively large.

[0014] Therefore, it has been found to be advantageous to adapt the characteristics of the second denoising operation depending on the difference, because the difference describes, for example, how much influence the first denoising operation has on the (first) sample information (e.g. in different areas of the image), and because a corresponding (possibly locally varying) adaptation of the second denoising operation leads to a good image quality while keeping latency and computational effort reasonably small.

[0015] According to one embodiment, the second denoising operation is configured to classify samples (e.g., pixels, such as y(i), where y can be a decoded input image for the PSAO operation, such as with sample index i) into multiple classes (e.g., groups of samples, Λ1, Λ2, or Λ0) depending on the difference (e.g., y2(i)-y1(i)) between the samples before the first denoising operation (e.g., y1(i)) and the samples after the first denoising operation (e.g., y2(i)). The second denoising operation is configured to modify the samples depending on the classification. The classification is, for example, based on the difference between the noisy (unfiltered) signal and the denoised (filtered) signal.

[0016] This embodiment is based on the discovery that filtering performance is substantially dependent on how the classification of samples is performed. Samples are successfully classified based on the difference between the noise signal and the denoised signal, and the classification is used to modify the samples in the second denoising operation so that the errors in the samples that occurred during filtering in the first denoising operation are minimized. In addition, based on the traditional classification, the sample modification in the second denoising operation can be performed faster, thereby reducing encoding latency.

[0017] Therefore, after one or a combination of DBF, or DBF operations, DBFV, or DBFV operations, DBFV and SAOV, or DBFV and SAOV operations, SAOV and DBFH, or SAOV and DBFH operations, DBFV and SAOV or DBFV, or DBFV and SAOV or DBFV operations, DBFV and SAOV and DBFV, or DBFV and SAOV and DBFV operations, or a combination of DBFV and DVFH or DBFV and DBFH operations, the second denoising operation may consider a sample adaptive offset operation, such as SAO, or SAO operations, SAOV, or SAOV operations, SAOH or SAOH (fourth) operations. Different combinations of filtering operations may further reduce encoding latency and may provide additional coding gain or quality enhancement.

[0018] It should be noted that in some embodiments, "samples" are represented as, for example, y(i) and y, which is the decoded input image for PSAO (or for a PSAO operation).

[0019] According to one embodiment, y (e.g., an input image, such as an input image of a PSAO or PSAO operation) and y2 (e.g., a first denoised image) are given as an output of an ALF or ALF operation (e.g., y) and an output of an SAO or SAO operation (e.g., y2), respectively. In this case, y and y2 are different.

[0020] According to another embodiment, when y2 is selected as ALF or the output of an ALF operation, for example, the input images y and y2 of a PSAO or a PSAO operation are the same image.

[0021] According to some embodiments, the input image y (e.g., the input image of PSAO or a PSAO operation) should always be the output of ALF or an ALF operation, because PSAO or a PSAO operation is, for example, applied after a loop filtering process (e.g., a combination of DBF+SAO+ALF or DBF operation+SAO operation+ALF operation). All mentioned embodiments provide an improved visual quality of the decoded video and / or an improved trade-off between bitrate, complexity and video quality.

[0022] According to an embodiment, the second denoising operation is configured to conditionally add an offset to a plurality of samples depending on the classification of the respective samples. For example, an offset is selected for each group of sample values ​​or sample positions sharing the same class index. In this embodiment, effective removal of coding artifacts is provided.

[0023] According to an embodiment, the first denoising operation includes a deblocking filtering operation and / or a sample adaptive offset filtering operation, the deblocking filtering operation for example using a low pass filtering of pixel values ​​along block boundaries to remove artifacts along block boundaries. In such an embodiment, different encoding artifacts can be effectively removed.

[0024] According to an embodiment, the sample adaptive offset filter is configured to classify samples of the image into a plurality of classes depending on one or more edge patterns (e.g., along one or more directions) in an image in an environment of the sample under consideration (e.g., including the sample under consideration) and / or depending on a sample value of the sample under consideration. The sample may be, for example, a pixel, such as y(i), where y may be a decoded input image for SAO or an SAO operation; for example, with a sample index i.

[0025] This embodiment is based on the finding that an efficient partitioning of the set of all sample positions improves the filtering performance at each position. For example, different influences of different edge patterns on pixel values ​​(where, for example, the error in the pixel value may depend on the presence of one or more edges in the environment of the considered pixel) may be taken into account (e.g., at least partially compensated) in this way. Furthermore, for example, a correlation of the pixel value error with the actual pixel value may be taken into account (and, for example, at least partially compensated) in this way.

[0026] As an illustration, it should be noted that conventional SAO or SAO operation may apply classification based on the following two characteristics (or at least one of the characteristics), for example:

[0027] 1) For example, assign each sample to one of, for example, 5 classes based on 1-D directional edge patterns, for example, along 4 directions, such as horizontal, vertical, 135° diagonal, and 45° diagonal.

[0028] 2) Classify each sample based on the sample value range, for example, equally divide into 32 frequency bands.

[0029] For example, a conventional SAO or SAO operation is applied for each coding tree unit (CTU) (or for multiple coding tree units), where the classification is selected from, for example, 5 different classifications as described above - for example 4 classifications based on edge information (horizontal + vertical + two diagonals) + for example a classification based on sample value range, which simply checks the sample value itself to assign it to, for example, one of 32 frequency bands.

[0030] According to an embodiment (or according to an aspect of the invention), the proposed classification is based on a different feature, namely the difference between the noisy (unfiltered) signal and the denoised (filtered) signal, for example when compared with the classification used in SAO or SAO operations (which are for example based on an evaluation of samples of a single image or frame or CTU). This embodiment is based on the finding that taking into account the difference between the unfiltered and filtered signal allows correcting quality errors that occur during the filtering operation.

[0031] According to an embodiment, the first denoising operation (or the above-mentioned first denoising operation) includes an ALF or adaptive loop filtering (ALF) operation, which uses an impulse response to perform filtering of, for example, pixel values, such as linear filtering, such as finite impulse response filtering or infinite impulse response filtering, and the filter coefficients are encoded in the bitstream. This embodiment is based on the following findings: different types of adaptive loop filtering (ALF) increase the coding gain, and such filtering (whose coefficients are encoded in the bitstream) can be adapted to specific video content and therefore provide a good compromise between bit rate and quality (for example, when compared to the case where the full bit budget is spent on the encoding of pixel values).

[0032] According to an embodiment, the video decoder is configured to apply a second denoising operation to sample information, such as to an image, or, for example, to pixel values, or, for example, to YUV values ​​of a YUV video representation, or, for example, to y(i), which is obtained using a combination (e.g., sequence) of one or more deblocking filtering operations (e.g., DBF or DBF operations, DBFH or DBFH operations and / or DBFV or DBFV operations), one or more sample adaptive offset filtering operations (e.g., SAO, or SAO operations, or SAOH, or SAOH operations and / or SAOV, or SAOV operations) and an adaptive loop filtering operation (ALF, or, for example, ALF operations). This embodiment is based on the finding that using a plurality of different processing steps as a first denoising operation allows for a good initial denoising. Moreover, it has been found that the total change in pixel values ​​through such a sequence of different processing steps, which is represented by the difference between the sample before the first denoising operation and the sample after the first denoising operation, constitutes a reliable criterion for adapting the (subsequent) second denoising operation.

[0033] According to an embodiment, the second denoising operation is configured to be applied to a plurality of coding tree units (CTUs) defining a video frame (e.g., a picture as a whole). Alternatively or additionally, the second denoising operation may be configured to be applied to a video frame, e.g., a picture as a whole consisting of coding tree units (which may, for example, define a video frame).

[0034] For example, a CTU may be a basic processing unit, and a CTU may use a larger block structure, such as up to 64×64 pixels, and may better subdivide an image into structures of variable sizes. For example, High Efficiency Video Coding (HEVC) uses only CTUs up to 64×64. However, larger CTUs (e.g., 128×128) may be used in a quadtree plus binary tree (QTBT) setting adopted by the first draft of a new video coding standard. This embodiment is based on the finding that block-based classification may reduce the complexity of the filtering process.

[0035] According to one embodiment, the classification of samples into multiple classes only occurs when one or more predetermined thresholds are reached. The one or more predetermined thresholds can be, for example, a single threshold T, which can be selected as a fixed positive constant, for example depending on human visual perception, or in the case of higher differences, more T0...TL (for example, more thresholds T0 to TL, or for example a set of thresholds T0 to TL), or for example a single threshold T, which can be, for example, encoded at the encoder or selected as a fixed positive constant, for example, the threshold T can be determined based on the rate distortion (RD) cost at the encoder. Thus, for example, unnecessary filtering operations are thereby excluded, which reduces encoding latency and reduces the consumption of operating resources.

[0036] According to an embodiment, the second denoising operation is configured to depend on the corresponding sample (e.g., y1[i], or Y1 or ) and the corresponding sample after the first denoising operation satisfies a first condition, (e.g., y2(i)-y1(i)>T) or a second condition (e.g., y2(i)-y1(i)<-T) or a third condition (e.g., -T<=y2(i)-y1(i)<=T), to classify the sample (e.g., the currently considered sample) into three or more classes. Alternatively, the second denoising operation is configured to classify the sample (e.g., the currently considered sample) into three or more classes depending on whether a difference (e.g., f(i)) based on a combination of multiple differences (e.g., weighted differences) between the corresponding samples before and after the first denoising operation, e.g., in a predetermined environment of the currently considered sample, satisfies a first condition (e.g., f(i)>T) or a second condition (e.g., f(i)<-T) or a third condition (e.g., -T<=f(i)<=T). An improved trade-off between compression performance and encoding speed is thus provided. Furthermore, the classification may be used to adapt the second denoising operation to the effectiveness of the first denoising operation, or to changes in one or more pixel values ​​caused by the first denoising operation.

[0037] According to an embodiment, the second denoising operation is configured to depend on the prior (eg, y1[i] or y1[i]) of the first denoising operation. ) and the corresponding samples after the first denoising operation (e.g., y2[i], or, for example, Y2, Y3, or Y), and depends on the magnitude or absolute value of the difference between the corresponding samples before the first denoising operation (e.g., y1[i], or, for example, Y1 or ) and the sign of the corresponding sample after the first denoising operation (e.g., y2[i], Y2, Y3, or Y) to classify the samples into three or more classes. If the absolute value of the difference is less than T or equal to T, the corresponding sample is assigned to, for example, class 3, Λ0, and if the absolute value of the difference is greater than T and the sign of the difference (e.g., y2(i)-y1(i)) is positive, the corresponding sample is assigned to, for example, class 1, Λ1. If the absolute value of the difference is greater than T and the sign of the difference y2(i)-y1(i) is negative, the corresponding sample is assigned to, for example, class 2, Λ2. For example, for a threshold value according to Λ1…Λ 2L For more than three classes, the corresponding sample is assigned to one of the more than three classes (e.g., from Λ0 to Λ 2L express).

[0038] and

[0039] and

[0040] Λ0=I∩(Λ1∪…∪Λ 2L ) c

[0041] This approach improves the accuracy of assigning samples to a class and provides successful partitioning (and / or classification) of the set of all sample locations.

[0042] According to one embodiment, the second denoising operation is configured to classify the samples into three categories according to:

[0043] Λ1={i∈I: (y2(i)-y1(i))>T} and Λ2={i∈I: (y2(i)-y1(i)<-T}

[0044] Λ0=I∩(Λ1∪Λ2) c

[0045] Wherein Λ1 is a set of samples (e.g., a set of pixels) associated with a first class, Λ2 is a set of samples (e.g., a set of pixels) associated with a second class, Λ0 is a set of samples (e.g., a set of pixels) associated with a third class, where T is a predetermined value, such as a fixed positive constant or preferably a positive value encoded in a bitstream representing the encoded video content, where I (=uppercase i) is a basic set of samples, where ∩ is a cut set operator - intersection, where U is a union operator - union, and where c is a complementary set operator. This new classification provides improved efficiency of the second denoising operation and allows the second denoising operation to be effectively adapted according to the pixel value changes caused by the first denoising operation. For example, the classification allows a discrete set of pixel value modifications to be performed in the second denoising operation.

[0046] According to one embodiment, the second denoising operation is configured to determine a combination of weighted differences (e.g. f(i)), such as a weighted sum, which combines a plurality of difference values ​​(e.g. y2(i)-y1(i)) between corresponding samples in a predetermined environment of the currently considered sample before the first denoising operation and after the first denoising operation, so as to obtain a combined difference value (e.g. f(i)) associated with, for example, the currently considered sample having sample index i. The difference associated with the currently considered sample may, for example, have a greater weighting than the difference associated with samples adjacent to the currently considered sample. Thus, an improved sample classification accuracy is provided, and the environment of the currently considered sample may also be taken into account in the classification. Thus, for example, dependencies between adjacent samples may be exploited, and the classification may be more "smooth", for example, due to the consideration of multiple sample value differences in a single classification step. Thus, the introduction of excessive artifacts in the second denoising operation may be avoided.

[0047] According to an embodiment, the second denoising operation is configured to determine the combined difference value f(i) according to:

[0048]

[0049] Wherein j is a sample index, N(i) is a set of (e.g., adjacent) samples or sample positions to be considered for determining a combined difference value associated with the sample under consideration with sample index i, wherein w(ij) (also denoted as w(i,j)) is a set of predetermined weighted values ​​(e.g., non-negative weighted values ​​or positive weighted values) associated with a plurality of sample positions in the environment of the sample under consideration, wherein y2(j)-y1(j) or y2(j)-y1(j) is the difference between the corresponding samples, e.g., with sample index j, before and after the first denoising operation. The weighted average difference value f(i) may optionally be used to construct Λ0…Λ2, e.g., as described above.

[0050] According to an embodiment, the second denoising operation is configured to select a quantization offset value, such as Q(d1), Q(d2), 0, for each sample i (e.g., a frame, such as each YUV component, or, for example, a group of sample positions sharing the same classification index l associated with the classification), depending on the classification C. For example, if the sample i under consideration is classified as the first class l=1, the offset value Q(d1) is added to the sample under consideration. For example, if the sample i under consideration is classified as the second class l=2, the offset value Q(d2) is added to the sample under consideration. For example, if the sample i under consideration is classified as the third class, the sample under consideration remains unchanged, so, for example, the value added to or subtracted from the pixel value can depend on the classification. Therefore, for example, a deviation in the pixel value associated with the difference between the pixel values ​​before and after the first denoising operation can be compensated.

[0051] According to an embodiment, the second denoising operation is configured to select the offset value using one or more offset values ​​encoded in the coded video information, for example using two offset values ​​Q(d1), Q(d2) encoded in the bitstream representing the coded video information. This embodiment minimizes the error between samples before and after the first denoising operation. Moreover, the effect of the second denoising operation can be optimized by appropriate signaling of the offset values ​​in the bitstream, while keeping the bitrate overhead fairly small, because the number of offset values ​​is relatively small, and because the offset values ​​can be updated only rarely.

[0052] According to one embodiment, the second denoising operation is configured to generate enhanced sample values ​​y according to, or more precisely, (or or ), where i is the index of the sample value currently being processed

[0053]

[0054] where dl (or d1) is the offset value corresponding to the class, I (= uppercase i) or l is the class index, Q is the quantization function, y or more precisely y(i) is the sample value input to the second denoising operation, for example a pixel value (which may be provided by the first denoising operation or based on the result of the first denoising operation, or it may be provided by the ALF process, for example), l is an indicator function that indicates into which class l the currently considered sample y or more precisely y(i) is classified. Thus, an accurate placement of the sample in the class is provided, which improves the classification operation.

[0055] According to an embodiment, the indicator function χΛ l , or more precisely, χΛ l (i) is defined as follows:

[0056]

[0057] as well as

[0058]

[0059] where Λ l is the class; 1 is the class index of each sample position i in the basic sample set I; i is the sample position in the basic sample set I, or the index representing the sample or sample position, C is the classification (for sample position i), ∈ is the set membership operator, is a non-set membership operator.

[0060] According to an embodiment, the second denoising operation is configured to use a first offset value, such as Q(d1), for all samples of the video frame classified as the first class (e.g., Λ1). Alternatively or supplementally, the second denoising operation is configured to use a second offset value, such as Q(d2), for all samples of the video frame classified as the second class (e.g., Λ1 or Λ2). Alternatively or supplementally, the second denoising operation is configured to use a third offset value, such as d0=0, for all samples of the video frame classified as the third class (e.g., Λ0).

[0061] According to an embodiment, the video decoder is configured to evaluate a bitstream flag, such as FLAG, to determine whether to perform the second denoising operation. For example, the video decoder is configured to selectively perform the second denoising operation depending on a bitstream flag that activates and deactivates the second denoising operation. Therefore, unnecessary filtering operations are minimized, which reduces the consumption of operating resources.

[0062] According to an embodiment, the video decoder is configured to apply the second denoising operation independently to different YUV components, for example independently to the Y component representing a single pixel and the U component and the V component. This embodiment is based on the finding that independent denoising of different color components improves denoising performance.

[0063] According to an embodiment, the second denoising operation is configured to classify the samples into a plurality of classes according to:

[0064] and and

[0065] Λ0=I∩(Λ1∪…∪Λ 2L ) c

[0066] where Λ l is a sample set associated with class 1, ..., 2L, such as a pixel set; Λ0 is a sample set associated with a class other than class 1, ..., 2L, such as a pixel set; T l-1 to T l is a predetermined value, such as a fixed positive constant or preferably a positive value encoded in the bitstream representing the encoded video content, I (= uppercase i) is the basic sample set, ∩ is the cut set operator - intersection, U is the union operator - union, and c is the complementary set operator. This new classification provides improved classification accuracy for multiple classes (e.g., more than three) and improved efficiency in detecting and correcting errors between samples before and after the first denoising operation.

[0067] According to an embodiment, T l-1 T l For the difference, for example, the difference y2(i)-y1(i), or Y-Y1, or Y2-Y1, or Y2-Y1, or Y2-Y1 between the samples before the first denoising operation (e.g., a combination of DBFV and DBFH or DBFV, or DBFV and DBFH, or DBFV operation) and after the first denoising operation (e.g., a combination of DBFV and DBFH or DBFV, or DBFV and DBFH, or DBFV operation). is a symmetric class boundary, for example symmetric about zero, so as to obtain the classification of each sample, for example, pixel location.

[0068] According to an embodiment, the second denoising operation is configured to select (or associate) a quantization offset value, such as Q(d1), Q(d2), ..., Q(d2L), 0, for each sample 1 (e.g., a frame, such as each YUV component, or, for example, a group of sample positions sharing the same classification index 1 associated with the classification) depending on the classification C. For example, when L=1, if the sample under consideration is classified as the first class l=1, the offset value Q(d1) is added to the sample under consideration, if the sample i under consideration is classified as the second class l=2, the offset value Q(d2) is added to the sample under consideration, and if the sample i under consideration is classified as the third class l=0, the sample under consideration remains unchanged. Therefore, an improved accuracy of placing the sample in the class is provided. Therefore, the value added to or subtracted from the pixel value can, for example, depend on the classification. For example, a deviation of the pixel value associated with the difference between the pixel values ​​before and after the first denoising operation can be compensated.

[0069] According to an embodiment, the second denoising operation is configured to select the offset value using the offset value encoded in the encoded video information. For example, the offset value may be selected using two offset values ​​Q(d1) and Q(d2) (when L=1) encoded in the bitstream representing the encoded video information. The offsets Q(d1), ..., Q(d2) are 2L ) should always be an even number and d0=0. Therefore, the second denoising operation can be applied to multiple CTUs or frames, thereby offsetting Q(d1), ..., Q(d 2L ) is encoded. This embodiment uses the offset information transmitted in the video bitstream to minimize errors. Therefore, the effect of the second denoising operation can be optimized by appropriate signaling of the offset value in the bitstream, while the bit rate overhead is kept low due to the relatively small number of offset values ​​and the few possible offset value updates.

[0070] According to an embodiment, the second denoising operation is configured to generate enhanced sample values ​​according to Or more precisely, Y(i) or or Where i is the index of the sample value currently being processed

[0071]

[0072] where d l is the offset value corresponding to the class, or 1 is the class index, Q is the quantization function, Y or more precisely Y(i) is the sample value input to the second denoising operation, such as a pixel value (which may be provided by the first denoising operation or based on the result of the first denoising operation, or it may be provided by the ALF process, for example), where χΛ lis an indicator function indicating into which class 1 the currently considered sample Y or more precisely Y(i) is classified. Thus, an accurate placement of the sample in the class is provided, which improves the classification operation. Furthermore, an enhanced sample value can be obtained efficiently.

[0073] According to an embodiment, the indicator function χΛ l , or more precisely, χΛ l (i) is defined as follows:

[0074]

[0075] as well as

[0076]

[0077] where Λ l is the class, l is the class index of each sample position i in the basic sample set I; i is the sample position in the basic sample set I, or the index representing the sample or sample position; C is the classification (for sample position i); ∈ is the set membership operator; is a non-set membership operator.

[0078] According to one embodiment, the second denoising operation is configured to detect the noises classified into the corresponding classes [e.g., Λ1, ..., Λ 2L ] use the first to 2L offset values, e.g., Q(d1), ...Q(d2L) for all samples of the video frame or CTU in ]. Alternatively or in addition, the second denoising operation is configured to use an offset value, e.g., d0=0, for all samples of the video frame classified as a class complementary to I (=capital i) (the class does not include the first to 2L, e.g., Λ0).

[0079] According to an embodiment, the video decoder is configured to evaluate a bitstream flag, such as FLAG, to determine whether to perform the second denoising operation. For example, the video decoder is configured to selectively perform the second denoising operation depending on a bitstream flag that activates and deactivates the second denoising operation. Therefore, unnecessary filtering operations are minimized, which reduces the consumption of operating resources.

[0080] According to an embodiment, the video decoder is configured to apply the second denoising operation independently to different YUV components, for example independently to the Y component representing a single pixel and the U component and the V component. This embodiment is based on the finding that independent denoising of different color components improves denoising performance.

[0081] According to an embodiment, the second denoising operation uses the same set of parameters, such as T0..., TL (ie, symmetric boundary), Q(d1)..., Q(d2L), for processing of a frame or each CTU.

[0082] According to an embodiment, the first denoising operation is a deblocking filtering operation, such as DBFV or a DBFV operation in a first direction, and wherein the second denoising operation is a first sample adaptive offset operation, such as SAOV or a SAOV operation, for example using associated parameters or an associated set of parameters.

[0083] According to an embodiment, the video decoder is configured to use the result of the second denoising operation, e.g. It is provided by SAOV, or a SAOV operation, to apply a third denoising operation, such as DBFH, or a DBFH operation, which is a deblocking filtering operation in a second direction different from the first direction. The video decoder is supplementarily configured to apply a fourth denoising operation based on a result of the third denoising operation, such as Y3, for example based on third denoising sample information, such as based on DBFH, or an output Y3 of the DBFH operation, the fourth denoising operation being a sample adaptive offset operation, such as SAOH, or a SAOH operation. The additional SAO filtering further reduces the encoding artifacts of DBF.

[0084] According to an embodiment, the second denoising operation and the fourth denoising operation use different parameters or parameter sets, such as T0, ...TL; Q(d1) ..., Q(d2L) for processing of a frame, or for a CTU. Using different parameter sets for the second and fourth denoising operations can provide further correction of quality errors.

[0085] According to an embodiment, the video decoder is configured to apply a fifth denoising operation, e.g., SAO, or SAO operation, or ALF, or ALF operation, or a combination of SAO and ALF or SAO and ALF operations, based on a result of a fourth denoising operation (e.g., SAOH or SAOH operation).

[0086] According to an embodiment, the fifth denoising operation is, for example, another SAO, or a SAO operation that uses different parameters or parameter sets for different Q(dl) of different CTUs.

[0087]

[0088] where XΛ l is an indicator function, for example defined by

[0089]

[0090] And Λ l ={i∈I:C(i)=l},

[0091] The fifth denoising operation uses different parameters or parameter sets, such as different Q(d1) for different CTUs.

[0092] Embodiments of the present invention create a video encoder for providing encoded video information based on input video information, such as a "video input" x(i) or X. The video encoder is configured to obtain first sample information, such as noisy first sample information, based on the input video information, for example using transformation, quantization, inverse quantization and inverse transformation, and optionally using prediction, which takes into account, for example, transformation of the input video information and / or prediction residuals into the spectral domain, quantization and prediction of transform coefficients. The first sample information preferably also takes into account inverse quantization and inverse transformation performed at the audio decoder side, such as values ​​of a YUV image representation, such as pixel values, such as y1, Y1, Y2, Etc. The first sample information may, for example, represent a reconstructed noise image, which may be obtained by a video decoder, for example, using entropy decoding, inverse quantization, and inverse transformation, for example, before a DBF, or deblocking filter (DBF) operation, or before a DBFV, or deblocking filter (DBFV) operation for vertical edges, or before a DBF and DBFV, or a combination of DBF and DBFV operations, and / or before a SAOV, or sample adaptive offset (SAOV) operation for vertical edges, and / or before a DBFH, or deblocking filter (DBFH) operation for horizontal edges, or before a SAO, or sample adaptive offset (SAO) operation, and / or before an ALF, or adaptive loop filter (ALF), for example, when a PSAO, or post-sample adaptive offset (PSAO) filtering operation is applied.

[0093] According to an embodiment, the video encoder is configured to apply a first denoising operation, such as a noise reduction operation, such as a combination of DBF, SAO, ALF, or a combination of DBF, SAO, ALF operations, or a combination of DBF and SAO, or a combination of DBF and SAO operations, or one of the denoising operations, such as when applying PSAO, or a PSAO filtering operation, or DBFV, or a DBFV operation and / or SAOV, or a SAOV operation and / or DBFH, or a DBFH operation, so as to obtain first denoised sample information, such as an output of SAO, or an output of a SAO operation, or an ALF, or an output of an ALF operation, such as y2, based on the first sample information, such as an output Y based on the combination of DBFV and DBFH or DBFV and DBFH operations, or based on the output Y2 of the DBFV or DBFV operation. The video encoder is configured to determine parameters of a second denoising operation (e.g. PSAO or PSAO operation), such as FLAG, Q(d1), Q(d2), based on a result of the first denoising operation, such as based on first denoising sample information, such as based on SAO or ALF, or an output of a SAO or ALF operation, such as a single threshold T, the single threshold T can be selected as a fixed positive constant, wherein the threshold T can be determined, for example, based on a rate-distortion cost at the encoder. The video encoder is optionally further configured to perform a second denoising operation. The video encoder is configured to depend on a difference, such as depending on y2(i)-y1(i)) or depending on Y2-Y1 or depending on Y-Y1, such as the difference between the output of a DBFV or DBFV operation and a sample before a DBFV or DBFV operation, or the difference between samples before a DBFH or DBFH operation and after a DBFH or DBFH operation, and For example, the difference between the output of DBFH or the output of the DBFH operation and the SAOV or the output of the SAOV operation, such as the difference between the samples before the first denoising operation and after the first denoising operation, to adjust or (or, for example) adapt the second denoising operation to obtain encoded video information.

[0094] This embodiment is based on the following discovery: adapting the second denoising operation according to the difference before and after the first denoising operation can correct the quality reduction caused by applying the first denoising operation. Therefore, this embodiment provides effective noise filtering with maintained video quality. For example, the difference between the samples before the first denoising operation and after the second denoising operation indicates in which areas the first denoising operation has a particularly large impact, and therefore when performing the subsequent second denoising operation, the knowledge of the areas in which the first denoising operation has a particularly large impact can be taken into account. For example, one or more parameters of the second denoising operation can be adjusted (e.g., in a locally variable manner) depending on the (local) difference between the sample (or corresponding multiple samples) before and after the first denoising operation. In other words, local changes in the difference can be used to locally adapt the second denoising operation. For example, it can be considered that the first type of second denoising operation or the first parameter or parameter set of the second denoising operation can provide the best results (and therefore should be applied) in the area where the difference is relatively small, while the second type of second denoising operation or the second parameter or parameter set of the second denoising operation can provide the best results (and therefore should be used) in the area where the difference is relatively large.

[0095] Therefore, it has been found to be advantageous to adapt the characteristics of the second denoising operation depending on the difference, because the difference describes, for example, how much influence the first denoising operation has on the (first) sample information (e.g. in different areas of the image), and because a corresponding (possibly locally varying) adaptation of the second denoising operation leads to a good image quality while keeping latency and computational effort reasonably small.

[0096] According to an embodiment, the second denoising operation is configured to classify samples (e.g., pixels, such as y(i), where y can be PSAO or an input image to a PSAO operation, or, for example, with a sample index i) into multiple classes, such as groups of samples, Λ1, Λ2, Λ0, based on the difference (e.g., y2(i)-y1(i)) between the samples before the first denoising operation (e.g., y1(i)) and the samples after the first denoising operation (e.g., y2(i)). The video encoder is also configured to determine parameters of the second denoising operation, for example, based on the classification. The second denoising operation is optionally configured to modify the samples, for example, based on the classification.

[0097] This embodiment is based on the discovery that filtering performance is substantially dependent on how the classification of samples is performed. Samples are successfully classified based on the difference between the noise signal and the denoised signal, and the classification is used to modify the samples in the second denoising operation so that the errors in the samples that occurred during filtering in the first denoising operation are minimized. In addition, based on the traditional classification, the sample modification in the second denoising operation can be performed faster, thereby reducing encoding latency.

[0098] According to an embodiment, the second denoising operation is configured to conditionally add an offset to a plurality of samples depending on the classification of the respective samples. For example, an offset is selected for each group of sample values ​​or sample positions that share the same class index. The classification is, for example, based on the difference between a noisy (e.g., unfiltered) signal and a denoised (e.g., filtered) signal. In this embodiment, effective removal of coding artifacts is provided.

[0099] With respect to the function that the second denoising operation (e.g., PSAO, SAOV, or SAOH, or a PSAO, SAOV, or SAOH operation) is configured to classify samples with sample indexes ...', the following should be noted. The second denoising operation may consider a sample adaptive offset operation, such as SAO, SAOV, SAOH (fourth), or a SAO operation, SAOV operation, SAOH (fourth) operation, after one or a combination of DBF, or DBF operation, DBFV, or DBFV operation, DBFV and SAOV, or DBFV and SAOV operations, SAOV or DBFH, or SAOV and DBFH operations, DBFV and SAOV or DBFV, or DBFV and SAOV or DBFV operations, or DBFV and SAOV and DBFV, or DBFV and SAOV and DBFV operations, or a combination of DBFV and DBFH or DBFV and DBFH operations that may be connected in series and may be performed continuously.

[0100] In one embodiment, the "samples" are y(i) and y, which are decoded input images for PSAO or PSAO filtering operations. According to one embodiment, y (e.g., input image, e.g., input image of PSAO or PSAO operation) and y2 (e.g., first denoised image) are given as the output of ALF or ALF operation (e.g., y) and the output of SAO or SAO operation (e.g., y2), respectively. In this case, y and y2 are different.

[0101] On the other hand, in another embodiment, when y2 is selected as ALF or the output of the ALF operation, the input image y (eg, the input image of the PSAO or the PSAO operation) and y2 are the same image.

[0102] According to an embodiment, in some embodiments, the input image y (e.g., the input image of PSAO or a PSAO operation) should be the output of ALF or an ALF operation, because in some embodiments, PSAO or a PSAO operation is applied after a loop filtering process (e.g., a combination of DBF+SAO+ALF or DBF+SAO+ALF operations).

[0103] In one embodiment, the first denoising operation comprises a deblocking filter, such as DBF, or DBF operation; or such as DBFH and / or DBFV, or DBFH and / or DBFV operation, which for example uses low-pass filtering of pixel values ​​along block boundaries, and / or SAO, or sample adaptive offset filtering operation to remove artifacts along block boundaries. In such an embodiment, different coding artifacts are effectively removed.

[0104] In one embodiment, a sample adaptive offset filter, such as PSAO, SAOV or SAOH, or a PSAO, SAOV or SAOH operation, is configured to classify samples of an image into a plurality of classes, for example depending on one or more edge patterns (e.g. along one or more directions) in an environment of the considered sample, for example including the considered sample, and / or depending on a sample value of the considered sample (e.g. a pixel, such as y(i), where y may be, for example, a decoded input image for the SAO or SAO operation with sample index i). This embodiment is based on the finding that an efficient partitioning of the set of all sample positions improves the filtering performance at each position. For example, different effects of different edge patterns on pixel values ​​(where, for example, an error in a pixel value may depend on the presence of one or more edges in the environment of the considered pixel) may be taken into account (e.g., at least partially compensated) in this manner. Furthermore, for example, a correlation of a pixel value error with an actual pixel value may be taken into account (and, for example, at least partially compensated) in this manner.

[0105] As a remark, it should be noted that conventional SAO or SAO operation may apply classification based on the following two characteristics (or at least one of the characteristics), for example:

[0106] 1) For example, assign each sample to one of, for example, 5 classes based on 1-D directional edge patterns, for example, along 4 directions, such as horizontal, vertical, 135° diagonal, and 45° diagonal.

[0107] 2) Classify each sample based on the sample value range, for example, equally divide into 32 frequency bands.

[0108] For example, for each CTU (or for multiple coding tree units) a conventional SAO or SAO operation is applied, where the classification is selected from, for example, 5 different classifications as described above - for example 4 classifications based on edge information (horizontal + vertical + two diagonals) + for example a classification based on sample value range, which simply checks the sample value itself to assign it to, for example, one of 32 frequency bands.

[0109] According to an embodiment (or according to an aspect of the invention), the proposed classification is based on a different feature, namely the difference between the noisy (unfiltered) signal and the denoised (filtered) signal, for example when compared with the classification used in SAO or SAO operations (which are for example based on an evaluation of samples of a single image or frame or CTU). This embodiment is based on the finding that taking into account the difference between the unfiltered and filtered signal allows correcting quality errors that occur during the filtering operation.

[0110] In one embodiment, the first denoising operation (or the above-mentioned first denoising operation) includes an ALF or adaptive loop filtering (ALF) operation, which uses an impulse response to perform filtering of, for example, pixel values, such as linear filtering, or such as finite impulse response filtering or such as infinite impulse response filtering, and the filter coefficients are encoded in the bitstream. This embodiment is based on the discovery that different types of adaptive loop filtering (ALF) increase coding gain. In addition, such filtering (whose coefficients are encoded in the bitstream) can be adapted to specific video content and therefore provides a good compromise between bit rate and quality (for example, when compared to the case where the full bit budget is spent on the encoding of pixel values).

[0111] In one embodiment, the video encoder is configured to apply a second denoising operation to sample information, such as an image; or such as a pixel value; or such as a YUV value of a YUV video representation; for example: y(i), which is obtained using a combination (e.g., sequence) of: one or more deblocking filtering operations, such as DBF, or DBF operations, or such as DBFV and / or DBFH, or DBFV and / or DBFH operations; one or more sample adaptive offset filtering operations, such as SAO, or SAO operations, or such as SAOH and / or SAOV, or SAOH and / or SAOV operations; and ALF, or an adaptive loop filtering (ALF) operation. This embodiment is based on the finding that using a plurality of different processing steps as a first denoising operation allows for good initial denoising. Moreover, it has been found that the total change in pixel values ​​through this sequence of different processing steps (which is represented by the difference between the sample before the first denoising operation and the sample after the first denoising operation) constitutes a reliable criterion for adapting the (subsequent) second denoising operation.

[0112] In one embodiment, the second denoising operation is configured to be applied to a plurality of coding tree units (CTUs) defining a video frame (e.g., a picture as a whole). Alternatively or additionally, the second denoising operation is configured to be applied to a video frame, e.g., a picture as a whole consisting of coding tree units (which may, for example, define a video frame).

[0113] For example, a CTU may be a basic processing unit, and a CTU may, for example, use a larger block structure of, for example, up to 64×64 pixels, and may better subdivide an image into structures of variable sizes. For example, High Efficiency Video Coding (HEVC) uses only CTUs of up to 64×64. However, larger CTUs (e.g., 128×128) may be used in a quadtree plus binary tree (QTBT) setting adopted by the first draft of a new video coding standard. This embodiment is based on the finding that block-based classification may reduce the complexity of the filtering process.

[0114] In one embodiment, samples are classified into multiple classes only when one or more predetermined thresholds are reached, such as T, which can be selected as a fixed positive constant, for example, depending on human vision, or a single threshold T, such as a single threshold T, which can be encoded at the encoder or selected as a fixed positive constant, such as the threshold T can be determined based on the rate distortion (RD) cost at the encoder. Therefore, for example, unnecessary filtering operations are eliminated, which reduces encoding latency and reduces the consumption of operating resources.

[0115] In one embodiment, the second denoising operation is configured to depend on the corresponding sample (e.g., y1[i], or Y1 or ) and the corresponding sample after the first denoising operation (e.g. y2[i]) satisfies a first condition (e.g. y2(i)-y1(i)>T) or a second condition (e.g. y2(i)-y1(i)<-T) or a third condition (e.g. -T<=y2(i)-y1(i)<=T) to classify the sample (e.g., the currently considered sample, e.g., pixel) into three or more classes. Alternatively, the second denoising operation is configured to classify the sample (e.g., f(i)) depending on whether a difference between the corresponding sample before and after the first denoising operation (e.g., y2[i]) and the corresponding sample after the first denoising operation (e.g., y2[i]) satisfies a first condition (e.g., y2(i)-y1(i)>T) or a second condition (e.g., y2(i)-y1(i)<-T) or a third condition (e.g., -T<=f(i)<=T) based on a combination of multiple differences (e.g., weighted differences) between the corresponding samples before and after the first denoising operation, e.g., in a predetermined environment of the currently considered sample, satisfies the first condition (e.g., f(i)>T) or a second condition (e.g., f(i)<-T) or a third condition (e.g., -T<=f(i)<=T). Thus, an improved trade-off between compression performance and encoding speed is provided. Furthermore, the classification may be used to adapt the second denoising operation to the effectiveness of the first denoising operation, or to changes in one or more pixel values ​​caused by the first denoising operation.

[0116] In one embodiment, the second denoising operation is configured to depend on the first denoising operation before (e.g., y1[i] or e.g., Y1 or ) and the corresponding samples after the first denoising operation (e.g., y2, or, for example, Y2, Y3, or Y), and depends on the magnitude or absolute value of the difference between the corresponding samples before the first denoising operation (e.g., y1[i] or, for example, Y1 or ) and the sign of the corresponding sample after the first denoising operation (e.g.: y2, or for example Y2, Y3 or Y) to classify the samples into three or more classes. If the absolute value of the difference is less than T or equal to T, the corresponding sample is assigned to class 3, Λ3, for example, and if the absolute value of the difference is greater than T and the sign of the difference (e.g. y2(i)-y1(i)) is positive, the corresponding sample is assigned to class 1, Λ1, for example, and if the absolute value of the difference is greater than T and the sign of the difference (e.g. y2(i)-y1(i)) is negative, the corresponding sample is assigned to class 2, Λ2, for example, or for example according to Λ1, ..., Λ 2L More than three categories, with more than one threshold assigned according to the following

[0117] and

[0118] and

[0119] Λ0=I∩(Λ1∪...∪Λ 2L ) c

[0120] This approach improves the accuracy of assigning samples to a class and provides successful partitioning (and / or classification) of the set of all sample locations.

[0121] In one embodiment, the second denoising operation is configured to classify samples into three categories according to the following.

[0122] Λ1={i∈I: (y2(i)-y1(i))>T} and Λ2={i∈I: (y2(i)-y1(i)<-T}

[0123] Λ0=I∩(Λ1∪Λ2) c

[0124] Wherein Λ1 is a set of samples (e.g., a set of pixels) associated with a first class, Λ2 is a set of samples (e.g., a set of pixels) associated with a second class, Λ0 is a set of samples (e.g., a set of pixels) associated with a third class, T is a predetermined value, such as a fixed positive constant or preferably a positive value encoded in a bitstream representing the encoded video content; I is a basic sample set; ∩ is a cut set operator - intersection; ∪ is a union operator - union; and c is a complementary set operator. This new classification provides improved efficiency of the second denoising operation and allows the second denoising operation to be effectively adapted according to the pixel value changes caused by the first denoising operation. For example, the classification allows a discrete set of pixel value modifications to be performed in the second denoising operation.

[0125] In one embodiment, the second denoising operation is configured to determine a combination of weighted differences (e.g. f(i)), e.g. a weighted sum, which combines a plurality of difference values ​​(e.g. y2(i)-y1(i)) between corresponding samples in a predetermined environment of the currently considered sample before the first denoising operation and after the first denoising operation, so as to obtain a combined difference value (e.g. f(i)) associated with, e.g., the currently considered sample having sample index i. The differences associated with the currently considered sample may, for example, have a greater weighting than the differences associated with samples adjacent to the currently considered sample. Thus, improved accuracy of sample classification is provided, and, for example, the environment of neighboring samples currently considered may be utilized, and, for example, the classification may be more "smooth" due to the consideration of multiple sample value differences in a single classification step. Thus, the introduction of excessive artifacts in the second denoising operation may be avoided.

[0126] According to an embodiment, the second denoising operation is configured to determine the combined difference value f(i) according to

[0127]

[0128] Wherein j is a sample index, N(i) is a set of samples to be considered for determining a combined difference value associated with a sample under consideration with sample index i, such as a set of adjacent samples, or, for example, a set of sample positions; w(ij), for example also denoted as w(i, j), is a set of predetermined weighted values ​​(e.g., non-negative weighted values ​​or positive weighted values) associated with a plurality of sample positions in the environment of the sample under consideration; y2(j)-y1(j) or y2(j)-y1(j) is the difference between the corresponding samples, for example, with sample index j, before and after the first denoising operation. Thus, an improved sample classification accuracy is provided. In this embodiment, the weighted average difference f(i) can optionally be used to construct Λ0, ..., Λ2, for example, as disclosed herein.

[0129] In one embodiment, the second denoising operation is configured to select a quantization offset value, e.g. Q(d1), Q(d2), 0, for each sample i (e.g. a frame, e.g. each YUV component or a group of sample set positions sharing the same classification index 1 associated with the classification) depending on the classification C. For example, if the sample under consideration is classified as a first class l=1, an offset value Q(d1) is added to the sample under consideration, wherein if the sample under consideration i is classified as a second class l=2, an offset value Q(d2) is added to the sample under consideration, and wherein if the sample under consideration i is classified as a third class, the sample under consideration remains unchanged. Thus, the value added to or subtracted from the pixel value may, for example, depend on the classification. Thus, for example, a deviation in the pixel value associated with the difference between the pixel values ​​before and after the first denoising operation may be compensated.

[0130] In one embodiment, the second denoising operation is configured to select the offset value using one or more offset values ​​encoded in the encoded video information, for example using two offset values ​​Q(d1), Q(d2) encoded in the bitstream representing the encoded video information. This embodiment minimizes the error between samples before and after the first denoising operation. Moreover, the effect of the second denoising operation can be optimized by appropriate signaling of the offset values ​​in the bitstream, while keeping the bitrate overhead fairly small, because the number of offset values ​​is relatively small, and because the offset values ​​can be updated only rarely.

[0131] In one embodiment, the second denoising operation is configured to generate enhanced sample values ​​according to Or more precisely, Where i is the index of the sample value currently being processed

[0132]

[0133] where d l is the offset value corresponding to the class; l is the class index; Q is the quantization function; y, or more precisely, y(i), is the sample value input to the second denoising operation, such as a pixel value, which can be provided by the first denoising operation or based on the result of the first denoising operation, or it can be provided by ALF processing, for example; l is an indicator function that indicates into which class 1 the currently considered sample y (or more precisely y(i)) is classified. Thus, an accurate placement of the sample in the class is provided, which improves the classification operation.

[0134] In one embodiment, the indicator function χΛ l , or more precisely xΛ l (i) is defined as follows:

[0135] and

[0136] Λ1 is the class; where l is the class index of each sample position i in the basic sample set I; i is the sample position in the basic sample set I, or an index representing a sample or sample position; C is the classification (for sample position i); ∈ is the set membership operator; is a non-set membership operator.

[0137] In one embodiment, the second denoising operation is configured to use a first offset value, such as Q(d1), for all samples of the video frame classified as the first class (e.g., Λ1). Alternatively or in addition, the second denoising operation is configured to use a second offset value, such as Q(d2), for all samples of the video frame classified as the second class (e.g., Λ2). Alternatively or in addition, the second denoising operation is configured to use a third offset value, such as d0=0, for all samples of the video frame classified as the third class (e.g., Λ0).

[0138] In one embodiment, the video encoder is configured to evaluate a bitstream flag, such as FLAG, to determine whether to perform the second denoising operation, for example, the video encoder is configured to selectively perform the second denoising operation depending on a bitstream flag that activates and deactivates the second denoising operation. Therefore, unnecessary filtering operations are minimized, which reduces the consumption of operating resources.

[0139] In one embodiment, the video encoder is configured to apply the second denoising operation independently to different YUV components, for example independently to the Y component representing a single pixel and the U component and the V component. This embodiment is based on the finding that independent denoising of different color components improves denoising performance.

[0140] In one embodiment, the video encoder is configured to determine one or more offset values, such as d1 or Q(d1) or d2 or Q(d2), used in the second denoising operation based on the input video representation and depending on the sample classification performed based on the difference between the samples before the first denoising operation and after the first denoising operation. Thus, improved accuracy of placing samples in classes is provided.

[0141] In one embodiment, the video encoder is configured to determine an offset value (dl) associated with a set of samples classified into a given class (e.g., a class with class index l) depending on a deviation (e.g., a mean deviation) between second denoised input samples (e.g., y(i)) classified into the given class and corresponding samples (e.g., x(i)) of the input video information.

[0142] In one embodiment, the video encoder is configured to determine an offset value (dl) associated with a set of samples classified into a given class so that a mean square error between a second denoised input sample (e.g., y(i)) classified into the given class and a corresponding sample (e.g., x(i)) of the input video information is minimized or brought below a predetermined threshold.

[0143] In one embodiment, the video encoder is configured to determine the offset value d associated with a set of samples classified into a given class with class index l according to l .

[0144]

[0145] Among them #(Λ l ) is the number of samples classified into the class with class index l; Λ l is a set of samples classified into a class with class index l; i is a sample index; x(i) is a sample value of the input video information; and y(i) is a sample value input to the second denoising operation.

[0146] In one embodiment, the second denoising operation is configured to classify the samples into a plurality of classes according to:

[0147] and

[0148] and

[0149] Λ0=I∩(Λ1∪…∪Λ 2L ) c

[0150] where Λ l is a sample set associated with class 1, ..., 2L, such as a pixel set; Λ0 is a sample set associated with a class other than class 1, ..., class 2L, such as a pixel set; Tl-1 to Tl or T l-1 to T l is a predetermined value, such as a fixed positive constant or preferably a positive value encoded in the bitstream representing the encoded video content; I (= uppercase i) is the basic sample set; ∩ is the cut set operator - intersection; ∪ is the union operator - union; and c is the complementary set operator. This new classification provides improved classification accuracy for multiple classes (more than three) and improved efficiency in detecting and correcting errors between samples before and after the first denoising operation.

[0151] In one embodiment, T1-1 to T1 or T l-1 to T l , for differences, e.g., the differences y2(i)-y1(i), or Y-Y1, or Y2-Y1, or between samples before a first denoising operation (e.g., a combination of DBFV and DBFH or DBFV, or DBFV and DBFH, or DBFV operation) and after a first denoising operation (e.g., a combination of DBFV and DBFH or DBFV, or DBFV and DBFH, or DBFV operation). is a symmetric class boundary, for example symmetric about zero, so as to obtain the classification of each sample, for example, pixel location.

[0152] In one embodiment, the second denoising operation is configured to select (or associate) a quantized offset value, such as Q(d1), Q(d2), ..., Q(d2L), 0, for each sample i, such as a frame, such as each YUV component, or a group of sample positions sharing the same classification index l associated with the classification, depending on the classification C. For example, when L=1, if the sample under consideration is classified as the first class l=1, the offset value Q(d1) is added to the sample under consideration. For example, if the sample under consideration i is classified as the second class l=2, the offset value Q(d2) is added to the sample under consideration. For example, if the sample under consideration i is classified as the third class l=0, the sample under consideration remains unchanged. Therefore, an improved accuracy of placing samples in a class is provided.

[0153] In one embodiment, the second denoising operation is configured to select the offset value using an offset value encoded in the encoded video information, for example using two offset values ​​Q(d1) and Q(d2) encoded in the bitstream representing the encoded video information (when L=1). The offsets Q(d1), ..., Q(d 2L ) should always be an even number and d0=0. Therefore, the second denoising operation can be applied to, for example, multiple CTUs or frames, thereby offsetting Q(d1), ..., Q(d 2L ) is encoded. This embodiment uses the offset information transmitted in the video bitstream to minimize the error.

[0154] In one embodiment, the second denoising operation is configured to generate an enhanced sample value Y according to: or Where i is the index of the sample value currently being processed

[0155]

[0156] Wherein dl is the offset value corresponding to the class; l is the class index; Q is the quantization function; Y, or more precisely, Y(i), is the sample value input to the second denoising operation, such as a pixel value, which can be provided by the first denoising operation or based on the result of the first denoising operation, or it can be provided by the ALF process, for example; l is an indicator function that indicates into which class l the currently considered sample Y (or, for example, more precisely, Y(i)) is classified. Thus, an accurate placement of the sample in the class is provided, which improves the classification operation. In addition, an enhanced sample value can be obtained efficiently.

[0157] In one embodiment, the indicator function χΛ l or more precisely χΛ l (i) is defined as follows:

[0158] and

[0159] Λ l is a class; where l is the class index of each sample position i in the basic sample set I; i is a sample position in the basic sample set I, or, for example, an index representing a sample or a sample position; C is a classification (for sample position i); ∈ is a set membership operator; and is a non-set membership operator.

[0160] In one embodiment, the second denoising operation is configured to perform denoising on the pixels classified into the corresponding classes (eg, Λ1, ..., Λ 2L ) uses the first to 2L offset values, e.g., Q(d1), ...Q(d2L), for all samples of the video frame or CTU. Alternatively or additionally, the second denoising operation is configured to use an offset value, e.g., d0=0, for all samples of the video frame classified as a class complementary to I (=uppercase i), e.g., the class does not include the first to 2L, e.g., Λ0.

[0161] In one embodiment, the video encoder is configured to evaluate a bitstream flag, such as FLAG, to determine whether to perform the second denoising operation, for example, the video decoder is configured to selectively perform the second denoising operation depending on a bitstream flag that activates and deactivates the second denoising operation. Therefore, unnecessary filtering operations are minimized, which reduces the consumption of operating resources.

[0162] In one embodiment, the video encoder is configured to apply the second denoising operation independently to different YUV components, for example independently to the Y component representing a single pixel and the U component and the V component. This embodiment is based on the finding that independent denoising of different color components improves denoising performance.

[0163] In one embodiment, the second denoising operation uses the same set of parameters, eg, T0..., TL (which is a symmetric boundary); Q(d1)..., Q(d2L), for processing of a frame or each CTU.

[0164] In one embodiment, the first denoising operation is a deblocking filtering operation, such as DBFV or a DBFV operation in a first direction, and wherein the second denoising operation is a first sample adaptive offset, such as SAOV or a SAOV operation, such as using associated parameters or an associated set of parameters.

[0165] According to an embodiment, the video decoder is configured to use the result of the second denoising operation, e.g. Provided by SAOV, or SAOV operation as an example, to apply a third denoising operation, such as DBFH, or DBFH operation, which is, for example, a deblocking filtering operation in a second direction different from the first direction. In this embodiment, the video decoder is further configured to apply a fourth denoising operation based on the result of the third denoising operation, such as Y3, for example based on the third denoising sample information, such as based on DBFH, or the output Y3 of the DBFH operation, the fourth denoising operation is a sample adaptive offset operation, such as SAOH, or SAOH operation.

[0166] In one embodiment, the second denoising operation and the fourth denoising operation use different parameters or parameter sets, such as T0...TL; Q(d1)..., Q(d2L), for processing of frames, or for example for CTUs.

[0167] In one embodiment, the video decoder is configured to apply a fifth denoising operation, e.g., SAO, or SAO operation, or ALF, or ALF operation, or a combination of SAO and ALF or SAO and ALF operations, based on the result of the fourth denoising operation (e.g., SAOH or SAOH operation).

[0168] According to an embodiment, the fifth denoising operation is, for example, another SAO, or a SAO operation that uses different parameters or parameter sets, such as different Q(d1), for different CTUs.

[0169]

[0170] where XΛ l is an indicator function, for example defined by

[0171]

[0172] and

[0173] Λ l ={i∈I:C(i)=l},

[0174] The fifth denoising operation uses different parameters or parameter sets, such as different Q(d1), for different CTUs.

[0175] Embodiments of the present invention create a method for providing decoded video information based on encoded video information (e.g., video input). The method includes obtaining first sample information, such as noise sample information, such as values ​​of a YUV image representation, such as pixel values, such as y1, based on the encoded video information, such as using entropy decoding, inverse quantization and inverse transformation of a bitstream, and optionally using prediction. The method also includes, based on the first sample information, applying a first denoising operation, such as a denoising operation; for example: a combination of DBF, SAO, ALF, or a combination of DBF, SAO, ALF operations, or a combination of DBF and SAO, or a combination of DBF and SAO operations, or one of the denoising operations, or a combination of DBFV and DBFH, or a combination of DBFV and DBFH operations, or for example only DBFV or DBFV operations, so as to obtain first denoised sample information, such as the output of SAO or SAO operations, or the output of ALF or ALF operations, or the output of the combination of DBFV and DBFH, or the output of the combination of DBFV and DBFH operations, or the output of DBFV or DBFV operations, such as y2. The deblocking filter DBF includes, for example, one or two filtering operations, namely, deblocking filter for vertical edges (DBFV) or / and deblocking filter for horizontal edges DBFH, which means DBF=DBFV or DBFH or DBFV+DBFH, and the deblocking filter can be a traditional deblocking filter.

[0176] The method according to this embodiment is based on the same considerations as the decoder described above.In addition, this disclosed embodiment may optionally be supplemented by any other features, functions and details disclosed herein in conjunction with the decoder, whether alone or in combination.

[0177] The method further includes applying a second denoising operation, such as PSAO, or a PSAO operation, or such as SAO, or a SAO operation, such as SAOV, or a SAOV operation, based on a result of the first denoising operation, such as based on the first denoising sample information, such as based on an output of SAO or ALF, or based on an output of a SAO operation or an ALF operation, or based on an output Y of a combination of DBFV and DBFH, or DBFV and DBFH operations, or based on an output Y2 of DBFV or a DBFV operation. The method further includes adjusting or, for example, adapting the second denoising operation, such as SAO, or a SAO operation, or SAOV, or a SAOV operation. Depends on the difference (e.g. y2(i)-y1(i) or Y-Y1 or Y2-Y1) between samples before and after the first denoising operation (e.g. DBFV and DBFH or DBFV, or a combination of DBFV and DBFH or DBFV operations) to obtain decoded video information.

[0178] SAOV and SAOH, or SAOV and SAOH operations herein indicate classification of SAOV and SAOH, which are based on Y2-Y1 or Y-Y1, respectively, such as the difference between the output of a DBFV or DBFV operation and the sample before the DBFV or DBFV operation, or the difference between the samples before the DBFH or DBFH operation and after the DBFH or DBFH operation, and For example, DBFH or the difference between the output of a DBFH operation and SAOV or the output of a SAOV operation.

[0179] Embodiments of the present invention create a method for providing encoded video information based on input video information (e.g., video input x(i)). The method includes obtaining first sample information, such as noise sample information, which takes into account, for example, a transformation of the input video information and / or prediction residual to the spectral domain, quantization and prediction of the transform coefficients. For example, the first sample information preferably also takes into account the inverse quantization and inverse transformation performed on the audio decoder side, for example, based on the values ​​of the YUV image representation of the input video information, such as pixel values, such as y1, for example using transformation, quantization, inverse quantization and inverse transformation, and optionally using prediction. For example, the first sample information can represent a reconstructed noise image, which can be obtained by a video encoder using entropy coding, inverse quantization and inverse transformation, for example before applying DBF or DBF operation, SAO or SAO operation and ALF or ALF operation.

[0180] The method according to this embodiment is based on the same considerations as the encoder described above.In addition, the embodiments disclosed herein may optionally be supplemented by any other features, functions and details disclosed herein in conjunction with the encoder, whether individually or in combination.

[0181] In one embodiment, the method includes applying a first denoising operation, such as a noise reduction operation, such as: a combination of DBF, SAO, ALF, or a combination of DBF, SAO, ALF operations, or a combination of DBF and SAO, or a combination of DBF and SAO operations, or one of the denoising operations, or a combination of DBFV and DBFH, or a combination of DBFV and DBFH operations, or for example only DBFV or DBFV operations, so as to obtain first denoised sample information, such as the output of SAO or a SAO operation, or the output of ALF or an ALF operation, or the output of a combination of DBFV and DBFH, or the output of a combination of DBFV and DBFH operations, or the output of DBFV or DBFV operations, such as y2. In this embodiment, the method further comprises determining parameters of a second denoising operation (e.g. PSAO, or PSAO operation, or SAOV, or SAOV operation, or SAOH, or SAOH operation), such as FLAG, Q(d1), Q(d2), based on a result of the first denoising operation, such as based on the first denoising sample information, such as based on the output of the SAO or ALF operation, and for example optionally also performing the second denoising operation. In this embodiment, the method further comprises adjusting or, for example, adapting the second denoising operation, depending on the difference (e.g. y2(i)-y1(i) or Y-Y1 or Y2-Y1) between samples before the first denoising operation (e.g. DBFV and DBFH or DBFV, or a combination of DBFV and DBFH or DBFV operations) and after the first denoising operation (e.g. DBFV and DBFH or DBFV, or a combination of DBFV and DBFH or DBFV operations) so as to obtain the encoded video information.

[0182] An embodiment creates a computer program for performing the above method when the computer program runs on a computer.

[0183] An embodiment creates an encoded video representation, such as a bitstream, representing video content. The encoded video representation includes: an encoded representation of transform coefficients; and a plurality of offset values ​​associated with a second denoising operation, such as dl, for example, defining offsets to be added to samples of different classes in the second denoising operation, wherein the samples are classified into a plurality of classes or a plurality of classes of samples, for example, depending on the difference between the samples before the first denoising operation and after the first denoising operation. In this embodiment, the second denoising operation is performed, for example, based on the result of the first denoising information. In this embodiment, the offset values ​​are associated with samples of a plurality of classes, depending on the difference between the samples before the first denoising operation (e.g., y1(i)) and the samples after the first denoising operation.

[0184] In an embodiment, the encoded video representation further comprises a flag indicating whether a second denoising operation should be performed.

[0185] These and other advantageous aspects are the subject matter of the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0186] Preferred embodiments of the present application are described below based on the accompanying drawings, in which:

[0187] Figure 1 shows a block diagram of a decoder (or video decoder) 100 according to an embodiment;

[0188] Figure 2 shows a block diagram of an encoder (or video encoder) 200 according to an embodiment;

[0189] Figure 3 shows a flowchart of a decoding method 300 according to an embodiment;

[0190] Figure 4 shows a flow chart of an encoding method 400 according to an embodiment;

[0191] Figure 5 shows a block diagram of a decoder 500 according to an embodiment;

[0192] Figure 6 shows a block diagram of an encoder according to an embodiment, wherein a PSAO operation is used as a second denoising operation;

[0193] Figure 7 shows a block diagram of a decoder according to an embodiment, wherein a PSAO operation is used as a second denoising operation;

[0194] Figure 8 Shows Figure 5 and 6 An example of a local neighborhood in the following embodiment;

[0195] Fig. 9 Shows Figure 5 and 6 An example of a local neighborhood in the following embodiment;

[0196] Fig.10 shows a block diagram of an encoder according to an embodiment, wherein a SAODBF operation is used as a second denoising operation;

[0197] Fig.11 A block diagram of an encoder according to an embodiment is shown, wherein a SAODBF operation is used as a second denoising operation. DETAILED DESCRIPTION

[0198] 1. According to Figure 1 Video decoder

[0199] Figure 1 A video decoder 100 according to an embodiment is shown.

[0200] The video decoder 100 receives a video input or video signal VI shown as en The decoding unit 101 of the video decoder 100 obtains the first sample information I based on the encoded video information. s1 The first sample information is a combination of a video signal and a noise signal (noise+signal information), and may include, for example, values ​​represented by a YUV image, such as pixel values. The decoding unit 101 is implemented, for example, using entropy decoding, inverse quantization and inverse transformation of a bitstream and optionally using prediction.

[0201] The first denoising unit 102 performs a first denoising operation to reduce noise, and based on the first sample information I s1 Get the first denoised sample information I ds1 . The first denoising unit 102 may, for example, include a denoising filter or a combination of denoising filters, including, for example, one of the following: a deblocking filter (DBF) operation, a sample adaptive offset (SAO) filter operation, an adaptive loop filter (ALF) operation, or a combination of a DBF operation and an SAO operation, or one of the denoising operations, or a combination of a deblocking filter for vertical edges (DBFV) and a deblocking filter for horizontal edges (DBFH), or only DBFV. The deblocking filter (DBF) may, for example, include one or two filtering operations, namely a deblocking filter for vertical edges (DBFV) or / and a deblocking filter DBFH for horizontal edges, which means that DBF=DBFV or DBFH or DBFV+DBFH, and the deblocking filter may be a conventional deblocking filter.

[0202] The second denoising unit 103 applies a second denoising operation based on the result of the first denoising operation to obtain the decoded video information VI output from the video decoder 100. de The second denoising unit 103 may for example comprise a denoising filter or a combination of denoising filters, comprising one of: a Post Sample Adaptive Offset (PSAO) filter operation, or a SAO filter operation, or a SAO filter for vertical edges (SAOV) operation.

[0203] The second denoising unit 103 is further configured to be dependent on the difference I between the samples before and after the first denoising operation. Δ The second denoising operation is adjusted to obtain decoded video information. For example, the offset may be included in a plurality of samples depending on the classification of the respective samples. For example, the samples may be classified based on the difference between the samples before the first denoising operation and after the first denoising operation. The offset is selected for each group of sample values ​​or sample positions that share the same class index. The indicated difference is determined by the calculation unit 104.

[0204] Therefore, the second denoising performed by the second denoising unit can be well adapted to the changes of one or more sample values ​​caused by the first denoising operation.

[0205] It should be noted that the video decoder 100 may optionally be supplemented by any of the features, functionalities, and details disclosed herein, both individually and in combination.

[0206] 2. According to Figure 2 Video encoder

[0207] Figure 2 FIG. 2 shows a video encoder 200 according to an embodiment. The video encoder 200 receives input video information, such as Figure 2 as shown in the Video Input or Video Signal VI.

[0208] The first encoding unit 201 is configured to obtain first sample information I based on the input video information VI s1 (It may, for example, include one or more pixel values.) The first encoding unit 201 includes, for example, a transform unit 205 that considers (or performs) a transform to a spectral domain (e.g., a transform of pixel values), a quantization unit 206 that considers (or performs) quantization of transform coefficients, and a prediction unit 207 that considers (or performs) prediction.

[0209] The first denoising unit 202 is configured to apply a first denoising operation, such as a noise reduction operation, so as to s1 Get the first denoised sample information I ds1 The second denoising unit 203 is configured to perform a second denoising operation. The video encoder 200 is, for example, configured to determine parameters of a second denoising operation (e.g., a PSAO operation), such as FLAG, Q(d1), Q(d2), based on a result of the first denoising operation, for example, based on the first denoising sample information, for example, based on the output of an SAO or ALF operation, for example, a single threshold T, and the single threshold T may be selected as a fixed positive constant, wherein the threshold T may be determined, for example, based on a rate-distortion cost at the encoder.

[0210] The second denoising unit 203 is further configured to be dependent on the difference I between the samples before and after the first denoising operation. Δ To adjust or, for example, adapt the second denoising operation to obtain the encoded video information VI en (For example, reference video information is obtained, which reflects video information that can be reconstructed at the video decoder side and which can be used to determine motion prediction parameters and to determine motion prediction residuals, which are ultimately used to derive encoded video information.) The indicated difference is determined by the calculation unit 204 .

[0211] The final entropy encoding is performed by the entropy encoding unit 208 in order to obtain the encoded video information. However, it should be noted that the entropy encoding unit may, for example, encode the quantized transform coefficients, the motion prediction information and one or more denoising parameters (as well as additional side information).

[0212] It should be noted that the video encoder 200 may optionally be supplemented by any of the features, functionalities, and details disclosed herein, both individually and in combination.

[0213] 3. According to Figure 3 Methods

[0214] Figure 3 A method 300 for providing decoded video information based on encoded video information according to an embodiment is shown.

[0215] The method comprises receiving coded video information at step 301 and obtaining first sample information based on the coded video information at step 302, for example using entropy decoding, inverse quantization and inverse transformation of a bitstream and optionally using prediction.

[0216] The method further comprises applying a first denoising operation (step 303), such as a noise reduction operation, to obtain first denoised sample information based on the first sample information.

[0217] The method 300 further comprises applying a second denoising operation based on the result of the first denoising operation in step 305 to obtain decoded information. The obtained decoded video information is output in step 306.

[0218] The method further comprises adjusting (step 304) or, for example, adapting the second denoising operation in dependence on the difference between samples before the first denoising operation and after the first denoising operation before applying the second denoising operation at step 305 in order to obtain the decoded video information.

[0219] The method 300 may optionally be supplemented by any of the features, functions and details disclosed herein, both individually and in combination.

[0220] 4. According to Figure 4 Methods

[0221] Figure 4 A method 400 for providing encoded video information based on input video information according to an embodiment is shown in FIG.

[0222] The method 400 comprises receiving input video information at step 401 and obtaining first sample information based on the input video information at step 402, for example using transformation, quantization, inverse quantization and inverse transformation, and optionally using prediction.

[0223] The method further comprises applying a first denoising operation (step 403), such as a noise reduction operation, to obtain first denoised sample information based on the first sample information.

[0224] The method 400 further comprises applying a second denoising operation based on the result of the first denoising operation at step 405. The method further comprises adjusting (step 404) or for example adapting the second denoising operation depending on the difference between samples before the first denoising operation and after the first denoising operation before applying the second denoising operation.

[0225] Operations 402-405 of method 400 substantially represent simulations of decoder operations within an encoding method. The output of the second denoising operation is considered when performing a lossless encoding operation (e.g., entropy encoding) that does not cause information loss when correspondingly decoding (e.g., entropy decoding) the encoded video information.

[0226] An entropy encoding step 406 is further performed to obtain encoding information.

[0227] In step 406, the obtained encoded video information is output.

[0228] The method 400 may optionally be supplemented by any of the features, functions and details disclosed herein, both individually and in combination.

[0229] 5. According to Figure 5 Video decoder

[0230] Figure 5 A video decoder 500 according to an embodiment is shown.

[0231] The video decoder 500 receives a video input or video signal VI shown as en The decoding unit 501 of the video decoder 500 obtains the first sample information I based on the encoded video information. s1 .

[0232] The first denoising unit 502 performs a first denoising operation to reduce noise, and based on the first sample information I s1 Get the first denoised sample information I ds1 The first denoising unit 502 includes a DBF filter 505 , a SAO filter 506 and an ALF filter 507 .

[0233] The second denoising unit 503 applies a second denoising operation based on the result of the first denoising operation so as to obtain decoded video information output from the video decoder 500. The second denoising unit 504 comprises a classifier 508 configured to classify the decoded video information based on the difference between the samples before the first denoising operation and the samples after the first denoising operation. ΔClassify the sample into samples of multiple classes; an offset determination unit 509, which is configured to determine an offset based on the classification performed by the classifier 508; and a modifier 510, which is configured to modify the sample depending on the classification performed by the classifier 508, i.e., by adding an offset determined by the offset determination unit 509. For example, the classification is performed by the classifier only when one or more predetermined thresholds are reached. The indicated difference is determined by the calculation unit 504.

[0234] The second denoising unit 503 is further configured to adjust the second denoising operation depending on the difference between the samples before the first denoising operation and after the first denoising operation so as to obtain the decoded video information.

[0235] The video decoder 500 may optionally be supplemented by any of the features, functions and details disclosed herein, both individually and in combination.

[0236] 6. Other Embodiments and Aspects (eg, using Post Sample Adaptive Offset (PSAO))

[0237] In the following, further aspects and embodiments according to the present invention will be described, which may be used alone or in combination with any other embodiments disclosed herein.

[0238] In addition, the embodiments disclosed in this section can optionally be supplemented by any of the other features, functions, and details disclosed herein, both individually and in combination.

[0239] For example, a post-sample adaptive offset (PSAO) operation that may be used as a second denoising operation (eg, in other embodiments disclosed herein) and its implementation in a decoder and an encoder according to embodiments are further described.

[0240] 6.1. Introduction

[0241] As described in the above background technology section, in video coding and decoding, loop filtering has emerged as a key module due to its significant improvement in compression performance.

[0242] The first loop filter adopted in video coding standards was the deblocking filter (DBF), which is mainly used to reduce blocking artifacts [1]. HEVC further adopts another nonlinear loop filter, sample adaptive offset (SAO) [2], to further reduce coding artifacts after DBF. Finally, adaptive loop filtering (ALF) [3] (the final filtering process in the loop filtering) has been adopted in the preliminary draft of VVC [4]. In particular, sample adaptive offset (SAO) is a process that modifies the decoded samples by conditionally adding an offset value to each sample after applying the deblocking filter (DBF). In order to conditionally add the offset value to each sample, a class C is applied to each sample position, and an offset value is selected for each group of sample positions that share the same class index associated with C.

[0243] According to this embodiment, in order to obtain additional coding efficiency, an additional SAO filtering process, namely Post-Sample Adaptive Offset (PSAO) with a new classification, is proposed.

[0244] For example, the PSAO (or PSAO operations) disclosed herein may optionally be used in combination with any of the concepts and techniques described herein, such as loop filtering including one or more of DBF, SAO, and ALF (or DBF, SAO, and ALF operations).

[0245] 6.2. Sample Adaptive Offset

[0246] The basic idea of ​​SAO is to use a class index l=1,...,L for each sample position i∈I to offset the value d l To this end, for example, the classification C is applied to all sample positions i∈I. This nonlinear filtering process can then be written as

[0247]

[0248] where XΛ l is an indicator function, for example defined by:

[0249]

[0250] as well as

[0251] Λ l ={i∈I:C(i)=l}. (2)

[0252] Offset value d l For example, is chosen so that l The associated mean square error

[0253]

[0254] is minimized. This is given, for example, by:

[0255]

[0256] where x is the original image. It can be seen that for L = 2, the best classification of (1)-(2) For example, it is given as

[0257]

[0258] And under some mild conditions

[0259]

[0260] The sample adaptive offset technique described herein may be optionally introduced into any other embodiment described herein. For example, the sample adaptive offset technique described herein may be used, for example, in the first denoising unit 102 and / or the second denoising unit 103 of the decoder 100 and / or in the first denoising unit 202 and / or the second denoising unit 203 of the encoder 200 and / or in the first denoising operation 303 and / or in the second denoising operation 305 of the decoding method 300 and / or in the first denoising operation 403 and / or in the second denoising operation 405 of the encoding method 400 and / or in the first denoising unit 502 of the decoder 500, as, for example, SAO filtering 506, and / or in the second denoising unit 503.

[0261] 6.3. Classification for PSAO

[0262] An example of classification for PSAO will be described below, where it should be noted that any details are optional.

[0263] PSAO (or PSAO operation) (which may, for example, function as a second denoising operation or which may, for example, be part of a second denoising operation) is, for example, as given in (1) above, and, for example, L=2, applied after three loop filtering processes, for example, DBF, SAO and ALF, according to the present specification. PSAO or PSAO operation, for example, the second denoising operation, is, for example, essentially a secondary SAO filtering process. In this case, the input image y in (1) is, for example, a reconstructed image after applying ALF or an ALF operation (for example, as a first denoising operation or as the first denoising operation disclosed herein). It can be seen from (4) that the ideal classification can be given, for example, as the sign of xi)-y(i). Because the original image x is not available at the decoder, y2-y1 is used instead of xy, for example. For example, by taking the sign of y2-y1 when y2-y1 is, for example, large enough to approximate the ideal classification in (4), two classes Λ1 and Λ2 are then constructed. Here, the two images y1 and y2 are, respectively, a reconstructed image before loop filtering (e.g., before applying loop filtering, such as a first denoising operation) and a reconstructed image (e.g., a reconstructed image or a filtered reconstructed image) after SAO (e.g., after applying an SAO operation, such as a second denoising operation).

[0264] This is for example in Figure 6 As shown in FIG. 1 , an encoder according to the present embodiment is shown. However, it should be noted that Figure 6 The specific construction and details shown in the drawings should be regarded as merely exemplary and are not essential to the concepts disclosed herein. Figure 6 An encoder block diagram is shown, where the unfiltered and filtered (denoised) images are defined as y1 and y2.

[0265] This gives, for example,

[0266]

[0267] Among them, for example,

[0268] Λ1={i∈I: (y2(i)-y1(i))>T} and Λ2={i∈I: (y2(i)-y1(i))<-T} (6)

[0269] And, for example,

[0270] Λ0=I∩(Λ1∪Λ2) c

[0271] where T is, for example, some fixed positive constant (which may optionally be determined by the encoder, and which may, for example, be signaled from the encoder to the decoder in the encoded video representation). Here, Q(d l ) is, for example, the quantized value d for l=1,2 l(It may optionally be determined by the encoder, and it may be signaled from the encoder to the decoder, for example, in the encoded video representation.) For the offset value d l , we apply, for example, (3) to l=1,2, and for example simply adopt d0=0 (for example for determination in an encoder, where, for example, y2 or y2(i) can replace y in equation (3)).

[0272] It should be noted that the "classification for PSAO" described here can be applied, for example, to Figure 6 The encoder 600 or according to Figure 7 , or in any other encoder and decoder disclosed herein (for example, in the second denoising unit 103 of the decoder 100 and / or in the second denoising unit 203 of the encoder 200 and / or in block 304 of the decoding method 300 (adjusting the second denoising operation) and / or in block 404 of the encoding method 400 (adjusting the second denoising operation and the classifier 508 of the second denoising unit 503 of the decoder 500).

[0273] 6.4. Decoder

[0274] In the following, an example of a decoder or (part of) the functionality of a decoder will be described, wherein it should be noted that the decoder may (for example) use the “classification for PSAO” discussed above. It should be noted that any details are optional.

[0275] After constructing Λ1 and Λ2, (5) (or operation (5)) is applied to the input image y (e.g., ALF or a decoded image after the ALF operation), for example, by classification as defined in (6). Unlike the SAO (or SAO operation) of (1) performed for each CTB, (5) (or operation (5)) is applied once for each frame, for example, which means that, for example, only two quantization offset values ​​Q(d1) and Q(d2) are required for each frame. It should be noted that, for example, there are three required parameters for each frame, namely, Q(d1), Q(d2) and FLAG parsed from the bitstream. For example, FLAG is 0 or 1, which determines whether PSAO (or PSAO operation) is applied or skipped for y. It should be noted that, for example, if FLAG=1, PSAO (or PSAO operation) with Q(d1) and Q(d2) is applied, while PSAO is skipped when FLAG=0. This entire process is applied to each YUV component independently, for example.

[0276] exist Figure 7 An example of a decoder that can, for example, use the functionality described herein is shown in . Figure 7 A decoder block diagram is shown, where the unfiltered and filtered (denoised) images are defined as y1 and y2.

[0277] 6.5. Encoder

[0278] In the following, examples of the functionality of an encoder or (part of) an encoder will be described, wherein it should be noted that the encoder may (for example) use the “classification for PSAO” discussed above. It should be noted that any details are optional.

[0279] After applying the classification in (6), for example, by (3), the offset values ​​d1 and d2 in (5) are obtained. Then, for example, using the quantized offset values ​​Q(d1) and Q(d2), the corresponding RD cost is calculated, and 0 or 1 is assigned to FLAG. For example, when FLAG=1, the quantized offset values ​​Q(d1) and Q(d2) are sent, otherwise only FLAG=0 is sent, in which case, for example, PSAO (or PSAO operation) is skipped at the decoder. This entire process is applied to each YUV component independently, for example.

[0280] 6.6. Notes

[0281] In the following, some remarks and details are provided, which may optionally be considered or used in any embodiment disclosed herein.

[0282] According to one aspect, the main novelty (or at least one main novelty) of the method (which can be applied, for example, in the encoder disclosed herein and the decoder disclosed herein) is the new classification, for example given in (6). Furthermore, in case SAO (or SAO operation) is applied according to an embodiment, for example (1) is applied for each CTB (for example, for each coding tree block or coding tree unit), while PSAO, for example, performs (5) for each frame (for example, for each YUV component). For PSAO, for example, for each YUV component at the encoder, only three parameters Q(d1), Q(d2) and FLAG are estimated. Finally, classification (6) can be summarized, for example, as follows:

[0283] Λ1={i∈I:f(i)>T} and Λ2={i∈I:f(i)<-T}

[0284] in

[0285]

[0286] and N(i) is the set of neighboring sample positions, e.g., for i=(i1, i2)∈I and w(ij), e.g., is some fixed positive weight. For example, one can take

[0287] N((i1,i2))={(i1,i2),(i1-1,i2),(i1+1,i2),(i1,i2-1),(i1,i2+1)}

[0288] And w(0,0)=4 and w(0,1)=w(1,0)=w(0,-1)=w(-1,0)=1. In addition, for example, we can simply take N(i1,i2)={(i1,i2)} and w(0,0)=1, and give (6).

[0289] Figure 8 Some examples of neighboring sample position sets N(i) according to some embodiments are shown in , for example, for a 4-neighborhood and for an 8-neighborhood.

[0290] Fig. 9 Other examples of local neighborhoods N((i1, i2)) according to other embodiments are shown, in particular containing 5 samples (left) and 9 samples (right).

[0291] 7. Other embodiments and aspects (eg, using sample adaptive offset for deblocking filtering (SAODBF))

[0292] In the following, further aspects and embodiments according to the present invention will be described, which may be used alone or in combination with any other embodiments disclosed herein.

[0293] In addition, the embodiments disclosed in this section can optionally be supplemented by any of the other features, functions, and details disclosed herein, both individually and in combination.

[0294] A Sample Adaptive Offset for Deblocking Filtering (SAODBF) operation, which may be used, for example, as a second denoising operation, and its implementation in a decoder and an encoder according to another embodiment are further described.

[0295] 7.1 Introduction

[0296] As mentioned in the background technology section above, in video coding, loop filtering has emerged as a key module due to its significant improvement in compression performance. The first loop filter adopted in video coding standards is deblocking filtering (DBF), which is mainly used to reduce blocking artifacts [1]. HEVC further adopts another nonlinear loop filter, such as sample adaptive offset (SAO), to further reduce coding artifacts after DBF [2].

[0297] As described above, sample adaptive offset (SAO) is a process of modifying decoded samples by conditionally adding an offset value to each sample after applying a deblocking filter (DBF). In order to conditionally add an offset value to each sample, a classification C is applied to each sample position, and an offset value is selected for each group of sample positions that share the same classification index associated with C. In this embodiment, additional SAO filtering is proposed, and SAODBF (or SAODBF operation) is applied in conjunction with a deblocking filter (DBF) with a new classification to obtain additional coding efficiency.

[0298] 7.2 Sample Adaptive Offset

[0299] The basic idea of ​​SAO is to add an offset value d to each reconstructed sample Y(i) depending on the class index l=1,...,L of multiple sample positions or each sample position i∈I. l To this end, for example, the classification C is applied to all sample positions i∈I. This nonlinear filtering process can then be written as

[0300]

[0301] Where Q(d l ) is the offset coefficient d l The quantized value of XΛ l is an indicator function defined, for example, by:

[0302]

[0303] and

[0304] Λ l ={i∈I:C(i)=l} (2′)

[0305] Offset value d l For example, is chosen so that l The associated mean square error

[0306]

[0307] is minimized. This is given, for example, by:

[0308]

[0309] where X is the original image and #(Λ l ) is the number of elements in Λl.

[0310] The sample adaptive offset technique described herein may optionally be introduced into any other embodiment described herein.

[0311] It should be noted that the "classification for PSAO" described here can be applied, for example, to Figure 6 The encoder 600 or according to Figure 7 , or in any other encoder and decoder disclosed herein (for example, in the second denoising unit 103 of the decoder 100 and / or in the second denoising unit 203 of the encoder 200 and / or in block 304 of the decoding method 300 (adjusting the second denoising operation) and / or in block 404 of the encoding method 400 (adjusting the second denoising operation and the classifier 508 of the second denoising unit 503 of the decoder 500)).

[0312] 7.3. Sample Adaptive Offset for Deblocking Filtering (SAODBF)

[0313] In this embodiment (or in general, in Section 7), two methods of Sample Adaptive Offset for Deblocking Filtering (SAODBF) are proposed (which may optionally be used in any of the embodiments disclosed herein).

[0314] These two operations are applied, for example, before applying SAO (or SAO operation), for example, immediately after DBF (or DBF operation), and they are additional SAO filters. Here and hereinafter, these two operations are defined as SAODBF1 and SAODBF2. These two operations follow the same process as provided in (1)′-(3)′ in conventional SAO (or conventional SAO operation), with a new classification to construct the class Λ in (2)′ or (2). l Finally, it should be noted that DBF (or DBF operation) consists of, for example, two filtering operations, e.g., one for vertical edges and another for horizontal edges, which are defined as DBFV (or DBFV operation) and DBFH (or DBFH operation), respectively, as Fig.10 As shown, an encoder according to this embodiment is shown. Fig.10 An encoder block diagram is shown, where the unfiltered and filtered images are defined as Y1 and Y.

[0315] 7.3.1 SAODBF1

[0316] In this section, the classification in SAODBF1 (or SAODBF1 operation) is described. For this purpose, T0, ..., T L For example, some positive constants, where T0<…<T L And T L =∞. Y1 and Y are, for example, the noisy image before applying DBF and the denoised image after DBF, respectively, for example, Fig.10 Then define the class of SAODBF1 as

[0317] Λ l ={i∈I:T l-1 ≤(Y1(i)-Y(i))<T l} and l+L ={i∈I:T l-1 ≤(Y(i)-Y1(i))<T l} (4′)

[0318] l = 1, ..., L, and

[0319] Λ0=I∩(Λ1∪…∪Λ 2L ) c

[0320] Then SAODBF1 (or SAODBF1 operation) is used as the class Λ in (1)′ and (4)′ l together, and this can be written as

[0321]

[0322] Where Q(d l ) is the offset coefficient d which is obtained, for example, as (3)′ l quantized value, and d0 is set to d0=0.

[0323] 7.3.1.1 Decoder

[0324] In the following, an example of a decoder or (part of) the functionality of a decoder will be described, wherein it should be noted that the decoder may (for example) use the “classification for PSAO” discussed above. It should be noted that any details are optional.

[0325] In constructing Λ1...Λ 2L After that, (5)′ is applied to the input image Y2 (decoded image after DBF (or DBF operation)), for example, through the classification in (4)′. Unlike the SAO (or SAO operation) of performing (1)′ for each CTU, (5)′ is applied once for each frame, for example, which means that, for example, only 2L quantization offset values ​​Q(d1)...Q(d 2L ), d0=0. Note that, for example, there are 2L+1 required parameters for each frame, namely the flags parsed from the bitstream and Q(d1)...Q(d 2L ). For example, flag is 0 or 1, which determines whether SAODBF1 (or SAODBF1 operation) is applied or skipped for Y2. Note that, for example, if flag=1, SAODBF1 (or SAODBF1 operation) is applied, while SAODBF1 operation is skipped when flag=0. This entire process can be applied to each YUV component independently, for example.

[0326] 7.3.1.2 Encoder

[0327] In the following, examples of the functionality of an encoder or (part of) an encoder will be described, wherein it should be noted that the encoder may (for example) use the “classification for PSAO” discussed above. It should be noted that any details are optional.

[0328] After applying the classification in (4)′, the offset values ​​d1 . . . d in (5)′ are obtained, for example, by (3)′. 2L Then, using, for example, quantized offset values ​​Q(d1)...Q(d 2L) to calculate the corresponding RD cost in order to assign 0 or 1 to the flag. When flag = 1, the quantized offset values ​​Q(d1)…Q(d 2L ) is sent, otherwise only flag=0 is sent, in which case SAODBF1 (or SAODBF1 operation) is skipped, for example, at the decoder. This whole process is applied, for example, independently to each YUV component.

[0329] 7.3.2 SAODRF2

[0330] In this method, for example, an additional SAO (or an additional SAO operation) is applied twice in the same process as (4)′-(5)′. The two additional SAOs (or SAO operations) are defined as SAOV and SAOH (or SAOV and SAOH operations). For example, SAOV (or SAOV operation) is applied after DBFV (or DBFV operation), and Y2 (the output of DBFV (or DBFV operation)) is used as, for example, an input image. SAOV (or SAOV operation) uses, for example, the difference between Y1 (the image before DBF (or DBF operation)) and Y2 for (4)′. On the other hand, SAOH (or SAOH operation) is applied after, for example, DBFH (or DBFH operation), and it uses, for example, Y3 (the output of DBFH (or DBFH operation)) as an input image. SAOH (or SAOH operation) uses, for example The difference between (the output of SAOV (or SAOV operation)) and Y3 is used for (4)'. This is for example Fig.11 Shown in.

[0331] 7.4 Some remarks

[0332] Similar to conventional SAO, the proposed additional SAO filters SAOBDF1 and SAOBDF2 may be applied, for example, to each CTU instead of, for example, to the entire image frame. In this case, the offset value d should be sent for each CTU. l And flag parameters. In addition, threshold parameters T0, ...T L-1 may be determined (e.g. selected by experiment) or may be determined at the encoder, such as the encoder selecting, for example, the best threshold parameter set T0, ... T in, for example, some pre-selected candidate set based on, for example, the RD cost. L-1 , and the selected option should then be sent, for example, for each image frame (or, for example, for each CTU).

[0333] In one embodiment, an additional SAO, such as PSAO, is applied, for example, after ALF, while in another embodiment, an additional SAO, such as PSAO, is applied, for example, after DBF (or, for example, after DBFV or DBFH).

[0334] In one embodiment, for example, there is only one threshold parameter T and for example three classes Λ0, Λ1, Λ2, while according to another embodiment, for example, there are L threshold parameters T0, ...T L-1 and for example 2L+1 classes Λ0, ... Λ 2L+1 , or 2L+1 classes Λ0, ...Λ 2L .

[0335] For the alternative method SAODBF2, the indicated additional SAO is applied eg twice (the first SAO eg after DBFV, the second SAO eg after DBFH), while in another embodiment it is applied eg once after ALF.

[0336] References

[0337] [1] P.List, A.Joch, J.Lainema, G.Bjontegaard, and M.Karczewicz, Adaptivedeblocking filter, IEEE Trans.Circuits Syst.Video Technol.13(2003), No.7.

[0338] [2] C.-M.Fu etal., Sample adaptive offsetTin the HEVC standard, IEEETrans.Circuits Syst.Video Technol.22(2012)No.12.

[0339] [3]Karczewicz, M.; Zhang, L.; Chien, W.; Li, X.: Geometry transformation-based adaptive in-loop filter, in Picture Coding Symposium (PCS), Nuremberg, Germany, 2016.

[0340] [4]Bross, B.; Chen, J.; Liu, S.: Versatile Video Coding (DrafT4), JVET-M1001, Marrakech, Marocco, January 2019.

[0341] 8.According to Figure 6 Example

[0342] Figure 66 is a block diagram of an encoder according to an embodiment of the present invention. The video encoder 600 is configured to receive a video input 610 and provide a bitstream 612 based thereon, the bitstream representing the video input in an encoded form.

[0343] The video encoder 600 includes a transform and quantization 620 that transforms and quantizes, for example, sample values ​​of a video input 610, or differences 622 between sample values ​​of the video input 610 and predicted sample values. For example, in some cases, prediction may not be used (e.g., when independent frames are encoded), while in other cases, such prediction may be applied. For example, the transform and quantization 620 may provide quantized transform coefficients 624 that represent pixel values ​​of the video input 610 or pixel values ​​of the difference between the video input and the prediction result 622. Entropy coding 630 may, for example, encode the quantized transform coefficients 624 (and additional control information) to obtain a bitstream 612.

[0344] The video encoder 600 also includes a reconstruction path 640, where reconstructed image information 642 is derived from the quantized transform coefficients 624. For example, the reconstruction path 640 "emulates" decoder-side reconstruction so that the video encoder 600 can obtain knowledge of what image information the video decoder will have available for, for example, motion compensation, which can be performed, for example, in the context of inter-frame prediction (e.g., using both prediction based on previous frames and bidirectional prediction using both previous and subsequent frames). In addition, the video encoder also includes a prediction mechanism 650, which can, for example, be capable of performing both intra-frame prediction (e.g., prediction within a single frame) and inter-frame prediction.

[0345] In the following, some additional details about the reconstruction path 640 will be described. For example, the reconstruction path 640 includes an inverse quantization and inverse transformation 644, which receives the quantized transform coefficients 624 and provides, based thereon, reconstructed image information 644a in the form of, for example, spectral values. Optionally, the predicted image information (e.g., predicted spectral values) 644b provided by the prediction path 650 is added to, for example, the reconstructed image information 644a by a prediction addition 645, thereby obtaining partially reconstructed image information 644c, which can also be denoted here by y1.

[0346] The partially reconstructed image information 644c is then input to a deblocking filter (DBF) 646, which provides deblocked filtered image information 646a. The reconstruction path 640 also includes a sample adaptive offset (SAO) process 648, which receives the deblocked filtered image information 646a (e.g., a block or frame of spectral values) and performs a sample adaptive offset (SAO) process. Thus, sample adaptive offset processed image information 648a is provided, which is also denoted by y2. In addition, the reconstruction path 640 also includes an adaptive loop filter (ALF) 649, which receives the same adaptive offset processed image information 648a and provides adaptive loop filter processed image information 649a based thereon, which can (for example) form reconstructed image information 642 (e.g., reconstructed pixel values) and can (for example) be denoted by y.

[0347] The prediction path 650 includes an intra prediction 652, which may, for example, receive the video input 610 and the partially reconstructed image information 644c, and which may, for example, provide intra prediction information 652a. In addition, the prediction path 650 may include a picture buffer 654, which may, for example, receive the reconstructed image information 642 (e.g., information denoted by y) and which may, for example, buffer the reconstructed image information 642 for later use. The buffered image information is denoted by 654a. The video encoder 600 also includes motion compensation 656, which may, for example, receive the buffered image information 654a (e.g., a block or frame of pixel values) and may determine motion compensation information 656a. For example, the motion compensation information 656a may include motion vector information, wherein the motion vector information may describe the movement of the image portion between subsequent frames.

[0348] Furthermore, the video encoder 600 may include an intra / inter mode selection 658, which may, for example, decide between intra prediction and inter prediction. The intra / inter mode selection 658 may, for example, provide appropriate prediction information 622 (e.g., prediction image information 622), which may, for example, be subtracted from the video input 610. Additionally, the intra / inter mode selection 658 may also provide prediction image information 644b, which may, for example, be added to the partially reconstructed image information 644a to thereby obtain reconstructed image information 644c in which prediction is taken into account.

[0349] Finally, the video encoder 600 may entropy encode the quantized transform coefficients associated with the video content, where the transform coefficients may be based on the original video input 610 or on a predicted processed version of the video input from which the prediction has been subtracted to thereby reduce the bit rate. The reconstruction path 640 may mimic the processing on the decoder side, e.g., having the same prediction basis on the encoder side and the decoder side (which helps keep the encoder and decoder synchronized).

[0350] Within the reconstruction path 640 , there are different filtering operations such as deblocking filtering (DBF) 646 , sample adaptive offset (SAO) processing 648 , and adaptive loop filtering (ALF) 649 .

[0351] It should be noted that the deblocking filter (DBF) operation 646, the sample adaptive offset (SAO) processing operation 648, and the adaptive loop filter (ALF) operation 649 can all be considered as denoising operations. However, it should be noted that the deblocking filter (DBF) 646 and the sample adaptive offset (SAO) 648 can be considered, for example, as a "first denoising operation", and the adaptive loop filter (ALF) 649 can be considered, for example, as a "second denoising operation". In addition, it should be noted that the adaptive loop filter (ALF) 649 can be adaptive (for example) depending on the difference between the output of the first denoising operation (e.g., y2) and the input of the first denoising operation (e.g., y1).

[0352] In a preferred embodiment, a post-sample adaptive offset (PSAO) such as described in Section 6 is added after the adaptive loop filter (ALF) 649. For example, the processed image information 649a provided by the adaptive loop filter (ALF) 649 and, for example, denoted by y is input to the post-sample adaptive offset (PSAO) processing (PSAO) 660, which should be considered optional. In other words, the image information provided by, for example, the adaptive loop filter (ALF) 649, denoted by 649a or y, is input to the post-sample adaptive offset (PSAO) processing 660, and the output result of the post-sample adaptive offset (PSAO) processing 660 may constitute the reconstructed image information 642 input to the prediction path 650. Therefore, the post-sample adaptive offset (PSAO) 660 may be adapted, for example, according to the difference between the sample adaptive offset (PSAO) processed image information 648a (also denoted by y2) and the image information 644c input to the deblocking filter (DBF) 646. However, the difference y2-y1 can be considered, for example, as the difference between samples before and after a first denoising operation (which can, for example, include a deblocking filter (DBF) 646 and a sample adaptive offset (SAO) process 648). For example, the difference y2-y1 (e.g., the difference between corresponding pixel values ​​of image information 648a and image information 644c) can be used for classification, which can be performed, for example, according to equation (4), where x is replaced by y1 and where y is replaced by y2. In addition, it should be noted that the optional post-sample adaptive offset (PSAO) process 660 can, for example, perform processing as provided in equation (5), for example, selectively replacing the offset value Q (d l ) is added to the output 649a of the adaptive loop filter (ALF) 649. In this regard, the pixel value of the image information 649a provided by the adaptive loop filter (ALF) 649 may be substituted for y in equation (5), and the output information of the post sample adaptive offset (PSAO) process 660 (e.g., the reconstructed image information 642) may be represented by the pixel value shown in equation (5). express.

[0353] Finally, the (optional) post-sample adaptive offset (PSAO) process 660 may provide improved reconstructed image information 642, and the pixel classification performed in the context of the post-sample adaptive offset (PSAO) process 660 is based on the difference between the pixel values ​​y2 and y1. Therefore, the strength of the modification of one or more pixel values ​​provided by the first denoising operation (e.g., by the deblocking filter (DBF) operation 646 and the sample adaptive offset (SAO) process 648) may have an impact on (or even determine) the classification, and may therefore affect the offset introduced by the post-sample adaptive offset (PSAO) process 660. Therefore, the offset introduced by the post-sample adaptive (PSAO) process 660 may, for example, have a correlation with the difference between y2 and y1.

[0354] However, it should be noted that according to Figure 6 The video encoder 600 may optionally be supplemented by any of the features, functionalities, and details described herein, both individually and in combination. In addition, it should be noted that any explanation provided with respect to the video encoder 600 may also optionally be applicable to any other embodiment.

[0355] 9. According to Figure 7 Example

[0356] Figure 7 FIG. 2 is a block diagram of a video decoder according to an embodiment of the present invention. Figure 7 The video decoder of is denoted as a whole by 700. The video decoder 700 is configured to receive a video stream 710, which may, for example, correspond to a bitstream 612 provided by the video encoder 600. In addition, the video decoder 700 is configured to provide a video output 712 (e.g., a representation of pixel values). The video decoder 700 includes an entropy decoder 720, which receives the bitstream 710 and may, for example, decode transform coefficients and optional additional parameters (e.g., quantization values ​​d ) that have been entropy encoded, for example, by the entropy encoder 630. l). Thus, entropy decoding 720 may correspond, for example, to entropy encoding 630. Thus, entropy decoding 720 provides decoded (quantized) transform coefficients 722. Furthermore, entropy decoding 720 may provide decoded filter information 724, which may, for example, include parameters for deblocking filtering (DBF), for sample adaptive offset (SAO) processing, and / or for post sample adaptive offset (PSAO) processing. The parameters (or filter information) 724 provided by entropy decoding 720 may, for example, correspond to parameters used by deblocking filtering (DBF) 646 and / or sample adaptive offset (SAO) processing 648 and / or adaptive loop filtering (ALF) 649 and / or post sample adaptive offset (PSAO) processing 660 in video encoder 600 (wherein it should be noted that these parameters may, for example, be entropy encoded by entropy encoding 630). Furthermore, video decoder 700 further includes inverse quantization and inverse transform 730, which may, for example, at least partially reverse the transform and quantization performed by block 620 of video encoder 600. For example, inverse quantization and inverse transformation 730 may correspond to inverse quantization and inverse transformation 644 of video encoder 600 .

[0357] The video decoder 700 further comprises a plurality of denoising operations or denoising blocks, which together form a denoising chain 740, which may, for example, provide a reconstructed video representation 712. Furthermore, the video decoder 700 further comprises a prediction 750, which may, for example, correspond to the prediction 650 of the video encoder 600.

[0358] For example, the processing chain 740 may receive the partially reconstructed image information 732 provided by the inverse quantization and inverse transformation 730, or the partially reconstructed image information 736 processed by prediction, and the prediction result 738 provided by the prediction process 750 is added to the partially reconstructed image information 732 provided by the inverse quantization and inverse transformation 730. The input information input into the denoising process 740 is represented as y1, for example, and may be equal to the partially reconstructed image information 732 in the absence of prediction, or may be equal to the partially reconstructed image information 736 processed by prediction in the presence of prediction.

[0359] The processing path 740 (which can be considered a denoising processing path or a post-processing path) can, for example, include a deblocking filter (DBF) 742, which provides deblocking filtered image information 742a based on the input information of the processing path 740 (e.g., image information y1), and the processing path can also include a sample adaptive offset (SAO) process 744, which provides sample adaptive offset processed image information 744a based on the deblocking filter processed image information 742a. ​​For example, the sample adaptive offset processed video information 744a can be represented by y2.

[0360] For example, the deblocking filter (DBF) 742 and the sample adaptive offset (SAO) process 744 may be considered (together) as a first denoising operation.

[0361] In addition, the processing path 740 also includes an adaptive loop filter (ALF) 746, which can, for example, receive the SAO processed image information 744a and provide adaptive loop filtered image information based thereon. The adaptive loop filtered image information can, for example, constitute the video output 712.

[0362] However, optionally, the processing path 740 further includes a post sample adaptive offset (PSAO) process 760, which receives image information provided by the adaptive loop filter (ALF) 746 (e.g., adaptive loop filtered image information) and provides reconstructed image information 762 based thereon, which reconstructed image information can (for example) replace the video output information 712. In other words, in the absence of the post sample adaptive offset (PSAO) process 760, the video output information 712 can be the output of the adaptive loop filter (ALF) 746, but in the presence of the (optional) post sample adaptive offset (PSAO) process 760, the video output information 712 can actually be provided by the post sample adaptive offset (PSAO) process 760. In other words, in the absence of the post-sample adaptive offset (PSAO) processing 760, the reconstructed image information 762 may be equal to the output of the adaptive loop filter (ALF) 746, but in the presence of the post-sample adaptive offset (PSAO) processing 760, the reconstructed video information 762 may be equal to the output of the post-sample adaptive offset (PSAO) processing 760 (or may be based on the output of the post-sample adaptive offset (PSAO) processing 760).

[0363] Furthermore, it should be noted that the deblocking filter (DBF) 742 may correspond to the deblocking filter (DBF) 646, the sample adaptive offset (SAO) processing 742 may correspond to the sample adaptive offset (SAO) processing 648, the adaptive loop filter (ALF) 746 may correspond to the adaptive loop filter (ALF) 649, and the post sample adaptive offset (PSAO) processing 760 may correspond to the post sample adaptive offset (PSAO) processing 660. In other words, the processing path 740 may include the same functions as those provided by the corresponding components of the video encoder 600. Therefore, the above explanations presented with respect to the video encoder 600 as described in Section 6 may also be applied in a corresponding manner.

[0364] In addition, it should be noted that the video decoder 700 also includes a prediction 750 corresponding to the prediction 650. For example, the prediction 750 includes a picture buffer 754, which receives the reconstructed picture information 762 (provided by the adaptive loop filter (ALF) 746 or the post-sample adaptive offset (PSAO) 760) and provides the buffered picture information 754a. In addition, the prediction 750 includes a motion compensation 756, which receives the buffered picture information 754a and provides the motion compensated picture information 756a (e.g., using additional information about the motion vector, which is included in the bitstream 710 in an encoded form). In addition, the prediction 750 includes an intra-frame prediction 752, which generally receives the partially decoded picture information 736 (also denoted by y1) and performs an intra-frame prediction (e.g., prediction within a block or frame, generally using the prediction information included in the bitstream 710 in an encoded form). For example, the intra-frame prediction 752 performs prediction based on picture information from the same frame, while the motion compensation 756 performs prediction based on picture information from one or more previous frames or even from one or more subsequent frames.

[0365] In addition, the prediction 750 includes an intra / inter mode selection 758, which decides where intra prediction or inter prediction or a combination of the two should be used, for example, based on a signaling input in coded form in the bitstream 710. The intra / inter mode selection 758 provides a prediction result 738, which is then added to the partially reconstructed image information 732 provided by the inverse quantization and inverse transform 730.

[0366] Finally, the video decoder 700 includes a processing path or denoising path or post-processing path 740 that performs multiple denoising operations. For example, the deblocking filter (DBF) 742 and the sample adaptive offset (SAO) operation 744 can be considered as a first denoising operation, and the adaptive loop filter (ALF) 746 and the post sample adaptive offset (PSAO) operation 760 can be considered as a second denoising operation. For example, the post sample adaptive offset (PSAO) operation 760 can be adapted depending on the difference between samples (or sample values) before and after the first denoising operation. For example, as described in Section 6.3, the classification used in the post sample adaptive offset (PSAO) operation 760 is based on the difference between image information y2 (e.g., its sample value) and information y1 (e.g., its sample value). In other words, the post sample adaptive offset (PSAO) 760 can use the classification as described in Section 6.3, where the classification can be performed, for example, according to equation (6). In addition, the actual function of the post-sample adaptive offset (PSAO) process 760 may correspond to the function described by equation (1) in Section 6.2, wherein the pixel value provided by the adaptive loop filter (ALF) 746 may replace the sample value y mentioned in equation (1), and wherein the result value mentioned in equation (1) is It can function to reconstruct image information or reconstruct video output 712, 762.

[0367] Furthermore, it should be noted that the functionality of blocks 742, 744, 746, 760 may correspond to the functionality of blocks 646, 648, 649, 660, wherein the parameters used by PSAO 760 may (for example) be determined in the video encoder 600 (for example) using equation (3), encoded in the bitstream by entropy encoding 630 and decoded by entropy encoding 720 and provided to the decoder-side PSAO 760 by entropy decoding 720.

[0368] Furthermore, it should be noted that the video decoder 700 may optionally be supplemented by any of the features, functions and details disclosed herein, either alone or in combination.

[0369] Figure 8 A graphical representation of some samples of N(i) is shown. For example, a four-neighborhood (e.g., a four-neighborhood of sample values ​​at sample location (x,y)) is shown at reference numeral 810, and an eight-neighborhood of the (center) sample value at sample location (x,y) is shown at reference numeral 820. For example, f(i) may be calculated using sample values ​​at the five sample locations represented by reference numeral 810, where the same or different weightings may be applied to different sample locations.

[0370] Similarly, sample values ​​at the nine positions shown at reference numeral 820 may optionally be used for evaluation of function f(i), wherein the same or different weightings may be applied to sample values ​​at different sample positions shown at reference numeral 820 .

[0371] also, Fig. 9 A schematic representation of an example of a local neighborhood N((i1, i2)) containing five samples (as shown by reference numeral 910) and nine samples (as shown by reference numeral 920) is shown. For example, local neighborhood 910 can be similar to local neighborhood 810, and local neighborhood 920 can be similar to local neighborhood 820, where sample values ​​at the center position can also be considered.

[0372] 10. According to Fig.10 Example

[0373] Fig.10 A block diagram of an encoder according to another embodiment of the present invention is shown.

[0374] Fig.10 FIG. 1 is a block diagram of an encoder according to an embodiment of the present invention. Fig.10 In the encoder block diagram of , the unfiltered and filtered images are represented by Y1 and Y.

[0375] It should be noted that according to Fig.10The encoder 1000 may be similar to Figure 6 600, so that the same components will not be explained here again. More specifically, it should be noted that transform and quantization 1020 corresponds to transform and quantization 620, entropy coding 1030 corresponds to entropy coding 630, inverse quantization and inverse transform 1044 corresponds to inverse quantization and inverse transform 644, intra prediction 1052 corresponds to intra prediction 652, picture cache 1054 corresponds to picture cache 654, motion compensation 1056 corresponds to motion compensation 656, and intra / inter mode selection 1058 corresponds to intra / inter mode selection 658. However, it should be noted that the reconstruction path 1040 is different from the reconstruction path 640 as a whole. Although inverse quantization and inverse transform 1044 corresponds to inverse quantization and inverse transform 644, and prediction addition 1045 corresponds to prediction addition 645, there are differences in the loop filtering operations. For example, deblocking filter (DBF) 646 is replaced by separate vertical and horizontal deblocking filters (DBFV and DBFH) 1046, 1047. For example, vertical deblocking filtering (DBFV) 1046 receives partially reconstructed image information 1044c (which may correspond to image information 644c, which may also be represented by Y1) and performs deblocking filtering in the vertical direction. In addition, horizontal deblocking filtering (DBFH) 1047 performs deblocking filtering in the horizontal direction based on the result of vertical deblocking filtering (DBFV) 1046. However, it should be noted that horizontal and vertical deblocking filtering (DBFV and DBFH) may also exchange their processing order. Therefore, after applying both vertical deblocking filtering and horizontal deblocking filtering (e.g., by blocks 1046, 1047), deblocked image information 1047a is obtained, which corresponds to image information 646a and is also represented by Y. In addition, the denoising path or post-processing path 1040 also includes a sample adaptive offset (SAO) process and an adaptive loop filter (ALF) process, which is represented by reference numeral 1049 and may correspond to, for example, the adaptive loop filter (ALF) 649 and the post-sample adaptive offset (PSAO) process 660. For example, the sample adaptive offset (SAO) and adaptive loop filter (ALF) processing 1049 may include a classification such as defined by equation (4)', which may be based on Fig.10 The image information Y and Y1 are shown.

[0376] See, e.g., Section 7.3.1.2 for more details.

[0377] Furthermore, the change may be performed, for example, by the SAO processing of block 1049 according to equation (5)′ (Where, for example, an adaptive loop filter (ALF), which is also part of processing block 1049, may then operate on the image information Y').

[0378] Furthermore, it should be noted that further details can be found, for example, in Section 7 of the specification.

[0379] However, it should be noted that according to Fig.10 The video encoder 1000 can optionally be supplemented by any of the features, functions and details disclosed herein, both individually and in combination.

[0380] 11.According to Fig.11 Example

[0381] Fig.11 1 shows a block diagram of an encoder or video encoder according to an embodiment of the present invention. The video encoder as a whole is denoted by 1100. In addition, it should be noted that there are, for example, unfiltered and filtered images, which are denoted by Y1, Y2, and Y3 (which may be represented, for example, by image information or sample values).

[0382] Furthermore, it should be noted that video encoder 1100 is very similar to video encoder 1000, and therefore the same blocks and functions will not be described again here. Specifically, blocks 1120, 1130, 1144, 1145, 1154, 1156, 1152, and 1158 correspond to blocks 1020, 1030, 1044, 1045, 1054, 1052, 1056, and 1058, respectively. In particular, the explanations provided with respect to the corresponding blocks of video encoder 600 also apply.

[0383] However, the processing path or post-processing path or denoising path 1140 is different from the corresponding functions in the video encoder 600, 1000. For example, the denoising path 1140 includes a series of processing blocks or processing functions, namely vertical deblocking filtering (DBFV) 1146, vertical sample adaptive offset (SAOV) processing 1160, horizontal deblocking filtering (DBFH) 1147, horizontal sample adaptive offset (SAOH) processing 1162 and another sample adaptive offset and adaptive loop filtering (SAO+ALF) 1149. In other words, there is a vertical denoising function including operations 1146 and 1160 and a horizontal denoising function including blocks 1147 and 1162, and there is also another sample adaptive offset processing and another adaptive loop filtering (block 1149). In this regard, it should be noted that the order of vertical denoising 1146, 1160 and horizontal denoising 1147, 1162 can be optionally interchanged, and the "final" processing block 1149 can be considered optional. Furthermore, it should be noted that in some embodiments, only one of vertical denoising 1146, 1160 and horizontal denoising 1147, 1162 may be applied.

[0384] Furthermore, it should be noted that any functionality of blocks 1160, 1162 may be performed, for example, as described in Section 7.3.2. For example, vertical sample adaptive offset (SAOV) processing 1162 may perform classification (e.g., according to equation (4)') based on the difference between image information Y2 and Y1 and using, for example, a process defined by equation (5)', where Y2 replaces Y and replace Similarly, the horizontal sample adaptive offset process 1162 can be based on Y3 and The horizontal sample adaptive offset processing 1162 can perform processing according to equation (4)', for example, wherein Y3 can play the role of Y, and wherein the output signal of the horizontal sample adaptive offset processing 1162 can be represented by equation (5)'. Given.

[0385] Finally, video encoder 1100 may, for example, perform the processing described in Section 7.3.2.

[0386] In addition, it should be noted that according to Fig.11 The video encoder 1100 can optionally be supplemented by any of the features, functions and details disclosed herein, both individually and in combination.

[0387] Furthermore, it should be noted that blocks 1046 , 1047 , 1049 may, for example, replace blocks 742 , 744 , 746 , 760 in video decoder 700 , wherein the output image information of block 1049 may, for example, form the video output 712 and the reconstructed image information 762 .

[0388] Similarly, blocks 1146 , 1160 , 1147 , 1162 , 1149 may optionally replace blocks 742 , 744 , 746 , 760 , wherein the output of block 1149 may, for example, replace the video output 712 and the reconstructed image information 762 .

[0389] 12. Conclusion:

[0390] In summary, embodiments according to the invention provide an improved trade-off between compression performance and visual quality and low encoding latency, which results in improved coding efficiency.Some embodiments also provide additional coding efficiency.

[0391] For example, different embodiments and aspects of the invention are described in the sections “Introduction”, “Sample Adaptive Offset”, “Classification for PSAO”, “Decoder”, “Encoder” and “Some Remarks”, where the features, functions and details from the section “Sample Adaptive Offset” can be optionally introduced into any other embodiment.

[0392] However, features, functions and details described in any other sections may also optionally be incorporated into an embodiment according to the present invention.

[0393] In addition, the embodiments described in the above sections can be used alone and can also be supplemented by any features, functions and details in another section.

[0394] In addition, it should be noted that the individual aspects described herein can be used alone or in combination. Therefore, details can be added to each of the individual aspects without adding details to another of the aspects.

[0395] In particular, the embodiments are also described in the claims. The embodiments described in the claims may optionally be supplemented by any of the features, functions and details described herein, both individually and in combination.

[0396] It should also be noted that the present disclosure explicitly or implicitly describes features that can be used in a video encoder (a device for providing an encoded representation of an input video signal) and a video decoder (a device for providing a decoded representation of a video signal based on the encoded representation of the video signal). Therefore, any feature described herein can be used in the context of a video encoder and in the context of a video decoder.

[0397] In addition, the features and functions disclosed herein in relation to the method may also be used in a device (configured to perform such function). In addition, any features and functions disclosed herein regarding a device may also be used in a corresponding method. In other words, the method disclosed herein may be supplemented by any features and functions described regarding a device.

[0398] Furthermore, any features and functions described herein may be implemented in hardware or software, or using a combination of hardware and software, as will be described in the "Implementation Alternatives" section.

[0399] Implementing alternatives

[0400] Although some aspects are described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, wherein a block or device corresponds to a method step or a feature of a method step. Similarly, the aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware device, such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such a device.

[0401] According to certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. The implementation may be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM or FLASH memory, on which electronically readable control signals are stored, which cooperates (or can cooperate) with a programmable computer system to perform the corresponding method. Therefore, the digital storage medium may be computer readable.

[0402] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0403] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, wherein when the computer program product runs on a computer, the program code is operative to perform one of the methods. The program code may, for example, be stored on a machine-readable carrier.

[0404] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0405] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0406] Therefore, another embodiment of the inventive method is a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitory.

[0407] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.The data stream or the sequence of signals may, for example, be configured to be transmitted via a data communication connection, for example via the Internet.

[0408] A further embodiment comprises a processing means, for example a computer or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0409] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0410] Another embodiment according to the invention comprises an apparatus or system configured to transmit (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may, for example, comprise a file server for transmitting the computer program to the receiver.

[0411] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may collaborate with a microprocessor to perform one of the methods described herein. Typically, these methods are preferably performed by any hardware device.

[0412] The means described herein may be implemented using hardware devices, or using computers, or using a combination of hardware devices and computers.

[0413] The apparatus described herein or any component of an apparatus described herein may be implemented at least partially in hardware and / or software.

[0414] The methods described herein may be performed using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

[0415] Any component of a method described herein or an apparatus described herein may be performed at least in part by hardware and / or software.

[0416] The embodiments described herein are merely for the purpose of illustrating the principles of the present invention. It should be understood that modifications and variations of the arrangements and details described herein will be apparent to other persons skilled in the art. Therefore, it is intended that the scope of the present invention be limited only by the scope of the forthcoming patent claims, rather than by the specific details presented by the description and explanation of the embodiments herein.

Claims

1. A video decoder (100) for encoding video information (VI en ) provides decoded video information (VI de ), The video decoder (100) is configured to en ) obtain the first sample information (I s1 ); The video decoder (100) is configured to apply a first denoising operation (102) so as to obtain a first sample information (I s1 ) obtain the first denoising sample information (I ds1 ); wherein the video decoder (100) is configured to apply a second denoising operation (103) based on a result of the first denoising operation (102); in, The video decoder (100) is configured to adjust the second denoising operation (103) depending on the difference between samples before the first denoising operation (102) and after the first denoising operation (102) so as to obtain the decoded video information (VI de ).

2. The video decoder (100) as claimed in claim 1, wherein the second denoising operation (103) is configured to classify the sample into samples of a plurality of classes depending on the difference between the sample before the first denoising operation (102) and the sample after the first denoising operation (102), And wherein the second denoising operation (103) is configured to modify the sample depending on the classification (C).

3. The video decoder (100) as claimed in claim 1, The second denoising operation (103) shown therein is configured to conditionally add an offset to a plurality of samples depending on the classification of the respective samples.

4. The video decoder (100) as claimed in claim 1, The first denoising operation (102) includes a deblocking filtering operation and / or a sample adaptive offset filtering operation.

5. The video decoder (100) as claimed in claim 4, Wherein the sample adaptive offset filtering operation is configured to classify samples of the image into a plurality of classes depending on one or more edge patterns of the image in the environment of the sample under consideration and / or depending on sample values ​​of the sample under consideration.

6. The video decoder (100) of claim 1, The first denoising operation (102) comprises an adaptive loop filtering (ALF) operation, wherein the adaptive loop filtering (ALF) operation performs filtering using an impulse response, and the filter coefficients are encoded in the bitstream.

7. The video decoder (100) of claim 1, in,The video decoder (100) is configured to apply the second denoising operation (103) to sample information obtained using a combination of one or more deblocking filtering operations, one or more sample adaptive offset filtering operations and an adaptive loop filtering operation.

8. The video decoder (100) of claim 1, wherein the second denoising operation (103) is configured to be applied to a plurality of coding tree units (CTUs) defining a video frame, and / or Wherein the second denoising operation (103) is configured to be applied to a video frame consisting of coding tree units defining the video frame.

9. The video decoder (100) of claim 2, Wherein classification (C) of the sample into a plurality of classes occurs only when one or more predetermined thresholds are reached.

10. The video decoder (100) of claim 1, wherein the second denoising operation (103) is configured to classify the sample into three or more classes depending on whether the difference between the corresponding samples before the first denoising operation (102) and after the first denoising operation (102) satisfies the first condition, the second condition or the third condition, Alternatively, the samples are classified into three or more classes depending on whether a combined difference value based on a plurality of differences between corresponding samples before and after the first denoising operation satisfies the first condition, the second condition, or the third condition.

11. The video decoder (100) of claim 1, in, The second denoising operation (103) is configured to classify samples into three or more classes depending on the magnitude of the difference between corresponding samples before the first denoising operation (102) and after the first denoising operation (102) and depending on the sign of the difference between corresponding samples before the first denoising operation (102) and after the first denoising operation (102).

12. The video decoder (100) of claim 1, The second denoising operation (103) is configured to classify the samples into three categories according to: Λ1={i∈l:(y2(i)-y i (i))>T} and Λ2={i∈I: (y2(i)-y1(i)<-T} Λ0=I∩(Λ1∪Λ2) c where Λ1 is the set of samples associated with the first class, where Λ2 is the sample set associated with the second class, where Λ0 is the sample set associated with the third class, Where T is a predetermined value; Where I (=capital i) is the basic sample set; where ∩ is the cut set operator - intersection; where ∪ is the union operator - union; where c is the complement set operator.

13. The video decoder (100) of claim 1, in, The second denoising operation (103) is configured to determine a combination of weighted differences to obtain a combined difference value associated with the sample currently considered, the combination of weighted differences combining a plurality of differences between corresponding samples in a predetermined environment of the sample currently considered before the first denoising operation (102) and after the first denoising operation (102).

14. The video decoder (100) of claim 13, The second denoising operation (103) is configured to determine the combined difference f(i) according to: where j is the sample index, wherein N(i) is the set of samples to be considered for determining the combined difference value associated with the considered sample with sample index i; wherein w(ij) is a set of predetermined weight values ​​associated with a plurality of sample positions in the environment of the considered sample; Where y2(j)-y1(j) is the difference between the corresponding samples before the first denoising operation (102) and after the first denoising operation (102).

15. The video decoder (100) of claim 2, in, The second denoising operation (103) is configured to select a quantization offset value depending on the classification (C).

16. The video decoder (100) of claim 2, The second denoising operation (103) is configured to use the encoded video information (VI en ) to select the offset value.

17. The video decoder (100) of claim 1, in, The second denoising operation (103) is configured to generate enhanced sample values ​​according to where d l is the offset value corresponding to the class; Where I (= uppercase i) is the class index; Where Q is the quantization function; Where y is the sample value input to the second denoising operation (103); where χΛ l is an indicator function that indicates to which class l the currently considered sample y is classified.

18. The video decoder (100) of claim 17, Wherein the indicator function χΛ l Defined by as well as Λ l It is a class; Where l is the class index of each sample position i in the basic sample set I; Where i is the sample position in the basic sample set I; Where C is the classification (for sample position i); where ∈ is the set membership operator; in is a non-set membership operator.

19. The video decoder (100) of claim 1, wherein the second denoising operation (103) is configured to use a first offset value for all samples of the video frame classified as a first class, and / or wherein the second denoising operation (103) is configured to use a second offset value for all samples of the video frame classified as a second class, and / or in, The second denoising operation (103) is configured to use a third offset value for all samples of the video frame classified into a third class.

20. The video decoder (100) of claim 1, in, The video decoder (100) is configured to evaluate a bitstream flag to decide whether to perform the second denoising operation (103).

21. The video decoder (100) of claim 1, in, The video decoder (100) is configured to apply the second denoising operation (103) independently to different YUV components.

22. The video decoder (100) of claim 1, The second denoising operation (103) is configured to classify the samples into a plurality of classes according to: and and Λ0=I∩(A1∪…∪A 2L ) c where Λ l is the set of samples associated with the 1st,...,2Lth classes; where Λ0 is the set of samples associated with classes other than the 1st,...,2Lth classes; Where T l-1 To T l is a predetermined value; Where I (=capital i) is the basic sample set; where ∩ is the cut set operator - intersection; where U is the union operator - union; where c is the complement set operator.

23. The video decoder (100) of claim 22, wherein: For the difference between the samples before the first denoising operation (102) and after the first denoising operation (102), T l-1 To T l is a symmetric class boundary in order to obtain the classification of each sample.

24. The video decoder (100) of claim 22, wherein the second denoising operation (103) is configured to use the first to second L offset values ​​for all samples of a video frame or coding tree unit (CTU) classified into the corresponding class, and / or Wherein the second denoising operation (103) is configured to use the offset value for all samples of the video frame classified as being in a class complementary to I (= uppercase i).

25. The video decoder (100) of claim 1, The second denoising operation (103) uses the same parameter set for processing a frame or each CTU.

26. The video decoder (100) of claim 1, Wherein the first denoising operation (102) is a deblocking filtering operation in a first direction, and wherein the second denoising operation is a first sample adaptive offset operation.

27. The video decoder (100) of claim 26, wherein the video decoder (100) is configured to apply a third denoising operation, the third denoising operation being a deblocking filtering operation in a second direction using a result of the second denoising operation (103), and in, The video decoder (100) is configured to apply a fourth denoising operation based on a result of the third denoising operation.

28. The video decoder (100) of claim 27, The second denoising operation (103) and the fourth denoising operation use different parameters or parameter sets for processing the frame.

29. The video decoder (100) of claim 28, in, The video decoder (100) is configured to apply a fifth denoising operation based on a result of the fourth denoising operation.

30. The video decoder (100) of claim 29, The fifth denoising operation uses different parameters or parameter sets.

31. A video encoder (200) for providing coded video information (VI) based on input video information (VI) en ), in, The video encoder (200) is configured to obtain first sample information (I) based on the input video information (VI) s1 ), the first sample information takes into account the transformation to the spectral domain (205), the quantization of the transformation coefficients (206) and the prediction (207), The video encoder (200) is configured to apply a first denoising operation (202) so as to generate a first denoising image based on the first sample information (I s1 ) obtain the first denoising sample information (I ds1 ); wherein the video encoder (200) is configured to determine parameters of a second denoising operation (203) based on a result of the first denoising operation (202); The video encoder (200) is configured to adjust the second denoising operation (203) depending on the difference between samples before the first denoising operation (202) and after the first denoising operation (202) so as to obtain the encoded video information (VI en ).

32. The video encoder (200) of claim 31, wherein the second denoising operation (203) is configured to classify the sample into samples of a plurality of classes depending on the difference between the sample before the first denoising operation (202) and the sample after the first denoising operation (202), And wherein the video encoder (200) is configured to determine parameters of the second denoising operation depending on the classification (C).

33. The video encoder (200) of claim 31, Wherein the second denoising operation (203) is configured to conditionally add an offset to a plurality of samples depending on a classification of the respective samples.

34. The video encoder (200) of claim 31, The first denoising operation (202) includes a deblocking filter and / or a sample adaptive offset filter operation.

35. The video encoder (200) of claim 34, Wherein the sample adaptive offset filtering operation is configured to classify samples of the image into a plurality of classes depending on one or more edge patterns of the image in the environment of the considered sample and / or depending on sample values ​​of the considered sample.

36. The video encoder (200) of claim 31, The first denoising operation (202) comprises an adaptive loop filtering (ALF) operation, wherein the adaptive loop filtering (ALF) operation performs filtering using an impulse response, and filter coefficients are encoded in a bitstream.

37. The video encoder (200) of claim 31, in, The video encoder (200) is configured to apply the second denoising operation (203) to sample information obtained using a combination of one or more deblocking filtering operations, one or more sample adaptive offset filtering operations, and an adaptive loop filtering operation.

38. The video encoder (200) of claim 31, wherein the second denoising operation (203) is configured to be applied to a plurality of coding tree units (CTUs) defining a video frame, and / or Wherein the second denoising operation (203) is configured to be applied to a video frame consisting of coding tree units defining the video frame.

39. The video encoder (200) of claim 32, The classification (C) of the sample into multiple classes only occurs when a predetermined threshold is reached.

40. The video encoder (200) of claim 31, wherein the second denoising operation (203) is configured to depend on whether the difference between the corresponding samples before the first denoising operation (202) and after the first denoising operation (202) satisfies the first condition, the second condition or the third condition, Or depending on whether the combined difference value satisfies the first condition, the second condition, or the third condition, the combined difference value classifies the sample into three or more classes based on a plurality of differences between corresponding samples before the first denoising operation (202) and after the first denoising operation (202).

41. The video encoder (200) of claim 31, in, The second denoising operation (203) is configured to classify samples into three or more classes depending on the magnitude of the difference between corresponding samples before the first denoising operation (202) and after the first denoising operation (202) and depending on the sign of the difference between corresponding samples before the first denoising operation (202) and after the first denoising operation (202).

42. The video encoder (200) of claim 31, The second denoising operation (203) is configured to classify the samples into three categories according to: Λ1={i∈I: (y2(i)-y1(i))>T} and Λ2={i∈I: (y2(i)-y1(i))<-T} Λ0=I∩(Λ1∪Λ2) where Λ1 is the set of samples associated with the first class, where Λ2 is the sample set associated with the second class, where Λ0 is the sample set associated with the third class, Where T is a predetermined value; Where I is the basic sample set; where ∩ is the cut set operator - intersection; where U is the union operator - union; where c is the complement set operator.

43. The video encoder (200) of claim 31, in, The second denoising operation (203) is configured to determine a combination of weighted differences to obtain a combined difference value associated with a currently considered sample, the combination of weighted differences combining a plurality of differences between corresponding samples in a predetermined environment of the currently considered sample before the first denoising operation (202) and after the first denoising operation (202); The second denoising operation (203) is configured to determine the combined difference f(i) according to: where j is the sample index, wherein N(i) is the set of samples to be considered for determining the combined difference value associated with the considered sample with sample index i; wherein w(ij) is a set of predetermined weight values ​​associated with a plurality of sample positions in the environment of the considered sample; Where y2(j)-y1(j) is the difference between the corresponding samples before the first denoising operation (202) and after the first denoising operation (202).

44. The video encoder (200) of claim 31, in, The second denoising operation (203) is configured to select a quantization offset value depending on the classification (C).

45. The video encoder (200) of claim 31, wherein the second denoising operation (203) is configured to use the encoded video information (VI e n) to select the offset value.

46. ​​The video encoder (200) of claim 31, in, The second denoising operation (203) is configured to generate enhanced sample values ​​according to where d l is the offset value corresponding to the class; Where l is the class index; Where Q is the quantization function; Where y is the sample value input to the second denoising operation (203); where χΛ l is an indicator function that indicates to which class l the currently considered sample y is classified.

47. The video encoder (200) of claim 46, Wherein the indicator function χΛ l Defined by and where Λ l It is a class; Where l is the class index of each sample position i in the basic sample set I; Where i is the sample position in the basic sample set I; Where C is the classification (for sample position i); where ∈ is the set membership operator; in is a non-set membership operator.

48. The video encoder (200) of claim 31, wherein the second denoising operation (203) is configured to use a first offset value for all samples of the video frame classified as the first class, and / or wherein the second denoising operation (203) is configured to use a second offset value for all samples of the video frame classified as a second class, and / or in, The second denoising operation (203) is configured to use a third offset value for all samples of the video frame classified into a third class.

49. The video encoder (200) of claim 31, in, The video encoder (200) is configured to evaluate a bitstream flag to decide whether to perform the second denoising operation (203).

50. The video encoder (200) of claim 31, The video encoder (200) is configured to apply the second denoising operation (203) independently to different YUV components.

51. The video encoder (200) of claim 31, in, The video encoder (200) is configured to determine one or more offset values ​​used in the second denoising operation (203) based on the input video representation and depending on sample classification performed based on differences between samples before the first denoising operation (202) and after the first denoising operation (202).

52. The video encoder (200) of claim 31, in, The video encoder (200) is configured to determine an offset value (d1) associated with a set of samples classified into a given class depending on a deviation between input samples of the second denoising operation (203) classified into the given class and corresponding samples of the input video information (VI).

53. The video encoder (200) of claim 52, in, The video encoder (200) is configured to determine an offset value (d1) associated with a set of samples classified into a given class so that a mean square error between input samples of the second denoising operation (203) classified into the given class and corresponding samples of the input video information (VI) is minimized or brought below a predetermined threshold.

54. The video encoder (200) of claim 52, in, The video encoder (200) is configured to determine an offset value d1 associated with a set of samples classified into a given class with class index 1 according to the following formula: Among them #(Λ l ) is the number of samples classified into the class with class index 1; where Λ l is the set of samples classified into the class with class index 1; Where i is the sample index; wherein x(i) is a sample value of the input video information (VI); and Where y(i) is the sample value input to the second denoising operation (203).

55. The video encoder (200) of claim 31, The second denoising operation (203) is configured to classify the samples into a plurality of classes according to: and and Λ0=I∩(Λ1∪…∪A 2L ) c where Λ l is the set of samples associated with the 1st,…,2Lth classes; where Λ0 is the set of samples associated with classes other than the 1st,…,2Lth classes; Where T l-1 To T l is a predetermined value; Where I (=capital i) is the basic sample set; where ∩ is the cut set operator - intersection; where ∪ is the union operator - union; where c is the complement set operator.

56. The video encoder (200) of claim 55, wherein: For the difference between the samples before the first denoising operation (202) and after the first denoising operation (202), T l-1 To T l is a symmetric class boundary in order to obtain the classification of each sample.

57. The video encoder (200) of claim 55, wherein the second denoising operation (203) is configured to use the first to second L offset values ​​for all samples of a video frame or coding tree unit (CTU) classified into the corresponding class, and / or Wherein the second denoising operation (203) is configured to use the offset value for all samples of the video frame classified as being in a class complementary to I (= uppercase i).

58. The video encoder (200) of claim 31, The second denoising operation (203) uses the same parameter set for processing a frame or each CTU.

59. The video encoder (200) of claim 31, The first denoising operation (202) is a deblocking filtering operation in a first direction, And wherein the second denoising operation (203) is a first sample adaptive offset operation.

60. The video encoder (200) of claim 59, wherein the video encoder (200) is configured to apply a third denoising operation, the third denoising operation being a deblocking filtering operation in a second direction using a result of the second denoising operation (203), and in, The video encoder (200) is configured to apply a fourth denoising operation based on a result of the third denoising operation.

61. The video encoder (200) of claim 60, The second denoising operation (203) and the fourth denoising operation use different parameters or parameter sets for processing frames.

62. The video encoder (200) of claim 61, in, The video encoder (200) is configured to apply a fifth denoising operation based on a result of the fourth denoising operation.

63. The video encoder (200) of claim 62, The fifth denoising operation uses different parameters or parameter sets.

64. A method (300) for providing decoded video information based on encoded video information, in, The method includes obtaining (302) first sample information based on the encoded video information; The method comprises applying (303) a first denoising operation to obtain first denoised sample information based on the first sample information; wherein the method comprises applying (305) a second denoising operation based on the result of the first denoising operation; Therein, the method comprises adjusting (304) the second denoising operation in dependence on a difference between samples before the first denoising operation and after the first denoising operation in order to obtain the decoded video information.

65. A method (400) for providing encoded video information based on input video information, The method comprises obtaining (402) first sample information based on the input video information, the first sample information taking into account transformation to the spectral domain, quantization and prediction of transformation coefficients, in, The method includes applying (403) a first denoising operation to obtain first denoised sample information based on the first sample information; Wherein the method comprises determining parameters of a second denoising operation based on the results of the first denoising operation; Therein, the method comprises adjusting (404) the second denoising operation in dependence on a difference between samples before the first denoising operation and after the first denoising operation in order to obtain the encoded video information.

66. A computer-readable storage medium comprising a computer program, which, when executed on a computer, causes the computer to execute the method according to claim 64 or 65.

Images