System and method for applying a deblocking filter to restored video data

Abstract

To provide a system and a method for adopting a deblocking filter.SOLUTION: A method includes steps of: receiving a value of a sample contained in two blocks adjacent to a vertical block boundary of a restored video data; determining whether or not that each block is a large block on the basis of that a width of each block is 32 or larger; calculating a first gradient value and a second gradient value for the two blocks; calculating a variable beta on the basis of a quantization parameter; setting (3*beta>>5) to a threshold value in the case where any one of the blocks is the large block; setting (beta>>3) to the threshold in the case where both the blocks are not the large block; and performing a filtering by adopting a wide and strong filter, a strong filter, or a weak filter to a sample value contained in each block on the basis of that a sum of the first and second gradient values is smaller than the threshold value.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present disclosure relates to video coding, and more particularly, to techniques for performing deblocking of restored video data. Background Art

[0002] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, laptop or desktop computers, tablet computers, digital recording devices, digital media players, video gaming devices, cellular telephones including so-called smart phones, medical imaging devices, and the like. Digital video can be encoded according to video encoding standards. Video encoding standards can incorporate video compression techniques. Examples of video encoding standards include ISO / IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC) and High-Efficiency Video Coding (HEVC). HEVC is described in High Efficiency Video Coding (HEVC), Rec. ITU-T H.265 (December 2016), which is incorporated herein by reference and is referred to herein as ITU-T H.265. Extensions and improvements to ITU-T H.265 are currently being considered for the development of next-generation video encoding standards. For example, the ITU-T Video Coding Experts Group (VCEG) and ISO / IEC (Moving Picture Experts Group (MPEG) (collectively referred to as the Joint Video Exploration Team (JVET)) are considering the potential need to standardize future video encoding technologies that have compression capabilities significantly exceeding those of the current HEVC standard. The Joint Exploration Model 7 (JEM 7), Algorithm Description of Joint Exploration Test Model 7 (JEM 7), ISO / IEC JTC1 / SC29 / WG11 Document: Video Coding Technology, describes the encoding features of the collaborative test model being studied by the JVET as having the potential to improve beyond the capabilities of ITU-T H.265, JVET-G1001, July 2017, Torino, IT, which is incorporated herein by reference. Note that the encoding features of JEM 7 are implemented in the JEM reference software.As used herein, the term JEM may collectively represent the algorithms included in JEM 7 and the implementation of the JEM reference software.

[0003] Video compression techniques reduce the data requirements for storing and transmitting video data by exploiting the inherent redundancy in a video sequence. Video compression techniques can successively divide a video sequence into smaller and smaller parts (e.g., groups of frames within the video sequence, frames within a group of frames, slices within a frame, coded tree units (e.g., macroblocks) within a slice, coded blocks within a coded tree unit, etc.). Intra prediction coding techniques (e.g., within-picture (spatial)) and inter prediction techniques (i.e., between-picture (temporal)) can be used to generate difference values between a unit of video data to be coded and a reference unit of video data. The difference values may be referred to as residual data. The residual data can be coded as quantized transform coefficients. Syntax elements can associate the residual data with the reference coding unit (e.g., intra prediction mode index, motion vector, and block vector). The residual data and syntax elements can be entropy coded. The entropy-coded residual data and syntax elements can be included in a compliant bitstream. The compliant bitstream and associated metadata may have a format according to a data structure. Summary of the Invention

[0004] In one embodiment, a method of filtering restored video data includes receiving an array of sample values including adjacent restored video blocks for components of the video data, and modifying the sample values in the adjacent restored video blocks according to a plurality of passes of a deblocking filter. Brief Description of the Drawings

[0005]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5A

Figure 5B

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14A

Figure 14B

[0006] Generally, the present disclosure describes various techniques for encoding video data. In particular, the present disclosure describes techniques for performing deblocking of the restored video data. The techniques of the present disclosure are described with respect to ITU-T H.264, ITU-T H.265, and JEM, but it should be noted that the techniques of the present disclosure are generally applicable to video encoding. For example, the encoding techniques described herein can be incorporated into a video encoding system (including a video encoding system based on a future video encoding standard) that includes block structures, intra prediction techniques, inter prediction techniques, transform techniques, filtering techniques, and / or entropy encoding techniques other than those included in ITU-T H.265. Therefore, the references to ITU-T H.264, ITU-T H.265, and JEM are for illustrative purposes and should not be construed as limiting the scope of the techniques described herein. Further, it should be noted that the incorporation by reference of documents herein should not be construed as limiting or creating ambiguity with respect to the terms used herein. For example, if the incorporated reference provides a definition of a term that is different from another incorporated reference and / or from the way the term is used herein, the term should be construed to broadly include each corresponding definition and / or to include each specific definition instead.

[0007] In one embodiment, a device for video encoding receives an array of sample values including adjacent restored video blocks for components of video data, and includes one or more processors configured to modify sample values within the adjacent restored video blocks according to a plurality of paths of a deblocking filter.

[0008] In one embodiment, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device to receive an array of sample values including adjacent restored video blocks for components of video data, and to modify sample values within the adjacent restored video blocks according to a plurality of paths of a deblocking filter.

[0009] In one embodiment, an apparatus includes means for receiving an array of sample values including adjacent restored video blocks for components of video data, and means for modifying sample values within the adjacent restored video blocks according to a plurality of paths of a deblocking filter.

[0010] Details of one or more embodiments are described in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

[0011] Video content typically includes a video sequence consisting of a series of frames. A series of frames may also be referred to as a group of pictures (GOP). Each video frame or picture can include a plurality of slices or tiles, and a slice or tile includes a plurality of video blocks. As used herein, the term video block generally may refer to a portion of a picture, or more specifically, to the largest array of sample values that can be predictedly encoded, its subdivision, and / or the corresponding structure. Further, the term current video block may refer to a portion of a picture that is being encoded or decoded. A video block can be defined as an array of sample values that can be predictedly encoded. In some cases, it should be noted that pixel values can be described as including sample values for each component of video data, which may also be referred to as color components (e.g., luma component (Y) and chroma components (Cb and Cr), or red, green, and blue components). In some cases, it should be noted that the terms pixel value and sample value are used interchangeably. Video blocks can be ordered within a picture according to a scanning pattern (e.g., raster scan). A video encoder can perform predictive encoding on video blocks and their subdivisions. Video blocks and their subdivisions may be referred to as nodes.

[0012] ITU-T H.264 specifies a macroblock structure that includes 16×16 luma samples. That is, in ITU-T H.264, a picture is divided into macroblocks. ITU-T H.265 specifies a similar Coding Tree Unit (CTU) structure, which may also be referred to as the Largest Coding Unit (LCU). In ITU-T H.265, a picture is divided into CTUs. In ITU-T H.265, for one picture, the CTU size can be set to include 16×16, 32×32, or 64×64 luma samples. In ITU-T H.265, a CTU is composed of respective Coding Tree Blocks (CTBs) for each component of the video data (e.g., luma (Y) and chroma (Cb and Cr)). Further, in ITU-T H.265, a CTU can be divided according to a quadtree (QT) partitioning structure, and as a result, the CTBs of the CTU are divided into Coding Blocks (CBs). That is, in ITU-T H.265, a CTU can be divided into quadtree leaf nodes. According to ITU-T H.265, one luma CB, together with two corresponding chroma CBs and related syntax elements, is called a Coding Unit (CU). In ITU-T H.265, the minimum allowable size of a CB can be signaled. In ITU-T H.265, the smallest minimum allowable size of a luma CB is 8×8 luma samples. In ITU-T H.265, the decision to encode a picture part using intra prediction or inter prediction is made at the CU level.

[0013] In ITU-T H.265, a CU is associated with a prediction unit (PU) structure having a root in the CU. In ITU-T H.265, the PU structure enables splitting of the luma CB and chroma CB for the purpose of generating corresponding reference samples. That is, in ITU-T H.265, the luma CB and chroma CB can be split into respective luma and chroma prediction blocks (PBs), where a PB contains a block of sample values to which the same prediction is applied. In ITU-T H.265, a CB can be split into 1, 2, or 4 PBs. ITU-T H.265 supports PB sizes from 64×64 samples down to 4×4 samples. In ITU-T H.265, square PBs are supported for intra prediction, where a CB can form a PB or be divided into 4 square PBs (i.e., the intra prediction PB size types include M×M or M / 2×M / 2, where M is the height and width of the square CB). In ITU-T H.265, in addition to square PBs, rectangular PBs are supported for inter prediction, where a CB can be bisected vertically or horizontally to form a PB (i.e., the inter prediction PB types include M×M, M / 2×M / 2, M / 2×M, or M×M / 2). Further, in ITU-T H.265, 4 asymmetric PB splits are supported for inter prediction, where a CB is divided into 2 PBs at one quarter of its height (upper or lower) or width (left or right) (i.e., the asymmetric partitions include M / 4×M left, M / 4×M right, M×M / 4 upper, and M×M / 4 lower). Reference sample values and / or predicted sample values for a PB are generated using intra prediction data (e.g., intra prediction mode syntax elements) or inter prediction data (e.g., motion data syntax elements) corresponding to the PB.

[0014] JEM defines a CTU that has a maximum size of 256×256 luma samples. JEM defines a quad-tree + binary-tree (QTBT) block structure. In JEM, the QTBT structure allows the quadtree leaf nodes to be further divided by a binary-tree structure (BT). That is, in JEM, the binary-tree structure allows the quadtree leaf nodes to be recursively divided vertically or horizontally. Figure 1 shows an example of a CTU (e.g., a CTU having a size of 256×256 luma samples) that is divided into quadtree leaf nodes and where the quadtree leaf nodes are further divided according to a binary tree. That is, in Figure 1, the dashed lines suggest additional binary-tree partitions in the quadtree. Thus, the binary-tree structure in JEM allows for square leaf nodes and rectangular leaf nodes, and each leaf node contains one CB. As shown in Figure 1, the pictures included in a GOP can contain multiple slices, each slice contains a series of CTUs, and each CTU can be divided according to the QTBT structure. Figure 1 shows an example of the QTBT division of one CTU included in one slice. Thus, the binary-tree structure in JEM allows for square leaf nodes and rectangular leaf nodes, and each leaf node contains one CB. In JEM, the CB is used for prediction without any division. That is, in JEM, the CB can be a block of sample values to which the same prediction is applied. Thus, the JEM QTBT leaf nodes can be similar to the PB in ITU-T H.265.

[0015] A video sampling format, sometimes called chroma format, can define the number of chroma samples contained in a CU relative to the number of luma samples contained in the CU. For example, for the 4:2:0 format, the sampling rate for the luma component is twice the sampling rate for the chroma component in both the horizontal and vertical directions. As a result, for a CU formatted according to the 4:2:0 format, the width and height of the sample array for the luma component are twice the width and height of each sample array for the chroma component. FIG. 2 is a conceptual diagram showing an example of an encoded unit formatted according to the 4:2:0 sample format. FIG. 2 shows the relative positions of the chroma samples with respect to the luma samples within the CU. As described above, a CU is typically defined according to the number of horizontal and vertical luma samples. Thus, as shown in FIG. 2, a 16×16 CU formatted according to the 4:2:0 sample format includes 16×16 samples of the luma component and 8×8 samples for each chroma component. Further, in the example shown in FIG. 2, the relative positions of the chroma samples with respect to the luma samples for video blocks adjacent to the 16×16 CU are shown. For a CU formatted according to the 4:2:2 format, the width of the sample array for the luma component is twice the width of the sample array for each chroma component, but the height of the sample array for the luma component is equal to the height of the sample array for each chroma component. Further, for a CU formatted according to the 4:4:4 format, the sample array for the luma component has the same width and height as the sample array for each chroma component.

[0016] As described above, the intra prediction data or the inter prediction data is used to generate reference sample values for a block of sample values. The difference between the sample values included in the current PB or another type of picture part structure and the associated reference samples (e.g., reference samples generated using prediction) may be referred to as residual data. The residual data can include respective arrays of difference values corresponding to respective components of the video data. The residual data can be within a pixel region. Transformations such as discrete cosine transform, discrete sine transform (DST), integer transform, wavelet transform, or conceptually similar transforms can be applied to the array of difference values to generate transform coefficients. Note that in ITU-T H.265, a CU is associated with a transform unit (TU) structure having a root at the CU level. That is, in ITU-T H.265, for the purpose of generating transform coefficients, the array of difference values can be re-partitioned (e.g., four 8×8 transforms can be applied to a 16×16 array of residual values). For each component of the video data, such re-partitioning of the difference values may be referred to as a transform block (TB). Note that in ITU-T H.265, it is not necessary to align the TBs with the PBs. Figure 3 shows an example of an alternative combination of PBs and TBs that can be used to encode a particular CB. Further, note that in ITU-T H.265, the TBs can have the following sizes: 4×4, 8×8, 16×16, and 32×32. In JEM, transform coefficients are generated without further partitioning, using the residual values corresponding to the CBs. That is, in JEM, a QTBT leaf node can be similar to both PBs and TBs in ITU-T H.265. In JEM, the core transform and subsequent secondary transform can be applied (in a video encoder) to generate transform coefficients. For a video decoder, the order of the transforms is reversed. Further, in JEM, whether to apply the secondary transform to generate transform coefficients may depend on the prediction mode.

[0017] The conversion coefficient can be quantized according to a quantization process. Quantization approximates the conversion coefficient by an amplitude limited to a specific set of values. Quantization may be used to vary the amount of data required to represent the group of conversion coefficients. Quantization can generally be described as being implemented by division of the conversion coefficient by a magnification factor and any associated rounding function (e.g., rounding to the nearest integer). Inverse quantization (or "dequantization") can include multiplication of the coefficient level value by a magnification factor. As used herein, it should be noted that the term quantization process generally may, in some cases, refer to division by a magnification factor to generate a level value or, in some cases, multiplication by a magnification factor to recover a conversion coefficient. That is, the quantization process may, in some cases, refer to quantization and, in some cases, to inverse quantization. The current block of video data is restored by performing inverse quantization on the level value, performing an inverse transform, and adding a set of predicted values to the resulting residual. The sample values of the restored block may be different from the sample values of the current video block input to the encoding process. Thus, it can be said that encoding is non-invertible. However, it should be noted that the difference in sample values can be considered acceptable to an observer of the restored video.

[0018] The quantized transform coefficients (which may be referred to as level values) can be entropy encoded according to entropy encoding techniques such as content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), probability interval partitioning entropy coding (PIPE), etc. Further, syntax elements such as syntax elements indicating prediction modes can also be entropy encoded. The entropy encoded and quantized transform coefficients and the corresponding entropy encoded syntax elements can form a compliant bitstream that can be used to regenerate video data. The binarization process can be performed on the syntax elements as part of the entropy encoding process. Binarization refers to the process of converting syntax values into a series of one or more bits. These bits may be referred to as "bins". FIGS. 4A-4B are conceptual diagrams showing examples of encoding blocks of video data. As shown in FIG. 4A, the current block of video data (e.g., CB corresponding to a video component) is encoded by subtracting a set of predicted values from the current block of video data to generate a residual, performing a transform on the residual, and quantizing the transform coefficients to generate level values. As shown in FIG. 4B, the current block of video data is decoded by performing inverse quantization on the level values, performing an inverse transform, and adding the set of predicted values to the resulting residual. It should be noted that in the examples of FIGS. 4A-4B, the sample values of the restored block are different from the sample values of the encoded current video block. Thus, it can be said that the encoding is irreversible. However, the difference in sample values can be considered acceptable or imperceptible to an observer of the restored video.

[0019] As shown in FIG. 4A, the quantized transform coefficients are encoded into a bitstream. The quantized transform coefficients and syntax elements (e.g., syntax elements indicating the coding structure of a video block) can be entropy encoded according to entropy coding techniques. Examples of entropy coding techniques include content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), probability interval partitioning entropy coding (PIPE), etc. The entropy encoded quantized transform coefficients and the corresponding entropy encoded syntax elements can form a compliant bitstream that can be used to reproduce video data at a video decoder. The entropy coding process can include performing binarization on the syntax elements. Binarization refers to the process of converting the value of a syntax value into a series of one or more bits. These bits may be referred to as "bins". Binarization is a lossless process and can include one or a combination of the following coding techniques: fixed length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-th exponential Golomb coding, and Golomb-Rice coding. For example, binarization may include representing the integer value of 5 of a syntax element as 00000101 using an 8-bit fixed length binarization technique, or representing the integer value of 5 as 11110 using a unary coding binarization technique. As used herein, each of the terms fixed length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-th exponential Golomb coding, and Golomb-Rice coding may refer to a general implementation of these techniques and / or a more specific implementation of these coding techniques. For example, an implementation of Golomb-Rice coding can be specifically defined according to a video coding standard, such as ITU-T H.265.The entropy encoding process further includes encoding the bin values using a reversible data compression algorithm. In the case of CABAC, for a particular bin, the context model can be selected from a set of available context models associated with the bin. In some examples, the context model can be selected based on the values of the previous bin and / or the previous syntax element. The context model can identify the probability that a bin has a particular value. For example, the context model can indicate a probability of 0.7 for encoding a bin with a value of 0 and a probability of 0.3 for encoding a bin with a value of 1. Note that in some cases, the probabilities of encoding a bin with a value of 0 and a bin with a value of 1 may not sum to 1. After selecting an available context model, the CABAC entropy encoder can arithmetically encode the bin based on the identified context model. The context model can be updated based on the value of the encoded bin. The context model can be updated based on related variables stored with the context, such as an adaptive window size, the number of bins encoded using the context. In accordance with ITU-T H.265, the CABAC entropy encoder can be implemented such that some syntax elements can be entropy encoded using arithmetic coding without the use of an explicitly assigned context model, and such encoding may sometimes be referred to as bypass coding.

[0020] As described above, intra prediction data or inter prediction data can associate a portion of a picture (e.g., PB or CB) with a corresponding reference sample. For intra prediction coding, an intra prediction mode can specify the position of a reference sample within a picture. In ITU-T H.265, the defined possible intra prediction modes include a planar (i.e., surface conforming) prediction mode (predMode:0), a DC (i.e., flat overall averaging) prediction mode (predMode:1), and 33 angular prediction modes (predMode:2 - 34). In JEM, the defined possible intra prediction modes include a planar prediction mode (predMode:0), a DC prediction mode (predMode:1), and 65 angular prediction modes (predMode:2 - 66). Note that the planar and DC prediction modes may be referred to as non-directional prediction modes, and the angular prediction modes may be referred to as directional prediction modes. Note that the techniques described herein may be generally applicable regardless of the number of defined possible prediction modes.

[0021] For inter-prediction coding, a motion vector (MV) identifies reference samples within a picture other than the picture of the video block to be coded, thereby exploiting the temporal redundancy of the video. For example, a current video block can be predicted from reference block(s) located within a previously coded frame(s), and the motion vector can be used to indicate the position of the reference block. The motion vector and related data can describe, for example, the horizontal component of the motion vector, the vertical component of the motion vector, the resolution for the motion vector (e.g., quarter-pel accuracy, half-pel accuracy, one-pel accuracy, two-pel accuracy, four-pel accuracy), the prediction direction, and / or the index value of the reference picture. Further, for example, coding standards such as ITU-T H.265 can support motion vector prediction. Motion vector prediction enables specifying a motion vector using the motion vectors of adjacent blocks. Examples of motion vector prediction include advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), so-called "combined" modes, and "skip" and "direct" motion estimation. Further, JEM supports advanced motion vector prediction (AMVP), spatial-temporal motion vector prediction (STMVP), pattern match motion vector derivation (PMMVD) mode which is a special merge mode based on frame rate up-conversion (FRUC) technology, and affine transform motion compensation prediction technology.

[0022] As described above, quantization can be realized by dividing the transform coefficient by a magnification factor, and can be further used to change the amount of data required to represent a group of transform coefficients. That is, increasing the magnification factor (or the degree of quantization) decreases the amount of data required to represent a group of coefficients. In ITU-T H.265, the degree of quantization can be determined by the quantization parameter QP. In ITU-T H.265, for an 8-bit bit length, QP can take 52 values from 0 to 51, and a change of 1 in QP generally corresponds to a change in the value of the quantization magnification factor of about 12%. More generally, it should be noted that in ITU-T H.265, the valid range of the QP value with respect to the source bit length is as follows. -6 * (bitdepth - 8)~+51 (inclusive). Therefore, for example, when the bit length is 10 bits, QP can take 64 values from -12 to 51, which can be mapped to values from 0 to 63 during dequantization. In ITU-T H.265, the quantization parameter can be updated for each CU, and each quantization parameter can be derived for each luma component and chroma component. It should be noted that as the degree of quantization increases (e.g., the transform coefficient is divided by a larger magnification value), the amount of distortion can increase (e.g., the restored video data may look more "blocky").

[0023] In some cases, block distortion can make the coded block boundaries of the restored video data visually perceptible to the user. To reduce block distortion, the restored sample values may be modified to minimize the distortion introduced by the video encoding process. Such modifications are generally referred to as filtering. Note that filtering can occur as part of an in-loop filtering process or a post-loop filtering process. In the case of an in-loop filtering process, the sample values resulting from the filtering process may be used for a predicted video block (e.g., stored in a reference frame buffer for subsequent encoding in a video encoder and subsequent decoding in a video decoder). In a post-loop filtering process, the sample values resulting from the filtering process are simply output as part of the decoding process (e.g., not used for subsequent encoding). For example, in the case of an in-loop filtering process, the sample values obtained as a result of filtering the restored block are used for subsequent decoding (e.g., stored in a reference buffer) and output (e.g., to a display). In the case of a post-loop filtering process, the restored block without modification is used for subsequent decoding, and the sample values obtained as a result of filtering the restored block are output.

[0024] Regarding the equations used herein, the following arithmetic operators may be used. + Addition - Subtraction * Multiplication including matrix multiplication / Integer division with truncation of the result to zero. For example, 7 / 4 and -7 / -4 are truncated to 1, and -7 / 4 and 7 / -4 are truncated to -1.

Number

[0025] Furthermore, the following mathematical functions can be used.

Number

Number

[0026] Furthermore, the following definitions of logical operators may apply. x && y Boolean logical "product" of x and y x || y Boolean logical "sum" of x and y ! Boolean logical "negation" x? y : z If x is true or not equal to 0, evaluate the value of y; otherwise, evaluate the value of z.

[0027] Furthermore, the following relational operators may apply. > Greater than >= Greater than or equal to < Less than <= Less than or equal to == Equal to != Not equal to

[0028] Furthermore, the following bitwise operators may apply. x >> y Arithmetic right shift of the two's complement integer representation of x by y bits. This function is defined only for non-negative integer values of y. The bit shifted into the most significant bit (MSB) as a result of the right shift has the same value as the MSB of x before the shift operation. x << y Arithmetic left shift of the two's complement integer representation of x by y bits. This function is defined only for non-negative integer values of y. As a result of the left shift, the bit shifted into the least significant bit (LSB) has a value equal to 0.

[0029] Deblocking (or de-blocking), deblock filtering, performing deblocking, or applying a deblock filter refers to a process of smoothing the video block boundaries of adjacent restored video blocks (i.e., making the boundaries less perceptible to an observer). Smoothing the boundaries of adjacent restored video blocks can include changing the sample values included in the rows or columns adjacent to the boundary. ITU-T H.265 provides cases where a deblock filter is applied to the restored sample values as part of the in-loop filtering process. ITU-T H.265 includes two types of deblock filters that can be used to change luma samples, namely, a strong filter that changes the sample values within three rows or columns adjacent to the boundary, and a weak filter that changes the sample values within the row or column directly adjacent to the boundary and conditionally changes the sample values within the second row or column from the boundary. Further, ITU-T H.265 includes one type of filter, namely a normal filter, that can be used to change chroma samples.

[0030] Figures 5A-5B show sample values included in video blocks P and Q having a boundary. As used herein, video blocks P and Q are used to refer to adjacent video blocks having a block boundary to which deblocking may be applied. The way in which the sample values are changed can be based on a defined filter, where pi and qi represent corresponding sample values within a column for a vertical boundary and sample values within a row for a horizontal boundary, and pi' and qi' represent the changed sample values. The defined filter can define the sample to be changed (or filtered) and the samples used to determine how the sample is changed. For example, as shown in Figure 5A, in one embodiment, the sample values in each of the first three columns adjacent to the deblocking boundary (referred to as the filtered samples) may be changed based on the sample values included in each of the first four columns adjacent to the deblocking boundary (referred to as the support samples).

[0031] As described above, ITU-T H.265 includes two types of filters that can be used to change luma samples, namely, a strong filter and a weak filter. Simplified definitions of the equations for the strong and weak filters for changing luma sample values are provided below. The definitions are simplified in that they do not include the clipping operation provided in ITU-T H.265 (i.e., in ITU-T H.265, the filtered value is clipped based on the value tC described below), but Section 8.7.2.5.7 of ITU-T H.265, which provides the complete definition, is referenced. Strong filter

Equation

Equation

Equation

[0032] Furthermore, ITU-T H.265 includes one type of filter that can be used to modify chroma samples, namely, the normal filter. A simplified definition of the equation of the normal filter for modifying chroma sample values is provided below. Normal filter

Equation

[0033] Deblocking can be performed based on the deblocking granularity. ITU-T H.265 provides a deblocking granularity of 8×8. That is, in ITU-T H.265, for a portion of a picture, each edge located on an 8×8 grid is evaluated to determine whether a boundary exists. Furthermore, in ITU-T H.265, a boundary strength (Bs) is determined for each boundary. In ITU-T H.265, Bs is determined as one of 0, 1, or 2 as follows. When P and Q are two adjacent coded blocks, the filter strength Bs is defined as follows. If one of the blocks (P or Q) has an intra prediction mode, Bs = 2; Otherwise, if P and Q belong to different TBs and P or Q has at least one non-zero transform coefficient, Bs = 1, Otherwise, if the reference pictures of P and Q are not equal, Bs = 1, Otherwise, if the difference between the x or y motion vector components of P and Q is 1 integer sample or more, Bs = 1, Otherwise, Bs = 0.

[0034] In ITU-T H.265, based on the QP (QP P which may also be referred to as QPq) used to code a CB including video blocks P and Q, the variable t C’ and β’ are determined. Figure 6 shows t C ’ and β’ to provide a table for determination. In ITU-T H.265, the index Q is determined as follows. For luma, Regarding β’:

Number

Number

Number

[0035] In ITU-T H.265, the variables β and Q are derived as follows.

Number

[0036] ITU-T H.265 defines the variable d, and d is determined as follows based on the luma sample value.

Number

[0037] Furthermore, in ITU-T H.265, the variable dpq is set to a value based on the values of d and β. Finally, in ITU-T H.265, each of Bs, tC, β, and d is used to determine which filter type to apply (e.g., a strong filter or a weak filter). Furthermore, in ITU-T H.265, for the chroma component, the normal filter is applied only when Bs is equal to 2. That is, in ITU-T H.265, deblocking is performed on the chroma component only when one block P or Q is generated using the intra prediction mode.

[0038] It should be noted that it may be useful to generally describe the deblocking filter according to a set of filter parameters. For example, for a set of sample values {a...b} included in a row or column, the corresponding deblocked sample value, y[n], may be specified based on the following formula.

Equation

[0039] Furthermore, in ITU-T H.265, the deblocking filter may be applied in a different way to CTU boundaries that coincide with slice and tile boundaries compared to CTU boundaries that do not coincide with slice and tile boundaries. Specifically, ITU-T H.265 specifies a flag present in the slice segment header, namely slice_loop_filter_across_slices_enabled_flag, which enables / disables the deblocking filter across CTU boundaries that coincide with the upper slice boundary and the left slice boundary. ITU-T H.265 provides the following definition for slice_loop_filter_across_slices_enabled_flag. A slice_loop_filter_across_slices_enabled_flag equal to 1 specifies that the in-loop filtering operation may be performed across the left and upper boundaries of the current slice, and a slice_loop_filter_across_slices_enabled_flag equal to 0 specifies that the in-loop operation is not performed across the left and upper boundaries of the current slice. The in-loop filtering operation includes the deblocking filter and the sample adaptive offset filter. If slice_loop_filter_across_slices_enabled_flag does not exist, it is assumed to be equal to pps_loop_filter_across_slices_enabled_flag.

[0040] Here, when pps_loop_filter_across_slices_enabled_flag exists within the picture parameter set (PPS) and ITU-T H.265 provides the following definition for pps_loop_filter_across_slices_enabled_flag, A pps_loop_filter_across_slices_enabled_flag equal to 1 specifies that in-loop filtering operations may be performed across the left and top boundaries of the slices that reference the PPS, and a pps_loop_filter_across_slices_enabled_flag equal to 0 specifies that in-loop filtering operations are not performed across the left and top boundaries of the slices that reference the PPS. In-loop filtering operations include deblocking filter and sample adaptive offset filter operations.

[0041] NOTE - Loop filtering across slice boundaries is It can be enabled while loop filtering across tile boundaries is disabled, and vice versa. Similarly, a flag loop_filter_across_tiles_enabled_flag present in the PPS enables / disables the deblocking filter across CTU boundaries that coincide with tile boundaries. ITU-T H.265 provides the following definition for loop_filter_across_tiles_enabled_flag: loop_filter_across_tiles_enabled_flag equal to 1 specifies that in-loop filtering operations may be performed across tile boundaries in pictures that reference the PPS. loop_filter_across_tiles_enabled_flag equal to 0 specifies that in-loop filtering operations are not performed across tile boundaries in pictures that reference the PPS. In-loop filtering operations include deblocking filters and sample adaptive offset filter operations. If loop_filter_across_tiles_enabled_flag is not present, the value of loop_filter_across_tiles_enabled_flag is inferred to be equal to 1.

[0042] As described above, for deblocking, the index Q is determined based on slice_beta_offset_div2 and slice_tc_offset_div2. In ITU-T H.265, the values of slice_beta_offset_div2 and slice_tc_offset_div2 may be included in the slice segment header and have the following definitions: slice_beta_offset_div2 and slice_tc_offset_div2 specify the deblocking parameter offsets for β and t C (divided by 2). The values of slice_beta_offset_div2 and slice_tc_offset_div2 shall both be in the range of -6 to 6, including both end values. If slice_beta_offset_div2 and slice_tc_offset_div2 do not exist, the values of slice_beta_offset_div2 and slice_tc_offset_div2 are respectively presumed to be equal to pps_beta_offset_div2 and pps_tc_offset_div2.

[0043] If pps_beta_offset_div2 and pps_tc_offset_div2 exist in the PPS and ITU-T H.265 provides the following definitions for pps_beta_offset_div2 and pps_tc_offset_div2, unless the default deblocking parameter offsets are overwritten by the deblocking parameter offsets present in the slice header of the slice referring to the PPS, pps_beta_offset_div2 and pps_tc_offset_div2 apply to the slice referring to the PPS for β and t CSpecify the default deblocking parameter offset for (divided by 2). The values of pps_beta_offset_div2 and pps_tc_offset_div2 shall both be in the range of -6 to 6, including both end values. If pps_beta_offset_div2 and pps_tc_offset_div2 do not exist, the values of pps_beta_offset_div2 and pps_tc_offset_div2 are assumed to be equal to 0.

[0044] As described above, ITU-T H.265 provides a blocking granularity of 8×8. In JEM, deblocking is performed according to a grid specified by the variable minCUWidth for horizontal boundaries and the variable minCUHeight for vertical boundaries, and the default values of minCUWidth and minCUHeight are 4. The value of d is also determined in JEM, and the calculation of d is the same as that of ITU-T H.265. Based on the above value of d, a determination is made as to whether to perform deblocking on the boundary. That is, if d < β, the deblocking filter is used for the current boundary, otherwise, no deblocking is performed on the boundary. Further, in JEM, the determination of whether to use a strong filter or a weak filter is the same as that in ITU-T H.265. Finally, in the JEM reference software, the luma filter coefficient for the strong deblocking filter is the same as the coefficient used in ITU-T H.265. Performing deblocking as implemented in ITU-T H.265 and JEM may not be ideal. Specifically, deblocking as implemented in ITU-T H.265 and JEM fails to consider various coding parameters and characteristics of the restored video data when performing deblocking.

[0045] FIG. 7 is a block diagram illustrating an example of a system that can be configured to encode (and / or decode) video data according to one or more techniques of the present disclosure. System 100 represents an example of a system that can encapsulate video data in accordance with one or more techniques of the present disclosure. As shown in FIG. 7, system 100 includes a source device 102, a communication medium 110, and a destination device 120. In the example shown in FIG. 7, source device 102 can include any device configured to encode video data and transmit the encoded video data to communication medium 110. Destination device 120 can include any device configured to receive the encoded video data via communication medium 110 and decode the encoded video data. Source device 102 and / or destination device 120 can include computing devices equipped for wired and / or wireless communication and can include, for example, set-top boxes, digital video recorders, televisions, desktops, laptops, or tablet computers, gaming consoles, medical imaging devices, and mobile devices including, for example, smartphones, cellular telephones, personal gaming devices.

[0046] The communication medium 110 can include any combination of wireless and wired communication media and / or storage devices. Examples of the communication medium 110 can include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other device that may be useful for facilitating communication between various devices and sites. The communication medium 110 can include one or more networks. For example, the communication medium 110 can include a network configured to enable access to the World Wide Web, such as the Internet. The network can operate according to a combination of one or more telecommunication protocols. The telecommunication protocols can include proprietary aspects and / or standardized telecommunication protocols. Examples of standardized telecommunication protocols include Digital Video Broadcasting (DVB) standards, Advanced Television Systems Committee (ATSC) standards, Integrated Services Digital Broadcasting (ISDB) standards, Data Over Cable Service Interface Specification (DOCSIS) standards, Global System Mobile Communications (GSM) standards, code division multiple access (CDMA) standards, 3rd Generation Partnership Project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, Internet Protocol (IP) standards, Wireless Application Protocol (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards.

[0047] A memory device can include any type of device or storage medium capable of storing data. The storage medium can include a tangible or non-transitory computer-readable medium. Examples of computer-readable media can include optical disks, flash memories, magnetic memories, or any other suitable digital storage media. In some examples, a memory device or a portion thereof may be described as non-volatile memory, and in other examples, a portion of the memory device may be described as volatile memory. Examples of volatile memory can include random access memory (RAM), dynamic random access memory (DRAM), and static random access memory (SRAM). Examples of non-volatile memory can include magnetic hard disks, optical disks, floppy disks, flash memories, or forms of electrically programmable memory (EPROM) or electrically erasable and programmable memory (EEPROM). Examples of the memory device(s) can include memory cards (e.g., Secure Digital (SD) memory cards), internal / external hard disk drives, and / or internal / external solid state drives. Data can be stored on the memory device according to a defined file format.

[0048] Referring again to FIG. 7, the source device 102 includes a video source 104, a video encoder 106, a data encapsulation device 107, and an interface 108. The video source 104 can include any device configured to capture and / or store video data. For example, the video source 104 can include a video camera and a storage device operably coupled thereto. The video encoder 106 can include any device configured to receive video data and generate a compliant bitstream representing the video data. A compliant bitstream can refer to a bitstream that a video decoder can receive and then reproduce the video data from. The form of the compliant bitstream can be defined according to a video coding standard. When generating the compliant bitstream, the video encoder 106 can compress the video data. The compression can be irreversible (recognizable or unrecognizable to the viewer) or reversible.

[0049] FIG. 8 is a block diagram illustrating an example of a video encoder 200 that can implement techniques for encoding video data described herein. Although the example video encoder 200 is shown as having separate functional blocks, such illustration is for explanatory purposes and it should be noted that it does not limit the video encoder 200 and / or its sub-components to a specific hardware or software architecture. The functions of the video encoder 200 can be implemented using any combination of hardware, firmware, and / or software implementations. In one example, the video encoder 200 can be configured to encode video data according to the techniques described herein. The video encoder 200 may perform intra-prediction encoding and inter-prediction encoding of picture portions and may thus be referred to as a hybrid video encoder in some cases. In the example shown in FIG. 8, the video encoder 200 receives a source video block. In some examples, the source video block can include a portion of a picture that is divided according to an encoding structure. For example, the source video data can include macroblocks, CTUs, CBs, their subdivisions, and / or other equivalent encoding units. In some examples, the video encoder can be configured to perform additional subdivision of the source video block. It should be noted that the techniques described herein are generally applicable to video encoding regardless of how the source video data is divided before and / or during encoding. In the example shown in FIG. 9, the video encoder 200 includes an adder 202, a transform coefficient generator 204, a coefficient quantization unit 206, an inverse quantization / transformation processing unit 208, an adder 210, an intra-prediction processing unit 212, an inter-prediction processing unit 214, a filter unit 216, and an entropy encoding unit 218. As shown in FIG. 8, the video encoder 200 receives a source video block and outputs a bitstream.

[0050] In the example shown in FIG. 8, the video encoder 200 can generate residual data by subtracting a predicted video block from a source video block. The adder 202 represents a component configured to perform this subtraction operation. In one embodiment, the subtraction of video blocks is performed in the pixel domain. The transform coefficient generator 204 applies a transform such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a conceptually similar transform to the residual block or its subdivision (e.g., four 8×8 transforms can be applied to a 16×16 array of residual values), generating a set of residual transform coefficients. The transform coefficient generator 204 can be configured to perform any and all combinations of transforms included in the family of discrete trigonometric transforms. The transform coefficient generator 204 can output the transform coefficients to the coefficient quantization unit 206.

[0051] The coefficient quantization unit 206 can be configured to perform quantization of the transform coefficients. As described above, the degree of quantization can be changed by adjusting the quantization ratio, which can be determined by a quantization parameter. The coefficient quantization unit 206 can be further configured to determine quantization values, restore quantization parameters, and output QP data that can be used by a video decoder to perform inverse quantization during video decoding. For example, the QP data signaled can include QP delta values. In ITU-T H.265, the degree of quantization applied to a set of transform coefficients can depend on a slice-level parameter, a parameter inherited from a previous coded unit, and / or optionally a CU-level delta value notified.

[0052] As shown in FIG. 8, the quantized transform coefficients are output to the inverse quantization / transformation processing unit 208. The inverse quantization / transformation processing unit 208 can be configured to apply inverse quantization and / or inverse transformation to generate restored residual data. As shown in FIG. 8, in the adder 210, the restored residual data can be added to the predicted video block. In this way, the encoded video block can be restored, and the resulting restored video block can be used to evaluate the encoding quality with respect to a given quality for a given prediction, transformation type, and / or quantization level. The video encoder 200 can be configured to execute a plurality of encoding paths (e.g., execute encoding while changing one or more encoding parameters). The rate distortion of the bitstream or other system parameters can be optimized based on the evaluation of the restored video block. Further, the restored video block can be stored and used as a reference for predicting subsequent blocks.

[0053] As described above, a video block can be encoded using intra prediction. The intra prediction processing unit 212 can be configured to select an intra prediction mode for the video block to be encoded. The intra prediction processing unit 212 can be configured to evaluate a frame and / or a portion thereof to determine the intra prediction mode to be used for encoding the current block. As shown in FIG. 8, the intra prediction processing unit 212 outputs intra prediction data (e.g., syntax elements) to the filter unit 216 and the entropy encoding unit 218.

[0054] The inter prediction processing unit 214 can be configured to perform inter prediction encoding on the current video block. The inter prediction processing unit 214 can be configured to receive a source video block and calculate motion vectors for PUs of the video block or the like. The motion vector can indicate the displacement of a PU of a video block in the current video frame with respect to a prediction block in the reference frame. Inter prediction encoding can use one or more reference pictures. Further, the motion prediction can be single prediction (using one motion vector) or dual prediction (using two motion vectors). The inter prediction processing unit 214 can be configured to select a prediction block, for example, by calculating a pixel difference determined by a sum of absolute difference (SAD), a sum of square difference (SSD), or other difference measurement methods. The motion vector and related data can describe, for example, the horizontal component of the motion vector, the vertical component of the motion vector, the resolution for the motion vector (e.g., quarter-pixel accuracy), the prediction direction, and / or the index value of the reference picture. Further, for example, an encoding standard such as ITU-T H.265 can support motion vector prediction. Motion vector prediction enables specifying a motion vector using the motion vectors of adjacent blocks. Examples of motion vector prediction include advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), so-called "combined" modes, and "skip" and "direct" motion estimations. The inter prediction processing unit 214 can be configured to perform motion vector prediction according to one or more of the above-described techniques. The inter prediction processing unit 214 can be configured to generate a prediction block using the motion prediction data. For example, the inter prediction processing unit 214 can place the predicted video block in a frame buffer (not shown in FIG. 8).Note that the inter prediction processing unit 214 can be further configured to apply one or more interpolation filters to the restored residual block to calculate pixel values less than an integer for use in motion prediction. The inter prediction processing unit 214 can output motion prediction data for the calculated motion vector to the filter unit 216 and the entropy encoding unit 218.

[0055] As described above, deblocking refers to the process of smoothing the boundaries of the restored video blocks. As shown in FIG. 8, the filter unit 216 receives the restored video block and the encoding parameters (e.g., intra prediction data, inter prediction data, and QP data), and outputs the changed restored video data. The filter unit 216 can be configured to perform deblocking and / or Sample Adaptive Offset (SAO) filtering. SAO filtering is a non-linear amplitude mapping that can be used to improve restoration by adding an offset to the restored video data. Note that, as shown in FIG. 8, the intra prediction processing unit 212 and the inter prediction processing unit 214 can receive the changed restored video block via the filter unit 216. That is, in some cases, deblocking can be performed in-loop, i.e., the predicted video blocks stored in the reference buffer can be filtered. In some cases, deblocking can be performed post-loop, i.e., for example, after the video data is restored and before it is output to the display. The techniques described herein may be applicable with in-loop deblocking, post-loop deblocking, and / or combinations thereof.

[0056] As described above, it may not be ideal to perform deblocking as implemented in ITU-T H.265 and JEM. In one embodiment, according to the techniques herein, the filter section 216 may be configured to select different filtering lines based on one or more of the block size on each side (one or both) of the boundary, the strength of the boundary, the prediction mode used by the blocks on each side of the boundary, the prediction mode of the samples being deblocked (e.g., intra, inter, skip) (e.g., use a weaker filter for boundaries close to reference samples), the QP value of the samples being deblocked, the block size corresponding to the samples being deblocked, the block size corresponding to the samples used for deblocking, the motion vectors for the blocks on each side of the deblocked boundary, the motion vectors for the samples being deblocked, and / or the motion vectors for the samples used for deblocking (in some cases, the number of samples deblocked on each side of the boundary may be different).

[0057] Samples on each side of a block boundary (perpendicular to the boundary edge) can be represented as follows: ...p8p7p6p5p4p3p2p1p0|q0q1q2q3q4q5q6q7q8... Where | represents the block boundary edge. When x is a positive integer starting from 0, sample p x represents the P side of the boundary. When y is a positive integer starting from 0, sample q x represents the Q side of the boundary.

[0058] In one embodiment, the P side represents samples outside the current CU, and the Q side represents samples inside the current CU.

[0059] In one embodiment, the P side represents samples inside the current CU, and the Q side represents samples outside the current CU.

[0060] In one embodiment, the P side represents samples outside the current block, and the Q side represents samples inside the current block.

[0061] In one embodiment, the P side represents samples inside the current block, and the Q side represents samples outside the current block.

[0062] Referring to FIG. 5A, samples p y>x and q y>x correspond to line R[x] when deblocking vertical edges.

[0063] Referring to FIG. 5B, samples p y>x and q y>x correspond to line R[y] when deblocking horizontal edges.

[0064] An example of a wider (more samples are deblocked), stronger filter for the P side of the boundary is called the WS00P P-side filter and is as follows:

Equation

[0065] An example of a wider (more samples are deblocked), stronger filter for the Q side of the boundary is called the WS00Q Q-side filter and is as follows:

Equation

[0066] A narrow (i.e., fewer samples are deblocked), strong filter for the P side of the boundary is called the HEVC_P P-side filter,

Number

[0067] An example of a narrow (i.e., fewer samples are deblocked), strong filter for the Q side of the boundary is referred to as the HEVC_Q Q-side filter,

Number

[0068] An example of a narrow (fewer samples are deblocked), strong filter for the P side of the boundary is referred to as the NS00P P-side filter and is as follows:

Number

[0069] An example of a narrow (fewer samples are deblocked), strong filter for the Q side of the boundary is referred to as the NS00Q Q-side filter and is as follows:

Number

[0070] An example of a narrow (fewer samples are deblocked), strong filter for the P side of the boundary is referred to as the NS00P P-side filter and is as follows:

Number

[0071] An example of a narrow (fewer samples are unblocked), strong filter for the Q side of the boundary is called the NS00Q Q-side filter and is as follows:

Number

[0072] An example of a narrow (fewer samples are unblocked), strong filter for the P side of the boundary is called the NS00P P-side filter and is as follows:

Number

[0073] An example of a narrow (fewer samples are unblocked), strong filter for the Q side of the boundary is called the NS00Q Q-side filter and is as follows:

Number

[0074] An example of a narrow (fewer samples are unblocked), weak filter for the P side of the boundary is called the NW00P P-side filter and is as follows:

Number

[0075] An example of a narrow (fewer samples are unblocked), weak filter for the Q side of the boundary is called the NW00Q Q-side filter and is as follows:

Number

[0076] An example of a filter for the P side of the boundary is called the F0P P side filter and is as follows:

Number

[0077] An example of a filter for the Q side of the boundary is called the F0Q Q side filter and is as follows:

Number

[0078] An example of a filter for the P side of the boundary is called the F1P P side filter and is as follows:

Number

[0079] An example of a filter for the Q side of the boundary is called the F1Q Q side filter and is as follows:

Number

[0080] In one embodiment, the distance of the deblocked sample from the boundary can be inversely proportional to the distance between the support sample to which the maximum tap value is assigned and the deblocked sample. Further, the distance of the second maximum tap value from the deblocked sample can be proportional to the distance of the sample from the boundary. The filters F2P and F2Q described below provide an exemplary implementation of such filtering. An example of a filter for the P side of the boundary is called the F2P P side filter and is as follows:

Number

[0081] An example of a filter for the Q side of the boundary is referred to as the F2Q Q-side filter and is as follows:

Number

[0082] An example of a filter for the P side of the boundary is referred to as the F4P P-side filter and is as follows:

Number

[0083] An example of a filter for the Q side of the boundary is referred to as the F4Q Q-side filter and is as follows:

Number

[0084] In one embodiment, according to the technology of this specification, gradient calculation may be used for the selection of filter parameters, i.e., the selection of the number of samples to be deblocked on one (or both) sides of the block boundary. The gradient may be calculated using the samples in line R[x].

[0085] In one embodiment, multiple gradients are calculated using the samples in line R[x], and these gradients are used for the selection of filter parameters, i.e., the selection of the number of samples to be deblocked on one (or both) sides of the block boundary. In another example, multiple gradients may be calculated using the samples in line R[x], and operations such as the average of the gradients, the maximum gradient, and the minimum gradient can be used for the selection of filter parameters, the number of samples to be deblocked on one (or both) sides of the block boundary.

[0086] In one embodiment, the function call xCalDQp(R[x]) calculates the gradient as follows.

Number

[0087] In one embodiment, the function call xCalDQq(R[x]) calculates the gradient as follows.

Number

[0088] In one embodiment, the function call xCalDQpLargeBlock(R[x]) calculates the gradient as follows.

Number

[0089] In one embodiment, the function call xCalDQqLargeBlock(R[x]) calculates the gradient as follows.

Number

[0090] In one embodiment, the function call xCalDQpLargeBlock(R[x]) calculates the gradient as follows.

Number

[0091] In one embodiment, the function call xCalDQqLargeBlock(R[x]) calculates the gradient as follows.

Number

[0092] In one embodiment, the function call xCalDQpLargeBlock(R[x]) calculates the gradient as follows.

Number

[0093] In one embodiment, the function call xCalDQqLargeBlock(R[x]) calculates the gradient as follows.

Number

[0094] In one embodiment, the function call xCalDQpLargeBlock(R[x]) calculates the gradient as follows.

Number

[0095] In one embodiment, the function call xCalDQqLargeBlock(R[x]) calculates the gradient as follows.

Number

[0096] In one embodiment, the function call xCalDQpLargeBlock(R[x]) calculates the gradient as follows.

Number

[0097] In one embodiment, the function call xCalDQqLargeBlock(R[x]) calculates the gradient as follows.

Number

[0098] In one embodiment, the function call xCalDQpLargeBlock(R[x]) calculates the gradient as follows.

Number

[0099] In one embodiment, the function call xCalDQqLargeBlock(R[x]) calculates the gradient as follows.

Number

[0100] In one embodiment, XCalDQpLargeBlock(R[x]) may be calculated using a subset of the second differences calculated at p1, p2, ..., p6, where the second difference at p n is abs(p n-1 -2 * p n +p n+1 ). Similarly, xCalDQqLargeBlock(R[x]) may be calculated using a subset of the second differences calculated at q1, q2, ..., q6. In one embodiment, the rounding offset in the function calls xCalDQpLargeBlock(R[x]) and xCalDQqLargeBlock(R[x]) may be aborted. In one embodiment, the function call xUseStrongFilteringLargeBlock(R[x0, d, bSidePisLargeBlk, bSideQisLargeBlk) calculates boolean variables as follows, and examples of the determined bSidePisLargeBlk and bSideQisLargeBlk are presented below.

Table 1

[0101] In one embodiment, the function call xUseStrongFilteringLargeBlock(R[x], d, bSidePisLargeBlk, bSideQisLargeBlk) calculates boolean variables as follows.

Table 2

[0102] In one embodiment, the function call xUseStrongFilteringLargeBlock(R[x], d, bSidePisLargeBlk, bSideQisLargeBlk) calculates boolean variables as follows. [Table 3]

[0103] In one embodiment, the function call xUseStrongFilteringLargeBlock(R[x], d, bSidePisLargeBlk, bSideQisLargeBlk) calculates boolean variables as follows. [Table 4]

[0104] In one embodiment, the rounding offset in the function call xUseStrongFilteringLargeBlock may be aborted.

[0105] In one embodiment, the conditions used when selecting the number of samples to be deblocked on one (or both) sides of the boundary correspond to the size (of the current block and / or neighboring blocks) perpendicular to the block boundary that exceeds a threshold. In some cases, when a subset of the transform coefficients of a block is set to zero based on the block size, the threshold used in the comparison may be based on the characteristics of the subset of zero coefficients. For example, for coefficients at column positions of 32 or more and row positions of 32 or more (where the indices for rows and columns start from 0), when the transform coefficient is set to zero, the size in the direction perpendicular to the block boundary is compared with the threshold of 32.

[0106] In one embodiment, a signal may be received in a bitstream indicating whether all transform coefficients for a block of samples are zero. Such a signal may be received, for example, for each color component, for a group of color components, for some spatial partitioning of the samples, or for some spatio-temporal partitioning of the samples. In HEVC, for each color component, a coded block flag (CBF), i.e., cbf_luma, cbf_cb, cbf_cr, is (explicitly or implicitly by use of inference rules if no explicit signal exists) signaled, and further, a flag indicating whether any of the color components in the transform tree contains non-zero transform coefficients is also (explicitly and implicitly) signaled, which is referred to as the residual quad tree root CBF, i.e., rqt_root_cbf.

[0107] In one embodiment, the number of samples to be deblocked on one (or both) sides of a boundary may be based on the type of edge being deblocked (e.g., vertical block edge, horizontal block edge), the channel type (e.g., luma, chroma), whether all transform coefficients are zero for the block of samples on one (or both) sides of the boundary, whether the block of samples on one (or both) sides of the boundary uses an encoding mode such as local illumination compensation (LIC) that can be obtained based on a linear model for illumination change, whether the block of samples on one (or both) sides of the boundary uses inter-component prediction (which can be based on a linear model), whether the block of samples on one (or both) sides of the boundary uses a prediction determined for a block smaller than the transform, whether the block of samples on one (or both) sides of the boundary uses a technique where a large block (e.g., CU) is divided into sub-blocks (e.g., sub-CU) and motion information is derived for these sub-blocks.

[0108] In one embodiment, according to the technology described herein, when the size of the current block perpendicular to the block boundary is greater than or equal to a threshold value (e.g., 32), a larger number of samples (e.g., 7) are deblocked on each side of the block boundary, and when the dimension of the current block perpendicular to the block boundary is less than the threshold value (e.g., 32), a smaller number of samples (e.g., 3) are deblocked on each side of the block boundary. For example, if ((the width of the current block >= 32 and the edge type is vertical), or (the height of the current block >= 32 and the edge type is horizontal)), then a larger number of samples are deblocked on each side of the block boundary.

[0109] In one embodiment, the technology described herein, when the dimension of the block perpendicular to the block boundary is greater than or equal to a threshold value (e.g., 32), a larger number of samples (e.g., 7) are deblocked on each side of the block boundary, and when the dimension of the block perpendicular to the block boundary is less than the threshold value (e.g., 32), a smaller number of samples (e.g., 3) are deblocked on each side of the block boundary. For example, if ((the width of one block >= 32 and the edge type is vertical), or (the height of one block >= 32 and the edge type is horizontal)), then a larger number of samples are deblocked for that block at the block boundary. Here, if the left side of the vertical boundary edge has a block size of 4 (rows) × 64 (columns) and the right side has a block size of 4 (rows) × 16 (columns), then a larger number of samples can be deblocked on the left side than on the right side.

[0110] In one embodiment, according to the technology described herein, the filter unit 216 is based on one or more of the distance of the deblocked sample from the boundary (in terms of the number of samples), the block size on each side of the boundary, the strength of the boundary, the prediction mode used by the blocks on each side of the boundary, the prediction mode of the deblocked sample (e.g., using a weaker filter for a boundary closer to the reference sample), the QP of the deblocked sample (e.g., using a stronger filter for a larger QP), the block size corresponding to the deblocked sample (e.g., using a stronger filter for a larger block size), the block size corresponding to the sample used for deblocking, the motion vectors for the blocks on each side of the deblocked boundary (e.g., if the MV difference is greater than a threshold, no deblocking is performed because samples on different sides of the boundary may belong to different objects), and / or the motion vector for the deblocked sample, the motion vector for the sample used for deblocking, to configure the filter parameters (e.g., including the number of coefficients) used for deblocking. It should be noted that the block size corresponding to the sample may include the block size of the CU to which the sample belongs, the block size of the TU to which the sample belongs, or the block size of the PU to which the sample belongs.

[0111] In one embodiment, according to the technology of this specification, the filter section 216 is based on the type of the de-blocked edge (e.g., vertical block edge, horizontal block edge), channel type (e.g., luma, chroma), whether all the transform coefficients are zero for the block of samples on one (or both) sides of the boundary, whether the block of samples on one (or both) sides of the boundary uses an encoding mode such as local illumination compensation (LIC) that can be obtained based on a linear model for illumination change, whether the block of samples on one (or both) sides of the boundary uses inter-component prediction (which can be obtained based on a linear model), whether the block of samples on one (or both) sides of the boundary uses a prediction determined for a block smaller than the transform, whether the block of samples on one (or both) sides of the boundary uses a technique in which a large block (e.g., CU) is divided into sub-blocks (e.g., sub-CU) and motion information is derived for these sub-blocks, and may be configured to select filter parameters (e.g., including the number of coefficients) used for de-blocking based on one or more of them.

[0112] In one embodiment, according to the techniques herein, the filter section 216 may be configured to select filter parameters (e.g., including the number of coefficients) for deblocking based on one or more of the following: the type of deblocked edge (e.g., vertical block edge, horizontal block edge), the channel type (e.g., luma, chroma), whether all transform coefficients are zero for the block of samples on one (or both) sides of the boundary, whether the block of samples on one (or both) sides of the boundary uses an encoding mode such as local illumination compensation (LIC) that can be based on a linear model for illumination change, whether the block of samples on one (or both) sides of the boundary uses inter-component prediction (which can be based on a linear model), whether the block of samples on one (or both) sides of the boundary uses a prediction determined for blocks smaller than the transform, and whether the block of samples on one (or both) sides of the boundary uses a technique where a large block (e.g., CU) is divided into sub-blocks (e.g., sub-CUs) and motion information is derived for these sub-blocks.

[0113] In one embodiment, selecting the filter parameters may include selecting broader and stronger filtering.

[0114] In one embodiment, according to the techniques herein, the filter section 216 may be configured to select a set of deblocking filter parameters (e.g., stronger filtering) for both sides when the size of the current block perpendicular to the block boundary is greater than or equal to a threshold (e.g., 32). For example, if ((the width of the current block >= 32 and the edge type is vertical) or (the height of the current block >= 32 and the edge type is horizontal)), then broader and stronger filtering is selected for each side of the block boundary.

[0115] In one embodiment, according to the technology described herein, the filter unit 216 may be configured to independently select filter parameters for each side of the block boundary based on the size of the block perpendicular to the block boundary on the corresponding side. For example, when the size of the block perpendicular to the block boundary on one side is greater than or equal to a threshold value (e.g., 32), a set of deblocking filter parameters (e.g., wider and stronger filtering) can be selected for that side. For example, if ((the width of one block >= 32 and the edge type is vertical), or (the height of one block >= 32 and the edge type is horizontal)), then wider and stronger filtering is selected for the side of the block boundary corresponding to that block.

[0116] In one embodiment, according to the technology described herein, the sub-CU boundary may be deblocked based on whether all the transform coefficients for the block (e.g., CU) on one (or both) sides of the boundary are zero. For example, when all the transform coefficients for the CU are zero, the sub-CU boundary is deblocked.

[0117] In one embodiment, according to the technology described herein, the sub-block boundary may be deblocked based on whether all the transform coefficients for the block on one (or both) sides of the boundary are zero. For example, when all the transform coefficients for the block are zero, the sub-block boundary is deblocked.

[0118] In one embodiment, according to the technology described herein, the sub-CU boundary may be deblocked based on whether all the transform coefficients for the block (e.g., CU) on one (or both) sides of the boundary are zero and whether the quantization step size is large (e.g., QP is greater than or equal to a threshold value). For example, when all the transform coefficients for the CU are zero and the QP is greater than the threshold value, the sub-CU boundary is deblocked.

[0119] In one embodiment, according to the technology described herein, the sub-block boundary may be de-blocked based on whether all transform coefficients are zero for the blocks on one (or both) sides of the boundary and whether the quantization step size is large (e.g., QP is greater than or equal to the threshold). For example, if all transform coefficients for a block are zero and QP is greater than the threshold, the sub-block boundary is de-blocked.

[0120] In one embodiment, according to the technology described herein, local illumination compensation (LIC) is used for the blocks on one (or both) sides of the boundary, and if all transform coefficients for that block are zero, the block boundary may be de-blocked.

[0121] In one embodiment, according to the technology described herein, all four boundaries of a block (i.e., left, right, top, bottom) may have their block boundaries de-blocked if local illumination compensation (LIC) is used for the block and all transform coefficients for that block are zero.

[0122] In one embodiment, according to the technology described herein, local illumination compensation (LIC) is used for the blocks on one (or both) sides of the boundary, all transform coefficients for that block are zero, and if the quantization step size is large (e.g., QP is greater than or equal to the threshold), the block boundary may be de-blocked.

[0123] In one embodiment, according to the technology described herein, all four boundaries of a block (i.e., left, right, top, bottom) may have their block boundaries de-blocked if local illumination compensation (LIC) is used for the block, all transform coefficients for that block are zero, and the quantization step size is large (e.g., QP is greater than or equal to the threshold).

[0124] In one embodiment, according to the technology described herein, inter-component prediction is used for the blocks on one (or both) sides of the boundary, and if all transform coefficients for that block are zero, the block boundary may be de-blocked.

[0125] In one embodiment, according to the techniques herein, component - to - component prediction is used for blocks on one (or both) sides of the boundary, all transform coefficients for the block are zero, and when the quantization step size is large (e.g., QP is greater than or equal to a threshold), the block boundary may be deblocked.

[0126] In one embodiment, according to the technology of this specification, the filter unit 216 may be configured to perform deblocking according to a plurality of filtering paths. In one embodiment, the filtering path may correspond to the processing / construction of all / subsets of the samples to be deblocked. The number of processing / construction of each sample in a given path may correspond to the path index / order. The subset of the samples to be deblocked may correspond to the path index / order. In one embodiment, any path may correspond to the processing / construction of all the samples that are deblocked exactly once. In one embodiment in such a case, the deblocked samples (as well as the non-deblocked samples) from the previous iteration may be used to construct the deblocked samples for the current iteration. In one embodiment in such a case, the deblocked samples (as well as the non-deblocked samples) from the previous and current iterations may be used to construct the deblocked samples. In this case, an ordering for constructing the deblocked samples may be specified. In one embodiment, the number of iterations may be determined based on one or more of the slice type, block size, skip flag of the current CU and its neighboring CUs, prediction mode (Intra / inter) of the current CU and its neighboring CUs, sample position to be deblocked, whether d < β, and / or the strong or weak filter determination conditions provided in JEM as described above, the distance from the boundary of the deblocked samples (in terms of the number of samples), block sizes on each side of the boundary, strength of the boundary, prediction modes used by the blocks on each side of the boundary, prediction mode of the deblocked samples, QP of the deblocked samples, sample block size corresponding to the deblocked samples, block size corresponding to the samples used for deblocking, motion vectors for the blocks on each side of the deblocked boundary, motion vectors for the deblocked samples, and / or motion vectors for the samples used for deblocking. In one embodiment, the number of iterations may be able to determine one or more filter parameters.

[0127] In one embodiment, N - pass deblocking with a given deblocking support can be expressed as follows. [Table 5]

[0128] Where f_pos() is a linear transformation of the following form. [Equation] Coeff_pos[] is an array of values that depend on the position pos of the deblocked samples. Note that each of the sampleValue[] used can be generated using deblocking in previous iterations. Also, the deblocking support does not change from iteration to iteration. By performing an analysis for each iteration, the following is obtained. For iterldx 0, since the samples used are non - deblocked samples, for each pos, [Equation] For Iterldx 1, [Equation] Since F_pos() is a linear transformation and the deblocking support does not change, the above can be rewritten as follows. For Iterldx 1, [Equation] Where g_pos() is a linear transformation similar to f_pos() that depends on the position pos but has different coefficient values. This simplification can be performed recursively for each iteration, resulting in a filtering operation that depends only on the original sample values at iteration 0, leading to an equivalent single pass. Due to the finite precision used in some cases, the final coefficient values may be approximated, leading to an approximate single pass representation of the multi-pass filtering operation.

[0129] In one embodiment, according to the technology of this specification, the filter section 216 can be configured to extend the filter line and the corresponding filter coefficients. In one embodiment, the sample line to be filtered can be extended up to 8 on one side. In one embodiment, for lines 0 and 1, the filter coefficients may be {1,1,1,1,1,2,2,2,1,1,1,1,1} / 16, for lines 2 and 3, the filter coefficients may be {1,1,1,1,1,1,1,1} / 8, and for other lines, the filter coefficients may be {1,2,2,2,1} / 8, where line 0 indicates the sample line closest to the boundary.

[0130] Note that in JEM, the restored samples are always used to filter the samples in the deblocking process. In one embodiment, according to the technology of this specification, the filter section 216 may be configured to use the modified sample values obtained as a result of deblocking for filtering other sample values. In one embodiment, the modified sample values may be used as input when filtering other sample values. In one embodiment, the filtering order may be specified. In one embodiment, the filtering order may perform deblocking from the line farthest from the boundary to the line closest to the boundary.

[0131] As described above, the deblocked samples are typically clipped to lie within a certain range of values. The range of values may be based on the original sample value and other parameters received in the bitstream. In one embodiment, the range of values is [original sample value - t_c, original sample value + t_c]. In one embodiment, according to the techniques herein, filter section 216 may be configured to adjust the clipping function based on one or more of the sample values, QP values, slice type, current prediction mode (Intra / Inter), current skip flag value, intra prediction mode of the samples being deblocked, motion vectors of the samples being deblocked, sample position (e.g., different samples may use different clipping functions), CU position (e.g., different CUs may use different clipping functions), and / or any of the other conditions above, in the last one or more rounds of multiple-pass deblocking.

[0132] In one embodiment, according to the techniques herein, filter section 216 may be configured to perform the filtering techniques described herein based on the block size. For example, one or more of the filtering techniques described herein may be applied at the boundary where the block size around the boundary is larger than a threshold. For example, the determination of whether to perform the filtering technique may be as follows. (1) Check each set of 4×4 samples for each side of the boundary (i.e., because the minimum CU size is 4×4 luma samples); (2) If any one set of 4×4 samples belongs to a CU of a size larger than a threshold (e.g., 64), the filtering technique will be performed at the current boundary. In one embodiment, according to the techniques herein, filter section 216 may be configured to perform the filtering techniques described herein based on one or more of the slice type, whether the block shape is rectangular, the skip flags of the current CU and its neighboring CUs, the prediction mode (Intra / Inter) of the current CU and its neighboring CUs, and the sample position of the samples being deblocked.

[0133] In one embodiment, according to the techniques of this specification, the filter section 216 may be configured to perform wider-stronger luma filtering as follows. First, determine whether the P side uses large blocks as follows: [Table 6] Next, determine whether the Q side uses large blocks as follows: [Table 7] Next, derive the following variables. [Equation] Next, evaluate Condition1 and Condition2 as follows. [Table 8]

[0134] When Condition1, Condition2, and bSidePisLargeBlk are true, a wider-stronger filter is applied to the P side of the boundary (e.g., WS00P).

[0135] When Condition1, Condition2, and bSideQisLargeBlk are true, a wider-stronger filter is applied to the Q side of the boundary (e.g., WS00Q).

[0136] In one embodiment, Condition2 may be modified as follows. [Table 9]

[0137] In one embodiment, according to the techniques of this specification, the filter section 216 may be configured to perform chroma filtering as follows. If ((p0 belongs to a CU with a vertical edge type and width >= 32) || (p0 belongs to a CU with a horizontal edge type and height >= 32)) && ((q0 belongs to a CU with a vertical edge type and width >= 32) || (q0 belongs to a CU with a horizontal edge type and height >= 32)), then a narrow strong filter (e.g., NS00P and NS00Q) can be used; otherwise, a narrow weak filter (e.g., NW00P and NW00Q) can be used.

[0138] In one embodiment, according to the technology described herein, the filter unit 216 may be configured to perform chroma filtering as follows. If ((p0 belongs to a CU with width >= 32) || (p0 belongs to a CU with height >= 32)) && ((q0 belongs to a CU with width >= 32) || (q0 belongs to a CU with height >= 32)), then a narrow strong filter (e.g., NS00P and NS00Q) can be used; otherwise, a narrow weak filter (e.g., NW00P and NW00Q) can be used.

[0139] In one embodiment, according to the technology of this specification, the filter unit 216 may be configured to perform deblock filtering according to the exemplary flowchart shown in FIG. 10. In one embodiment, the filter unit 216 may be configured to perform deblock filtering on luma samples according to the flowchart shown in FIG. 10. FIG. 10 shows an example in which one of the types of deblocking, namely, a wider and stronger filter, a strong filter, a weak filter, and no filtering, may be applied to the current block (e.g., one of the P blocks or Q blocks). In one embodiment, applying a wider and stronger filter may include applying the WS00P and WS00Q filters described above. In one embodiment, applying a strong filter may include applying the HEVC_P and HEVC_Q filters described above. In one embodiment, applying a weak filter may include applying the weak filter in HEVC described above as the weak filter. As shown in FIG. 10, based on whether the large block condition is true at 402, whether the large block gradient condition is true at 404, whether the large block strong filter condition is true at 406, whether the gradient condition is true at 410, and whether the strong filter condition is true at 414, a wider filter is applied at 408, no filter is applied at 412, a strong filter is applied at 416, and a weak filter is applied at 418.

[0140] In one embodiment, the large block condition may include whether the following are true. [Table 10] where EDGE_VER is the vertical boundary type, EDGE_HOR is the horizontal boundary type, cur_block_width is, for example, the current block width in luma samples, cur_block_height is, for example, the current block height in luma samples, The adjacent_block_width is, for example, the width of adjacent blocks in luma samples, and the adjacent_block_height is, for example, the height of adjacent blocks in luma samples.

[0141] In one embodiment, the large block gradient condition may include whether the above-mentioned Condition1 is true. In one embodiment, the large block strong filter condition may include whether one of the above-exemplified Condition2 is true.

[0142] In one embodiment, the gradient condition may include whether d < β, where d is determined as follows.

Equation

[0143] In one embodiment, the strong filter condition may include whether the following is true.

Table 11

[0144] In one embodiment, the large block condition may include whether the following is true. [Table 12] where EDGE_VER is a vertical boundary type, EDGE_HOR is a horizontal boundary type, cur_block_width is the current block width in, for example, a chroma sample, cur_block_height is the current block height in, for example, a chroma sample, adjacent_block_width is the adjacent block width in, for example, a chroma sample, and adjacent_block_height is the adjacent block height in, for example, a chroma sample.

[0145] In one embodiment, the filter unit 216 may be configured to perform deblock filtering for chroma samples of a P block or a Q block based on the following set of conditions. [Table 13]

[0146] [Table 14]

[0147] where EDGE_VER is a vertical boundary type, EDGE_HOR is a horizontal boundary type, cur_Q_block_width is the current Q block width in, for example, a chroma sample, cur_Q_block_height is the current Q block height in, for example, a chroma sample, cur_P_block_width is the current P block width in, for example, a chroma sample, cur_P_block_height is, for example, the current P block height in the chroma sample, TH_w is a width threshold (e.g., 32 samples), TH_h is a height threshold (e.g., 32 samples).

[0148] Note that the threshold (e.g., TH_w and / or TH_h) may include a default value (e.g., 16 or 32) in some embodiments, may be notified in a parameter set in some embodiments, may be notified in the slice header in some embodiments, and may be the CTU size in the current part of the video in some embodiments.

[0149] Referring to FIG. 11, in one embodiment, the large block condition may be replaced by a luma filter condition. That is, for example, if a strong filter is applied to a luma block, at 504, a wider and stronger filter may be applied to the collocated chroma block, or otherwise, at 506, a weak filter may be applied to the collocated chroma block.

[0150] In one embodiment, according to the technology of this specification, the filter unit 216 may be configured to perform deblock filtering according to the exemplary flowchart shown in FIG. 12. In one embodiment, the filter unit 216 may be configured to perform deblock filtering for chroma samples according to the flowchart shown in FIG. 12. FIG. 12 shows an example in which one of the types of deblocking, i.e., a wider and stronger filter or a weaker filter, can be applied to the current block (e.g., a P block or a Q block). In one embodiment, applying a wider and stronger filter may include applying the NS00P and NSOOQ filters described above. In one embodiment, applying a weaker filter may include applying the NW00P and NW00Q filters described above. As shown in FIG. 12, based on whether the large block condition is true at 602 and whether the large block strong filter condition is true at 604, a wider and stronger filter is applied at 608, and a weaker filter is applied at 606.

[0151] In one embodiment, the large block condition may include whether the following are true.

Table 15

[0152] Note that the threshold (e.g., TH_w and / or TH_h) may include a default value (e.g., 16 or 32) in some embodiments, may be notified in a parameter set in some embodiments, may be notified in a slice header in some embodiments, and may be the CTU size in the current part of the video in some embodiments. In one embodiment, the threshold is greater than 4.

[0153] In one embodiment, a broader strong filter condition may include whether both the first condition and the second condition are true. That is, when both the first condition and the second condition are true, the strong filter condition can be true. In one embodiment, the first condition may be true when d < β, and d is determined as follows.

Equation

[0154] In one embodiment, for Rc[0] and Rc[1],

Equation

[0155] In one embodiment, for Rc[0],

Equation

[0156] In one embodiment, note that the edge is deblocked as a segment, and the segment length can be a function of the minimum size allowed for the CU / TU / PU / subPU. Further, if the 2xN and Nx2 CUs are the shortest and thinnest blocks allowed within the chroma channel, the chroma segment length may be 2.

[0157] Note that according to the above chroma filtering, two lines perpendicular to the edge to be deblocked are processed as a basic segment. In one embodiment, four lines may be processed as a basic segment. In one embodiment, when four lines are processed as a basic segment, the first condition may be true when d < β, and d is determined as follows.

Equation

[0158] Further, in one embodiment, when four lines are processed as a basic segment, for Rc[0] and Rc[3] ((abs(p3 - p0)+abs(q3 - q0)<(β>3)) && (d<(β>>2)) && (abs(q0 - p0)<((tC * 5 + 1)>>1))) the second condition may be true when it is true.

[0159] As described above, in ITU-T H.265, the deblocking filter may be applied in a different way to CTU boundaries that coincide with slice and tile boundaries compared to CTU boundaries that do not coincide with slice and tile boundaries. Specifically, in ITU-T H.265, the slice_loop_filter_across_slices_enabled_flag enables / disables the deblocking filter across CTU boundaries that coincide with the upper slice boundary and the left slice boundary. In one embodiment, according to the technology described herein, when the support samples of the deblocking filter cross a boundary (e.g., picture / slice / tile), the deblocking filter that uses the support samples may not be allowed. In one embodiment, according to the technology described herein, when the support samples of the deblocking filter cross a boundary (e.g., slice) and the use of sample values across the boundary (e.g., slice) is disabled, a padding operation may be used to generate the support sample values. For example, to generate support samples, one of numerical scalar, circular, replicate, or symmetric padding may be used. The numerical scalar padding operation pads according to a constant value, the circular padding operation pads with a circular repetition of the sample values, the replicate padding operation pads by repeating the edge sample values, and the symmetric padding operation pads with a mirror reflection of the sample values.

[0160] As described above, in ITU-T H.265, the filtered value is clipped based on the value t C Specifically, for the strong filter in the above-mentioned ITU-T H.265, the p i ’ value is clipped to (p i -2 * t C , p i +2 * t C ), and the q i ’ value is clipped to (q i -2 * t C , q i +2 * t C) is clipped. As described above, in ITU-T H.265, the variable t C ’ (therefore, the value of t C ) is determined based on the index Q determined based on qP L , which is equal to (QP q + QP P + 1) / 2. In some cases of video encoding (e.g., the proposed techniques for encoding high dynamic range (HDR) video), the value of QP may be changed at the CU level or CTU level. In this case, the range of the clipping operation provided in ITU-T H.265 based on the index Q may be insufficient. In one embodiment, according to the techniques herein, different values of t C ’ may be determined for the P-side samples and Q-side samples. That is, the P-side t C ’ value, i.e., t CP ’, and the corresponding P-side t C value, i.e., t Cp ’, are used to clip the p i ’ value, and the Q-side t C ’ value, i.e., t CQ ’, may be clipped, and the corresponding Q-side t C value, i.e., t CQ , is used to clip the q i ’ value. In one embodiment, the corresponding P-side index Q, i.e., Q p , and the Q-side index Q, i.e., Q q , may be determined by replacing qP L with the values corresponding to QP P and QP q in the above formula for the Q index. Therefore, according to the techniques herein, the p i ’ value may be clipped to (p i - 2 * t CP , p i + 2 * t CP ), and the q i ’ value may be clipped to (q i - 2 * t CQ , q i + 2* t CQ ) may be clipped to p i ’ value and q i ’ values may include filtered values generated according to any filter described herein. Therefore, the corresponding t CP and t CQ Based on p i ’ value and q i ’ The technique of clipping values may be applicable to any filter described herein.

[0161] Note that in some cases, a video block (e.g., a CU) may include an internal TU boundary, and block distortion may appear within the video block. In some cases, when the video block has a size larger than 64, the deblocking of the internal TU boundary may be disabled. In one embodiment, according to the techniques herein, even when the video block has a size larger than 64, deblocking may be performed along the video block boundary and also along any internal TU boundary.

[0162] Referring to FIGS. 5A-5B, in some cases, a P block or a Q block may include a plurality of objects. For example, referring to FIG. 5A, in one embodiment, columns p7-p3 may correspond to a first object, columns p2-p0 may correspond to a second object, and columns q0-q7 may correspond to a third object. In such a case, if the samples of columns p2-p0 are filtered using the samples of one or more columns of p3 and q0-q3 as support samples, the resulting filter sample values of columns p2-p0 may appear blurred. Further, in some cases, deblocking may result in the obscuring of dominant sample values and / or introduce other visual distortions.

[0163] As described above, the corresponding deblocked sample value, y[n] having support samples, may be specified based on the following equation.

Number

[0164] In one embodiment, according to the technology of this specification, in order to avoid blurring or distortion caused by separate objects included in the filtered sample and the support sample, one or more clipping operations may be applied to the term x[n + m]. In one embodiment, the term x[n + m] is Clip3(x[n] - 2 * t C , x[n] + 2 * t C , x[n + m]).

[0165] In one embodiment, if abs(x[n + m] - x[n]) is greater than the threshold value, the term x[n + m] may be modified such that the support sample x[n + m] is excluded from the sum. Note that when Coeff[m] corresponds to a uniform distribution (i.e., coeff[m] = 1 / filter length, and the filter length is (abs(a - b + 1))), coeff[m] is calculated as 1 / (filter length - excluded sample). When Coeff[m] corresponds to a Gaussian distribution (i.e.,

Number

[0166] In one embodiment, different filters may be applied to different sample positions with respect to the boundary. For example, samples close to the boundary may be filtered using a strong filter, and samples far from the boundary may be a filter using a weak filter. For example, the samples in columns p0 to p1 may be filtered according to a strong filter, and the samples in columns p2 to p4 may be filtered according to a weak filter. In one embodiment, for chroma deblocking (or luma deblocking), a threshold value (e.g., 2, 3, 4) may be used such that a strong filter is applied if the position distance between the current sample and the sample closest to the boundary is less than the threshold value. In one embodiment, the threshold value may be based on one or more of the block size on each side (one or both sides) of the boundary, the strength of the boundary, the prediction mode used by the blocks on each side of the boundary, the prediction mode of the sample being deblocked, the QP of the sample being deblocked, the block size corresponding to the sample being deblocked, the block size corresponding to the sample used for deblocking, the motion vector for the blocks on each side of the boundary being deblocked, the motion vector for the sample being deblocked, and / or the motion vector for the sample used for deblocking.

[0167] In some cases of video coding, it should be noted that the luminance transform coefficient corresponding to the CU (for example, after quantization) may be 0, and the CU may be divided into sub-PUs for motion compensation (for example, ATMVP). In such cases, according to the technology of this specification, luminance deblocking may be performed along the sub-PU boundary and, furthermore, in some embodiments, along the CU boundary.

[0168] In some cases of video coding, it should be noted that the chrominance transform coefficient corresponding to the CU (for example, after quantization) may be 0, and the CU may be divided into sub-PUs for motion compensation (for example, ATMVP). In such cases, according to the technology of this specification, chrominance deblocking may be performed along the sub-PU boundary and, furthermore, in some embodiments, along the CU boundary.

[0169] As described above, in ITU-T H.265, for luma, Bs,t C , β, and d are each used to determine which filter type to apply (for example, a strong filter or a weak filter). In particular, if d is less than β, the variable dStrong is determined as follows. d_strong = abs(p3 - p0) + abs(q0 - q3) Whether a strong filter is applied or a weak filter is applied is determined as follows based on the value of d_strong.

Table 16

[0170] In one embodiment, according to the technology of this specification, dStrong can be determined as follows.

Equation

[0171] If (Cur_EDGE_VER && cur_block_width==4), then parallel deblocking of the vertical boundary of the current block is not executed; If (Cur_EDGE_VER && adjacent_block_width==4), then parallel deblocking of the vertical boundary of the adjacent block is not executed; If (Cur_EDGE_HOR && cur_block_height==4), then parallel deblocking of the horizontal boundary of the current block is not executed; If (Cur_EDGE_HOR && adjacent_block_width==4), then parallel deblocking of the horizontal boundary of the adjacent block is not executed. Where Cur_EDGE_VER is the current vertical boundary, Cur_EDGE_HOR is the current horizontal boundary, cur_block_width is, for example, the current block width in luma samples, cur_block_height is, for example, the current block height in luma samples, adjacent_block_width is, for example, the adjacent block width in luma samples, and adjacent_block_height is, for example, the adjacent block height in luma samples.

[0172] In one embodiment, according to the technology described herein, for each of the above conditions, deblocking can be performed on the lum samples at the boundary as follows. If (Cur_EDGE_VER && cur_block_width == 4 && adjacent_block_width > 4), then perform deblocking only on the adjacent block samples for Cur_EDGE_VER; If (Cur_EDGE_VER && cur_block_width == 4 && adjacent_block_width == 4), then do not perform deblocking on Cur_EDGE_VER; If (Cur_EDGE_VER && cur_block_width > 4 && adjacent_block_width == 4), then perform deblocking only on the current block samples for Cur_EDGE_VER; If (Cur_EDGE_VER && cur_block_width > 4 && adjacent_block_width > 4), then perform deblocking on the current block samples and the adjacent block samples for Cur_EDGE_VER; If (Cur_EDGE_HOR && cur_block_height == 4 && adjacent_block_height > 4), then perform deblocking only on the adjacent block samples for Cur_EDGE_HOR; If (Cur_EDGE_HOR && cur_block_height == 4 && adjacent_block_height == 4), then do not perform deblocking on Cur_EDGE_HOR; If (Cur_EDGE_HOR && cur_block_height == 4 && adjacent_block_height == 4), then perform deblocking only on the current block samples for Cur_EDGE_HOR; If (Cur_EDGE_HOR && cur_block_height>4 && adjacent_block_height>4), then perform deblocking on the current block sample and the adjacent block sample for Cur_EDGE_HOR.

[0173] Similar to the above, for chroma deblocking, parallel deblocking may be restricted when each of the cur_block_height, adjacent_block_height, cur_block_width, and adjacent_block_width blocks is equal to the threshold value 2. Therefore, according to the technology of this specification, for chroma samples, deblocking may be performed as described above where the threshold value 4 is replaced with the threshold value 2 in the conditional statement.

[0174] In one embodiment, according to the technology of this specification, instead of performing deblocking on the edges of blocks having a size below the threshold value, a narrower filter may be applied to the samples on the edge. For example, when (Cur_EDGE_VER && cur_block_width==4 && adjacent_block_width>4), deblocking may be performed as follows. For Cur_EDGE_VER, perform deblocking on the adjacent block samples according to the filter width, and perform deblocking on one column of adjacent samples at Cur_EDGE_VER for the current block.

[0175] Similarly, for each of the cases described above, a narrower filter may be applied to the samples on the edge. Thus, generally, according to the techniques herein, a video encoder (and / or video decoder) may determine, for example, when parallel deblocking is restricted due to overlapping deblocking filters (e.g., having a filter width greater than half of the width (or height) of the block), and may be configured to modify the samples that would otherwise be deblocked. Note that in some cases, parallel deblocking may be restricted based on the samples used for deblocking support. According to the techniques herein, a video encoder (and / or video decoder) may be configured to determine when parallel deblocking is restricted for the samples within a block that are used for deblocking support for a plurality of deblocking filters.

[0176] As described above, for the F4PP side filter, one of the calculations includes the following.

Number

[0177] Note that if the q side is size 8 and a strong HEVC filter is used on the edge opposite the current edge, samples q5 and q6 can be changed by a strong HEVC deblocking operation for the opposite side. Parallel processing cannot occur at deblocking edges that are parallel to each other. In one embodiment, to prevent this, the p side should use a long filter (i.e., a stronger filter) only if the length of the q side (perpendicular to the edge) is 16 or more. This condition (e.g., the LargeBlk condition) is checked in one of the following ways.

Table 17

Table 18

[0178] In one embodiment, according to the technology of this specification, the set of deblocking filters may use bilinear operations. In one embodiment, the block boundary samples p i and q i are replaced by linear interpolation as follows for i = 0 to S - 1.

Equation

[0179] In one embodiment, f i , Middle s,t , P s , g i , and Q t may be determined as provided in Table 1.

Table 19

[0180] Regarding Table 1, note that for 7,5; 7,3; 5,3; 5,7; and 3,7, the weights of pi and q i in Middle are not the same and are derived from 7,7 by adding additional terms.

[0181] In one embodiment, according to the technology of this specification, the set of deblocking filters may use bilinear operations if either side is 32 or more.

[0182] In one embodiment, according to this technology, if either side is 32 or more, bilinear deblocking may be performed as provided in Table 2.

Table 20

[0183] In one embodiment, according to the present technology, when either side is 32 or more, bilinear blocking may be performed as provided in Table 3. [Table 21]

[0184] In one embodiment, according to the present technology, when either side is 32 or more, bilinear blocking may be performed as provided in Table 4. [Table 22]

[0185] In one embodiment, according to the present technology, when either side is 32 or more, bilinear blocking may be performed as provided in Table 5. [Table 23]

[0186] In one embodiment, according to the present technology, when either side is 32 or more, bilinear blocking may be performed as provided in Table 6. [Table 24]

[0187] In one embodiment, according to the present technology, when either side is 32 or more, bilinear blocking may be performed as provided in Table 7. [Table 25]

[0188] In one embodiment, according to the technology described herein, if any side is 16 or more, bilinear operations may be used. In such a case, in Tables 2 to 7, 32 may be replaced by 16. In one embodiment of Tables 5, 6, and 7, the final columns (s,t) of the rows having a P-side length not equal to the Q-side length may each use filtering of (3,3), (5,5), (3,3). In one embodiment, it may be additionally conditioned whether a strong filter condition is true with respect to whether the set of deblocking filters uses bilinear operations. For example, any of the above strong filter conditions may be mentioned. In one embodiment, the following may be additionally conditioned with respect to whether the set of deblocking filters uses bilinear operations. The variables dpq0, dpq3, dp, dq, and d are derived as follows.

Equation

Equation

Table 26

[0189] In some embodiments, the following should be noted. The control parameter values for luma and chroma (e.g., β, tC, etc.) are not the same and may be signaled using different sets of syntax elements; the control parameter values for chroma may be derived from the control parameter values for luma; deblocking may be performed only on subPU edges that align with 8×8 (luma) and 4×4 (chroma) boundaries; deblocking of the edges of the current block may be based on the use of linear model (LM) chroma; deblocking of the edges of the current block may be based on the use of separate split trees; deblocking of the edges of the current block may be based on the use of pulse code modulation (PCM); and / or deblocking of the edges of the current block may be based on the use of transform quantization bypass mode. Note that PCM is a reversible coding mode for blocks of samples. In one example of PCM coding, samples are directly represented with a predefined number of bits. The bit length used for PCM may be signaled in the parameter set.

[0190] Regarding deblocking of the edges of the current block based on the use of LM chroma, and / or deblocking of the edges of the current block may be based on the use of separate split trees. In one example, deblocking may be performed on the edges of the current block when LM chroma is used for chroma blocks, and / or when separate trees are used for luma and chroma, and / or when the received transform coefficients are zero. For one example, for separate trees, deblocking may be performed only on chroma edges (e.g., TU edges, PU edges, subPU edges, CU edges) that align with the 4×4 chroma grid.

[0191] In one example, when the large block condition is false and the strong filter condition is true, NS00P and NS00Q are used to deblock the edges.

[0192] In one embodiment, when the large block condition is false and the strong filter condition is true, the (s,t)=3,3 filter is used to deblock edges.

[0193] Note that in some cases, the deblocking boundary may include the horizontal CTU boundary. For example, referring to FIG. 5B, in some cases, the sample py,x may be included in the CTU on the CTU that includes the sample qy,x. For the purpose of encoding the upper line within the current CTU, a typical video coder implementation stores N lines of samples in the lower line of the CTU on top of the current CTU. For example, where the deblocking boundary in FIG. 5B is the CTU boundary, the video coder stores the value px,0 for performing intra prediction encoding of the line qx,0. The CTU line buffer refers to the line of sample values on top of the current CTU that are stored for encoding the current CTU. As the number of lines included in the CTU line buffer increases, the memory cost of implementing the video coder increases. Note that in some cases, data corresponding to the sample values is also stored (e.g., prediction mode (and related information, e.g., intra prediction mode, bi-pred / uni-pred, motion vector, reference index, etc.), block size, coefficient coding flag, etc.). Therefore, in order to avoid an increase in implementation cost, it is desirable to avoid increasing the number of lines included in the CTU line buffer only for the purpose of performing deblocking. For example, if all the encoding features of the proposed video coding standard require the CTU line buffer to store 4 lines of sample values, a deblocking filter that requires increasing the CTU line buffer to store 7 lines of sample values will increase the implementation cost.

[0194] As described above, JEM describes the coding characteristics under the common test model research by JVET for video coding technology that potentially improves beyond the capabilities of ITU-T H.265. Further, in response to the "Joint Call for Proposals on Video Compression with Capabilities beyond HEVC" jointly issued by VCEG and MPEG, multiple descriptions of video coding have been proposed by various groups at the 10th Meeting of ISO / IEC JTC1 / SC29 / WG11, 16 - 20 April 2018, San Diego, CA. As a result of multiple descriptions of video coding, a draft text of the video coding specification is described in "Versatile Video Coding (Draft 1)", 10th Meeting of ISO / IEC JTC1 / SC29 / WG11, 16 - 20 April 2018, San Diego, CA, document JVET-J1001-v2, which is incorporated herein by reference and referred to as JVET-J1001. "Versatile Video Coding (Draft 2)", 11th Meeting of ISO / IEC JTC1 / SC29 / WG11, 10 - 18 July 2018, Ljubljana, SI, document JVET-K1001-v4, which is incorporated herein by reference and referred to as JVET-K1001, is an updated version of JVET-J1001. The technologies proposed in each of JVET-J1001 and JVET-K1001 have been implemented and evaluated using the test model (VTM) and the benchmark set (BMS). The existing deblocking filter in the BMS changes three samples perpendicular to the edge.

[0195] CE2-related: Longer Tap Deblocking Filter,」 11th Meeting of ISO / IEC JTC1 / SC29 / WG11 10-18 July 2018, Ljubljana, SI, document JVET-K0369-r3, which is referred to as JVET-K0369 in this specification, describes a deblocking filter that modifies up to seven samples perpendicular to the deblocked edge. Further, to limit the CTU line buffer size, the filter described in JVET-K0369 restricts the filtering operation of horizontal edges that overlap the CTU boundary. Specifically, JVET-K0369 describes a deblocking filter that changes sample values according to Table 8A, and provides the locations where the deblocking filter is changed for horizontal edges that overlap the CTU boundary as shown in Table 8B.

Table 27

Table 28

[0196] Note that changing the deblocking filter provided by JVET-K0369 for horizontal edges that overlap the CTU boundary does not reduce the line buffer requirements for chroma sample values. Further, as shown in Table 8B, in addition to "padding" the filter coefficients to zero for p7~p4, the filter coefficient values are changed for p3~q7. Therefore, JVET-K0369 requires storing an additional set of filters used to filter the CTU boundary, which requires additional memory to store the coefficients.

[0197] In one embodiment, according to the technology of this specification, the use of long-tap filters that can include changing and / or having a filter support including at least three or more lines from px,0 to px,i can be restricted. In one embodiment, for luma and / or chroma deblocking, when the condition (EDGE_TYPE is EDGE_HOR && the current boundary is aligned with the CTU boundary) is satisfied, a long-tap filter is not applied to the P side. Here, EDGE_TYPE being EDGE_HOR indicates that the current boundary is a horizontal boundary. In one embodiment, for luma and / or chroma deblocking, when the following condition (EDGE_TYPE is EDGE_HOR && curPos.y % CTUSize in luma samples == 0) is satisfied, a long-tap filter is not applied to P. Here, curPos.y is the vertical luma position of the current block to be deblocked. In one embodiment, for luma and / or chroma deblocking, when the following condition (EDGE_TYPE is EDGE_HOR && curPosC.y % CTUSize in chroma samples == 0) is satisfied, a long-tap filter is not applied to P. Here, curPosC.y is the vertical chroma position of the current block to be deblocked. In one embodiment, for luma and / or chroma deblocking, when the following condition (EDGE_TYPE is EDGE_HOR && the current boundary is aligned with the CTU boundary) is satisfied, a long-tap filter is not applied to the Q side and the P side. In one embodiment, for luma and / or chroma deblocking, when the following condition (EDGE_TYPE is EDGE_HOR && curPos.y % CTUSize in luma samples == 0) is satisfied, a long-tap filter is not applied to the Q side and the P side. In one embodiment, for luma and / or chroma deblocking, when the following condition (EDGE_TYPE is EDGE_HOR && curPosC.y % CTUSize in chroma samples == 0) is satisfied, a long-tap filter is not applied to the Q side and the P side.In one embodiment, when a long tap filter is not applied, another filter is applied that modifies fewer samples and / or utilizes fewer lines from px,0 to px,i (i.e., one, two, or three lines), including filter support. For example, the weak or strong filters described herein may be applied when it is not allowed to apply a long tap filter. Note that, as provided in ITU-T H.265, the % operator provides the remainder of x divided by y.

[0198] In one embodiment, when a long tap filter is not applied, sample values exceeding a target line buffer threshold (e.g., three or four) may be made unavailable, and a predetermined value may be used for the corresponding sample positions. Table 9 shows an example where the long tap filter includes the long tap filter described above with respect to Table 8A, and the target line buffer threshold is 4. Accordingly, the sample values of p4 to p7 are unavailable. As shown in Table 9, the values of p4 to p7 are not changed for deblocking. Further, as shown in p4 to p7 of Table 9, the filter coefficients are indicated as NA, indicating that the respective sample values of p4 to p7 are not available in the line buffer. In one embodiment, for each of p4 to p7, the sample value may be set to the sample value of p3, and the filter coefficients of Table 8A may be used to derive the changed sample values for p3' to q2'. [Table 29]

[0199] Furthermore, in one embodiment, a value derived from available sample values may be used for the corresponding sample positions. In one embodiment, for each of p4 to p7, the sample value may be set to the average sample value of p3 and p2, and the filter coefficients of Table 8A may be used to derive the changed sample values for p3' to q2'.

[0200] In one embodiment, when a long tap filter is not applied, the filtering process may be changed based on the position of the deblocked samples (e.g., based on whether the sample value is on the CTU horizontal boundary or within a certain distance from the CTU horizontal boundary), and a filter that does not access / deblock samples exceeding the target line buffer threshold may be selected. For example, regarding the example shown in Table 9, a rule different from the rule for deriving the sample values for p4 to p7 may be applied to p3' and p2'.

[0201] In one embodiment, when a long tap filter is not applied, the control process may be changed based on the position of the deblocked samples, and a filter that does not access / deblock samples exceeding the target line buffer threshold may be selected. For example, the (s = 3, t = 7) filter in Table 1 for luma, the (s = 3, t = 5) filter in Table 1 for luma, F1P for luma, and / or a weak chroma filter for chroma may be selected.

[0202] In one embodiment, when a long tap filter is not applied, the deblocking grid may be changed so that samples exceeding the target line buffer threshold are not accessed / deblocked. For example, the deblocking grid may be moved so that the horizontal edge is at a distance of 4 below the horizontal CTU edge.

[0203] As described above, in ITU-T H.265, based on the QP values (sometimes referred to as QPP and QPQ) used to encode the CB containing video blocks P and Q, the variables tC' and β' are determined. The derivation of the index Q for the luma channel has been described above. For the chroma channel, in ITU-T H.265, when the chroma is 4:2:0, the variable QpC is determined as specified in the table shown in FIG. 13 based on the following index qPi.

Number

[0204] Note that in ITU-T H.265, when the chroma format is 4:2:2 or 4:4:4, QpC is set equal to Min(qPi, 51).

[0205] For chroma, t C ’ is determined using the table shown in Figure 6, and the index Q is determined as follows for t C ’.

Equation

[0206] The techniques proposed in JVET-J1001 and JVET-K1001 respectively suggest that separate splitting trees can be used to split the luma channel and the chroma channel. When separate splitting trees are used to split the luma channel and the chroma channel, it may be useful to increase up to the amount by which the QP value for the chroma channel can be varied with respect to the QP value for the luma channel. That is, for example, for each component of the chroma channel, the corresponding QP offset value, which can be signaled at the slice level, can be increased. ITU-T H.265 provides the following chroma channel QP offset syntax elements, and it should be noted that pps_cb_qp_offset and pps_cr_qp_offset specify the offsets with respect to the luma quantization parameter Qp’y used to derive QP’Cb and Qp’Cr respectively. The values of pps_cb_qp_offset and pps_cr_qp_offset shall be in the range of -12 to +12, inclusive of both end values. When ChromaArrayType is equal to 0, pps_cb_qp_offset and pps_cr_qp_offset shall not be used in the decoding process and the decoder shall ignore their values.

[0207] slice_cb_qp_offset specifies the difference added to the value of pps_cb_qp_offset when determining the value of the quantization parameter Qp’Cb. The value of slice_cb_qp_offset shall be in the range of -12 to +12, inclusive of both end values. If slice_cb_qp_offset does not exist, it is assumed to be equal to 0. The value of pps_cb_qp_offset + slice_cb_qp_offset shall be in the range of -12 to +12, inclusive of both end values.

[0208] The slice_cr_qp_offset specifies the difference to be added to the value of pps_cr_qp_offset when determining the value of quantization parameter Qp’Cr. The value of slice_cr_qp_offset shall be in the range of -12 to +12, including both end values. If slice_cr_qp_offset does not exist, it is presumed to be equal to 0. The value of pps_cr_qp_offset + slice_cr_qp_offset shall be in the range of -12 to +12, including both end values.

[0209] A change to the derivation of the chroma QP value may cause the deblocking of the chroma channel when the deblocking parameter is based on the QP value. According to the technology described herein, for example, when separate split trees may be used to split the luma channel and the chroma channel, the derivation of the QP value based on the deblocking parameter may be changed.

[0210] In one embodiment according to the technology described herein, cQpPicOffset may be derived as follows. For Cb,

Equation

Equation

Equation

Equation

[0211] In some cases, additional luma and chroma QP offset values may be used for blocks that are undergoing a certain type of processing (e.g., adaptive color conversion). These QP offsets may be used to derive the QP for luma and chroma. As a result, the deblocking process may depend on the additional luma and chroma QP offsets.

[0212] In some embodiments, when separate split trees are used to split the luma channel and the chroma channel, the chroma QP value may be calculated based on the split tree type. For example, in one embodiment, the chroma QP value may be determined as follows.

Number

[0213] In one embodiment, Qp blk_P and / or Qp bik_Q may be derived from one or more of the following combinations: the QP values of multiple corresponding luma blocks, the number of samples of the luma block corresponding to the chroma block, the luma QP value corresponding to a predetermined chroma position, etc. In some embodiments, Qp blk_P and / or Qp bik_Qcan be derived, for example, using functions such as integer averaging with rounding operations, maximum value functions, etc. Note that it is possible to have sub-luma blocks corresponding to chroma blocks. FIGS. 14A to 14B show examples of possible luma partitions corresponding to chroma blocks P and Q, and each luma block has a QP value of QP_X which is a QP value. In the example shown in FIG. 14A, chroma block P is collocated with luma blocks having QP values of QP_1 and QP_3, and chroma block Q is collocated with luma blocks having QP values of QP_2 and QP_4. In the example shown in FIG. 14B, chroma block P is collocated with luma blocks having QP values of QP_1, QP_3, and QP_5, and chroma block Q is collocated with luma blocks having QP values of QP_2 and QP_4. In one embodiment, for the example shown in FIG. 14A, Qp blk_P and Qp bik_Q can be derived as follows.

Equation

[0214] In one embodiment, for the example shown in FIG. 14B, Qp blk_P and Qp bik_Q can be derived as follows.

Equation

[0215] In one embodiment, Qp blk_P and / or Qp bik_Q can be derived by identifying a set of chroma positions and, for each chroma position, identifying the corresponding luma position. For each corresponding luma position, the corresponding QP value can be determined. The corresponding QP value can be used to derive Qp blk_P and / or Qp bik_Q .

[0216] As described above, in one embodiment, a broader strong filter condition may include whether both the first condition and the second condition are true, and the first condition may be true when d < β, where d is determined as follows. [Number] In the formula, R C [N] corresponds to the chroma line that is perpendicular to the deblocked edge and is at a distance N from the top of the current chroma segment being deblocked. In one embodiment, the filter condition may include a condition that is true when d < β, where d is determined as follows. [Number]

[0217] In some embodiments, the condition may be checked for the x sample segment (e.g., x = 2) of the chroma deblocking boundary. This reduces the number of lines in the worst case for which the gradient needs to be calculated. Note that in the worst case, the first condition calculates the gradient (xCalcDQP for every line), while the above condition calculates the gradient every two lines.

[0218] As described above, ITU-T H.265, variable β and t C are used for filtering decisions and clipping operations. For example, β and / or t C are used to determine whether a strong filter is used and / or to clip the filtered sample values. Note that in JVET-K1001, the peak signal-to-noise ratio (PSNR) is higher than that of ITU-T H.265 for a given quantization level. Therefore, in some cases, β and / or t CIt may be useful to change. That is, when the distortion level is lower at a given quantization level, the amount of perceived block noise is less, and thus less deblocking is required. In one embodiment, β may be changed such that β = β << n. In one embodiment, β may be changed such that β = β >> n. In one embodiment, t C = t C may be changed such that t = t << n. In one embodiment, t C = t C may be changed such that t = t >> n. In one embodiment, n may be determined based on one or more of the following combinations: slice type, QP value, block size, bit length, intra prediction mode, motion vector (e.g., magnitude), channel type, and / or component type, etc. For example, in one embodiment, t C = t C >> 2 may be used for intra slices, and t C = t C >> 4 may be used for inter slices. In one embodiment, t C = t C >> 2 may be used for the luma component, and t C = t C >> 4 may be used for the chroma component.

[0219] "CE2-2.1.1: Long deblocking filters and fixes," 11th Meeting of ISO / IEC JTC1 / SC29 / WG11, 10 - 18 July 2018, Ljubljana, SI, document JVET-K0307-vl, which is referred to as JVET-K0307 in this specification, describes long filters and the determination of luma components. Based on the above filtering technology, the filtering technology in JVET-K0307 may be modified to enable the use of long asymmetric filters. For long asymmetric filters, the number of samples deblocked on the larger block side is larger than that on the smaller block side. The deblocking decision process for selecting from this extended filter set is described in detail below. The extended filter set may be used for strong deblocking throughout the deblocking.

[0220] In one embodiment, according to the technology of this specification, when either side has a large block and the modified strong filter condition is satisfied, a stronger luma filter is used. In one embodiment, a large luma block corresponds to a width of 32 or more for vertical edges and a height of 32 or more for horizontal edges.

[0221] In one embodiment, a stronger luma filter can be defined as follows. Then, block boundary samples p i and q i are replaced by linear interpolation for i = 0 to S - 1 as follows.

Equation

Table 30

[0222] In one embodiment, the control process is further based on the gradients calculated for two lines of four sample segments, the comparison of the absolute pixel value differences with t C and the comparison of other absolute pixel value differences with β. For the larger block side, more gradients are calculated. The control process may be as follows. 1. Variables dpq0, dpq3, dp, dq, and d are derived as follows. First, dp0, dp3, dq0, dq3 are derived in the same way as in ITU-T H.265. Next, dpq0, dpq3, dp, dq, d are derived in the same way as in ITU-T H.265. Similar to ITU-T H.265, when d is less than 6, the following ordered steps are applied. a. dpq is derived in the same way as in ITU-T H.265. b. sp3 - Abs(p3 - p0), derived in the same way as in ITU-T H.265.

Table 31

Table 32

Table 33

Equation

Table 34

[0223] In one embodiment, the control process may be as follows. The variables dpq0, dpq3, dp, dq, and d are derived as follows. First, dp0, dp3, dq0, and dq3 are derived in the same manner as ITU-T H.265. Next, dpq0, dpq3, dp, dq, and d are derived in the same manner as ITU-T H.265. Similar to ITU-T H.265, when d is less than β, the following ordered steps are applied. dpq is derived in the same manner as ITU-T H.265. sp3 - Abs(p3 - p0) is derived in the same manner as ITU-T H.265. [Table 35] Derive in the same manner as ITU-T H.265. [Table 36] Derive the following in the same manner as ITU-T H.265. [Table 37]

[0224] Note that the conditions (p side is 32 or more && q side is 16 or more), and (q side is 32 or more && p side is 16 or more) determine whether a stronger luma filter can be applied. In other embodiments, additional conditions (e.g., one or more preceding conditions) may be applied and may be used to determine whether a stronger luma filter can be applied. In one embodiment, the additional conditions may be as follows. [Table 38] However, for (7,7,7,3,3,7), the decision process is (p side is 32 or more || q side is 32 or more). The idea is to apply either a stronger luma filter preceding the decision process with a lower threshold selection where 3*β >> 5 instead of β >> 3.

[0225] In one embodiment, according to the technology described herein, luma may be deblocked according to a 4×4 luma sample grid (or according to an 8×8 luma sample grid). In this example, the stronger luma filters described above as the WS00P P-side filter and the WS00Q Q-side filter may be used for large blocks. Here, a large block is a block with a width of 32 or more with respect to a vertical edge, a height of 32 or more with respect to a horizontal edge, and an adjacent block of 16 or more. The control process may be further based on the gradient calculated for two lines of four sample segments, the comparison of the absolute pixel value difference with t C and the comparison of other absolute pixel value differences with β. Further, when p0 belongs to the CTU on the current CTU, the following limited support luma filter provided in Table 8B above may be used.

[0226] In some cases, a subset of samples is not accessible. In such cases, the control process using this subset of samples may be affected. This may introduce asymmetry in the gradient calculation. In some embodiments, in this case, another control process may be used.

[0227] In one embodiment, the control process may be as follows. The variables dpq0, dpq3, dp, dq, and d are derived as follows. First, dp0, dp3, dq0, and dq3 are derived in the same manner as ITU-T H.265. [Table 39] Next, dpq0, dpq3, dp, dq, and d are derived in the same manner as in ITU-T H.265. Similar to ITU-T H.265, when d is less than β, the following ordered steps are applied. dpq is derived in the same manner as in ITU-T H.265. sp3 - Abs(p3 - p0) is derived in the same manner as in ITU-T H.265. [Table 40] sq3 = Abs(q0 - q3) is derived in the same manner as in ITU-T H.265. [Table 41] Similar to the derivation in ITU-T H.265, [Table 42] When StrongFilterCondition is true and LongTapDeblocking is true, a stronger filter for luma with a length perpendicular to the boundary edge of 32 or more is used. Otherwise, when StrongFilterCondition is true and LongTapDeblocking is false, another strong filter (e.g., the strong filters HEVC_P and HEVC_Q of HEVC) is used.

[0228] In one embodiment, the threshold used for comparison may also be based on the position. For example, whether the deblocked edge is aligned with the CTU boundary.

[0229] In one embodiment, for a 4×4 luma deblocking grid, one of the following may be required. When the block width / height is equal to 4 for the vertical / horizontal edge respectively, and the HEVC filter on / off conditions (i.e., d < Beta, where d = d0 + d3, and d0 = dp0 + dq0, and d3 = dp3 + dq3) are evaluated to be true for each edge, this method performs an HEVC normal / weak filter with a maximum of one sample change. Thus, the following HEVC condition, i.e., |δ| < 10(t C ) is checked, where δ = (9 << (q0 - p0) - (3 * (q1 - p1)) + 8) >> 4, and if the condition is evaluated to be true, samples p0 and q0 are changed, otherwise filtering is not applied. When the block width / height is equal to 4 for the vertical / horizontal edge respectively, a maximum of three samples are used for filter determination, and only one sample is changed by the filter. That is, in the strong / weak filter condition check, p2 i replaces p3 i , and both the strong filter and the weak filter are only permitted to change p0 and q0.

[0230] In one embodiment, according to the technology described herein, chroma may be deblocked according to a 2×2 chroma sample grid (or according to a 4×4 luma sample grid). In this example, the above-mentioned HEVC_PP side and HEVC_QQ side filters may be used. Further, when p0 belongs to the CTU on the current CTU, the weak chroma filter described above as NW00P may be used. In one embodiment, when the strong luma filter condition of HEVC calculated for chroma is true and any of the following conditions is true, a strong filter may be used. - The edge type is vertical, p0 belongs to a CU with width >= 16 (chroma samples), and q0 belongs to a CU with width >= 16 (chroma samples) - The edge type is horizontal, p0 belongs to a CU with height >= 16 (chroma samples), and q0 belongs to a CU with height >= 16 (chroma samples).

[0231] Referring back to FIG. 8, the entropy encoding unit 218 receives the quantized transform coefficients and the prediction syntax data (i.e., intra prediction data, motion prediction data). Note that in some examples, the coefficient quantization unit 206 can perform a scan of the matrix containing the quantized transform coefficients before the coefficients are output to the entropy encoding unit 218. In other examples, the entropy encoding unit 218 can perform the scan. The entropy encoding unit 218 can be configured to perform entropy encoding according to one or more of the techniques described herein. In this way, the video encoder 200 represents an example of a device that receives an array of sample values including adjacent reconstructed video blocks for components of video data and is configured to modify the sample values within the adjacent reconstructed video blocks according to a plurality of passes of a deblocking filter.

[0232] Referring back to FIG. 7, the data encapsulation unit 107 can receive the encoded video data and generate a compliant bitstream, e.g., a series of NAL units, according to a defined data structure. A device that receives the compliant bitstream can regenerate the video data therefrom. Further, a device that receives the compliant bitstream may perform a sub-bitstream extraction process, where sub-bitstream extraction refers to a process by which a device that receives the compliant bitstream forms a new compliant bitstream by discarding and / or modifying data within the received bitstream. Note that the term compliant bitstream may be used instead of the term compliant bitstream.

[0233] Referring again to FIG. 7, interface 108 may include any device configured to receive data generated by data encapsulation unit 107 and transmit and / or store that data on a communication medium. Interface 108 can include a network interface card such as an Ethernet card, and can include an optical transceiver, a radio frequency transceiver, or any other type of device capable of transmitting and / or receiving information. Further, interface 108 can include a computer system interface capable of enabling files to be stored on a storage device. For example, interface 108 can include a chipset that supports the Peripheral Component Interconnect (PCI) bus protocol and the Peripheral Component Interconnect Express (PCIe) bus protocol, its own bus protocol, the Universal Serial Bus (USB) protocol, FC, or any other logical and physical structure that can be used to interconnect peer devices.

[0234] Referring again to FIG. 7, the target device 120 includes an interface 122, a data decapsulation unit 123, a video decoder 124, and a display 126. The interface 122 can include any device configured to receive data from a communication medium. The interface 122 can include a network interface card such as an Ethernet card, and can include an optical transceiver, a radio frequency transceiver, or any other type of device capable of receiving and / or transmitting information. Further, the interface 122 can include an interface for a computer system that enables obtaining a compliant video bitstream from a storage device. For example, the interface 122 can include a chipset that supports the PCI bus protocol and the PCIe bus protocol, proprietary bus protocols, the USB protocol, FC, or any other logical and physical structure that can be used to interconnect peer devices. The data decapsulation unit 123 may be configured to receive and analyze any of the exemplary parameter sets described herein.

[0235] The video decoder 124 can include any device configured to receive a bitstream and / or an acceptable variation thereof and then reproduce video data. The display 126 can include any device configured to display video data. The display 126 can include one of various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display. The display 126 can include a high-resolution display or an ultra-high-resolution display. In the example shown in FIG. 7, the video decoder 124 is described as outputting data to the display 126, but it should be noted that the video decoder 124 can be configured to output video data to various types of devices and / or their sub-components. For example, the video decoder 124 can be configured to output video data to any communication medium as described herein.

[0236] FIG. 9 is a block diagram showing an example of a video decoder that can be configured to decode video data according to one or more techniques of the present disclosure. In one embodiment, the video decoder 300 can be configured to decode the transform data and restore the residual data from the transform coefficients based on the decoded transform data. The video decoder 300 may be configured to perform intra prediction decoding and inter prediction decoding, and for this reason, it may be referred to as a hybrid decoder. In the example shown in FIG. 9, the video decoder 300 includes an entropy decoding unit 302, an inverse quantization unit 304, an inverse transform coefficient processing unit 306, an intra prediction processing unit 308, an inter prediction processing unit 310, an adder 312, a filter unit 314, and a reference buffer 316. The video decoder 300 can be configured to decode video data to match the video encoding system. Although the example video decoder 300 is shown as having separate functional blocks, such an illustration is for explanatory purposes, and it should be noted that the video decoder 300 and / or its sub-components are not limited to a specific hardware or software architecture. The functions of the video decoder 300 can be realized using any combination of hardware, firmware, and / or software implementation forms.

[0237] As shown in FIG. 9, the entropy decoding unit 302 receives the entropy-encoded bitstream. The entropy decoding unit 302 can be configured to decode the syntax elements and the quantized coefficients from the bitstream according to a process reciprocal to the entropy encoding process. The entropy decoding unit 302 can be configured to perform entropy decoding according to any of the entropy encoding techniques described above. The entropy decoding unit 802 can determine the values of the syntax elements in the encoded bitstream to match the video encoding standard. As shown in FIG. 9, the entropy decoding unit 302 can determine the values of the quantized coefficients and the decision data from the bitstream. In the example shown in FIG. 9, the inverse quantization unit 304 receives the quantized coefficient values and outputs the transform coefficients. The inverse transform processing unit 306 receives the transform coefficients and outputs the restored residual data.

[0238] Referring again to FIG. 9, the restored residual data can be provided to the adder 312. The adder 312 can add the restored residual data to the predicted video block to generate the restored video data. The predicted video block can be determined according to prediction video techniques (i.e., intra prediction and inter prediction). The intra prediction processing unit 308 can be configured to receive intra prediction syntax elements and obtain the predicted video block from the reference buffer 316. The reference buffer 316 can include a memory device configured to store one or more frames of video data. The intra prediction syntax elements can identify intra prediction modes such as the intra prediction mode described above. The inter prediction processing unit 308 can receive inter prediction syntax elements, generate motion vectors, and identify predicted blocks within one or more reference frames stored in the reference buffer 316. The inter prediction processing unit 310 can, in some cases, perform interpolation based on an interpolation filter to generate a motion-compensated block. The syntax elements can include identifiers of interpolation filters that will be used for motion prediction with sub-pixel accuracy. The inter prediction processing unit 310 can use the interpolation filter to calculate the interpolated values for sub-integer pixels of the reference block.

[0239] The filter unit 314 can be configured to perform filtering on the restored video data. For example, the filter unit 314 can be configured to perform deblocking and / or sample adaptive offset (SAO) filtering based on parameters specified in the bitstream, for example. Further, it should be noted that in some examples, the filter unit 314 can be configured to perform its own arbitrary filtering (e.g., visual enhancement such as mosquito noise reduction). The filter unit 314 can operate in the same manner as the filter unit 216. As shown in FIG. 9, the restored video block can be output by the video decoder 300. In this way, the video decoder 300 can receive an array of sample values including adjacent restored video blocks for the components of the video data, and can be configured to change the sample values in the adjacent restored video blocks according to multiple passes of the deblocking filter.

[0240] In one or more examples, the described functions can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the function can be stored or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium can include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium or a communication medium, including any medium that facilitates transfer of a computer program from one place to another according to a communication protocol. In this way, the computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or a carrier wave. The data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for the implementation of the techniques described in this disclosure. A computer program product can include a computer-readable medium.

[0241] By way of example, and without limitation, such computer-readable storage media can include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage devices, magnetic disk storage devices, other magnetic storage devices, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and that is accessible by a computer. Also, any connection can be properly termed a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that the computer-readable media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead are directed to non-transitory tangible storage media. As used in the present invention, disk and disc include Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, and Blu-ray (registered trademark) disc, where disk typically magnetically reproduces data and disc optically reproduces data using a laser. The above combinations should also be included within the scope of computer-readable media.

[0242] The commands can be executed by one or more processors such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, as used herein, the term "processor" can refer to either the foregoing structures, or any other structure suitable for implementation of the techniques described herein. Additionally, in some aspects, the functions described herein can be provided within dedicated hardware modules and / or software modules configured to encode and decode, or incorporated into a composite codec. Also, this technology can be implemented entirely within one or more circuits or logic elements.

[0243] The techniques of the present disclosure can be implemented in a variety of devices or apparatuses including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chip sets). Various components, modules, or units are described in the present disclosure to emphasize the functional aspects of devices configured to execute the disclosed techniques, but implementation by different hardware units is not necessarily required. Rather, as described above, the various units can be combined with codec hardware units, or provided by a collection of interoperating hardware units including one or more of the foregoing processors, along with suitable software and / or firmware.

[0244] Furthermore, each functional block and various functions of the base station apparatus and the terminal apparatus used in each of the above-described implementation forms can generally be realized or executed by an integrated circuit or an electric circuit that is a plurality of integrated circuits. A circuit designed to execute the functions described in this specification may include a general-purpose processor, a digital signal processor (DSP), an application-specific or general-purpose application integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gates or transistor logic, or individual hardware components, or a combination thereof. The general-purpose processor may be a microprocessor, or alternatively, the processor may be a conventional processor, controller, microcontroller, or state machine. The above-described general-purpose processor or each circuit may be composed of a digital circuit or an analog circuit. Furthermore, if an integrated circuit technology that replaces the current integrated circuit appears due to the progress of semiconductor technology, an integrated circuit using this technology can also be used.

[0245] Various embodiments have been described. These and other embodiments are within the scope of the following claims.

[0246] <Cross-reference> This non-provisional application claims priority under 35 U.S.C. § 119 to U.S. Provisional Patent Application No. 62 / 651,058, filed Mar. 30, 2018; U.S. Provisional Patent Application No. 62 / 654,379, filed Apr. 7, 2018; U.S. Provisional Patent Application No. 62 / 655,029, filed Apr. 9, 2018; U.S. Provisional Patent Application No. 62 / 656,291, filed Apr. 11, 2018; U.S. Provisional Patent Application No. 62 / 677,629, filed May 29, 2018; U.S. Provisional Patent Application No. 62 / 679,716, filed Jun. 1, 2018; U.S. Provisional Patent Application No. 62 / 696,309, filed Jul. 10, 2018; U.S. Provisional Patent Application No. 62 / 711,420, filed Jul. 27, 2018; U.S. Provisional Patent Application No. 62 / 714,755, filed Aug. 5, 2018; U.S. Provisional Patent Application No. 62 / 732,556, filed Sep. 17, 2018; U.S. Provisional Patent Application No. 62 / 733,067, filed Sep. 18, 2018; U.S. Provisional Patent Application No. 62 / 735,090, filed Sep. 22, 2018; U.S. Provisional Patent Application No. 62 / 737,596, filed Sep. 27, 2018, the contents of which are hereby incorporated by reference herein.

Claims

1. An apparatus comprising: a processor; and a memory storing instructions that cause the apparatus to process a method for filtering restored video data by the processor. Receiving values of samples included in a first block and a second block adjacent to a vertical block boundary of the restored video data. Determining whether the first block is a large block based on whether the width of the first block is 32 or more. Determining whether the second block is a large block based on whether the width of the second block is 32 or more. Calculating a first gradient value for the first block. Calculating a second gradient value for the second block. Calculating a variable beta based on a quantization parameter. When only one of the first block or the second block is the large block, and the width of the one block is 32 or more and the width of the other block is equal to 16, setting (3 * beta >> 5) to a threshold value. When neither the first block nor the second block is the large block, setting (beta >> 3) to the threshold value. Filtering the values of the samples included in the first block and the second block using a long filter based on whether the sum of the first gradient value and the second gradient value is less than the threshold value. The apparatus is characterized by this.

2. A method for filtering restored video data, comprising: Receiving values of samples included in a first block and a second block adjacent to a vertical block boundary of the restored video data. Determining whether the first block is a large block based on whether the width of the first block is 32 or more. Determine whether the second block is a large block based on whether the width of the second block is 32 or more, calculate a first gradient value for the first block, calculate a second gradient value for the second block, calculate variable beta based on a quantization parameter, when only one of the first block or the second block is the large block, and the width of that one block is 32 or more and the width of the other block is equal to 16, set (3 * beta >> 5) to a threshold value, when neither the first block nor the second block is the large block, set (beta >> 3) to the threshold value, filter the values of the samples included in the first block and the second block using a long filter based on whether the sum of the first gradient value and the second gradient value is less than the threshold value. A method characterized by this.

Images