Image processing device, image processing method, program, and recording medium

Abstract

To provide various DFs.SOLUTION: A reduced second luminance filter obtained by reducing filter properties of a second luminance filter having higher filter strength than a first luminance filter, or a reduced first luminance filter obtained by reducing filter properties of the first luminance filter is used as a second color difference filter having higher filter strength than a first color difference filter with respect to a pixel of a color difference component located in the vicinity of a block boundary of a decoded image. This technology is applicable to, for example, coding and decoding of an image.SELECTED DRAWING: Figure 16

Description

Technical Field

[0001] The present technology relates to an image processing apparatus, an image processing method, a program, and a recording medium, and particularly relates to an image processing apparatus, an image processing method, a program, and a recording medium that can provide various DFs (deblocking filters), for example.

Background Art

[0002] In H.265 / HEVC, which is one of the standard specifications of the image encoding method, a deblocking filter is applied to the block boundaries of the decoded image in order to suppress the deterioration of image quality caused by block distortion that occurs during encoding. In H.265 / HEVC, there are two types of deblocking filters that can be applied to the luminance component, namely a weak filter and a strong filter, while there is only one type of weak filter for the chrominance component.

[0003] Also, currently, for the purpose of further improving the encoding efficiency compared to H.265 / HEVC, the standardization work of VVC (Versatile Video Coding), which is the next-generation image encoding method, is being promoted by JVET (Joint Video Experts Team), a joint standardization organization of ITU-T and ISO / IEC (see, for example, Non-Patent Document 1).

[0004] In the VVC standardization work, in Non-Patent Document 1 below, a method has been proposed in which the deblocking filter that can be applied to the chrominance component is changed to two types, similar to the deblocking filter that can be applied to the luminance component, and a strong filter can also be applied to the chrominance component.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Regarding DF, there is a demand to provide various DFs.

[0007] This technology has been made in view of such a situation, and enables the provision of various DFs.

Means for Solving the Problems

[0008] The image processing apparatus of this technology includes a decoding unit that decodes an encoded stream to generate a decoded image, and a third color difference filter whose filter design is different from that of a first color difference filter applied to pixels of a color difference component located near a block boundary of the decoded image generated by the decoding unit, and is a filter with a stronger filter intensity than the first color difference filter, and is applied to pixels of a color difference component located near a block boundary inside a CTU (Coding Tree Unit), and a filter unit that applies the third color difference filter to pixels of a color difference component located near the block boundary of the CTU.

[0009] The image processing method of this technology includes: decoding an encoded stream to generate a decoded image; and applying a third chromatic difference filter to pixels of chromatic difference components located near the block boundaries of the CTU (Coding Tree Unit), which has a stronger filter strength than the first chromatic difference filter applied to pixels of chromatic difference components located near the block boundaries of the decoded image, and which has a different filter design from the second chromatic difference filter applied to pixels of chromatic difference components located near the block boundaries of the CTU.

[0010] The program for this technology is a program for executing a process that includes decoding an encoded stream to generate a decoded image, and applying a third color difference filter to pixels of color difference components located near the block boundaries of the CTU (Coding Tree Unit). This third color difference filter has a stronger filter strength than the first color difference filter applied to pixels of color difference components located near the block boundaries of the decoded image, and its filter design differs from the second color difference filter applied to pixels of color difference components located near the block boundaries within the CTU.

[0011] The recording medium of this technology is a recording medium on which a program is recorded for executing a process that includes decoding an encoded stream to generate a decoded image, and applying a third color difference filter to pixels of color difference components located near the block boundaries of the CTU (Coding Tree Unit). This third color difference filter has a stronger filter strength than the first color difference filter applied to pixels of color difference components located near the block boundaries of the decoded image, and its filter design differs from the second color difference filter applied to pixels of color difference components located near the block boundaries of the CTU.

[0012] In the image processing apparatus, image processing method, program, and recording medium of this technology, an encoded stream is decoded to generate a decoded image. A third chromatic difference filter is applied to the pixels of the chromatic difference components located near the block boundaries of the CTU (Coding Tree Unit), and has a stronger filter strength than the first chromatic difference filter applied to the pixels of the chromatic difference components located near the block boundaries of the CTU. The third chromatic difference filter has a different filter design from the second chromatic difference filter applied to the pixels of the chromatic difference components located near the block boundaries of the CTU.

[0013] Furthermore, an image processing device can be realized by having a computer execute a program. The program can be provided by recording it on a recording medium or by transmitting it via a transmission medium. [Brief explanation of the drawing]

[0014] [Figure 1] This table explains how to calculate the bS in HEVC. [Figure 2] This table explains the calculation of bS in Non-Patent Document 1. [Figure 3] This is an explanatory diagram showing an example of pixels for the color difference components (U component, V component) within two adjacent blocks Bp and Bq, separated by a vertical block boundary BB. [Figure 4] This is a table illustrating the calculation of bS in one embodiment of the present disclosure. [Figure 5] This is a block diagram showing an example of the configuration of an image encoding device 10, which is one aspect of the image processing device according to the same embodiment. [Figure 6] This is a block diagram showing an example of the configuration of an image decoding device 60, which is one aspect of the image processing apparatus according to the same embodiment. [Figure 7] This is a block diagram showing an example of a detailed configuration of the deblocking filter 26 according to the same embodiment. [Figure 8] This table shows an example of bS calculated by the boundary strength calculation unit 261. [Figure 9] This flowchart shows an example of the processing flow by the deblocking filter 26 according to the same embodiment. [Figure 10] This is a flowchart illustrating the flow of the boundary strength calculation process performed by the boundary strength calculation unit 261. [Figure 11] This is a block diagram showing an example of the DF300 configuration as a new defensive line. [Figure 12] This figure shows an example of the structure of a decoded image processed by the DF300. [Figure 13] This is a flowchart explaining the process of the DF300. [Figure 14] This is a diagram explaining the DF of HEVC. [Figure 15] This is a diagram explaining the new defender. [Figure 16] This figure shows an example of a pixel with a color difference component at a block boundary. [Figure 17] This diagram shows filter NC1 and the required pixels when a filter based on filter Y1 is adopted as filter NC1. [Figure 18] This diagram shows the filter NC1 and the required pixels when a filter based on filter OF is adopted as filter NC1. [Figure 19] This diagram shows filter NC1 and the required pixels when a filter based on filter Y2 is adopted as filter NC1. [Figure 20] This figure shows an example of how to apply filter NC1 to a decoded image. [Figure 21] This figure shows an example of how to apply filter NC1 to a decoded image. [Figure 22] This is a block diagram showing an example configuration of one embodiment of a computer. [Modes for carrying out the invention]

[0015] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, thus omitting redundant explanations.

[0016] Furthermore, the scope disclosed herein is not limited to the contents of the examples, and the contents of the following references REF1 to REF3, which were publicly known at the time of filing, are also incorporated herein by reference. In other words, the contents described in the following references REF1 to REF3 also serve as the basis for determining the support requirement. For example, even if the Quad-Tree Block Structure described in reference REF2 and the QTBT (Quad Tree Plus Binary Tree) Block Structure described in reference REF3 are not directly defined in the detailed description of the invention, they are within the scope of this disclosure and satisfy the support requirement of the claims. Similarly, even if technical terms such as Parsing, Syntax, and Semantics are not directly defined in the detailed description of the invention, they are within the scope of this disclosure and satisfy the support requirement of the claims. REF1:Recommendation ITU-T H.264 (04 / 2017) “Advanced video coding for generic audiovisual services”, April 2017 REF2:Recommendation ITU-T H.265,(12 / 2016) “High efficiency video coding”, December 2016 REF3: J. Chen, E. Alshina, GJ Sullivan, J.-R. Ohm, J. Boyce, “Algorithm Description of Joint Exploration Test Model (JEM7)”, JVET-G1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 7th Meeting: Torino, IT, 13-21 July 2017

[0017] Furthermore, unless otherwise specified, the following explanations use a YUV420 format signal as an example, where the luminance component is represented as the Y component and the chrominance components as the U and V components. However, the techniques described below can also be applied to signals in other formats such as YUV444 and YUV422. In addition, the representation of the luminance and chrominance components differs depending on the signal being addressed; for example, the techniques described below can also be applied to signals where the luminance and chrominance components are represented as YCbCr.

[0018] Furthermore, the following terms used herein are defined as follows: Color difference-related parameters refer to all parameters related to color difference. For example, color difference-related parameters may include information about the conversion coefficients of color difference components, such as the conversion coefficients of the color difference components contained in each TU (Transform Unit), or flags indicating the presence or absence of significance coefficients (non-zero conversion coefficients) for the color difference components in each TU. However, color difference-related parameters are not limited to these examples and can be a variety of parameters related to color difference. The question of whether or not a deblocking filter should be applied means whether or not a deblocking filter should be applied. For example, determining whether or not a deblocking filter should be applied means determining whether or not a deblocking filter should be applied. Furthermore, the result of determining whether or not a deblocking filter should be applied is the result of determining whether or not a deblocking filter should be applied, and for example, this result may indicate either that it should be applied or that it does not need to be applied. Large block determination refers to the determination of whether or not a block being evaluated is a large block. In this specification, the block being evaluated may be a block that straddles a block boundary, as will be described later. Large block determination can be performed by comparing the size of the block with a predetermined threshold. The cases in which large block determination is performed and the details of large block determination will be described later.

[0019] <1. Overview> [1-1. Existing Methods]

[0020] The deblocking filter process in existing image encoding schemes such as HEVC includes a process to determine whether or not to apply the filter, a process to determine the filter strength, and a filtering process (filter application process). Below, the process related to existing deblocking filters will be explained using the HEVC deblocking filter as an example. In the following, the deblocking filter for the color difference component of the decoded image (including images locally decoded during encoding) will be mainly explained, and the explanation of the deblocking filter for the luminance component of the decoded image will be omitted as appropriate.

[0021] The first step in applying a deblocking filter is to determine whether or not it should be applied. This determination determines whether or not the deblocking filter should be applied to the block boundaries of the decoded image. In HEVC, block boundaries are identified based on the Quad-Tree Block Structure described in reference REF2. Specifically, among the edges of the smallest block unit, an 8x8 pixel block (sample grid), edges that satisfy the condition that they are at least one of either a TU (Transform Unit) boundary or a PU (Prediction Unit) boundary are identified as block boundaries in HEVC.

[0022] The determination of whether or not the application is necessary is performed based on the boundary strength (bS) of the block boundary. In HEVC, bS is calculated for every four lines of the identified block boundary. If the block boundary is a vertical boundary, the above lines correspond to rows perpendicular to the vertical boundary. If the block boundary is a horizontal boundary, the above lines correspond to columns perpendicular to the horizontal boundary.

[0023] Figure 1 is a table explaining the calculation of bS in HEVC. As shown in Figure 1, in HEVC, bS is calculated based on the truth value (whether it is met or not) of condition A, which is a condition related to intra prediction; condition B1, which is a condition related to the significance coefficient of the Y component; and condition B2, which is a condition related to the motion vector (MV) and the reference picture. Referring to Figure 1, bS is set to 2 when condition A is true. Also, if condition A is false and at least one of condition B1 or condition B2 is true, bS is set to 1. And if condition A, condition B1, and condition B2 are all false, bS is set to 0. The conditions A, B1, and B2 shown in Figure 1 are as follows.

[0024] -Condition A: Of the Coding Units (CUs) that include the pixel of the uppermost line among the lines to be calculated for bS and straddle the block boundary, at least one of the coding modes is the intra-prediction mode. -Condition B1: The block boundary is a TU boundary, and the significance coefficient of the Y component exists in at least one of the two TUs that enclose the block boundary and include the pixel of the uppermost line among the lines for which bS is calculated. -Condition B2: Between two CUs that include the pixel of the uppermost line among the lines to be calculated for bS and straddle the block boundary, the absolute difference in MV is 1 pixel or more, or the reference pictures for motion compensation are different, or the number of MVs is different.

[0025] Furthermore, in HEVC, a deblocking filter may be applied to the luminance component (Y component) of the decoded image for block boundaries where the bS set as described above is 1 or greater. Therefore, in HEVC, the determination of whether or not a deblocking filter is required for the luminance component of the decoded image may differ depending on whether or not conditions B1 and B2 are met.

[0026] In HEVC, two types of deblocking filters are available for the luminance component of the decoded image: a strong filter with high filter intensity and a weak filter with low filter intensity. If bS is 1 or greater, the processing of the deblocking filter for the luminance component of the decoded image involves further determination of whether or not to apply the filter based on additional conditions, followed by determination of the filter intensity and filtering. Details of these processes are described in the above-mentioned reference REF2, and will not be explained here.

[0027] On the other hand, in HEVC, the deblocking filter applied to the chromatic difference components (U and V components) of the decoded image is applied only to block boundaries where bS is 2. Therefore, as shown in Figure 1, whether or not conditions B1 and B2 are met does not affect the determination of whether or not to apply the deblocking filter to the chromatic difference components of the decoded image in HEVC.

[0028] Furthermore, in HEVC, the only deblocking filter that can be applied to the chrominance component of the decoded image is the weak filter. Therefore, no filter strength determination process is necessary for the chrominance component of the decoded image; if bS is 2, the weak filter is applied to the chrominance component of the decoded image.

[0029] Incidentally, as described in reference REF3 above, in VVC, block partitioning using the QTBT Block Structure can select larger blocks than in HEVC, which uses the Quad-Tree Block Structure. When the block size is large in flat regions (regions where the change in pixel values ​​within the region is small), block distortion is likely to occur. Therefore, in VVC, where larger blocks can be selected, if only a weak filter is used as the deblocking filter applied to the chrominance component of the decoded image, as in HEVC, there is a risk that significant block distortion will remain in the chrominance component. In light of this situation, it is desirable to improve the deblocking filter applied to the chrominance component of the decoded image.

[0030] For example, Non-Patent Document 1 proposes a method in which a strong filter can be applied to the color difference component by changing the deblocking filter that can be applied to the color difference component to two types, similar to the deblocking filter that can be applied to the luminance component. Furthermore, Non-Patent Document 1 states that a deblocking filter can be applied to the color difference component of the decoded image not only when bS is 2, but also when bS is 1.

[0031] Figure 2 is a table illustrating the calculation of bS in Non-Patent Document 1. As shown in Figure 2, in Non-Patent Document 1, bS is calculated based on the above-mentioned conditions A, B1, and B2, similar to the HEVC example shown in Figure 2. However, as mentioned above, in Non-Patent Document 1, a deblocking filter can be applied to the chromatic difference component of the decoded image not only when bS is 2, but also when bS is 1. Therefore, as shown in Figure 2, in Non-Patent Document 1, the determination of whether or not a deblocking filter is necessary for the chromatic difference component (U component, V component) of the decoded image may differ depending on whether or not conditions B1 and B2 are met.

[0032] The following describes the deblocking filter applicable to the color difference component of a decoded image in Non-Patent Document 1, including the process for determining whether the filter is applicable, the process for determining the filter strength, and the filtering process, with reference to Figure 3. Figure 3 is an explanatory diagram showing an example of pixels of the color difference component (U component, V component) within two adjacent blocks Bp and Bq separated by a vertical block boundary BB. Although a vertical boundary is used as an example here, the matters described here are naturally applicable to horizontal boundaries as well. Furthermore, Figure 3 shows an example where blocks Bp and Bq are 4x4 in the color difference component, but the matters described here are similarly applicable to blocks of other sizes.

[0033] In the example in Figure 3, the pixels of the color difference component within block Bp are p i,j This is represented by the symbol i, where i is the column index and j is the row index. The column index i is numbered 0, 1, 2, 3 in order from the column closest to the block boundary BB (from left to right in the diagram). The row index j is numbered 0, 1, 2, 3 from top to bottom. On the other hand, the pixels of the color difference component within block Bq are q k,j This is represented by the symbol k, where k is the column index and j is the row index. The column index k is numbered 0, 1, 2, 3 in order from the column closest to the block boundary BB (from right to left in the diagram).

[0034] As explained with reference to Figure 2, after bS is calculated, the following three conditions are used to determine whether the filter should be applied and to determine the filter strength. In the case of the YUV420 format, this process is performed for every two lines in the color difference component. For example, in the example shown in Figure 3, the determination for lines L11 and L12 is performed separately from the determination for lines L21 and L22. Note that the determination for each line is performed using the pixels of the line to be determined. Below, the process of determining whether the filter should be applied, determining the filter strength, and filtering will be explained using lines L11 and L12 as examples.

[0035] First, in the applicability determination process, it is determined in order whether the following conditions C91 and condition C92 are true.

[0036] - Condition C91 : (bS == 2 || bS == 1 && (block_width > 16 && block_height > 16)) - Condition C92: d < beta

[0037] Note that in the above condition C91, block_width and block_height are, as shown in FIG. 3, the horizontal size and vertical size of the block (for example, CU) applicable to the block boundary to be determined, respectively.

[0038] Also, the variable beta in the above condition C92 is an edge determination threshold value, and the initial value of the variable beta is given according to the quantization parameter. Also, the value of the variable beta can be specified by the user with the parameter in the slice header. Also, the variable d in the above condition C92 is calculated by the following formulas (1) to (7).

[0039] dp0 = Abs(p 2,0 - 2 * p 1,0 + p 0,0 ) …(1) dp1 = Abs(p 2,1 - 2 * p 1,1 + p 0,1 ) …(2) dq0 = Abs(q 2,0 - 2 * q 1,0 + q 0,0 ) …(3) dq1 = Abs(q 2,1 - 2 * q 1,1 + q 0,1 ) …(4) dpq0 = dp0 + dq0 …(5) dpq1 = dp1 + dq1 …(6) d = dpq0 + dpq1 …(7)

[0040] Note that condition C92 described above is the same as the condition used in the process of determining whether or not to apply the deblocking filter applied to the luminance component in HEVC (hereinafter referred to as the condition for the luminance component), except that the referenced lines are different. In the condition for the luminance component, the pixels of the first line and the pixels of the fourth line were referenced, and the determination was made every four lines. On the other hand, in the YUV420 format, the pixel density of the chromatic difference components (U component, V component) is half that of the luminance component, so in condition C92 described above, the pixels of line L11, which is the first line, and the pixels of line L12, which is the second line, are referenced, and the determination is made every two lines.

[0041] If at least one of the above conditions C91 or C92 is false, the deblocking filter is not applied to the color difference component of the decoded image. On the other hand, if both conditions C91 and C92 are true, the process proceeds to the filter strength determination process.

[0042] In the filter strength determination process, it is determined whether the following condition C93 is true in order to decide which filter, a strong filter or a weak filter, to apply.

[0043] -Condition C93:(block_width>16&&block_height>16)

[0044] Note that in condition C93 above, block_width and block_height are the horizontal and vertical dimensions of the block that spans the block boundary being evaluated, similar to block_width and block_height in condition C91.

[0045] If condition C93 is true, a strong filter is applied to the color difference component of the decoded image at the target block boundary; if condition C93 is false, a weak filter is applied to the color difference component of the decoded image at the target block boundary.

[0046] The strong filter applied to the chromatic difference component in Non-Patent Literature 1 is similar to the strong filter applied to the luminance component in HEVC, and is expressed as shown in the following equations (8) to (13).

[0047] p0' = Clip3(p0-2*tc, p0+2*t C ,(p2+2*p1+2*p0+2*q0+q1+4)>>3) …(8) p1' = Clip3(p1-2*tc, p1+2*t) C (p2+p1+p0+q0+2)>>2) …(9) p2' = Clip3(p2-2*tc, p2+2*t) C ,(2*p3+3*p2+p1+p0+q0+4)>>3) …(10) q0' = Clip3(q0 - 2*tc, q0 + 2*t C ,(p1+2p0+2q0+2q1+q2+4)>>3) …(11) q1' = Clip3(q1-2*tc, q1+2*t C (p0+q0+q1+q2+2)>>2) …(12) q2' = Clip3(q2 - 2*t c q² + 2*t C ,(p0+q0+q1+3*q2+2*q3+4)>>3) …(13)

[0048] Note that in equations (8) to (13) above, p i , and q k is the pixel value of the color difference component before applying the deblocking filter. Also, p i ', and q k ′ represents the pixel value of the chromatic difference component after applying the deblocking filter. Here, i and k are the column indices within block Bp and block Bq, respectively, and the row indices are omitted in equations (8) to (13). Also, t C These are parameters given according to the quantization parameters. Also, Clip3(a,b,c) represents a clipping operation that clips the value c within the range a≦c≦b.

[0049] The weak filter applied to the chromatic difference component in Non-Patent Document 1 is the same as the weak filter applied to the chromatic difference component in HEVC, so its explanation is omitted here.

[0050] The above describes the processing of a deblocking filter that can be applied to the color difference component of a decoded image, as described in Non-Patent Document 1. According to the method described above, it is possible to apply a strong filter not only to the luminance component but also to the color difference component, depending on the conditions.

[0051] However, as explained with reference to Figure 2, the condition B1 used to calculate bS in Non-Patent Literature 1 depends on the presence or absence of a significance coefficient for the luminance component (Y component), similar to the case of HEVC, and even with other conditions included, information on the chromatic difference components (U component, V component) is not used. However, the spatial patterns of the luminance component and the spatial patterns of each chromatic difference component do not necessarily coincide. Therefore, if the determination of whether or not to apply a deblocking filter to the chromatic difference component is made according to conditions based on the information of the luminance component, there is a risk that the deblocking filter will not be applied appropriately even if block distortion occurs, and block distortion may remain.

[0052] Furthermore, if bS is 1, for condition C91 used in the application necessity determination process in Non-Patent Document 1 to be true, both the horizontal and vertical sizes of the block that is subject to determination must be greater than 16. However, as described in reference document REF3, the shape of a block (e.g., CU) in VVC can be a non-square rectangle, not just a square. Block distortion tends to occur more easily depending on the size in the direction perpendicular to the block boundary than on the size in the same direction as the block boundary. Therefore, depending on the shape of the block, the application necessity determination process in Non-Patent Document 1 may not properly apply the deblocking filter, and block distortion may remain.

[0053] Furthermore, the strong filter described in Non-Patent Document 1 is the same as the strong filter applied in HEVC. On the other hand, as mentioned above, in VVC, blocks of a larger size may be selected than those used in HEVC, so even if the strong filter described in Non-Patent Document 1 is applied, there is a risk that block distortion may not be sufficiently reduced.

[0054] [1-2. Overview of one embodiment of this disclosure] Therefore, with the above circumstances as a starting point, we have created one embodiment of the present disclosure. The image processing device according to one embodiment of the present disclosure performs an application necessity determination process to determine whether or not to apply a deblocking filter to the color difference component of the decoded image, based on the boundary intensity (bS) calculated using color difference-related parameters related to the color difference of the decoded image. The outline of one embodiment of the present disclosure will be described below.

[0055] Figure 4 is a table illustrating the calculation of bS in this embodiment. As shown in Figure 4, bS is calculated based on condition A, which is a condition related to intra prediction; condition B1-Y, which is a condition related to the significance coefficient of the Y component; condition B1-U, which is a condition related to the significance coefficient of the U component; condition B1-V, which is a condition related to the significance coefficient of the V component; and condition B2, which is a condition related to MV and the reference picture.

[0056] Referring to Figure 4, bS is set to 16 when condition A is true. Also, if condition A is false and condition B2 is true, bS is set to 1. Furthermore, if both condition A and condition B2 are false, and at least one of conditions B1-Y, B1-U, and B1-V is true, bS is set to a value between 2 and 14. Finally, if all of conditions A, B1-Y, B1-U, B1-V, and B2 are false, bS is set to 0. Note that conditions A, B1-Y, and B2 shown in Figure 4 are the same as conditions A, B1, and B2 explained with reference to Figure 1, respectively. The method for calculating bS in this embodiment will be explained in more detail later.

[0057] Furthermore, conditions B1-U and B1-V shown in Figure 4 correspond to conditions that use the presence or absence of a significance coefficient for the U component and the V component, respectively, as the basis for determination, instead of the presence or absence of a significance coefficient for the Y component in condition B1-Y, and are expressed as follows. Note that the truth value of the following conditions B1-U and B1-V can be determined based on a flag (an example of a color difference-related parameter) indicating the presence or absence of a significance coefficient for the color difference component in each TU.

[0058] -Condition B1-U: The block boundary is a TU boundary, and the significance coefficient of the U component exists in at least one of the two TUs that enclose the block boundary and include the pixel of the uppermost line among the lines for which bS is calculated. -Condition B1-V: The block boundary is a TU boundary, and the significance coefficient of the V component exists in at least one of the two TUs that enclose the block boundary and include the pixel of the uppermost line among the lines for which bS is calculated.

[0059] In this embodiment, the necessity of applying a deblocking filter to the color difference component of the decoded image is determined based on bS calculated using the above-described color difference-related conditions B1-U and B1-V. This configuration makes it possible to apply the deblocking filter more appropriately to the color difference component.

[0060] Furthermore, in this embodiment, as will be described later, the determination of whether or not to apply a deblocking filter to the color difference component of the decoded image is made based on the size in the direction perpendicular to the block boundary. With this configuration, it becomes possible to apply the deblocking filter more appropriately even when the shape of the block is a non-square rectangle.

[0061] Furthermore, in this embodiment, as will be described later, a strong filter with greater intensity (stronger low-pass characteristics) than the strong filter in Non-Patent Document 1 may be applied to the color difference component of the decoded image. In addition, in order to apply such a strong filter more appropriately, in this embodiment the filter intensity is determined by a different method than the filter intensity determination process in Non-Patent Document 1. With this configuration, block distortion can be further reduced.

[0062] The above describes an overview of one embodiment of the present disclosure. Below, the configuration and operation of this embodiment, which achieve the effects described above, will be described in detail.

[0063] <2. Outline of the device configuration> First, using Figures 5 and 6, we will describe the schematic configuration of an apparatus as an example to which the technology disclosed herein can be applied. The technology disclosed herein is applicable, for example, to an image coding apparatus and an image decoding apparatus.

[0064] [2-1. Image encoding device] Figure 5 is a block diagram showing an example of the configuration of an image encoding device 10, which is one embodiment of an image processing device according to one embodiment of the present disclosure.

[0065] Referring to Figure 5, the image coding device 10 includes a sorting buffer 11, a control unit 12, a subtraction unit 13, an orthogonal transform unit 14, a quantization unit 15, a reversible coding unit 16, a storage buffer 17, an inverse quantization unit 21, an inverse orthogonal transform unit 22, an addition unit 23, an in-loop filter 24, a frame memory 30, a switch 31, a mode setting unit 32, an intra-prediction unit 40, and an inter-prediction unit 50.

[0066] The sorting buffer 11 sorts a series of images to be encoded (original images) according to the GOP (Group of Pictures) structure used in the encoding process. The sorting buffer 11 outputs the sorted images to the control unit 12, subtraction unit 13, intra prediction unit 40, and inter prediction unit 50.

[0067] The control unit 12 divides the image into processing unit blocks based on an external or pre-specified block size. The block division by the control unit 12 may form CUs (Units of Composition) of a Quad-Tree Block Structure or a QTBT (Quad Tree Plus Binary Tree) Block Structure as processing units. The control unit 12 also determines the parameters for the encoding process, for example, based on RDO (Rate-Distortion Optimization). The determined parameters are then supplied to each component.

[0068] The subtraction unit 13 calculates the prediction error, which is the difference between the image input from the sorting buffer 11 and the predicted image, and outputs the calculated prediction error to the orthogonal transformation unit 14.

[0069] The orthogonal transformation unit 14 performs orthogonal transformation processing for each of the one or more transformation blocks (TUs) set within each domain. The orthogonal transformation here may be, for example, a discrete cosine transform or a discrete sine transform. More specifically, the orthogonal transformation unit 14 converts the prediction error input from the subtraction unit 13 into a frequency domain transformation coefficient for each transformation block, from a spatial domain image signal. The orthogonal transformation unit 14 then outputs the transformation coefficient to the quantization unit 15.

[0070] Furthermore, the orthogonal transformation unit 14 may generate a flag indicating the presence or absence of a significance coefficient in each TU (for each Y component, U component, and V component) based on the transformation coefficients obtained by the orthogonal transformation, and output it to the reversible encoding unit 16 and the in-loop filter 24. The flags indicating the presence or absence of a significance coefficient for the U component and the V component in each TU, generated by the orthogonal transformation unit 14, are included in the color difference related parameters.

[0071] The quantization unit 15 is supplied with conversion coefficients input from the orthogonal transformation unit 14 and a rate control signal from the rate control unit 18, which will be described later. The quantization unit 15 quantizes the conversion coefficients and outputs the quantized conversion coefficients (hereinafter also called quantized data) to the reversible coding unit 16 and the inverse quantization unit 21. The quantization unit 15 also changes the bit rate of the quantized data input to the reversible coding unit 16 by switching the quantization scale based on the rate control signal from the rate control unit 18.

[0072] The reversible encoding unit 16 generates an encoded stream by encoding the quantized data input from the quantization unit 15. The reversible encoding unit 16 also encodes various parameters referenced by the decoder and inserts these encoded parameters into the encoded stream. The parameters encoded by the reversible encoding unit 16 may include the parameters determined by the control unit 12 described above.

[0073] Furthermore, the parameters encoded by the reversible encoding unit 16 may include color difference-related parameters. The color difference-related parameters encoded by the reversible encoding unit 16 include, for example, a flag indicating the presence or absence of a significance coefficient for the U component in each TU input from the orthogonal transformation unit 14, as described above, and a flag indicating the presence or absence of a significance coefficient for the V component in each TU. The reversible encoding unit 16 outputs the generated encoded stream to the storage buffer 17.

[0074] The storage buffer 17 temporarily stores the encoded stream input from the reversible encoding unit 16 using a storage medium such as semiconductor memory. The storage buffer 17 then outputs the stored encoded stream to a transmission unit (not shown) (for example, a communication interface or a connection interface to peripheral devices) at a rate corresponding to the bandwidth of the transmission path.

[0075] The rate control unit 18 monitors the available capacity of the storage buffer 17. The rate control unit 18 generates a rate control signal according to the available capacity of the storage buffer 17 and outputs the generated rate control signal to the quantization unit 15. For example, when the available capacity of the storage buffer 17 is low, the rate control unit 18 generates a rate control signal to reduce the bit rate of the quantized data. Also, for example, when the available capacity of the storage buffer 17 is sufficiently large, the rate control unit 18 generates a rate control signal to increase the bit rate of the quantized data.

[0076] The inverse quantization unit 21, the inverse orthogonal transformation unit 22, and the summing unit 23 constitute a local decoder. The local decoder is responsible for locally decoding the decoded image from the encoded data.

[0077] The inverse quantization unit 21 inversely quantizes the quantized data using the same quantization parameters as those used by the quantization unit 15, and restores the conversion coefficients. The inverse quantization unit 21 then outputs the restored conversion coefficients to the inverse orthogonal transformation unit 22.

[0078] The inverse orthogonal transform unit 22 restores the prediction error by performing an inverse orthogonal transform process on the transformation coefficients input from the inverse quantization unit 21. The inverse orthogonal transform unit 22 then outputs the restored prediction error to the summing unit 23.

[0079] The summing unit 23 generates a decoded image (reconstructed image) by adding the restored prediction error input from the inverse orthogonal transform unit 22 and the predicted image input from the intra prediction unit 40 or the inter-prediction unit 50. The summing unit 23 then outputs the generated decoded image to the in-loop filter 24 and the frame memory 30.

[0080] The in-loop filter 24 applies a series of in-loop filters to improve the image quality of the decoded image. For example, as described in "2.5. In-loop filtering" of reference REF3, the four in-loop filters may be applied in the order of a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter. The in-loop filter 24 shown in Figure 5 includes, for example, a bilateral filter 25, a deblocking filter 26a, an adaptive offset filter 27, and an adaptive loop filter 28, and the above four in-loop filters may be applied in order. However, the in-loop filter 24 is not limited to this configuration, and it may be possible to appropriately select which of the four in-loop filters to apply and in what order they are applied. The deblocking filter 26a will be described in detail later.

[0081] The in-loop filter 24 outputs the decoded image to the frame memory 30 after the in-loop filter has been applied.

[0082] The frame memory 30 stores the decoded image before filtering, which is input from the adder 23, and the decoded image to which the in-loop filter has been applied, which is input from the in-loop filter 24, using a storage medium.

[0083] Switch 31 reads the decoded image before filtering, which is used for intra prediction, from the frame memory 30 and supplies the read decoded image to the intra prediction unit 40 as a reference image. Switch 31 also reads the decoded image after filtering, which is used for inter prediction, from the frame memory 30 and supplies the read decoded image to the inter prediction unit 50 as a reference image.

[0084] The mode setting unit 32 sets a prediction coding mode for each block based on a comparison of costs input from the intra prediction unit 40 and the inter prediction unit 50. For blocks in which the intra prediction mode is set, the mode setting unit 32 outputs the prediction image generated by the intra prediction unit 40 to the subtraction unit 13 and the addition unit 23, and outputs information related to the intra prediction to the reversible coding unit 16. For blocks in which the inter prediction mode is set, the mode setting unit 32 outputs the prediction image generated by the inter prediction unit 50 to the subtraction unit 13 and the addition unit 23, and outputs information related to the inter prediction to the reversible coding unit 16.

[0085] The intra-prediction unit 40 performs intra-prediction processing based on the original image and the decoded image. For example, the intra-prediction unit 40 evaluates the prediction error and the cost based on the amount of code generated for each of the prediction mode candidates included in the search range. Next, the intra-prediction unit 40 selects the prediction mode with the minimum cost as the optimal prediction mode. The intra-prediction unit 40 also generates a predicted image according to the selected optimal prediction mode. Then, the intra-prediction unit 40 outputs information related to intra-prediction, including prediction mode information indicating the optimal prediction mode, the corresponding cost, and the predicted image to the mode setting unit 32.

[0086] The interpretation unit 50 performs interpretation processing (motion compensation) based on the original image and the decoded image. For example, the interpretation unit 50 evaluates the prediction error and the cost based on the amount of code generated for each of the prediction mode candidates included in a certain search range. Next, the interpretation unit 50 selects the prediction mode with the lowest cost, i.e., the prediction mode with the highest compression ratio, as the optimal prediction mode. The interpretation unit 50 also generates a predicted image according to the selected optimal prediction mode. Finally, the interpretation unit 50 outputs information regarding interpretation, the corresponding cost, and the predicted image to the mode setting unit 32.

[0087] [2-2. Image Decoding Device] Next, the decoding of the encoded data described above will be explained. Figure 6 is a block diagram showing an example of the configuration of an image decoding device 60, which is one aspect of the image processing apparatus according to this embodiment. Referring to Figure 6, the device comprises a storage buffer 61, a reversible decoding unit 62, an inverse quantization unit 63, an inverse orthogonal transform unit 64, an adder 65, an in-loop filter 66, a reordering buffer 72, a D / A (Digital to Analogue) conversion unit 73, a frame memory 80, selectors 81a and 81b, an intra-prediction unit 90, and an inter-prediction unit 100.

[0088] The storage buffer 61 temporarily stores the encoded stream received from the image encoding device 10 via a transmission unit (not shown, for example, a communication interface or a connection interface with peripheral devices) using a storage medium.

[0089] The reversible decoding unit 62 decodes the encoded stream input from the storage buffer 61 according to the encoding scheme used during encoding, and generates quantized data. The reversible decoding unit 62 outputs the generated quantized data to the inverse quantization unit 63.

[0090] Furthermore, the reversible decoding unit 62 parses various parameters from the encoded stream. The parameters parsed by the reversible decoding unit 62 may include, for example, information regarding intra-prediction and information regarding inter-prediction. The reversible decoding unit 62 outputs the information regarding intra-prediction to the intra-prediction unit 90. The reversible decoding unit 62 also outputs the information regarding inter-prediction to the inter-prediction unit 100.

[0091] Furthermore, the parameters parsed by the reversible decoding unit 62 may include color difference-related parameters. The reversible decoding unit 62 outputs the color difference-related parameters to the in-loop filter 66. The color difference-related parameters parsed by the reversible decoding unit 62 include, for example, flags indicating the presence or absence of a significance coefficient for the U component in each TU, and flags indicating the presence or absence of a significance coefficient for the V component in each TU.

[0092] The inverse quantization unit 63 inversely quantizes the quantized data input from the reversible decoding unit 62 using the same quantization steps as used during encoding, and restores the conversion coefficients. The inverse quantization unit 63 outputs the restored conversion coefficients to the inverse orthogonal transformation unit 64.

[0093] The inverse orthogonal transform unit 64 generates a prediction error by performing an inverse orthogonal transform on the conversion coefficients input from the inverse quantization unit 63, according to the orthogonal transform scheme used during encoding. The inverse orthogonal transform unit 64 outputs the generated prediction error to the adder unit 65.

[0094] The summing unit 65 generates a decoded image by adding the prediction error input from the inverse orthogonal transform unit 64 and the predicted image input from the selector 71b. The summing unit 65 then outputs the generated decoded image to the in-loop filter 66 and the frame memory 80.

[0095] The in-loop filter 66 applies a series of in-loop filters to improve the image quality of the decoded image. For example, as described in "2.5. In-loop filtering" of reference REF3, the four in-loop filters may be applied in the order of a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter. The in-loop filter 66 shown in Figure 6 includes, for example, a bilateral filter 67, a deblocking filter 26b, an adaptive offset filter 69, and an adaptive loop filter 70, and the above four in-loop filters may be applied in order. However, the in-loop filter 66 is not limited to this configuration, and it may be possible to appropriately select which of the four in-loop filters to apply and in what order. The deblocking filter 26b will be described in detail later.

[0096] The in-loop filter 66 outputs the decoded image to which the in-loop filter has been applied to the sorting buffer 72 and the frame memory 80.

[0097] The reordering buffer 72 generates a series of images in time by rearranging the images input from the in-loop filter 66. The reordering buffer 72 then outputs the generated images to the D / A conversion unit 73.

[0098] The D / A conversion unit 73 converts the digital image input from the sorting buffer 72 into an analog image signal. The D / A conversion unit 73 then displays the image by outputting the analog image signal to a display (not shown) connected to, for example, the image decoding device 60.

[0099] The frame memory 80 stores the decoded image before filtering, which is input from the adder 65, and the decoded image to which the in-loop filter has been applied, which is input from the in-loop filter 66, using a storage medium.

[0100] The selector 81a switches the output destination of the image from the frame memory 80 between the intra-prediction unit 90 and the inter-prediction unit 100 for each block in the image, according to the prediction mode information acquired by the reversible decoding unit 62. For example, if the intra-prediction mode is specified, the selector 81a outputs the decoded image before filtering supplied from the frame memory 80 to the intra-prediction unit 90 as a reference image. If the inter-prediction mode is specified, the selector 81a outputs the decoded image after filtering to the inter-prediction unit 100 as a reference image.

[0101] The selector 81b switches the source of the predicted image to be supplied to the adder 65 between the intra-prediction unit 90 and the inter-prediction unit 100, according to the prediction mode information acquired by the reversible decoding unit 62. For example, if the intra-prediction mode is specified, the selector 81b supplies the predicted image output from the intra-prediction unit 90 to the adder 65. If the inter-prediction mode is specified, the selector 81b supplies the predicted image output from the inter-prediction unit 100 to the adder 65.

[0102] The intra-prediction unit 90 performs intra-prediction processing based on the intra-prediction information input from the reversible decoding unit 62 and the reference image from the frame memory 80, and generates a predicted image. The intra-prediction unit 90 then outputs the generated predicted image to the selector 81b.

[0103] The interpretation unit 100 performs interpretation processing based on interpretation information input from the reversible decoding unit 62 and a reference image from the frame memory 80, and generates a predicted image. The interpretation unit 100 then outputs the generated predicted image to the selector 81b.

[0104] <3. Deblocking Filter> [3-1. Example of a deblocking filter configuration] This section describes an example of the configuration of the deblocking filter 26a of the image encoding device 10 shown in Figure 5 and the deblocking filter 26b of the image decoding device 60 shown in Figure 6. Note that the configurations of the deblocking filters 26a and 26b may be the same. Therefore, in the following description, unless there is a need to distinguish between the two, the deblocking filters 26a and 26b will be collectively referred to as the deblocking filter 26.

[0105] As described above, the deblocking filter 26 according to this embodiment determines whether or not to apply a deblocking filter to the color difference component of the decoded image based on bS calculated using color difference-related parameters related to color difference. Furthermore, as described above, the deblocking filter 26 according to this embodiment determines whether or not to apply a deblocking filter to the color difference component of the decoded image based on the size in the direction orthogonal to the block boundary. Furthermore, as described above, the deblocking filter 26 according to this embodiment can apply a strong filter with a greater intensity (stronger low-pass characteristics) than the strong filter in Non-Patent Literature 1 to the color difference component of the decoded image. In addition, in order to apply such a strong filter more appropriately, in this embodiment the filter intensity is determined by a method different from the filter intensity determination process in Non-Patent Literature 1. In the following, the functions of the deblocking filter 26 relating to the deblocking filter applied to the color difference component of the decoded image will be mainly described, and the functions of the deblocking filter 26 relating to the deblocking filter applied to the luminance component will be omitted as appropriate.

[0106] Figure 7 is a block diagram showing an example of a detailed configuration of the deblocking filter 26 according to this embodiment. Referring to Figure 7, the deblocking filter 26 includes a boundary intensity calculation unit 261, a determination unit 263, and a filtering unit 269.

[0107] (1) Boundary strength calculation unit The boundary intensity calculation unit 261 calculates bS (boundary intensity) using color difference-related parameters that relate to color difference, targeting the block boundaries of the decoded image. When the target signal is in YUV420 format, the boundary intensity calculation unit 261 calculates bS in units of 4 lines in the luminance component of the decoded image, i.e., in units of 2 lines in the color difference component of the decoded image.

[0108] In this embodiment, the color difference-related parameters used by the boundary intensity calculation unit 261 to calculate bS include a flag indicating the presence or absence of a significance coefficient for the U component in each TU, and a flag indicating the presence or absence of a significance coefficient for the V component in each TU. As shown in Figure 7, the boundary intensity calculation unit 261 receives flags indicating the presence or absence of significance coefficients for each component (Y component, U component, V component) in each TU from the orthogonal transformation unit 14 or the reversible decoding unit 62.

[0109] The boundary intensity calculation unit 261 calculates bS based on conditions A, B1-Y, B1-U, B1-V, and B2, as described with reference to Figure 4. In other words, the boundary intensity calculation unit 261 calculates bS based on whether or not a significant coefficient of the color difference component exists in the TU that straddles the block boundary for which bS is to be calculated. Furthermore, the boundary intensity calculation unit 261 according to this embodiment can calculate bS by independently determining whether or not a significant coefficient of each component of the Y, U, and V exists in the TU that straddles the block boundary for which bS is to be calculated. With this configuration, a bS more suitable for the U and V components can be calculated than when bS is calculated based on whether or not a significant coefficient of the Y component exists, as described with reference to Figure 2, and it becomes possible to apply the deblocking filter more appropriately.

[0110] Referring to Figure 8, the calculation of bS by the boundary strength calculation unit 261 will be explained in more detail. Figure 8 is a table showing an example of bS calculated by the boundary strength calculation unit 261. The bS calculated by the boundary strength calculation unit 261 can be represented by multiple bits. In the example shown in Figure 8, bS is represented by 5 bits. In addition, the bS may be calculated such that each of the multiple bits includes at least one bit corresponding to the Y component, U component, and V component. With this configuration, when the determination unit 263, which will be described later, determines whether or not to apply a deblocking filter based on bS, it can easily make a determination by referring to the corresponding bits of bS for each component to be determined.

[0111] Furthermore, the boundary strength calculation unit 261 may calculate bS such that each bit in bS corresponds to the truth value of each condition. In the example shown in Figure 8, bS is calculated such that if each condition is true, the bit corresponding to that condition is 1, and if each condition is false, the bit corresponding to that condition is 0. Also, in the example shown in Figure 8, bS is represented by 5 bits, with the 5th bit of bS corresponding to condition A regarding intra prediction, the 4th bit of bS corresponding to condition B1-Y regarding the significance coefficient of the Y component, the 3rd bit of bS corresponding to condition B1-U regarding the significance coefficient of the U component, the 2nd bit of bS corresponding to condition B1-V regarding the significance coefficient of the V component, and the 1st bit of bS corresponding to condition B2 regarding MV and reference picture. However, the correspondence between each bit of bS and each condition is not limited to the example shown in Figure 8. For example, the order of the 4th, 3rd, and 2nd bits of bS corresponding to the Y, U, and V components may be changed.

[0112] (2) Judgment section As shown in Figure 7, the determination unit 263 includes an application necessity determination unit 265 that determines whether or not a deblocking filter is necessary to apply to the color difference component of the decoded image, and a filter intensity determination unit 267 that determines the filter intensity of the deblocking filter applied to the color difference component of the decoded image. The functions of the application necessity determination unit 265 and the filter intensity determination unit 267 will be described in order below.

[0113] In the following explanation, we will mainly describe the determination of whether or not to apply a deblocking filter to the color difference component of the decoded image, and the determination of the filter strength, while the determination for the luminance component will be omitted as appropriate. Furthermore, the application necessity determination unit 265 and the filter strength determination unit 267 in this embodiment determine whether or not to apply a deblocking filter and the filter strength independently for the U component and the V component, respectively.

[0114] The application necessity determination unit 265 targets the block boundaries of the decoded image and determines whether or not to apply a deblocking filter to the color difference component of the decoded image based on the bS (boundary intensity) calculated by the boundary intensity calculation unit 261 as described above.

[0115] Furthermore, the application necessity determination unit 265 may determine whether or not to apply the deblocking filter to the color difference component of the decoded image based on the block size of the blocks flanking the block boundary. In the following, such a determination based on block size may be referred to as a large block determination. Also, the application necessity determination unit 265 is not required to always perform a large block determination for all block boundaries, and may decide whether or not to perform a large block determination depending on bS. The cases in which a large block determination is performed and the details of the large block determination will be described later.

[0116] The application necessity determination unit 265 in this embodiment determines whether or not the deblocking filter needs to be applied based on the determination of the following conditions C1 and C2.

[0117] -Condition C1:(bS==16||(Condition C11&&Condition C12)) -Condition C2:d <beta

[0118] In the above condition C1, condition C11 is a condition for determining whether or not to perform a large block determination, and condition C12 is a condition related to the large block determination. When bS is 16, that is, when condition A related to intra prediction is met, condition C1 can be determined to be true without the need to perform a large block determination. Therefore, condition C11 for determining whether or not to perform a large block determination can be true when bS has a value related to intra prediction. In this way, by skipping the large block determination and determining condition C1 to be true when bS is 16, the amount of processing required for the large block determination can be reduced.

[0119] Furthermore, if condition C11 of condition C1 is false, the evaluation of condition C12 (large block evaluation) is not performed, and condition C1 is determined to be false. This configuration makes it possible to reduce the amount of processing required for large block evaluation.

[0120] Condition C11 may be true if the conditions regarding the significance coefficient of each component, or the above-mentioned condition B2, are true. In other words, condition C11 may differ depending on the component being evaluated. For example, if the component being evaluated is U, condition C11 may be a condition like condition C11-U below, and if the component being evaluated is V, condition C11 may be a condition like condition C11-V below.

[0121] -Condition C11-U:(bS&0x04||bS&0x01) -Condition C11-V:(bS&0x02||bS&0x01)

[0122] Furthermore, the application necessity determination unit 265 performs a large block determination based on the size of the blocks enclosing the block boundary in the direction perpendicular to the block boundary. With this configuration, when the shape of the block is a non-square rectangle, it becomes possible to determine whether or not to apply the deblocking filter based on the size in the direction perpendicular to the block boundary, which is likely to affect the occurrence of block distortion.

[0123] Furthermore, the application necessity determination unit 265 may perform a large block determination based on whether the size of the blocks enclosing the block boundary in the direction perpendicular to the block boundary is greater than a predetermined threshold. The threshold used in this large block determination is not limited, but may be, for example, 16. When the size in the direction perpendicular to the block boundary is small, and especially when it is 16 or less, block noise is less noticeable, and with this configuration, it is possible to avoid applying unnecessary deblocking filters. For example, the condition C12 for large block determination may be the following condition.

[0124] -Condition C12:(EDGE_VER&&block_width>16)||(EDGE_HOR&&block_height>16)

[0125] In the above condition C12, EDGE_VER means that the block boundary being evaluated is a vertical boundary, and EDGE_HOR means that the block boundary being evaluated is a horizontal boundary.

[0126] Furthermore, since the above condition C2 is identical to the above-mentioned condition C92, its explanation is omitted here. Note that the above condition C2 is checked only if condition C1 is true. If condition C1 is false, the condition C2 is not checked, and it is determined that the deblocking filter should not be applied. Checking condition C2 requires the calculation of the variable d as shown in equations (1) to (7) above, and since this requires more processing than checking condition C1, it is possible to reduce the processing load by checking condition C2 after condition C1.

[0127] Furthermore, after determining whether or not to apply a deblocking filter based on conditions C1 and C2 as described above, the filter intensity determination unit 267 further determines the filter intensity of the deblocking filter to be applied to the color difference component of the decoded image. The deblocking filters that can be applied in this embodiment may be of two types, as will be described later: a weak filter having a weaker intensity and a strong filter having a stronger intensity. The filtering unit 269, which will be described later, then applies either the weak filter or the strong filter according to the filter intensity determined by the filter intensity determination unit 267.

[0128] The filter strength determination unit 267 determines the filter strength when it is determined that a deblocking filter should be applied. By determining the filter strength after determining whether or not a deblocking filter should be applied, it is possible to suppress the processing required for determining the filter strength.

[0129] Furthermore, the filter intensity determination unit 267 determines the filter intensity based on the waveform of the color difference component of pixels located near the block boundary. The determination based on the waveform will be explained below. The filter strength determination unit 267 determines the filter strength based on condition C3, which is derived from the following waveform.

[0130] -Condition C3: (Condition C31&&Condition C32&&Condition C33) -Condition C31:|p3-p0|+|q3-q0|<(beta>>3) -Condition C32:|p2-2*p1+p0|+|q2-2*q1+q0|<(beta>>2) -Condition C33:|p0-q0|<((t c *5+1)>>1)

[0131] The filter intensity determination unit 267 determines the above condition C3 for pixels located near the block boundary that are included in two lines. Conditions C31, C32, and C33 used in C3 are determined for each line. Note that p i , q k , p i ', q k ′, beta, and t C Since this has already been explained above, I will omit the explanation here.

[0132] Conditions C31, C32, and C33 are conditions that are determined using the pixels contained in each line. More specifically, condition C31 is a condition relating to the flatness of the color difference components of the pixels contained in each line within a block. Condition C32 is a condition relating to the determination of the continuity of the color difference components of the pixels contained in each line within a block. Condition C33 is a condition relating to the gap (difference) between blocks of color difference components of the pixels contained in each line, and more specifically, it is a condition for determining the gap between blocks using the pixel values ​​adjacent to the block boundary.

[0133] If condition C31 is true, the flatness of the chrominance component waveform is high within each block. Also, if condition C32 is true, the chrominance component waveform is highly continuous within each block. Furthermore, if condition C32 is true, the chrominance component waveform has a large gap at the block boundary.

[0134] As described above, condition C3 is determined to be true only if all of the above conditions C31, C32, and C33 are true. The filter strength determination unit 267 also determines condition C3 for each line. However, as described above, the filter strength is determined in units of two lines. In other words, the filter strength is determined so that if condition C3 is true for both of two consecutive lines, a strong filter is applied to those two lines, and if it is false, a weak filter is applied to those two lines.

[0135] (3) Filtering section The filtering unit 269 applies a deblocking filter to the color difference components of pixels located near the block boundary, based on the determination result of the application necessity determination unit 265 regarding the necessity of applying the deblocking filter. Furthermore, as described above, the filtering unit 269 applies either a weak filter or a strong filter as a deblocking filter according to the filter intensity determined by the filter intensity determination unit 267.

[0136] In this embodiment, the weak filter applied to the color difference component by the filtering unit 269 may be the same as, for example, the weak filter applied to the color difference component of the decoded image in Non-Patent Document 1 or HEVC described above. On the other hand, the strong filter applied to the color difference component in this embodiment may be different from the strong filter applied to the color difference component in Non-Patent Document 1 (the strong filter applied to the luminance component in HEVC). An example of a strong filter applied to the color difference component in this embodiment will be described below.

[0137] In this embodiment, the coefficient of the strong filter applied to the color difference component may be 2 at the center of the strong filter application range and 1 at other positions. Alternatively, the filtering unit 269 may define the strong filter application range as extending from the block boundary to three pixels on both sides, and apply the strong filter to the color difference components of pixels included in the application range, using the three pixels on both sides of the center of the application range as reference pixels. For example, a strong filter with p0 as the center position of the applicable range can be expressed as shown in equation (14) below.

[0138] p0'=Clip3(p 0- w*t C ,p0+w*t C ,((p3+p2+p1+2*p0+q0+q1+q2+4)>>3)) …(14)

[0139] In equation (14) above, w is a weight that can be set as appropriate, for example, it may be set to 1 or 2. Also, Clip3(a,b,c) represents a clipping operation that clips the value c within the range a≦c≦b, as described above.

[0140] By applying such a strong filter, it becomes possible to apply a deblocking filter that is stronger than the strong filter applied to the color difference component in Non-Patent Document 1 mentioned above.

[0141] By the way, if the center position of the range to which the strong filter is applied is the second or third pixel from the block boundary, the reference pixels will include pixels that are five or more pixels away from the block boundary. However, pixels that are five or more pixels away from the block boundary are not used in determining the filter strength and may not be suitable to be used as reference pixels. Therefore, the filtering unit 269 may use the pixel value of the fourth pixel from the block boundary as the pixel value of the reference pixel by padding it, instead of using pixels that are five or more pixels away from the block boundary.

[0142] For example, a strong filter with p1 as the center position of the applicable range can be expressed as shown in equation (15) below. p1' = Clip3(p 1- w*t C ,p1+w*t C ,((p4+p3+p2+2*p1+p0+q0+q1+4)>>3)) =Clip3(p 1- w*t C ,p1+w*t C ,((p3+p3+p2+2*p1+p0+q0+q1+4)>>3)) =Clip3(p 1- w*t C ,p1+w*t C ,((2*p3+p2+2*p1+p0+q0+q1+4)>>3)) …(15)

[0143] Similarly, a strong filter with p2 as the center position of the applicable range can be expressed as shown in equation (16) below. p2' = Clip3(p 2- w*t C ,p2+w*t C ,((p5+p4+p3+2*p2+p1+p0+q0+4)>>3)) =Clip3(p 2- w*t C ,p2+w*t C ,((p3+p3+p3+2*p2+p1+p0+q0+4)>>3)) =Clip3(p 2- w*t C ,p2+w*t C ,((3*p3+2*p2+p1+p0+q0+4)>>3)) …(16)

[0144] Similarly, strong filters with q0 to q3 as the center of the applicable range can be expressed as shown in equations (17) to (19) below. q0' = Clip3(q0 - w*t C ,q0+w*t C ,((p2+p1+p0+2*q0+q1+q2+q3+4)>>3)) …(17) q1' = Clip3(q1 - w*tC ,q1+w*t C ,((p1+p0+q0+2*q1+q2+2*q3+4)>>3)) …(18) q2' = Clip3(q2 - w*t C ,q2+w*t C ,((p0+q0+q1+2*q2+3*q3+4)>>3)) …(19)

[0145] [3-2. Processing Flow] The configuration example of the deblocking filter 26 according to this embodiment has been described above. Next, the processing flow by the deblocking filter 26 according to this embodiment will be described. Figure 9 is a flowchart showing an example of the processing flow by the deblocking filter 26 according to this embodiment. In the following, only the processing related to the features of this embodiment will be described, and explanations of other processing will be omitted as appropriate.

[0146] First, the boundary strength calculation unit 261 calculates bS (boundary strength) (S10). Here, the method for calculating bS will be explained in more detail with reference to Figure 10. Figure 10 is a flowchart illustrating the flow of the boundary strength calculation process (S10) performed by the boundary strength calculation unit 261.

[0147] First, the boundary strength calculation unit 261 initializes bS to 0 (S102). Next, the boundary strength calculation unit 261 determines whether condition A, which is a condition related to intra prediction, is true or false (S104). If condition A is true (YES in S104), bS is set to 16 (S106).

[0148] On the other hand, if condition A is false (NO in S104), the boundary strength calculation unit 261 determines the truth value of condition B2, which is a condition relating to the motion vector (MV) and the reference picture (S108). If condition B2 is true (YES in S108), bS is set to 1 (S110).

[0149] On the other hand, if condition B2 is false (NO in S108), the boundary strength calculation unit 261 determines the truth value of condition B1-Y, which is a condition regarding the presence or absence of a significance coefficient for the Y component (S112). If condition B1-Y is true (YES in S112), 8 is added to bS (S114), and then the process proceeds to step S116. On the other hand, if condition B1-Y is false (NO in S112), the process proceeds directly to step S116.

[0150] In step S116, the boundary strength calculation unit 261 determines the truth value of condition B1-U, which is a condition regarding the presence or absence of a significance coefficient for the U component. If condition B1-U is true (YES in S116), 4 is added to bS (S118), and then the process proceeds to step S120. On the other hand, if condition B1-U is false (NO in S116), the process proceeds directly to step S120.

[0151] In step S120, the boundary strength calculation unit 261 determines the truth value of condition B1-V, which is a condition regarding the presence or absence of a significance coefficient for the V component. If condition B1-V is true (YES in S120), 2 is added to bS (S122), and then the boundary strength calculation process (S10) ends. If condition B1-V is false (NO in S120), the boundary strength calculation process (S10) ends as is.

[0152] Returning to Figure 9, let's continue explaining the processing flow by the deblocking filter 26. In step S20, the application necessity determination unit 265 of the determination unit 263 determines whether the above-mentioned condition C1 is true or false. If condition C1 is false (NO in S20), the process ends.

[0153] On the other hand, if condition C1 is true (YES in S20), the application necessity determination unit 265 determines the truth value of the above-mentioned condition C2 (S30). If condition C2 is false (NO in S30), the process ends.

[0154] On the other hand, if condition C2 is true (YES in S30), the filter strength determination unit 267 of the determination unit 263 determines the filter strength by determining the truth or falsity of the above-mentioned condition C3 (S40). If condition C3 is true (YES in S40), the filtering unit 269 applies a strong filter to the color difference components of pixels located near the block boundary (S50). On the other hand, if condition C3 is false (NO in S40), the filtering unit 269 applies a weak filter to the color difference components of pixels located near the block boundary (S60).

[0155] The processing flow by the deblocking filter 26 according to this embodiment has been described above. The above processing, as described with reference to Figures 9 and 10, can be performed in units of 4 lines in the luminance component of the decoded image, or in units of 2 lines in the chrominance component of the decoded image, for example, in the case of the YUV420 format.

[0156] <New DF>

[0157] The following explains the new DF (Deblocking Filter).

[0158] The following documents, which are prior art, are incorporated herein by reference and are relevant to this technology.

[0159] [JVET-L0072 (version 1 - date 2018-09-25 00:23:50)] K. Andersson, Z. Zhang, R. Sjoberg: CE11: Long deblocking filters for luma (CE11.1.1) and for both luma and chroma (CE11.1.9), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 12th Meeting, Macao, CN, 3-12 Oct. 2018.

[0160] [JVET-L0224 (version 1 - date 2018-09-25 01:59:53)] Anand Meher Kotra, Biao Wang, Semih Esenlik, Han Gao, Zhijie Zhao, Jianle Chen: CE11.1.8: Longer tap Luma deblocking filter, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 12th Meeting, Macao, CN, 3-12 Oct. 2018.

[0161] [JVET-L0403r1 (version 3 - date 2018-10-04 05:13:00)] Dmytro Rusanovskyy, Marta Karczewicz: CE11: Test on long deblocking filtering from JVET-J0021 / JVET-K0334 (CE11.1.4). Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 12th Meeting, Macao, CN, 3-12 Oct. 2018.

[0162] [JVET-L0405r1 (version 2 - date 2018-10-03 07:14:31)] Weijia Zhu, Kiran Misra, Phil Cowan, Andrew Segall: CE11: Deblocking modifications for Large CUs both luma and chroma (Test 11.1.7a and CE11.1.7b). Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 12th Meeting, Macao, CN, 3-12 Oct. 2018.

[0163] [JVET-L0327-v1 (version 1 - date 2018-09-25 02:33:13)] Masaru Ikeda, Teruhiko Suzuki: CE11: Long-tap deblocking filter for luma and chroma (CE11.1.6). Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 12th Meeting, Macao, CN, 3-12 Oct. 2018.

[0164] Furthermore, the scope disclosed herein is not limited to the contents of the examples, and the contents of the following reference REF4, which were publicly known at the time of filing, are also incorporated herein by reference. In other words, the contents of the following reference REF4 also serve as the basis for determining the support requirement. REF4: [JVET-K1002-v2 (version 3 - date 2018-10-02 16:37:03)] Jianle Chen, Yan Ye, Seung Hwan Kim: Algorithm description for Versatile Video Coding and Test Model 2 (VTM 2), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 11th Meeting, Ljubljana, SI, 10-18 July 2018.

[0165] In VVC, the encoding block size is larger (8 times larger than in AVC and 2 times larger than in HEVC), resulting in significant degradation at block boundaries.

[0166] For large blocks with a large block size, the HEVC deblocking filter (DF) may not be able to completely remove block noise if it is strong.

[0167] Furthermore, in VVC, different block divisions can be selected for the luminance component and the chrominance component in the case of an intra-processor, and therefore the block size can be optimized for each of the luminance and chrominance components. It has been found that encoding efficiency can be improved by using large blocks for the chrominance component and small blocks with a smaller block size for the luminance component, and the importance of DF for large blocks of the chrominance component is increasing.

[0168] For large blocks, block noise (block distortion) tends to be greater. To properly remove large block noise, a strong filter strength, i.e., a greater blurring of the image, is required.

[0169] Filters that blur images more are long-tap filters (filters with a large number of taps). Therefore, in order to properly remove large block noise, it is necessary to use long-tap filters as the DF (Digital Filter) applied to the luminance and chrominance components.

[0170] When using a long-tap filter as the DF, a line buffer is needed at the block boundary, i.e., at the horizontal boundary when the DF is applied in raster scan order, for the number of pixels required for filtering. For example, if eight pixels arranged vertically perpendicular to the horizontal boundary are used for DF filtering, and four of those eight pixels are in the block above the horizontal boundary, then a line buffer is needed to store the pixels (and their pixel values) for four lines (rows).

[0171] If a large-capacity line buffer is installed in the filter factor (DF), the cost will increase. If the line buffer capacity is reduced to prioritize cost, a long-tap filter cannot be used in the DF, which degrades the DF's performance and makes it difficult to adequately remove block noise.

[0172] Therefore, it is desirable to determine the specifications of the DF by balancing cost and performance. In AVC and HEVC, the filter design for DF does not take into account the capacity of the line buffer.

[0173] Therefore, this technology proposes a new DF (hereinafter also referred to as the new DF).

[0174] Unless otherwise specified, the following explanation focuses on the color difference component, and the luminance component will not be discussed.

[0175] Figure 11 is a block diagram showing an example configuration of the DF300 as a new DF.

[0176] The DF300 can be used as the DF26.

[0177] In Figure 11, the parts corresponding to DF26 in Figure 7 are given the same reference numerals, and their explanations will be omitted as appropriate below.

[0178] In Figure 11, the DF300 includes a boundary strength calculation unit 261, a determination unit 310, a filtering unit 320, a line buffer 330, and a control unit 340.

[0179] Therefore, DF300 is similar to DF26 in Figure 7 in that it has a boundary strength calculation unit 261. However, DF300 differs from DF26 in that it has a determination unit 310 and a filtering unit 320 instead of the determination unit 263 and filtering unit 269. Furthermore, DF300 differs from DF26 in that it newly has a control unit 340.

[0180] Although not shown in Figure 7, DF26 in Figure 7 also has a line buffer, similar to DF300. However, the capacity of the line buffer 330 in DF300 and the line buffer in DF26 may differ.

[0181] The determination unit 310 includes an application necessity determination unit 311 and a filter strength determination unit 312.

[0182] The application necessity determination unit 311 is supplied with bS from the boundary intensity calculation unit 261. The application necessity determination unit 311 is also supplied with decoded images from outside the DF300 (addition unit 23 in Figure 5 and addition unit 65 in Figure 6) or from the line buffer 330.

[0183] The application necessity determination unit 311, similar to the application necessity determination unit 265 in Figure 7, performs the application necessity determination process using bS from the boundary intensity calculation unit 261, as well as decoded images from outside the DF300 and from the line buffer 330.

[0184] The application necessity determination unit 311 performs a step determination to determine whether there is a high probability of a step at the block boundary, depending on bS. For example, the application necessity determination unit 311 determines that there is a high probability of a step at the block boundary if bS is greater than 0 (1 or more). Then, if the application necessity determination unit 311 determines that there is a high probability of a step at the block boundary in the step determination, it performs a filter application determination to determine whether to apply DF to the pixels of the color difference component near the block boundary. The application necessity determination process performed by the application necessity determination unit 311 consists of the above step determination and filter application determination.

[0185] The application necessity determination unit 311 supplies the filter application determination result to the filter strength determination unit 312 as the determination result of the application necessity determination process.

[0186] The filter intensity determination unit 312 receives the filter application determination result from the application necessity determination unit 311, as well as the decoded image from outside the DF300 or from the line buffer 330.

[0187] If the filter application determination from the application necessity determination unit 311 indicates that the DF should be applied, the filter intensity determination unit 312, similar to the filter intensity determination unit 267 in Figure 7, uses the decoded image from outside the DF 300 or from the line buffer 330 to determine the filter intensity of the DF applied to the color difference component of the decoded image, i.e., it performs a filter type determination to determine the filter type of the DF applied to the color difference component of the decoded image. The filter intensity determination unit 321 then supplies the determination result of the filter type determination to the filtering unit 320.

[0188] In the new DF, the filter types applied to the color difference components of the decoded image include, for example, two types of filters: a weak filter with a weaker filter strength (compared to a strong filter) and a strong filter with a stronger filter strength (compared to a weak filter).

[0189] The filtering unit 320 receives the filter type determination result from the filter intensity determination unit 312, as well as decoded images from outside the DF300 or from the line buffer 330.

[0190] The filtering unit 320, similar to the filtering unit 269 in Figure 7, performs filtering by applying a strong filter or a weak filter, as determined by the filter type determination result from the filter intensity determination unit 312, to the decoded image. That is, the filtering unit 320 performs the filtering calculation on the target pixel, which is the pixel of the color difference component to be filtered, from the decoded image from outside the DF300 or from the line buffer 330, using pixels of the color difference component in the vicinity of the target pixel. Here, the pixels used for filtering are also called filter constituent pixels.

[0191] The filtering unit 320 outputs the pixels (of the color difference component) obtained by filtering the target pixels as filtered pixels (pixels that constitute the filtered image after filtering).

[0192] The line buffer 330 receives the decoded image from outside the DF300. The line buffer 330 stores the pixels of the color difference component of the decoded image from outside the DF300 as appropriate. The line buffer 330 has a storage capacity to store pixels of the color difference component for a predetermined number of lines (rows), and when it has stored the pixels up to that capacity, it stores new pixels by overwriting the oldest pixels.

[0193] The control unit 340 controls each block that makes up the DF300.

[0194] In this embodiment, the DF300 processes the decoded images in the order of the raster scan. If the decoded images are processed not in the order of the raster scan, but for example, from top to bottom, and then repeated from left to right, the horizontal (left and right) and vertical (up and down) directions described below will be reversed (swapped).

[0195] Figure 12 shows an example of the configuration of a decoded image processed by the DF300.

[0196] The blocks that make up the decoded image include CTUs, which contain blocks such as PUs and TUs.

[0197] The intensity calculation unit 261, determination unit 310, and filtering unit 320 of the DF300 in Figure 11 can process data using CTU as the unit, and in this case, they have a buffer (hereinafter also referred to as an internal buffer) (not shown) that can store CTU.

[0198] For the purposes of this discussion, we will refer to the CTU boundary within a block boundary as the CTU boundary, and the block boundaries other than the CTU boundary as the internal boundary. Furthermore, the CTU that is being processed by DF300 will be referred to as the CTU of interest.

[0199] The intensity calculation unit 261, the determination unit 310, and the filtering unit 320 store the pixels of the lines (rows) included in the CTU of interest in an internal buffer and process the CTU of interest.

[0200] In the CTU of interest, for horizontal internal boundaries, the pixels of the blocks above and below the internal boundary are stored in the internal buffer, so there is no need to store them in the line buffer 330.

[0201] Furthermore, for the CTU of interest, the pixels of the block below the upper horizontal CTU boundary are stored in the internal buffer, and therefore do not need to be stored in the line buffer 330. However, the pixels of the block above the upper horizontal CTU boundary are pixels within the CTU of the row above the CTU of interest, and are therefore not stored in the internal buffer. For this reason, the pixels of the block above the upper horizontal CTU boundary of the CTU of interest need to be stored in the line buffer 330.

[0202] The above explains why the line buffer 330 is necessary.

[0203] Figure 13 is a flowchart illustrating the processing of DF300 in Figure 11.

[0204] In the DF300, the line buffer 330 stores the pixels of the color difference component of the decoded image supplied from outside the DF300 as appropriate.

[0205] Then, in step S211, the boundary strength calculation unit 261 calculates bS as described above and supplies it to the application necessity determination unit 311, and the process proceeds to step S212.

[0206] In step S212, the application necessity determination unit 311 performs a step difference determination to determine whether bS is greater than 0.

[0207] In step S212, if bS is determined not to be greater than 0, that is, if there is no possibility of a step at the block boundary, the process terminates. Therefore, in this case, the DF300 filter is not applied to the decoded image.

[0208] On the other hand, if the step difference determination in step S212 determines that bS is greater than 0, that is, if there is a possibility of a step difference at the block boundary, the process proceeds to step S213.

[0209] In step S213, the application necessity determination unit 311 performs a filter application determination to determine whether to apply DF to the pixels of the color difference component near the block boundary.

[0210] If, in the filter application determination in step S213, it is determined that DF should not be applied (not applied), the process terminates.

[0211] On the other hand, if it is determined in the filter application determination in step S213 that DF should be applied, the process proceeds to step S214.

[0212] In step S214, the filter strength determination unit 312 determines the filter type, and the process proceeds to step S215.

[0213] In step S215, the filtering unit 320 performs a filtering process by applying the strong filter or weak filter, as determined by the filter type determination result in step S214, to the decoded image, and the process is completed.

[0214] Figure 14 is a diagram illustrating the DF of HEVC.

[0215] HEVC's DF (Define Filter) has filters Y1, Y2, and C1.

[0216] Filter Y1 is a strong filter for luminance components, which filters the luminance component as the target of DF (Digital Filter).

[0217] Filter Y2 is a weak filter for the luminance component that filters the luminance component as the target of DF.

[0218] Filter C1 is a color difference component filter that filters the color difference component as the target of DF.

[0219] In HEVC, there is only one filter, filter C1, that filters the chromatic difference component as the target of DF (Define Filter), and there is no distinction like that between strong filters and weak filters.

[0220] Figure 15 is a diagram illustrating the new defensive line (DF).

[0221] The new DF includes filters NY1, NY2, NC1, and NC2.

[0222] Filter NY1 is a strong filter for luminance components, which filters the luminance component as the target of DF (Define Filter). For example, filter Y1 can be used as filter NY1.

[0223] Filter NY2 is a weak filter for the luminance component that filters the luminance component as the target of DF. For example, filter Y2 can be used as filter NY2.

[0224] Filter NC1 is a strong filter for color difference components (a filter with stronger filter strength than filter NC2) that filters the color difference component as the target of DF. As filter NC1, for example, filters Y1, Y2, or a filter based on the original filter OF described later (filters Y1, Y2, or OF itself, or filters with reduced filter characteristics of filters Y1, Y2, or OF) can be used.

[0225] Filter NC2 is a weak filter for color difference components (a filter with weaker filter strength than filter NC1) that filters the color difference component as the target of DF. For example, filter C1 can be used as filter NC2.

[0226] Figure 16 shows an example of a pixel with a color difference component at a block boundary.

[0227] In Figure 16, the left-right direction represents the up-down direction of the decoded image, and the up-down direction represents the chromatic difference component (or its magnitude). Therefore, in Figure 16, the block boundaries indicated by vertical lines are the horizontal boundaries (horizontal boundaries) of the decoded image.

[0228] The new DF will be explained using the example of applying the DF to pixels of blocks Bp and Bq adjacent to the horizontal boundary above and below. In this case, the DF applied to the pixels of blocks Bp and Bq is a vertical filter applied in the vertical direction.

[0229] For applying DF to pixels in blocks adjacent to the left and right of a vertical boundary, you can use the new DF or other filters. The explanation for applying DF to pixels in blocks adjacent to the left and right of a vertical boundary will be omitted below.

[0230] In FIG. 16, the pixels of block Bp above the horizontal boundary and the pixels of block Bq below it are represented by p and q, respectively, in the same manner as in FIG. 3. However, in FIG. 16, the indices j of p and q are omitted. The pixel p (p) is also called a p-side pixel, and the pixel q (q) is also called a q-side pixel. i,j and q k,j In FIG. 16, the pixels of block Bp above the horizontal boundary and the pixels of block Bq below it are represented by p and q, respectively, in the same manner as in FIG. 3. However, in FIG. 16, the indices j of p and q are omitted. The pixel p (p) is also called a p-side pixel, and the pixel q (q) is also called a q-side pixel. i,j and q k,j In FIG. 16, the pixels of block Bp above the horizontal boundary and the pixels of block Bq below it are represented by p and q, respectively, in the same manner as in FIG. 3. However, in FIG. 16, the indices j of p and q are omitted. The pixel p (p) is also called a p-side pixel, and the pixel q (q) is also called a q-side pixel. i (p i,j ) is also called a p-side pixel, and the pixel q k (q k,j ) is also called a q-side pixel.

[0231] In blocks Bp and Bq adjacent above and below the horizontal boundary, |p3 - p0| + |q3 - q0| represents the flatness between blocks Bp and Bq. Also, |p2 - 2*p1 + p0| + |q2 - 2*q1 + q0| represents the continuity between blocks Bp and Bq, and |p0 - q0| represents the gap between blocks Bp and Bq.

[0232] Hereinafter, in the new DF, variations of filter NC1 as a strong filter applied to the color difference component will be described.

[0233] <When adopting a filter based on filter Y1 as filter NC1>

[0234] FIG. 17 is a diagram showing filter NC1 and required pixels when adopting a filter based on filter Y1 as filter NC1.

[0235] Filters NC1 based on filter Y1 include a Y1 normal filter, a Y1 minus 1 asymmetric filter, a Y1 minus 1 symmetric filter, a Y1 minus 2 asymmetric filter, and a Y1 minus 2 symmetric filter.

[0236] When adopting the Y1 normal filter as filter NC1, in the filter application determination, the same determination as the luminance component application determination for determining whether to apply DF to the luminance component of HEVC is performed.

[0237] In the determination of whether to apply the luminance component, it is determined whether the formula (20) is satisfied. If the formula (20) is satisfied, it is determined that DF is applied, and if not, it is determined that DF is not applied.

[0238] dp0 = Abs( p2,0 - 2 * p1,0 + p0,0 ) dp3 = Abs( p2,3 - 2 * p1,3 + p0,3 ) dq0 = Abs( q2,0 - 2 * q1,0 + q0,0 ) dq3 = Abs( q2,3 - 2 * q1,3 + q0,3 ) d = dp0 + dp3 + dq0 + dq3 < beta ···(20)

[0239] In the formula (20), Abs(A) represents the absolute value of A.

[0240] In the determination of whether to apply the luminance component, using the pixels pi,j and qk,j of the luminance component in the first line and the pixels pi,j and qk,j of the luminance component in the fourth line of the blocks Bp and Bq, it is determined whether the formula (20) is satisfied. And when the formula (20) is satisfied, it is determined that DF is applied to the pixels of the four lines of the blocks Bp and Bq.

[0241] Here, assuming that the decoded image in the YUV420 format is the target of DF, the blocks Bp and Bq have pixels of two lines of chrominance components. In the determination of whether to apply the filter when adopting the Y1 normal filter, using the pixels of one line of the chrominance components among the pixels of two lines of the chrominance components, it is determined whether the formula (20) is satisfied. That is, for example, with dp3 and dq3 in the formula (20) set to 0, it is determined whether the formula (20) is satisfied.

[0242] In this case, the pixels of the chrominance components used for the determination of whether to apply the filter (hereinafter also referred to as the pixels for application determination) are the six pixels of pixels p0 to p2 and q0 to q2.

[0243] Therefore, when the horizontal boundary is the CTU boundary, for the filter application determination, as the storage capacity of the line buffer 330, a capacity for storing the color difference components of three lines of pixels p0 to p2 of the block Bp above the horizontal boundary (hereinafter also referred to as the capacity for three lines of pixels) is required.

[0244] When adopting the Y1 normal filter as the filter NC1, in the filter type determination, the same determination as the luminance component application determination for determining whether to apply DF to the luminance component of HEVC is performed. That is, in the filter type determination, it is determined whether the formula (21) is satisfied.

[0245] |p3 - p0|+|q3 - q0| < (beta>>3) |p2 - 2*p1 + p0|+ |q2 - 2*q1 + q0| < (beta>>2) |p0 - q0| < ((tc*5+1)>>1) ···(21)

[0246] Note that A>>B represents shifting A to the right by B bits, and A<<B represents shifting A to the left by B bits.

[0247] In the filter type determination, when the formula (21) is satisfied, it is determined that the filter NC1 as a strong filter is applied to the color difference component, and when the formula (21) is not satisfied, it is determined that the filter NC2 as a weak filter is applied to the color difference component.

[0248] The pixels of the color difference component used for the filter type determination of the formula (21) (hereinafter also referred to as type determination pixels) are eight pixels of pixels p0 to p3 and q0 to q3.

[0249] Therefore, when the horizontal boundary is a CTU boundary, for filter type determination, as the storage capacity of the line buffer 330, the capacity for four pixel lines of pixels p0 to p3 of the block Bp above the horizontal boundary is required.

[0250] As the Y1 normal filter, the strong filter of the luminance component of HEVC can be adopted. In this case, the Y1 normal filter is represented by Equation (22).

[0251] p2′ = Clip3( p2 - 2*tC, p2 + 2*tC, ( 2*p3 + 3*p2 + p1 + p0 + q0 + 4 ) >> 3 ) p1′ = Clip3( p1 - 2*tC, p1 + 2*tC, ( p2 + p1 + p0 + q0 + 2 ) >> 2 ) p0′ = Clip3( p0 - 2*tC, p0 + 2*tC, ( p2 + 2*p1 + 2*p0 + 2*q0 + q1 + 4 ) >> 3 ) q0′ = Clip3( q0 - 2*tC, q0 + 2*tC, ( p1 + 2 * p0 + 2 * q0 + 2 * q1 + q2 + 4 ) >> 3 ) q1′ = Clip3( q1 - 2*tC, q1 + 2*tC, ( p0 + q0 + q1 + q2 + 2 ) >> 2 ) q2′ = Clip3( q2 - 2*tC, q2 + 2*tC, ( p0 + q0 + q1 + 3 *q2 + 2*q3 + 4 ) >> 3 ) ···(22)

[0252] Note that Clip3(A, B, C) is a function that represents A when C < A, B when C > B, and C in other cases, respectively.

[0253] The filter configuration pixels of the color difference components used for the filter processing of the Y1 normal filter in Equation (22) are eight pixels from pixel p0 to p3 and q0 to q3. Further, the target pixels of the color difference components to be subjected to the filter processing (pixels for which filter pixels are required) are six pixels from pixel p0 to p2 and q0 to q2.

[0254] Therefore, when the horizontal boundary is the CTU boundary, for the filter processing of the Y1 normal filter, as the storage capacity of the line buffer 330, a capacity of four pixel lines from pixel p0 to p3 of the block Bp above the horizontal boundary is required.

[0255] From the above, when adopting the Y1 normal filter as the filter NC1, the storage capacity of the line buffer 330 is restricted by the type determination pixel with the most pixels on the p side among the application determination pixels, type determination pixels, and filter configuration pixels, and as the storage capacity of the line buffer 330, a capacity of four pixel lines from pixel p0 to p3 of the block Bp above the horizontal boundary is required.

[0256] When adopting the Y1 minus 1 asymmetric filter as the filter NC1, the filter application determination is performed, for example, in the same manner as in the case of the Y1 normal filter.

[0257] Therefore, the application determination pixels of the color difference components used for the filter application determination are six pixels from pixel p0 to p2 and q0 to q2, which are the same as those of the Y1 normal filter.

[0258] As a result, when the horizontal boundary is the CTU boundary, for the filter application determination, as the storage capacity of the line buffer 330, a capacity of three pixel lines from pixel p0 to p2 of the block Bp above the horizontal boundary is required.

[0259] When a Y1-1 asymmetric filter is adopted as filter NC1, the filter type determination checks whether it satisfies equation (23), which is obtained by replacing (only one pixel) of pixel p3 (the one on the p side furthest from the horizontal boundary) in equation (21), the luminance component application determination that determines whether to apply DF to the luminance component of HEVC, with pixel p2 on the horizontal boundary side.

[0260] |p2 - p0|+|q3 - q0| < (beta>>3) |p2 - 2*p1 + p0|+ |q2 - 2*q1 + q0| < (beta>>2) |p0 - q0| < ((tc*5+1)>>1) ···(twenty three)

[0261] In the filter type determination, if equation (23) is satisfied, it is determined that filter NC1, as a strong filter, will be applied to the color difference component; if equation (23) is not satisfied, it is determined that filter NC2, as a weak filter, will be applied to the color difference component.

[0262] The seven pixels used for determining the type of the color difference component in the filter type determination in equation (23) are pixels p0 to p2 and q0 to q3.

[0263] Therefore, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity in the line buffer 330 equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0264] As a Y1-1 asymmetric filter, we can use the Y1 normal filter of equation (22), that is, a filter in which pixel p3 of the strong filter for the luminance component of HEVC is replaced with pixel p2 on the horizontal boundary side. In this case, the Y1-1 asymmetric filter is expressed by equation (24).

[0265] p2′ = Clip3( p2 - 2*tC, p2 + 2*tC, ( 5*p2 + 3*p2 + p1 + p0 + q0 + 4 ) >> 3 ) p1′ = Clip3( p1 - 2*tC, p1 + 2*tC, ( p2 + p1 + p0 + q0 + 2 ) >> 2 ) p0′ = Clip3( p0 - 2*tC, p0 + 2*tC, ( p2 + 2*p1 + 2*p0 + 2*q0 + q1 + 4 ) >> 3 ) q0′ = Clip3( q0 - 2*tC, q0 + 2*tC, ( p1 + 2 * p0 + 2 * q0 + 2 * q1 + q2 + 4 ) >> 3 ) q1′ = Clip3( q1 - 2*tC, q1 + 2*tC, ( p0 + q0 + q1 + q2 + 2 ) >> 2 ) q2′ = Clip3( q2 - 2*tC, q2 + 2*tC, ( p0 + q0 + q1 + 3 *q2 + 2*q3 + 4 ) >> 3 ) ···(24)

[0266] The chromatic difference components used in the filtering process of the Y1-1 asymmetric filter in equation (24) consist of seven pixels: pixels p0 to p2 and q0 to q3. Furthermore, the pixels targeted for filtering of the chromatic difference components are six pixels: pixels p0 to p2 and q0 to q2.

[0267] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the Y1-1 asymmetric filter requires the storage capacity of the line buffer 330 to be equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0268] The Y1-1 asymmetric filter in equation (24) is a filter in which the pixel p3 in the first equation for calculating pixel p2' in equation (22) is replaced with the pixel p2 on the horizontal boundary side. Therefore, the tap coefficient (filter coefficient) of pixel p2 in the first equation for calculating pixel p2' is changed from 3 in equation (22) to 5 (=2+3).

[0269] Based on the above, if a Y1-minus-1 asymmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0270] Here, when a Y1-minus-1 asymmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p2 in block Bp and pixels q0 to q3 in block Bq, and are asymmetric with respect to the horizontal boundary (asymmetric between the p side and the q side).

[0271] When a Y1-minus-1 symmetric filter is adopted as filter NC1, the filter application determination is performed in the same way as, for example, the case of a Y1 normal filter.

[0272] Therefore, the pixels used to determine the application of the color difference component for filter application are the same six pixels, p0 to p2 and q0 to q2, as for the Y1 normal filter.

[0273] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary, for the filter application determination.

[0274] When a Y1-minus-1 symmetric filter is adopted as filter NC1, the filter type determination checks whether the equation (25) is satisfied, which is obtained by replacing pixel p3 in equation (21), the luminance component application determination that determines whether to apply DF to the luminance component of HEVC, with pixel p2 on the horizontal boundary side, and replacing pixel q3 with pixel q2 on the horizontal boundary side.

[0275] |p2 - p0|+|q2 - q0| < (beta>>3) |p2 - 2*p1 + p0|+ |q2 - 2*q1 + q0| < (beta>>2) |p0 - q0| < ((tc*5+1)>>1) ···(twenty five)

[0276] In the filter type determination, if equation (25) is satisfied, it is determined that filter NC1, as a strong filter, will be applied to the color difference component; if equation (25) is not satisfied, it is determined that filter NC2, as a weak filter, will be applied to the color difference component.

[0277] The six pixels used to determine the type of the color difference component in the filter type determination in equation (25) are pixels p0 to p2 and q0 to q2.

[0278] Therefore, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity in the line buffer 330 equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0279] As a Y1-minus-1 symmetric filter, one can employ the Y1 normal filter of equation (22), that is, a filter obtained by replacing pixel p3 of the strong filter for the luminance component of HEVC with pixel p2 on the horizontal boundary side, and replacing pixel q3 with pixel q2 on the horizontal boundary side. In this case, the Y1-minus-1 symmetric filter is expressed by equation (26).

[0280] p2′ = Clip3( p2 - 2*tC, p2 + 2*tC, ( 5*p2 + 3*p2 + p1 + p0 + q0 + 4 ) >> 3 ) p1′ = Clip3( p1 - 2*tC, p1 + 2*tC, ( p2 + p1 + p0 + q0 + 2 ) >> 2 ) p0′ = Clip3( p0 - 2*tC, p0 + 2*tC, ( p2 + 2*p1 + 2*p0 + 2*q0 + q1 + 4 ) >> 3 ) q0′ = Clip3( q0 - 2*tC, q0 + 2*tC, ( p1 + 2 * p0 + 2 * q0 + 2 * q1 + q2 + 4 ) >> 3 ) q1′ = Clip3( q1 - 2*tC, q1 + 2*tC, ( p0 + q0 + q1 + q2 + 2 ) >> 2 ) q2′ = Clip3( q2 - 2*tC, q2 + 2*tC, ( p0 + q0 + q1 + 5 *q2 + 4 ) >> 3 ) ...(26)

[0281] The chromatic difference components used in the filtering process of the Y1-1 symmetric filter in equation (26) consist of six pixels: pixels p0 to p2 and q0 to q2. Furthermore, the pixels targeted for filtering of the chromatic difference components are the six pixels: pixels p0 to p2 and q0 to q2.

[0282] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the Y1-1 symmetric filter requires the storage capacity of the line buffer 330 to be equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0283] The Y1-1 symmetric filter in equation (26) is a filter obtained by replacing pixel p3 in the first equation for calculating pixel p2' in equation (22) with pixel p2 on the horizontal boundary side. As a result, the tap coefficient of pixel p2 in the first equation for calculating pixel p2' is changed from 3 in equation (22) to 5 (=2+3). Furthermore, the Y1-1 symmetric filter in equation (26) is a filter obtained by replacing pixel q3 in the sixth equation for calculating pixel q2' in equation (22) with pixel q2 on the horizontal boundary side. As a result, the tap coefficient of pixel q2 in the sixth equation for calculating pixel q2' is changed from 3 in equation (22) to 5 (=3+2).

[0284] Based on the above, if a Y1-minus-1 symmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0285] Here, when a Y1-minus-1 symmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p2 in block Bp and pixels q0 to q2 in block Bq, and are symmetric with respect to the horizontal boundary.

[0286] When a Y1-2 asymmetric filter is adopted as filter NC1, the filter application determination checks whether the condition satisfies equation (27), which is obtained by setting dp3 and dq3 in equation (20) to 0 and replacing pixels p2,0 with pixels p1,0 on the horizontal boundary side.

[0287] dp0 = Abs( p1,0 - 2 * p1,0 + p0,0 ) = Abs( p0,0 - p1,0 ) dq0 = Abs( q2,0 - 2 * q1,0 + q0,0 ) d = dp0 + dq0 < (beta>>1) ...(27)

[0288] In the filter application determination, if equation (27) is satisfied, it is determined that the DF should be applied; otherwise, it is determined that the DF should not be applied.

[0289] Therefore, the pixels used for determining the application of the color difference component in the filter application determination are the five pixels p0 to p1 and q0 to q2.

[0290] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary, for the filter application determination.

[0291] When a Y1-2 asymmetric filter is adopted as filter NC1, the filter type determination checks whether it satisfies equation (28), which is obtained by substituting pixels p3 and p2 (the two on the p side furthest from the horizontal boundary) in equation (21), the luminance component application determination that determines whether to apply DF to the luminance component of HEVC, with pixel p1 on the horizontal boundary side.

[0292] |p1 - p0|+|q3 - q0| < (beta>>3) |p1 - 2*p1 + p0|+ |q2 - 2*q1 + q0| < (beta>>2) = |p0 - p1|+ |q2 - 2*q1 + q0| < (beta>>2) |p0 - q0| < ((tc*5+1)>>1) ...(28)

[0293] In the filter type determination, if equation (28) is satisfied, it is determined that filter NC1, as a strong filter, will be applied to the color difference component; if equation (28) is not satisfied, it is determined that filter NC2, as a weak filter, will be applied to the color difference component.

[0294] The six pixels used to determine the type of the color difference component in the filter type determination in equation (28) are pixels p0 to p1 and q0 to q3.

[0295] Therefore, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity in the line buffer 330 equivalent to two pixel lines, specifically pixel p0 or p1, of block Bp above the horizontal boundary.

[0296] As a Y1-2 asymmetric filter, one can employ a filter obtained by replacing pixels p3 and p2 of the Y1 normal filter in equation (22), that is, the strong filter for the luminance component of HEVC, with pixel p1 on the horizontal boundary side. In this case, the Y1-2 asymmetric filter is expressed by equation (29).

[0297] p1′ = Clip3( p1 - 2*tC, p1 + 2*tC, ( 2*p1 + p0 + q0 + 2 ) >> 2 ) p0′ = Clip3( p0 - 2*tC, p0 + 2*tC, ( 3*p1 + 2*p0 + 2*q0 + q1 + 4 ) >> 3 ) q0′ = Clip3( q0 - 2*tC, q0 + 2*tC, ( p1 + 2*p0 + 2*q0 + 2*q1 + q2 + 4 ) >> 3 ) q1′ = Clip3( q1 - 2*tC, q1 + 2*tC, ( p0 + q0 + q1 + q2 + 2 ) >> 2 ) q2′ = Clip3( q2 - 2*tC, q2 + 2*tC, ( p0 + q0 + q1 + 3*q2 + 2*q3 + 4 ) >> 3 ) ···(29)

[0298] The chromatic difference components used in the filtering process of the Y1-2 asymmetric filter in equation (29) consist of six pixels: pixels p0 to p1 and q0 to q3. The target pixels for the chromatic difference components to be filtered are five pixels: pixels p0 to p1 and q0 to q2.

[0299] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the Y1-2 asymmetric filter requires the storage capacity of the line buffer 330 to be equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0300] The Y1-2 asymmetric filter in equation (29) is an equation that omits the first equation in equation (22) that calculates pixel p2'. Furthermore, the Y1-2 asymmetric filter in equation (29) is a filter in which pixel p2 in the equations for calculating pixels p1' and p0' in equation (22) is replaced with pixel p1 on the horizontal boundary side. As a result, the tap coefficient of pixel p1 in the equation for calculating pixel p1' in equation (29) is changed from 1 to 2 (=1+1) in equation (22), and the tap coefficient of pixel p1 in the equation for calculating pixel p0' in equation (29) is changed from 2 to 3 (=1+2) in equation (22).

[0301] Based on the above, if a Y1-2 asymmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0302] Here, when a Y1-2 asymmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p1 in block Bp and pixels q0 to q3 in block Bq, and are asymmetric with respect to the horizontal boundary.

[0303] When a Y1-2 symmetric filter is adopted as filter NC1, the filter application determination checks whether the result satisfies equation (30), which is obtained by setting dp3 and dq3 in equation (20) to 0 and substituting pixels p2,0 and q2,0 with pixels p1,0 and q1,0 on the horizontal boundary side, respectively.

[0304] dp0 = Abs( p1,0 - 2 * p1,0 + p0,0 ) = Abs( p0,0 - p1,0 ) dq0 = Abs( q1,0 - 2 * q1,0 + q0,0 ) = Abs( q0,0 - q1,0 ) d = dp0 + dq0 < (beta>>1) ...(30)

[0305] In the filter application determination, if equation (30) is satisfied, it is determined that the DF should be applied; otherwise, it is determined that the DF should not be applied.

[0306] Therefore, the pixels used for determining the application of the color difference component in the filter application determination are the four pixels p0 to p1 and q0 to q1.

[0307] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary, for the filter application determination.

[0308] When a Y1-2 symmetric filter is adopted as filter NC1, the filter type determination checks whether the equation (31) is satisfied, which is obtained by replacing pixels p3 and p2 in equation (21), which is used to determine whether DF is applied to the luminance component of HEVC, with the horizontal boundary pixel p1, and replacing pixels q3 and q2 with the horizontal boundary pixel q1.

[0309] |p1 - p0|+|q1 - q0| < (beta>>3) |p1 - 2*p1 + p0|+ |q1 - 2*q1 + q0| < (beta>>2) = |p0 - p1|+ |q0 - q1| < (beta>>2) |p0 - q0| < ((tc*5+1)>>1) ...(31)

[0310] In the filter type determination, if equation (31) is satisfied, it is determined that filter NC1, as a strong filter, should be applied to the color difference component; if equation (31) is not satisfied, it is determined that filter NC2, as a weak filter, should be applied to the color difference component.

[0311] The four pixels used to determine the type of the color difference component in the filter type determination in equation (31) are pixels p0 to p1 and q0 to q1.

[0312] Therefore, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity in the line buffer 330 equivalent to two pixel lines, specifically pixel p0 or p1, of block Bp above the horizontal boundary.

[0313] As a Y1-2 symmetric filter, one can employ a filter obtained by replacing pixels p3 and p2 of the Y1 normal filter in equation (22), that is, a strong filter for the luminance component of HEVC, with pixels p1 on the horizontal boundary side, and replacing pixels q3 and q2 with pixels q1 on the horizontal boundary side. In this case, the Y1-2 symmetric filter is expressed by equation (32).

[0314] p1′ = Clip3( p1 - 2*tC, p1 + 2*tC, ( 2*p1 + p0 + q0 + 2 ) >> 2 ) p0′ = Clip3( p0 - 2*tC, p0 + 2*tC, ( 3*p1 + 2*p0 + 2*q0 + q1 + 4 ) >> 3 ) q0′ = Clip3( q0 - 2*tC, q0 + 2*tC, ( p1 + 2*p0 + 2*q0 + 3*q1 + 4 ) >> 3 ) q1′ = Clip3( q1 - 2*tC, q1 + 2*tC, ( p0 + q0 + 2*q1 + 2 ) >> 2 ) ...(32)

[0315] The chromatic difference components used in the filtering process of the Y1-2 symmetric filter in equation (32) consist of four pixels: pixels p0 to p1 and q0 to q1. Furthermore, the pixels targeted for filtering of the chromatic difference components are the same four pixels: pixels p0 to p1 and q0 to q1.

[0316] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the Y1-2 symmetric filter requires the storage capacity of the line buffer 330 to be equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0317] The Y1-2 symmetric filter in equation (32) is an equation that omits the first equation for finding pixel p2' and the sixth equation for finding pixel q2' in equation (22). Furthermore, the Y1-2 symmetric filter in equation (32) is a filter that replaces pixel p2 in the equations for finding pixels p1' and p0' in equation (22) with pixel p1, and replaces pixel q2 in the equations for finding pixels q1' and q0' in equation (22) with pixel q1 on the horizontal boundary side. As a result, the tap coefficient of pixel p1 in the equation for finding pixel p1' in equation (31) is changed from 1 to 2 (=1+1) in equation (22), and the tap coefficient of pixel p1 in the equation for finding pixel p0' in equation (31) is changed from 2 to 3 (=1+2) in equation (22). Furthermore, the tap coefficient of pixel q1 in the formula for finding pixel q0' in equation (31) has been changed from 2 in equation (22) to 3 (=2+1), and the tap coefficient of pixel q1 in the formula for finding pixel q1' in equation (31) has been changed from 1 in equation (22) to 2 (=1+1).

[0318] Based on the above, if a Y1-minus 2 symmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0319] Here, when a Y1-2 symmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p1 of block Bp and pixels q0 to q1 of block Bq, and are symmetric with respect to the horizontal boundary.

[0320] <When using a filter based on filter OF as filter NC1>

[0321] Figure 18 shows the filter NC1 and the required pixels when a filter based on filter OF is adopted as filter NC1.

[0322] Filters NC1 based on the OF filter include the OF normal filter, OF -1 asymmetric filter, OF -1 symmetric filter, OF -2 asymmetric filter, and OF -2 symmetric filter.

[0323] When an OF normal filter is used as filter NC1, the filter application determination is performed in the same way as for the Y1 normal filter, that is, in the same way as the luminance component application determination in equation (20).

[0324] Therefore, the pixels used to determine the application of the color difference component for filter application are the same six pixels, p0 to p2 and q0 to q2, as for the Y1 normal filter.

[0325] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary, for the filter application determination.

[0326] When an OF normal filter is adopted as filter NC1, the filter type determination is made by checking whether equation (21) is satisfied, similar to the Y1 normal filter.

[0327] Therefore, the pixels used to determine the type of the color difference component used for filter type determination are the same eight pixels as for the Y1 normal filter: pixels p0 to p3 and q0 to q3.

[0328] As a result, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity in the line buffer 330 equivalent to the capacity of four pixel lines, from pixel p0 to p3, of block Bp above the horizontal boundary.

[0329] As an OF normal filter, a strong filter for the luminance component of HEVC can be used, that is, a filter obtained by changing the tap coefficient of the Y1 normal filter in equation (22) or the clip parameters A and B of the clip function Clip3(A, B, C). As an OF normal filter, for example, the filter in equation (33) can be used.

[0330] p2′ = Clip3( p2 - tC, p2 + tC, ( 3*p3 + 2*p2 + p1 + p0 + q0 + 4 ) >> 3 ) p1′ = Clip3( p1 - tC, p1 + tC, ( 2*p3 + p2 + 2*p1 + p0 + q0 + q1 + 4 ) >> 3 ) p0′ = Clip3( p0 - tC, p0 + tC, ( p3 + p2 + p1 + 2*p0 + q0 + q1 + q2 + 4 ) >> 3 ) q0′ = Clip3( q0 - tC, q0 + tC, ( p2 + p1 + p0 + 2 * q0 + q1 + q2 + q3 + 4 ) >> 3 ) q1′ = Clip3( q1 - tC, q1 + tC, ( p1 + p0 + q0 + 2*q1 + q2 + 2*q3 + 4 ) >> 3 ) q2′ = Clip3( q2 - tC, q2 + tC, ( p0 + q0 + q1 + 2*q2 + 3*q3 + 4 ) >> 3 ) ...(33)

[0331] The OF normal filter in equation (33) is a strong filter for the luminance component of HEVC, that is, a filter in which the tap coefficients of the Y1 normal filter in equation (22) and the clip parameters A and B of the clip function Clip3(A, B, C) are changed. For example, in the OF normal filter in equation (33), 2tC, which constitutes the clip parameter in equation (22), is replaced with tC.

[0332] The chromatic difference components used in the filtering process of the OF normal filter in equation (33) consist of eight pixels: pixels p0 to p3 and q0 to q3. The target pixels for the chromatic difference components to be filtered are six pixels: pixels p0 to p2 and q0 to q2.

[0333] Therefore, if the horizontal boundary is a CTU boundary, the filtering process of the OF normal filter requires the storage capacity of the line buffer 330 to be equivalent to the capacity of four pixel lines, from pixel p0 to p3, of block Bp above the horizontal boundary.

[0334] Based on the above, if an OF normal filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of four pixel lines, from pixel p0 to p3, of block Bp above the horizontal boundary.

[0335] When an OF-1 asymmetric filter is adopted as filter NC1, the filter application determination is performed in the same way as, for example, the case of an OF normal filter.

[0336] Therefore, the pixels used to determine the application of the color difference component for filter application are the same six pixels, p0 to p2 and q0 to q2, as for the OF normal filter.

[0337] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary, for the filter application determination.

[0338] When an OF-1 asymmetric filter is adopted as filter NC1, the filter type determination is made by checking whether equation (23) is satisfied, similar to the Y1-1 asymmetric filter.

[0339] Therefore, the pixels used to determine the type of the chromatic difference component in the OF-1 asymmetric filter are the same seven pixels, p0 to p2 and q0 to q3, as in the Y1-1 asymmetric filter.

[0340] As a result, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity in the line buffer 330 equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0341] As an OF-1 asymmetric filter, a filter can be used in which pixel p3 of the OF normal filter in equation (33) is replaced with pixel p2 on the horizontal boundary side. In this case, the OF-1 asymmetric filter is expressed by equation (34).

[0342] p2′ = Clip3( p2 - tC, p2 + tC, ( 5*p2 + p1 + p0 + q0 + 4 ) >> 3 ) p1′ = Clip3( p1 - tC, p1 + tC, ( 3*p2 + 2*p1 + p0 + q0 + q1 + 4 ) >> 3 ) p0′ = Clip3( p0 - tC, p0 + tC, ( 2*p2 + p1 + 2*p0 + q0 + q1 + q2 + 4 ) >> 3 ) q0′ = Clip3( q0 - tC, q0 + tC, ( p2 + p1 + p0 + 2 * q0 + q1 + q2 + q3 + 4 ) >> 3 ) q1′ = Clip3( q1 - tC, q1 + tC, ( p1 + p0 + q0 + 2*q1 + q2 + 2*q3 + 4 ) >> 3 ) q2′ = Clip3( q2 - tC, q2 + tC, ( p0 + q0 + q1 + 2*q2 + 3*q3 + 4 ) >> 3 ) ...(34)

[0343] The chromatic difference components used in the filtering process of the OF-1 asymmetric filter in equation (34) consist of seven pixels: pixels p0 to p2 and q0 to q3. The target pixels for the chromatic difference components to be filtered are six pixels: pixels p0 to p2 and q0 to q2.

[0344] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the OF-1 asymmetric filter requires the storage capacity of the line buffer 330 to be equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0345] The OF-1 asymmetric filter in equation (34) is a filter in which pixel p3 in the equation for calculating pixel p2' or p0' in equation (33) is replaced with pixel p2 on the horizontal boundary side. Therefore, the tap coefficient of pixel p2 in the equation for calculating pixel p2' is changed from 2 in equation (33) to 5 (=3+2). Furthermore, the tap coefficient of pixel p2 in the equation for calculating pixel p1' is changed from 1 in equation (33) to 3 (=2+1), and the tap coefficient of pixel p2 in the equation for calculating pixel p2' is changed from 1 in equation (33) to 2 (=1+1).

[0346] Based on the above, when an OF-1 asymmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0347] Here, when an OF-1 asymmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p2 in block Bp and pixels q0 to q3 in block Bq, and are asymmetric with respect to the horizontal boundary.

[0348] When an OF minus 1 symmetric filter is adopted as filter NC1, the filter application determination is performed in the same way as, for example, the case of an OF normal filter.

[0349] Therefore, the pixels used to determine the application of the color difference component for filter application are the same six pixels, p0 to p2 and q0 to q2, as for the OF normal filter.

[0350] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary, for the filter application determination.

[0351] When an OF minus 1 symmetric filter is adopted as filter NC1, the filter type determination is made by checking whether it satisfies equation (25), similar to the 1 minus 1 symmetric filter.

[0352] Therefore, the type determination pixels for the chromatic difference component used to determine the filter type of the OF minus 1 symmetric filter are the same as for the Y1 minus 1 symmetric filter, consisting of six pixels: pixels p0 to p2 and q0 to q2.

[0353] As a result, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity in the line buffer 330 equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0354] As an OF-minus-1 symmetric filter, one can employ a filter obtained by replacing pixel p3 of the OF normal filter in equation (33) with pixel p2 on the horizontal boundary side, and also replacing pixel q3 with pixel q2 on the horizontal boundary side. In this case, the OF-minus-1 symmetric filter is represented by equation (35).

[0355] p2′ = Clip3( p2 - tC, p2 + tC, ( 5*p2 + p1 + p0 + q0 + 4 ) >> 3 ) p1′ = Clip3( p1 - tC, p1 + tC, ( 3*p2 + 2*p1 + p0 + q0 + q1 + 4 ) >> 3 ) p0′ = Clip3( p0 - tC, p0 + tC, ( 2*p2 + p1 + 2*p0 + q0 + q1 + q2 + 4 ) >> 3 ) q0′ = Clip3( q0 - tC, q0 + tC, ( p2 + p1 + p0 + 2 * q0 + q1 + 2*q2 + 4 ) >> 3 ) q1′ = Clip3( q1 - tC, q1 + tC, ( p1 + p0 + q0 + 2*q1 + 3*q2 + 4 ) >> 3 ) q2′ = Clip3( q2 - tC, q2 + tC, ( p0 + q0 + q1 + 5*q2 + 4 ) >> 3 ) ...(35)

[0356] The chromatic difference components used in the filtering process of the OF-1 symmetric filter in equation (35) consist of six pixels: pixels p0 to p2 and q0 to q2. Furthermore, the pixels targeted for filtering of the chromatic difference components are the six pixels: pixels p0 to p2 and q0 to q2.

[0357] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the OF minus 1 symmetric filter requires the storage capacity of the line buffer 330 to be equivalent to the capacity of three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0358] The OF-1 symmetric filter in equation (35) is a filter in which pixel p3 in the equation for finding pixels p2' or p0' in equation (33) is replaced with pixel p2 on the horizontal boundary side. As a result, the tap coefficient of pixel p2 in the equation for finding pixel p2' is changed from 2 in equation (33) to 5 (=3+2). Furthermore, the tap coefficient of pixel p2 in the equation for finding pixel p1' is changed from 1 in equation (33) to 3 (=2+1), and the tap coefficient of pixel p2 in the equation for finding pixel p0' is changed from 1 in equation (33) to 2 (=1+1).

[0359] Furthermore, the OF minus 1 symmetric filter in equation (35) is a filter in which pixel q3 in the equation for finding pixels q0' or q2' in equation (33) is replaced with pixel q2 on the horizontal boundary side. As a result, the tap coefficient of pixel q2 in the equation for finding pixel q0' is changed from 1 to 2 (=1+1) in equation (33). In addition, the tap coefficient of pixel q2 in the equation for finding pixel q1' is changed from 1 to 3 (=1+2) in equation (33), and the tap coefficient of pixel q2 in the equation for finding pixel q2' is changed from 2 to 5 (=2+3) in equation (33).

[0360] Based on the above, if an OF-minus-1 symmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0361] Here, when an OF-minus-1 symmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p2 of block Bp and pixels q0 to q2 of block Bq, and are symmetric with respect to the horizontal boundary.

[0362] When an OF-2 asymmetric filter is adopted as filter NC1, the filter application determination is made by checking whether equation (27) is satisfied, similar to the Y1-2 asymmetric filter.

[0363] Therefore, the pixels used to determine the application of the chromatic difference component for the OF-2 asymmetric filter are the same five pixels as for the Y1-2 asymmetric filter: pixels p0 to p1 and q0 to q2.

[0364] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary, for the filter application determination.

[0365] When an OF-2 asymmetric filter is adopted as filter NC1, the filter type determination is made by checking whether equation (28) is satisfied, similar to the Y1-2 asymmetric filter.

[0366] Therefore, the chromatic difference component type determination pixels used for determining the filter type of the OF-2 asymmetric filter are the same as those for the Y1-2 asymmetric filter, consisting of six pixels: p0 to p1 and q0 to q3.

[0367] As a result, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity equivalent to two pixel lines, specifically pixel p0 or p1, of block Bp above the horizontal boundary, in the line buffer 330.

[0368] As an OF-minus-2 asymmetric filter, a filter can be used in which pixels p3 and p2 of the OF normal filter in equation (33) are replaced with pixels p1 on the horizontal boundary side. In this case, the OF-minus-2 asymmetric filter is represented by equation (36).

[0369] p1′ = Clip3( p1 - tC, p1 + tC, ( 5*p1 + p0 + q0 + q1 + 4 ) >> 3 ) p0′ = Clip3( p0 - tC, p0 + tC, ( 3*p1 + 2*p0 + q0 + q1 + q2 + 4 ) >> 3 ) q0′ = Clip3( q0 - tC, q0 + tC, ( 2*p1 + p0 + 2 * q0 + q1 + q2 + q3 + 4 ) >> 3 ) q1′ = Clip3( q1 - tC, q1 + tC, ( p1 + p0 + q0 + 2*q1 + q2 + 2*q3 + 4 ) >> 3 ) q2′ = Clip3( q2 - tC, q2 + tC, ( p0 + q0 + q1 + 2*q2 + 3*q3 + 4 ) >> 3 ) ...(36)

[0370] The chromatic difference components used in the filtering process of the OF-2 asymmetric filter in equation (33) consist of six pixels: pixels p0 to p1 and q0 to q3. The target pixels for the chromatic difference components to be filtered are five pixels: pixels p0 to p1 and q0 to q2.

[0371] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the OF-2 asymmetric filter requires the storage capacity of the line buffer 330 to be equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0372] The OF-2 asymmetric filter in equation (36) is an equation that lacks the equation for calculating pixel p2' in equation (33).

[0373] Furthermore, the OF-2 asymmetric filter in equation (36) is a filter in which pixels p2 and p3 in the equations for calculating pixels p1', p0', and q0' in equation (33) are replaced with pixel p1 on the horizontal boundary side. As a result, the tap coefficient of pixel p1 in the equation for calculating pixel p1' in equation (36) is changed from 2 in equation (33) to 5 (=2+1+2). Also, the tap coefficient of pixel p1 in the equation for calculating pixel p0' in equation (36) is changed from 1 in equation (33) to 3 (=1+1+1), and the tap coefficient of pixel p1 in the equation for calculating pixel q0' in equation (36) is changed from 1 in equation (33) to 2 (=1+1).

[0374] Based on the above, when an OF-2 asymmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0375] Here, when an OF-2 asymmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p1 of block Bp and pixels q0 to q3 of block Bq, and are asymmetric with respect to the horizontal boundary (the OF-2 asymmetric filter is an asymmetric filter).

[0376] When an OF minus 2 symmetric filter is adopted as filter NC1, the filter application determination is made by checking whether equation (30) is satisfied, similar to the Y1 minus 2 symmetric filter.

[0377] Therefore, the pixels used to determine the application of the chromatic difference component for the OF minus 2 symmetric filter are the same four pixels, p0 to p1 and q0 to q1, as with the Y1 minus 2 symmetric filter.

[0378] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary, for the filter application determination.

[0379] When an OF minus 2 symmetric filter is adopted as filter NC1, the filter type determination is made by checking whether equation (31) is satisfied, similar to the Y1 minus 2 symmetric filter.

[0380] Therefore, the type determination pixels for the chromatic difference component used to determine the filter type of the OF minus 2 symmetric filter are the same four pixels, p0 to p1 and q0 to q1, as with the Y1 minus 2 symmetric filter.

[0381] As a result, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity equivalent to two pixel lines, specifically pixel p0 or p1, of block Bp above the horizontal boundary, in the line buffer 330.

[0382] As an OF minus 2 symmetric filter, one can employ a filter in which pixels p3 and p2 of the OF normal filter in equation (33) are replaced with pixels p1 on the horizontal boundary side, and pixels q2 and q3 are replaced with pixels q1 on the horizontal boundary side. In this case, the OF minus 2 symmetric filter is represented by equation (37).

[0383] p1′ = Clip3( p1 - tC, p1 + tC, ( 5*p1 + p0 + q0 + q1 + 4 ) >> 3 ) p0′ = Clip3( p0 - tC, p0 + tC, ( 3*p1 + 2*p0 + q0 + q1 + q2 + 4 ) >> 3 ) q0′ = Clip3( q0 - tC, q0 + tC, ( 2*p1 + p0 + 2 * q0 + 3*q1 + 4 ) >> 3 ) q1′ = Clip3( q1 - tC, q1 + tC, ( p1 + p0 + q0 + 5*q1 + 4 ) >> 3 ) ...(37)

[0384] The chromatic difference components used in the filtering process of the OF-2 symmetric filter in equation (37) consist of four pixels: pixels p0 to p1 and q0 to q1. Furthermore, the pixels targeted for filtering of the chromatic difference components are the same four pixels: pixels p0 to p1 and q0 to q1.

[0385] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the OF minus 2 symmetric filter requires a storage capacity in the line buffer 330 equivalent to two pixel lines, specifically pixels p0 or p1, of block Bp above the horizontal boundary.

[0386] The OF-2 symmetric filter in equation (37) is an equation that omits the first equation for finding pixel p2' and the sixth equation for finding pixel q2' in equation (33).

[0387] Furthermore, the OF-2 symmetric filter in equation (37) is a filter in which pixels p2 and p3 in the equations for calculating pixels p1', p0', and q0' in equation (33) are replaced with pixel p1 on the horizontal boundary side. As a result, the tap coefficient of pixel p1 in the equation for calculating pixel p1' in equation (37) is changed from 2 in equation (33) to 5 (=2+1+2). Also, the tap coefficient of pixel p1 in the equation for calculating pixel p0' in equation (37) is changed from 1 in equation (33) to 3 (=1+1+1), and the tap coefficient of pixel p1 in the equation for calculating pixel q0' in equation (37) is changed from 1 in equation (33) to 2 (=1+1).

[0388] Furthermore, the OF-2 symmetric filter in equation (37) is a filter in which pixels q2 and q3 in the equation for calculating pixels q0' and q1' in equation (33) are replaced with pixel q1 on the horizontal boundary side. As a result, the tap coefficient of pixel q1 in the equation for calculating pixel q0' in equation (37) is changed from 1 to 3 (=1+1+1) in equation (33). Also, the tap coefficient of pixel q1 in the equation for calculating pixel q1' in equation (37) is changed from 1 to 3 (=1+1+1) in equation (33), and the tap coefficient of pixel p1 in the equation for calculating pixel q0' in equation (37) is changed from 2 to 5 (=2+1+2) in equation (33).

[0389] Based on the above, when an OF-2 symmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0390] Here, when an OF-2 symmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p1 of block Bp and pixels q0 to q1 of block Bq, and are symmetric with respect to the horizontal boundary (the OF-2 symmetric filter is a symmetric filter).

[0391] <When using a filter based on filter Y2 as filter NC1>

[0392] Figure 19 shows the filter NC1 and the required pixels when a filter based on filter Y2 is adopted as filter NC1.

[0393] Filters NC1 based on filter Y2 include the Y2 normal filter, the Y2-1 asymmetric filter, and the Y2-1 symmetric filter.

[0394] When a Y2 normal filter is used as filter NC1, the filter application determination is performed in the same way as for a Y1 normal filter, that is, in the same way as the luminance component application determination in equation (20).

[0395] Therefore, the pixels used to determine the application of the color difference component for filter application are the same six pixels, p0 to p2 and q0 to q2, as for the Y1 normal filter.

[0396] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary, for the filter application determination.

[0397] When a Y2 normal filter is adopted as filter NC1, the filter type determination is performed in accordance with the standard weakness determination, which determines whether or not to apply a weakness filter to the luminance component of HEVC.

[0398] Here, the standard week determination checks whether equation (38), and optionally equations (39) and (40), are satisfied.

[0399] | (9*(q0,i - p0,i)-3*(q1,i - p1,i)+8) >> 4 | < tc*10 ...(38) |p2,0 - 2*p1,0 + p0,0|+ |p2,3 - 2*p1,3 + p0,3| < (beta+(beta>>1)) >> 3 ...(39) |q2,0 - 2*q1,0 + q0,0|+ |q2,3 - 2*q1,3 + q0,3| < (beta+(beta>>1)) >> 3 ...(40)

[0400] In the standard week determination, first, a week on / off determination is performed for each line (column) of blocks Bp and Bq to determine whether equation (38) is satisfied. Then, if there is a line that satisfies equation (38) in the week on / off determination, a p1 determination to determine whether equation (39) is satisfied and a q1 determination to determine whether equation (40) is satisfied are performed using the first and fourth lines.

[0401] In the standard weakness determination, if the p1 determination in equation (39) is satisfied, then the pixel p of block Bp of the line that satisfies the weakness on / off determination in equation (38) i However, it is set to a pixel that can be the target pixel. Similarly, if the q1 judgment in equation (40) is satisfied, the pixel q of block Bq of the line that satisfies the weak on / off judgment in equation (38) i However, it is set to a pixel that can become the target pixel.

[0402] The filter type determination for the Y2 normal filter is performed in accordance with the standard weakness determination, which determines whether or not to apply a weakness filter to the luminance component of HEVC.

[0403] In other words, the filter type determination for the Y2 normal filter checks whether equation (41) is satisfied, and, if necessary, whether equations (39) and (40) are also satisfied.

[0404] | (9*(q0 - p0)-3*(q1 - p1)+8) >> 4 | < tc*10 ...(41) |p2 - 2*p1 + p0| < (beta+(beta>>1)) >> 4 ...(42) |q2 - 2*q1 + q0| < (beta+(beta>>1)) >> 4 ...(43)

[0405] In determining the filter type of the Y2 normal filter, first, a weak on / off determination is performed on each of the two lines (columns) of pixels for the color difference components of blocks Bp and Bq to determine whether equation (41) is satisfied. Then, if a line satisfying equation (41) exists in the weak on / off determination, a p1 determination is performed on one of the two lines of pixels for the color difference components of blocks Bp and Bq (for example, the first line) to determine whether equation (42) is satisfied, and a q1 determination is performed to determine whether equation (43) is satisfied.

[0406] In the filter type determination of the Y2 normal filter, if the p1 determination in equation (42) is satisfied, then the pixel p of block Bp of the line (column) that satisfies the weak on / off determination in equation (41) i However, it is set to a pixel that can be the target pixel. Similarly, if the q1 judgment in equation (43) is satisfied, the pixel q of block Bq of the line that satisfies the weak on / off judgment in equation (41) i However, it is set to a pixel that can become the target pixel.

[0407] In filter NC2, which is based on filter Y2, only pixels that are set to be potential target pixels become target pixels, and filter pixels are determined.

[0408] From equations (41) to (43), the six pixels used to determine the type of the color difference component for determining the filter type of the Y2 normal filter are pixels p0 to p2 and q0 to q2.

[0409] As a result, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity in the line buffer 330 equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0410] As the Y2 normal filter, a weak filter for the luminance component of HEVC can be used. In this case, the Y2 normal filter is expressed by equation (44).

[0411] D = Clip3( -tC, tC, D ), D = ( 9 * ( q0 - p0 ) - 3 * ( q1 - p1 ) + 8 ) >> 4 p0′ = Clip1C( p0 + D ) q0' = Clip1C( q0 - D ) Dp = Clip3( -( tC >> 1 ), tC >> 1, ( ( ( p2 + p0 + 1 ) >> 1 ) - p1 + D ) >> 1 ) p1′ = Clip1C( p1 + Dp ) Dq = Clip3( -( tC >> 1 ), tC >> 1, ( ( ( q2 + q0 + 1 ) >> 1 ) - q1 - D ) >> 1 ) q1' = Clip1C( q1 + Dq ) ...(44)

[0412] Here, if we represent the number of bits in the color difference component as BC, then Clip1C(A) = Clip(0, (1 << BC) - 1, A).

[0413] The chromatic difference components used in the filtering process of the Y2 normal filter in equation (44) consist of six pixels: pixels p0 to p2 and q0 to q2. The target pixels for the chromatic difference components to be filtered are four pixels: pixels p0 to p1 and q0 to q1.

[0414] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the Y2 normal filter requires the storage capacity of the line buffer 330 to be equivalent to three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0415] Based on the above, if a Y2 normal filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of three pixel lines, from pixel p0 to p2, of block Bp above the horizontal boundary.

[0416] When using the Y2-1 asymmetric filter as filter NC1, the filter application determination is made by checking whether equation (27) is satisfied, similar to the Y1-2 asymmetric filter.

[0417] Therefore, the pixels used to determine the application of the Y2-1 asymmetric filter are the five pixels p0 to p1 and q0 to q2, similar to the Y1-2 asymmetric filter.

[0418] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary, for the filter application determination.

[0419] When a Y2-1 asymmetric filter is adopted as filter NC1, the filter type determination checks whether equations (45) to (47) are satisfied, which are obtained by replacing pixel p2 in equations (41) to (43) used for determining the filter type of a Y2 normal filter with pixel p1 on the horizontal boundary side.

[0420] | (9*(q0 - p0)-3*(q1 - p1)+8) >> 4 | < tc*10 ...(45) |p1 - 2*p1 + p0| < (beta+(beta>>1)) >> 4 = |p0 - p1| < (beta+(beta>>1)) >> 4 ...(46) |q2 - 2*q1 + q0| < (beta+(beta>>1)) >> 4 ...(47)

[0421] Equation (45) represents the week on / off determination, and equations (46) and (47) represent the p1 determination and q1 determination, respectively.

[0422] The filter type determination for the Y2-1 asymmetric filter is the same as for the Y2 normal filter, except that equations (45) to (47) are used instead of equations (41) to (43), so the explanation is omitted.

[0423] From equations (45) to (47), the pixels used to determine the type of the color difference component for determining the filter type of the Y2-1 asymmetric filter are the five pixels p0 to p1 and q0 to q2.

[0424] As a result, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity equivalent to two pixel lines, specifically pixel p0 or p1, of block Bp above the horizontal boundary, in the line buffer 330.

[0425] As a Y2-1 asymmetric filter, one can employ a filter represented by equation (48), which is obtained by substituting pixel p2 in equation (44) of the Y2 normal filter with pixel p1 on the horizontal boundary side.

[0426] D = Clip3( -tC, tC, D ), D = ( 9 * ( q0 - p0 ) - 3 * ( q1 - p1 ) + 8 ) >> 4 p0′ = Clip1C( p0 + D ) q0' = Clip1C( q0 - D ) Dp = Clip3( -( tC >> 1 ), tC >> 1, ( ( ( p1 + p0 + 1 ) >> 1 ) - p1 + D ) >> 1 ) p1′ = Clip1C( p1 + Dp ) Dq = Clip3( -( tC >> 1 ), tC >> 1, ( ( ( q2 + q0 + 1 ) >> 1 ) - q1 - D ) >> 1 ) q1' = Clip1C( q1 + Dq ) ...(48)

[0427] The chromatic difference component used in the filtering process of the Y2-1 asymmetric filter in equation (48) consists of five pixels: pixels p0 to p1 and q0 to q2. The target pixels for the chromatic difference component to be filtered are four pixels: pixels p0 to p1 and q0 to q1.

[0428] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the Y2-1 asymmetric filter requires the storage capacity of the line buffer 330 to be equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0429] Based on the above, if a Y2-1 asymmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0430] Here, when a Y2-1 asymmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p1 of block Bp and pixels q0 to q2 of block Bq, and are asymmetric with respect to the horizontal boundary.

[0431] When a Y2-1 symmetric filter is adopted as filter NC1, the filter application determination is made by checking whether equation (30) is satisfied, similar to the Y1-2 symmetric filter.

[0432] Therefore, the pixels used to determine the application of the Y2-1 symmetric filter are the four pixels p0 to p1 and q0 to q1, similar to the Y1-2 symmetric filter.

[0433] As a result, if the horizontal boundary is a CTU boundary, the line buffer 330 requires storage capacity equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary, for the filter application determination.

[0434] When a Y2-1 symmetric filter is adopted as filter NC1, the filter type determination checks whether equations (49) to (51) are satisfied, obtained by replacing pixel p2 in equations (41) to (43) of the filter type determination for a Y2 normal filter with pixel p1 on the horizontal boundary side, and replacing pixel q2 with pixel q1 on the horizontal boundary side.

[0435] | (9*(q0 - p0)-3*(q1 - p1)+8) >> 4 | < tc*10 ...(49) |p1 - 2*p1 + p0| < (beta+(beta>>1)) >> 4 = |p0 - p1| < (beta+(beta>>1)) >> 4 ...(50) |q1 - 2*q1 + q0| < (beta+(beta>>1)) >> 4 = |q0 - q1| < (beta+(beta>>1)) >> 4 ...(51)

[0436] Equation (49) represents the week on / off determination, and equations (50) and (51) represent the p1 determination and q1 determination, respectively.

[0437] The filter type determination for the Y2-1 symmetric filter is the same as for the Y2 normal filter, except that equations (49) to (51) are used instead of equations (41) to (43), so the explanation is omitted.

[0438] From equations (49) to (51), the four pixels used to determine the type of the color difference component for determining the filter type of the Y2-1 symmetric filter are pixels p0 to p1 and q0 to q1.

[0439] As a result, if the horizontal boundary is a CTU boundary, the filter type determination requires a storage capacity equivalent to two pixel lines, specifically pixel p0 or p1, of block Bp above the horizontal boundary, in the line buffer 330.

[0440] As a Y2-1 symmetric filter, one can employ a filter represented by equation (52), which is obtained by replacing pixel p2 in equation (44) of the Y2 normal filter with pixel p1 on the horizontal boundary side, and also replacing pixel q2 with pixel q1 on the horizontal boundary side.

[0441] D = Clip3( -tC, tC, D ), D = ( 9 * ( q0 - p0 ) - 3 * ( q1 - p1 ) + 8 ) >> 4 p0′ = Clip1C( p0 + D ) q0' = Clip1C( q0 - D ) Dp = Clip3( -( tC >> 1 ), tC >> 1, ( ( ( p1 + p0 + 1 ) >> 1 ) - p1 + D ) >> 1 ) p1′ = Clip1C( p1 + Dp ) Dq = Clip3( -( tC >> 1 ), tC >> 1, ( ( ( q1 + q0 + 1 ) >> 1 ) - q1 - D ) >> 1 ) q1' = Clip1C( q1 + Dq ) ...(52)

[0442] The chromatic difference components used in the filtering process of the Y2-1 symmetric filter in equation (52) consist of four pixels: pixels p0 to p1 and q0 to q1. Furthermore, the pixels targeted for filtering of the chromatic difference components are the same four pixels: pixels p0 to p1 and q0 to q1.

[0443] Therefore, if the horizontal boundary is a CTU boundary, the filtering of the Y2-1 symmetric filter requires the storage capacity of the line buffer 330 to be equivalent to two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0444] Based on the above, if a Y2-1 symmetric filter is adopted as filter NC1, the memory capacity of the line buffer 330 will need to be equivalent to the capacity of two pixel lines, from pixel p0 to p1, of block Bp above the horizontal boundary.

[0445] Here, when a Y2-1 symmetric filter is adopted as filter NC1, the pixels used are pixels p0 to p1 of block Bp and pixels q0 to q1 of block Bq, and are symmetric with respect to the horizontal boundary.

[0446] As described above, this technology can provide various filters as a dataframe (DF).

[0447] <How to apply filter NC1>

[0448] Figure 20 shows an example of how to apply filter NC1 to a decoded image.

[0449] Here, "filter characteristics" is a term that comprehensively defines the properties of a filter, such as filter strength, filter constituent pixels, filter coefficient values, filter tap length, and filter shape (symmetric / asymmetric). It refers to the filter properties resulting from the filter parameters used when performing filtering, or from encoding parameters (such as quantization parameters) related to the filter characteristics. Furthermore, changing the filter characteristics means changing the properties of the filter itself, including changing the filter strength, filter constituent pixels, filter coefficient values, filter tap length, and filter shape (symmetric / asymmetric). Changing the filter characteristics also includes changing the filter parameters or encoding parameters (such as quantization parameters) related to the filter characteristics. Furthermore, reducing the filter characteristics (Reduction / Reduce) includes functionally reducing the filter properties such as filter strength, filter constituent pixels, filter coefficient values, filter tap length, and filter shape (symmetric / asymmetric), taking into account implementation costs such as line buffer capacity. Furthermore, reducing filter characteristics (Reduction / Reduce) also includes functionally changing filter parameters, or encoding parameters related to filter characteristics (such as quantization parameters), taking into account implementation costs such as line buffer capacity. In addition, a filter with reduced filter characteristics will be called a Reduced Filter (Reduction Filter or Reduced Filter).

[0450] Figure 20 shows the filter NC1 applied to the CTU boundary, the filter NC1 applied to the internal boundary, the difference in image quality between the CTU boundary and the internal boundary when filter NC1 is applied, and the required memory capacity (Line buffer size) for the line buffer 330.

[0451] Here, the Y1-1 asymmetric filter, Y1-1 symmetric filter, Y1-2 asymmetric filter, and Y1-2 symmetric filter shown in Figure 17 are filters with reduced filter characteristics (filter strength) compared to the Y1 normal filter because they have fewer filter constituent pixels. Therefore, the Y1-1 asymmetric filter, Y1-1 symmetric filter, Y1-2 asymmetric filter, and Y1-2 symmetric filter can be called reduced filters, which have reduced filter characteristics compared to the Y1 normal filter.

[0452] Furthermore, the OF-1 asymmetric filter, OF-1 symmetric filter, OF-2 asymmetric filter, and OF-2 symmetric filter shown in Figure 18 have fewer filter constituent pixels compared to the OF normal filter, resulting in reduced filter characteristics (filter strength). Therefore, the OF-1 asymmetric filter, OF-1 symmetric filter, OF-2 asymmetric filter, and OF-2 symmetric filter can be described as reduced filters with reduced filter characteristics compared to the OF normal filter.

[0453] Furthermore, the Y2-1 asymmetric and Y2-1 symmetric filters shown in Figure 19 have fewer filter constituent pixels compared to the Y2 normal filter, resulting in reduced filter characteristics (filter strength). Therefore, the Y2-1 asymmetric and Y2-1 symmetric filters can be described as reduced filters with reduced filter characteristics compared to the Y2 normal filter.

[0454] For now, the Y1 normal filter, OF normal filter, and Y2 normal filter will be referred to as normal filters. The reduction filters have weaker filter strength compared to normal filters, but because they have fewer filter constituent pixels (on the p side), the memory capacity required for the line buffer 330 is smaller compared to normal filters.

[0455] Incidentally, for large block-sized blocks, such as CTU boundaries and internal boundaries, it is required to apply filter NC1, which has a strong filter strength (filter characteristics), to adequately remove block noise.

[0456] However, if a normal filter with strong filter strength is used for filter NC1, the number of filter constituent pixels (on the p side) of the normal filter increases, which increases the required memory capacity of the line buffer 330 at the CTU boundary.

[0457] Therefore, at the CTU boundary, a reduction filter with fewer filter constituent pixels (on the p side) can be used as filter NC1. In this case, the memory capacity required for the line buffer 330 can be reduced. From the third row onward in Figure 20, a reduction filter is applied at the CTU boundary in order to reduce the memory capacity required for the line buffer 330.

[0458] On the other hand, applying filter NC1 to the internal boundary does not affect the memory capacity required for the line buffer 330. Therefore, a normal filter with a large number of filter constituent pixels but strong filter intensity can be applied as filter NC1 to the internal boundary. In this case, block noise can be sufficiently removed.

[0459] Furthermore, if a reduction filter is applied to the CTU boundary and a normal filter is applied to the internal boundary, a difference in image quality may occur between the CTU boundary and the internal boundary due to the difference in filter strength between the reduction filter and the normal filter.

[0460] Therefore, a reduction filter similar to that used at the CTU boundary can be applied to the internal boundary. In this case, it is possible to suppress the difference in image quality between the CTU boundary and the internal boundary.

[0461] The control unit 340 (Figure 11) controls which of the filters described in Figures 17 to 19 is applied (selected) as filter NC1 at the block boundary.

[0462] For example, if you want to apply a strong DF (Field Filter) on average within a single screen, you can apply different DF designs to the CTU (Center of Tuning) boundary and the internal boundary; that is, apply a reduction filter to the CTU boundary and a normal filter to the internal boundary.

[0463] Furthermore, if you want to apply the DF evenly within a single screen, you can apply the same filter design to both the CTU boundary and the internal boundary; in other words, you can apply the reduction filter to both the CTU boundary and the internal boundary.

[0464] Figure 21 shows another example of how to apply filter NC1 to a decoded image.

[0465] In Figure 21, for example, in the first row, filter NC1 applies the HEVC color difference component filter C1 (Figure 14) to the p-side of the CTU boundary, and the Y1 normal filter is applied to the q-side. Furthermore, filter NC1 applies the Y1 normal filter to both the p-side and q-side of the internal boundary.

[0466] As described above, in this technology, for pixels of chromatic difference components located near the block boundaries of the decoded image, a Y1-1 asymmetric filter (reduced second luminance filter) or a Y2-1 asymmetric filter (reduced first luminance filter) with reduced filter characteristics, which is a stronger filter for the luminance component than a weak filter (first luminance filter) for the luminance component, can be applied as a filter NC1 (second chromatic difference filter) with stronger filter strength than filter NC2 (first chromatic difference filter).

[0467] Furthermore, in this technology, the second luminance reduction filter can be a vertical second luminance reduction filter that performs the second luminance reduction filter vertically, and the second chromatic difference filter can be a vertical second chromatic difference filter that performs the second chromatic difference filter vertically.

[0468] Furthermore, in this technology, the second luminance reduction vertical filter can be a filter in which the filter coefficients or clipping parameters have been changed compared to the second luminance filter.

[0469] Furthermore, in this technology, the second luminance reduction vertical filter can be an asymmetric filter in which the filter characteristics of the filter applied to pixels located above the block boundary are reduced.

[0470] Furthermore, this technology allows the second luminance filter to be a strong filter for the luminance component compliant with the H.265 / HEVC standard.

[0471] Furthermore, in this technology, the first luminance reduction filter can be made into a vertical first luminance reduction filter, which performs the first luminance reduction filter in the vertical direction.

[0472] Furthermore, in this technology, the control unit 340 can control the DF300 (filter unit) so that it applies a second luminance reduction vertical filter as a second color difference vertical filter to the block boundaries of coding tree blocks, which are fixed-size blocks in a sequence unit.

[0473] Furthermore, in this technology, the control unit 340 can control the filter section so that the reduction second luminance vertical filter is applied as a second color difference vertical filter to the block boundaries of the blocks separated from the coding tree block.

[0474] Furthermore, in this technology, the control unit 340 can control the filter section so that a second vertical luminance filter, which applies a second luminance filter vertically to the block boundary of a block divided from a coding tree block, is applied as a second vertical chromatic difference filter.

[0475] <Description of a computer using this technology>

[0476] Next, the series of processes described above can be performed by hardware or by software. When the series of processes are performed by software, the programs that make up that software are installed on a general-purpose computer or the like.

[0477] Figure 22 is a block diagram showing an example configuration of one embodiment of a computer on which the program that performs the series of processes described above is installed.

[0478] The program can be pre-recorded on the hard disk 905 or ROM 903, which are recording media built into the computer.

[0479] Alternatively, the program can be stored (recorded) on a removable recording medium 911 driven by the drive 909. Such a removable recording medium 911 can be provided as so-called packaged software. Examples of removable recording media 911 include flexible disks, CD-ROMs (Compact Disc Read Only Memory), MO (Magneto Optical) disks, DVDs (Digital Versatile Discs), magnetic disks, semiconductor memory, etc.

[0480] In addition to installing the program from the removable storage medium 911 as described above, the program can also be downloaded to the computer via a communication network or broadcasting network and installed on the built-in hard disk 905. That is, the program can be transferred wirelessly to the computer from a download site via a satellite for digital satellite broadcasting, or transferred via a wired connection to the computer via a network such as a LAN (Local Area Network) or the Internet.

[0481] The computer has a built-in CPU (Central Processing Unit) 902, and an input / output interface 910 is connected to the CPU 902 via a bus 901.

[0482] When the CPU 902 receives a command from the user via the input / output interface 910, such as by operating the input unit 907, it executes a program stored in the ROM (Read Only Memory) 903 accordingly. Alternatively, the CPU 902 loads a program stored in the hard disk 905 into the RAM (Random Access Memory) 904 and executes it.

[0483] As a result, the CPU 902 performs processing according to the flowchart described above, or processing according to the configuration of the block diagram described above. The CPU 902 then outputs the processing results as needed, for example, via the input / output interface 910 from the output unit 906, or transmits them from the communication unit 908, or records them on the hard disk 905.

[0484] The input section 907 consists of a keyboard, mouse, microphone, etc. The output section 906 consists of an LCD (Liquid Crystal Display), speakers, etc.

[0485] In this specification, the processes performed by a computer according to a program do not necessarily have to be performed chronologically in the order described in the flowchart. That is, the processes performed by a computer according to a program include processes that are executed in parallel or individually (e.g., parallel processing or object-based processing).

[0486] Furthermore, the program may be processed by a single computer (processor), or it may be processed in a distributed manner by multiple computers. Moreover, the program may be transferred to a remote computer for execution.

[0487] Furthermore, in this specification, a system means a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are located in the same enclosure or not. Therefore, multiple devices housed in separate enclosures and connected via a network, and a single device in which multiple modules are housed in one enclosure, are both considered systems. <5. Conclusion> As described above, according to the embodiments of this disclosure, it is possible to apply a deblocking filter more appropriately to the color difference components of the decoded image.

[0488] While preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations may be conceived within the scope of the technical ideas described in the claims, and these will naturally also fall within the technical scope of the present disclosure.

[0489] (Color difference related parameters) For example, in the above embodiment, an example was described in which a flag indicating the presence or absence of a significance coefficient for the color difference component in each TU is used as a color difference related parameter, but this technology is not limited to such an example. For example, the conversion coefficient of the color difference component itself may be included in the color difference related parameter. In such a case, the boundary intensity calculation unit 261 may calculate bS by determining the presence or absence of a significance coefficient for the color difference component in each TU from the conversion coefficient of the color difference component. Also, in relation to the above embodiment, Figure 4 shows an example in which the value of bS differs not only depending on whether conditions B1-Y, B1-U, or B1-V are met, but also depending on whether condition B2 is met. However, as in the alternative example shown in Figure 14, the determination of whether condition B2 is met for both color difference components U and V may be omitted, for example, in order to suppress an increase in processing costs.

[0490] (Large block detection threshold) In the above embodiment, an example was described in which the threshold used for large block determination is 16. However, this technology is not limited to this example, and the threshold may be set to 8 or 32. Furthermore, in the case of the YUV444 format, a threshold higher than or equal to the threshold used for the YUV420 format may be used for large block determination.

[0491] (Strong filter) In the above embodiment, an example was described in which a strong filter expressed by equations (15) to (19) is applied to the chromatic difference component. However, the strong filter applied in this technology is not limited to this example. Any strong filter applied to the chromatic difference component can have a stronger filter strength than a weak filter. For example, the strong filter applied to the chromatic difference component in Non-Patent Literature 1 (the strong filter applied to the luminance component in HEVC) may be applied to the chromatic difference component in this technology.

[0492] (Applicability of this technology) This technology can be applied to any image encoding and decoding scheme. In other words, as long as it does not contradict the technology described above, the specifications for various processes related to image encoding and decoding, such as transformation (inverse transformation), quantization (dequantization), encoding (decoding), and prediction, are arbitrary and not limited to the examples given above. Furthermore, some of these processes may be omitted, as long as they do not contradict the aforementioned technology.

[0493] (block) Furthermore, in this specification, the term "block" (not a block indicating a processing unit) used to describe a sub-region or processing unit of an image (picture) refers to any sub-region within a picture, and its size, shape, and characteristics are not limited unless otherwise specified. For example, "block" includes any sub-region (processing unit) such as TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), transformation block, subblock, macroblock, tile, or slice, as described in the above-mentioned references REF1 to REF3.

[0494] (Processing unit) The data units to which the various types of information described above are set, and the data units targeted by the various processes, are arbitrary and not limited to the examples given above. For example, this information and these processes may be set for each TU (Transform Unit), TB (Transform Block), PU (Prediction Unit), PB (Prediction Block), CU (Coding Unit), LCU (Largest Coding Unit), subblock, block, tile, slice, picture, sequence, or component, or they may target the data of those data units. Of course, these data units can be set for each piece of information or process, and it is not necessary for all information and processes to have the same data unit. The storage location of this information is arbitrary and may be stored in the header or parameter set of the data units mentioned above. It may also be stored in multiple locations.

[0495] Furthermore, in the above embodiment, deblocking filtering is performed on the color difference components in units of two lines, but this technology is not limited to such examples. For example, in the case of the YUV444 format, deblocking filtering may be performed on the color difference components in units of four lines. In such a case, the application necessity determination unit 265 may refer to the first and third lines when determining the condition C3 described above.

[0496] (Control information) Control information relating to the technology described above may be transmitted from the encoding side to the decoding side. For example, control information (e.g., enabled_flag) that controls whether or not to allow (or prohibit) the application of the technology described above may be transmitted. Alternatively, control information indicating the targets to which the technology described above applies (or does not apply) may be transmitted. For example, control information specifying the block size (upper or lower limit, or both), frames, components, or layers to which the technology applies (or to which its application is permitted or prohibited) may be transmitted.

[0497] (Block size information) When specifying the size of a block to which this technology is applied, the block size may be specified not only directly but also indirectly. For example, the block size may be specified using identification information that identifies the size. Alternatively, the block size may be specified by a ratio or difference with the size of a reference block (e.g., LCU or SCU). For example, when transmitting information that specifies the block size as a syntax element, the information that specifies the size indirectly as described above may be used. By doing so, the amount of information can be reduced, and coding efficiency may be improved. Furthermore, this specification of block size also includes specifying a range of block sizes (e.g., specifying a range of acceptable block sizes).

[0498] (others) In this specification, "flag" refers to information used to identify multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the values ​​that this "flag" can take are, for example, two values, 1 / 0, or three or more values. In other words, the number of bits that constitute this "flag" is arbitrary, and can be one bit or multiple bits. Furthermore, identification information (including flags) can be included not only in the form of the identification information itself in the bitstream, but also in the form of differential information of the identification information relative to a certain reference information in the bitstream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also differential information relative to the reference information.

[0499] Furthermore, various types of information (metadata, etc.) related to encoded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the encoded data. Here, the term "associate" means, for example, making it possible to use (link) one piece of data when processing the other. In other words, associated data may be combined into a single piece of data, or they may be individual pieces of data. For example, information associated with encoded data (image) may be transmitted on a different transmission path than the encoded data (image). Also, for example, information associated with encoded data (image) may be recorded on a different recording medium (or a different recording area on the same recording medium) than the encoded data (image). Note that this "association" does not have to be with the entire data, but only with a part of the data. For example, an image and the information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a part within a frame.

[0500] In this specification, terms such as "combine," "multiplex," "add," "integrate," "include," "store," "insert," "insert," and "place" mean combining multiple things into one, such as combining encoded data and metadata into a single data, and represent one method of "associating" as described above.

[0501] This technology can also be implemented as any configuration that makes up a device or system, such as a processor as a system LSI (Large Scale Integration), a module using multiple processors, a unit using multiple modules, or a set with additional functions added to a unit (i.e., a part of the device).

[0502] Furthermore, the embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0503] For example, this technology can be configured as cloud computing, where a single function is shared and processed collaboratively by multiple devices via a network.

[0504] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0505] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices.

[0506] Furthermore, the effects described herein are merely illustrative and not limiting, and other effects may also occur. [Explanation of Symbols]

[0507] 10 Image encoding device, 16 Lossless encoding unit, 26 Deblocking filter (DF), 60 Image decoding device, 62 Lossless decoding unit, 261 Boundary intensity calculation unit, 263 Determination unit, 265 Application necessity determination unit, 267 Filter intensity determination unit, 269 Filtering unit, 300 DF, 310 Determination unit, 311 Application necessity determination unit, 312 Filter intensity determination unit, 320 Filtering unit, 330 Line buffer, 340 Control unit, 901 Bus, 902 CPU, 903 ROM, 904 RAM, 905 Hard disk, 906 Output unit, 907 Input unit, 908 Communication unit, 909 Drive, 910 Input / Output interface, 911 Removable recording medium

Claims

1. A decoding unit that decodes a bitstream and generates a decoded image, A filter unit applies a third color difference filter, which has a stronger filter strength than the first color difference filter applied to pixels of color difference components located near block boundaries in the decoded image decoded by the decoding unit, and has a different filter design from the second color difference filter, which is applied to pixels of color difference components located near block boundaries within the CTU (Coding Tree Unit), to pixels of color difference components located near block boundaries in the CTU. An image processing device equipped with the following features.

2. The filter unit performs the filtering operation of the third color difference filter on pixels of color difference components located near the block boundary of the CTU by padding during the filtering operation of the second color difference filter. The image processing apparatus according to claim 1.

3. The filter unit performs the filtering operation of the third color difference filter on pixels of color difference components located near the block boundary of the CTU by replacing the pixels of the color difference component furthest from the block boundary of the CTU by padding during the filtering operation of the second color difference filter. The image processing apparatus according to claim 2.

4. The third color difference filter is a filter whose filter design differs from that of the second color difference filter in terms of the filter operation or the tap length of the filter. The image processing apparatus according to claim 1.

5. The third color difference filter is a filter whose filter operation or filter tap is asymmetrical with respect to the second color difference filter, as a filter design. The image processing apparatus according to claim 4.

6. Decoding a bitstream to generate a decoded image, A third color difference filter, which has a stronger filter strength than the first color difference filter applied to pixels of color difference components located near block boundaries in the decoded image, and has a different filter design from the second color difference filter applied to pixels of color difference components located near block boundaries within the CTU (Coding Tree Unit), is applied to pixels of color difference components located near block boundaries in the CTU. Image processing methods including [specific details omitted].

7. In the filtering operation of the second color difference filter, the pixels of the color difference component located near the block boundary of the CTU are replaced by padding, thereby performing the filtering operation of the third color difference filter on the pixels of the color difference component located near the block boundary of the CTU. The image processing method according to claim 6.

8. In the filtering operation of the second color difference filter, the pixels of the color difference component furthest from the block boundary of the CTU are replaced by padding, thereby performing the filtering operation of the third color difference filter on the pixels of the color difference component located near the block boundary of the CTU. The image processing method according to claim 7.

9. The third color difference filter is a filter whose filter design differs from that of the second color difference filter in terms of the filter operation or the tap length of the filter. The image processing method according to claim 6.

10. The third color difference filter is a filter whose filter operation or filter tap is asymmetrical with respect to the second color difference filter, as a filter design. The image processing method according to claim 9.

11. Decoding a bitstream to generate a decoded image, A third color difference filter, which has a stronger filter strength than the first color difference filter applied to pixels of color difference components located near block boundaries in the decoded image, and has a different filter design from the second color difference filter applied to pixels of color difference components located near block boundaries within the CTU (Coding Tree Unit), is applied to pixels of color difference components located near block boundaries in the CTU. A program that performs a process that includes the following.

12. Decoding a bitstream to generate a decoded image, A third color difference filter, which has a stronger filter strength than the first color difference filter applied to pixels of color difference components located near block boundaries in the decoded image, and has a different filter design from the second color difference filter applied to pixels of color difference components located near block boundaries within the CTU (Coding Tree Unit), is applied to pixels of color difference components located near block boundaries in the CTU. A recording medium on which a program is stored to execute a process that includes [a specific action].

Images