Image processing apparatus and image processing method

Abstract

To provide an image processing device and an image processing method.SOLUTION: An image processing device according to a present disclosure, is configured to adopt a block boundary of a decoded image as a target, and determine a deblocking filter application necessity with respect to color difference components of the decoded image on the basis of boundary strength calculated by using a color difference related parameter related to a color difference component obtained by an orthogonal conversion; and apply a deblocking filter to the color difference components of pixels positioned in a vicinity of the target on the basis of a determination result. Therein: the boundary strength is independently calculated for each of first and second components by independently determining the presence / absence of a significant coefficient to each of the first and second components in the color difference components to a block interposing the block boundary of the calculation target of the boundary strength on the basis of the color difference related parameter; and the deblocking filter application necessity is determined on the basis of a determination based on the boundary strength, and a determination based on a size of a direction orthogonal to the block boundary of the block interposing the block boundary.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus and an image processing method.

Background Art

[0002] In H.265 / HEVC, which is one of the standard specifications of video coding methods, a deblocking filter is applied to the block boundaries of the decoded image in order to suppress the degradation of image quality caused by block distortion that occurs during encoding. In H.265 / HEVC, there are two types of deblocking filters that can be applied to the luminance component, namely a weak filter and a strong filter, while there is only one type of weak filter that can be applied to the chrominance component.

[0003] Also, currently, for the purpose of further improving the coding efficiency compared to H.265 / HEVC, the standardization work of FVC (Future Video Coding), which is a next-generation video coding method, is being promoted by JVET (Joint Video Experts Team), which is a joint standardization organization of ITU-T and ISO / IEC (see, for example, Non-Patent Document 1).

[0004] In the standardization work of FVC, in Non-Patent Document 2 below, a method has been proposed in which the deblocking filter that can be applied to the chrominance component is changed to two types, similar to the deblocking filter that can be applied to the luminance component, and a strong filter can also be applied to the chrominance component.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The necessity of applying the deblocking filter as described above is determined using the boundary strength of the block boundary. The calculation of the boundary strength of the block boundary is performed based on the information of the luminance component without using the information of the chrominance component. However, when determining the necessity of applying the deblocking filter to the chrominance component of the decoded image using the boundary strength specified based on the information of the luminance component, the deblocking filter cannot be applied appropriately, and there is a risk that block distortion remains.

[0007] Therefore, the present disclosure proposes a mechanism that can more appropriately apply the deblocking filter to the chrominance component of the decoded image.

Means for Solving the Problems

[0008] According to the present disclosure, a decoding unit that decodes an encoded stream to generate a decoded image, a determination unit that targets a block boundary of the decoded image and determines whether or not to apply a deblocking filter to a color difference component of the decoded image based on a boundary strength calculated using a color difference related parameter which is information regarding a conversion coefficient of a color difference component obtained by orthogonal transformation, and a filtering unit that applies a deblocking filter to a color difference component of a pixel located in the vicinity of the block boundary based on a determination result of whether or not to apply the deblocking filter, are provided. The boundary strength is independently calculated for each of the first component and the second component by independently determining whether or not a significant coefficient of the first component among the components included in the color difference component exists in a block sandwiching the block boundary which is a calculation target of the boundary strength, and whether or not a significant coefficient of the second component exists, based on the color difference related parameter. The determination unit determines whether or not to apply the deblocking filter by a determination based on the boundary strength and a large block determination based on whether or not a size in the orthogonal direction is larger than 8 among a direction orthogonal to and a direction parallel to the block boundary of the block sandwiching the block boundary. An image processing apparatus is provided.

[0009] Also, according to the present disclosure, decoding an encoded stream to generate a decoded image, targeting a block boundary of the decoded image, and based on a boundary strength calculated using a color difference related parameter which is information regarding a transform coefficient of a color difference component obtained by orthogonal transformation, determining whether to apply a deblocking filter to a color difference component of the decoded image, and based on a determination result of whether to apply the deblocking filter, applying a deblocking filter to a color difference component of a pixel located in the vicinity of the block boundary, wherein the boundary strength is based on the color difference related parameter, and independently determining whether a significant coefficient of a first component among components included in the color difference component exists in blocks sandwiching the block boundary which is a calculation target of the boundary strength, and whether a significant coefficient of a second component exists, and is independently calculated for each of the first component and the second component, and the determining is performed by a determination based on the boundary strength and a large block determination based on whether a size in the orthogonal direction among a direction orthogonal to and a direction parallel to the block boundary of the blocks sandwiching the block boundary is greater than 8, thereby determining whether to apply the deblocking filter, and an image processing method is provided.

[0010] Also, according to the present disclosure, a determination unit that determines whether or not to apply a deblocking filter to a chrominance component of the decoded image, based on a boundary strength calculated using a chrominance-related parameter, which is information regarding a conversion coefficient of a chrominance component obtained by orthogonal transformation, targeting a block boundary of the decoded image decoded by local decoding processing; a filtering unit that applies a deblocking filter to a chrominance component of a pixel located in the vicinity of the block boundary, based on a determination result of whether or not to apply the deblocking filter; and an encoding unit that encodes an image using the decoded image to which the deblocking filter has been applied by the filtering unit. The boundary strength is independently calculated for each of the first component and the second component by independently determining whether or not a significant coefficient of the first component among the components included in the chrominance component and whether or not a significant coefficient of the second component exist in blocks sandwiching the block boundary, which is the calculation target of the boundary strength, based on the chrominance-related parameter. The determination unit determines whether or not to apply the deblocking filter by a determination based on the boundary strength and a large block determination based on whether or not a size in the orthogonal direction among a direction orthogonal to and a direction parallel to the block boundary of the blocks sandwiching the block boundary is greater than 8. An image processing apparatus is provided.

[0011] Also, according to the present disclosure, for the block boundary of the decoded image decoded by the local decoding process, based on the boundary strength calculated using the color difference related parameter which is information on the conversion coefficient of the color difference component obtained by orthogonal transformation, determining whether to apply a deblocking filter to the color difference component of the decoded image, applying a deblocking filter to the color difference component of the pixels located in the vicinity of the block boundary based on the determination result of whether to apply the deblocking filter, and encoding an image using the decoded image to which the deblocking filter has been applied, wherein the boundary strength is based on the color difference related parameter, for the blocks sandwiching the block boundary which is the calculation target of the boundary strength, independently determining whether there is a significant coefficient of a first component among the components included in the color difference component and whether there is a significant coefficient of a second component, and independently calculated for each of the first component and the second component, and the determining is based on a determination based on the boundary strength and a large block determination based on whether the size in the orthogonal direction among the orthogonal direction and the parallel direction to the block boundary of the blocks sandwiching the block boundary is greater than 8, thereby determining whether to apply the deblocking filter, and an image processing method is provided.

Advantages of the Invention

[0012] As described above, according to the present disclosure, it is possible to more appropriately apply a deblocking filter to the color difference component of the decoded image.

[0013] Note that the above effects are not necessarily limited, and together with or instead of the above effects, any of the effects shown in this specification or other effects that can be grasped from this specification may be achieved.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12A

Figure 12B

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0015] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0016] Also, the scope disclosed in this specification is not limited to the content of the embodiments, and the content of the following reference documents REF1 to REF3, which were known at the time of filing, is also incorporated herein by reference. That is, the content described in the following reference documents REF1 to REF3 is also used as a basis for judging support requirements. For example, even if the Quad-Tree Block Structure described in Reference Document REF2 and the QTBT (Quad Tree Plus Binary Tree) Block Structure described in Reference Document REF3 are not directly defined in the detailed description of the invention, they are within the scope of the present disclosure and are considered to meet the support requirements of the claims. Also, for example, with respect to technical terms such as Parsing, Syntax, and Semantics, even if they are not directly defined in the detailed description of the invention, they are within the scope of the present disclosure and are considered to meet the support requirements of the claims. REF1: Recommendation ITU-T H.264 (04 / 2017) “Advanced video coding for generic audiovisual services”, April 2017 REF2: Recommendation ITU-T H.265,(12 / 2016) “High efficiency video coding”, December 2016 REF3: J. Chen, E. Alshina, G. J. Sullivan, J.-R. Ohm, J. Boyce, “Algorithm Description of Joint Exploration Test Model (JEM7)”, JVET-G1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 7th Meeting: Torino, IT, 13-21 July 2017

[0017] In addition, in the following, unless otherwise specified, signals in the YUV420 format will be used as examples for explanation, and the luminance component may be represented as the Y component, and the color difference components may be represented as the U component and the V component. However, the techniques described below can be similarly applied to signals in other formats such as the YUV444 format and the YUV422 format. Also, the expressions of the luminance component and the color difference components vary depending on the signal being targeted. For example, the techniques described below can be similarly applied to signals in which the luminance component and the color difference components are expressed in YCbCr.

[0018] In addition, the following terms used in this specification are defined as follows. The color difference-related parameter means all parameters related to the color difference. For example, the color difference-related parameter may include information related to the conversion coefficient of the color difference component included in each TU (Transform Unit), or a flag indicating the presence or absence of the significant coefficient (non-zero conversion coefficient) of the color difference component in each TU. However, the color difference-related parameter is not limited to such examples and can be various parameters related to the color difference. Whether to apply a deblocking filter means whether to apply the deblocking filter. For example, determining whether to apply a deblocking filter means determining whether to apply the deblocking filter. Also, the determination result of whether to apply a deblocking filter is the result of determining whether to apply the deblocking filter. For example, the determination result may be information indicating either that it should be applied or that it does not need to be applied. Large block determination means determining whether the block to be determined is a large block. In this specification, as will be described later, the block to be determined can be a block sandwiching a block boundary. Also, the large block determination can be performed by comparing the size (block size) of the block with a predetermined threshold. Note that the cases where the large block determination is performed and the details of the large block determination will be described later.

[0019] Note that the description will be made in the following order. 1. Overview 1-1. Existing deblocking filters 1-2. Overview of the technology according to the present disclosure 2. Overview of the apparatus 2-1. Image encoding apparatus 2-2. Image decoding apparatus 3. Deblocking filter 3-1. Configuration example 3-2. Flow of processing 3-3. Modification example 4. Hardware configuration example 5. Conclusion

[0020] <1. Overview> [1-1. Existing methods]

[0021] The processing related to the deblocking filter in existing image coding methods such as HEVC includes a determination process for whether to apply the filter, a determination process for the filter strength, and a filtering process (filter application process). In the following, the processing related to the existing deblocking filter will be described by taking the HEVC deblocking filter as an example. Note that in the following, the deblocking filter for the chrominance component of the decoded image (including the image locally decoded during encoding) will be mainly described, and the description of the deblocking filter for the luminance component of the decoded image will be omitted as appropriate.

[0022] As the processing related to the deblocking filter, first, a determination process for whether to apply the filter is performed. The determination process for whether to apply the filter is a process for determining whether to apply the deblocking filter to the block boundary of the decoded image. Note that in HEVC, the block boundary is specified based on the block structure of the Quad-Tree Block Structure described in Reference REF2. Specifically, among the edges of the 8×8 pixel block (sample grid), which is the smallest block unit, an edge that satisfies the condition of being at least one of the TU (Transform Unit) boundary or the PU (Prediction Unit) boundary is specified as the block boundary in HEVC.

[0023] The determination process for whether to apply the filter is performed based on the boundary strength (hereinafter sometimes referred to as bS) of the block boundary. In HEVC, bS is calculated for every four lines of the specified block boundary. When the block boundary is a vertical boundary, the above lines correspond to the rows orthogonal to the vertical boundary. When the block boundary is a horizontal boundary, the above lines correspond to the columns orthogonal to the horizontal boundary.

[0024] FIG. 1 is a table for explaining the calculation of bS in HEVC. As shown in FIG. 1, in HEVC, bS is calculated based on the truth (satisfied or not satisfied) of condition A which is a condition related to intra prediction, condition B1 which is a condition related to significant coefficients of the Y component, and condition B2 which is a condition related to the motion vector (MV) and the reference picture. Referring to FIG. 1, when condition A is true, bS is set to 2. Also, when condition A is false and at least one of condition B1 and condition B2 is true, bS is set to 1. And when condition A, condition B1, and condition B2 are all false, bS is set to 0. Note that condition A, condition B1, and condition B2 shown in FIG. 1 are the following conditions.

[0025] - Condition A: Among the CUs (Coding Units) that include the pixels of the topmost line among the lines for which bS is calculated and straddle the block boundary, at least one of the coding modes is the intra prediction mode. - Condition B1: The block boundary is the TU boundary, and among the two TUs that include the pixels of the topmost line among the lines for which bS is calculated and straddle the block boundary, there exists a significant coefficient of the Y component in at least one of them. - Condition B2: Between the two CUs that include the pixels of the topmost line among the lines for which bS is calculated and straddle the block boundary, the absolute value of the difference in MV is 1 pixel or more, or the motion compensation reference pictures are different, or the number of MVs is different.

[0026] Furthermore, in HEVC, for the block boundary where the bS set as described above is 1 or more, a deblocking filter for the luminance component (Y component) of the decoded image can be applied. Therefore, in HEVC, depending on whether condition B1 and condition B2 are satisfied or not, the determination result of whether to apply the deblocking filter for the luminance component of the decoded image can be different.

[0027] In HEVC, as a deblocking filter for the luminance component of the decoded image, a strong filter with a large filter strength and a weak filter with a small filter strength are prepared. When bS is 1 or more, for the process related to the deblocking filter for the luminance component of the decoded image, after further application necessity determination processing based on further conditions is performed, filter strength determination processing and filtering processing follow. Details of these processes are described in the above reference document REF2, and the description here is omitted.

[0028] On the other hand, the deblocking filter for the chrominance components (U component, V component) of the decoded image in HEVC is applied only to block boundaries where bS is 2. Therefore, as shown in FIG. 1, whether conditions B1 and B2 are satisfied does not affect the application necessity determination of the deblocking filter for the chrominance components of the decoded image in HEVC.

[0029] Also, in HEVC, the deblocking filter that can be applied to the chrominance components of the decoded image is only the weak filter. Therefore, for the chrominance components of the decoded image, filter strength determination processing is unnecessary, and when bS is 2, a weak filter is applied to the chrominance components of the decoded image.

[0030] By the way, as described in the above reference document REF3, in block division by QTBT Block Structure in FVC, blocks of a larger size can be selected than in block division by Quad-Tree Block Structure in HEVC. When the block size is large in a flat region (a region where the change in pixel values within the region is small), block distortion is likely to occur. Therefore, in FVC where blocks of a larger size can be selected, if the deblocking filter that can be applied to the chrominance components of the decoded image is only the weak filter as in HEVC, there is a risk that significant block distortion will remain in the chrominance components. In view of such a situation, it is desired to improve the deblocking filter for the chrominance components of the decoded image.

[0031] For example, in Non-Patent Document 2, a method is proposed in which the deblocking filter applicable to the chrominance components is changed to two types in the same manner as the deblocking filter applicable to the luminance component, and a strong filter can be applied to the chrominance components as well. Further, in Non-Patent Document 2, it is described that the deblocking filter can be applied to the chrominance components of the decoded image not only when bS is 2 but also when bS is 1.

[0032] FIG. 2 is a table for explaining the calculation of bS in Non-Patent Document 2. As shown in FIG. 2, in Non-Patent Document 2, bS is calculated based on the above-described Conditions A, B1, and B2 in the same manner as the example of HEVC shown in FIG. 2. However, as described above, in Non-Patent Document 2, the deblocking filter can be applied to the chrominance components of the decoded image not only when bS is 2 but also when bS is 1. Therefore, as shown in FIG. 2, in Non-Patent Document 2, depending on whether Conditions B1 and B2 are satisfied, the determination result of whether to apply the deblocking filter to the chrominance components (U component, V component) of the decoded image can be different.

[0033] Hereinafter, with reference to FIG. 3, the determination process for whether to apply the deblocking filter, the determination process for the filter strength, and the filtering process for the deblocking filter applicable to the chrominance components of the decoded image in Non-Patent Document 2 will be described. FIG. 3 is an explanatory diagram showing an example of pixels of the chrominance components (U component, V component) in two adjacent blocks Bp and Bq sandwiching a vertical block boundary BB. Here, the vertical boundary is taken as an example for explanation, but of course, the matters described here are equally applicable to the horizontal boundary. Further, FIG. 3 shows an example in which blocks Bp and Bq are 4×4 in the chrominance components, but the matters described here are equally applicable to blocks of other sizes.

[0034] In the example of FIG. 3, the pixels of the chrominance components in block Bp are p i,jIt is denoted by the symbol. Here, i is the column index and j is the row index. The column index i is numbered 0, 1, 2, 3 in order from the column close to the block boundary BB (from left to right in the figure). The row index j is numbered 0, 1, 2, 3 from top to bottom. On the other hand, the pixels of the color difference component in the block Bq are q k,j It is denoted by the symbol. Here, k is the column index and j is the row index. The column index k is numbered 0, 1, 2, 3 in order from the column close to the block boundary BB (from right to left in the figure).

[0035] After bS is calculated as described with reference to FIG. 2, the applicability determination process and the filter strength determination process are performed using the following three conditions. Such processes are performed for every two lines in the color difference component in the case of the YUV420 format. For example, in the example shown in FIG. 3, the determination regarding line L11 and line L12 and the determination regarding line L21 and line L22 are performed separately. Note that the determination for each line is performed using the pixels of the line to be determined. Hereinafter, the applicability determination process, the filter strength determination process, and the filtering process will be described taking line L11 and line L12 as an example.

[0036] First, in the applicability determination process, it is determined in order whether the following condition C91 and condition C92 are true.

[0037] -Condition C91: (bS == 2 || bS == 1 && (block_width > 16 && block_height > 16)) -Condition C92: d < beta

[0038] Note that in the above condition C91, block_width and block_height are, as shown in FIG. 3, the horizontal size and the vertical size of the block (for example, CU) related to the block boundary to be determined, respectively.

[0039] In addition, the variable beta in the above condition C92 is an edge determination threshold value, and the initial value of the variable beta is given according to the quantization parameter. Also, the value of the variable beta can be specified by the user with the parameter in the slice header. Further, the variable d in the above condition C92 is calculated by the following formulas (1) to (7).

[0040] dp0 = Abs(p 2,0 - 2 * p 1,0 + p 0,0 ) …(1) dp1 = Abs(p 2,1 - 2 * p 1,1 + p 0,1 ) …(2) dq0 = Abs(q 2,0 - 2 * q 1,0 + q 0,0 ) …(3) dq1 = Abs(q 2,1 - 2 * q 1,1 + q 0,1 ) …(4) dpq0 = dp0 + dq0 …(5) dpq1 = dp1 + dq1 …(6) d = dpq0 + dpq1 …(7)

[0041] Note that the above condition C92 is the same as the condition (hereinafter referred to as the condition in the luminance component) used in the determination process of whether to apply the deblocking filter applied to the luminance component in HEVC, except that the referenced line is different. In the condition in the luminance component, the pixels of the first line and the fourth line are referenced, and the determination is made every four lines. On the other hand, in the YUV420 format, since the pixel density of the chrominance components (U component, V component) is half of the pixel density of the luminance component, in the above condition C92, the pixels of the line L11 which is the first line and the pixels of the line L12 which is the second line are referenced, and the determination is made every two lines.

[0042] When at least one of the above conditions C91 and C92 is false, the deblocking filter is not applied to the color difference component of the decoded image. On the other hand, when both of the above conditions C91 and C92 are true, the process proceeds to the filter strength determination process.

[0043] In the filter strength determination process, in order to determine which filter to apply between the strong filter and the weak filter, it is determined whether the following condition C93 is true.

[0044] -Condition C93: (block_width>16&&block_height>16)

[0045] Note that block_width and block_height in the above condition C93 are the horizontal size and vertical size of the block related to the block boundary to be determined, respectively, similar to block_width and block_height in condition C91.

[0046] When the above condition C93 is true, a strong filter is applied to the color difference component of the decoded image at the target block boundary. When the above condition C93 is false, a weak filter is applied to the color difference component of the decoded image at the target block boundary.

[0047] The strong filter applied to the color difference component in Non-Patent Document 2 is the same as the strong filter applied to the luminance component in HEVC, and is represented as follows in formulas (8) to (13).

[0048] p 0 ′=Clip3(p 0 -2*tc,p 0 +2*t C ,(p 2 +2*p 1 +2*p 0 +2*q 0 +q 1 +4)>>3) …(8) p 1 ′=Clip3(p1 -2*tc,p 1 +2*t C ,(p 2 +p 1 +p 0 +q 0 +2)>>2) …(9) p 2 ′=Clip3(p 2 -2*tc,p 2 +2*t C ,(2*p 3 +3*p 2 +p 1 +p 0 +q 0 +4)>>3) …(10) q 0 ′=Clip3(q 0 -2*tc,q 0 +2*t C ,(p 1 +2p 0 +2q 0 +2q 1 +q 2 +4)>>3) …(11) q 1 ′=Clip3(q 1 -2*tc,q 1 +2*t C ,(p 0 +q 0 +q 1 +q 2 +2)>>2) …(12) q 2 ′=Clip3(q 2 -2*t c ,q 2 +2*t C ,(p 0 +q 0 +q 1 +3*q 2 +2*q 3 +4)>>3) …(13)

[0049] Note that in the above formulas (8) to (13), p i , and q k are the pixel values of the color difference components before applying the deblocking filter. Also, p i ′, and q k′ is the pixel value of the color difference component after applying the deblocking filter. Here, i and k are the indexes of the columns in the block Bp and block Bq described above, respectively. In equations (8) to (13), the index of the row is omitted. Also, t C is a parameter given according to the quantization parameter. Also, Clip3(a, b, c) represents a clipping process that clips the value c within the range of a ≤ c ≤ b.

[0050] Since the weak filter applied to the color difference component in Non-Patent Document 2 is the same as the weak filter applied to the color difference component in HEVC, the description here is omitted.

[0051] As described above, the processing related to the deblocking filter that can be applied to the color difference component of the decoded image in Non-Patent Document 2 has been explained. According to the method described above, it is possible to apply a strong filter not only to the luminance component but also to the color difference component according to the conditions.

[0052] However, as described with reference to FIG. 2, the condition B1 used for calculating bS in Non-Patent Document 2 depends on the presence or absence of significant coefficients of the luminance component (Y component) as in the case of HEVC, and information on the color difference components (U component, V component) is not used even including other conditions. However, the spatial pattern of the luminance component and the spatial pattern of each color difference component do not necessarily match. Therefore, if the determination of whether to apply the deblocking filter to the color difference component is made according to the conditions based on the information of the luminance component, there is a risk that the deblocking filter may not be appropriately applied despite the occurrence of block distortion, and the block distortion may remain.

[0053] Also, when bS is 1, for condition C91 used in the applicability determination process in Non-Patent Document 2 to be true, it is necessary that both the horizontal size and the vertical size of the block applied to the block boundary to be determined are larger than 16. However, as described in Reference Document REF3, the shape of a block (e.g., CU) in FVC can be not only a square but also a non-square rectangle. And block distortion tends to occur more easily depending on the size in the direction orthogonal to the block boundary than the size in the same direction as the block boundary. Therefore, depending on the shape of the block, in the applicability determination process of Non-Patent Document 2, the deblocking filter may not be appropriately applied, and there was a risk that block distortion would remain.

[0054] Also, the strong filter in Non-Patent Document 2 is the same as the strong filter applied in HEVC. On the other hand, as described above, in FVC, blocks of a larger size than the block partitioning in HEVC can be selected. Therefore, even if the strong filter in Non-Patent Document 2 is applied, there was a risk that block distortion could not be sufficiently reduced.

[0055] [1-2. Outline of an Embodiment of the Present Disclosure] Therefore, an embodiment of the present disclosure has been created focusing on the above circumstances. An image processing apparatus according to an embodiment of the present disclosure performs an applicability determination process for determining whether to apply a deblocking filter to a color difference component of a decoded image based on a boundary strength (bS) calculated using a color difference related parameter related to the color difference of the decoded image. Hereinafter, an outline of an embodiment of the present disclosure will be described.

[0056] FIG. 4 is a table for explaining the calculation of bS in the present embodiment. As shown in FIG. 4, it is calculated based on condition A which is a condition related to intra prediction, condition B1-Y which is a condition related to the significant coefficient of the Y component, condition B1-U which is a condition related to the significant coefficient of the U component, condition B1-V which is a condition related to the significant coefficient of the V component, and condition B2 which is a condition related to MV and the reference picture.

[0057] Referring to FIG. 4, when condition A is true, bS is set to 16. Also, when condition A is false and condition B2 is true, bS is set to 1. And when condition A and condition B2 are false and any one of condition B1-Y, condition B1-U, and condition B1-V is true, bS is set to a value between 2 and 14. And when condition A, condition B1-Y, condition B1-U, condition B1-V, and condition B2 are all false, bS is set to 0. Note that condition A, condition B1-Y, and condition B2 shown in FIG. 4 are the same as condition A, condition B1, and condition B2 described with reference to FIG. 1, respectively. Also, the method for calculating bS according to this embodiment will be described in more detail later.

[0058] Also, condition B1-U and condition B1-V shown in FIG. 4 correspond to conditions for determining the presence or absence of a significant coefficient of the U component and the presence or absence of a significant coefficient of the V component, respectively, instead of the presence or absence of a significant coefficient of the Y component in condition B1-Y, and are expressed as follows. Note that the truth or falsehood of condition B1-U and condition B1-V below can be determined based on a flag (an example of a color difference-related parameter) indicating the presence or absence of a significant coefficient of the color difference component in each TU.

[0059] - Condition B1-U: The block boundary is a TU boundary, and among the two TUs sandwiching the block boundary including the pixels of the topmost line among the lines for which bS is calculated, at least one of them has a significant coefficient of the U component. - Condition B1-V: The block boundary is a TU boundary, and among the two TUs sandwiching the block boundary including the pixels of the topmost line among the lines for which bS is calculated, at least one of them has a significant coefficient of the V component.

[0060] In this embodiment, based on bS calculated using the above-described conditions B1-U and B1-V related to the color difference, it is determined whether or not to apply a deblocking filter to the color difference components of the decoded image. With such a configuration, it becomes possible to more appropriately apply a deblocking filter to the color difference components.

[0061] In addition, in the present embodiment, as will be described later, based on the size in the direction orthogonal to the block boundary, it is determined whether or not to apply a deblocking filter to the color difference component of the decoded image. With such a configuration, even when the shape of the block is a non-square rectangle, it is possible to more appropriately apply the deblocking filter.

[0062] Further, in the present embodiment, as will be described later, a strong filter having a greater strength (stronger low-pass characteristic) than the strong filter in Non-Patent Document 2 can be applied to the color difference component of the decoded image. Also, in order to more appropriately apply such a strong filter, in the present embodiment, the filter strength is determined by a method different from the filter strength determination process in Non-Patent Document 2. With such a configuration, block distortion can be further reduced.

[0063] The outline of one embodiment of the present disclosure has been described above. Hereinafter, the configuration and operation of the present embodiment for realizing the above-described effects will be sequentially described in detail.

[0064] <2. Schematic Configuration of Apparatus> First, with reference to FIGS. 5 and 6, the schematic configuration of an apparatus as an example to which the technology disclosed in this specification can be applied will be described. The technology disclosed in this specification is applicable to, for example, an image encoding apparatus and an image decoding apparatus.

[0065] [2-1. Image Encoding Apparatus] FIG. 5 is a block diagram showing an example of the configuration of an image encoding apparatus 10 which is an aspect of an image processing apparatus according to an embodiment of the present disclosure.

[0066] Referring to FIG. 5, the image encoding apparatus 10 includes a rearrangement buffer 11, a control unit 12, a subtraction unit 13, an orthogonal transformation unit 14, a quantization unit 15, a reversible encoding unit 16, an accumulation buffer 17, an inverse quantization unit 21, an inverse orthogonal transformation unit 22, an addition unit 23, an in-loop filter 24, a frame memory 30, a switch 31, a mode setting unit 32, an intra prediction unit 40, and an inter prediction unit 50.

[0067] The rearrangement buffer 11 rearranges a series of images constituting the video to be encoded according to the GOP (Group of Pictures) structure to which the encoding process is applied. The rearrangement buffer 11 outputs the rearranged images to the control unit 12, the subtraction unit 13, the intra prediction unit 40, and the inter prediction unit 50.

[0068] The control unit 12 divides an image into blocks of a processing unit based on an external or a block size of a specified processing unit in advance. By the block division by the control unit 12, a CU of a Quad-Tree Block Structure or a QTBT (Quad Tree Plus Binary Tree) Block Structure may be formed as a processing unit. Further, the control unit 12 determines parameters related to the encoding process based on, for example, RDO (Rate-Distortion Optimization). The determined parameters are supplied to each unit.

[0069] The subtraction unit 13 calculates a prediction error, which is a difference between the image input from the rearrangement buffer 11 and the prediction image, and outputs the calculated prediction error to the orthogonal transformation unit 14.

[0070] The orthogonal transformation unit 14 executes an orthogonal transformation process for each of one or more transformation blocks (TUs) set in each region. The orthogonal transformation here may be, for example, a discrete cosine transform or a discrete sine transform. More specifically, the orthogonal transformation unit 14 converts the prediction error input from the subtraction unit 13 into conversion coefficients in the frequency domain from the image signal in the spatial domain for each transformation block. Then, the orthogonal transformation unit 14 outputs the conversion coefficients to the quantization unit 15.

[0071] Further, the orthogonal transformation unit 14 may generate, for each component (for each of the Y component, U component, and V component), a flag indicating the presence or absence of significant coefficients in each TU based on the transformation coefficients obtained by orthogonal transformation, and output the flag to the reversible coding unit 16 and the in-loop filter 24. Note that the flag indicating the presence or absence of significant coefficients of the U component in each TU and the flag indicating the presence or absence of significant coefficients of the V component in each TU, which are generated by the orthogonal transformation unit 14, are included in the color difference related parameters.

[0072] The quantization unit 15 is supplied with the transformation coefficients input from the orthogonal transformation unit 14 and the rate control signal from the rate control unit 18 described later. The quantization unit 15 quantizes the transformation coefficients and outputs the quantized transformation coefficients (hereinafter also referred to as quantization data) to the reversible coding unit 16 and the inverse quantization unit 21. Further, the quantization unit 15 changes the bit rate of the quantization data input to the reversible coding unit 16 by switching the quantization scale based on the rate control signal from the rate control unit 18.

[0073] The reversible coding unit 16 generates a coded stream by coding the quantization data input from the quantization unit 15. Further, the reversible coding unit 16 codes various parameters referred to by the decoder and inserts the coded parameters into the coded stream. The parameters coded by the reversible coding unit 16 may include the parameters determined by the control unit 12 described above.

[0074] Furthermore, the parameters coded by the reversible coding unit 16 may include color difference related parameters. The color difference related parameters coded by the reversible coding unit 16 include, for example, the flag indicating the presence or absence of significant coefficients of the U component in each TU and the flag indicating the presence or absence of significant coefficients of the V component in each TU, which are input from the orthogonal transformation unit 14 as described above. The reversible coding unit 16 outputs the generated coded stream to the accumulation buffer 17.

[0075] The accumulation buffer 17 temporarily accumulates the encoded stream input from the reversible encoding unit 16 using a storage medium such as a semiconductor memory. Then, the accumulation buffer 17 outputs the accumulated encoded stream to a transmission unit (e.g., a communication interface or a connection interface to a peripheral device, etc., not shown) at a rate corresponding to the bandwidth of the transmission path.

[0076] The rate control unit 18 monitors the free capacity of the accumulation buffer 17. Then, the rate control unit 18 generates a rate control signal according to the free capacity of the accumulation buffer 17 and outputs the generated rate control signal to the quantization unit 15. For example, when the free capacity of the accumulation buffer 17 is small, the rate control unit 18 generates a rate control signal for reducing the bit rate of the quantized data. Also, for example, when the free capacity of the accumulation buffer 17 is sufficiently large, the rate control unit 18 generates a rate control signal for increasing the bit rate of the quantized data.

[0077] The inverse quantization unit 21, the inverse orthogonal transformation unit 22, and the addition unit 23 constitute a local decoder. The local decoder has the role of locally decoding a decoded image from the encoded data.

[0078] The inverse quantization unit 21 inverse quantizes the quantized data with the same quantization parameters as those used by the quantization unit 15 and restores the transform coefficients. Then, the inverse quantization unit 21 outputs the restored transform coefficients to the inverse orthogonal transformation unit 22.

[0079] The inverse orthogonal transformation unit 22 restores the prediction error by performing an inverse orthogonal transformation process on the transform coefficients input from the inverse quantization unit 21. Then, the inverse orthogonal transformation unit 22 outputs the restored prediction error to the addition unit 23.

[0080] The addition unit 23 generates a decoded image (reconstructed image) by adding the restored prediction error input from the inverse orthogonal transformation unit 22 and the predicted image input from the intra prediction unit 40 or the inter prediction unit 50. Then, the addition unit 23 outputs the generated decoded image to the in-loop filter 24 and the frame memory 30.

[0081] The in-loop filter 24 applies a series of in-loop filters for the purpose of improving the image quality of the decoded image. For example, as described in "2.5. In-loop filtering" of Reference REF3, four in-loop filters may be applied in the order of a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter. The in-loop filter 24 shown in FIG. 5 includes, for example, a bilateral filter 25, a deblocking filter 26a, an adaptive offset filter 27, and an adaptive loop filter 28, and the above four in-loop filters may be applied in order. However, the in-loop filter 24 is not limited to such a configuration, and it is possible to appropriately select which of the four in-loop filters to apply and in what order. Note that the deblocking filter 26a will be described in detail later.

[0082] The in-loop filter 24 outputs the decoded image to which the in-loop filter has been applied to the frame memory 30.

[0083] The frame memory 30 stores, using a storage medium, the decoded image before filtering input from the addition unit 23 and the decoded image to which the in-loop filter has been applied input from the in-loop filter 24.

[0084] The switch 31 reads out the decoded image before filtering used for intra prediction from the frame memory 30 and supplies the read decoded image to the intra prediction unit 40 as a reference image. Also, the switch 31 reads out the decoded image after filtering used for inter prediction from the frame memory 30 and supplies the read decoded image to the inter prediction unit 50 as a reference image.

[0085] The mode setting unit 32 sets a prediction coding mode for each block based on the comparison of the costs input from the intra prediction unit 40 and the inter prediction unit 50. For a block in which the intra prediction mode is set, the mode setting unit 32 outputs the prediction image generated by the intra prediction unit 40 to the subtraction unit 13 and the addition unit 23, and outputs information related to the intra prediction to the reversible coding unit 16. Also, for a block in which the inter prediction mode is set, the mode setting unit 32 outputs the prediction image generated by the inter prediction unit 50 to the subtraction unit 13 and the addition unit 23, and outputs information related to the inter prediction to the reversible coding unit 16.

[0086] The intra prediction unit 40 executes intra prediction processing based on the original image and the decoded image. For example, the intra prediction unit 40 evaluates the cost based on the prediction error and the amount of code generated for each of the prediction mode candidates included in the search range. Next, the intra prediction unit 40 selects the prediction mode with the minimum cost as the optimal prediction mode. Also, the intra prediction unit 40 generates a prediction image according to the selected optimal prediction mode. Then, the intra prediction unit 40 outputs information related to the intra prediction including the prediction mode information indicating the optimal prediction mode, the corresponding cost, and the prediction image to the mode setting unit 32.

[0087] The inter prediction unit 50 executes inter prediction processing (motion compensation) based on the original image and the decoded image. For example, the inter prediction unit 50 evaluates the cost based on the prediction error and the amount of code generated for each of the prediction mode candidates included in a certain search range. Next, the inter prediction unit 50 selects the prediction mode with the minimum cost, that is, the prediction mode with the highest compression rate, as the optimal prediction mode. Also, the inter prediction unit 50 generates a prediction image according to the selected optimal prediction mode. Then, the inter prediction unit 50 outputs information related to the inter prediction, the corresponding cost, and the prediction image to the mode setting unit 32.

[0088] [2-2. Image Decoding Device] Next, the decoding of the data encoded as described above will be explained. FIG. 6 is a block diagram showing an example of the configuration of an image decoding apparatus 60, which is an aspect of the image processing apparatus according to the present embodiment. Referring to FIG. 6, it includes an accumulation buffer 61, a reversible decoding unit 62, an inverse quantization unit 63, an inverse orthogonal transformation unit 64, an addition unit 65, an in-loop filter 66, a rearrangement buffer 72, a D / A (Digital to Analogue) conversion unit 73, a frame memory 80, selectors 81a and 81b, an intra prediction unit 90, and an inter prediction unit 100.

[0089] The accumulation buffer 61 temporarily stores, using a storage medium, the encoded stream received from the image encoding apparatus 10 via a transmission unit (not shown) (for example, a communication interface or a connection interface with a peripheral device, etc.).

[0090] The reversible decoding unit 62 decodes the encoded stream input from the accumulation buffer 61 according to the encoding method used during encoding, and generates quantization data. The reversible decoding unit 62 outputs the generated quantization data to the inverse quantization unit 63.

[0091] Also, the reversible decoding unit 62 parses various parameters from the encoded stream. The parameters parsed by the reversible decoding unit 62 may include, for example, information related to intra prediction and information related to inter prediction. The reversible decoding unit 62 outputs the information related to intra prediction to the intra prediction unit 90. Also, the reversible decoding unit 62 outputs the information related to inter prediction to the inter prediction unit 100.

[0092] Also, the parameters parsed by the reversible decoding unit 62 may include color difference related parameters. The reversible decoding unit 62 outputs the color difference related parameters to the in-loop filter 66. Note that the color difference related parameters parsed by the reversible decoding unit 62 include, for example, a flag indicating the presence or absence of significant coefficients of the U component in each of the above-described TUs, and a flag indicating the presence or absence of significant coefficients of the V component in each of the TUs.

[0093] The inverse quantization unit 63 inverse-quantizes the quantization data input from the reversible decoding unit 62 with the same quantization step as that used during encoding, and restores the transform coefficients. The inverse quantization unit 63 outputs the restored transform coefficients to the inverse orthogonal transform unit 64.

[0094] The inverse orthogonal transform unit 64 performs an inverse orthogonal transform on the transform coefficients input from the inverse quantization unit 63 according to the orthogonal transform method used during encoding, thereby generating a prediction error. The inverse orthogonal transform unit 64 outputs the generated prediction error to the addition unit 65.

[0095] The addition unit 65 generates a decoded image by adding the prediction error input from the inverse orthogonal transform unit 64 and the predicted image input from the selector 71b. Then, the addition unit 65 outputs the generated decoded image to the in-loop filter 66 and the frame memory 80.

[0096] The in-loop filter 66 applies a series of in-loop filters for the purpose of improving the image quality of the decoded image. For example, as described in "2.5. In-loop filtering" of Reference REF3, four in-loop filters may be applied in the order of a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter. The in-loop filter 66 shown in FIG. 6 includes, for example, a bilateral filter 67, a deblocking filter 26b, an adaptive offset filter 69, and an adaptive loop filter 70, and the above four in-loop filters may be applied in sequence. However, the in-loop filter 66 is not limited to such a configuration, and it is possible to appropriately select which of the four in-loop filters to apply and in what order. Note that the deblocking filter 26b will be described in detail later.

[0097] The in-loop filter 66 outputs the decoded image to which the in-loop filter has been applied to the rearrangement buffer 72 and the frame memory 80.

[0098] The rearrangement buffer 72 generates a series of images in time series by rearranging the images input from the in-loop filter 66. Then, the rearrangement buffer 72 outputs the generated images to the D / A conversion unit 73.

[0099] The D / A conversion unit 73 converts the digital-formatted image input from the rearrangement buffer 72 into an analog-formatted image signal. Then, the D / A conversion unit 73 causes a display (not shown) connected to, for example, the image decoding device 60 to display a video by outputting the analog image signal thereto.

[0100] The frame memory 80 stores, using a storage medium, the decoded image before filtering input from the addition unit 65 and the decoded image to which the in-loop filter has been applied and input from the in-loop filter 66.

[0101] The selector 81a switches, for each block in the image, the output destination of the image from the frame memory 80 between the intra prediction unit 90 and the inter prediction unit 100 according to the prediction mode information acquired by the reversible decoding unit 62. For example, when the intra prediction mode is specified, the selector 81a outputs the decoded image before filtering supplied from the frame memory 80 to the intra prediction unit 90 as a reference image. Also, when the inter prediction mode is specified, the selector 81a outputs the decoded image after filtering to the inter prediction unit 100 as a reference image.

[0102] The selector 81b switches, according to the prediction mode information acquired by the reversible decoding unit 62, the output source of the prediction image to be supplied to the addition unit 65 between the intra prediction unit 90 and the inter prediction unit 100. For example, when the intra prediction mode is specified, the selector 81b supplies the prediction image output from the intra prediction unit 90 to the addition unit 65. Also, when the inter prediction mode is specified, the selector 81b supplies the prediction image output from the inter prediction unit 100 to the addition unit 65.

[0103] The intra prediction unit 90 performs intra prediction processing based on the information regarding intra prediction input from the reversible decoding unit 62 and the reference image from the frame memory 80, and generates a prediction image. Then, the intra prediction unit 90 outputs the generated prediction image to the selector 81b.

[0104] The inter prediction unit 100 performs inter prediction processing based on the information regarding inter prediction input from the reversible decoding unit 62 and the reference image from the frame memory 80, and generates a prediction image. Then, the inter prediction unit 100 outputs the generated prediction image to the selector 81b.

[0105] <3. Deblocking Filter> [3-1. Configuration Example of Deblocking Filter] In this section, an example of the configuration of the deblocking filter 26a of the image encoding apparatus 10 shown in FIG. 5 and the deblocking filter 26b of the image decoding apparatus 60 shown in FIG. 6 will be described. Note that the configurations of the deblocking filter 26a and the deblocking filter 26b may be common. Therefore, in the following description, when there is no need to particularly distinguish between the two, the deblocking filter 26a and the deblocking filter 26b are collectively referred to as the deblocking filter 26.

[0106] As described above, the deblocking filter 26 according to the present embodiment determines whether to apply a deblocking filter to the color difference component of the decoded image based on bS calculated using a color difference-related parameter related to the color difference. Also, as described above, the deblocking filter 26 according to the present embodiment determines whether to apply a deblocking filter to the color difference component of the decoded image based further on the size in the direction orthogonal to the block boundary. Further, as described above, the deblocking filter 26 according to the present embodiment can apply a strong filter having a greater strength (stronger low-pass characteristics) than the strong filter in Non-Patent Document 2 to the color difference component of the decoded image. Also, in order to apply such a strong filter more appropriately, in the present embodiment, the filter strength is determined by a method different from the filter strength determination process in Non-Patent Document 2. Note that hereinafter, the function of the deblocking filter 26 regarding the deblocking filter mainly applied to the color difference component of the decoded image will be described, and the function of the deblocking filter 26 regarding the deblocking filter applied to the luminance component will be omitted as appropriate.

[0107] FIG. 7 is a block diagram showing an example of the detailed configuration of the deblocking filter 26 according to the present embodiment. Referring to FIG. 7, the deblocking filter 26 includes a boundary strength calculation unit 261, a determination unit 263, and a filtering unit 269.

[0108] (1) Boundary Strength Calculation Unit The boundary strength calculation unit 261 targets the block boundary of the decoded image and calculates bS (boundary strength) using a color difference-related parameter related to the color difference. When a signal in the YUV420 format is the target, the boundary strength calculation unit 261 calculates bS in units of 4 lines in the luminance component of the decoded image, that is, in units of 2 lines in the color difference component of the decoded image.

[0109] In this embodiment, the color difference related parameters used by the boundary strength calculation unit 261 for calculating bS include a flag indicating the presence or absence of a significant coefficient of the U component in each TU, and a flag indicating the presence or absence of a significant coefficient of the V component in each TU. As shown in FIG. 7, flags indicating the presence or absence of significant coefficients of each component (Y component, U component, V component) in each TU are input to the boundary strength calculation unit 261 from the orthogonal transformation unit 14 or the inverse decoding unit 62.

[0110] The boundary strength calculation unit 261 calculates bS based on condition A, condition B1-Y, condition B1-U, condition B1-V, and condition B2 described with reference to FIG. 4. That is, the boundary strength calculation unit 261 calculates bS based on whether or not a significant coefficient of the color difference component exists in the TUs sandwiching the block boundary that is the calculation target of bS. Further, the boundary strength calculation unit 261 according to this embodiment can calculate bS by independently determining whether or not significant coefficients of each of the Y component, U component, and V component exist in the TUs sandwiching the block boundary that is the calculation target of bS. With such a configuration, as described with reference to FIG. 2, a bS more suitable for the U component and V component is calculated than calculating bS based on whether or not a significant coefficient of the Y component exists, and it becomes possible to apply the deblocking filter more appropriately.

[0111] With reference to FIG. 8, the calculation of bS by the boundary strength calculation unit 261 will be described in more detail. FIG. 8 is a table showing an example of bS calculated by the boundary strength calculation unit 261. The bS calculated by the boundary strength calculation unit 261 can be represented by a plurality of bits. In the example shown in FIG. 8, bS is represented by 5 bits. Also, bS may be calculated such that at least one bit corresponding to each of the Y component, U component, and V component is included in the plurality of bits. With such a configuration, when the determination unit 263 described later determines whether or not to apply the deblocking filter based on bS, it becomes possible to easily perform the determination by referring to the bits of bS corresponding to each component to be determined.

[0112] Further, the boundary strength calculation unit 261 may calculate bS such that each bit included in bS corresponds to the truth or falsehood of each condition. In the example shown in FIG. 8, when each condition is true, the bit corresponding to the condition is 1, and when each condition is false, bS is calculated such that the bit corresponding to the condition is 0. Further, in the example shown in FIG. 8, bS is represented by 5 bits, the fifth bit of bS is condition A related to intra prediction, the fourth bit of bS is condition B1-Y related to the significant coefficient of the Y component, the third bit of bS is condition B1-U related to the significant coefficient of the U component, the second bit of bS is condition B1-V related to the significant coefficient of the V component, and the first bit of bS is condition B2 related to MV and the reference picture, respectively. However, the correspondence between each bit of bS and each condition is not limited to the example shown in FIG. 8. For example, the order of the fourth, third, and second bits of bS corresponding to the Y component, U component, and V component, respectively, may be interchanged.

[0113] (2) Determination unit As shown in FIG. 7, the determination unit 263 includes an application necessity determination unit 265 that determines whether or not to apply a deblocking filter to the color difference component of the decoded image, and a filter strength determination unit 267 that determines the filter strength of the deblocking filter applied to the color difference component of the decoded image. Hereinafter, the functions of the application necessity determination unit 265 and the filter strength determination unit 267 will be sequentially described.

[0114] In the following description, the determination of whether or not to apply a deblocking filter to the color difference component of the decoded image and the filter strength will be mainly described, and the determination for the luminance component will be appropriately omitted. Further, the application necessity determination unit 265 and the filter strength determination unit 267 according to the present embodiment independently determine whether or not to apply a deblocking filter and the filter strength for each of the U component and the V component.

[0115] The application necessity determination unit 265 targets the block boundary of the decoded image and determines whether or not to apply a deblocking filter to the color difference component of the decoded image based on bS (boundary strength) calculated as described above by the boundary strength calculation unit 261.

[0116] Further, the applicability determination unit 265 may further determine whether to apply a deblocking filter to the color difference component of the decoded image based on the block sizes of the blocks sandwiching the block boundary. In the following, the determination based on such block sizes may be referred to as large block determination. Also, the applicability determination unit 265 does not necessarily perform large block determination for all block boundaries, and may determine whether to perform large block determination according to bS. The cases where large block determination is performed and the details of large block determination will be described later.

[0117] The applicability determination unit 265 according to the present embodiment determines whether to apply a deblocking filter by determining the following condition C1 and condition C2.

[0118] -Condition C1: (bS == 16 || (condition C11 && condition C12)) -Condition C2: d < beta

[0119] Condition C11 in the above condition C1 is a condition for determining whether to perform large block determination, and condition C12 is a condition for large block determination. When bS is 16, that is, when the condition A regarding intra prediction is satisfied, condition C1 can be determined to be true without the need to perform large block determination. Therefore, condition C11 for determining whether to perform large block determination can be true when bS has a value related to inter prediction. In this way, by skipping large block determination and determining condition C1 to be true when bS is 16, the processing amount related to large block determination can be suppressed.

[0120] Also, even when condition C11 of condition C1 is false, the determination of condition C12 (large block determination) is not performed, and condition C1 is determined to be false. With such a configuration, the processing amount related to large block determination can be suppressed.

[0121] Condition C11 may be a condition regarding the significant coefficients of each component, or may be true when the above-described condition B2 is true. That is, condition C11 may vary depending on the component to be determined. For example, condition C11 when the component to be determined is the U component may be a condition such as the following condition C11-U, and condition C11 when the component to be determined is the V component may be a condition such as the following condition C11-V.

[0122] - Condition C11-U: (bS & 0x04 || bS & 0x01) - Condition C11-V: (bS & 0x02 || bS & 0x01)

[0123] Further, the application necessity determination unit 265 performs a large block determination based on the size in the direction orthogonal to the block boundary in the blocks sandwiching the block boundary. With such a configuration, when the shape of the block is a non-square rectangle, it becomes possible to determine whether to apply the deblocking filter based on the size in the direction orthogonal to the block boundary, which is likely to affect the occurrence of block distortion.

[0124] Further, the application necessity determination unit 265 may perform a large block determination based on whether the size in the direction orthogonal to the block boundary in the blocks sandwiching the block boundary is greater than a predetermined threshold value. The threshold value used in this large block determination is not limited, but may be 16, for example. When the size in the direction orthogonal to the block boundary is small, particularly when it is 16 or less, block noise is less noticeable, and with such a configuration, it becomes possible to avoid applying an unnecessary deblocking filter. For example, condition C12 regarding the large block determination may be a condition such as the following.

[0125] - Condition C12: (EDGE_VER && block_width > 16) || (EDGE_HOR && block_height > 16)

[0126] In the above condition C12, EDGE_VER means that the block boundary to be determined is a vertical boundary, and EDGE_HOR means that the block boundary to be determined is a horizontal boundary.

[0127] Also, since the above condition C2 is the same as the above-mentioned condition C92, the description here is omitted. Note that the determination of the above condition C2 is performed when condition C1 is true. When condition C1 is false, it is determined that the deblocking filter is not applied without performing the determination of condition C2. The determination of condition C2 requires a process of calculating the variable d as in the above-mentioned formulas (1) to (7), and since the processing amount is larger than the determination of condition C1, it is possible to suppress the processing amount by performing the determination of condition C2 after condition C1.

[0128] Also, after the filter strength determination unit 267 determines whether or not to apply the deblocking filter according to conditions C1 and C2 as described above, the filter strength determination unit 267 further determines the filter strength of the deblocking filter applied to the color difference component of the decoded image. The deblocking filter that can be applied in this embodiment may be two types, a weak filter having a weaker strength and a strong filter having a stronger strength, as will be described later. Then, the filtering unit 269 described later applies either the weak filter or the strong filter according to the filter strength determined by the filter strength determination unit 267.

[0129] When it is determined that the deblocking filter is to be applied, the filter strength determination unit 267 determines the filter strength. By performing the determination of the filter strength after the determination of whether or not to apply the deblocking filter, it is possible to suppress the processing related to the determination of the filter strength.

[0130] Also, the filter strength determination unit 267 determines the filter strength based on the waveform of the color difference component of the pixel located in the vicinity of the block boundary. Hereinafter, the determination based on the waveform will be described. The filter strength determination unit 267 determines the filter strength according to condition C3 based on the following waveform.

[0131] -Condition C3: (Condition C31 && Condition C32 && Condition C33) -Condition C31: |p 3 -p 0 | + |q 3 -q 0 | < (beta >> 3) -Condition C32: |p 2 -2 * p 1 + p 0 | + |q 2 -2 * q 1 + q 0 | < (beta >> 2) -Condition C33: |p 0 -q 0 | < ((t c * 5 + 1) >> 1)

[0132] The filter strength determination unit 267 determines the above Condition C3 for the pixels included in two lines among the pixels located near the block boundary. The Condition C31, Condition C32, and Condition C33 used in the above C3 are determined for each line. Note that for p i , q k , p i ′, q k ′, beta, and t C Since they have already been described above, the description here is omitted.

[0133] Condition C31, Condition C32, and Condition C33 are conditions for determination using the pixels included in each line. More specifically, Condition C31 is a condition regarding the flatness within the block of the color difference components of the pixels included in each line. Also, Condition C32 is a condition regarding the determination of the continuity within the block of the color difference components of the pixels included in each line. Also, Condition C33 is a condition regarding the gap (difference) between the blocks of the color difference components of the pixels included in each line, and more specifically, it is a condition for determining the gap between the blocks using the pixel values adjacent to the block boundary.

[0134] When condition C31 is true, the flatness of the waveform of the color difference component is high within each block. Also, when condition C32 is true, the waveform of the color difference component has high continuity within each block. Further, when condition C32 is true, the waveform of the color difference component has a large gap at the block boundary.

[0135] As described above, condition C3 is determined to be true when all of the above conditions C31, C32, and C33 are true. Also, the filter strength determination unit 267 determines the above condition C3 line by line. However, as described above, the filter strength is determined in units of two lines. That is, when the above condition C3 is true in both of two consecutive lines, a strong filter is applied to the two lines, and when it is false, a weak filter is applied to the two lines, so that the filter strength is determined.

[0136] (3) Filtering unit The filtering unit 269 applies a deblocking filter to the color difference components of the pixels located in the vicinity of the block boundary based on the determination result of whether to apply the deblocking filter by the application necessity determination unit 265. Also, as described above, the filtering unit 269 applies a weak filter or a strong filter as a deblocking filter according to the filter strength determined by the filter strength determination unit 267.

[0137] The weak filter applied to the color difference component by the filtering unit 269 according to the present embodiment may be the same as the weak filter applied to the color difference component of the decoded image in Non-Patent Document 2 or HEVC described above, for example. On the other hand, the strong filter applied to the color difference component in the present embodiment may be different from the strong filter applied to the color difference component in Non-Patent Document 2 (the strong filter applied to the luminance component in HEVC). Hereinafter, an example of the strong filter applied to the color difference component in the present embodiment will be described.

[0138] In this embodiment, the coefficient of the strong filter applied to the color difference component may be 2 at the center position of the application target range of the strong filter and 1 at other positions. Further, the filtering unit 269 may use, as the application target range of the strong filter, the range from both sides of the block boundary to three pixels, and use, as reference pixels, the three pixels on both sides of the center position of the application target range, and apply the strong filter to the color difference components of the pixels included in the application target range. For example, for p 0 The strong filter with p

[0139] as the center position of the application target range is represented by the following formula (14). 0 p 0- ′ = Clip3(p C , p 0 + w * t C , ((p 3 + p 2 + p 1 + 2 * p 0 + q 0 + q 1 + q 2 + 4) >> 3)) …(14)

[0140] Note that in the above formula (14), w is a weight that can be appropriately set, and may be set to 1 or 2, for example. Also, Clip3(a, b, c) represents a clipping process that clips the value c within the range a ≤ c ≤ b as described above.

[0141] By applying such a strong filter, it becomes possible to apply a stronger deblocking filter than the strong filter applied to the color difference component in Non-Patent Document 2 described above.

[0142] Incidentally, when the center position of the application range of the strong filter is the second or third pixel from the block boundary, the reference pixels include pixels that are more than five pixels away from the block boundary. However, pixels that are more than five pixels away from the block boundary are not used for determining the filter strength and may not be suitable for use as reference pixels. Therefore, the filtering unit 269 may use the pixel value of the fourth pixel from the block boundary instead of the pixels that are more than five pixels away from the block boundary as the pixel value of the reference pixel by padding.

[0143] For example, p 1 The strong filter with the center position of the application range is expressed as follows in Equation (15). p 1 ′ = Clip3(p 1- w * t C , p 1 + w * t C , ((p 4 + p 3 + p 2 + 2 * p 1 + p 0 + q 0 + q 1 + 4) >> 3)) = Clip3(p 1- w * t C , p 1 + w * t C , ((p 3 + p 3 + p 2 + 2 * p 1 + p 0 + q 0 + q 1 + 4) >> 3)) = Clip3(p 1- w * t C , p 1 + w * t C , ((2 * p 3 + p 2 + 2 * p 1 + p 0 + q 0 + q 1 + 4) >> 3)) …(15)

[0144] Similarly, p 2The strong filter with the center position of the application target range is expressed as the following formula (16). p 2 ′=Clip3(p 2- w*t C ,p 2 +w*t C ,((p 5 +p 4 +p 3 +2*p 2 +p 1 +p 0 +q 0 +4)>>3)) =Clip3(p 2- w*t C ,p 2 +w*t C ,((p 3 +p 3 +p 3 +2*p 2 +p 1 +p 0 +q 0 +4)>>3)) =Clip3(p 2- w*t C ,p 2 +w*t C ,((3*p 3 +2*p 2 +p 1 +p 0 +q 0 +4)>>3)) …(16)

[0145] Also, similarly, the strong filters with q 0 ~q 3 as the center positions of the application target range are expressed as the following formulas (17) to (19), respectively. q 0 ′=Clip3(q 0 -w*t C ,q 0 +w*t C ,((p 2 +p 1 +p 0 +2*q 0 +q 1 +q 2 +q 3 +4)>>3)) …(17) q1 ′ = Clip3(q 1 - w * t C , q 1 + w * t C , ((p 1 + p 0 + q 0 + 2 * q 1 + q 2 + 2 * q 3 + 4) >> 3)) …(18) q 2 ′ = Clip3(q 2 - w * t C , q 2 + w * t C , ((p 0 + q 0 + q 1 + 2 * q 2 + 3 * q 3 + 4) >> 3)) …(19)

[0146] [3 - 2. Processing flow] The configuration example of the deblocking filter 26 according to the present embodiment has been described above. Subsequently, the processing flow by the deblocking filter 26 according to the present embodiment will be described. FIG. 9 is a flowchart showing an example of the processing flow by the deblocking filter 26 according to the present embodiment. Hereinafter, among the processes by the deblocking filter 26, the processes related to the features of the present embodiment will be described, and the description of other processes will be omitted as appropriate.

[0147] First, the boundary strength calculation unit 261 calculates bS (boundary strength) (S10). Here, with reference to FIG. 10, the calculation method of bS will be described in more detail. FIG. 10 is a flowchart for explaining the flow of the boundary strength calculation process (S10) executed by the boundary strength calculation unit 261.

[0148] First, the boundary strength calculation unit 261 initializes bS to 0 (S102). Subsequently, the boundary strength calculation unit 261 determines the truth or falsehood of condition A, which is a condition related to intra prediction (S104). If condition A is true (YES in S104), bS is set to 16 (S106).

[0149] On the other hand, when condition A is false (NO in S104), the boundary strength calculation unit 261 determines the truth or falsehood of condition B2, which is a condition related to the motion vector (MV) and the reference picture (S108). When condition B2 is true (YES in S108), bS is set to 1 (S110).

[0150] On the other hand, when condition B2 is false (NO in S108), the boundary strength calculation unit 261 determines the truth or falsehood of condition B1-Y, which is a condition related to the presence or absence of a significant coefficient of the Y component (S112). When condition B1-Y is true (YES in S112), after 8 is added to bS (S114), the process proceeds to step S116. On the other hand, when condition B1-Y is false (NO in S112), the process directly proceeds to step S116.

[0151] In step S116, the boundary strength calculation unit 261 determines the truth or falsehood of condition B1-U, which is a condition related to the presence or absence of a significant coefficient of the U component. When condition B1-U is true (YES in S116), after 4 is added to bS (S118), the process proceeds to step S120. On the other hand, when condition B1-U is false (NO in S116), the process directly proceeds to step S120.

[0152] In step S120, the boundary strength calculation unit 261 determines the truth or falsehood of condition B1-V, which is a condition related to the presence or absence of a significant coefficient of the V component. When condition B1-V is true (YES in S120), after 2 is added to bS (S122), the boundary strength calculation process (S10) ends. When condition B1-V is false (NO in S120), the boundary strength calculation process (S10) directly ends.

[0153] Returning to FIG. 9, the description of the processing flow by the deblocking filter 26 is continued. In step S20, the applicability determination unit 265 of the determination unit 263 determines the truth or falsehood of the above-described condition C1. When condition C1 is false (NO in S20), the process ends.

[0154] On the other hand, when condition C1 is true (YES in S20), the applicability determination unit 265 determines the truth or falsehood of condition C2 described above (S30). When condition C2 is false (NO in S30), the process ends.

[0155] On the other hand, when condition C2 is true (YES in S30), the filter strength determination unit 267 of the determination unit 263 determines the filter strength by determining the truth or falsehood of condition C3 described above (S40). When condition C3 is true (YES in S40), the filtering unit 269 applies a strong filter to the color difference component of the pixel located near the block boundary (S50). On the other hand, when condition C3 is false (NO in S40), the filtering unit 269 applies a weak filter to the color difference component of the pixel located near the block boundary (S60).

[0156] The flow of the process by the deblocking filter 26 according to the present embodiment has been described above. The above-described process described with reference to FIGS. 9 and 10 can be performed in units of 4 lines in the luminance component of the decoded image, that is, in units of 2 lines in the color difference component of the decoded image, for example, in the case of the YUV420 format.

[0157] [3-3. Modification Example] FIG. 11 is a table showing a modification example of bS calculated by the boundary strength calculation unit 261. Similar to the example of FIG. 8, bS is also represented by 5 bits here. Further, at least one bit corresponding to each of the Y component, U component, and V component is included in a plurality of bits of bS. The 5th bit (the most significant bit) of bS corresponds to condition A related to intra prediction, the 4th bit of bS corresponds to condition B1-Y related to the significant coefficient of the Y component, the 3rd bit of bS corresponds to condition B1-U related to the significant coefficient of the U component, the 2nd bit of bS corresponds to condition B1-V related to the significant coefficient of the V component, and the 1st bit (the least significant bit) of bS corresponds to condition B2 related to MV and the reference picture, respectively.

[0158] Unlike the example of FIG. 8, in this modified example, particularly when condition B1-Y is false (therefore, bS is less than 8), the first bit of bS is set to 1. As a result, in the example of FIG. 8, bS takes one of 10 values: 0, 1, 2, 4, 6, 8, 10, 12, 14, or 16, whereas in this modified example, bS takes one of 13 values: 0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, or 16.

[0159] FIGS. 12A and 12B are flowcharts for explaining an example of the flow of the boundary strength calculation process corresponding to the modified example of FIG. 11. Before the start of the process, it is assumed that bS is initialized to 0. Referring to FIG. 12A, first, the boundary strength calculation unit 261 determines the truth or falsehood of condition A, which is a condition related to intra prediction (S150). If condition A is true, bS is set to 16 (S152), and the boundary strength calculation process ends.

[0160] If condition A is false, the boundary strength calculation unit 261 determines the truth or falsehood of condition B1-Y, which is a condition related to the presence or absence of a significant coefficient in the Y component (S154). If condition B1-Y is true, 8 is added to bS (S156), and if condition B1-Y is false, the addition of 8 is skipped (S158). Next, the truth or falsehood of condition B1-U, which is a condition related to the presence or absence of a significant coefficient in the U component, is determined (S160). If condition B1-U is true, 4 is added to bS (S162), and if condition B1-U is false, the addition of 4 is skipped (S164). Next, the truth or falsehood of condition B1-V, which is a condition related to the presence or absence of a significant coefficient in the V component, is determined (S166). If condition B1-V is true, 2 is added to bS (S168), and if condition B1-V is false, the addition of 2 is skipped (S170).

[0161] Referring to FIG. 12B, the subsequent processing branches depending on whether condition B1-Y, which is a condition regarding the presence or absence of a significant coefficient of the Y component, is true or false (S172). If condition B1-Y is true, the fourth bit of bS is equal to 1, so bS is set to a value of 8 or more (14, 12, 10, or 8) (S174), and the boundary strength calculation process ends without performing the MV check described later.

[0162] On the other hand, if condition B1-Y is false, the boundary strength calculation unit 261 performs an MV check (S176). The MV check here means determining the truth or falsehood of condition B2, which is a condition regarding the motion vector and the reference picture. If condition B2 is true, 1 is added to bS (S178), and if condition B2 is false, the addition of 1 is skipped (S180).

[0163] <4. Hardware Configuration Example> The above-described series of processes can be executed by hardware or by software. When the series of processes are executed by software, the program constituting the software is installed in a computer. Here, the computer includes a computer incorporated in dedicated hardware, and, for example, a general-purpose personal computer that can execute various functions by installing various programs.

[0164] FIG. 13 is a block diagram showing a hardware configuration example of a computer that executes the above-described series of processes by a program.

[0165] In the computer 800 shown in FIG. 13, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are interconnected via a bus 804.

[0166] The bus 804 is also connected to an input / output interface 810. Connected to the input / output interface 810 are an input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815.

[0167] The input unit 811 consists of, for example, a keyboard, a mouse, a microphone, a touch panel, input terminals, etc. The output unit 812 consists of, for example, a display, a speaker, output terminals, etc. The storage unit 813 consists of, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 814 consists of, for example, a network interface. The drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0168] In the computer configured as described above, the CPU 801 loads and executes, for example, a program stored in the storage unit 813 via the input / output interface 810 and the bus 804 into the RAM 803, thereby performing the above-described series of processes. The RAM 803 also appropriately stores data and the like necessary for the CPU 801 to execute various processes.

[0169] The program executed by the computer (CPU 801) can be recorded and applied, for example, on a removable medium 821 as a package medium or the like. In that case, the program can be installed in the storage unit 813 via the input / output interface 810 by mounting the removable medium 821 on the drive 815.

[0170] Also, this program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In that case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0171] Alternatively, this program can be pre-installed in the ROM 802 or the storage unit 813.

[0172] <5. Connection> As described above, according to the embodiment of the present disclosure, it is possible to more appropriately apply a deblocking filter to the color difference component of the decoded image.

[0173] As described above, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings. However, the technical scope of the present disclosure is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field of the present disclosure can conceive various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that these also belong to the technical scope of the present disclosure.

[0174] (Color difference related parameter) For example, in the above embodiment, an example in which a flag indicating the presence or absence of a significant coefficient of the color difference component in each TU is used as the color difference related parameter has been described. However, the present technology is not limited to such an example. For example, the conversion coefficient itself of the color difference component may be included in the color difference related parameter. In such a case, the boundary strength calculation unit 261 may calculate bS by determining the presence or absence of a significant coefficient of the color difference component in each TU from the conversion coefficient of the color difference component. Further, in relation to the above embodiment, FIG. 4 shows an example in which the value of bS differs depending on whether or not condition B2 is satisfied, in addition to whether or not conditions B1-Y, B1-U, or B1-V are satisfied. However, as in the alternative example shown in FIG. 14, for example, in order to suppress an increase in processing cost, the determination of whether or not condition B2 is satisfied for both color difference components U and V may be omitted.

[0175] (Threshold for large block determination) In the above embodiment, an example in which the threshold used in the large block determination is 16 has been described. However, the present technology is not limited to such an example, and it may be set to 8 or 32. Further, in the case of the YUV444 format, a threshold equal to or higher than the threshold used in the YUV420 format may be used in the large block determination.

[0176] (Strong filter) In the above embodiment, an example in which the strong filters represented by formulas (15) to (19) are applied to the color difference components has been described. However, the strong filters applied in the present technology are not limited to such examples. The strong filter applied to the color difference components may be any filter having a stronger filter strength than the weak filter. For example, the strong filter applied to the color difference components in Non-Patent Document 2 (the strong filter applied to the luminance component in HEVC) may be applied to the color difference components in the present technology.

[0177] (Applicable Object of the Present Technology) The present technology can be applied to any image encoding / decoding method. That is, as long as it does not conflict with the present technology described above, the specifications of various processes related to image encoding / decoding, such as transformation (inverse transformation), quantization (inverse quantization), encoding (decoding), prediction, etc., are arbitrary and not limited to the examples described above. Also, as long as it does not conflict with the present technology described above, a part of these processes may be omitted.

[0178] (Block) In addition, in this specification, the "block" (not the block indicating the processing unit) used in the description as a partial region or a processing unit of an image (picture) indicates an arbitrary partial region in the picture, and its size, shape, characteristics, etc. are not limited. For example, the "block" includes any partial region (processing unit) such as the TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), transform block, sub-block, macro-block, tile, or slice described in the above reference documents REF1 to REF3.

[0179] (Processing Unit) The data unit in which the various types of information described above are set and the data unit to which the various types of processing are applied are each arbitrary and are not limited to the examples described above. For example, these pieces of information and processing may be set for each of TU (Transform Unit), TB (Transform Block), PU (Prediction Unit), PB (Prediction Block), CU (Coding Unit), LCU (Largest Coding Unit), sub-block, block, tile, slice, picture, sequence, or component, respectively, or the data of these data units may be targeted. Of course, this data unit can be set for each piece of information and processing, and it is not necessary for the data units of all information and processing to be unified. Note that the storage location of this information is arbitrary and may be stored in the header or parameter set of the data unit described above. Also, it may be stored in a plurality of locations.

[0180] Also, in the above embodiment, the deblocking filter processing for the color difference components is performed in units of two lines, but the present technology is not limited to such an example. For example, in the case of the YUV444 format, the deblocking filter processing for the color difference components may be performed in units of four lines. In such a case, the applicability determination unit 265 may make a determination with reference to the first line and the third line in the determination of the above-described condition C3.

[0181] (Control Information) The control information related to the present technology described above may be transmitted from the encoding side to the decoding side. For example, control information (e.g., enabled_flag) for controlling whether to permit (or prohibit) the application of the present technology described above may be transmitted. Also, for example, control information indicating the target (or non-target) to which the present technology described above is applied may be transmitted. For example, control information specifying the block size (upper limit or lower limit, or both), frame, component, or layer, etc., to which the present technology is applied (or to which the application is permitted or prohibited) may be transmitted.

[0182] (Block size information) When specifying the size of the block to which the present technology is applied, not only the block size may be directly specified, but also the block size may be indirectly specified. For example, the block size may be specified using identification information for identifying the size. Also, for example, the block size may be specified by the ratio or difference from the size of a reference block (for example, LCU, SCU, etc.). For example, when transmitting information specifying the block size as a syntactic element or the like, as that information, information indirectly specifying the size as described above may be used. By doing so, the amount of information of that information can be reduced, and the coding efficiency may be improved in some cases. Also, the specification of this block size includes the specification of the range of the block size (for example, the specification of the range of the allowable block size, etc.).

[0183] (Others) Note that in this specification, a "flag" is information for identifying a plurality of states, and includes not only information used for identifying two states of true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, a binary value of 1 / 0, or may be three or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be 1 bit or a plurality of bits. Also, identification information (including flags) is not only in the form of including the identification information in a bit stream, but also in the form of including the difference information of the identification information with respect to a certain reference information in the bit stream. Therefore, in this specification, "flags" and "identification information" include not only that information, but also difference information with respect to reference information. Since it is also assumed that the difference information of the identification information with respect to a certain reference information is included in the bit stream, in this specification, "flags" and "identification information" include not only that information, but also difference information with respect to reference information.

[0184] In addition, various types of information (such as metadata) related to the encoded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the encoded data. Here, the term "associate" means, for example, enabling the use (linking) of one piece of data when processing the other piece of data. That is, the data associated with each other may be grouped as one piece of data or may be individual pieces of data. For example, the information associated with the encoded data (image) may be transmitted on a transmission path different from that of the encoded data (image). Also, for example, the information associated with the encoded data (image) may be recorded on a recording medium different from that of the encoded data (image) (or a different recording area of the same recording medium). Note that this "association" may be for a part of the data instead of the entire data. For example, an image and the information corresponding to the image may be associated with each other in any unit such as a plurality of frames, one frame, or a part within a frame.

[0185] Note that in this specification, terms such as "synthesize", "multiplex", "add", "integrate", "include", "store", "insert", "plug in", "insert" mean, for example, grouping a plurality of things into one, such as grouping the encoded data and the metadata into one piece of data, and mean one method of the above-mentioned "associate".

[0186] This technology can also be implemented as any component constituting the apparatus or system, for example, a processor as a system LSI (Large Scale Integration) or the like, a module using a plurality of processors or the like, a unit using a plurality of modules or the like, a set obtained by adding other functions to the unit (that is, a part of the configuration of the apparatus).

[0187] In this specification, the term "system" refers to a collection of multiple components (such as devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, a plurality of devices housed in separate enclosures and connected via a network, and a single device with multiple modules housed in one enclosure are both systems.

[0188] Also, for example, the configuration described as a single device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described as multiple devices (or processing units) above may be combined and configured as a single device (or processing unit). Of course, other configurations may be added to the configuration of each device (or each processing unit) in addition to those described above. Furthermore, if the overall configuration and operation of the system are substantially the same, a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or another processing unit). Also, for example, the present technology can adopt a cloud computing configuration in which one function is shared and jointly processed by multiple devices via a network.

[0189] Also, for example, each step described in the above flowchart can be executed by a single device or can be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes included in that single step can be executed by a single device or can be shared and executed by multiple devices. In other words, the multiple processes included in a single step can also be executed as processes of multiple steps. Conversely, the processes described as multiple steps can also be executed as a single step.

[0190] Note that the program executed by the computer may be such that the processing of the steps of describing the program is executed in time series along the order described in this specification, or may be executed in parallel, or may be executed individually at a necessary timing such as when a call is made. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the order described above. Further, the processing of the steps of describing this program may be executed in parallel with the processing of other programs, or may be executed in combination with the processing of other programs.

[0191] Also, the effects described in this specification are merely illustrative or exemplary and not limiting. That is, the technology according to the present disclosure may exhibit other effects apparent to those skilled in the art from the description of this specification, together with or instead of the above effects.

[0192] Note that the following configurations also belong to the technical scope of the present disclosure. (1) A decoding unit that decodes a coded stream to generate a decoded image, A determination unit that determines whether or not to apply a deblocking filter to the color difference component of the decoded image based on the boundary strength calculated using a color difference-related parameter related to the color difference, targeting the block boundary of the decoded image. A filtering unit that applies a deblocking filter to the color difference component of the pixels located in the vicinity of the block boundary based on the determination result of whether or not to apply the deblocking filter. An image processing apparatus comprising the above. (2) The color difference-related parameter includes information regarding the conversion coefficient of the color difference component. The boundary strength is calculated based on whether or not a significant coefficient of the color difference component exists in the blocks sandwiching the block boundary that is the calculation target of the boundary strength. The image processing apparatus according to (1) above. (3) The boundary strength is calculated by independently determining whether significant coefficients of each component exist in the blocks sandwiching the block boundary that is the object of calculating the boundary strength, and the image processing apparatus according to (2) above. (4) The boundary strength is represented by a plurality of bits, and at least one bit corresponding to each component is included in the plurality of bits, and the image processing apparatus according to any one of (1) to (3) above. (5) The determination unit determines whether the deblocking filter needs to be applied based on the determination based on the boundary strength and the large block determination using the block size of the blocks sandwiching the block boundary, and the image processing apparatus according to any one of (1) to (4) above. (6) The determination unit performs the large block determination when the boundary strength has a value related to inter prediction, and the image processing apparatus according to (5) above. (7) The determination unit performs the large block determination based on the size in the direction orthogonal to the block boundary in the blocks sandwiching the block boundary, and the image processing apparatus according to (5) or (6) above. (8) The determination unit performs the large block determination based on whether the size in the direction orthogonal to the block boundary in the blocks sandwiching the block boundary is greater than 16, and the image processing apparatus according to (7) above. (9) The filtering unit applies a weak filter or a strong filter as the deblocking filter to the color difference components of the pixels located in the vicinity of the block boundary, and the image processing apparatus according to any one of (1) to (8) above. (10) The coefficient of the strong filter is 2 at the center position of the application range of the strong filter and 1 at other positions, and the image processing apparatus according to (9) above. (11) The filtering unit sets the range of application of the strong filter to three pixels on each side from the block boundary, and applies the strong filter to the color difference components of the pixels included in the application target range, using the three pixels on each side of the center position of the application target range as reference pixels. The image processing apparatus according to (9) or (10) above. (12) Instead of the pixels that are five or more pixels away from the block boundary, the filtering unit uses the pixel value of the fourth pixel from the block boundary as the pixel value of the reference pixel by padding. The image processing apparatus according to (11) above. (13) The filtering unit performs clipping processing based on a parameter t specified based on the quantization parameter C and applies the strong filter. The image processing apparatus according to any one of (10) to (12) above. (14) The determination unit further determines the filter strength of the deblocking filter applied to the color difference components of the pixels located in the vicinity of the block boundary. The filtering unit applies the weak filter or the strong filter to the color difference components of the pixels located in the vicinity of the block boundary according to the filter strength determined by the determination unit. The image processing apparatus according to any one of (9) to (13) above. (15) The determination unit determines the filter strength after determining whether the deblocking filter needs to be applied. The image processing apparatus according to (14) above. (16) The determination unit determines the filter strength based on the waveform of the color difference components of the pixels located in the vicinity of the block boundary. The image processing apparatus according to (15) above. (17) The determination unit determines the filter strength based on the conditions regarding flatness, continuity, and gaps of the color difference components of the pixels included in two lines among the pixels located in the vicinity of the block boundary. When it is determined that all of the conditions regarding flatness, the conditions regarding continuity, and the conditions regarding gaps are satisfied, the filtering unit applies the strong filter, and when it is determined that at least one condition is not satisfied, the weak filter is applied. The image processing apparatus according to (16). (18) Decoding the encoded stream to generate a decoded image, targeting the block boundary of the decoded image, and determining whether to apply a deblocking filter to the color difference component of the decoded image based on the boundary strength calculated using a color difference-related parameter related to the color difference, applying a deblocking filter to the color difference components of the pixels located near the block boundary based on the determination result of whether to apply the deblocking filter, An image processing method including. (19) A determination unit that targets the block boundary of the locally decoded decoded image and determines whether to apply a deblocking filter to the color difference component of the decoded image based on the boundary strength calculated using a color difference-related parameter related to the color difference, A filtering unit that applies a deblocking filter to the color difference components of the pixels located near the block boundary based on the determination result of whether to apply the deblocking filter, An encoding unit that encodes an image using the decoded image to which the deblocking filter has been applied by the filtering unit, An image processing apparatus comprising. (20) targeting the block boundary of the locally decoded decoded image and determining whether to apply a deblocking filter to the color difference component of the decoded image based on the boundary strength calculated using a color difference-related parameter related to the color difference, applying a deblocking filter to the color difference components of the pixels located near the block boundary based on the determination result of whether to apply the deblocking filter, encoding an image using the decoded image to which the deblocking filter has been applied, An image processing method including

Explanation of symbols

[0193] 10 Image encoding device 16 Reversible encoding section 26 Deblocking filter 60 Image decoding device 62 Reversible decoding section 261 Boundary strength calculation section 263 Determination section 265 Application necessity determination section 267 Filter strength determination section 269 Filtering section

Claims

1. a decoding unit that decodes the encoded stream to generate a decoded image; a determination unit that determines whether or not a deblocking filter needs to be applied to a chrominance component of the decoded image based on a boundary strength calculated using a chrominance-related parameter that is information about a transform coefficient of the chrominance component obtained by an orthogonal transform, for a block boundary of the decoded image; a filtering unit that applies a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; Equipped with the boundary strength is calculated independently for each of the first component and the second component by independently determining, based on the color difference-related parameters, whether or not a significant coefficient of a first component and whether or not a significant coefficient of a second component are present among components included in the color difference components in blocks sandwiching the block boundary, which is a target for calculating the boundary strength; The determination unit is determining whether or not the deblocking filter needs to be applied based on a determination based on the boundary strength and a large block determination based on whether or not a size of a block between the block boundary and a direction parallel to the block boundary is greater than 8 in the orthogonal direction; Image processing device.

2. The determination unit performs the large block determination when the boundary strength has a value related to inter prediction. The image processing device according to claim 1 .

3. the color difference components are composed of a U component and a V component, the first component being the U component and the second component being the V component; The image processing device according to claim 1 .

4. The color difference related parameters include a flag indicating whether a significant coefficient is included in a transform block of the orthogonal transform. The image processing device according to claim 1 .

5. The boundary strength is represented by a number of bits, The image processing device according to claim 1 , wherein the plurality of bits includes at least one bit corresponding to each component.

6. The image processing device according to claim 1 , wherein the filtering unit applies, as the deblocking filter, a weak filter or a strong filter to color difference components of pixels located in the vicinity of the block boundary.

7. The image processing device according to claim 6 , wherein a coefficient of the strong filter is 2 at a central position of an application range of the strong filter, and is 1 at other positions.

8. 7. The image processing device according to claim 6, wherein the filtering unit defines an application range of the strong filter from the block boundary to three pixels on either side thereof, and applies the strong filter to color difference components of pixels included in the application range by using three pixels on either side of a center position of the application range as reference pixels.

9. The image processing device according to claim 8 , wherein the filtering unit uses, as the pixel value of the reference pixel, a pixel value of a fourth pixel from the block boundary by padding, instead of a pixel that is five pixels or more away from the block boundary.

10. The filtering unit is configured to filter a parameter t C The image processing device according to claim 8 , wherein the strong filter is applied by performing clipping processing based on

11. The determination unit further determines a filter strength of the deblocking filter to be applied to a chrominance component of a pixel located near the block boundary, The image processing device according to claim 6 , wherein the filtering unit applies the weak filter or the strong filter to color difference components of pixels located in the vicinity of the block boundary, depending on the filter strength determined by the determination unit.

12. The image processing device according to claim 11 , wherein the determining unit determines the filter strength after determining whether or not a deblocking filter needs to be applied.

13. The image processing device according to claim 12 , wherein the determining unit determines the filter strength based on a waveform of a color difference component of a pixel located in the vicinity of the block boundary.

14. the determination unit determines the filter strength based on a condition regarding flatness, a condition regarding continuity, and a condition regarding a gap of color difference components of pixels included in two lines among pixels located near the block boundary; The image processing device according to claim 13 , wherein the filtering unit applies the strong filter when it is determined that all of the conditions related to the flatness, the condition related to the continuity, and the condition related to the gap are satisfied, and applies the weak filter when it is determined that at least one of the conditions is not satisfied.

15. decoding the encoded stream to generate a decoded image; determining whether or not a deblocking filter needs to be applied to the chrominance component of the decoded image based on a boundary strength calculated using a chrominance-related parameter that is information about a transform coefficient of the chrominance component obtained by an orthogonal transform, for a block boundary of the decoded image; applying a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; Including, the boundary strength is calculated independently for each of the first component and the second component by independently determining, based on the color difference-related parameters, whether or not a significant coefficient of a first component and whether or not a significant coefficient of a second component are present among components included in the color difference components in blocks sandwiching the block boundary, which is a target for calculating the boundary strength; The determining step comprises: determining whether or not the deblocking filter needs to be applied based on a determination based on the boundary strength and a large block determination based on whether or not a size of a block between the block boundary and a direction parallel to the block boundary is greater than 8 in the orthogonal direction; Image processing methods.

16. a determination unit that determines whether or not a deblocking filter needs to be applied to a chrominance component of a decoded image decoded by a local decoding process, based on a boundary strength calculated using a chrominance-related parameter that is information about a transform coefficient of the chrominance component obtained by an orthogonal transform; a filtering unit that applies a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; an encoding unit that encodes an image using the decoded image to which the deblocking filter has been applied by the filtering unit; Equipped with the boundary strength is calculated independently for each of the first component and the second component by independently determining, based on the color difference-related parameters, whether or not a significant coefficient of a first component and whether or not a significant coefficient of a second component are present among components included in the color difference components in blocks sandwiching the block boundary, which is a target for calculating the boundary strength; The determination unit is determining whether or not the deblocking filter needs to be applied based on a determination based on the boundary strength and a large block determination based on whether or not a size of a block between the block boundary and a direction parallel to the block boundary is greater than 8 in the orthogonal direction; Image processing device.

17. determining whether or not a deblocking filter needs to be applied to a chrominance component of a decoded image decoded by a local decoding process, based on a boundary strength calculated using a chrominance-related parameter that is information about a transform coefficient of the chrominance component obtained by an orthogonal transform; applying a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; encoding an image using the decoded image to which the deblocking filter has been applied; Including, the boundary strength is calculated independently for each of the first component and the second component by independently determining, based on the color difference-related parameters, whether or not a significant coefficient of a first component and whether or not a significant coefficient of a second component are present among components included in the color difference components in blocks sandwiching the block boundary, which is a target for calculating the boundary strength; The determining step comprises: determining whether or not the deblocking filter needs to be applied based on a determination based on the boundary strength and a large block determination based on whether or not a size of a block between the block boundary and a direction parallel to the block boundary is greater than 8 in the orthogonal direction; Image processing methods.

Images