Image processing device and image processing method

Abstract

To provide an image processing device and an image processing method.SOLUTION: An image processing device according to a present disclosure, is configured to adopt a block boundary of a decoded image as a target, and determine a deblocking filter application necessity with respect to color difference components of the decoded image on the basis of boundary strength calculated by using a color difference related parameter related to a color difference component obtained by an orthogonal conversion; and apply a deblocking filter to the color difference components of pixels positioned in a vicinity of the target on the basis of a determination result. Therein: the boundary strength is independently calculated for each of first and second components by independently determining the presence / absence of a significant coefficient to each of the first and second components in the color difference components to a block interposing the block boundary of the calculation target of the boundary strength on the basis of the color difference related parameter; and the deblocking filter application necessity is determined on the basis of a determination based on the boundary strength, and a determination based on a size of a direction orthogonal to the block boundary of the block interposing the block boundary.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device and an image processing method. [Background technology]

[0002] In H.265 / HEVC, a standard specification for video coding, a deblocking filter is applied to the block boundaries of decoded images to suppress image quality degradation caused by block distortion that occurs during coding. In H.265 / HEVC, there are two types of deblocking filters that can be applied to the luminance component: a weak filter and a strong filter, while there is only one type of deblocking filter that can be applied to the chrominance component: a weak filter.

[0003] Furthermore, with the aim of further improving coding efficiency over H.265 / HEVC, the Joint Video Experts Team (JVET), a joint standardization organization of ITU-T and ISO / IEC, is currently working on standardizing FVC (Future Video Coding), a next-generation video coding method (see, for example, Non-Patent Document 1).

[0004] In the standardization work for FVC, Non-Patent Document 2 below proposes a method in which the deblocking filters that can be applied to the chrominance components are changed to two types, similar to the deblocking filters that can be applied to the luminance components, so that a strong filter can also be applied to the chrominance components. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] J. Chen, E. Alshina, GJ Sullivan, J.-R. Ohm, J. Boyce, “Algorithm Description of Joint Exploration Test Model (JEM7)”, JVET-G1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 7th Meeting: Torino, IT, 13-21 July 2017 [Non-patent document 2] Seung-Hwan Kim, Jie Zhao, Misra Kiran and Andrew Segall, “Improvement of chroma deblocking filter”, JVET-D0108, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 4th Meeting: Chengdu, CN, 15-21 October 2016 Summary of the Invention [Problem to be solved by the invention]

[0006] Whether or not to apply the above-described deblocking filter is determined using the boundary strength of the block boundary, and the boundary strength of the block boundary is calculated based on the luminance component information without using the chrominance component information. However, if the boundary strength specified based on the luminance component information is used to determine whether or not to apply the deblocking filter to the chrominance component of the decoded image, the deblocking filter may not be applied appropriately, and block distortion may remain.

[0007] Therefore, the present disclosure proposes a mechanism that enables a deblocking filter to be applied more appropriately to the chrominance components of a decoded image. [Means for solving the problem]

[0008] According to the present disclosure, there is provided a decoding unit that decodes an encoded stream to generate a decoded image, and a decoding unit that performs an orthogonal transform on a block boundary of the decoded image and calculates a chrominance-related parameter that is information about a transform coefficient of a chrominance component obtained by an orthogonal transform. , represented by at least 5 bits an image processing device including: a determination unit that determines, based on boundary strength, whether a deblocking filter needs to be applied to the chrominance components of the decoded image; and a filtering unit that applies a deblocking filter to the chrominance components of pixels located near the block boundary based on the result of the determination of whether the deblocking filter needs to be applied; wherein the boundary strength is calculated independently for each of the first and second components by determining independently whether a significant coefficient of a first component and a significant coefficient of a second component are present among the components included in the chrominance components in blocks sandwiching the block boundary for which the boundary strength is to be calculated; the boundary strength is expressed by a plurality of bits including at least one bit corresponding to each of the chrominance components; and the determination unit determines whether the deblocking filter needs to be applied based on the boundary strength and a large block determination based on the size of the blocks sandwiching the block boundary in a direction perpendicular to the block boundary or a direction parallel to the block boundary.

[0009] Furthermore, according to the present disclosure, a method for generating a decoded image by decoding an encoded stream, and a method for generating a decoded image by decoding a block boundary of the decoded image, the method comprising: , represented by at least 5 bitsan image processing method including: determining, based on boundary strength, whether a deblocking filter needs to be applied to the chrominance components of the decoded image; and applying a deblocking filter to the chrominance components of pixels located near the block boundary based on the result of the determination of whether a deblocking filter needs to be applied; wherein the boundary strength is calculated independently for each of the first and second components by determining independently whether a significant coefficient of a first component and a significant coefficient of a second component are present among components included in the chrominance components in blocks sandwiching the block boundary for which the boundary strength is to be calculated; the boundary strength is expressed by a plurality of bits including at least one bit corresponding to each of the chrominance components; and the determination of whether a deblocking filter needs to be applied is performed based on the boundary strength and a large block determination based on the size of the blocks sandwiching the block boundary in a direction perpendicular to the block boundary or a direction parallel to the block boundary.

[0010] Furthermore, according to the present disclosure, a block boundary of a decoded image decoded by a process using local decoding is targeted, and the block boundary is calculated using chrominance-related parameters, which are information about transform coefficients of chrominance components obtained by orthogonal transform. , represented by at least 5 bitsan encoding unit that encodes an image using the decoded image to which the deblocking filter has been applied by the filtering unit; and an encoding unit that encodes an image using the decoded image to which the deblocking filter has been applied by the filtering unit, wherein the boundary strength is calculated independently for each of the first and second components by independently determining whether a significant coefficient of a first component and a significant coefficient of a second component are present among components included in the chrominance components in blocks sandwiching the block boundary for which the boundary strength is to be calculated, and the boundary strength is expressed by a plurality of bits including at least one bit corresponding to each of the chrominance components; and the determination unit determines whether the deblocking filter needs to be applied based on the boundary strength and a large block determination based on the size of the blocks sandwiching the block boundary in a direction perpendicular to the block boundary or a direction parallel to the block boundary.

[0011] Furthermore, according to the present disclosure, a block boundary of a decoded image decoded by a process using local decoding is targeted, and the block boundary is calculated using chrominance-related parameters, which are information about transform coefficients of chrominance components obtained by orthogonal transform. , represented by at least 5 bitsand encoding the decoded image to which the deblocking filter has been applied, the image processing method including: determining, based on boundary strength, whether a deblocking filter needs to be applied to the chrominance components of the decoded image; applying a deblocking filter to the chrominance components of pixels located near the block boundary based on a result of the determination of whether a deblocking filter needs to be applied; and encoding the image using the decoded image to which the deblocking filter has been applied, wherein the boundary strength is calculated independently for each of the first and second components by determining independently whether a significant coefficient of a first component and a significant coefficient of a second component are present among components included in the chrominance components in blocks sandwiching the block boundary for which the boundary strength is to be calculated, the boundary strength being expressed by a plurality of bits including at least one bit corresponding to each of the chrominance components, and the determination of whether the deblocking filter needs to be applied is based on the boundary strength and large block determination based on the size of the blocks sandwiching the block boundary in a direction perpendicular to the block boundary or a direction parallel to the block boundary. [Effects of the Invention]

[0012] As described above, according to the present disclosure, it is possible to more appropriately apply a deblocking filter to the chrominance components of a decoded image.

[0013] The above effects are not necessarily limiting, and any of the effects described in this specification or other effects that can be understood from this specification may be achieved in addition to or instead of the above effects. [Brief explanation of the drawings]

[0014] [Figure 1] 10 is a table for explaining calculation of bS in HEVC. [Figure 2] 10 is a table for explaining the calculation of bS in Non-Patent Document 2. [Figure 3]FIG. 10 is an explanatory diagram showing an example of pixels of color difference components (U component, V component) in two adjacent blocks Bp and Bq across a vertical block boundary BB. [Figure 4] 10 is a table for explaining calculation of bS according to an embodiment of the present disclosure. [Figure 5] FIG. 2 is a block diagram showing an example of the configuration of an image encoding device 10, which is one aspect of the image processing device according to the embodiment. [Figure 6] FIG. 10 is a block diagram showing an example of the configuration of an image decoding device 60, which is one aspect of the image processing device according to the embodiment. [Figure 7] FIG. 2 is a block diagram showing an example of a detailed configuration of a deblocking filter 26 according to the embodiment. [Figure 8] 10 is a table showing an example of bS calculated by the boundary strength calculation unit 261. [Figure 9] 10 is a flowchart showing an example of the flow of processing by the deblocking filter 26 according to the embodiment. [Figure 10] 10 is a flowchart illustrating the flow of a boundary strength calculation process executed by a boundary strength calculation unit 261. [Figure 11] 10 is a table showing modified examples of bS calculated by the boundary strength calculation unit 261. [Figure 12A] 12 is the first half of a flowchart illustrating an example of the flow of a boundary strength calculation process corresponding to the modified example of FIG. 11. [Figure 12B] 12 is the second half of a flowchart illustrating an example of the flow of boundary strength calculation processing corresponding to the modified example of FIG. 11. [Figure 13] FIG. 2 is an explanatory diagram illustrating an example of a hardware configuration. [Figure 14] 10 is a table showing a modified example of the calculation of bS. DETAILED DESCRIPTION OF THE INVENTION

[0015] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0016] Furthermore, the scope of the disclosure in this specification is not limited to the contents of the examples. The contents of the following reference documents REF1 to REF3, which were publicly known at the time of filing, are also incorporated by reference. In other words, the contents of the following reference documents REF1 to REF3 also serve as the basis for determining the support requirements. For example, even if the Quad-Tree Block Structure described in Reference REF2 and the QTBT (Quad Tree Plus Binary Tree) Block Structure described in Reference REF3 are not directly defined in the detailed description of the invention, they are still within the scope of this disclosure and meet the support requirements of the claims. Similarly, for example, technical terms such as parsing, syntax, and semantics are also within the scope of this disclosure and meet the support requirements of the claims, even if they are not directly defined in the detailed description of the invention. REF1:Recommendation ITU-T H.264 (04 / 2017) “Advanced video coding for generic audiovisual services”, April 2017 REF2:Recommendation ITU-T H.265,(12 / 2016) “High efficiency video coding”, December 2016 REF3: J. Chen, E. Alshina, GJ Sullivan, J.-R. Ohm, J. Boyce, “Algorithm Description of Joint Exploration Test Model (JEM7)”, JVET-G1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 7th Meeting: Torino, IT, 13-21 July 2017

[0017] Unless otherwise specified, the following description will be given taking a signal in YUV420 format as an example, and the luminance component will be represented as the Y component, and the chrominance components as the U component and V component, respectively. However, the technology described below can be similarly applied to signals in other formats, such as the YUV444 format and the YUV422 format. The representation of the luminance component and chrominance component differs depending on the target signal, and the technology described below can be similarly applied to signals in which the luminance component and chrominance component are represented in YCbCr, for example.

[0018] Additionally, the following terms used in this specification are defined as follows: The chrominance-related parameters refer to general parameters related to chrominance. For example, the chrominance-related parameters may include information about the transform coefficients of the chrominance components included in each TU (Transform Unit), a flag indicating whether or not each TU has a significant coefficient (a non-zero transform coefficient) of the chrominance component, and other information about the transform coefficients of the chrominance components. However, the chrominance-related parameters are not limited to these examples, and may be various parameters related to chrominance. The necessity of applying a deblocking filter means whether or not a deblocking filter should be applied. For example, determining whether or not a deblocking filter should be applied means determining whether or not a deblocking filter should be applied. Furthermore, the determination result of whether or not a deblocking filter should be applied is the result of determining whether or not a deblocking filter should be applied, and for example, the determination result can be information indicating either that a deblocking filter should be applied or that it does not need to be applied. Large block determination refers to determining whether a block to be determined is a large block. In this specification, the block to be determined may be a block that sandwiches a block boundary, as will be described later. Large block determination may also be performed by comparing the size of the block (block size) with a predetermined threshold. Cases in which large block determination is performed and details of large block determination will be described later.

[0019] The explanation will be given in the following order. 1. Overview 1-1. Existing deblocking filters 1-2. Overview of the technology disclosed herein 2. Equipment Overview 2-1. Image encoding device 2-2. Image decoding device 3. Deblocking filter 3-1.Configuration example 3-2. Processing flow 3-3. Modified Examples 4. Hardware configuration example 5. Conclusion

[0020] <1. Overview> [1-1. Existing methods]

[0021] The processing of a deblocking filter in an existing image encoding method such as HEVC includes a process for determining whether application is necessary, a process for determining filter strength, and a filtering process (filter application process). The processing of an existing deblocking filter will be described below using an HEVC deblocking filter as an example. Note that the following mainly describes a deblocking filter for the chrominance component of a decoded image (including an image locally decoded during encoding), and a description of a deblocking filter for the luminance component of a decoded image will be omitted as appropriate.

[0022] The deblocking filter process first performs an application necessity determination process. The application necessity determination process is a process for determining whether or not a deblocking filter should be applied to a block boundary of a decoded image. In HEVC, block boundaries are identified based on the block structure of the Quad-Tree Block Structure described in Reference Document REF2. Specifically, among edges of an 8x8 pixel block (sample grid), which is the smallest block unit, an edge that satisfies the condition of being at least either a TU (Transform Unit) boundary or a PU (Prediction Unit) boundary is identified as a block boundary in HEVC.

[0023] The process of determining whether or not to apply the threshold is performed based on the boundary strength (hereinafter, sometimes referred to as bS) of the block boundary. In HEVC, bS is calculated for every four lines of the identified block boundary. If the block boundary is a vertical boundary, the above lines correspond to rows that are perpendicular to the vertical boundary. If the block boundary is a horizontal boundary, the above lines correspond to columns that are perpendicular to the horizontal boundary.

[0024] FIG. 1 is a table for explaining the calculation of bS in HEVC. As shown in FIG. 1, in HEVC, bS is calculated based on the truth or falsity (satisfaction or non-satisfaction) of condition A, which is a condition related to intra prediction, condition B1, which is a condition related to significant coefficients of the Y component, and condition B2, which is a condition related to motion vectors (MVs) and reference pictures. Referring to FIG. 1, if condition A is true, bS is set to 2. Also, if condition A is false and at least one of conditions B1 and B2 is true, bS is set to 1. And if condition A, condition B1, and condition B2 are all false, bS is set to 0. Note that conditions A, B1, and B2 shown in FIG. 1 are the following conditions.

[0025] Condition A: The coding mode of at least one of the CUs (Coding Units) that include pixels on the top line of the lines for which bS is calculated and that sandwich a block boundary is an intra prediction mode. Condition B1: The block boundary is a TU boundary, and a significant coefficient of the Y component exists in at least one of two TUs that contain a pixel on the top line of the lines for which bS is calculated and sandwich the block boundary. Condition B2: Between two CUs that include pixels on the top line of the lines for which bS is calculated and sandwich a block boundary, the absolute value of the difference in MV is 1 pixel or more, or the reference pictures for motion compensation are different, or the number of MVs is different.

[0026] Furthermore, in HEVC, a deblocking filter may be applied to the luminance component (Y component) of a decoded image for block boundaries where the bS set as described above is equal to or greater than 1. Therefore, in HEVC, the result of determining whether or not to apply a deblocking filter to the luminance component of a decoded image may differ depending on whether or not Condition B1 and Condition B2 are satisfied.

[0027] In HEVC, a strong filter with high filter strength and a weak filter with low filter strength are provided as deblocking filters for the luminance component of a decoded image. When bS is 1 or greater, the deblocking filter processing for the luminance component of a decoded image includes a process for determining whether or not further application is necessary based on further conditions, followed by a process for determining filter strength and a filtering process. Details of these processes are described in the above-mentioned reference document REF2, and will not be described here.

[0028] On the other hand, a deblocking filter for the chrominance components (U component, V component) of a decoded image in HEVC is applied only to block boundaries where bS is 2. Therefore, as shown in Fig. 1, whether or not conditions B1 and B2 are satisfied does not affect the determination of whether or not to apply a deblocking filter to the chrominance components of a decoded image in HEVC.

[0029] In addition, in HEVC, the only deblocking filter that can be applied to the chrominance component of a decoded image is a weak filter. Therefore, there is no need to determine the filter strength for the chrominance component of a decoded image. When bS is 2, a weak filter is applied to the chrominance component of a decoded image.

[0030] As described in the above-mentioned reference REF3, block division using the QTBT block structure in FVC may select blocks of a larger size than block division using the quad-tree block structure in HEVC. When the block size in a flat region (a region with small changes in pixel values within the region) is large, blocking artifacts are likely to occur. Therefore, in FVC, where blocks of a larger size may be selected, if a weak filter is the only deblocking filter that can be applied to the chrominance component of a decoded image, as in HEVC, significant blocking artifacts may remain in the chrominance component. In view of this situation, it is desirable to improve the deblocking filter for the chrominance component of a decoded image.

[0031] For example, Non-Patent Document 2 proposes a method in which the deblocking filter that can be applied to the chrominance component is changed to two types, similar to the deblocking filter that can be applied to the luminance component, and a strong filter can also be applied to the chrominance component. Non-Patent Document 2 also describes that a deblocking filter can be applied to the chrominance component of a decoded image not only when bS is 2, but also when bS is 1.

[0032] Fig. 2 is a table for explaining the calculation of bS in Non-Patent Document 2. As shown in Fig. 2, in Non-Patent Document 2, bS is calculated based on the above-mentioned conditions A, B1, and B2, similarly to the example of HEVC shown in Fig. 2. However, as described above, in Non-Patent Document 2, a deblocking filter may be applied to the chrominance components of the decoded image not only when bS is 2, but also when bS is 1. Therefore, as shown in Fig. 2, in Non-Patent Document 2, the result of determining whether or not to apply a deblocking filter to the chrominance components (U component, V component) of the decoded image may differ depending on whether or not conditions B1 and B2 are satisfied.

[0033] The following describes the application necessity determination process, filter strength determination process, and filtering process for a deblocking filter that can be applied to the chrominance components of a decoded image in Non-Patent Document 2, with reference to Fig. 3. Fig. 3 is an explanatory diagram showing an example of pixels of chrominance components (U components, V components) in two adjacent blocks Bp and Bq on either side of a vertical block boundary BB. Note that while a vertical boundary is used as an example, the matters described here can naturally be applied to horizontal boundaries as well. Also, while Fig. 3 shows an example in which blocks Bp and Bq are 4x4 in the chrominance components, the matters described here can be applied to blocks of other sizes as well.

[0034] In the example of FIG. 3, the pixels of the chrominance component in the block Bp are p i,jwhere i is the column index and j is the row index. The column index i is numbered 0, 1, 2, 3 (from left to right in the figure) in order from the column closest to the block boundary BB. The row index j is numbered 0, 1, 2, 3 from top to bottom. On the other hand, the color difference component pixels in block Bq are numbered q k,j where k is the column index and j is the row index. The column index k is numbered 0, 1, 2, 3 (from right to left in the figure) starting from the column closest to the block boundary BB.

[0035] After bS is calculated as described with reference to FIG. 2, the application necessity determination process and the filter strength determination process are performed using the following three conditions. In the case of the YUV420 format, this process is performed for every two lines of the color difference components. For example, in the example shown in FIG. 3, a determination is made separately for lines L11 and L12 and for lines L21 and L22. Note that the determination for each line is performed using the pixels of the line to be determined. Below, the application necessity determination process, filter strength determination process, and filtering process will be described using lines L11 and L12 as an example.

[0036] First, in the application necessity determination process, it is determined in order whether the following condition C91 and condition C92 are true.

[0037] -Condition C91:(bS==2||bS==1&&(block_width>16&&block_height>16))) -Condition C92:d <beta

[0038] In the above condition C91, block_width and block_height are the horizontal and vertical sizes, respectively, of a block (for example, a CU) that overlaps the block boundary that is the target of the determination, as shown in FIG.

[0039] The variable beta in the above condition C92 is an edge determination threshold, and the initial value of the variable beta is given according to the quantization parameter. The value of the variable beta can be specified by the user using a parameter in the slice header. The variable d in the above condition C92 is calculated using the following equations (1) to (7).

[0040] dp0=Abs(p 2,0 -2*p 1,0 +p 0,0 ) …(1) dp1=Abs(p 2,1 -2*p 1,1 +p 0,1 ) …(2) dq0=Abs(q 2,0 -2*q 1,0 +q 0,0 ) …(3) dq1=Abs(q 2,1 -2*q 1,1 +q 0,1 ) …(4) dpq0=dp0+dq0 …(5) dpq1=dp1+dq1 …(6) d=dpq0+dpq1 …(7)

[0041] Note that the above condition C92 is similar to the condition used in the process of determining whether to apply a deblocking filter applied to the luma component in HEVC (hereinafter referred to as the luma component condition), except that the lines referenced are different. In the luma component condition, the pixels on the first line and the pixels on the fourth line are referenced, and a determination is made every four lines. On the other hand, in the YUV420 format, the pixel density of the color difference components (U component, V component) is half that of the luma component, so in the above condition C92, the pixels on the first line, L11, and the pixels on the second line, L12, are referenced, and a determination is made every two lines.

[0042] If at least one of the above conditions C91 and C92 is false, the deblocking filter is not applied to the chrominance components of the decoded image. On the other hand, if both the above conditions C91 and C92 are true, the process proceeds to the filter strength determination process.

[0043] In the filter strength determination process, in order to determine whether to apply a strong filter or a weak filter, it is determined whether the following condition C93 is true.

[0044] -Condition C93:(block_width>16&&block_height>16)

[0045] Note that, like the block_width and block_height in the condition C91, the block_width and block_height in the above condition C93 are the horizontal and vertical sizes, respectively, of the block that overlaps the block boundary that is the target of the determination.

[0046] If the above condition C93 is true, a strong filter is applied to the chrominance components of the decoded image at the target block boundary, and if the above condition C93 is false, a weak filter is applied to the chrominance components of the decoded image at the target block boundary.

[0047] The strong filter applied to the chrominance component in Non-Patent Document 2 is the same as the strong filter applied to the luminance component in HEVC, and is expressed by the following equations (8) to (13).

[0048] p0′=Clip3(p0-2*tc,p0+2*t C ,(p2+2*p1+2*p0+2*q0+q1+4)>>3) …(8) p1′=Clip3(p1-2*tc,p1+2*t C ,(p2+p1+p0+q0+2)>>2) …(9) p2′=Clip3(p2-2*tc,p2+2*t C,(2*p3+3*p2+p1+p0+q0+4)>>3) …(10) q0′=Clip3(q0-2*tc,q0+2*t C ,(p1+2p0+2q0+2q1+q2+4)>>3) …(11) q1′=Clip3(q1-2*tc,q1+2*t C ,(p0+q0+q1+q2+2)>>2) …(12) q2′=Clip3(q2-2*t c ,q2+2*t C ,(p0+q0+q1+3*q2+2*q3+4)>>3) …(13)

[0049] In the above equations (8) to (13), p i , and q k is the pixel value of the chrominance component before applying the deblocking filter. i ′, and q k ' is the pixel value of the chrominance component after the deblocking filter is applied. Here, i and k are the column indices in the above-mentioned blocks Bp and Bq, respectively, and the row indices are omitted in equations (8) to (13). C is a parameter given according to the quantization parameter. Clip3(a,b,c) represents a clipping process that clips the value c within the range of a≦c≦b.

[0050] The weak filter applied to the chrominance component in Non-Patent Document 2 is the same as the weak filter applied to the chrominance component in HEVC, and therefore a description thereof will be omitted here.

[0051] Thus far, we have described the process relating to a deblocking filter that can be applied to the chrominance component of a decoded image in Non-Patent Document 2. According to the above-mentioned technique, it is possible to apply a strong filter not only to the luminance component but also to the chrominance component depending on the conditions.

[0052] However, as explained with reference to FIG. 2, the condition B1 used to calculate bS in Non-Patent Document 2 depends on the presence or absence of a significant coefficient of the luminance component (Y component), as in the case of HEVC, and does not use information on the chrominance components (U component, V component) even if other conditions are included. However, the spatial pattern of the luminance component and the spatial pattern of each of the chrominance components do not necessarily match. Therefore, if the determination of whether to apply a deblocking filter to the chrominance component is made according to a condition based on information about the luminance component, there is a risk that the deblocking filter will not be applied appropriately even though blocking artifacts have occurred, and the blocking artifacts will remain.

[0053] Furthermore, when bS is 1, in order for condition C91 used in the application necessity determination process in Non-Patent Document 2 to be true, both the horizontal and vertical sizes of the block across the block boundary to be determined must be greater than 16. However, as described in Reference Document REF3, the shape of a block (e.g., CU) in FVC can be not only square but also non-square rectangular. Blocking artifacts tend to occur more easily depending on the size in the direction perpendicular to the block boundary than on the size in the same direction as the block boundary. Therefore, depending on the shape of the block, the application necessity determination process in Non-Patent Document 2 may not properly apply the deblocking filter, resulting in remaining blocking artifacts.

[0054] Furthermore, the strong filter in Non-Patent Document 2 is the same as the strong filter applied in HEVC. However, as described above, in FVC, blocks of a size larger than those in the block division in HEVC may be selected, so even if the strong filter in Non-Patent Document 2 is applied, there is a risk that block distortion may not be sufficiently reduced.

[0055] [1-2. Overview of an embodiment of the present disclosure] Therefore, focusing on the above circumstances, an embodiment of the present disclosure has been created. An image processing device according to an embodiment of the present disclosure performs application necessity determination processing for determining whether or not a deblocking filter needs to be applied to the chrominance components of a decoded image, based on a boundary strength (bS) calculated using a chrominance-related parameter related to the chrominance of the decoded image. An outline of an embodiment of the present disclosure will be described below.

[0056] Fig. 4 is a table for explaining how bS is calculated in this embodiment. As shown in Fig. 4, bS is calculated based on condition A, which is a condition related to intra prediction, condition B1-Y, which is a condition related to significant coefficients of the Y component, condition B1-U, which is a condition related to significant coefficients of the U component, condition B1-V, which is a condition related to significant coefficients of the V component, and condition B2, which is a condition related to MV and reference pictures.

[0057] Referring to FIG. 4, if condition A is true, bS is set to 16. If condition A is false and condition B2 is true, bS is set to 1. If condition A and condition B2 are false and any one of conditions B1-Y, B1-U, and B1-V is true, bS is set to a value between 2 and 14. If condition A, condition B1-Y, condition B1-U, condition B1-V, and condition B2 are all false, bS is set to 0. Note that condition A, condition B1-Y, and condition B2 shown in FIG. 4 are the same as condition A, condition B1, and condition B2 described with reference to FIG. 1, respectively. The method for calculating bS according to this embodiment will be described in more detail later.

[0058] 4 correspond to conditions in which the presence or absence of a significant coefficient of the U component and the presence or absence of a significant coefficient of the V component are used for determination instead of the presence or absence of a significant coefficient of the Y component in condition B1-Y, and are expressed as follows: The truth or falsity of the following conditions B1-U and B1-V can be determined based on a flag (an example of a color difference-related parameter) indicating the presence or absence of a significant coefficient of the color difference component in each TU.

[0059] Condition B1-U: The block boundary is a TU boundary, and at least one of the two TUs that contain the pixel on the top line of the bS calculation target line and sandwich the block boundary has a significant coefficient of the U component. Condition B1-V: The block boundary is a TU boundary, and at least one of the two TUs that contain the pixel on the top line of the bS calculation target line and sandwich the block boundary has a significant coefficient of the V component.

[0060] In this embodiment, whether or not a deblocking filter needs to be applied to the chrominance components of the decoded image is determined based on bS calculated using the above-described chrominance-related conditions B1-U and B1-V. This configuration makes it possible to more appropriately apply a deblocking filter to the chrominance components.

[0061] Furthermore, in this embodiment, as will be described later, the determination of whether to apply a deblocking filter to the chrominance components of the decoded image is also based on the size of the block in a direction perpendicular to the block boundary. With this configuration, it is possible to more appropriately apply a deblocking filter even when the block shape is a rectangle that is not a square.

[0062] Furthermore, in this embodiment, as will be described later, a strong filter having a strength (stronger low-pass characteristics) than the strong filter in Non-Patent Document 2 may be applied to the chrominance components of the decoded image. Furthermore, in order to more appropriately apply such a strong filter, in this embodiment, the filter strength is determined by a method different from the filter strength determination process in Non-Patent Document 2. With this configuration, it is possible to further reduce block distortion.

[0063] The above is an overview of one embodiment of the present disclosure. The configuration and operation of this embodiment for achieving the above-mentioned effects will be described in detail below.

[0064] 2. Outline of the device First, a schematic configuration of an example device to which the technology disclosed in this specification can be applied will be described with reference to Figures 5 and 6. The technology disclosed in this specification can be applied to, for example, an image encoding device and an image decoding device.

[0065] [2-1. Image encoding device] FIG. 5 is a block diagram showing an example of a configuration of an image encoding device 10, which is one aspect of an image processing device according to an embodiment of the present disclosure.

[0066] Referring to Figure 5, the image encoding device 10 includes a sorting buffer 11, a control unit 12, a subtraction unit 13, an orthogonal transformation unit 14, a quantization unit 15, a lossless encoding unit 16, an accumulation buffer 17, an inverse quantization unit 21, an inverse orthogonal transformation unit 22, an addition unit 23, an in-loop filter 24, a frame memory 30, a switch 31, a mode setting unit 32, an intra prediction unit 40, and an inter prediction unit 50.

[0067] The reordering buffer 11 reorders a series of images constituting a video to be encoded according to a GOP (Group of Pictures) structure for encoding processing. The reordering buffer 11 outputs the reordered images to the control unit 12, the subtraction unit 13, the intra prediction unit 40, and the inter prediction unit 50.

[0068] The control unit 12 divides an image into blocks, which are processing units, based on an externally or pre-specified block size of the processing unit. The block division by the control unit 12 may form CUs of a quad-tree block structure or a QTBT (quad tree plus binary tree) block structure as processing units. The control unit 12 also determines parameters for the encoding process based on, for example, rate-distortion optimization (RDO). The determined parameters are supplied to each unit.

[0069] The subtraction unit 13 calculates a prediction error, which is the difference between the image input from the sorting buffer 11 and the predicted image, and outputs the calculated prediction error to the orthogonal transformation unit 14.

[0070] The orthogonal transform unit 14 performs orthogonal transform processing on each of one or more transform blocks (TUs) set in each region. The orthogonal transform here may be, for example, a discrete cosine transform or a discrete sine transform. More specifically, the orthogonal transform unit 14 converts the prediction errors input from the subtraction unit 13 from image signals in the spatial domain into transform coefficients in the frequency domain for each transform block. The orthogonal transform unit 14 then outputs the transform coefficients to the quantization unit 15.

[0071] Furthermore, the orthogonal transform unit 14 may generate a flag indicating the presence or absence of a significant coefficient in each TU for each component (Y component, U component, V component) based on the transform coefficients obtained by the orthogonal transform, and output the flag to the lossless encoding unit 16 and the in-loop filter 24. Note that the flag indicating the presence or absence of a significant coefficient of the U component in each TU and the flag indicating the presence or absence of a significant coefficient of the V component in each TU, which are generated by the orthogonal transform unit 14, are included in the color difference related parameters.

[0072] The quantization unit 15 is supplied with the transform coefficients input from the orthogonal transform unit 14 and a rate control signal from a rate control unit 18, which will be described later. The quantization unit 15 quantizes the transform coefficients and outputs the quantized transform coefficients (hereinafter also referred to as quantized data) to the lossless encoding unit 16 and the inverse quantization unit 21. The quantization unit 15 also changes the quantization scale based on the rate control signal from the rate control unit 18, thereby changing the bit rate of the quantized data input to the lossless encoding unit 16.

[0073] The lossless encoding unit 16 generates an encoded stream by encoding the quantized data input from the quantization unit 15. The lossless encoding unit 16 also encodes various parameters referenced by a decoder and inserts the encoding parameters into the encoded stream. The parameters encoded by the lossless encoding unit 16 may include parameters determined by the control unit 12 described above.

[0074] Furthermore, the parameters encoded by the lossless encoding unit 16 may include chrominance-related parameters. The chrominance-related parameters encoded by the lossless encoding unit 16 include, for example, a flag indicating whether or not there is a significant coefficient of the U component in each TU input from the orthogonal transform unit 14 as described above, and a flag indicating whether or not there is a significant coefficient of the V component in each TU. The lossless encoding unit 16 outputs the generated encoded stream to the accumulation buffer 17.

[0075] The accumulation buffer 17 temporarily accumulates the encoded stream input from the lossless encoding unit 16 using a storage medium such as a semiconductor memory. Then, the accumulation buffer 17 outputs the accumulated encoded stream to a transmission unit (not shown) (for example, a communication interface or a connection interface with a peripheral device) at a rate according to the bandwidth of the transmission path.

[0076] The rate control unit 18 monitors the free space in the accumulation buffer 17. Then, the rate control unit 18 generates a rate control signal according to the free space in the accumulation buffer 17 and outputs the generated rate control signal to the quantization unit 15. For example, when the free space in the accumulation buffer 17 is small, the rate control unit 18 generates a rate control signal for lowering the bit rate of the quantized data. Also, for example, when the free space in the accumulation buffer 17 is sufficiently large, the rate control unit 18 generates a rate control signal for increasing the bit rate of the quantized data.

[0077] The inverse quantization unit 21, the inverse orthogonal transformation unit 22, and the addition unit 23 constitute a local decoder. The local decoder plays a role in locally decoding a decoded image from the coded data.

[0078] The inverse quantization unit 21 inverse quantizes the quantized data using the same quantization parameters as those used by the quantization unit 15 to restore the transform coefficients. The inverse quantization unit 21 then outputs the restored transform coefficients to the inverse orthogonal transform unit 22.

[0079] The inverse orthogonal transform unit 22 restores the prediction error by performing an inverse orthogonal transform process on the transform coefficients input from the inverse quantization unit 21. Then, the inverse orthogonal transform unit 22 outputs the restored prediction error to the adder 23.

[0080] The adder 23 generates a decoded image (reconstructed image) by adding the reconstructed prediction error input from the inverse orthogonal transformer 22 and the predicted image input from the intra prediction unit 40 or the inter prediction unit 50. Then, the adder 23 outputs the generated decoded image to the in-loop filter 24 and the frame memory 30.

[0081] The in-loop filter 24 applies a series of in-loop filters to improve the image quality of the decoded image. For example, as described in "2.5. In-loop filtering" in Reference Document REF3, four in-loop filters may be applied in the following order: a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter. The in-loop filter 24 shown in FIG. 5 includes, for example, a bilateral filter 25, a deblocking filter 26a, an adaptive offset filter 27, and an adaptive loop filter 28, and the four in-loop filters may be applied in this order. However, the in-loop filter 24 is not limited to this configuration, and it may be possible to appropriately select which of the four in-loop filters to apply and in what order. The deblocking filter 26a will be described in detail later.

[0082] The in-loop filter 24 outputs the decoded image to which the in-loop filter has been applied to the frame memory 30 .

[0083] The frame memory 30 uses a storage medium to store the decoded image before filtering input from the adder 23 and the decoded image to which the in-loop filter has been applied input from the in-loop filter 24 .

[0084] The switch 31 reads out a decoded image before filtering to be used for intra prediction from the frame memory 30, and supplies the read decoded image as a reference image to the intra prediction unit 40. The switch 31 also reads out a decoded image after filtering to be used for inter prediction from the frame memory 30, and supplies the read decoded image as a reference image to the inter prediction unit 50.

[0085] The mode setting unit 32 sets a predictive coding mode for each block based on a comparison of costs input from the intra prediction unit 40 and the inter prediction unit 50. For a block for which intra prediction mode is set, the mode setting unit 32 outputs a predicted image generated by the intra prediction unit 40 to the subtraction unit 13 and the addition unit 23, and outputs information related to the intra prediction to the lossless coding unit 16. Furthermore, for a block for which inter prediction mode is set, the mode setting unit 32 outputs a predicted image generated by the inter prediction unit 50 to the subtraction unit 13 and the addition unit 23, and outputs information related to the inter prediction to the lossless coding unit 16.

[0086] The intra prediction unit 40 performs intra prediction processing based on the original image and the decoded image. For example, the intra prediction unit 40 evaluates a cost based on a prediction error and the amount of code generated for each prediction mode candidate included in a search range. Next, the intra prediction unit 40 selects the prediction mode with the smallest cost as the optimal prediction mode. The intra prediction unit 40 also generates a predicted image according to the selected optimal prediction mode. The intra prediction unit 40 then outputs information related to intra prediction, including prediction mode information indicating the optimal prediction mode, the corresponding cost, and the predicted image, to the mode setting unit 32.

[0087] The inter prediction unit 50 performs inter prediction processing (motion compensation) based on the original image and the decoded image. For example, the inter prediction unit 50 evaluates a cost based on a prediction error and the amount of code generated for each prediction mode candidate included in a certain search range. Next, the inter prediction unit 50 selects the prediction mode with the smallest cost, i.e., the prediction mode with the highest compression rate, as the optimal prediction mode. The inter prediction unit 50 also generates a predicted image according to the selected optimal prediction mode. The inter prediction unit 50 then outputs information related to inter prediction, the corresponding cost, and the predicted image to the mode setting unit 32.

[0088] [2-2. Image Decoding Device] Next, decoding of the data encoded as above will be described. Fig. 6 is a block diagram showing an example of the configuration of an image decoding device 60, which is one aspect of the image processing device according to this embodiment. Referring to Fig. 6, the image decoding device 60 includes an accumulation buffer 61, a lossless decoding unit 62, an inverse quantization unit 63, an inverse orthogonal transform unit 64, an adder 65, an in-loop filter 66, a sorting buffer 72, a D / A (Digital to Analogue) conversion unit 73, a frame memory 80, selectors 81a and 81b, an intra prediction unit 90, and an inter prediction unit 100.

[0089] The accumulation buffer 61 temporarily accumulates, using a storage medium, an encoded stream received from the image encoding device 10 via a transmission unit (not shown) (for example, a communication interface or a connection interface with a peripheral device).

[0090] The lossless decoding unit 62 decodes the coded stream input from the accumulation buffer 61 in accordance with the coding method used for coding, and generates quantized data. The lossless decoding unit 62 outputs the generated quantized data to the inverse quantization unit 63.

[0091] The lossless decoding unit 62 also parses various parameters from the encoded stream. The parameters parsed by the lossless decoding unit 62 may include, for example, information about intra prediction and information about inter prediction. The lossless decoding unit 62 outputs the information about intra prediction to the intra prediction unit 90. The lossless decoding unit 62 also outputs the information about inter prediction to the inter prediction unit 100.

[0092] Furthermore, the parameters parsed by the lossless decoding unit 62 may include chrominance-related parameters. The lossless decoding unit 62 outputs the chrominance-related parameters to the in-loop filter 66. Note that the chrominance-related parameters parsed by the lossless decoding unit 62 include, for example, a flag indicating whether or not there is a significant coefficient of the U component in each TU described above, and a flag indicating whether or not there is a significant coefficient of the V component in each TU.

[0093] The inverse quantization unit 63 inverse-quantizes the quantized data input from the lossless decoding unit 62 using the same quantization step as that used in encoding, thereby restoring the transform coefficients. The inverse quantization unit 63 outputs the restored transform coefficients to the inverse orthogonal transform unit 64.

[0094] The inverse orthogonal transform unit 64 generates prediction errors by performing inverse orthogonal transform on the transform coefficients input from the inverse quantization unit 63 in accordance with the orthogonal transform method used during encoding. The inverse orthogonal transform unit 64 outputs the generated prediction errors to the adder 65.

[0095] The adder 65 generates a decoded image by adding the prediction error input from the inverse orthogonal transformer 64 and the predicted image input from the selector 71b. Then, the adder 65 outputs the generated decoded image to the in-loop filter 66 and the frame memory 80.

[0096] The in-loop filter 66 applies a series of in-loop filters to improve the image quality of the decoded image. For example, as described in "2.5. In-loop filtering" in Reference Document REF3, four in-loop filters may be applied in the following order: a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter. The in-loop filter 66 shown in FIG. 6 includes, for example, a bilateral filter 67, a deblocking filter 26b, an adaptive offset filter 69, and an adaptive loop filter 70, and the four in-loop filters may be applied in this order. However, the in-loop filter 66 is not limited to this configuration, and it may be possible to appropriately select which of the four in-loop filters to apply and in what order. The deblocking filter 26b will be described in detail later.

[0097] The in-loop filter 66 outputs the decoded image to which the in-loop filter has been applied to the sorting buffer 72 and the frame memory 80 .

[0098] The sorting buffer 72 generates a series of images in time series by sorting the images input from the in-loop filter 66. Then, the sorting buffer 72 outputs the generated images to the D / A conversion unit 73.

[0099] The D / A conversion unit 73 converts the digital images input from the rearrangement buffer 72 into analog image signals. Then, the D / A conversion unit 73 outputs the analog image signals to, for example, a display (not shown) connected to the image decoding device 60, thereby displaying the video.

[0100] The frame memory 80 uses a storage medium to store the decoded image before filtering input from the adder 65 and the decoded image to which the in-loop filter has been applied input from the in-loop filter 66 .

[0101] The selector 81a switches the output destination of an image from the frame memory 80 between the intra prediction unit 90 and the inter prediction unit 100 for each block in the image, according to prediction mode information acquired by the lossless decoding unit 62. For example, when an intra prediction mode is specified, the selector 81a outputs the decoded image before filtering supplied from the frame memory 80 as a reference image to the intra prediction unit 90. Furthermore, when an inter prediction mode is specified, the selector 81a outputs the decoded image after filtering to the inter prediction unit 100 as a reference image.

[0102] The selector 81b switches the output source of the predicted image to be supplied to the adder 65 between the intra prediction unit 90 and the inter prediction unit 100, according to the prediction mode information acquired by the lossless decoding unit 62. For example, when an intra prediction mode is specified, the selector 81b supplies the predicted image output from the intra prediction unit 90 to the adder 65. Furthermore, when an inter prediction mode is specified, the selector 81b supplies the predicted image output from the inter prediction unit 100 to the adder 65.

[0103] The intra prediction unit 90 performs intra prediction processing based on the information related to intra prediction input from the lossless decoding unit 62 and the reference image from the frame memory 80, and generates a predicted image. Then, the intra prediction unit 90 outputs the generated predicted image to the selector 81b.

[0104] The inter prediction unit 100 performs inter prediction processing based on information related to inter prediction input from the lossless decoding unit 62 and the reference image from the frame memory 80, and generates a predicted image. Then, the inter prediction unit 100 outputs the generated predicted image to the selector 81b.

[0105] <3. Deblocking filter> [3-1. Example of deblocking filter configuration] This section describes an example of the configuration of the deblocking filter 26a of the image encoding device 10 shown in Fig. 5 and the deblocking filter 26b of the image decoding device 60 shown in Fig. 6. Note that the deblocking filter 26a and the deblocking filter 26b may have the same configuration. Therefore, in the following description, when there is no particular need to distinguish between the two, the deblocking filter 26a and the deblocking filter 26b will be collectively referred to as the deblocking filter 26.

[0106] As described above, the deblocking filter 26 according to this embodiment determines whether or not to apply a deblocking filter to the chrominance components of the decoded image based on bS calculated using chrominance-related parameters related to chrominance. Furthermore, as described above, the deblocking filter 26 according to this embodiment determines whether or not to apply a deblocking filter to the chrominance components of the decoded image based additionally on the size in the direction perpendicular to the block boundary. Furthermore, as described above, the deblocking filter 26 according to this embodiment may apply a strong filter with greater strength (stronger low-pass characteristics) than the strong filter in Non-Patent Document 2 to the chrominance components of the decoded image. Furthermore, in order to more appropriately apply such a strong filter, this embodiment determines the filter strength using a method different from the filter strength determination process in Non-Patent Document 2. Note that, hereinafter, the functions of the deblocking filter 26 related to the deblocking filter applied to the chrominance components of the decoded image will be mainly described, and the functions of the deblocking filter 26 related to the deblocking filter applied to the luminance component will be omitted as appropriate.

[0107] 7 is a block diagram showing an example of a detailed configuration of the deblocking filter 26 according to this embodiment. Referring to FIG. 7, the deblocking filter 26 includes a boundary strength calculation unit 261, a determination unit 263, and a filtering unit 269.

[0108] (1) Boundary strength calculation section The boundary strength calculation unit 261 calculates bS (boundary strength) using chrominance-related parameters related to chrominance for block boundaries of the decoded image. When a YUV420 format signal is being processed, the boundary strength calculation unit 261 calculates bS for every four lines in the luminance component of the decoded image, i.e., every two lines in the chrominance component of the decoded image.

[0109] In this embodiment, the chrominance-related parameters used by the boundary strength calculation unit 261 to calculate bS include a flag indicating whether or not there is a significant coefficient of the U component in each TU, and a flag indicating whether or not there is a significant coefficient of the V component in each TU. As shown in Fig. 7 , the boundary strength calculation unit 261 receives, as input, from the orthogonal transform unit 14 or the lossless decoding unit 62, a flag indicating whether or not there is a significant coefficient of each component (Y component, U component, V component) in each TU.

[0110] The boundary strength calculation unit 261 calculates bS based on condition A, condition B1-Y, condition B1-U, condition B1-V, and condition B2 described with reference to FIG. 4. That is, the boundary strength calculation unit 261 calculates bS based on whether significant coefficients of the chrominance components are present in the TUs sandwiching the block boundary for which bS is calculated. Furthermore, the boundary strength calculation unit 261 according to this embodiment can calculate bS by independently determining whether significant coefficients of the Y component, U component, and V component are present in the TUs sandwiching the block boundary for which bS is calculated. With this configuration, bS suitable for the U component and V component can be calculated, and a deblocking filter can be applied more appropriately, compared to calculating bS based on whether a significant coefficient of the Y component is present as described with reference to FIG. 2.

[0111] Calculation of bS by the boundary strength calculation unit 261 will be described in more detail with reference to FIG. 8. FIG. 8 is a table showing an example of bS calculated by the boundary strength calculation unit 261. The bS calculated by the boundary strength calculation unit 261 can be represented by a plurality of bits. In the example shown in FIG. 8, bS is represented by five bits. Furthermore, bS may be calculated such that the plurality of bits includes at least one bit corresponding to each of the Y component, U component, and V component. With this configuration, when the determination unit 263 (described later) determines whether or not to apply a deblocking filter based on bS, the determination can be easily made by referring to the bit of bS corresponding to each component to be determined.

[0112] Furthermore, the boundary strength calculation unit 261 may calculate bS such that each bit included in bS corresponds to the truth or falsity of each condition. In the example shown in FIG. 8, bS is calculated such that if each condition is true, the bit corresponding to the condition is 1, and if each condition is false, the bit corresponding to the condition is 0. In the example shown in FIG. 8, bS is expressed by 5 bits, and the 5th bit of bS corresponds to condition A regarding intra prediction, the 4th bit of bS corresponds to condition B1-Y regarding significant coefficients of the Y component, the 3rd bit of bS corresponds to condition B1-U regarding significant coefficients of the U component, the 2nd bit of bS corresponds to condition B1-V regarding significant coefficients of the V component, and the 1st bit of bS corresponds to condition B2 regarding MV and reference pictures. However, the correspondence between each bit of bS and each condition is not limited to the example shown in FIG. 8. For example, the order of the 4th bit, 3rd bit, and 2nd bit of bS, which correspond to the Y component, U component, and V component, respectively, may be interchanged.

[0113] (2) Judgment section 7, the determination unit 263 includes an application necessity determination unit 265 that determines whether a deblocking filter needs to be applied to the chrominance component of the decoded image, and a filter strength determination unit 267 that determines the filter strength of the deblocking filter to be applied to the chrominance component of the decoded image. Below, the functions of the application necessity determination unit 265 and the filter strength determination unit 267 will be described in order.

[0114] In the following description, the determination of whether or not to apply a deblocking filter to the chrominance components of the decoded image and the filter strength will be mainly described, and the description of the determination for the luminance component will be omitted as appropriate. Furthermore, the application necessity determination unit 265 and the filter strength determination unit 267 according to this embodiment determine whether or not to apply a deblocking filter and the filter strength for each of the U component and the V component independently.

[0115] The application necessity determination unit 265 targets block boundaries of the decoded image and determines whether or not a deblocking filter needs to be applied to the chrominance components of the decoded image based on the bS (boundary strength) calculated by the boundary strength calculation unit 261 as described above.

[0116] The application necessity determination unit 265 may further determine whether or not a deblocking filter needs to be applied to the chrominance components of the decoded image based on the block sizes of the blocks sandwiching the block boundary. Note that, hereinafter, such a determination based on block size may be referred to as a large block determination. Also, the application necessity determination unit 265 does not always need to perform large block determination for all block boundaries, and may determine whether or not to perform large block determination based on bS. Note that cases in which large block determination is performed and details of large block determination will be described later.

[0117] The application necessity determining unit 265 according to this embodiment determines whether or not a deblocking filter needs to be applied based on the determination of the following condition C1 and the determination of the following condition C2.

[0118] -Condition C1:(bS==16||(Condition C11&&Condition C12)) -Condition C2:d <beta

[0119] Condition C11 in the above condition C1 is a condition for determining whether or not to perform large block determination, and condition C12 is a condition related to the large block determination. When bS is 16, that is, when condition A related to intra prediction is satisfied, condition C1 may be determined to be true without the need to perform large block determination. Therefore, condition C11 for determining whether or not to perform large block determination may be true when bS has a value related to inter prediction. Note that by skipping large block determination and determining condition C1 to be true when bS is 16 in this way, the amount of processing required for large block determination can be reduced.

[0120] Also, if the condition C11 of the condition C1 is false, the determination of the condition C12 (large block determination) is not performed, and the condition C1 is determined to be false. With this configuration, the amount of processing required for the large block determination can be reduced.

[0121] Condition C11 may be true when a condition regarding the significant coefficient of each component or the above-mentioned condition B2 is true. That is, condition C11 may differ depending on the component to be determined. For example, when the component to be determined is the U component, condition C11 may be a condition such as condition C11-U below, and when the component to be determined is the V component, condition C11 may be a condition such as condition C11-V below.

[0122] -Condition C11-U:(bS&0x04||bS&0x01) -Condition C11-V:(bS&0x02||bS&0x01)

[0123] Furthermore, the application necessity determination unit 265 performs large block determination based on the size of the blocks on either side of the block boundary in a direction perpendicular to the block boundary. With this configuration, when the shape of the block is a non-square rectangle, it becomes possible to determine whether or not to apply a deblocking filter based on the size in the direction perpendicular to the block boundary, which is likely to affect the occurrence of block distortion.

[0124] The application necessity determination unit 265 may also perform large block determination based on whether the size of blocks on either side of a block boundary in a direction perpendicular to the block boundary is greater than a predetermined threshold. The threshold used in this large block determination is not limited, but may be, for example, 16. When the size in the direction perpendicular to the block boundary is small, particularly when it is 16 or less, block noise is less noticeable, so this configuration makes it possible to avoid applying unnecessary deblocking filters. For example, condition C12 for large block determination may be the following condition:

[0125] -Condition C12:(EDGE_VER&&block_width>16)||(EDGE_HOR&&block_height>16)

[0126] In the above condition C12, EDGE_VER means that the block boundary to be determined is a vertical boundary, and EDGE_HOR means that the block boundary to be determined is a horizontal boundary.

[0127] Furthermore, the above condition C2 is the same as the above-mentioned condition C92, and therefore description thereof will be omitted here. Note that the determination of the above condition C2 is made when the condition C1 is true, and when the condition C1 is false, the determination of the condition C2 is not made, and it is determined that a deblocking filter is not to be applied. The determination of the condition C2 requires a process of calculating the variable d as in the above-mentioned formulas (1) to (7), and therefore requires a larger amount of processing than the determination of the condition C1. Therefore, by making the determination of the condition C2 after the condition C1, it is possible to reduce the amount of processing.

[0128] Furthermore, after determining whether or not a deblocking filter needs to be applied based on conditions C1 and C2 as described above, the filter strength determination unit 267 further determines the filter strength of the deblocking filter to be applied to the chrominance component of the decoded image. As will be described later, the deblocking filters that can be applied in this embodiment may be of two types: a weak filter having weaker strength and a strong filter having stronger strength. The filtering unit 269, which will be described later, applies either the weak filter or the strong filter depending on the filter strength determined by the filter strength determination unit 267.

[0129] When it is determined that a deblocking filter is to be applied, the filter strength determination unit 267 determines the filter strength. By determining the filter strength after determining whether or not a deblocking filter needs to be applied, it is possible to reduce the processing involved in determining the filter strength.

[0130] Furthermore, the filter strength determination unit 267 determines the filter strength based on the waveform of the color difference component of a pixel located near a block boundary. The determination based on the waveform will be described below. The filter strength determination unit 267 determines the filter strength based on the following waveform-based condition C3.

[0131] -Condition C3: (Condition C31&&Condition C32&&Condition C33) -Condition C31:|p3-p0|+|q3-q0|<(beta>>3) -Condition C32:|p2-2*p1+p0|+|q2-2*q1+q0|<(beta>>2) -Condition C33:|p0-q0|<((t c *5+1)>>1)

[0132] The filter strength determination unit 267 performs the determination of the above condition C3 for pixels included in two lines among pixels located near the block boundary. The conditions C31, C32, and C33 used in the above C3 are determined for each line. Note that the p i , q k , pi ′, q k ′, beta, and t C Since this has already been explained above, the explanation will be omitted here.

[0133] Conditions C31, C32, and C33 are conditions used for determining the gap between blocks using the pixels included in each line. More specifically, condition C31 is a condition related to the flatness of the chrominance components of the pixels included in each line within a block. Condition C32 is a condition related to determining the continuity of the chrominance components of the pixels included in each line within a block. Condition C33 is a condition related to the gap between blocks (difference) between the chrominance components of the pixels included in each line, and more specifically, is a condition for determining the gap between blocks using pixel values adjacent to the block boundary.

[0134] If condition C31 is true, the waveform of the chrominance component is highly flat within each block. If condition C32 is true, the waveform of the chrominance component is highly continuous within each block. If condition C32 is true, the waveform of the chrominance component has large gaps at block boundaries.

[0135] As described above, condition C3 is determined to be true when all of the above conditions C31, C32, and C33 are true. Furthermore, the filter strength determination unit 267 determines the above condition C3 for each line. However, as described above, the filter strength is determined in units of two lines. That is, if the above condition C3 is true for both of two consecutive lines, a strong filter is applied to those two lines, and if it is false, a weak filter is applied to those two lines.

[0136] (3) Filtering section The filtering unit 269 applies a deblocking filter to the chrominance components of pixels located near block boundaries, based on the determination result of whether or not to apply a deblocking filter made by the application necessity determination unit 265. Furthermore, as described above, the filtering unit 269 applies a weak filter or a strong filter as a deblocking filter, depending on the filter strength determined by the filter strength determination unit 267.

[0137] The weak filter applied to the chrominance component by the filtering unit 269 according to this embodiment may be the same as the weak filter applied to the chrominance component of a decoded image in, for example, the above-mentioned Non-Patent Document 2 or HEVC. On the other hand, the strong filter applied to the chrominance component in this embodiment may be different from the strong filter applied to the chrominance component in Non-Patent Document 2 (the strong filter applied to the luminance component in HEVC). An example of the strong filter applied to the chrominance component in this embodiment will be described below.

[0138] In this embodiment, the coefficient of the strong filter applied to the chrominance components may be 2 at the center position of the application range of the strong filter, and 1 at other positions. Furthermore, the filtering unit 269 may set the application range of the strong filter from the block boundary to three pixels on either side, and apply the strong filter to the chrominance components of the pixels included in the application range, using three pixels on either side of the center position of the application range as reference pixels. For example, a strong filter with p0 as the center position of the application range is expressed as in the following equation (14).

[0139] p0′=Clip3(p 0- w*t C ,p0+w*t C ,((p3+p2+p1+2*p0+q0+q1+q2+4)>>3)) …(14)

[0140] In the above equation (14), w is a weight that can be set appropriately, and may be set to, for example, 1 or 2. Also, Clip3(a, b, c) represents the clipping process that clips the value c within the range a≦c≦b, as described above.

[0141] By applying such a strong filter, it becomes possible to apply a deblocking filter that is stronger than the strong filter applied to the chrominance component in Non-Patent Document 2 mentioned above.

[0142] Incidentally, when the center position of the range to which the strong filter is applied is the second or third pixel from the block boundary, the reference pixels include pixels that are five pixels or more away from the block boundary. However, pixels that are five pixels or more away from the block boundary are not used in determining the filter strength, and may not be suitable for use as reference pixels. Therefore, the filtering unit 269 may use the pixel value of the fourth pixel from the block boundary as the pixel value of the reference pixel by padding it, instead of the pixel that is five pixels or more away from the block boundary.

[0143] For example, a strong filter with p1 as the center position of the application range is expressed as in the following equation (15). p1′=Clip3(p 1- w*t C ,p1+w*t C ,((p4+p3+p2+2*p1+p0+q0+q1+4)>>3)) =Clip3(p 1- w*t C ,p1+w*t C ,((p3+p3+p2+2*p1+p0+q0+q1+4)>>3)) =Clip3(p 1- w*t C ,p1+w*t C ,((2*p3+p2+2*p1+p0+q0+q1+4)>>3)) …(15)

[0144] Similarly, a strong filter with p2 as the center position of the application range is expressed as in the following equation (16). p2′=Clip3(p 2- w*t C ,p2+w*t C ,((p5+p4+p3+2*p2+p1+p0+q0+4)>>3)) =Clip3(p 2- w*t C ,p2+w*t C ,((p3+p3+p3+2*p2+p1+p0+q0+4)>>3)) =Clip3(p 2- w*t C ,p2+w*t C ,((3*p3+2*p2+p1+p0+q0+4)>>3)) …(16)

[0145] Similarly, strong filters with q0 to q3 as the center positions of the application range are expressed by the following equations (17) to (19), respectively. q0′=Clip3(q0-w*t C ,q0+w*t C ,((p2+p1+p0+2*q0+q1+q2+q3+4)>>3)) …(17) q1′=Clip3(q1-w*t C ,q1+w*t C ,((p1+p0+q0+2*q1+q2+2*q3+4)>>3)) …(18) q2′=Clip3(q2-w*t C ,q2+w*t C ,((p0+q0+q1+2*q2+3*q3+4)>>3)) …(19)

[0146] [3-2. Processing flow] An example of the configuration of the deblocking filter 26 according to this embodiment has been described above. Next, a processing flow by the deblocking filter 26 according to this embodiment will be described. Fig. 9 is a flowchart showing an example of the processing flow by the deblocking filter 26 according to this embodiment. Note that, of the processing by the deblocking filter 26, the following description will focus on processing related to the characteristics of this embodiment, and descriptions of other processing will be omitted as appropriate.

[0147] First, the boundary strength calculation unit 261 calculates bS (boundary strength) (S10). Here, the method of calculating bS will be described in more detail with reference to Fig. 10. Fig. 10 is a flowchart for explaining the flow of the boundary strength calculation process (S10) executed by the boundary strength calculation unit 261.

[0148] First, the boundary strength calculation unit 261 initializes bS to 0 (S102). Next, the boundary strength calculation unit 261 determines whether condition A, which is a condition related to intra prediction, is true or false (S104). If condition A is true (YES in S104), bS is set to 16 (S106).

[0149] On the other hand, if condition A is false (NO in S104), boundary strength calculation unit 261 determines whether condition B2, which is a condition regarding motion vectors (MVs) and reference pictures, is true (S108). If condition B2 is true (YES in S108), bS is set to 1 (S110).

[0150] On the other hand, if the condition B2 is false (NO in S108), the boundary strength calculation unit 261 determines whether the condition B1-Y, which is a condition regarding the presence or absence of a significant coefficient of the Y component, is true or false (S112). If the condition B1-Y is true (YES in S112), 8 is added to bS (S114), and then the process proceeds to step S116. On the other hand, if the condition B1-Y is false (NO in S112), the process proceeds directly to step S116.

[0151] In step S116, the boundary strength calculation unit 261 determines whether condition B1-U, which is a condition regarding the presence or absence of a significant coefficient in the U component, is true or false. If condition B1-U is true (YES in S116), 4 is added to bS (S118), and then the process proceeds to step S120. On the other hand, if condition B1-U is false (NO in S116), the process proceeds directly to step S120.

[0152] In step S120, the boundary strength calculation unit 261 determines whether a condition B1-V, which is a condition regarding the presence or absence of a significant coefficient in the V component, is true or false. If the condition B1-V is true (YES in S120), 2 is added to bS (S122), and then the boundary strength calculation process (S10) ends. If the condition B1-V is false (NO in S120), the boundary strength calculation process (S10) ends.

[0153] 9, the description of the processing flow by the deblocking filter 26 will continue. In step S20, the application necessity determination unit 265 of the determination unit 263 determines whether the above-mentioned condition C1 is true or false. If the condition C1 is false (NO in S20), the processing ends.

[0154] On the other hand, if the condition C1 is true (YES in S20), the application necessity determining unit 265 determines whether the above-mentioned condition C2 is true (S30). If the condition C2 is false (NO in S30), the processing ends.

[0155] On the other hand, if condition C2 is true (YES in S30), the filter strength determination unit 267 of the determination unit 263 determines the filter strength by determining whether the above-mentioned condition C3 is true (S40). If condition C3 is true (YES in S40), the filtering unit 269 applies a strong filter to the chrominance components of pixels located near the block boundary (S50). On the other hand, if condition C3 is false (NO in S40), the filtering unit 269 applies a weak filter to the chrominance components of pixels located near the block boundary (S60).

[0156] The above has described the flow of processing by the deblocking filter 26 according to this embodiment. Note that, in the case of the YUV420 format, for example, the above-described processing described with reference to Fig. 9 and Fig. 10 can be performed in units of four lines in the luminance component of the decoded image, that is, in units of two lines in the chrominance component of the decoded image.

[0157] [3-3. Modifications] 11 is a table showing a modified example of bS calculated by the boundary strength calculation unit 261. As in the example of FIG. 8, bS is expressed by five bits here as well. The multiple bits of bS include at least one bit corresponding to each of the Y component, U component, and V component. The fifth bit (most significant bit) of bS corresponds to condition A regarding intra prediction, the fourth bit of bS corresponds to condition B1-Y regarding significant coefficients of the Y component, the third bit of bS corresponds to condition B1-U regarding significant coefficients of the U component, the second bit of bS corresponds to condition B1-V regarding significant coefficients of the V component, and the first bit (least significant bit) of bS corresponds to condition B2 regarding MV and reference picture.

[0158] Unlike the example of Fig. 8, in this modification, the first bit of bS is set to 1, especially when condition B1-Y is false (and therefore bS is less than 8). As a result, while bS can take one of 10 possible values, 0, 1, 2, 4, 6, 8, 10, 12, 14, or 16, in the example of Fig. 8, bS can take one of 13 possible values, 0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, or 16, in this modification.

[0159] 12A and 12B are flowcharts illustrating an example of the flow of boundary strength calculation processing corresponding to the modified example of FIG. 11. Before the start of processing, bS is assumed to be initialized to 0. Referring to FIG. 12A, first, the boundary strength calculation unit 261 determines whether condition A, which is a condition related to intra prediction, is true or false (S150). If condition A is true, bS is set to 16 (S152), and the boundary strength calculation processing ends.

[0160] If condition A is false, the boundary strength calculation unit 261 determines whether condition B1-Y, which is a condition regarding the presence or absence of a significant coefficient in the Y component, is true (S154). If condition B1-Y is true, 8 is added to bS (S156), and if condition B1-Y is false, the addition of 8 is skipped (S158). Next, the boundary strength calculation unit 261 determines whether condition B1-U, which is a condition regarding the presence or absence of a significant coefficient in the U component, is true (S160). If condition B1-U is true, 4 is added to bS (S162), and if condition B1-U is false, the addition of 4 is skipped (S164). Next, the boundary strength calculation unit 261 determines whether condition B1-V, which is a condition regarding the presence or absence of a significant coefficient in the V component, is true (S166). If condition B1-V is true, 2 is added to bS (S168), and if condition B1-V is false, the addition of 2 is skipped (S170).

[0161] 12B, the subsequent processing branches depending on whether condition B1-Y, which is a condition regarding the presence or absence of a significant coefficient of the Y component, is true or false (S172). If condition B1-Y is true, the fourth bit of bS is equal to 1, and therefore bS is set to a value equal to or greater than 8 (14, 12, 10, or 8) (S174), and the boundary strength calculation processing ends without performing the MV check described below.

[0162] On the other hand, if the condition B1-Y is false, the boundary strength calculation unit 261 executes an MV check (S176). The MV check here means determining whether the condition B2, which is a condition regarding the motion vector and the reference picture, is true or false. If the condition B2 is true, 1 is added to bS (S178), and if the condition B2 is false, the addition of 1 is skipped (S180).

[0163] <4. Hardware configuration example> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0164] FIG. 13 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program.

[0165] In a computer 800 shown in FIG. 13, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are interconnected via a bus 804.

[0166] An input / output interface 810 is also connected to the bus 804. To the input / output interface 810, an input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected.

[0167] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 812 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 813 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 814 includes, for example, a network interface. The drive 815 drives removable media 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0168] In the computer configured as above, the CPU 801 performs the above-described series of processes by, for example, loading a program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executing the program. The RAM 803 also stores data necessary for the CPU 801 to execute various processes as appropriate.

[0169] The program executed by the computer (CPU 801) can be applied by recording it on removable media 821 such as package media, for example. In this case, the program can be installed in the storage unit 813 via the input / output interface 810 by inserting the removable media 821 into the drive 815.

[0170] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0171] Alternatively, this program can be installed in advance in the ROM 802 or the storage unit 813.

[0172] <5. Conclusion> As described above, according to the embodiments of the present disclosure, it is possible to more appropriately apply a deblocking filter to the chrominance components of a decoded image.

[0173] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0174] (Color difference related parameters) For example, in the above embodiment, an example was described in which a flag indicating whether or not a significant coefficient of a chrominance component in each TU is used as the chrominance-related parameter. However, the present technology is not limited to such an example. For example, the transform coefficients of the chrominance components themselves may be included in the chrominance-related parameters. In such a case, the boundary strength calculation unit 261 may calculate bS by determining whether or not a significant coefficient of a chrominance component in each TU is present from the transform coefficients of the chrominance components. Also, in relation to the above embodiment, FIG. 4 shows an example in which the value of bS varies depending not only on whether or not condition B1-Y, B1-U, or B1-V is satisfied, but also on whether or not condition B2 is satisfied. However, as in an alternative example shown in FIG. 14, for example, in order to suppress an increase in processing cost, the determination of whether or not condition B2 is satisfied for both the chrominance components U and V may be omitted.

[0175] (Large block threshold) In the above embodiment, an example has been described in which the threshold value used in large block determination is 16, but the present technology is not limited to this example, and the threshold value may be set to 8 or 32. Furthermore, in the case of the YUV444 format, a threshold value equal to or greater than the threshold value used in the YUV420 format may be used in large block determination.

[0176] (Strong filter) In the above embodiment, an example has been described in which the strong filters expressed by equations (15) to (19) are applied to the chrominance components, but the strong filters applied in the present technology are not limited to such an example. The strong filter applied to the chrominance components may be a filter with a stronger filter strength than the weak filter. For example, the strong filter applied to the chrominance components in Non-Patent Document 2 (a strong filter applied to the luminance component in HEVC) may be applied to the chrominance components in the present technology.

[0177] (Target of application of this technology) This technology can be applied to any image encoding / decoding method. In other words, as long as it does not conflict with the above-mentioned technology, the specifications of various processes related to image encoding / decoding, such as transform (inverse transform), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and are not limited to the above-mentioned examples. Furthermore, as long as it does not conflict with the above-mentioned technology, some of these processes may be omitted.

[0178] (block) Furthermore, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region of an image (picture) or a processing unit refers to any partial region within a picture, and its size, shape, characteristics, etc. are not limited unless otherwise specified. For example, a "block" is intended to include any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), transform block, sub-block, macroblock, tile, or slice described in the above-mentioned reference documents REF1 to REF3.

[0179] (processing unit) The data units in which the various pieces of information described above are set and the data units targeted by the various processes are each arbitrary and are not limited to the above examples. For example, these pieces of information and processes may be set for each TU (Transform Unit), TB (Transform Block), PU (Prediction Unit), PB (Prediction Block), CU (Coding Unit), LCU (Largest Coding Unit), sub-block, block, tile, slice, picture, sequence, or component, or may target data of these data units. Of course, these data units may be set for each piece of information or process, and the data units for all pieces of information and processes do not need to be unified. Note that the storage location of these pieces of information is arbitrary and may be stored in the headers or parameter sets of the above-mentioned data units, or may be stored in multiple locations.

[0180] In the above embodiment, deblocking filter processing is performed on the chrominance components in units of two lines, but the present technology is not limited to this example. For example, in the case of the YUV444 format, deblocking filter processing may be performed on the chrominance components in units of four lines. In such a case, the application necessity determination unit 265 may make a determination regarding the above-described condition C3 by referring to the first and third lines.

[0181] (control information) Control information related to the present technology described above may be transmitted from the encoding side to the decoding side. For example, control information (e.g., enabled_flag) that controls whether or not to permit (or prohibit) application of the present technology described above may be transmitted. Also, for example, control information indicating targets to which the present technology described above is to be applied (or targets to which it is not to be applied) may be transmitted. For example, control information specifying a block size (upper or lower limit, or both), frame, component, or layer to which the present technology is to be applied (or permitted or prohibited to be applied) may be transmitted.

[0182] (Block size information) When specifying the size of a block to which the present technology is applied, the block size may be specified not only directly but also indirectly. For example, the block size may be specified using identification information that identifies the size. Furthermore, for example, the block size may be specified by a ratio or difference from the size of a reference block (e.g., LCU, SCU, etc.). For example, when transmitting information specifying the block size as a syntax element, the information indirectly specifying the size as described above may be used as the information. By doing so, the amount of information can be reduced, and coding efficiency may be improved. Furthermore, the specification of the block size also includes specification of a range of block sizes (e.g., specification of a range of allowable block sizes, etc.).

[0183] (others) In this specification, a "flag" is information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. In other words, the number of bits that make up this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be included not only in the bitstream, but also as an identification relative to certain reference information. Since it is also possible that difference information from other information may be included in the bitstream, in this specification, "flag" and "identification information" include not only that information but also difference information from the reference information.

[0184] Furthermore, various types of information (metadata, etc.) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, making one piece of data available (linked) when processing the other piece of data. In other words, mutually associated data may be combined into one piece of data or may be individual pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of the same recording medium). Note that this "association" may refer to only a portion of the data, rather than the entire data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0185] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0186] The present technology can also be implemented as any configuration that constitutes an apparatus or a system, for example, a processor as a system LSI (Large Scale Integration), a module using multiple processors, a unit using multiple modules, a set in which other functions are added to a unit, or the like (i.e., a configuration of a part of an apparatus).

[0187] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0188] Also, for example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit). Also, for example, the present technology can be configured as a cloud computing system in which one function is shared and processed jointly by multiple devices via a network.

[0189] Also, for example, each step described in the above flowchart can be executed by one device or can be shared and executed by multiple devices. Furthermore, if one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0190] In addition, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0191] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0192] The following configurations also fall within the technical scope of the present disclosure. (1) a decoding unit that decodes the coded stream to generate a decoded image; a determination unit that determines whether or not a deblocking filter needs to be applied to a chrominance component of the decoded image based on a boundary strength calculated using a chrominance-related parameter related to chrominance, the boundary strength being targeted at a block boundary of the decoded image; a filtering unit that applies a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; An image processing device comprising: (2) the chrominance-related parameters include information about transform coefficients of chrominance components; The image processing device according to (1), wherein the boundary strength is calculated based on whether or not significant coefficients of the color difference components exist in blocks sandwiching the block boundary that is the target of calculation of the boundary strength. (3) The image processing device according to (2), wherein the boundary strength is calculated by independently determining whether or not a significant coefficient of each component exists in the blocks sandwiching the block boundary for which the boundary strength is to be calculated. (4) the boundary strength is represented by a plurality of bits; The image processing device according to any one of (1) to (3), wherein the plurality of bits include at least one bit corresponding to each component. (5) The image processing device according to any one of (1) to (4), wherein the determination unit determines whether or not the deblocking filter needs to be applied based on the boundary strength and a large block determination using the block sizes of the blocks on either side of the block boundary. (6) The image processing device according to (5), wherein the determination unit performs the large block determination when the boundary strength has a value related to inter prediction. (7) The image processing device according to (5) or (6), wherein the determination unit performs the large block determination based on sizes of blocks on either side of the block boundary in a direction perpendicular to the block boundary. (8) The image processing device according to (7), wherein the determination unit performs the large block determination based on whether or not the size of the blocks sandwiching the block boundary in a direction perpendicular to the block boundary is greater than 16. (9) The image processing device according to any one of (1) to (8), wherein the filtering unit applies a weak filter or a strong filter as the deblocking filter to color difference components of pixels located near the block boundary. (10) The image processing device according to (9), wherein the coefficient of the strong filter is 2 at the center position of the range to which the strong filter is applied and 1 at other positions. (11) The image processing device according to (9) or (10), wherein the filtering unit defines an application range of the strong filter from the block boundary to three pixels on either side, and applies the strong filter to the color difference components of pixels included in the application range using three pixels on either side of a center position of the application range as reference pixels. (12) The image processing device according to (11), wherein the filtering unit uses the pixel value of the fourth pixel from the block boundary as the pixel value of the reference pixel instead of a pixel that is five pixels or more away from the block boundary by padding the pixel value. (13) The filtering unit is configured to filter a parameter t C The image processing device according to any one of (10) to (12), wherein the strong filter is applied by performing clipping processing based on the above formula. (14) the determination unit further determines a filter strength of the deblocking filter to be applied to a chrominance component of a pixel located near the block boundary; The image processing device according to any one of (9) to (13), wherein the filtering unit applies the weak filter or the strong filter to the color difference components of pixels located near the block boundary, depending on the filter strength determined by the determination unit. (15) The image processing device according to (14), wherein the determining unit determines the filter strength after determining whether or not the deblocking filter needs to be applied. (16) The image processing device according to (15), wherein the determining unit determines the filter strength based on waveforms of color difference components of pixels located near the block boundary. (17) the determination unit determines the filter strength based on a condition regarding flatness, a condition regarding continuity, and a condition regarding a gap of color difference components of pixels included in two lines among pixels located near the block boundary; The image processing device according to (16), wherein the filtering unit applies the strong filter when it is determined that all of the conditions related to the flatness, the continuity, and the gap are satisfied, and applies the weak filter when it is determined that at least one of the conditions is not satisfied. (18) decoding the coded stream to generate a decoded image; determining whether or not a deblocking filter needs to be applied to a chrominance component of the decoded image based on a boundary strength calculated using a chrominance-related parameter related to chrominance, with respect to a block boundary of the decoded image; applying a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; An image processing method comprising: (19) a determination unit that determines whether or not a deblocking filter needs to be applied to a chrominance component of a locally decoded image based on a boundary strength calculated using a chrominance-related parameter related to chrominance, with respect to a block boundary of the locally decoded image; a filtering unit that applies a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; an encoding unit that encodes an image using the decoded image to which the deblocking filter has been applied by the filtering unit; An image processing device comprising: (20) determining whether or not a deblocking filter needs to be applied to a chrominance component of a locally decoded image based on a boundary strength calculated using a chrominance-related parameter related to chrominance, for a block boundary of the locally decoded image; applying a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; encoding an image using the decoded image to which the deblocking filter has been applied; An image processing method comprising: [Explanation of symbols]

[0193] 10 Image encoding device 16 Lossless encoding section 26 Deblocking Filter 60 Image decoding device 62 Lossless Decoding Unit 261 Boundary strength calculation unit 263 Judgment section 265 Application necessity judgment part 267 Filter strength determination unit 269 Filtering Section

Claims

1. a decoding unit that decodes the coded stream to generate a decoded image; a determination unit that determines whether or not a deblocking filter needs to be applied to the chrominance components of the decoded image based on a boundary strength expressed by at least 5 bits, the boundary strength being calculated using a chrominance-related parameter that is information about transform coefficients of the chrominance components obtained by orthogonal transform, with respect to a block boundary of the decoded image; a filtering unit that applies a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; Equipped with the boundary strength is calculated independently for each of the first component and the second component by independently determining whether or not a significant coefficient of a first component and a significant coefficient of a second component are present among components included in the color difference components in blocks sandwiching the block boundary for which the boundary strength is to be calculated; the boundary strength is represented by a plurality of bits including at least one bit corresponding to each of the color difference components; The determination unit determining whether or not the deblocking filter needs to be applied based on a determination based on the boundary strength and a large block determination based on the size of blocks sandwiching the block boundary in a direction perpendicular to the block boundary and a direction parallel to the block boundary; Image processing device.

2. The boundary strength is expressed by five bits, each bit corresponding to a condition regarding intra prediction, a condition regarding a significant coefficient of a Y component, a condition regarding a significant coefficient of a U component of the color difference component, a condition regarding a significant coefficient of a V component of the color difference component, and a condition regarding a motion vector and a reference picture. The image processing device according to claim 1 .

3. The boundary strength is If the condition regarding the intra prediction is true, a bit of the five bits corresponding to the condition regarding the intra prediction is set to a first value indicating true, and the other bits are set to a second value indicating not true; when the condition regarding the intra prediction is false and the condition regarding the motion vector and the reference picture is true, a bit among the five bits corresponding to the condition regarding the motion vector and the reference picture is set to the first value, and the other bits are set to the second value; If the condition regarding intra prediction is false and the condition regarding motion vectors and reference pictures is false, a bit corresponding to the condition regarding the intra prediction and a bit corresponding to the condition regarding the motion vector and the reference picture among the five bits are set to the second value; each other bit is set to the first value when the condition corresponding to the bit is true and is set to the second value when the condition corresponding to the bit is false; The image processing device according to claim 2 .

4. The boundary strength is If the condition regarding the intra prediction is true, a bit among the five bits corresponding to the condition regarding the intra prediction is set to a first value indicating true, and the other bits are set to a second value indicating not true; If the condition regarding the intra prediction is false, a bit corresponding to the condition regarding the intra prediction among the five bits is set to the second value; three bits among the five bits corresponding to a condition regarding a significant coefficient of the Y component, a condition regarding a significant coefficient of the U component, and a condition regarding a significant coefficient of the V component are set to the first value when the condition corresponding to the bit is true, and are set to the second value when the condition corresponding to the bit is false; If the condition regarding intra prediction is false and the condition regarding significant coefficients of the Y component is true, a bit among the five bits corresponding to a condition regarding the motion vector and the reference picture is set to the second value; If the condition regarding intra prediction is false and the condition regarding significant coefficients of the Y component is false, a bit corresponding to a condition regarding the motion vector and the reference picture is set to the first value when the condition regarding the motion vector and the reference picture is true, and is set to the second value when the condition regarding the motion vector and the reference picture is false; The image processing device according to claim 2 .

5. the five bits correspond to the condition regarding the intra prediction, the condition regarding the significant coefficient of the Y component, the condition regarding the significant coefficient of the U component, the condition regarding the significant coefficient of the V component, and the condition regarding the motion vector and the reference picture, in order from the most significant bit to the least significant bit of the five bits; 5. The image processing device according to claim 2, wherein the image processing device is a computer.

6. the determination unit performs the large block determination when the boundary strength has a value related to inter prediction. The image processing device according to claim 1 .

7. the color difference components are composed of a U component and a V component, the first component is the U component, and the second component is the V component; The image processing device according to claim 1 .

8. The color difference related parameters include a flag indicating whether a significant coefficient is included in a transform block of the orthogonal transform. The image processing device according to claim 1 .

9. The image processing device according to claim 1 , wherein the determining unit determines whether the block is large based on whether or not the size of the blocks sandwiching the block boundary in a direction perpendicular to the block boundary is greater than 8.

10. decoding the coded stream to generate a decoded image; determining whether or not a deblocking filter needs to be applied to the chrominance components of the decoded image based on a boundary strength expressed by at least 5 bits, the boundary strength being calculated using a chrominance-related parameter that is information about transform coefficients of the chrominance components obtained by orthogonal transform, with respect to a block boundary of the decoded image; applying a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; Including, the boundary strength is calculated independently for each of the first component and the second component by independently determining whether or not a significant coefficient of a first component and a significant coefficient of a second component are present among components included in the color difference components in blocks sandwiching the block boundary for which the boundary strength is to be calculated; the boundary strength is represented by a plurality of bits including at least one bit corresponding to each of the color difference components; The determining step comprises: determining whether or not the deblocking filter needs to be applied based on a determination based on the boundary strength and a large block determination based on the sizes of blocks sandwiching the block boundary in a direction perpendicular to the block boundary and a direction parallel to the block boundary; Image processing methods.

11. a determination unit that determines whether or not a deblocking filter needs to be applied to a chrominance component of a decoded image decoded by a local decoding process, based on a boundary strength expressed by at least 5 bits, calculated using a chrominance-related parameter that is information about a transform coefficient of the chrominance component obtained by an orthogonal transform, for a block boundary of the decoded image; a filtering unit that applies a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; an encoding unit that encodes an image using the decoded image to which the deblocking filter has been applied by the filtering unit; Equipped with the boundary strength is calculated independently for each of the first component and the second component by independently determining whether or not a significant coefficient of a first component and a significant coefficient of a second component are present among components included in the color difference components in blocks sandwiching the block boundary for which the boundary strength is to be calculated; the boundary strength is represented by a plurality of bits including at least one bit corresponding to each of the color difference components; The determination unit determining whether or not the deblocking filter needs to be applied based on a determination based on the boundary strength and a large block determination based on the size of blocks sandwiching the block boundary in a direction perpendicular to the block boundary and a direction parallel to the block boundary; Image processing device.

12. The boundary strength is expressed by five bits, each bit corresponding to a condition regarding intra prediction, a condition regarding a significant coefficient of a Y component, a condition regarding a significant coefficient of a U component of the color difference component, a condition regarding a significant coefficient of a V component of the color difference component, and a condition regarding a motion vector and a reference picture. The image processing device according to claim 11 .

13. The boundary strength is If the condition regarding the intra prediction is true, a bit of the five bits corresponding to the condition regarding the intra prediction is set to a first value indicating true, and the other bits are set to a second value indicating not true; when the condition regarding the intra prediction is false and the condition regarding the motion vector and the reference picture is true, a bit among the five bits corresponding to the condition regarding the motion vector and the reference picture is set to the first value, and the other bits are set to the second value; If the condition regarding intra prediction is false and the condition regarding motion vectors and reference pictures is false, a bit corresponding to the condition regarding the intra prediction and a bit corresponding to the condition regarding the motion vector and the reference picture among the five bits are set to the second value; each other bit is set to the first value when the condition corresponding to the bit is true and is set to the second value when the condition corresponding to the bit is false; The image processing device according to claim 12.

14. The boundary strength is If the condition regarding the intra prediction is true, a bit among the five bits corresponding to the condition regarding the intra prediction is set to a first value indicating true, and the other bits are set to a second value indicating not true; If the condition regarding the intra prediction is false, a bit corresponding to the condition regarding the intra prediction among the five bits is set to the second value; three bits among the five bits corresponding to a condition regarding a significant coefficient of the Y component, a condition regarding a significant coefficient of the U component, and a condition regarding a significant coefficient of the V component are set to the first value when the condition corresponding to the bit is true, and are set to the second value when the condition corresponding to the bit is false; If the condition regarding intra prediction is false and the condition regarding significant coefficients of the Y component is true, a bit among the five bits corresponding to a condition regarding the motion vector and the reference picture is set to the second value; If the condition regarding intra prediction is false and the condition regarding significant coefficients of the Y component is false, a bit corresponding to a condition regarding the motion vector and the reference picture is set to the first value when the condition regarding the motion vector and the reference picture is true, and is set to the second value when the condition regarding the motion vector and the reference picture is false; The image processing device according to claim 12.

15. the five bits correspond to the condition regarding the intra prediction, the condition regarding the significant coefficient of the Y component, the condition regarding the significant coefficient of the U component, the condition regarding the significant coefficient of the V component, and the condition regarding the motion vector and the reference picture, in order from the most significant bit to the least significant bit of the five bits; 15. The image processing device according to claim 12.

16. determining whether or not a deblocking filter needs to be applied to the chrominance components of the decoded image based on a boundary strength expressed by at least 5 bits, the boundary strength being calculated using a chrominance-related parameter that is information about transform coefficients of the chrominance components obtained by orthogonal transform, for a block boundary of the decoded image decoded by a local decoding process; applying a deblocking filter to a chrominance component of a pixel located near the block boundary based on a result of the determination of whether or not the deblocking filter needs to be applied; encoding an image using the decoded image to which the deblocking filter has been applied; Including, the boundary strength is calculated independently for each of the first component and the second component by independently determining whether or not a significant coefficient of a first component and a significant coefficient of a second component are present among components included in the color difference components in blocks sandwiching the block boundary for which the boundary strength is to be calculated; the boundary strength is represented by a plurality of bits including at least one bit corresponding to each of the color difference components; The determining step comprises: determining whether or not the deblocking filter needs to be applied based on a determination based on the boundary strength and a large block determination based on the sizes of blocks sandwiching the block boundary in a direction perpendicular to the block boundary and a direction parallel to the block boundary; Image processing methods.

Images