Deblocking filter device, decoding device, and program

Abstract

To provide a deblocking filter device, a decoding device, and a program for suppressing degradation of an image quality.SOLUTION: In a decoding device 2, a deblocking filter device includes: a deblocking filter 230 that performs a filtering process on a boundary between a first reconstructed block and a second reconstructed block adjacent to the first reconstructed block; and a filter control part 231 that controls a boundary filter strength of the deblocking filter based on whether or not at least one of the first reconstructed block and the second reconstructed block is encoded using a JCCR (Joint coding of chroma residual) that generates one joint prediction residual from a prediction residual of a Cb color difference component and a Cr color difference component.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a deblocking filter device, a decoding device, and a program. [Background technology]

[0002] High Efficiency Video Coding (HEVC) and the next-generation coding standard Versatile Video Coding (VVC) employ a deblocking filter as an in-loop filter to suppress distortion at block boundaries when performing block-by-block coding. The deblocking filter controls its boundary filtering strength depending on whether or not at least one of two adjacent blocks has a nonzero transform coefficient. This is because the inverse transform of the nonzero transform coefficients distributes the energy of the prediction residual across the entire block, potentially creating discontinuities at the boundary between the two blocks. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] JVET-Q2001 “Versatile Video Coding (Draft 8)” Summary of the Invention

[0004] A deblocking filter device according to a first aspect includes a deblocking filter that performs filtering on a boundary between a first reconstructed block and a second reconstructed block adjacent to the first reconstructed block, and a filter control unit that controls a boundary filter strength of the deblocking filter based on whether at least one of the first reconstructed block and the second reconstructed block is coded using JCCR (Joint coding of chroma residual), which generates one joint prediction residual from prediction residuals of a Cb color difference component and a Cr color difference component.

[0005] A decoding device according to a second aspect includes a deblocking filter device according to the first aspect and an entropy decoding unit that acquires a flag for each of the first reconstructed block and the second reconstructed block indicating whether or not the block has been encoded using the JCCR, and the filter control unit controls the boundary filter strength of the deblocking filter based on the flag for each of the first reconstructed block and the second reconstructed block.

[0006] A program according to a third aspect causes a computer to function as the deblocking filter device according to the first aspect. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an encoding device according to an embodiment. [Figure 2] 10A and 10B are diagrams for explaining the operation of a deblocking filter according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating a configuration of a decoding device according to an embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of the operation of the filter control unit according to the embodiment. [Figure 5] FIG. 11 is a diagram illustrating a configuration of an encoding device according to a second modification. [Figure 6] FIG. 11 is a diagram illustrating a configuration of a decoding device according to a second modification. DETAILED DESCRIPTION OF THE INVENTION

[0008] The VVC standard draft adopts a technology called adaptive color transform (ACT), which converts the color space (RGB space) of prediction residuals into YCgCo space when the chroma format of input video is 4:4:4, and performs encoding processes such as conversion and entropy encoding on the color-space-converted prediction residuals (see Non-Patent Document 1). The encoding device can control whether to apply ACT to each block to be encoded, and outputs an ACT application flag for each block to be encoded as a stream. The decoding device performs entropy decoding and inverse conversion processes on the blocks encoded using ACT to restore the prediction residuals, and then inversely converts the color space (YCgCo space) of the restored prediction residuals into RGB space.

[0009] In the conventional technology described above, when at least one of two adjacent blocks has a non-zero transform coefficient, the decoding device applies a deblocking filter to the boundary between the two adjacent blocks. On the other hand, when neither of the two adjacent blocks has a non-zero transform coefficient, the decoding device may not apply a deblocking filter to the boundary between the two adjacent blocks.

[0010] However, for blocks to which ACT has been applied, after the prediction residual is restored from the transform coefficients by inverse transform processing, the color space of the prediction residual is inversely transformed from the YCgCo color space to the RGB color space by color space inverse transform. Therefore, if a block of a certain color component has a non-zero transform coefficient, the non-zero transform coefficient will affect blocks of other color components during color space inverse transform.

[0011] Therefore, if deblocking filter control based on the presence or absence of non-zero transform coefficients is applied to blocks coded using ACT, there is a concern that the boundary filter strength of the deblocking filter may not be controlled appropriately, resulting in degradation of image quality.

[0012] Similar problems can occur when another coding tool, Joint Coding of Chroma Residual (JCCR), is applied.

[0013] Therefore, an object of the present disclosure is to provide a deblocking filter device, a decoding device, and a program that suppress degradation of image quality.

[0014] An encoding device and a decoding device according to an embodiment will be described with reference to the drawings. The encoding device and the decoding device according to the embodiment encode and decode moving images, respectively, as typified by MPEG. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0015] [First embodiment] <Encoder Configuration> First, the configuration of the encoding device according to this embodiment will be described. Fig. 1 is a diagram showing the configuration of an encoding device 1 according to this embodiment.

[0016] As shown in FIG. 1, the encoding device 1 includes a block division unit 100, a residual generation unit 110, a switching unit 111, a color space conversion unit 112, a transform / quantization unit 120, an entropy encoding unit 130, an inverse quantization / inverse transform unit 140, a switching unit 143, a color space inverse transform unit 144, a synthesis unit 150, a deblocking filter 160, a memory 170, and a prediction unit 180.

[0017] The block division unit 100 divides an original image, which is an input image in units of frames (or pictures) constituting a video, into a plurality of image blocks, and outputs the image blocks obtained by division to the residual generation unit 110. The size of the image blocks is, for example, 32x32 pixels, 16x16 pixels, 8x8 pixels, or 4x4 pixels. The shape of the image blocks is not limited to squares and may be rectangular (non-square). An image block is a unit for which the encoding device 1 performs encoding processing (i.e., a block to be encoded) and a unit for which the decoding device performs decoding processing (i.e., a block to be decoded). Such an image block is sometimes called a CU (Coding Unit).

[0018] The input image may be an RGB signal with a chroma format of 4:4:4. The RGB space is an example of a first color space. The "R" component corresponds to the first component, the "G" component corresponds to the second component, and the "B" component corresponds to the third component. The block division unit 100 outputs blocks by dividing the R, G, and B components that make up the image into blocks. In the following description of the encoding device, when there is no need to distinguish between the individual color components, they are simply referred to as the block to be encoded.

[0019] The residual generation unit 110 calculates a prediction residual representing the difference (error) between the current block to be coded output by the block division unit 100 and a predicted block obtained by predicting the current block to be coded by the prediction unit 180. Specifically, the residual generation unit 110 calculates the prediction residual by subtracting each pixel value of the predicted block from each pixel value of the current block to be coded, and outputs the calculated prediction residual to the switching unit 111. In this embodiment, the residual generation unit 110 generates a prediction residual for each color component based on the difference between the current block to be coded for each color component and the predicted block for each color component.

[0020] The switching unit 111 outputs the prediction residual of each color component output by the residual generation unit 110 to either the transform / quantization unit 120 or the color space conversion unit 112. When color space conversion processing (ACT processing) is not performed, the switching unit 111 outputs the prediction residual to the transform / quantization unit 120, and when ACT processing is performed, the switching unit 111 outputs the prediction residual to the color space conversion unit 112.

[0021] The color space conversion unit 112 performs ACT processing on the prediction residual of each color component, and outputs the prediction residual after the ACT processing to the transform / quantization unit 120. The color space conversion unit 112 generates a new prediction residual by performing the following conversion calculation on the R component, G component, and B component of the prediction residual of the current block to be coded.

[0022] Co=RB t=B+(Co>>1) Cg=Gt Y=t+(Cg>>1)

[0023] Here, ">>" represents a right shift operation. The "Y" component corresponds to the first component, the "Cg" component corresponds to the second component, and the "Co" component corresponds to the third component. Such a YCgCo space is an example of a second color space.

[0024] The switching unit 111 and the color space conversion unit 112 can control whether or not to perform color conversion processing for each block to be coded. The entropy coding unit 130 signals, in the bitstream, a flag (ACT application flag) indicating whether or not color conversion processing has been performed on the coding block.

[0025] The ACT processing in the color space conversion unit 112 may generate a prediction residual composed of a new color component by performing addition, subtraction, multiplication, division, shifting, etc. on each color component. The ACT processing does not have to be a conversion that affects all color components. For example, the color space conversion unit 112 may apply an ACT processing in which the first component is left unchanged, the average value of the second and third components is set as the new second component, and the difference between the second and third components is set as the new third component.

[0026] The transform / quantization unit 120 performs transform processing and quantization processing on a block-by-block basis. The transform / quantization unit 120 includes a transform unit 121 and a quantization unit 122.

[0027] The transform unit 121 performs a transform process on the prediction residuals (called prediction residuals regardless of whether ACT processing is applied) output by the switching unit 111 or the color space transform unit 112 to calculate transform coefficients, and outputs the calculated transform coefficients to the quantization unit 122. Specifically, the transform unit 121 generates transform coefficients for each color component by performing a transform process on the prediction residuals of each color component in units of blocks. The transform process may be, for example, a frequency transform such as DCT, DST, or discrete wavelet transform. The transform unit 121 also outputs information related to the transform process to the entropy coding unit 130.

[0028] The transform process includes a transform skip, which is adopted in the HEVC and VVC standard proposals and does not perform a transform process. In the transform skip mode of HEVC, transform coefficients are obtained by scaling prediction residuals without performing horizontal and vertical transform processes. However, the transform skip according to this embodiment also includes a transform that applies only horizontal transform processes or only vertical transform processes. Furthermore, the transform unit 121 may perform a secondary transform process that applies further transform processes to the transform coefficients obtained by the transform process. Furthermore, the secondary transform process may be applied only to a partial region of the transform coefficients.

[0029] The quantization unit 122 quantizes the transform coefficients output from the transform unit 121 using the quantization parameters and the scaling list, and outputs the quantized transform coefficients to the entropy coding unit 130 and the inverse quantization / inverse transform unit 140. The quantization unit 122 also outputs information related to the quantization process (specifically, information on the quantization parameters and the scaling list used in the quantization process) to the entropy coding unit 130 and the inverse quantization unit 141.

[0030] The entropy coding unit 130 performs entropy coding on the quantized transform coefficients output by the quantization unit 122, compresses the data, generates a bitstream (encoded data), and outputs the bitstream to the decoding side. For entropy coding, Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. can be used. In addition, the entropy coding unit 130 includes information related to the conversion process input from the conversion unit 121 in the bitstream and signals it to the decoding side, and also includes information related to the prediction process input from the prediction unit 180 in the bitstream and signals it to the decoding side.

[0031] Furthermore, the entropy coding unit 130 includes a color space conversion flag indicating whether or not ACT is applied in the bitstream for each block to be coded, and signals this to the decoding side. Such a color space conversion flag is also called an ACT application flag. When the ACT application flag is on ("1"), it indicates that ACT is applied to the corresponding block to be coded. When the ACT application flag is off ("0"), it indicates that ACT is not applied to the corresponding block to be coded. Note that an ACT non-application flag may be used instead of the ACT application flag. In that case, when the ACT non-application flag is on ("1"), it indicates that ACT is not applied to the corresponding block to be coded. When the ACT non-application flag is off ("0"), it indicates that ACT is applied to the corresponding block to be coded.

[0032] The inverse quantization and inverse transform unit 140 performs inverse quantization processing and inverse transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142.

[0033] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the transform coefficients by inverse quantizing the quantized transform coefficients output by the quantization unit 122 using a quantization parameter (Qp) and a scaling list, and outputs the restored transform coefficients to the inverse transform unit 142.

[0034] The inverse transform unit 142 performs inverse transform processing corresponding to the transform processing performed by the transform unit 121. For example, if the transform unit 121 performs discrete cosine transform, the inverse transform unit 142 performs inverse discrete cosine transform. The inverse transform unit 142 performs inverse transform processing on the transform coefficients output by the inverse quantization unit 141 to restore prediction residuals, and outputs restored prediction residuals to the switching unit 143.

[0035] The switching unit 143 outputs the restored prediction residual of each color component output by the inverse transform unit 142 to either the synthesis unit 150 or the color space inverse transform unit 144. The switching unit 143 outputs the restored prediction residual for blocks to which ACT has been applied to the synthesis unit 150, and outputs the restored prediction residual for blocks to which ACT has not been applied to the color space inverse transform unit 144.

[0036] The color space inverse conversion unit 144 performs color space inverse conversion processing (inverse ACT processing), which is the inverse processing of the ACT processing performed by the color space conversion unit 112, and outputs the prediction residual after the inverse ACT processing to the synthesis unit 150. Specifically, the inverse conversion from the YCgCo space to the RGB space is performed by performing the following inverse conversion calculation using the Y component, Cg component, and Co component of the restored prediction residual.

[0037] t=Y-(Cg>>1) G=Cg+t B=t-(Co>>1) R=Co+B

[0038] The synthesis unit 150 synthesizes, on a pixel-by-pixel basis, the reconstructed prediction residual output by the inverse transform unit 142 or the color space inverse transform unit 144 with the predicted block output by the prediction unit 180. The synthesis unit 150 adds each pixel value of the reconstructed prediction residual to each pixel value of the predicted block to reconstruct (reconstruct) the block to be coded, and outputs the reconstructed block to the deblocking filter 160. Note that the reconstructed block may also be called a reconstructed block.

[0039] The deblocking filter 160 performs filtering on the reconstructed block output by the synthesis unit 150, and outputs the reconstructed block after filtering to the memory 170. The filter control unit 161 controls the deblocking filter 160. The deblocking filter 160 and the filter control unit 161 will be described in detail later.

[0040] The memory 170 stores the restored blocks after filtering output by the deblocking filter 160 and accumulates the restored blocks as restored images on a frame-by-frame basis. The memory 170 outputs the stored restored blocks or restored images to the prediction unit 180.

[0041] The prediction unit 180 performs prediction processing on a block-by-block basis. The prediction unit 180 generates a prediction block for each color component by performing prediction processing such as intra prediction and inter prediction on the current block to be coded. The prediction unit 180 includes an inter prediction unit 181, an intra prediction unit 182, and a switching unit 183.

[0042] The inter prediction unit 181 performs inter prediction using correlation between frames. Specifically, the inter prediction unit 181 uses the restored image stored in the memory 170 as a reference image to calculate a motion vector by a technique such as block matching, predicts the block to be coded, generates an inter prediction block, and outputs the generated inter prediction block to the switching unit 183. Here, the inter prediction unit 181 selects an optimal inter prediction method from inter prediction using multiple reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 181 outputs information related to the inter prediction (motion vector, etc.) to the entropy coding unit 130.

[0043] The intra prediction unit 182 performs intra prediction using spatial correlation within a frame. Specifically, the intra prediction unit 182 generates an intra prediction block by referencing restored pixels surrounding the current block to be coded among the restored images stored in the memory 170, and outputs the generated intra prediction block to the switching unit 183. The intra prediction unit 182 selects an intra prediction mode to be applied to the current block to be coded from among multiple intra prediction modes, and predicts the current block to be coded using the selected intra prediction mode.

[0044] The switching unit 183 switches between the inter-predicted block output by the inter-prediction unit 181 and the intra-predicted block output by the intra-prediction unit 182, and outputs either the predicted block to the residual generation unit 110 and the synthesis unit 150.

[0045] Next, the deblocking filter 160 and the filter control unit 161 according to this embodiment will be described.

[0046] The deblocking filter 160 performs filtering on the block boundary between two blocks, a restored block (first block) and a restored block (second block) adjacent to the restored block, and outputs each of the filtered restored blocks to the memory 170. The filtering is a process for reducing signal degradation caused by processing on a block-by-block basis, and smooths signal gaps at the block boundary between two adjacent blocks.

[0047] The filter control unit 161 controls the deblocking filter 160. Specifically, the filter control unit 161 controls a boundary filter strength (Bs) indicating whether or not to perform filtering on the block boundaries of a block pair, and the filter strength of the deblocking filter 160. The boundary filter strength Bs refers to a parameter for determining whether or not to apply filtering and the type of filtering. Note that the control of whether or not to perform filtering can be considered as the control of whether or not to set the boundary filter strength Bs to 1 or more or to zero.

[0048] FIG. 2 is a diagram illustrating the operation of the deblocking filter 160 according to this embodiment. In the example shown in FIG. 2, the deblocking filter 160 performs filtering on block boundaries of each 8×8 pixel block. The deblocking filter 160 also performs filtering in units of four rows or four columns. Block P (first reconstructed block) and block Q (second reconstructed block) shown in FIG. 2 are one unit of filtering by the deblocking filter 160, and illustrate an example in which the block size is 4×4 pixels. Each of blocks P and Q may be referred to as a sub-block. Block Q is a reconstructed block corresponding to the current block to be coded, and block P is a reconstructed block adjacent to block Q.

[0049] The filter control unit 161 determines the boundary filter strength Bs based on the following Table 1. In this embodiment, the value of the boundary filter strength Bs is set to 0, 1, or 2.

[0050] [Table 1]

[0051] As shown in FIG. 2 and Table 1, when intra prediction is applied to at least one of blocks P and Q, the filter control unit 161 sets the Bs value to 2.

[0052] The filter control unit 161 sets the Bs value to 1 if motion compensation prediction (inter prediction) is applied to both blocks P and Q and at least one of the following conditions (a) to (d) is met, and sets the Bs value to 0 otherwise.

[0053] (a) The absolute value of the difference between the motion vectors of blocks P and Q is equal to or greater than a threshold value (for example, one pixel).

[0054] (b) The number of motion vectors or reference images of blocks P and Q are different.

[0055] (c) At least one of blocks P and Q contains significant transform coefficients (ie, non-zero transform coefficients).

[0056] (d) ACT is applied to at least one of blocks P and Q.

[0057] The filter control unit 161 controls the deblocking filter 160 so as not to perform deblocking filtering when the value of the boundary filter strength Bs is 0. Hereinafter, an example will be described using the vertical block boundary shown in FIG.

[0058] When the value of the boundary filter strength Bs is 1 or 2, the filter control unit 161 may control the deblocking filter 160 to perform deblocking filtering if the following formula (1) is satisfied.

[0059]

number

[0060] When performing deblocking filtering, the filter control unit 161 may apply a strong filter when all of the following conditional expressions (2) to (7) are satisfied, and may apply a weak filter in other cases.

[0061]

number

[0062] However, the values ​​of the thresholds β and tC change depending on the average value Qav of the quantization parameters of the adjacent blocks P and Q.

[0063] <Configuration of the decoding device> Next, the decoding device according to this embodiment will be described, focusing on the differences from the encoding device 1. Fig. 3 is a diagram showing the configuration of a decoding device 2 according to this embodiment.

[0064] As shown in FIG. 3, the decoding device 2 includes an entropy decoding unit 200, an inverse quantization / inverse transform unit 210, a switching unit 215, a color space inverse transform unit 216, a synthesis unit 220, a deblocking filter 230, a memory 240, and a prediction unit 250.

[0065] The entropy decoding unit 200 decodes the coded data (bitstream), acquires quantized transform coefficients corresponding to the block to be decoded, and outputs the acquired quantized transform coefficients to the inverse quantization and inverse transform unit 210. The entropy decoding unit 200 also acquires information related to the transform process and the quantization process, and outputs the information related to the transform process and the quantization process to the inverse quantization and inverse transform unit 210. The entropy decoding unit 200 also acquires information related to the prediction process, and outputs the information related to the prediction process to the prediction unit 250. The entropy decoding unit 200 acquires a color space conversion flag for each block to be decoded, and outputs the acquired color space conversion flag to the switching unit 215 and the filter control unit 231.

[0066] The inverse quantization and inverse transform unit 210 performs inverse quantization processing and inverse transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212.

[0067] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 inverse quantizes the quantized transform coefficients output by the entropy decoding unit 200 using a quantization parameter (Qp) and a scaling list to restore the transform coefficients of the block to be decoded, and outputs the restored transform coefficients to the inverse transform unit 212.

[0068] The inverse transform unit 212 performs inverse transform processing corresponding to the transform processing performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs inverse transform processing on the transform coefficients output by the inverse quantization unit 211 to restore prediction residuals, and outputs the restored prediction residuals to the switching unit 215.

[0069] Based on the color space conversion flag, the switching unit 215 outputs the prediction residual of each color component output by the inverse conversion unit 212 to either the synthesis unit 220 or the color space inverse conversion unit 216. The switching unit 111 outputs the prediction residual for a block to which color space inverse conversion processing (ACT) has been applied to the transform / quantization unit 120, and outputs the prediction residual for a block to which ACT has been applied to the color space inverse conversion unit 216.

[0070] The color space inverse transform unit 216 performs color space inverse transform processing (inverse ACT processing), which is the inverse processing of the ACT processing performed by the color space transform unit 112 of the encoding device 1, and outputs the prediction residual after the inverse ACT processing to the synthesis unit 220. Specifically, the color space inverse transform unit 216 performs the following inverse transform calculation using the Y component, Cg component, and Co component of the restored prediction residual.

[0071] t=Y-(Cg>>1) G=Cg+t B=t-(Co>>1) R=Co+B

[0072] The synthesis unit 220 decodes (reconstructs) the original block by synthesizing the prediction residual output by the switching unit 215 or the color space inverse conversion unit 216 and the prediction block output by the prediction unit 250 on a pixel-by-pixel basis, and outputs the restored block to the deblocking filter 230.

[0073] The deblocking filter 230 performs filtering on the reconstructed blocks output by the synthesis unit 220, and outputs the filtered reconstructed blocks to the memory 240. Specifically, the deblocking filter 230 performs filtering on the block boundary between two blocks consisting of a reconstructed block (first block) and a reconstructed block (second block) adjacent to the reconstructed block, and outputs each of the filtered reconstructed blocks to the memory 240. The function of the deblocking filter 230 is similar to that of the deblocking filter 160 of the encoding device 1.

[0074] The filter control unit 231 controls the deblocking filter 230. The filter control unit 231 controls the deblocking filter 230. Specifically, the filter control unit 231 controls a boundary filter strength (Bs) indicating whether or not to perform filtering on a block boundary of a block pair, and the filter strength of the deblocking filter 230. The function of the filter control unit 231 is similar to the function of the filter control unit 161 of the encoding device 1. The function of the filter control unit 231 is to determine the boundary filter strength Bs based on Table 1 above.

[0075] That is, the filter control unit 231 according to this embodiment controls the boundary filter strength Bs of the deblocking filter 230 based on whether or not at least one of the adjacent blocks P and Q is coded using adaptive color transformation (ACT).

[0076] As described above, the entropy decoding unit 200 acquires a flag (color space conversion flag) indicating whether or not the blocks have been coded using adaptive color conversion for each of the blocks P and Q. The filter control unit 231 controls the boundary filter strength Bs of the deblocking filter 230 based on the color space conversion flag for each of the blocks P and Q.

[0077] When at least one of the blocks P and Q is coded using adaptive color transform, the filter control unit 231 controls the boundary filter strength Bs so that filtering is performed by the deblocking filter 230 (i.e., sets the boundary filter strength Bs to 1). Specifically, even if neither the block P nor the block Q has a non-zero transform coefficient, the filter control unit 231 controls the boundary filter strength Bs so that filtering is performed by the deblocking filter 230 when at least one of the blocks P and Q is coded using adaptive color transform.

[0078] The memory 240 stores the restored blocks output by the synthesis unit 220 and accumulates the restored blocks as restored images on a frame-by-frame basis. The memory 240 outputs the restored blocks or the restored images to the prediction unit 250. The memory 240 also outputs the restored images on a frame-by-frame basis to the outside of the decoding device 2.

[0079] The prediction unit 250 performs prediction for each color component in units of blocks. The prediction unit 250 includes an inter prediction unit 251, an intra prediction unit 252, and a switching unit 253.

[0080] The inter prediction unit 251 performs inter prediction using correlation between frames. Specifically, the inter prediction unit 251 predicts the block to be coded using the restored image stored in the memory 240 as a reference image based on information related to inter prediction (for example, motion vector information) output by the entropy decoding unit 200, thereby generating an inter prediction block.

[0081] The intra prediction unit 252 performs intra prediction using spatial correlation within a frame. Specifically, the intra prediction unit 252 generates an intra prediction block by referring to restored pixels around the block to be coded in the restored image stored in the memory 240, using an intra prediction mode according to information related to intra prediction (for example, intra prediction mode information) output by the entropy decoding unit 200, and outputs the generated intra prediction block to the switching unit 253.

[0082] The switching unit 253 switches between the inter-predicted block output by the inter-prediction unit 251 and the intra-predicted block output by the intra-prediction unit 252, and outputs either the predicted block to the synthesis unit 220.

[0083] As described above, the decoding device 2 according to this embodiment includes an entropy decoding unit 200 that decodes an encoded stream and outputs transform coefficients corresponding to a block P; an inverse quantization and inverse transform unit 210 that performs inverse quantization and inverse transform processing on the transform coefficients output by the entropy decoding unit 200 to reconstruct a prediction residual corresponding to the block P (first block); a synthesis unit 220 that reconstructs the block P by synthesizing the reconstructed prediction residual with a prediction block obtained by predicting the block P; a deblocking filter 230 that performs filtering on the boundary between the reconstructed block P and a reconstructed block Q (second block) adjacent to the block P; and a filter control unit 231 that controls a boundary filter strength Bs of the deblocking filter 230 based on whether or not at least one of the block P and the block Q has been coded using adaptive color transformation (ACT).

[0084] For blocks to which ACT has been applied, prediction residuals are restored from transform coefficients by inverse transform processing, and then the color space of the prediction residuals is inversely transformed from YCgCo space to RGB space by color space inverse transform (inverse ACT). Therefore, if a block of a certain color component has a non-zero transform coefficient, that non-zero transform coefficient will affect blocks of other color components during color space inverse transform.

[0085] For this reason, the filter control unit 231 according to this embodiment not only controls the boundary filter strength Bs of the deblocking filter 230 based on the presence or absence of non-zero transform coefficients, but also controls the boundary filter strength Bs of the deblocking filter 230 taking into account whether ACT is applied. This makes it possible to appropriately control the boundary filter strength Bs of the deblocking filter 230, thereby suppressing degradation of image quality even when ACT is applied.

[0086] <Filter control section operation> Next, the operations of the filter control unit 161 and the filter control unit 231 according to this embodiment will be described. Since the filter control unit 161 on the encoding side and the filter control unit 231 on the decoding side perform the same operation, the filter control unit 231 on the decoding side will be described here as an example. Fig. 4 is a diagram showing an example of the operation of the filter control unit 231 according to this embodiment. Note that the determination order shown in Fig. 4 is an example, and the determination order may be changed.

[0087] 4, in step S1, the filter control unit 231 determines whether intra prediction has been applied to at least one of a pair of blocks consisting of blocks P and Q. If intra prediction has been applied to at least one of the pair of blocks (step S1: YES), in step S2, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs=2.

[0088] If intra prediction is not applied to either of the block pair (step S1: NO), in step S3, the filter control unit 231 determines whether the difference between the motion vectors of the current block pair is equal to or greater than a threshold. If the difference between the motion vectors of the current block pair is equal to or greater than the threshold (step S3: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs=1.

[0089] If the difference between the motion vectors of the block pair is not equal to or greater than the threshold (step S3: NO), in step S5, the filter control unit 231 determines whether the number of motion vectors of the block pair or the reference images are different. If the number of motion vectors of the block pair or the reference images are different (step S5: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs=1.

[0090] If the number of motion vectors or reference images of the block pair are the same (step S5: NO), in step S6, the filter control unit 231 determines whether or not at least one block of the block pair includes a non-zero transform coefficient. If at least one block of the block pair includes a non-zero transform coefficient (step S6: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs=1.

[0091] If none of the blocks in the block pair contains a non-zero transform coefficient (step S6: NO), in step S7, the filter control unit 231 determines whether at least one of the blocks in the block pair has been coded using ACT, based on the color space conversion flag of each block output by the entropy decoding unit 200. If at least one of the blocks in the block pair has been coded using ACT (step S7: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs=1. On the other hand, if none of the blocks in the block pair has been coded using ACT (step S7: NO), in step S8, the filter control unit 231 controls the deblocking filter 230 not to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs=0.

[0092] <Change example 1> The filter control unit 231 on the decoding side may determine whether at least one block of the block pair (block P and block Q) includes a non-zero transform coefficient based on a flag signaled from the encoding device 1.

[0093] Specifically, the entropy coding unit 130 of the coding device 1 includes in the coded stream for each block a flag (tu_coded_flag) indicating whether or not the block contains a non-zero transform coefficient. For example, the entropy coding unit 130 sets the flag (tu_coded_flag) to “1” for a block that contains a non-zero transform coefficient, and sets the flag (tu_coded_flag) to “0” for a block that does not contain a non-zero transform coefficient.

[0094] The entropy decoding unit 200 of the decoding device 2 acquires a flag (tu_coded_flag) for each block and outputs the acquired flag (tu_coded_flag) to the filter control unit 231. The filter control unit 231 interprets a block for which the flag (tu_coded_flag) is "1" as not containing a non-zero transform coefficient. Then, the filter control unit 231 sets the boundary filter strength Bs of the deblocking filter 230 as shown in Table 2 below.

[0095] [Table 2]

[0096] As shown in Table 2, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering when tu_coded_flag of at least one block of a block pair is "1." Specifically, the filter control unit 231 sets the boundary filter strength Bs=1.

[0097] <Change example 2> As described above, for a block to which ACT has been applied, the prediction residual is restored from the transform coefficients by inverse transform processing, and then the color space of the prediction residual is inversely transformed from the YCgCo color space to the RGB color space by inverse ACT. Therefore, if a block of a certain color component has a non-zero transform coefficient, the non-zero transform coefficient will affect blocks of other color components during inverse ACT.

[0098] For this reason, when ACT is applied, control of the boundary filter strength Bs of the deblocking filter 230 based on the presence or absence of non-zero transform coefficients does not function appropriately. For this reason, in this modified example, the boundary filter strength Bs of the deblocking filter 230 is controlled based on the prediction residual after inverse transform processing, rather than the transform coefficients. As a result, even if non-zero transform coefficients affect blocks of other color components during inverse ACT, the boundary filter strength Bs of the deblocking filter 230 can be appropriately controlled based on the prediction residual after the effect.

[0099] That is, the filter control unit 161 and the filter control unit 231 according to this modified example set the boundary filter strength Bs of the deblocking filter 230 based on Table 3 below.

[0100] [Table 3]

[0101] FIG. 5 is a diagram showing the configuration of the encoding device 1 according to this modified example.

[0102] As shown in Fig. 5, in the encoding device 1, the filter control unit 161 receives as input a prediction residual that is the same as the prediction residual (reconstructed prediction residual) input to the synthesis unit 150. For a block to which ACT is applied, the prediction residual after inverse ACT is input to the filter control unit 161. The filter control unit 161 controls the deblocking filter 160 to perform deblocking filtering when the reconstructed prediction residual of at least one block of a block pair (block P and block Q) includes a non-zero value. Specifically, the filter control unit 161 sets the boundary filter strength Bs=1.

[0103] FIG. 6 is a diagram showing the configuration of a decoding device 2 according to this modified example.

[0104] As shown in Fig. 6, in the decoding device 2, the filter control unit 231 receives as input a prediction residual that is the same as the prediction residual (reconstructed prediction residual) input to the synthesis unit 220. For a block to which ACT is applied, the prediction residual after inverse ACT is input to the filter control unit 231. The filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering when the reconstructed prediction residual of at least one block of a block pair (block P and block Q) includes a non-zero value. Specifically, the filter control unit 231 sets the boundary filter strength Bs=1.

[0105] [Second embodiment] In the above-described embodiment and its modified examples, an example in which ACT, which is one of the coding tools, is applied has been described. However, when another coding tool, Joint Coding of Chroma Residual (JCCR), is applied, the same problem as described above may occur. For this reason, the above-described embodiment and its modified examples may be applied to JCCR, and ACT in the above-described embodiment and its modified examples may be appropriately read as JCCR. For example, the boundary filter strength Bs may be controlled as shown in Table 4 below.

[0106] [Table 4]

[0107] JCCR is a coefficient coding mode for chrominance components (see Non-Patent Document 1). In JCCR, the encoding device 1 generates one joint prediction residual from the prediction residual of the first chrominance component (Cb component) and the prediction residual of the second chrominance component (Cr component) by utilizing the correlation between the prediction residuals of the first chrominance component (Cb component) and the second chrominance component (Cr component). For example, the encoding device 1 generates a joint prediction residual by combining the prediction residual of the second chrominance component, whose positive and negative polarities are inverted, with the prediction residual of the first chrominance component. The encoding device 1 then performs transformation, quantization, and entropy coding on the generated joint prediction residual before transmitting it.

[0108] The decoding device 2 reconstructs the prediction residual of the first chrominance component and the prediction residual of the second chrominance component from the transmitted joint prediction residual. In this way, by transmitting only one joint prediction residual for two chrominance components, coding efficiency is improved.

[0109] As described above, the deblocking filter device according to this embodiment includes a deblocking filter (160, 230) that performs a filtering process on the boundary between a first reconstructed block (block P) and a second reconstructed block (block Q), and a filter control unit (161, 231) that controls a boundary filter strength Bs of the deblocking filter (160, 230) based on whether or not at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) has been coded using JCCR (Joint coding of chroma residual), which generates one joint prediction residual from the respective prediction residuals of the Cb color difference component and the Cr color difference component.

[0110] The filter control unit (161, 231) may control the boundary filter strength Bs so that filtering is performed by the deblocking filter (160, 230) when at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) is coded using JCCR.

[0111] The filter control unit (161, 231) may control the boundary filter strength Bs so as to perform filtering by the deblocking filter (160, 230) when at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) is coded using JCCR, even if there are no non-zero transform coefficients in both the first reconstructed block (block P) and the second reconstructed block (block Q).

[0112] In this embodiment, the entropy decoding unit 200 of the decoding device 2 may acquire a flag indicating whether or not the first reconstructed block (block P) and the second reconstructed block (block Q) have been coded using JCCR. The filter control unit 231 may control the boundary filter strength Bs of the deblocking filter 230 based on the flag for each of the first reconstructed block (block P) and the second reconstructed block (block Q). For example, when the flag for at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) is "1", the filter control unit 231 may control the boundary filter strength Bs so that the deblocking filter 230 performs filtering.

[0113] In this embodiment, the entropy decoding unit 200 of the decoding device 2 may further acquire, for each of the first reconstructed block (block P) and the second reconstructed block (block Q), tu_coded_flag indicating whether or not the block contains a non-zero transform coefficient. The filter control unit 231 may control the boundary filter strength Bs of the deblocking filter 230 further based on the tu_coded_flag for each of the first reconstructed block (block P) and the second reconstructed block (block Q). For example, when the tu_coded_flag for at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) is "1," the filter control unit 231 may control the boundary filter strength Bs so that the deblocking filter 230 performs filtering.

[0114] For each block of the first chrominance component (Cb component) and the second chrominance component (Cr component) to which JCCR is applied, the above-mentioned tu_coded_flag is set to "1," but in reality, no transform coefficient exists for one of the color components. Therefore, as in the above-mentioned Modification Example 2, it is preferable to determine whether or not a reconstructed prediction residual contains a non-zero value for a certain color component, instead of determining whether or not a non-zero transform coefficient exists.

[0115] [Other embodiments] In the above-described embodiment, an example of controlling the boundary filter strength Bs has been described as the control of the deblocking filters (160, 230). However, the control is not limited to the boundary filter strength Bs, and the filter length or switching between multiple filters may also be controlled.

[0116] A program may be provided that causes a computer to execute each process performed by the encoding device 1 described above. Also, a program may be provided that causes a computer to execute each process performed by the decoding device 2. The program may be recorded on a computer-readable medium. Using a computer-readable medium makes it possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0117] The circuits that execute the processes performed by the encoding device 1 may be integrated, and the encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC). The circuits that execute the processes performed by the decoding device 2 may be integrated, and the decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC).

[0118] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope that does not deviate from the gist of the invention.

[0119] This application claims priority from Japanese Patent Application No. 2020-101293 (filed June 10, 2020), the entire contents of which are incorporated herein by reference.

Claims

1. a deblocking filter that performs filtering on a boundary between a first reconstructed block and a second reconstructed block adjacent to the first reconstructed block; a filter control unit that controls a boundary filter strength of the deblocking filter based on whether or not at least one of the first reconstruction block and the second reconstruction block applies color conversion processing; the color conversion process is a process of converting a prediction residual in a first color space into a prediction residual in a second color space different from the first color space, the filter control unit controls the boundary filter strength to perform the filtering process when at least one of the first reconstructed block and the second reconstructed block is coded using the color conversion process; The filter control unit controls the boundary filter strength so as to perform the filtering process on a predetermined color component when a prediction residual obtained by the color conversion process includes a non-zero value for the predetermined color component. Deblocking filter device.

2. The filter control unit controls the boundary filter strength so as to perform the filtering process when at least one of the first reconstructed block and the second reconstructed block is coded using the color conversion process, even if there are no non-zero transform coefficients in both the first reconstructed block and the second reconstructed block. The deblocking filter device according to claim 1 .

3. A deblocking filter device according to claim 1 or 2; an entropy decoding unit that acquires, for each of the first reconstructed block and the second reconstructed block, a flag indicating whether or not the first reconstructed block and the second reconstructed block have been coded using the color conversion process; The filter control unit controls a boundary filter strength of the deblocking filter based on the flag for each of the first reconstructed block and the second reconstructed block. Decryption device.

4. The filter control unit controls the boundary filter strength so as to perform the filtering process when the flag of at least one of the first reconstructed block and the second reconstructed block is "1." The decoding device according to claim 3 .

5. The entropy decoding unit further obtains, for each of the first reconstructed block and the second reconstructed block, a tu_coded_flag indicating whether the first reconstructed block and the second reconstructed block include a non-zero transform coefficient; The filter control unit controls a boundary filter strength of the deblocking filter further based on the tu_coded_flag for each of the first reconstructed block and the second reconstructed block.

5. The decoding device according to claim 3 or 4.

6. The filter control unit controls the boundary filter strength so as to perform the filtering process when the tu_coded_flag of at least one of the first reconstructed block and the second reconstructed block is "1." The decoding device according to claim 5 .

7. A program that causes a computer to function as the deblocking filter device according to claim 1 or 2.

Images