Deblocking Filter Device, Decoding Device, and Program

Abstract

To provide a deblocking filter device, a decoding device, and a program that suppress deterioration of image quality.SOLUTION: In a decoding device 2, a deblocking filter performs filter processing on the boundary between a first reconstructed block and a second reconstructed block adjacent to the first reconstructed block, and controls the border filter strength of the deblocking filter on the basis of whether at least one of the first reconstructed block and the second reconstructed block is encoded using JCCR (Joint coding of chroma residual) to generate one joint prediction residual from the prediction residuals of each of the Cb chroma component and the Cr chroma component.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a deblocking filter device, a decoding device, and a program.

Background Art

[0002] In HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding), which is the next-generation coding method, as an in-loop filter, a deblocking filter for suppressing distortion at the boundary of a block when performing encoding processing in block units is adopted. In the control of the deblocking filter, the boundary filtering strength of the deblocking filter is controlled according to whether or not there is a non-zero transform coefficient in at least one of two adjacent blocks. This is because the energy of the prediction residual is distributed over the entire block by the inverse transform of the non-zero transform coefficient, so there is a high possibility of discontinuity at the boundary between the two blocks.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

[0004] The deblocking filter device according to the first aspect includes a deblocking filter that performs a filtering process on a boundary between a first reconstruction block and a second reconstruction block adjacent to the first reconstruction block, and at least one of the first reconstruction block and the second reconstruction block is encoded using JCCR (Joint coding of chroma residual) that generates one joint prediction residual from the prediction residuals of each of the Cb color difference component and the Cr color difference component. A filter control unit that controls the boundary filter strength of the deblocking filter based on whether or not it is present.

[0005] The decoding device according to the second aspect includes the deblocking filter device according to the first aspect, and an entropy decoding unit that acquires, for each of the first reconstruction block and the second reconstruction block, a flag indicating whether or not it is encoded using the JCCR. The filter control unit controls the boundary filter strength of the deblocking filter based on the flag for each of the first reconstruction block and the second reconstruction block.

[0006] The program according to the third aspect causes a computer to function as the deblocking filter device according to the first aspect.

Brief Description of Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0008] In the VVC standard proposal, when the chroma format of the input video is 4:4:4, a technology called Adaptive Colour Transform (ACT) is adopted, which converts the colour space (RGB space) of the prediction residual to the YCgCo space and performs encoding processes such as conversion processing and entropy encoding processing on the prediction residual after the colour space conversion (see Non-Patent Document 1). The encoding device can control whether to apply ACT for each block to be encoded, and outputs an ACT application flag to the stream for each block to be encoded. The decoding device performs entropy decoding, inverse conversion processing, etc. on the block encoded using ACT to restore the prediction residual, and inversely converts the colour space (YCgCo space) of the restored prediction residual to the RGB space.

[0009] In the above-described conventional technology, the decoding device applies a deblocking filter to the boundary of two adjacent blocks when at least one of the two adjacent blocks has non-zero conversion coefficients. On the other hand, when non-zero conversion coefficients do not exist in both of the two adjacent blocks, it is also possible not to apply a deblocking filter to the boundary of the two blocks.

[0010] However, for a block to which ACT is applied, after the prediction residual is restored from the conversion coefficients by inverse conversion processing, the colour space of the prediction residual is inversely converted from the YCgCo space to the RGB space by colour space inverse conversion. Therefore, when a non-zero conversion coefficient exists in a block of a certain colour component, the non-zero conversion coefficient will affect the blocks of other colour components during colour space inverse conversion.

[0011] Therefore, when applying deblocking filter control based on the presence or absence of non-zero conversion coefficients to a block encoded using ACT, there is a concern that the boundary filter strength of the deblocking filter cannot be appropriately controlled, causing image quality degradation.

[0012] Even when applying Joint coding of chroma residual (JCCR), which is another encoding tool, the same problems as described above may occur.

[0013] Therefore, an object of the present disclosure is to provide a deblocking filter device, a decoding device, and a program that suppress degradation of image quality.

[0014] With reference to the drawings, an encoding device and a decoding device according to an embodiment will be described. The encoding device and the decoding device according to the embodiment respectively perform encoding and decoding of moving images represented by MPEG. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0015] [First Embodiment] <Configuration of Encoding Device> First, the configuration of the encoding device according to the present embodiment will be described. FIG. 1 is a diagram showing the configuration of an encoding device 1 according to the present embodiment.

[0016] As shown in FIG. 1, the encoding device 1 includes a block division unit 100, a residual generation unit 110, a switching unit 111, a color space conversion unit 112, a conversion / quantization unit 120, an entropy encoding unit 130, an inverse quantization / inverse conversion unit 140, a switching unit 143, a color space inverse conversion unit 144, a synthesis unit 150, a deblocking filter 160, a memory 170, and a prediction unit 180.

[0017] The block division unit 100 divides the original image, which is an input image in units of frames (or pictures) constituting the moving image, into a plurality of image blocks, and outputs the image blocks obtained by the division to the residual generation unit 110. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels, etc. The shape of the image block is not limited to a square and may be a rectangle (non-square). The image block is a unit for which the encoding device 1 performs encoding processing (i.e., the encoding target block), and is also a unit for which the decoding device performs decoding processing (i.e., the decoding target block). Such an image block is sometimes called a CU (Coding Unit).

[0018] The input image is an RGB signal, and the chroma format may be 4:4:4. The RGB space is an example of the first color space. The "R" component corresponds to the first component, the "G" component corresponds to the second component, and the "B" component corresponds to the third component. The block division unit 100 outputs blocks by performing block division on each of the R component, G component, and B component constituting the image. In the following description of the encoding device, when the color components are not distinguished, it is simply called the encoding target block.

[0019] The residual generation unit 110 calculates a prediction residual representing the difference (error) between the encoding target block output by the block division unit 100 and the prediction block obtained by predicting the encoding target block by the prediction unit 180. Specifically, the residual generation unit 110 calculates the prediction residual by subtracting the pixel values of the prediction block from the pixel values of the encoding target block, and outputs the calculated prediction residual to the switching unit 111. In the present embodiment, the residual generation unit 110 generates the prediction residual of each color component based on the difference between the encoding target block of each color component and the prediction block of each color component.

[0020] The switching unit 111 outputs the prediction residuals of each color component output by the residual generation unit 110 to either the conversion / quantization unit 120 or the color space conversion unit 112. When the color space conversion process (ACT process) is not performed, the switching unit 111 outputs the prediction residuals to the conversion / quantization unit 120, and when the ACT process is performed, the switching unit 111 outputs the prediction residuals to the color space conversion unit 112.

[0021] The color space conversion unit 112 performs the ACT process on the prediction residuals of each color component, and outputs the prediction residuals after the ACT process to the conversion / quantization unit 120. The color space conversion unit 112 generates new prediction residuals by performing the following conversion calculations on the R component, G component, and B component of the prediction residuals of the block to be encoded.

[0022] Co = R - B t = B + (Co >> 1) Cg = G - t Y = t + (Cg >> 1)

[0023] Here, ">>" represents a right shift operation. Also, the "Y" component corresponds to the first component, the "Cg" component corresponds to the second component, and the "Co" component corresponds to the third component. Such a YCgCo space is an example of the second color space.

[0024] The switching unit 111 and the color space conversion unit 112 can control whether to perform color conversion processing for each block to be encoded. The entropy encoding unit 130 signals in the bit stream a flag (ACT application flag) indicating whether color conversion processing has been performed on the encoded block.

[0025] Note that the ACT process in the color space conversion unit 112 may generate prediction residuals composed of new color components by addition, subtraction, multiplication, division, shift processing, etc. for each color component. Also, the ACT process does not necessarily need to be a conversion that affects all color components. For example, the color space conversion unit 112 may apply an ACT process that maintains the first component without change, sets the average value of the second and third components as the new second component, and sets the difference between the second and third components as the new third component.

[0026] The conversion / quantization unit 120 performs conversion processing and quantization processing in block units. The conversion / quantization unit 120 includes a conversion unit 121 and a quantization unit 122.

[0027] The conversion unit 121 performs conversion processing on the prediction residual (referred to as the prediction residual regardless of whether the ACT process is applied) output by the switching unit 111 or the color space conversion unit 112 to calculate conversion coefficients, and outputs the calculated conversion coefficients to the quantization unit 122. Specifically, the conversion unit 121 generates conversion coefficients for each color component by performing conversion processing on the prediction residual of each color component in block units. The conversion processing may be, for example, frequency conversion such as DCT, DST, or discrete wavelet transform. Also, the conversion unit 121 outputs information regarding the conversion processing to the entropy encoding unit 130.

[0028] The conversion processing includes conversion skip that does not perform conversion processing and is adopted in the HEVC and VVC standard drafts. In the conversion skip mode of HEVC, scaling is applied to the prediction residual without performing horizontal and vertical conversion processing to obtain the conversion coefficients. However, the conversion skip according to this embodiment includes conversions that apply only horizontal conversion processing and conversions that apply only vertical conversion processing. Also, the conversion unit 121 may perform secondary conversion processing in which further conversion processing is applied to the conversion coefficients obtained by the conversion processing. Also, the secondary conversion processing may be applied only to a partial region of the conversion coefficients.

[0029] The quantization unit 122 quantizes the conversion coefficients output from the conversion unit 121 using a quantization parameter and a scaling list, and outputs the quantized conversion coefficients to the entropy encoding unit 130 and the inverse quantization / inverse conversion unit 140. Also, the quantization unit 122 outputs information regarding the quantization processing (specifically, information on the quantization parameter and the scaling list used in the quantization processing) to the entropy encoding unit 130 and the inverse quantization unit 141.

[0030] The entropy encoding unit 130 performs entropy encoding on the quantized transform coefficients output by the quantization unit 122, compresses the data to generate a bitstream (encoded data), and outputs the bitstream to the decoding side. For entropy encoding, Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), or the like can be used. Further, the entropy encoding unit 130 includes information related to the conversion process input from the conversion unit 121 in the bitstream and signals it to the decoding side, or includes information related to the prediction process input from the prediction unit 180 in the bitstream and signals it to the decoding side.

[0031] Furthermore, the entropy encoding unit 130 includes, in the bitstream for each coding target block, a color space conversion flag indicating whether or not ACT is applied, and signals it to the decoding side. Such a color space conversion flag is also referred to as an ACT application flag. When the ACT application flag is on (“1”), it indicates that ACT is applied to the corresponding coding target block. When the ACT application flag is off (“0”), it indicates that ACT is not applied to the corresponding coding target block. Note that an ACT non-application flag may be used instead of the ACT application flag. In that case, when the ACT non-application flag is on (“1”), it indicates that ACT is not applied to the corresponding coding target block. When the ACT non-application flag is off (“0”), it indicates that ACT is applied to the corresponding coding target block.

[0032] The inverse quantization / inverse transformation unit 140 performs inverse quantization processing and inverse transformation processing in block units. The inverse quantization / inverse transformation unit 140 includes an inverse quantization unit 141 and an inverse transformation unit 142.

[0033] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the transform coefficients by inverse quantizing the quantized transform coefficients output by the quantization unit 122 using the quantization parameter (Qp) and the scaling list, and outputs the restored transform coefficients to the inverse transformation unit 142.

[0034] The inverse conversion unit 142 performs an inverse conversion process corresponding to the conversion process performed by the conversion unit 121. For example, when the conversion unit 121 performs a discrete cosine transform, the inverse conversion unit 142 performs an inverse discrete cosine transform. The inverse conversion unit 142 performs an inverse conversion process on the conversion coefficients output by the inverse quantization unit 141 to restore the prediction residual, and outputs the restored prediction residual, which is the restored prediction residual, to the switching unit 143.

[0035] The switching unit 143 outputs the restored prediction residual of each color component output by the inverse conversion unit 142 to either the synthesis unit 150 or the color space inverse conversion unit 144. The switching unit 143 outputs the restored prediction residual to the synthesis unit 150 for the block to which ACT is applied, and outputs the restored prediction residual to the color space inverse conversion unit 144 for the block to which ACT is not applied.

[0036] The color space inverse conversion unit 144 performs a color space inverse conversion process (inverse ACT process), which is the inverse process of the ACT process performed by the color space conversion unit 112, and outputs the prediction residual after the inverse ACT process to the synthesis unit 150. Specifically, an inverse conversion from the YCgCo space to the RGB space is performed by performing the following inverse conversion calculation using the Y component, Cg component, and Co component of the restored prediction residual.

[0037] t = Y - (Cg >> 1) G = Cg + t B = t - (Co >> 1) R = Co + B

[0038] The synthesis unit 150 synthesizes the restored prediction residual output by the inverse conversion unit 142 or the color space inverse conversion unit 144 with the prediction block output by the prediction unit 180 on a pixel-by-pixel basis. The synthesis unit 150 adds the pixel values of the restored prediction residual and the pixel values of the prediction block to restore (reconstruct) the block to be encoded, and outputs the restored block to the deblocking filter 160. Note that the restored block may also be referred to as a reconstructed block.

[0039] The deblocking filter 160 performs filter processing on the restored block output by the synthesis unit 150, and outputs the restored block after the filter processing to the memory 170. The filter control unit 161 controls the deblocking filter 160. Details of the deblocking filter 160 and the filter control unit 161 will be described later.

[0040] The memory 170 stores the restored block after the filter processing output by the deblocking filter 160, and accumulates the restored blocks as a restored image in units of frames. The memory 170 outputs the stored restored block or restored image to the prediction unit 180.

[0041] The prediction unit 180 performs prediction processing in units of blocks. The prediction unit 180 generates a prediction block for each color component by performing prediction processing such as intra prediction and inter prediction on the block to be encoded. The prediction unit 180 includes an inter prediction unit 181, an intra prediction unit 182, and a switching unit 183.

[0042] The inter prediction unit 181 performs inter prediction using the correlation between frames. Specifically, the inter prediction unit 181 uses the restored image stored in the memory 170 as a reference image, calculates a motion vector by a method such as block matching, predicts the block to be encoded, generates an inter prediction block, and outputs the generated inter prediction block to the switching unit 183. Here, the inter prediction unit 181 selects the optimal inter prediction method from among inter predictions using a plurality of reference images (typically, bi-prediction) and inter predictions using one reference image (uni-directional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 181 outputs information related to inter prediction (such as motion vectors) to the entropy encoding unit 130.

[0043] The intra prediction unit 182 performs intra prediction using the spatial correlation within a frame. Specifically, the intra prediction unit 182 generates an intra prediction block by referring to the restored pixels around the block to be encoded among the restored images stored in the memory 170, and outputs the generated intra prediction block to the switching unit 183. The intra prediction unit 182 selects an intra prediction mode to be applied to the block to be encoded from among a plurality of intra prediction modes, and predicts the block to be encoded using the selected intra prediction mode.

[0044] The switching unit 183 switches between the inter prediction block output by the inter prediction unit 181 and the intra prediction block output by the intra prediction unit 182, and outputs one of the prediction blocks to the residual generation unit 110 and the synthesis unit 150.

[0045] Next, the deblocking filter 160 and the filter control unit 161 according to the present embodiment will be described.

[0046] The deblocking filter 160 performs filter processing on the block boundary between two blocks composed of a restored block (first block) and a restored block adjacent to the restored block (second block), and outputs each restored block after the filter processing to the memory 170. The filter processing is a process for reducing signal degradation caused by block-based processing, and is a filter processing for smoothing the signal gap at the block boundary between two adjacent blocks.

[0047] The filter control unit 161 controls the deblocking filter 160. Specifically, the filter control unit 161 controls the boundary filter strength (Bs) indicating whether to perform filter processing on the block boundary of a block pair, and the filter strength of the deblocking filter 160. The boundary filter strength Bs refers to a parameter for determining whether to apply filter processing and the type of the filter processing. Note that the control of whether to perform filter processing can be regarded as the control of setting the boundary filter strength Bs to 1 or more or to zero.

[0048] FIG. 2 is a diagram for explaining the operation of the deblocking filter 160 according to the present embodiment. In the example shown in FIG. 2, the deblocking filter 160 performs filter processing on the block boundary for each 8×8 pixel block. Also, the deblocking filter 160 performs filter processing in units of 4 rows or 4 columns. In the blocks P (first reconstructed block) and Q (second reconstructed block) shown in FIG. 2, an example in which one unit of the filter processing of the deblocking filter 160 and the block size is 4×4 pixels is shown. Each of the blocks P and Q may be called a sub-block. The block Q is a restored block corresponding to the block to be encoded, and the block P is a restored block adjacent to the block Q.

[0049] The filter control unit 161 determines the boundary filter strength Bs based on Table 1 below. In the present embodiment, the value of the boundary filter strength Bs is either 0, 1, or 2.

[0050]

Table 1

[0051] As shown in FIG. 2 and Table 1, when intra prediction is applied to at least one of the blocks P and Q, the filter control unit 161 sets the Bs value to 2.

[0052] When motion compensation prediction (inter prediction) is applied to both blocks P and Q and at least one of the following conditions (a) to (d) is satisfied, the filter control unit 161 sets the Bs value to 1; otherwise, it sets the Bs value to 0.

[0053] (a) The absolute value of the difference between the motion vectors of blocks P and Q is greater than or equal to a threshold value (e.g., 1 pixel).

[0054] (b) The number of motion vectors or the reference images of blocks P and Q are different.

[0055] (c) At least one of blocks P and Q includes significant transform coefficients (i.e., non-zero transform coefficients).

[0056] (d) ACT is applied to at least one of blocks P and Q.

[0057] When the value of the boundary filter strength Bs is 0, the filter control unit 161 controls the deblocking filter 160 so as not to perform deblocking filter processing. Hereinafter, the vertical block boundary shown in FIG. 2 will be described as an example.

[0058] When the value of the boundary filter strength Bs is 1 or 2, the filter control unit 161 may control the deblocking filter 160 to perform deblocking filter processing when the following equation (1) is satisfied.

[0059]

Equation

[0060] Note that when performing deblocking filter processing, the filter control unit 161 may apply a strong filter when all of the following conditional expressions (2) to (7) are satisfied, and apply a weak filter otherwise.

[0061]

Equation

[0062] However, the values of the threshold β and tC change according to the average quantization parameter Qav of the adjacent blocks P and Q.

[0063] <Configuration of the decoding device> Next, the differences between the decoding device according to the present embodiment and the encoding device 1 will be mainly described. FIG. 3 is a diagram showing the configuration of the decoding device 2 according to the present embodiment.

[0064] As shown in FIG. 3, the decoding device 2 includes an entropy decoding unit 200, an inverse quantization / inverse transformation unit 210, a switching unit 215, a color space inverse transformation unit 216, a synthesis unit 220, a deblocking filter 230, a memory 240, and a prediction unit 250.

[0065] The entropy decoding unit 200 decodes the encoded data (bit stream), obtains the quantization conversion coefficients corresponding to the blocks to be decoded, and outputs the obtained quantization conversion coefficients to the inverse quantization / inverse transformation unit 210. Further, the entropy decoding unit 200 obtains information regarding the conversion process and the quantization process, and outputs the information regarding the conversion process and the quantization process to the inverse quantization / inverse transformation unit 210. Furthermore, the entropy decoding unit 200 obtains information regarding the prediction process, and outputs the information regarding the prediction process to the prediction unit 250. The entropy decoding unit 200 obtains the color space conversion flag for each block to be decoded, and outputs the obtained color space conversion flag to the switching unit 215 and the filter control unit 231.

[0066] The inverse quantization / inverse transformation unit 210 performs inverse quantization processing and inverse transformation processing in block units. The inverse quantization / inverse transformation unit 210 includes an inverse quantization unit 211 and an inverse transformation unit 212.

[0067] The inverse quantization unit 211 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 inverse quantizes the quantized transform coefficients output by the entropy decoding unit 200 using the quantization parameter (Qp) and the scaling list, thereby restoring the transform coefficients of the block to be decoded, and outputs the restored transform coefficients to the inverse transform unit 212.

[0068] The inverse transform unit 212 performs an inverse transform process corresponding to the transform process performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 211 to restore the prediction residual, and outputs the restored prediction residual to the switching unit 215.

[0069] The switching unit 215 outputs the prediction residual of each color component output by the inverse transform unit 212 to either the synthesis unit 220 or the color space inverse transform unit 216 based on the color space conversion flag. The switching unit 111 outputs the prediction residual to the transform / quantization unit 120 for the block to which the color space inverse transform process (ACT) is applied, and outputs the prediction residual to the color space inverse transform unit 216 for the block to which the ACT is not applied.

[0070] The color space inverse transform unit 216 performs a color space inverse transform process (inverse ACT process), which is the inverse process of the ACT process performed by the color space transform unit 112 of the encoding device 1, and outputs the prediction residual after the inverse ACT process to the synthesis unit 220. Specifically, the color space inverse transform unit 216 performs the following inverse transform calculation using the Y component, Cg component, and Co component of the restored prediction residual.

[0071] t = Y - (Cg >> 1) G = Cg + t B = t - (Co >> 1) R = Co + B

[0072] The synthesis unit 220 decodes (reconstructs) the original block by synthesizing the prediction residual output by the switching unit 215 or the color space inverse transform unit 216 and the prediction block output by the prediction unit 250 on a pixel-by-pixel basis, and outputs the restored block to the deblocking filter 230.

[0073] The deblocking filter 230 performs a filtering process on the restored block output by the synthesizing unit 220, and outputs the restored block after the filtering process to the memory 240. Specifically, the deblocking filter 230 performs a filtering process on the block boundary of two blocks including the restored block (first block) and the restored block adjacent to the restored block (second block), and outputs each restored block after the filtering process to the memory 240. The function of the deblocking filter 230 is the same as that of the deblocking filter 160 of the encoding device 1.

[0074] The filter control unit 231 controls the deblocking filter 230. The filter control unit 231 controls the deblocking filter 230. Specifically, the filter control unit 231 controls the boundary filter strength (Bs: Boundary strength) indicating whether to perform a filtering process on the block boundary of the block pair, and the filter strength of the deblocking filter 230. The function of the filter control unit 231 is the same as that of the filter control unit 161 of the encoding device 1. The function of the filter control unit 231 determines the boundary filter strength Bs based on Table 1 above.

[0075] That is, the filter control unit 231 according to the present embodiment controls the boundary filter strength Bs of the deblocking filter 230 based on whether at least one of the adjacent blocks P and Q is encoded using adaptive color conversion (ACT).

[0076] As described above, the entropy decoding unit 200 acquires a flag (color space conversion flag) indicating whether encoding is performed using adaptive color conversion for each of the blocks P and Q. The filter control unit 231 controls the boundary filter strength Bs of the deblocking filter 230 based on the color space conversion flags for each of the blocks P and Q.

[0077] When at least one of block P and block Q is encoded using adaptive color conversion, the filter control unit 231 controls the boundary filter strength Bs so that the deblocking filter 230 performs filter processing (that is, sets the boundary filter strength Bs = 1). Specifically, even when there are no non-zero transform coefficients in both block P and block Q, if at least one of block P and block Q is encoded using adaptive color conversion, the filter control unit 231 controls the boundary filter strength Bs so that the deblocking filter 230 performs filter processing.

[0078] The memory 240 stores the restored blocks output by the synthesizing unit 220 and accumulates the restored blocks as a restored image in units of frames. The memory 240 outputs the restored blocks or the restored image to the prediction unit 250. Also, the memory 240 outputs the restored image in units of frames to the outside of the decoding device 2.

[0079] The prediction unit 250 performs prediction for each color component in units of blocks. The prediction unit 250 includes an inter-prediction unit 251, an intra-prediction unit 252, and a switching unit 253.

[0080] The inter-prediction unit 251 performs inter-prediction using the correlation between frames. Specifically, the inter-prediction unit 251 uses the restored image stored in the memory 240 as a reference image based on the information related to inter-prediction (for example, motion vector information) output by the entropy decoding unit 200 to predict the block to be encoded and generates an inter-prediction block, and outputs the generated inter-prediction block to the switching unit 253.

[0081] The intra prediction unit 252 performs intra prediction using the spatial correlation within a frame. Specifically, the intra prediction unit 252 uses an intra prediction mode corresponding to the information regarding intra prediction (e.g., intra prediction mode information) output by the entropy decoding unit 200, and refers to the restored pixels around the block to be encoded among the restored images stored in the memory 240 to generate an intra prediction block, and outputs the generated intra prediction block to the switching unit 253.

[0082] The switching unit 253 switches between the inter prediction block output by the inter prediction unit 251 and the intra prediction block output by the intra prediction unit 252, and outputs one of the prediction blocks to the synthesis unit 220.

[0083] As described above, the decoding apparatus 2 according to the present embodiment includes an entropy decoding unit 200 that outputs transform coefficients corresponding to block P by decoding an encoded stream, an inverse quantization / inverse transform unit 210 that restores a prediction residual corresponding to block P (first block) by performing inverse quantization processing and inverse transform processing on the transform coefficients output by the entropy decoding unit 200, a synthesis unit 220 that restores block P by synthesizing the restored prediction residual and the prediction block obtained by predicting block P, a deblocking filter 230 that performs a filter process on the boundary between the restored block P and the restored block Q (second block) adjacent to block P, and a filter control unit 231 that controls the boundary filter strength Bs of the deblocking filter 230 based on whether at least one of block P and block Q is encoded using adaptive color conversion (ACT).

[0084] For the block to which ACT is applied, after the prediction residual is restored from the transform coefficients by the inverse transform process, the color space of the prediction residual is inversely transformed from the YCgCo space to the RGB space by the color space inverse transform (inverse ACT). Therefore, when there are non-zero transform coefficients in a block of a certain color component, the non-zero transform coefficients will affect the blocks of other color components during the color space inverse transform.

[0085] Therefore, the filter control unit 231 according to this embodiment controls the boundary filter strength Bs of the deblocking filter 230 not only based on the presence or absence of non-zero conversion coefficients, but also considers whether or not ACT is applied to control the boundary filter strength Bs of the deblocking filter 230. As a result, the boundary filter strength Bs of the deblocking filter 230 can be appropriately controlled, so that even when ACT is applied, deterioration of image quality can be suppressed.

[0086] <Operation of Filter Control Unit> Next, the operations of the filter control unit 161 and the filter control unit 231 according to this embodiment will be described. Since the filter control unit 161 on the encoding side and the filter control unit 231 on the decoding side perform the same operation, here, the filter control unit 231 on the decoding side will be described as an example. FIG. 4 is a diagram showing an example of the operation of the filter control unit 231 according to this embodiment. Note that the order of the determinations shown in FIG. 4 is an example, and the order of the determinations may be changed.

[0087] As shown in FIG. 4, in step S1, the filter control unit 231 determines whether or not intra prediction is applied to at least one of the block pairs consisting of blocks P and Q. When intra prediction is applied to at least one of the block pairs (step S1: YES), in step S2, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 2.

[0088] When intra prediction is not applied to any of the block pairs (step S1: NO), in step S3, the filter control unit 231 determines whether or not the difference between the motion vectors of the target block pair is equal to or greater than a threshold value. When the difference between the motion vectors of the target block pair is equal to or greater than the threshold value (step S3: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

[0089] If the difference between the motion vectors of the block pair is not greater than the threshold value (step S3: NO), in step S5, the filter control unit 231 determines whether the number of motion vectors of the block pair or the reference image is different. If the number of motion vectors of the block pair or the reference image is different (step S5: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

[0090] If the number of motion vectors of the block pair or the reference image is the same (step S5: NO), in step S6, the filter control unit 231 determines whether at least one of the blocks of the block pair contains a non-zero transform coefficient. If at least one of the block pairs contains a non-zero transform coefficient (step S6: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

[0091] If none of the blocks in the block pair contains a non-zero transform coefficient (step S6: NO), in step S7, the filter control unit 231 determines whether at least one of the blocks in the block pair is encoded using ACT based on the color space conversion flag of each block output by the entropy decoding unit 200. If at least one of the blocks in the block pair is encoded using ACT (step S7: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1. On the other hand, if none of the blocks in the block pair is encoded using ACT (step S7: NO), in step S8, the filter control unit 231 controls the deblocking filter 230 not to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 0.

[0092] <Modified Example 1> The filter control unit 231 on the decoding side may determine whether at least one of the blocks in the block pair (block P and block Q) contains a non-zero transform coefficient based on a flag signaled from the encoding device 1.

[0093] Specifically, the entropy encoding unit 130 of the encoding device 1 includes a flag (tu_coded_flag) indicating whether or not it contains a non-zero transform coefficient in the encoding stream for each block. For example, the entropy encoding unit 130 sets the flag (tu_coded_flag) to "1" for a block containing a non-zero transform coefficient, and sets the flag (tu_coded_flag) to "0" for a block not containing a non-zero transform coefficient.

[0094] The entropy decoding unit 200 of the decoding device 2 acquires a flag (tu_coded_flag) for each block and outputs the acquired flag (tu_coded_flag) to the filter control unit 231. The filter control unit 231 interprets that a block with the flag (tu_coded_flag) being "1" does not include non-zero transform coefficients. Then, as shown in Table 2 below, the filter control unit 231 sets the boundary filter strength Bs of the deblocking filter 230.

[0095]

Table 2

[0096] As shown in Table 2, when the tu_coded_flag of at least one block of the block pair is "1", the filter control unit 231 controls the deblocking filter 230 to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

[0097] <Modification Example 2> As described above, for the block to which ACT is applied, after the prediction residual is restored from the transform coefficients by the inverse transform process, the color space of the prediction residual is inversely transformed from the YCgCo space to the RGB space by the inverse ACT. Therefore, when there are non-zero transform coefficients in a block of a certain color component, the non-zero transform coefficients affect the blocks of other color components during the inverse ACT.

[0098] Therefore, when ACT is applied, the control of the boundary filter strength Bs of the deblocking filter 230 based on the presence or absence of non-zero transform coefficients does not function properly. For this reason, in this modification example, the boundary filter strength Bs of the deblocking filter 230 is controlled based on the prediction residual after the inverse transform process instead of the transform coefficients. Thereby, even if the non-zero transform coefficients affect the blocks of other color components during the inverse ACT, the boundary filter strength Bs of the deblocking filter 230 can be appropriately controlled based on the prediction residual after being affected.

[0099] That is, the filter control unit 161 and the filter control unit 231 according to this modification example set the boundary filter strength Bs of the deblocking filter 230 based on Table 3 below.

[0100] [Table 3]

[0101] FIG. 5 is a diagram showing the configuration of the encoding device 1 according to this modification example.

[0102] As shown in FIG. 5, in the encoding device 1, the same prediction residual as the prediction residual (restored prediction residual) input to the synthesis unit 150 is input to the filter control unit 161. For the block to which ACT is applied, the prediction residual after inverse ACT is input to the filter control unit 161. When the restored prediction residual of at least one block of the block pair (block P and block Q) includes a non-zero value, the filter control unit 161 controls the deblocking filter 160 to perform deblocking filter processing. Specifically, the filter control unit 161 sets the boundary filter strength Bs = 1.

[0103] FIG. 6 is a diagram showing the configuration of the decoding device 2 according to this modification example.

[0104] As shown in FIG. 6, in the decoding device 2, the same prediction residual as the prediction residual (restored prediction residual) input to the synthesis unit 220 is input to the filter control unit 231. For the block to which ACT is applied, the prediction residual after inverse ACT is input to the filter control unit 231. When the restored prediction residual of at least one block of the block pair (block P and block Q) includes a non-zero value, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

[0105] [Second Embodiment] In the above-described embodiments and their modified examples, an example in which ACT, one of the encoding tools, is applied has been described. However, the same problems as described above may occur even when Joint coding of chroma residual (JCCR), another encoding tool, is applied. For this reason, the above-described embodiments and their modified examples may be applied to JCCR, and ACT in the above-described embodiments and their modified examples may be appropriately replaced with JCCR. For example, the boundary filter strength Bs can be controlled as shown in Table 4 below.

[0106]

Table 4

[0107] JCCR is a coefficient encoding mode for chrominance components (see Non-Patent Document 1). In JCCR, the encoding device 1 utilizes the correlation of the prediction residuals of the first chrominance component (Cb component) and the second chrominance component (Cr component) to generate one joint prediction residual from the prediction residual of the first chrominance component and the prediction residual of the second chrominance component. For example, the encoding device 1 generates a joint prediction residual by synthesizing the prediction residual of the second chrominance component with its sign reversed with the prediction residual of the first chrominance component. Then, the encoding device 1 performs conversion processing, quantization processing, and entropy encoding processing on the generated joint prediction residual and transmits it.

[0108] The decoding device 2 reconstructs the prediction residual of the first chrominance component and the prediction residual of the second chrominance component from the transmitted joint prediction residual. In this way, by transmitting only one joint prediction residual for two chrominance components, the encoding efficiency is improved.

[0109] As described above, the deblocking filter device according to the present embodiment includes a deblocking filter (160, 230) that performs a filter process on the boundary between the first reconstruction block (block P) and the second reconstruction block (block Q), and at least one of the first reconstruction block (block P) and the second reconstruction block (block Q) is coded using JCCR (Joint coding of chroma residual) that generates one joint prediction residual from the prediction residuals of the Cb color difference component and the Cr color difference component respectively. A filter control unit (161, 231) that controls the boundary filter strength Bs of the deblocking filter (160, 230) based on whether or not it is encoded.

[0110] When at least one of the first reconstruction block (block P) and the second reconstruction block (block Q) is coded using JCCR, the filter control unit (161, 231) may control the boundary filter strength Bs so as to perform the filter process by the deblocking filter (160, 230).

[0111] Even when there are no non-zero transform coefficients in both the first reconstruction block (block P) and the second reconstruction block (block Q), when at least one of the first reconstruction block (block P) and the second reconstruction block (block Q) is coded using JCCR, the filter control unit (161, 231) may control the boundary filter strength Bs so as to perform the filter process by the deblocking filter (160, 230).

[0112] In this embodiment, the entropy decoding unit 200 of the decoding apparatus 2 may obtain, for each of the first reconstruction block (block P) and the second reconstruction block (block Q), a flag indicating whether or not it is encoded using JCCR. The filter control unit 231 may control the boundary filter strength Bs of the deblocking filter 230 based on the flags for each of the first reconstruction block (block P) and the second reconstruction block (block Q). For example, when the flag of at least one of the first reconstruction block (block P) and the second reconstruction block (block Q) is "1", the filter control unit 231 may control the boundary filter strength Bs so as to perform filter processing by the deblocking filter 230.

[0113] In this embodiment, the entropy decoding unit 200 of the decoding apparatus 2 may further obtain, for each of the first reconstruction block (block P) and the second reconstruction block (block Q), a tu_coded_flag indicating whether or not it includes non-zero transform coefficients. The filter control unit 231 may further control the boundary filter strength Bs of the deblocking filter 230 based on the tu_coded_flag for each of the first reconstruction block (block P) and the second reconstruction block (block Q). For example, when the tu_coded_flag of at least one of the first reconstruction block (block P) and the second reconstruction block (block Q) is "1", the filter control unit 231 may control the boundary filter strength Bs so as to perform filter processing by the deblocking filter 230.

[0114] For each block of the first color difference component (Cb component) and the second color difference component (Cr component) to which JCCR is applied, although the above-described tu_coded_flag is set to "1", actually there are no transform coefficients in one of the color components. Therefore, similar to the above-described modification example 2, it is preferable to use, for a certain color component, determination as to whether or not the reconstructed prediction residual includes non-zero values instead of determination as to whether or not non-zero transform coefficients exist.

[0115] [Other Embodiments] In the above-described embodiments, an example of controlling the boundary filter strength Bs as the control of the deblocking filters (160, 230) has been mainly described. However, not limited to the control of the boundary filter strength Bs, control of the filter length or switching control of a plurality of filters may be performed.

[0116] A program for causing a computer to execute each process performed by the above-described encoding apparatus 1 may be provided. Also, a program for causing a computer to execute each process performed by the decoding apparatus 2 may be provided. The program may be recorded on a computer-readable medium. By using a computer-readable medium, it is possible to install the program in a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0117] The circuits that execute each process performed by the encoding apparatus 1 may be integrated, and the encoding apparatus 1 may be configured by a semiconductor integrated circuit (chipset, SoC). The circuits that execute each process performed by the decoding apparatus 2 may be integrated, and the decoding apparatus 2 may be configured by a semiconductor integrated circuit (chipset, SoC).

[0118] As described above, the embodiments have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist.

[0119] This application claims the priority of Japanese Patent Application No. 2020-101293 (filed on June 10, 2020), the entire content of which is incorporated herein by reference.

Claims

1. A deblocking filter that performs a filtering process on a boundary between a first reconstruction block and a second reconstruction block adjacent to the first reconstruction block; A filter control unit that controls a boundary filter strength of the deblocking filter based on whether at least one of the first reconstruction block and the second reconstruction block is encoded using a color conversion process; The color conversion process is an encoding tool that converts a color space of a prediction residual from a first color space to a second color space different from the first color space; The color conversion process includes a process of generating a prediction residual composed of new color components by any one of addition, subtraction, multiplication, division, and shift processes for each color component in the first color space; When at least one of the first reconstruction block and the second reconstruction block is encoded using the color conversion process, the filter control unit controls the boundary filter strength so as to perform the filtering process; When a prediction residual obtained by an inverse color conversion process includes a non-zero value for a predetermined color component, the filter control unit controls the boundary filter strength so as to perform the filtering process for the predetermined color component A deblocking filter device.

2. Even when there are no non-zero conversion coefficients in both the first reconstruction block and the second reconstruction block, when at least one of the first reconstruction block and the second reconstruction block is encoded using the color conversion process, the filter control unit controls the boundary filter strength so as to perform the filtering process The deblocking filter device according to claim 1.

3. The deblocking filter device according to claim 1 or 2; An entropy decoder that acquires, for each of the first reconstruction block and the second reconstruction block, a flag indicating whether it is encoded using the color conversion process; The filter control unit controls the boundary filter strength of the deblocking filter based on the flag for each of the first reconstruction block and the second reconstruction block. Decoding device. Claim 4 When the flag of at least one of the first reconstruction block and the second reconstruction block is "1", the filter control unit controls the boundary filter strength so as to perform the filter process. The decoding device according to claim 3. Claim 5 The entropy decoding unit further obtains a tu_coded_flag indicating whether each of the first reconstruction block and the second reconstruction block includes a non-zero transform coefficient. The filter control unit further controls the boundary filter strength of the deblocking filter based on the tu_coded_flag for each of the first reconstruction block and the second reconstruction block. The decoding device according to claim 3 or 4. Claim 6 When the tu_coded_flag of at least one of the first reconstruction block and the second reconstruction block is "1", the filter control unit controls the boundary filter strength so as to perform the filter process. The decoding device according to claim 5. Claim 7 A program for causing a computer to function as the deblocking filter device according to claim 1 or 2.

Images