Deblocking filter device, decoding device, and program

Abstract

To provide a deblocking filter device, a decoding device, and a program for suppressing degradation of an image quality.SOLUTION: In a decoding device 2, a deblocking filter device includes: a deblocking filter 230 that performs a filtering process on a boundary between a first reconstructed block and a second reconstructed block adjacent to the first reconstructed block; and a filter control part 231 that controls a boundary filter strength of the deblocking filter based on whether or not at least one of the first reconstructed block and the second reconstructed block is encoded using a JCCR (Joint coding of chroma residual) that generates one joint prediction residual from a prediction residual of a Cb color difference component and a Cr color difference component.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a deblocking filter device, a decoding device, and a program.

Background Art

[0002] In HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding), which is a next-generation coding method, a deblocking filter is adopted as an in-loop filter to suppress distortion at the boundary of blocks when performing coding processing in block units. In the control of the deblocking filter, the boundary filtering strength of the deblocking filter is controlled according to whether there is a non-zero transform coefficient in at least one of two adjacent blocks. This is because the energy of the prediction residual is distributed throughout the block due to the inverse transform of the non-zero transform coefficient, and thus there is a high possibility of discontinuity occurring at the boundary between the two blocks.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

[0004] A deblocking filter apparatus according to the first embodiment includes a deblocking filter that performs filtering on the boundary between a first reconstruction block and a second reconstruction block adjacent to the first reconstruction block, and a filter control unit that controls the boundary filtering intensity of the deblocking filter based on whether or not at least one of the first reconstruction block and the second reconstruction block is encoded using JCCR (Joint coding of chroma residual), which generates a single joint predicted residual from the predicted residuals of the Cb chroma difference component and the Cr chroma difference component, respectively.

[0005] The decoding device according to the second embodiment comprises a deblocking filter device according to the first embodiment and an entropy decoding unit that acquires a flag for each of the first and second reconstructed blocks indicating whether or not the JCCR is encoded, and the filter control unit controls the boundary filter strength of the deblocking filter based on the flag for each of the first and second reconstructed blocks.

[0006] The program according to the third embodiment causes the computer to function as a deblocking filter device according to the first embodiment. [Brief explanation of the drawing]

[0007] [Figure 1] This figure shows the configuration of the encoding device according to the embodiment. [Figure 2] This is a diagram illustrating the operation of the deblocking filter according to the embodiment. [Figure 3] This figure shows the configuration of the decoding device according to the embodiment. [Figure 4] This figure shows an example of the operation of the filter control unit according to the embodiment. [Figure 5] This diagram shows the configuration of the encoding device related to modification example 2. [Figure 6] This diagram shows the configuration of the decoding device related to modification example 2. [Modes for carrying out the invention]

[0008] The VVC draft employs a technique called Adaptive Colour Transform (ACT) (see Non-Patent Literature 1) in which, when the chroma format of the input video is 4:4:4, the color space (RGB space) of the predicted residual is converted to the YCgCo space, and then encoding processing such as conversion processing and entropy encoding processing is performed on the predicted residual after the color space conversion. The encoding device can control whether or not to apply ACT for each block to be encoded, and outputs an ACT application flag as a stream for each block to be encoded. The decoding device performs entropy decoding and inverse conversion processing on the block encoded using ACT to restore the predicted residual, and then inversely converts the color space (YCgCo space) of the restored predicted residual back to the RGB space.

[0009] In the conventional technology described above, the decoder applies a deblocking filter to the boundary between two adjacent blocks if at least one of the two adjacent blocks contains a non-zero conversion coefficient. On the other hand, if neither of the two adjacent blocks contains a non-zero conversion coefficient, it is possible not to apply a deblocking filter to the boundary between the two blocks.

[0010] However, for blocks to which ACT is applied, after the predicted residuals are restored from the conversion coefficients by the inverse transformation process, the color space of the predicted residuals is inversely transformed from the YCgCo space to the RGB space by the color space inverse transformation. Therefore, if a non-zero conversion coefficient exists in a block of a certain color component, that non-zero conversion coefficient will affect other color component blocks during the color space inverse transformation.

[0011] Therefore, applying deblocking filter control based on the presence or absence of non-zero conversion coefficients to blocks encoded using ACT may lead to image quality degradation because the boundary filter strength of the deblocking filter cannot be properly controlled.

[0012] Similar problems can occur when other coding tools, such as Joint coding of chroma residual (JCCR), are applied.

[0013] Therefore, the purpose of this disclosure is to provide a deblocking filter device, a decoding device, and a program that suppress image quality degradation.

[0014] An encoding device and a decoding device according to an embodiment will be described with reference to the drawings. The encoding device and decoding device according to the embodiment perform encoding and decoding of moving images, such as MPEG, respectively. In the following drawings, identical or similar parts are denoted by the same or similar reference numerals.

[0015] [First Embodiment] <Configuration of the encoding device> First, the configuration of the encoding device according to this embodiment will be described. Figure 1 is a diagram showing the configuration of the encoding device 1 according to this embodiment.

[0016] As shown in Figure 1, the encoding device 1 includes a block division unit 100, a residual generation unit 110, a switching unit 111, a color space conversion unit 112, a conversion / quantization unit 120, an entropy encoding unit 130, an inverse quantization / inverse conversion unit 140, a switching unit 143, a color space inverse conversion unit 144, a synthesis unit 150, a deblocking filter 160, a memory 170, and a prediction unit 180.

[0017] The block division unit 100 divides the original image, which is an input image in units of frames (or pictures) that constitute the moving image, into multiple image blocks, and outputs the image blocks obtained by the division to the residual generation unit 110. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the image block is not limited to a square; it may also be rectangular (non-square). The image block is a unit that the encoding device 1 performs encoding processing on (i.e., the block to be encoded), and a unit that the decoding device performs decoding processing on (i.e., the block to be decoded). Such an image block is sometimes called a CU (Coding Unit).

[0018] The input image is an RGB signal, and the chroma format may be 4:4:4. The RGB space is an example of a first color space. The "R" component corresponds to the first component, the "G" component corresponds to the second component, and the "B" component corresponds to the third component. The block division unit 100 outputs blocks by dividing each of the R, G, and B components that constitute the image into blocks. In the following description of the encoding device, when each color component is not distinguished, it is simply referred to as the encoded block.

[0019] The residual generation unit 110 calculates a predicted residual, which represents the difference (error) between the block to be encoded output by the block division unit 100 and the predicted block obtained by the prediction unit 180 predicting the block to be encoded. Specifically, the residual generation unit 110 calculates the predicted residual by subtracting each pixel value of the predicted block from each pixel value of the block to be encoded, and outputs the calculated predicted residual to the switching unit 111. In this embodiment, the residual generation unit 110 generates the predicted residual for each color component by the difference between the block to be encoded for each color component and the predicted block for each color component.

[0020] The switching unit 111 outputs the prediction residuals of each color component output by the residual generation unit 110 to either the conversion / quantization unit 120 or the color space conversion unit 112. When the color space conversion process (ACT process) is not performed, the switching unit 111 outputs the prediction residuals to the conversion / quantization unit 120, and when the ACT process is performed, the switching unit 111 outputs the prediction residuals to the color space conversion unit 112.

[0021] The color space conversion unit 112 performs the ACT process on the prediction residuals of each color component and outputs the prediction residuals after the ACT process to the conversion / quantization unit 120. The color space conversion unit 112 generates new prediction residuals by performing the following conversion calculations on the R component, G component, and B component of the prediction residuals of the block to be encoded.

[0022] Co = R - B t = B + (Co >> 1) Cg = G - t Y = t + (Cg >> 1)

[0023] Here, ">>" represents a right shift operation. Also, the "Y" component corresponds to the first component, the "Cg" component corresponds to the second component, and the "Co" component corresponds to the third component. Such a YCgCo space is an example of the second color space.

[0024] The switching unit 111 and the color space conversion unit 112 can control whether to perform the color conversion process for each block to be encoded. The entropy encoding unit 130 signals in the bit stream a flag (ACT application flag) indicating whether the color conversion process has been performed on the encoded block.

[0025] [[ID=Z5]] Note that the ACT process in the color space conversion unit 112 may generate prediction residuals composed of new color components by addition, subtraction, multiplication, division, shift processing, etc. for each color component. Also, the ACT process does not necessarily need to be a conversion that affects all color components. For example, the color space conversion unit 112 may apply an ACT process in which the first component is maintained without change, the average value of the second component and the third component is used as the new second component, and the difference between the second component and the third component is used as the new third component.

[0026] The conversion and quantization unit 120 performs conversion and quantization processing in block units. The conversion and quantization unit 120 comprises a conversion unit 121 and a quantization unit 122.

[0027] The conversion unit 121 performs a conversion process on the predicted residuals (referred to as predicted residuals regardless of whether ACT processing is applied) output by the switching unit 111 or the color space conversion unit 112 to calculate conversion coefficients, and outputs the calculated conversion coefficients to the quantization unit 122. Specifically, the conversion unit 121 generates conversion coefficients for each color component by performing a conversion process on the predicted residuals of each color component in block units. The conversion process can be any frequency conversion such as DCT, DST, or discrete wavelet transform. The conversion unit 121 also outputs information related to the conversion process to the entropy coding unit 130.

[0028] The conversion process includes conversion skipping, which does not perform any conversion, as adopted in the HEVC and VVC standards. In HEVC's conversion skip mode, conversion coefficients are obtained by scaling the predicted residuals without performing horizontal and vertical conversion processing. However, the conversion skipping according to this embodiment also includes conversions that apply conversion processing only horizontally or only vertically. Furthermore, the conversion unit 121 may perform a secondary conversion process that applies further conversion processing to the conversion coefficients obtained by the conversion process. In addition, the secondary conversion process may be applied only to a portion of the conversion coefficients.

[0029] The quantization unit 122 quantizes the conversion coefficients output from the conversion unit 121 using the quantization parameters and scaling list, and outputs the quantized conversion coefficients to the entropy coding unit 130 and the inverse quantization / inverse conversion unit 140. The quantization unit 122 also outputs information related to the quantization process (specifically, information on the quantization parameters and scaling list used in the quantization process) to the entropy coding unit 130 and the inverse quantization unit 141.

[0030] The entropy coding unit 130 performs entropy coding on the quantization conversion coefficients output by the quantization unit 122, compresses the data, generates a bitstream (encoded data), and outputs the bitstream to the decoding side. Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc., can be used for entropy coding. In addition, the entropy coding unit 130 signals to the decoding side by including information about the conversion process input from the conversion unit 121 in the bitstream, and by including information about the prediction process input from the prediction unit 180 in the bitstream.

[0031] Furthermore, the entropy encoding unit 130 signals the decoding side by including a color space conversion flag in the bitstream for each block to be encoded, indicating whether or not ACT is applied. Such a color space conversion flag is also called an ACT application flag. When the ACT application flag is on ("1"), it indicates that ACT is applied to the corresponding block to be encoded. When the ACT application flag is off ("0"), it indicates that ACT is not applied to the corresponding block to be encoded. Alternatively, an ACT non-application flag may be used instead of the ACT application flag. In that case, when the ACT non-application flag is on ("1"), it indicates that ACT is not applied to the corresponding block to be encoded. When the ACT non-application flag is off ("0"), it indicates that ACT is applied to the corresponding block to be encoded.

[0032] The inverse quantization / inverse transformation unit 140 performs inverse quantization and inverse transformation processing on a block-by-block basis. The inverse quantization / inverse transformation unit 140 comprises an inverse quantization unit 141 and an inverse transformation unit 142.

[0033] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the conversion coefficients by inverse quantization of the quantization conversion coefficients output by the quantization unit 122 using the quantization parameter (Qp) and the scaling list, and outputs the restored conversion coefficients to the inverse conversion unit 142.

[0034] The inverse transform unit 142 performs an inverse transform process corresponding to the transform process performed by the transform unit 121. For example, if the transform unit 121 performs a discrete cosine transform, the inverse transform unit 142 performs an inverse discrete cosine transform. The inverse transform unit 142 performs an inverse transform process on the transformation coefficients output by the inverse quantization unit 141 to restore the predicted residuals, and outputs the restored predicted residuals to the switching unit 143.

[0035] The switching unit 143 outputs the predicted residuals of each color component output by the inverse conversion unit 142 to either the synthesis unit 150 or the color space inverse conversion unit 144. For blocks to which ACT is applied, the switching unit 143 outputs the predicted residuals to the synthesis unit 150, and for blocks to which ACT is not applied, it outputs the predicted residuals to the color space inverse conversion unit 144.

[0036] The color space inverse conversion unit 144 performs a color space inverse conversion process (inverse ACT process), which is the reverse process of the ACT process performed by the color space conversion unit 112, and outputs the predicted residuals after the inverse ACT process to the synthesis unit 150. Specifically, the inverse conversion from the YCgCo space to the RGB space is performed by performing the following inverse conversion calculation using the Y component, Cg component, and Co component of the restored predicted residuals.

[0037] t = Y - (Cg >> 1) G = Cg + t B = t - (Co >> 1) R=Co+B

[0038] The synthesis unit 150 synthesizes the restored predicted residuals output by the inverse transform unit 142 or the color space inverse transform unit 144 with the predicted blocks output by the prediction unit 180 on a pixel-by-pixel basis. The synthesis unit 150 adds each pixel value of the restored predicted residuals to each pixel value of the predicted blocks to restore (reconstruct) the blocks to be encoded, and outputs the restored blocks to the deblocking filter 160. The restored blocks are sometimes called reconstructed blocks.

[0039] The deblocking filter 160 performs filtering on the restored blocks output by the synthesis unit 150 and outputs the filtered restored blocks to the memory 170. The filter control unit 161 controls the deblocking filter 160. Details of the deblocking filter 160 and the filter control unit 161 will be described later.

[0040] Memory 170 stores the reconstructed blocks after filtering output by the deblocking filter 160 and accumulates the reconstructed blocks as reconstructed images on a frame-by-frame basis. Memory 170 outputs the stored reconstructed blocks or reconstructed images to the prediction unit 180.

[0041] The prediction unit 180 performs prediction processing on a block-by-block basis. The prediction unit 180 generates prediction blocks for each color component by performing prediction processing such as intra-prediction and inter-prediction on the blocks to be encoded. The prediction unit 180 includes an inter-prediction unit 181, an intra-prediction unit 182, and a switching unit 183.

[0042] The interpretation unit 181 performs interpretation using correlations between frames. Specifically, the interpretation unit 181 uses the restored image stored in memory 170 as a reference image, calculates motion vectors using methods such as block matching, predicts the blocks to be encoded, generates interpretation blocks, and outputs the generated interpretation blocks to the switching unit 183. Here, the interpretation unit 181 selects the optimal interpretation method from among interpretation using multiple reference images (typically biprediction) and interpretation using one reference image (unidirectional prediction), and performs interpretation using the selected interpretation method. The interpretation unit 181 outputs information related to interpretation (motion vectors, etc.) to the entropy encoding unit 130.

[0043] The intra-prediction unit 182 performs intra-prediction using spatial correlations within the frame. Specifically, the intra-prediction unit 182 generates intra-prediction blocks by referring to restored pixels surrounding the block to be encoded from the restored image stored in memory 170, and outputs the generated intra-prediction blocks to the switching unit 183. The intra-prediction unit 182 selects an intra-prediction mode to apply to the block to be encoded from among multiple intra-prediction modes, and predicts the block to be encoded using the selected intra-prediction mode.

[0044] The switching unit 183 switches between the inter-prediction block output by the inter-prediction unit 181 and the intra-prediction block output by the intra-prediction unit 182, and outputs one of the prediction blocks to the residual generation unit 110 and the synthesis unit 150.

[0045] Next, the deblocking filter 160 and the filter control unit 161 according to this embodiment will be described.

[0046] The deblocking filter 160 performs filtering on the block boundary between two blocks, consisting of a restored block (first block) and a restored block adjacent to it (second block), and outputs each restored block after filtering to the memory 170. The filtering process is a process to reduce signal degradation caused by block-level processing, and is a filtering process that smooths the signal gap at the block boundary between two adjacent blocks.

[0047] The filter control unit 161 controls the deblocking filter 160. Specifically, the filter control unit 161 controls the boundary filter strength (Bs), which indicates whether or not to perform filtering on the block boundaries of a block pair, and the filter strength of the deblocking filter 160. The boundary filter strength Bs is a parameter used to determine whether or not to apply filtering and the type of filtering. The control of whether or not to perform filtering can be considered as controlling whether or not to set the boundary filter strength Bs to 1 or more or to zero.

[0048] Figure 2 is a diagram illustrating the operation of the deblocking filter 160 according to this embodiment. In the example shown in Figure 2, the deblocking filter 160 performs filtering on the block boundaries of each 8x8 pixel block. The deblocking filter 160 also performs filtering in units of 4 rows or 4 columns. Blocks P (first reconstructed block) and Q (second reconstructed block) shown in Figure 2 represent one unit of filtering by the deblocking filter 160, and show an example where the block size is 4x4 pixels. Blocks P and Q may also be called subblocks. Block Q is a reconstructed block corresponding to the block to be encoded, and block P is a reconstructed block adjacent to block Q.

[0049] The filter control unit 161 determines the boundary filter strength Bs based on Table 1 below. In this embodiment, the value of the boundary filter strength Bs is one of 0, 1, or 2.

[0050] [Table 1]

[0051] As shown in Figure 2 and Table 1, the filter control unit 161 sets the Bs value to 2 when intra prediction is applied to at least one of blocks P and Q.

[0052] The filter control unit 161 sets the Bs value to 1 if motion compensation prediction (interpretation) is applied to both blocks P and Q and at least one of the following conditions (a) to (d) is met, and sets the Bs value to 0 otherwise.

[0053] (a) The absolute value of the difference between the motion vectors of blocks P and Q is greater than or equal to a threshold (e.g., 1 pixel).

[0054] (b) The number of motion vectors or reference images for blocks P and Q are different.

[0055] (c) At least one of blocks P and Q contains a significant transformation coefficient (i.e., a non-zero transformation coefficient).

[0056] (d) ACT is applied to at least one of blocks P and Q.

[0057] The filter control unit 161 controls the deblocking filter 160 so that it does not perform deblocking filtering when the value of the boundary filter intensity Bs is 0. The following explanation will use the vertical block boundary shown in Figure 2 as an example.

[0058] The filter control unit 161 may control the deblocking filter 160 to perform deblocking filter processing if the value of the boundary filter intensity Bs is 1 or 2 and the following equation (1) is satisfied.

[0059]

number

[0060] Furthermore, when the filter control unit 161 performs deblocking filtering, it may apply a strong filter if all of the following conditions (2) to (7) are met, and apply a weak filter otherwise.

[0061]

number

[0062] However, the threshold values ​​β and tC vary depending on the mean value Qav of the quantization parameters of adjacent blocks P and Q.

[0063] <Configuration of the decryption device> Next, the decoding device according to this embodiment will be described, primarily focusing on the differences between it and the encoding device 1. Figure 3 shows the configuration of the decoding device 2 according to this embodiment.

[0064] As shown in Figure 3, the decoding device 2 includes an entropy decoding unit 200, an inverse quantization / inverse conversion unit 210, a switching unit 215, a color space inverse conversion unit 216, a synthesis unit 220, a deblocking filter 230, a memory 240, and a prediction unit 250.

[0065] The entropy decoding unit 200 decodes the encoded data (bitstream), obtains the quantization conversion coefficients corresponding to the decoded block, and outputs the obtained quantization conversion coefficients to the inverse quantization / inverse conversion unit 210. The entropy decoding unit 200 also obtains information regarding the conversion and quantization processes and outputs this information to the inverse quantization / inverse conversion unit 210. Furthermore, the entropy decoding unit 200 obtains information regarding the prediction process and outputs this information to the prediction unit 250. The entropy decoding unit 200 obtains a color space conversion flag for each decoded block and outputs the obtained color space conversion flags to the switching unit 215 and the filter control unit 231.

[0066] The inverse quantization / inverse transformation unit 210 performs inverse quantization and inverse transformation processing on a block-by-block basis. The inverse quantization / inverse transformation unit 210 comprises an inverse quantization unit 211 and an inverse transformation unit 212.

[0067] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 recovers the conversion coefficients of the decoded block by inverse quantization of the quantization conversion coefficients output by the entropy decoding unit 200 using the quantization parameter (Qp) and the scaling list, and outputs the recovered conversion coefficients to the inverse conversion unit 212.

[0068] The inverse transform unit 212 performs an inverse transform process corresponding to the transform process performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 211 to restore the predicted residuals and outputs the restored predicted residuals to the switching unit 215.

[0069] The switching unit 215 outputs the predicted residuals of each color component output by the inverse conversion unit 212 to either the synthesis unit 220 or the color space inverse conversion unit 216, based on the color space conversion flag. The switching unit 111 outputs the predicted residuals to the conversion / quantization unit 120 for blocks to which color space inverse conversion processing (ACT) has been applied, and outputs the predicted residuals to the color space inverse conversion unit 216 for blocks to which ACT has been applied.

[0070] The color space inverse conversion unit 216 performs a color space inverse conversion process (inverse ACT process), which is the inverse process of the ACT process performed by the color space conversion unit 112 of the encoding device 1, and outputs the predicted residual after the inverse ACT process to the synthesis unit 220. Specifically, the color space inverse conversion unit 216 performs the following inverse conversion calculation using the Y component, Cg component, and Co component of the reconstructed predicted residual.

[0071] t = Y - (Cg >> 1) G = Cg + t B = t - (Co >> 1) R=Co+B

[0072] The synthesis unit 220 decodes (reconstructs) the original block by combining the predicted residual output by the switching unit 215 or the color space inverse conversion unit 216 with the predicted block output by the prediction unit 250 on a pixel-by-pixel basis, and outputs the restored block to the deblocking filter 230.

[0073] The deblocking filter 230 performs filtering on the restored blocks output by the synthesis unit 220 and outputs the filtered restored blocks to the memory 240. Specifically, the deblocking filter 230 performs filtering on the block boundary of two blocks consisting of a restored block (first block) and a restored block adjacent to it (second block), and outputs each filtered restored block to the memory 240. The function of the deblocking filter 230 is the same as that of the deblocking filter 160 of the encoding device 1.

[0074] The filter control unit 231 controls the deblocking filter 230. Specifically, the filter control unit 231 controls the boundary filter strength (Bs), which indicates whether or not to perform filtering on the block boundaries of block pairs, and the filter strength of the deblocking filter 230. The function of the filter control unit 231 is the same as that of the filter control unit 161 of the encoding device 1. The function of the filter control unit 231 is to determine the boundary filter strength Bs based on Table 1 above.

[0075] In other words, the filter control unit 231 according to this embodiment controls the boundary filter intensity Bs of the deblocking filter 230 based on whether or not at least one of adjacent blocks P and Q is encoded using adaptive color conversion (ACT).

[0076] As described above, the entropy decoding unit 200 obtains a flag (color space conversion flag) for each of block P and block Q indicating whether or not it is encoded using adaptive color conversion. The filter control unit 231 controls the boundary filter intensity Bs of the deblocking filter 230 based on the color space conversion flag for each of block P and block Q.

[0077] The filter control unit 231 controls the boundary filter intensity Bs to perform filtering by the deblocking filter 230 if at least one of block P and block Q is encoded using adaptive color conversion (i.e., it sets the boundary filter intensity Bs = 1). Specifically, even if there are no non-zero conversion coefficients in both block P and block Q, the filter control unit 231 controls the boundary filter intensity Bs to perform filtering by the deblocking filter 230 if at least one of block P and block Q is encoded using adaptive color conversion.

[0078] Memory 240 stores the restored blocks output by the synthesis unit 220 and accumulates the restored blocks as restored images on a frame-by-frame basis. Memory 240 outputs the restored blocks or restored images to the prediction unit 250. Memory 240 also outputs the restored images on a frame-by-frame basis to the outside of the decoding device 2.

[0079] The prediction unit 250 performs predictions for each color component in block units. The prediction unit 250 includes an inter-prediction unit 251, an intra-prediction unit 252, and a switching unit 253.

[0080] The interpretation unit 251 performs interpretation using correlations between frames. Specifically, the interpretation unit 251 predicts the blocks to be encoded using the restored image stored in the memory 240 as a reference image based on interpretation information (e.g., motion vector information) output by the entropy decoding unit 200, generates interpretation prediction blocks, and outputs the generated interpretation prediction blocks to the switching unit 253.

[0081] The intra-prediction unit 252 performs intra-prediction using spatial correlations within the frame. Specifically, the intra-prediction unit 252 uses an intra-prediction mode corresponding to the intra-prediction information (e.g., intra-prediction mode information) output by the entropy decoding unit 200 to generate an intra-prediction block by referring to the restored pixels surrounding the encoding target block in the restored image stored in the memory 240, and outputs the generated intra-prediction block to the switching unit 253.

[0082] The switching unit 253 switches between the inter-prediction block output by the inter-prediction unit 251 and the intra-prediction block output by the intra-prediction unit 252, and outputs one of the prediction blocks to the synthesis unit 220.

[0083] As described above, the decoding device 2 according to this embodiment includes: an entropy decoding unit 200 that outputs conversion coefficients corresponding to block P by decoding an encoded stream; an inverse quantization / inverse transform unit 210 that restores the predicted residuals corresponding to block P (first block) by performing inverse quantization and inverse transform processing on the conversion coefficients output by the entropy decoding unit 200; a synthesis unit 220 that restores block P by combining the restored predicted residuals and the predicted block obtained by predicting block P; a deblocking filter 230 that performs filtering on the boundary between the restored block P and the restored block Q (second block) adjacent to block P; and a filter control unit 231 that controls the boundary filter intensity Bs of the deblocking filter 230 based on whether or not at least one of block P and block Q is encoded using adaptive color transformation (ACT).

[0084] For blocks to which ACT is applied, the predicted residuals are restored from the conversion coefficients through the inverse transformation process, and then the color space of the predicted residuals is inversely transformed from the YCgCo space to the RGB space through the inverse color space transformation (inverse ACT). Therefore, if a block of a certain color component has non-zero conversion coefficients, those non-zero conversion coefficients will affect other color component blocks during the inverse color space transformation.

[0085] Therefore, the filter control unit 231 according to this embodiment controls the boundary filter intensity Bs of the deblocking filter 230 not only based on the presence or absence of non-zero conversion coefficients, but also by considering whether or not ACT is applied. This makes it possible to appropriately control the boundary filter intensity Bs of the deblocking filter 230, thereby suppressing image quality degradation even when ACT is applied.

[0086] <Operation of the filter control unit> Next, the operation of the filter control unit 161 and the filter control unit 231 according to this embodiment will be described. Since the encoding-side filter control unit 161 and the decoding-side filter control unit 231 perform the same operation, the decoding-side filter control unit 231 will be used as an example for this explanation. Figure 4 shows an example of the operation of the filter control unit 231 according to this embodiment. Note that the order of determination shown in Figure 4 is just one example, and the order of determination may be changed.

[0087] As shown in Figure 4, in step S1, the filter control unit 231 determines whether intra prediction is applied to at least one of the block pair consisting of block P and Q. If intra prediction is applied to at least one of the block pair (step S1: YES), in step S2, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 2.

[0088] If no intra prediction is applied to any of the block pairs (step S1: NO), in step S3, the filter control unit 231 determines whether the difference in motion vectors of the target block pairs is greater than or equal to a threshold. If the difference in motion vectors of the target block pairs is greater than or equal to a threshold (step S3: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs=1.

[0089] If the difference between the motion vectors of the block pairs is not greater than or equal to a threshold (step S3: NO), in step S5, the filter control unit 231 determines whether the number of motion vectors or the reference image of the block pairs are different. If the number of motion vectors or the reference image of the block pairs are different (step S5: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter intensity Bs=1.

[0090] If the number of motion vectors or reference images of the block pairs are the same (step S5: NO), in step S6, the filter control unit 231 determines whether at least one of the blocks in the block pair contains a non-zero transformation coefficient. If at least one of the block pairs contains a non-zero transformation coefficient (step S6: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter intensity Bs = 1.

[0091] If neither block in the block pair contains a non-zero conversion coefficient (step S6: NO), in step S7, the filter control unit 231 determines whether at least one block in the block pair is encoded using ACT based on the color space conversion flag of each block output by the entropy decoding unit 200. If at least one block in the block pair is encoded using ACT (step S7: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter intensity Bs=1. On the other hand, if neither block in the block pair is encoded using ACT (step S7: NO), in step S8, the filter control unit 231 controls the deblocking filter 230 not to perform deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter intensity Bs=0.

[0092] <Example of change 1> The decoder-side filter control unit 231 may determine whether at least one of the block pairs (block P and block Q) contains non-zero conversion coefficients based on a flag signaled by the encoding device 1.

[0093] Specifically, the entropy coding unit 130 of the coding device 1 includes a flag (tu_coded_flag) in the coded stream for each block, indicating whether or not it contains non-zero conversion coefficients. For example, the entropy coding unit 130 sets the flag (tu_coded_flag) to "1" for blocks that contain non-zero conversion coefficients, and sets the flag (tu_coded_flag) to "0" for blocks that do not contain non-zero conversion coefficients.

[0094] The entropy decoding unit 200 of the decoding device 2 acquires a flag (tu_coded_flag) for each block and outputs the acquired flag (tu_coded_flag) to the filter control unit 231. The filter control unit 231 interprets blocks with a flag (tu_coded_flag) of "1" as not containing non-zero conversion coefficients. The filter control unit 231 then sets the boundary filter intensity Bs of the deblocking filter 230 as shown in Table 2 below.

[0095] [Table 2]

[0096] As shown in Table 2, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering if the tu_coded_flag of at least one block in the block pair is "1". Specifically, the filter control unit 231 sets the boundary filter strength Bs=1.

[0097] <Example of change 2> As described above, for blocks to which ACT is applied, the predicted residuals are restored from the conversion coefficients by the inverse transformation process, and then the color space of the predicted residuals is inversely transformed from the YCgCo space to the RGB space by the inverse ACT process. Therefore, if a non-zero conversion coefficient exists in a block of a certain color component, that non-zero conversion coefficient will affect other color component blocks during the inverse ACT process.

[0098] Therefore, when ACT is applied, the control of the boundary filter intensity Bs of the deblocking filter 230 based on the presence or absence of non-zero transformation coefficients does not function properly. For this reason, in this modified example, the boundary filter intensity Bs of the deblocking filter 230 is controlled based on the predicted residuals after the inverse transformation process, rather than the transformation coefficients. This allows the boundary filter intensity Bs of the deblocking filter 230 to be properly controlled based on the predicted residuals after the impact, even if non-zero transformation coefficients affect the blocking of other color components during inverse ACT.

[0099] In other words, the filter control unit 161 and filter control unit 231 in this modified example set the boundary filter strength Bs of the deblocking filter 230 based on Table 3 below.

[0100] [Table 3]

[0101] Figure 5 shows the configuration of the encoding device 1 related to this modification example.

[0102] As shown in Figure 5, in the encoding device 1, the filter control unit 161 receives the same predicted residual (reconstructed predicted residual) that is input to the synthesis unit 150. For blocks to which ACT is applied, the predicted residual after inverse ACT is input to the filter control unit 161. The filter control unit 161 controls the deblocking filter 160 to perform deblocking filtering if the reconstruction predicted residual of at least one of the block pairs (block P and block Q) contains a non-zero value. Specifically, the filter control unit 161 sets the boundary filter strength Bs=1.

[0103] Figure 6 shows the configuration of the decoding device 2 in this modified example.

[0104] As shown in Figure 6, in the decoding device 2, the filter control unit 231 receives the same predicted residual (reconstructed predicted residual) as the one input to the combining unit 220. For blocks to which ACT is applied, the predicted residual after inverse ACT is input to the filter control unit 231. The filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering if the reconstruction predicted residual of at least one of the block pairs (block P and block Q) contains a non-zero value. Specifically, the filter control unit 231 sets the boundary filter strength Bs=1.

[0105] [Second Embodiment] In the embodiments and modifications thereof described above, an example in which ACT, one of the coding tools, is applied was explained. However, similar problems may arise when another coding tool, such as Joint coding of chroma residual (JCCR), is applied. For this reason, the embodiments and modifications thereof described above may be applied to JCCR, and ACT in the embodiments and modifications thereof may be appropriately replaced with JCCR. For example, the boundary filter intensity Bs can be controlled as shown in Table 4 below.

[0106] [Table 4]

[0107] JCCR is a coefficient coding mode for chromatic difference components (see Non-Patent Document 1). In JCCR, the coding device 1 uses the correlation between the predicted residuals of the first chromatic difference component (Cb component) and the second chromatic difference component (Cr component) to generate a single congruent predicted residual from the predicted residuals of the first and second chromatic difference components. For example, the coding device 1 generates a congruent predicted residual by inverting the positive and negative signs of the predicted residual of the second chromatic difference component and combining it with the predicted residual of the first chromatic difference component. Then, the coding device 1 performs transformation, quantization, and entropy coding on the generated congruent predicted residual before transmission.

[0108] The decoding device 2 reconstructs the predicted residuals of the first chromatic difference component and the second chromatic difference component from the transmitted combined predicted residuals. By transmitting only one combined predicted residual for each of the two chromatic difference components in this way, coding efficiency is improved.

[0109] As described above, the deblocking filter apparatus according to this embodiment includes a deblocking filter (160, 230) that performs filtering on the boundary between a first reconstruction block (block P) and a second reconstruction block (block Q), and a filter control unit (161, 231) that controls the boundary filter intensity Bs of the deblocking filter (160, 230) based on whether or not at least one of the first reconstruction block (block P) and the second reconstruction block (block Q) is encoded using JCCR (Joint coding of chroma residual), which generates a single joint predicted residual from the predicted residuals of the Cb chroma difference component and the Cr chroma difference component, respectively.

[0110] The filter control units (161, 231) may control the boundary filter intensity Bs to perform filtering by the deblocking filters (160, 230) if at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) is encoded using JCCR.

[0111] The filter control units (161, 231) may control the boundary filter intensity Bs to perform filtering by the deblocking filters (160, 230) even if there are no non-zero conversion coefficients in both the first reconstructed block (block P) and the second reconstructed block (block Q), provided that at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) is encoded using JCCR.

[0112] In this embodiment, the entropy decoding unit 200 of the decoding device 2 may acquire a flag for each of the first reconstructed block (block P) and the second reconstructed block (block Q) indicating whether or not it is encoded using JCCR. The filter control unit 231 may control the boundary filter intensity Bs of the deblocking filter 230 based on the flag for each of the first reconstructed block (block P) and the second reconstructed block (block Q). For example, the filter control unit 231 may control the boundary filter intensity Bs to perform filtering by the deblocking filter 230 if the flag for at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) is "1".

[0113] In this embodiment, the entropy decoding unit 200 of the decoding device 2 may further acquire a tu_coded_flag for each of the first reconstruction block (block P) and the second reconstruction block (block Q) indicating whether or not it contains non-zero conversion coefficients. The filter control unit 231 may further control the boundary filter intensity Bs of the deblocking filter 230 based on the tu_coded_flag for each of the first reconstruction block (block P) and the second reconstruction block (block Q). For example, the filter control unit 231 may control the boundary filter intensity Bs to perform filtering by the deblocking filter 230 if the tu_coded_flag for at least one of the blocks of the first reconstruction block (block P) and the second reconstruction block (block Q) is "1".

[0114] For each block of the first chromatic difference component (Cb component) and the second chromatic difference component (Cr component) to which JCCR is applied, although the tu_coded_flag described above is set to "1", in reality, there is no conversion coefficient for one of the color components. For this reason, similar to the modification example 2 described above, it is preferable to use a determination of whether the reconstructed predicted residual contains a non-zero value, instead of determining whether a non-zero conversion coefficient exists for a given color component.

[0115] [Other embodiments] In the embodiments described above, an example of controlling the boundary filter intensity Bs as a control of the deblocking filters (160, 230) was mainly explained. However, control is not limited to the boundary filter intensity Bs; control of the filter length or switching between multiple filters may also be performed.

[0116] A program may be provided that causes a computer to perform each of the processes performed by the encoding device 1 described above. Similarly, a program may be provided that causes a computer to perform each of the processes performed by the decoding device 2. The program may be recorded on a computer-readable medium. Using a computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, but may be, for example, a CD-ROM or DVD-ROM.

[0117] The circuits that perform each process carried out by the encoding device 1 may be integrated, and the encoding device 1 may be configured as a semiconductor integrated circuit (chipset, SoC). The circuits that perform each process carried out by the decoding device 2 may be integrated, and the decoding device 2 may be configured as a semiconductor integrated circuit (chipset, SoC).

[0118] Although the embodiments have been described in detail above with reference to the drawings, the specific configuration is not limited to those described above, and various design changes can be made without departing from the gist of the invention.

[0119] This application claims priority to Japanese Patent Application No. 2020-101293 (filed June 10, 2020), the entirety of which is incorporated into this specification by reference.

Claims

1. A deblocking filter that performs filtering on the boundary between a first reconfiguration block and a second reconfiguration block adjacent to the first reconfiguration block, The first reconstruction block and the second reconstruction block include a filter control unit that controls the boundary filter intensity of the deblocking filter based on whether or not a color conversion process is applied, The color conversion process is a process that converts the predicted residuals of the first color space into the predicted residuals of a second color space that is different from the first color space. The filter control unit controls the boundary filter intensity to perform the filtering process if at least one of the first reconstruction block and the second reconstruction block is encoded using the color conversion process. The filter control unit controls the boundary filter intensity so that the filter is applied to a predetermined color component if the predicted residual obtained by the color conversion process includes a non-zero value for that predetermined color component. Deblocking filter device.

2. The filter control unit controls the boundary filter intensity to perform the filtering process even if there are no non-zero conversion coefficients in both the first and second reconstruction blocks, provided that at least one of the first and second reconstruction blocks is encoded using the color conversion process. The deblocking filter apparatus according to claim 1.

3. A deblocking filter device according to claim 1 or 2, The system includes an entropy decoding unit that obtains a flag for each of the first and second reconstruction blocks indicating whether or not the color conversion process has been used for encoding, The filter control unit controls the boundary filter intensity of the deblocking filter based on the flags for the first and second reconstruction blocks, respectively. Decoding device.

4. The filter control unit controls the boundary filter intensity to perform the filtering process if the flag in at least one of the first and second reconstruction blocks is "1". The decoding device according to claim 3.

5. The entropy decoding unit further obtains a tu_coded_flag for each of the first and second reconstruction blocks, indicating whether or not it contains non-zero conversion coefficients. The filter control unit further controls the boundary filter intensity of the deblocking filter based on the tu_coded_flag for each of the first and second reconstruction blocks. The decoding device according to claim 3 or 4.

6. The filter control unit controls the boundary filter strength to perform the filtering process when the tu_coded_flag of at least one of the first and second reconstruction blocks is "1". The decoding device according to claim 5.

7. A program characterized by causing a computer to function as a deblocking filter device according to claim 1 or 2.

Images