Deblocking filter device, decoding device, and program product

Abstract

The present application relates to a deblocking filter, a decoding device and a program product. A deblocking filter device of a first mode includes: a deblocking filter that performs a filtering process on a boundary of a first reconstructed block and a second reconstructed block adjacent to the first reconstructed block; and a filter control unit that controls a boundary filtering strength of the deblocking filter based on whether at least one of the first reconstructed block and the second reconstructed block is encoded using JCCR (Joint coding of chroma residual) that generates one joint prediction residual from prediction residuals of Cb color difference components and Cr color difference components respectively.

Description

[0001] This application is a divisional application of the invention patent application filed on June 8, 2021, with application number 202180005451.2. Technical Field

[0002] This invention relates to a deblocking filter, a decoding device, and a program. Background Technology

[0003] In HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding), which is the next-generation coding scheme, deblocking filtering is used as a loop filter to suppress distortion at the boundaries of blocks when encoding in blocks. In the control of deblocking filtering, the boundary filtering strength of the deblocking filter is controlled based on the presence of non-zero transform coefficients in at least one of two adjacent blocks. This is because, since the energy of the predicted residual is distributed throughout the block through the inverse transform of the non-zero transform coefficients, the possibility of discontinuities at the boundaries between two blocks is high.

[0004] Existing technical documents

[0005] Non-patent literature

[0006] Non-patent document 1: JVET-Q2001 “Versatile Video Coding (Draft 8)” Summary of the Invention

[0007] The first type of deblocking filter apparatus includes: a deblocking filter that performs filtering processing on the boundary between a first reconstructed block and a second reconstructed block adjacent to the first reconstructed block; and a filter control unit that controls the boundary filtering intensity of the deblocking filter based on whether at least one of the first reconstructed block and the second reconstructed block is encoded using JCCR (Joint Coding of Chroma Residual) which generates a joint prediction residual from the respective prediction residuals of the Cb chromatic difference component and the Cr chromatic difference component.

[0008] The second decoding apparatus includes: a first deblocking filter; and an entropy decoding unit that acquires a flag indicating whether the JCCR is used for encoding for each of the first reconstruction block and the second reconstruction block, and the filter control unit controls the boundary filtering strength of the deblocking filter based on the flag for each of the first reconstruction block and the second reconstruction block.

[0009] The third method of the program enables the computer to function as a deblocking filter device, which is the first method. Attached Figure Description

[0010] Figure 1 This is a diagram showing the configuration of the encoding device according to an embodiment.

[0011] Figure 2 This is a diagram illustrating the operation of the deblocking filter in the implementation method.

[0012] Figure 3 This is a diagram showing the configuration of the decoding device according to an embodiment.

[0013] Figure 4 This is a diagram showing an operational example of the filter control unit according to an embodiment.

[0014] Figure 5 This is a diagram showing the configuration of the encoding device in Modified Example 2.

[0015] Figure 6 This is a diagram showing the configuration of the decoding device in Modified Example 2. Detailed Implementation

[0016] In the VVC standard scheme, a technique called Adaptive Color Transform (ACT) is used: given an input image with a chroma format of 4:4:4, the color space of the prediction residual (RGB space) is transformed to the YCgCo space, and then the prediction residual after the color space transformation is processed. Entropy encoding and other encoding processes are described (see Non-Patent Document 1). The encoding device can control whether ACT is applied to each encoded object block, and streams ACT with a flag applied to each encoded object block. The decoding device performs entropy decoding on the blocks encoded using ACT. Inverse transformation processing is used to restore the prediction residual, and the color space (YCgCo space) of the restored prediction residual is inversely transformed into RGB space.

[0017] In the aforementioned conventional techniques, when at least one of two adjacent blocks has non-zero transform coefficients, the decoding device applies deblocking filtering to the boundary between the two blocks. On the other hand, when neither of the two adjacent blocks has non-zero transform coefficients, deblocking filtering may not be applied to the boundary between the two blocks.

[0018] However, for blocks using ACT, after restoring the prediction residual from the transform coefficients through inverse transform processing, the color space of the prediction residual is transformed from YCgCo space to RGB space through inverse color space transform. Therefore, if a block of a certain color component has non-zero transform coefficients, these non-zero transform coefficients will affect blocks of other color components during the inverse color space transform.

[0019] Therefore, if deblocking filter control based on the presence or absence of non-zero transform coefficients is applied to blocks encoded using ACT, there is a possibility that the boundary filtering strength of the deblocking filter cannot be properly controlled, leading to image quality degradation.

[0020] The same problem may occur when using Joint coding of chromaresidual (JCCR) as another coding tool.

[0021] Therefore, the object of the present invention is to provide a deblocking filter, a decoding device, and a program capable of suppressing image quality degradation.

[0022] The encoding and decoding apparatus of the embodiments will be described with reference to the accompanying drawings. The encoding and decoding apparatus of the embodiments encode and decode moving images represented by MPEG, respectively. In the following description of the drawings, the same or similar parts are given the same or similar reference numerals.

[0023] [First Implementation Method]

[0024] <Composition of the Encoding Device>

[0025] First, the configuration of the encoding device in this embodiment will be explained. Figure 1 This is a diagram showing the configuration of the encoding device 1 in this embodiment.

[0026] like Figure 1 As shown, the encoding device 1 includes a block segmentation unit 100, a residual generation unit 110, a switching unit 111, a color space transformation unit 112, a transform quantization unit 120, an entropy encoding unit 130, an inverse quantization and inverse transformation unit 140, a switching unit 143, a color space inverse transformation unit 144, a synthesis unit 150, a deblocking filter 160, a memory 170, and a prediction unit 180.

[0027] The block segmentation unit 100 segments the input image, i.e., the original image, which constitutes the frame (or image) unit of the moving image, into multiple image blocks, and outputs the segmented image blocks to the residual generation unit 110. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the image block is not limited to a square, but can also be rectangular (non-square). The image block is the unit for encoding processing by the encoding device 1 (i.e., the encoding target block), and is also the unit for decoding processing by the decoding device (i.e., the decoding target block). Such an image block is sometimes also called a CU (Coding Unit).

[0028] The input image can be an RGB signal, and the chroma format can be 4:4:4. RGB space is an example of a first color space. The "R" component corresponds to the first component, the "G" component corresponds to the second component, and the "B" component corresponds to the third component. The block segmentation unit 100 segments each of the R, G, and B components constituting the image into blocks, thereby outputting blocks. In the following description of the encoding apparatus, the blocks are referred to as encoding target blocks without distinguishing between the individual color components.

[0029] The residual generation unit 110 calculates the prediction residual, which represents the difference (error) between the encoded target block output by the block segmentation unit 100 and the prediction block obtained by the prediction unit 180 predicting the encoded target block. Specifically, the residual generation unit 110 calculates the prediction residual by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the encoded target block, and outputs the calculated prediction residual to the switching unit 111. In this embodiment, the residual generation unit 110 generates the prediction residual for each color component based on the difference between the encoded target block of each color component and the prediction block of each color component.

[0030] The switching unit 111 outputs the predicted residuals of each color component output by the residual generation unit 110 to either the transform quantization unit 120 or the color space transformation unit 112. The switching unit 111 outputs the predicted residuals to the transform quantization unit 120 without performing color space transformation processing (ACT processing), and outputs the predicted residuals to the color space transformation unit 112 with ACT processing.

[0031] The color space transformation unit 112 performs ACT processing on the prediction residuals of each color component and outputs the ACT-processed prediction residuals to the transform quantization unit 120. The color space transformation unit 112 generates new prediction residuals by performing the following transformation calculations on the R, G, and B components of the prediction residuals for the encoded object block.

[0032] Co = R - B

[0033] t = B + (Co >> 1)

[0034] Cg = G - t

[0035] Y = t + (Cg >> 1)

[0036] Here, ">>" indicates a right shift operation. Furthermore, the "Y" component corresponds to the first component, the "Cg" component to the second component, and the "Co" component to the third component. This YCgCo color space is an example of a second color space.

[0037] The switching unit 111 and the color space conversion unit 112 can control whether color conversion processing is applied to each encoded object block. The entropy encoding unit 130 sends a signal in the bit stream indicating whether color conversion processing (ACT application flag) has been performed on the encoded block.

[0038] Furthermore, the ACT processing of the color space transformation unit 112 only requires addition for each color component. Subtraction multiplication division The shifting process and other methods can generate a prediction residual composed of new color components. In addition, the ACT process does not need to be a transformation that affects all color components. For example, the color space transformation unit 112 can apply an ACT process that does not change and maintains the first component, takes the average of the second and third components as the new second component, and takes the difference between the second and third components as the new third component.

[0039] The transformation and quantization unit 120 performs transformation and quantization processing on a block-by-block basis. The transformation and quantization unit 120 includes a transformation unit 121 and a quantization unit 122.

[0040] The transform unit 121 performs transform processing on the prediction residuals (referred to as prediction residuals regardless of whether ACT processing is applied) output by the switching unit 111 or the color space transform unit 112, calculates transform coefficients, and the vectorization unit 122 outputs the calculated transform coefficients. Specifically, the transform unit 121 generates transform coefficients for each color component by performing transform processing on the prediction residuals of each color component in block units. The transform processing can be, for example, frequency transforms such as DCT, DST, or discrete wavelet transform. In addition, the transform unit 121 outputs information related to the transform processing to the entropy coding unit 130.

[0041] The transformation processing includes transform skipping, which is used in the HEVC and VVC standard schemes, without transformation processing. In the HEVC transform skipping mode, the prediction residual is scaled without horizontal and vertical transformation processing to obtain the transformation coefficients. However, the transform skipping in this embodiment also includes transformations that apply only horizontal transformation processing and transformations that apply only vertical transformation processing. In addition, the transformation unit 121 can also perform a secondary transformation processing that further applies transformation processing to the transformation coefficients obtained through the transformation processing. Furthermore, the secondary transformation processing can be applied only to a portion of the transformation coefficients.

[0042] The quantization unit 122 quantizes the transform coefficients output from the transform unit 121 using quantization parameters and a scaling list, and outputs the quantized transform coefficients to the entropy encoding unit 130 and the inverse quantization and inverse transform unit 140. In addition, the quantization unit 122 outputs information related to the quantization process (specifically, information about the quantization parameters and scaling list used in the quantization process) to the entropy encoding unit 130 and the inverse quantization unit 141.

[0043] The entropy coding unit 130 performs entropy coding on the quantization transform coefficients output by the quantization unit 122, compresses the data, generates a bitstream (coded data), and outputs the bitstream to the decoding side. Entropy coding can use Huffman coding, CABAC (Context-Based Adaptive Binary Arithmetic Coding), etc. Alternatively, the entropy coding unit 130 can include information related to the transform processing input from the transform unit 121 in the bitstream and send it to the decoding side as a signal, or include information related to the prediction processing input from the prediction unit 180 in the bitstream and send it to the decoding side as a signal.

[0044] Furthermore, the entropy coding unit 130 includes a color space transformation flag indicating whether or not ACT is applied in the bitstream for each encoded object block and sends it to the decoding side as a signal. This color space transformation flag is also called the ACT application flag. When the ACT application flag is on ("1"), it indicates that ACT is applied to the corresponding encoded object block. When the ACT application flag is off ("0"), it indicates that ACT is not applied to the corresponding encoded object block. Alternatively, an ACT non-application flag can be used instead of the ACT application flag. In this case, when the ACT non-application flag is on ("1"), it indicates that ACT is not applied to the corresponding encoded object block. When the ACT non-application flag is off ("0"), it indicates that ACT is applied to the corresponding encoded object block.

[0045] The inverse quantization and inverse transformation unit 140 performs inverse quantization and inverse transformation processing on a block-by-block basis. The inverse quantization and inverse transformation unit 140 includes an inverse quantization unit 141 and an inverse transformation unit 142.

[0046] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 performs inverse quantization on the quantization transform coefficients output by the quantization unit 122 using quantization parameters (Qp) and a scaling list to restore the transform coefficients, and outputs the restored transform coefficients to the inverse transform unit 142.

[0047] The inverse transform unit 142 performs the inverse transform processing corresponding to the transform processing performed by the transform unit 121. For example, if the transform unit 121 performs a discrete cosine transform, the inverse transform unit 142 performs an inverse discrete cosine transform. The inverse transform unit 142 performs the inverse transform processing on the transform coefficients output by the inverse quantization unit 141, restores the prediction residual, and outputs the restored prediction residual, i.e., the restored prediction residual, to the switching unit 143.

[0048] The switching unit 143 outputs the restoration prediction residuals of each color component output by the inverse transformation unit 142 to either the compositing unit 150 or the color space inverse transformation unit 144. For blocks where ACT is not applied, the switching unit 143 outputs the restoration prediction residuals to the compositing unit 150, and for blocks where ACT is applied, it outputs the restoration prediction residuals to the color space inverse transformation unit 144.

[0049] The color space inverse transformation unit 144 performs the inverse processing of the ACT processing performed by the color space transformation unit 112, namely the color space inverse transformation processing (inverse ACT processing), and outputs the prediction residual after inverse ACT processing to the synthesis unit 150. Specifically, using the Y component, Cg component, and Co component of the restored prediction residual, an inverse transformation calculation is performed as follows, thereby performing an inverse transformation from YCgCo space to RGB space.

[0050] t = Y - (Cg >> 1)

[0051] G = Cg + t

[0052] B = t - (Co >> 1)

[0053] R = Co + B

[0054] The compositing unit 150 combines the restored prediction residual output from the inverse transform unit 142 or the color space inverse transform unit 144 with the prediction block output from the prediction unit 180 on a pixel-by-pixel basis. The compositing unit 150 adds each pixel value of the restored prediction residual to each pixel value of the prediction block to restore (reconstruct) the encoded object block, and outputs the restored block to the deblocking filter 160. The restored block is sometimes also referred to as a reconstructed block.

[0055] The deblocking filter 160 filters the restored block output from the synthesis unit 150 and outputs the filtered restored block to the memory 170. The filter control unit 161 controls the deblocking filter 160. Details of the deblocking filter 160 and the filter control unit 161 will be described later.

[0056] The memory 170 stores the restored blocks after filtering output from the deblocking filter 160, storing the restored blocks as restored images in frame units. The memory 170 outputs the stored restored blocks or restored images to the prediction unit 180.

[0057] The prediction unit 180 performs prediction processing on a block-by-block basis. The prediction unit 180 generates a prediction block for each color component by performing intra-frame prediction and inter-frame prediction for the coded target block. The prediction unit 180 includes an inter-frame prediction unit 181, an intra-frame prediction unit 182, and a switching unit 183.

[0058] The inter-frame prediction unit 181 performs inter-frame prediction utilizing inter-frame correlation. Specifically, the inter-frame prediction unit 181 uses the restored image stored in the memory 170 as a reference image, calculates motion vectors using methods such as block matching, predicts the target block to be encoded, generates an inter-frame prediction block, and outputs the generated inter-frame prediction block to the switching unit 183. Here, the inter-frame prediction unit 181 selects the most suitable inter-frame prediction method from inter-frame prediction using multiple reference images (typically, dual prediction) and inter-frame prediction using a single reference image (unidirectional prediction), and performs inter-frame prediction using the selected inter-frame prediction method. The inter-frame prediction unit 181 outputs information related to inter-frame prediction (motion vectors, etc.) to the entropy coding unit 130.

[0059] The intra-prediction unit 182 performs intra-prediction utilizing intra-frame spatial correlation. Specifically, the intra-prediction unit 182 generates an intra-prediction block by referring to restored pixels surrounding the coded target block in the restored image stored in the memory 170, and outputs the generated intra-prediction block to the switching unit 183. The intra-prediction unit 182 selects an intra-prediction mode to apply to the coded target block from a plurality of intra-prediction modes, and uses the selected intra-prediction mode to predict the coded target block.

[0060] The switching unit 183 switches between the inter-frame prediction block output by the inter-frame prediction unit 181 and the intra-frame prediction block output by the intra-frame prediction unit 182, and outputs any prediction block to the residual generation unit 110 and the synthesis unit 150.

[0061] Next, the deblocking filter 160 and the filter control unit 161 of this embodiment will be described.

[0062] The deblocking filter 160 performs filtering on the block boundary between two blocks consisting of a restored block (first block) and an adjacent restored block (second block), and outputs the filtered restored blocks to the memory 170. The filtering process is used to mitigate signal degradation caused by block unit processing, and it smooths the signal gap at the block boundary between two adjacent blocks.

[0063] The filter control unit 161 controls the deblocking filter 160. Specifically, the filter control unit 161 controls the boundary filter strength (Bs), which indicates whether filtering is applied to the block boundaries of block pairs, and the filtering strength of the deblocking filter 160. The boundary filter strength Bs is a parameter used to determine whether filtering is applied and the type of filtering. Furthermore, the control of whether filtering is applied can be considered as controlling whether the boundary filter strength Bs is 1 or higher, or whether the boundary filter strength Bs is zero.

[0064] Figure 2 This is a diagram illustrating the operation of the deblocking filter 160 in this embodiment. Figure 2 In the example shown, the deblocking filter 160 filters the block boundaries of each 8×8 pixel block as the object. Additionally, the deblocking filter 160 can filter in units of 4 rows or 4 columns. Figure 2 The blocks P (first reconstructed block) and Q (second reconstructed block) shown are units of filtering by the deblocking filter 160, illustrating an example with a block size of 4×4 pixels. Blocks P and Q can also be referred to as sub-blocks. Block Q is the reconstructed block corresponding to the encoded object block, and block P is the reconstructed block adjacent to block Q.

[0065] The filter control unit 161 determines the boundary filter strength Bs according to Table 1 below. In this embodiment, the value of the boundary filter strength Bs is any one of 0, 1, and 2.

[0066] [Table 1]

[0067] like Figure 2 As shown in Table 1, when intra-frame prediction is applied to at least one of block P and block Q, the filter control unit 161 sets the Bs value to 2.

[0068] If motion compensation prediction (inter-frame prediction) is applied to both blocks P and Q and at least one of the following conditions (a) to (d) is met, the filter control unit 161 sets the Bs value to 1; otherwise, it sets the Bs value to 0.

[0069] (a) The absolute value of the difference between the motion vectors of block P and block Q is above a threshold (e.g., 1 pixel).

[0070] (b) The motion vectors of blocks P and Q are different in value or reference image.

[0071] (c) At least one of blocks P and Q contains meaningful transformation coefficients (i.e., non-zero transformation coefficients).

[0072] (d) ACT was applied to at least one of block P and block Q.

[0073] When the boundary filter strength Bs is 0, the filter control unit 161 controls the deblocking filter 160 not to perform deblocking filtering. In the following text, ... Figure 2 The vertical block boundary shown is used as an example for illustration.

[0074] When the value of the boundary filter intensity Bs is 1 or 2, and the following equation (1) is satisfied, the filter control unit 161 can control the deblocking filter 160 to perform deblocking filtering.

[0075] [Mathematical Expression 1]

[0076] In addition, when performing deblocking filtering, the filter control unit 161 applies strong filtering if all of the following conditional expressions (2) to (7) are satisfied, and applies weak filtering otherwise.

[0077] [Mathematical Expression 2]

[0078] Wherein, the threshold β and t C The value is based on the average of the quantization parameters of adjacent blocks P and Q. av change.

[0079] <Composition of the Decoding Device>

[0080] Next, the decoding device of this embodiment will be described, focusing on its differences from the encoding device 1. Figure 3 This is a diagram showing the configuration of the decoding device 2 in this embodiment.

[0081] like Figure 3 As shown, the decoding device 2 includes an entropy decoding unit 200, an inverse quantization and inverse transformation unit 210, a switching unit 215, a color space inverse transformation unit 216, a synthesis unit 220, a deblocking filter 230, a memory 240, and a prediction unit 250.

[0082] The entropy decoding unit 200 decodes the encoded data (bitstream), obtains the quantization transform coefficients corresponding to the decoded target block, and outputs the obtained quantization transform coefficients to the inverse quantization and inverse transform unit 210. Additionally, the entropy decoding unit 200 obtains information related to transform processing and quantization processing, and outputs this information to the inverse quantization and inverse transform unit 210. Furthermore, the entropy decoding unit 200 obtains information related to prediction processing, and outputs this information to the prediction unit 250. The entropy decoding unit 200 obtains the color space transformation mark for each decoded target block, and outputs the obtained color space transformation mark to the switching unit 215 and the filter control unit 231.

[0083] The inverse quantization and inverse transformation unit 210 performs inverse quantization and inverse transformation processing on a block-by-block basis. The inverse quantization and inverse transformation unit 210 includes an inverse quantization unit 211 and an inverse transformation unit 212.

[0084] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 performs inverse quantization on the quantization transform coefficients output by the entropy decoding unit 200 using quantization parameters (Qp) and a scaling list to recover the transform coefficients of the decoded object block, and outputs the recovered transform coefficients to the inverse transform unit 212.

[0085] The inverse transform unit 212 performs an inverse transform process corresponding to the transform process performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 211, restores the prediction residual, and outputs the restored prediction residual to the switching unit 215.

[0086] The switching unit 215 outputs the prediction residuals of each color component output by the inverse color transform unit 212 to either the compositing unit 220 or the inverse color transform unit 216 based on the color space transformation marker. For blocks that have not undergone inverse color space transformation (ACT), the switching unit 215 outputs the prediction residuals to the compositing unit 220, and for blocks that have undergone ACT, it outputs the prediction residuals to the inverse color transform unit 216.

[0087] The color space inverse transformation unit 216 performs the inverse processing of the ACT processing performed by the color space transformation unit 112 of the encoding device 1, namely, the color space inverse transformation processing (inverse ACT processing), and outputs the prediction residual after inverse ACT processing to the synthesis unit 220. Specifically, the color space inverse transformation unit 216 uses the Y component, Cg component, and Co component of the restored prediction residual to perform the following inverse transformation calculation.

[0088] t = Y - (Cg >> 1)

[0089] G = Cg + t

[0090] B = t - (Co >> 1)

[0091] R = Co + B

[0092] The compositing unit 220 decodes (reconstructs) the original block by combining the prediction residual output by the switching unit 215 or the color space inverse transformation unit 216 with the prediction block output by the prediction unit 250 in pixel units, and outputs the restored block to the deblocking filter 230.

[0093] The deblocking filter 230 filters the restored blocks output by the synthesis unit 220 and outputs the filtered restored blocks to the memory 240. Specifically, the deblocking filter 230 filters the block boundary between two blocks formed by the restored block (first block) and the adjacent restored block (second block), and outputs the filtered restored blocks to the memory 240. The function of the deblocking filter 230 is the same as that of the deblocking filter 160 of the encoding device 1.

[0094] The filter control unit 231 controls the deblocking filter 230. Specifically, the filter control unit 231 controls the boundary filter strength (Bs), which indicates whether the block boundaries of block pairs are filtered, and the filtering strength of the deblocking filter 230. The function of the filter control unit 231 is the same as that of the filter control unit 161 of the encoding device 1. The filter control unit 231 determines the boundary filter strength Bs according to Table 1 above.

[0095] That is, the filter control unit 231 of this embodiment controls the boundary filtering intensity Bs of the deblocking filter 230 based on whether at least one of the adjacent blocks P and Q is encoded using adaptive color transformation (ACT).

[0096] As described above, the entropy decoding unit 200 obtains a flag (color space transformation flag) indicating whether the representation has been encoded using adaptive color transformation for each of blocks P and Q. The filter control unit 231 controls the boundary filtering intensity Bs of the deblocking filter 230 based on the color space transformation flag for each of blocks P and Q.

[0097] When at least one of blocks P and Q is encoded using adaptive color transformation, the filter control unit 231 controls the boundary filter strength Bs to cause the deblocking filter 230 to perform filtering (i.e., the boundary filter strength Bs is set to 1). Specifically, even when neither block P nor block Q has non-zero transform coefficients, when at least one of blocks P and Q is encoded using adaptive color transformation, the filter control unit 231 controls the boundary filter strength Bs to cause the deblocking filter 230 to perform filtering.

[0098] The memory 240 stores the filtered restored blocks output by the filter control unit 231, and stores the filtered restored blocks as restored images on a frame-by-frame basis. The memory 240 outputs the filtered restored blocks or restored images to the prediction unit 250. In addition, the memory 240 outputs the filtered restored images on a frame-by-frame basis to the external decoding device 2.

[0099] The prediction unit 250 performs prediction on a block-by-block basis for each color component. The prediction unit 250 includes an inter-frame prediction unit 251, an intra-frame prediction unit 252, and a switching unit 253.

[0100] The inter-frame prediction unit 251 performs inter-frame prediction using inter-frame correlation. Specifically, based on information related to inter-frame prediction (e.g., motion vector information) output by the entropy decoding unit 200, the inter-frame prediction unit 251 uses the restored image stored in the memory 240 as a reference image to predict the encoded target block, generates an inter-frame prediction block, and outputs the generated inter-frame prediction block to the switching unit 253.

[0101] The intra-prediction unit 252 performs intra-prediction that utilizes intra-frame spatial correlation. Specifically, the intra-prediction unit 252 uses an intra-prediction mode corresponding to intra-prediction-related information (such as intra-prediction mode information) output by the entropy decoding unit 200, refers to restored pixels located around the coded target block in the restored image stored in the memory 240, generates an intra-prediction block, and outputs the generated intra-prediction block to the switching unit 253.

[0102] The switching unit 253 switches between the inter-frame prediction block output by the inter-frame prediction unit 251 and the intra-frame prediction block output by the intra-frame prediction unit 252, and outputs any prediction block to the synthesis unit 220.

[0103] Thus, the decoding apparatus 2 of this embodiment includes: an entropy decoding unit 200, which decodes the encoded stream and outputs transform coefficients corresponding to block P; an inverse quantization and inverse transform unit 210, which performs inverse quantization and inverse transform processing on the transform coefficients output by the entropy decoding unit 200 to restore the prediction residual corresponding to block P (the first block); a synthesis unit 220, which restores block P by synthesizing the restored prediction residual with a prediction block obtained by predicting block P; a deblocking filter 230, which performs filtering processing on the boundary of the restored block P and the restored block Q (the second block) adjacent to block P; and a filter control unit 231, which controls the boundary filtering intensity Bs of the deblocking filter 230 based on whether at least one of block P and block Q uses adaptive color transformation (ACT) for encoding.

[0104] For blocks that have applied ACT, after restoring the prediction residuals from the transform coefficients through inverse transform processing, the color space of the prediction residuals is inversely transformed from YCgCo space to RGB space through inverse color space transform (inverse ACT). Therefore, when a block of a certain color component has non-zero transform coefficients, these non-zero transform coefficients will affect blocks of other color components during the inverse color space transform.

[0105] Therefore, the filter control unit 231 of this embodiment controls the boundary filtering intensity Bs of the deblocking filter 230 not only based on the presence or absence of non-zero transform coefficients, but also considers whether or not ACT is applied to control the boundary filtering intensity Bs of the deblocking filter 230. As a result, the boundary filtering intensity Bs of the deblocking filter 230 can be appropriately controlled, and thus image quality degradation can be suppressed even when ACT is applied.

[0106] <Operation of the Filter Control Unit>

[0107] Next, the operation of the filter control unit 161 and the filter control unit 231 in this embodiment will be described. Since the filter control unit 161 on the encoding side and the filter control unit 231 on the decoding side perform the same operation, the filter control unit 231 on the decoding side will be used as an example for explanation. Figure 4 This is a diagram illustrating an operational example of the filter control unit 231 of this embodiment. Additionally, Figure 4 The order of judgment shown is an example, and the order of judgment can be changed.

[0108] like Figure 4 As shown, in step S1, the filter control unit 231 determines whether intra-frame prediction has been applied to at least one of the block pair consisting of block P and block Q. If intra-frame prediction has been applied to at least one of the block pair (step S1: Yes), in step S2, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filtering strength Bs = 2.

[0109] If no intra-frame prediction is applied to either side of the block pair (step S1: No), in step S3, the filter control unit 231 determines whether the difference in motion vectors of the target block pair is above a threshold. If the difference in motion vectors of the target block pair is above the threshold (step S3: Yes), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filtering strength Bs = 1.

[0110] If the difference between the motion vectors of the block pair is not above a threshold (step S3: No), in step S5, the filter control unit 231 determines whether the values ​​of the motion vectors of the block pair or the reference image are different. If the values ​​of the motion vectors of the block pair or the reference image are different (step S5: Yes), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering processing. Specifically, the filter control unit 231 sets the boundary filtering intensity Bs = 1.

[0111] If the values ​​of the motion vectors of the block pair or the reference image are the same (step S5: No), in step S6, the filter control unit 231 determines whether at least one block of the block pair contains non-zero transform coefficients. If at least one of the block pairs contains non-zero transform coefficients (step S6: Yes), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

[0112] If neither block in the block pair contains non-zero transform coefficients (step S6: No), in step S7, the filter control unit 231 determines whether at least one block in the block pair has been encoded using ACT based on the color space transformation markers of each block output by the entropy decoding unit 200. If at least one block in the block pair has been encoded using ACT (step S7: Yes), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1. On the other hand, if neither block in the block pair has been encoded using ACT (step S7: No), in step S8, the filter control unit 231 controls the deblocking filter 230 not to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 0.

[0113] <Variation Example 1>

[0114] The filter control unit 231 on the decoding side can also determine whether at least one block of the block pair (block P and block Q) contains non-zero transform coefficients based on the markers sent by the encoding device 1 using signals.

[0115] Specifically, the entropy coding unit 130 of the coding device 1 includes a flag (tu_coded_flag) in the coding stream for each block, indicating whether it contains non-zero transform coefficients. For example, the entropy coding unit 130 sets the flag (tu_coded_flag) to "1" for blocks containing non-zero transform coefficients and sets the flag (tu_coded_flag) to "0" for blocks that do not contain non-zero transform coefficients.

[0116] The entropy decoding unit 200 of the decoding device 2 acquires a flag (tu_coded_flag) for each block and outputs the acquired flag (tu_coded_flag) to the filter control unit 231. The filter control unit 231 interprets a block with a flag (tu_coded_flag) of "1" as containing non-zero transform coefficients. Furthermore, the filter control unit 231 sets the boundary filtering strength Bs of the deblocking filter 230 as shown in Table 2 below.

[0117] [Table 2]

[0118] As shown in Table 2, when the tu_coded_flag of at least one block in the block pair is "1", the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filtering strength Bs = 1.

[0119] <Variation Example 2>

[0120] As described above, for blocks that have applied ACT, after restoring the prediction residuals from the transform coefficients through inverse transform processing, the color space of the prediction residuals is inversely transformed from YCgCo space to RGB space through inverse ACT. Therefore, when a block of a certain color component has non-zero transform coefficients, these non-zero transform coefficients will affect blocks of other color components during inverse ACT.

[0121] Therefore, when applying ACT, controlling the boundary filtering strength Bs of the deblocking filter 230 based on the presence or absence of non-zero transform coefficients is ineffective. Thus, in this modified example, the boundary filtering strength Bs of the deblocking filter 230 is controlled based on the prediction residual after inverse transform processing, rather than on the transform coefficients. Consequently, even if non-zero transform coefficients affect blocks of other color components during inverse ACT, the boundary filtering strength Bs of the deblocking filter 230 can be appropriately controlled based on the prediction residual affected by this effect.

[0122] That is, the filter control unit 161 and the filter control unit 231 of this modified example set the boundary filtering intensity Bs of the deblocking filter 230 based on Table 3 below.

[0123] [Table 3]

[0124] Figure 5 This is a diagram showing the configuration of the encoding device 1 in this modified example.

[0125] like Figure 5As shown, in the encoding device 1, the same prediction residual (restored prediction residual) as the prediction residual input to the synthesis unit 150 is input to the filter control unit 161. For blocks applying ACT, the prediction residual after inverse ACT is input to the filter control unit 161. If the restored prediction residual of at least one block in the block pair (block P and block Q) contains a non-zero value, the filter control unit 161 controls the deblocking filter 160 to perform deblocking filtering. Specifically, the filter control unit 161 sets the boundary filter strength Bs = 1.

[0126] Figure 6 This is a diagram showing the configuration of the decoding device 2 in this modified example.

[0127] like Figure 6 As shown, in the decoding device 2, the same prediction residual (reconstructed prediction residual) as the prediction residual input to the synthesis unit 220 is input to the filter control unit 231. For blocks applying ACT, the prediction residual after inverse ACT is input to the filter control unit 231. If the reconstructed prediction residual of at least one block in the block pair (block P and block Q) contains a non-zero value, the filter control unit 231 controls the deblocking filter 230 to perform deblocking filtering. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

[0128] [Second Implementation]

[0129] In the above embodiments and variations, an example of applying ACT as one of the coding tools has been described. However, the same problems may occur when applying Joint Coding of Chroma Residual (JCCR) as another coding tool. Therefore, the above embodiments and variations can be applied to JCCR, and ACT in the above embodiments and variations can be appropriately replaced with JCCR. For example, the boundary filter strength Bs can be controlled as shown in Table 4 below.

[0130] [Table 4]

[0131] JCCR is a coefficient encoding mode for color difference components (see Non-Patent Document 1). In JCCR, the encoding device 1 utilizes the correlation between the prediction residuals of the first color difference component (Cb component) and the second color difference component (Cr component) to generate a joint prediction residual from the prediction residuals of the first color difference component and the prediction residual of the second color difference component. For example, the encoding device 1 combines the prediction residual of the second color difference component (after reversing its positive and negative values) with the prediction residual of the first color difference component to generate a joint prediction residual. Furthermore, the encoding device 1 performs transformation processing, quantization processing, and entropy encoding processing on the generated joint prediction residual before transmitting it.

[0132] Decoding device 2 reconstructs the prediction residuals of the first color difference component and the second color difference component based on the transmitted joint prediction residual. In this way, by transmitting only one joint prediction residual for the two color difference components, coding efficiency can be improved.

[0133] Thus, the deblocking filter apparatus of this embodiment includes: a deblocking filter (160, 230) that performs filtering processing on the boundary between the first reconstructed block (block P) and the second reconstructed block (block Q); and a filter control unit (161, 231) that controls the boundary filtering intensity Bs of the deblocking filter (160, 230) based on whether at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) uses JCCR (Joint coding of chroma residual) to generate a joint prediction residual from the prediction residuals of the Cb chroma component and the Cr chroma component, respectively.

[0134] When at least one of the first reconstruction block (block P) and the second reconstruction block (block Q) is encoded using JCCR, the filter control unit (161, 231) can control the boundary filter intensity Bs to enable the deblocking filter (160, 230) to perform filtering.

[0135] Even if there are no non-zero transform coefficients in either the first reconstruction block (block P) or the second reconstruction block (block Q), and at least one of the first reconstruction block (block P) or the second reconstruction block (block Q) is encoded using JCCR, the filter control unit (161, 231) can control the boundary filter intensity Bs to enable the deblocking filter (160, 230) to perform filtering.

[0136] In this embodiment, the entropy decoding unit 200 of the decoding device 2 can obtain a flag indicating whether JCCR encoding has been performed for each of the first reconstruction block (block P) and the second reconstruction block (block Q). The filter control unit 231 can control the boundary filtering intensity Bs of the deblocking filter 230 based on the flag for each of the first reconstruction block (block P) and the second reconstruction block (block Q). For example, if the flag of at least one block in the first reconstruction block (block P) and the second reconstruction block (block Q) is "1", the filter control unit 231 can control the boundary filtering intensity Bs to cause the deblocking filter 230 to perform filtering processing.

[0137] In this embodiment, the entropy decoding unit 200 of the decoding device 2 can also obtain a tu_coded_flag indicating whether non-zero transform coefficients are included for each of the first reconstruction block (block P) and the second reconstruction block (block Q). The filter control unit 231 can also control the boundary filtering intensity Bs of the deblocking filter 230 based on the tu_coded_flag for each of the first reconstruction block (block P) and the second reconstruction block (block Q). For example, when the tu_coded_flag of at least one block of the first reconstruction block (block P) and the second reconstruction block (block Q) is "1", the filter control unit 231 can control the boundary filtering intensity Bs to cause the deblocking filter 230 to perform filtering processing.

[0138] For each block applying the first color difference component (Cb component) and the second color difference component (Cr component) of JCCR, although the tu_coded_flag mentioned above is set to "1", one of the color components actually does not have transformation coefficients. Therefore, similar to the above variation 2, it is preferable that, for a certain color component, the determination of whether there are non-zero transformation coefficients is changed to the determination of whether the reconstructed prediction residual contains non-zero values.

[0139] [Other Implementation Methods]

[0140] In the above embodiments, the description mainly focuses on controlling the boundary filter strength Bs as an example of controlling the deblocking filters (160, 230). However, it is not limited to controlling the boundary filter strength Bs; it can also be used to control the filter length or switch between multiple filters.

[0141] A program can be provided that enables a computer to perform the processes performed by the encoding device 1 described above. Additionally, a program can be provided that enables a computer to perform the processes performed by the decoding device 2. The program can be stored on a computer-readable medium. If a computer-readable medium is used, the program can be installed on the computer. Here, the computer-readable medium storing the program can be a non-transitory storage medium. There are no particular limitations on the non-transitory storage medium; for example, it can be a CD-ROM, DVD-ROM, or other storage media.

[0142] Alternatively, the circuitry for each process performed by the encoding device 1 can be integrated, and the encoding device 1 can be constructed using a semiconductor integrated circuit (chipset, SoC). Similarly, the circuitry for each process performed by the decoding device 2 can be integrated, and the decoding device 2 can be constructed using a semiconductor integrated circuit (chipset, SoC).

[0143] The embodiments have been described in detail above with reference to the accompanying drawings. However, the specific configuration is not limited to the above configuration, and various design variations can be made without departing from the main idea.

[0144] This application claims priority to Japanese Patent Application No. 2020-101293 (filed on June 10, 2020), the entire contents of which are incorporated herein by reference.

Claims

1. A deblocking filter, wherein, The deblocking filter device includes: A deblocking filter filters the boundary between the first reconstructed block and the second reconstructed block adjacent to the first reconstructed block; and The filter control unit controls the boundary filtering intensity of the deblocking filter based on whether color transformation processing is applied to at least one of the first reconstruction block and the second reconstruction block. The color transformation process includes: generating and updating the processing corresponding to one or more color components of the prediction residual through any one of the following processing methods: addition, subtraction, multiplication, division, and shifting, targeting one or more color components of the prediction residual. When the filter control unit uses the color transformation process for encoding in at least one of the first reconstruction block and the second reconstruction block, it controls the boundary filter intensity to perform the filtering process. The filter control unit controls the boundary filter strength to perform the filtering process on the predetermined color component when the prediction residual updated by the color transformation process contains a non-zero value.

2. The deblocking filter device according to claim 1, wherein, Even when there are no non-zero transform coefficients in either the first reconstruction block or the second reconstruction block, the filter control unit still controls the boundary filter intensity to perform the filtering process when the color transformation process is applied to at least one of the first reconstruction block and the second reconstruction block.

3. A decoding device, wherein, The decoding device includes: The deblocking filter device according to claim 1 or 2; and The entropy decoding unit acquires a flag indicating whether the color transformation processing has been applied for each of the first and second reconstruction blocks. The filter control unit controls the boundary filtering strength of the deblocking filter based on the flags for each of the first reconstruction block and the second reconstruction block.

4. The decoding device according to claim 3, wherein, When the flag of at least one of the first reconstruction block and the second reconstruction block is "1", the filter control unit controls the boundary filter intensity to perform the filtering process.

5. The decoding apparatus according to claim 3 or 4, wherein, The entropy decoding unit also obtains a tu_coded_flag indicating whether each of the first and second reconstruction blocks contains non-zero transform coefficients for each of them. The filter control unit further controls the boundary filtering strength of the deblocking filter based on the tu_coded_flag for each of the first reconstruction block and the second reconstruction block.

6. A program product, characterized in that, Includes a program that, when executed by a computer, causes the computer to function as the deblocking filter device as described in claim 1 or 2.

Images