Image encoding apparatus and method, image decoding apparatus and method, and storage medium

Abstract

The present application provides an image encoding apparatus and method, an image decoding apparatus and method, and a storage medium. In a prediction process, one of a first mode, a second mode, and a third mode can be used, the first mode is used to derive prediction pixels of a block using pixels in an image including the block, the second mode is used to derive prediction pixels of the block using pixels in an image different from the image including the block, and the third mode is used to generate prediction pixels of the block using both the pixels in the image including the block and the pixels in the image different from the image including the block. In a case where the third mode is used in at least one of a first block and a second block, an intensity of a deblocking filter process to be performed with respect to a boundary between the first block and the second block is set to an intensity same as in a case where the first mode is used in at least one of the first block and the second block.

Description

[0001] (This application is a divisional application of the application filed on November 11, 2019, with application number 201980083160.8 and titled "Image Encoding Device, Image Encoding Method, Image Decoding Device and Image Decoding Method".) Technical Field

[0002] This invention relates to an image encoding technology and an image decoding technology. Background Technology

[0003] As coding methods for compressed recording of moving images, H.264 / AVC (hereinafter referred to as H.264) and HEVC (High-Efficiency Video Coding) (hereinafter referred to as HEVC) coding methods are known. In HEVC, to improve coding efficiency, basic blocks with a larger size than traditional macroblocks (16 pixels × 16 pixels) are used. These large basic blocks are called CTUs (Code Tree Units), and the maximum size of a basic block is 64 pixels × 64 pixels. CTUs are further divided into sub-blocks, each of which is a unit used for prediction or transformation.

[0004] Furthermore, in HEVC, an adaptive deblocking filter is applied to the block boundaries of the reconstructed image obtained by adding the signal processed by inverse quantization / inverse transformation to the predicted image, thereby suppressing visually obvious block distortion and preventing image quality degradation from propagating to the predicted image. Patent Document 1 discloses a technique related to such deblocking filtering.

[0005] In recent years, as the successor to HEVC, efforts have been made to standardize international coding methods for higher efficiency. The Joint Video Experts Group (JVET) was established between ISO / IEC and ITU-T, promoting the standardization of the VVC (Multi-Functional Video Coding) coding method (hereinafter referred to as VVC). To improve coding efficiency, in addition to traditional intra-frame prediction and inter-frame prediction, new prediction methods using both intra-frame and inter-frame prediction pixels (hereinafter referred to as weighted intra / inter-frame prediction) have been investigated.

[0006] Citation List

[0007] Patent documents

[0008] Patent Document 1: Japanese Patent Application Publication No. 2014-507863 Summary of the Invention

[0009] The problem the invention aims to solve

[0010] In VVC, the introduction of deblocking filtering is studied, similar to HEVC. Furthermore, in VVC, in addition to traditional intra-frame and inter-frame prediction, the introduction of weighted intra / inter-frame prediction is also studied. This weighted intra / inter-frame prediction uses both intra-frame and inter-frame predicted pixels to generate new predicted pixels. In HEVC, the deblocking filter strength determination method is based on prediction methods such as intra / inter-frame prediction. On the other hand, in weighted intra / inter-frame prediction, which is a new prediction method, the strength of the deblocking filter is determined using the same method as inter-frame prediction. However, this cannot adequately suppress distortion at block boundaries. This invention provides a technique for appropriately determining the strength of the deblocking filter processing for weighted intra / inter-frame prediction and suppressing distortion generated at block boundaries.

[0011] Solution for solving the problem

[0012] According to one aspect of the present invention, an image encoding apparatus is provided, characterized in that it comprises: an encoding unit for encoding an image by performing prediction processing on each block; a determining unit for determining, based on a mode used in the prediction processing of a first block and a mode used in the prediction processing of a second block adjacent to the first block, at least the intensity of a deblocking filter processing to be performed on the boundary between the first block and the second block; and a processing unit for performing deblocking filter processing on the boundary corresponding to the intensity determined by the determining unit, wherein the encoding unit is capable of using one of the following modes in the prediction processing: a first mode for deriving the encoding using pixels in an image including the block to be encoded. The determination component determines the following: a first block and a second block. The first block is defined as a block containing a block of code. The second block is defined as a second mode for deriving the predicted pixels in the block of code using pixels from an image different from the image containing the block of code. The third mode is defined as a third mode for generating the predicted pixels in the block of code using both pixels from the image containing the block of code and pixels from an image different from the image containing the block of code. The determination component sets the intensity of the deblocking filter processing to be performed on the boundary between the first block and the second block to the same intensity as when the first mode is used in at least one of the first block and the second block.

[0013] The effects of the invention

[0014] According to the configuration of the invention, the intensity of the deblocking filter processing for weighted intra / inter-frame prediction can be appropriately determined, and distortion generated at block boundaries can be suppressed.

[0015] Other features and advantages of the invention will become apparent from the following description taken in conjunction with the accompanying drawings. Note that the same reference numerals in all the drawings denote the same or similar components. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with the description herein, serve to explain the principles of the invention.

[0017] Figure 1 This is a block diagram illustrating an example of the functional configuration of an image encoding device;

[0018] Figure 2 This is a block diagram illustrating an example of the functional configuration of an image decoding device;

[0019] Figure 3 This is a flowchart of the encoding process;

[0020] Figure 4 This is a flowchart of the decoding process;

[0021] Figure 5 This is a block diagram illustrating an example of the hardware configuration of a computer device;

[0022] Figure 6 This is a view showing an example of the configuration of the bit stream; and

[0023] Figure 7 This is a view showing an example of deblocking filter processing. Detailed Implementation

[0024] Embodiments of the invention will now be described with reference to the accompanying drawings. Note that the embodiments described below illustrate examples of detailed implementation of the invention and are one of the detailed embodiments of the configuration described in the claims. For example, in the following description, terms such as "basic block" and "sub-block" are used. However, these embodiments can be applied to various processing units referred to as "blocks" and "units" in image coding techniques.

[0025] [First Embodiment]

[0026] First, refer to Figure 1 The block diagram illustrates an example of the functional configuration of the image encoding device according to this embodiment. The control unit 199 controls the operation of the entire image encoding device. The block segmentation unit 102 segments the input image (images of individual frames of a moving image, or a still image) into multiple basic blocks and outputs each basic block (segmented image).

[0027] The prediction unit 103 divides each basic block into multiple sub-blocks (segmented images). For each sub-block, the prediction unit 103 generates a prediction image by performing intra-frame prediction, inter-frame prediction, or weighted intra-frame / inter-frame prediction (which weights and combines intra-frame and inter-frame predictions). Then, for each sub-block, the prediction unit 103 obtains the difference between the prediction image and the sub-block as the prediction error. In addition, the prediction unit 103 generates prediction information, which includes information on how the basic block is divided into sub-blocks, prediction modes, and information required for prediction, such as motion vectors.

[0028] The conversion / quantization unit 104 performs an orthogonal transformation on the prediction error of each sub-block, thereby obtaining the transformation coefficients (orthogonal transformation coefficients) of each sub-block. The conversion / quantization unit 104 quantizes the transformation coefficients of each sub-block, thereby generating the quantization coefficients of the sub-block.

[0029] The inverse quantization / inverse transformation unit 105 generates transform coefficients by inverse quantization of the quantization coefficients of each sub-block generated by the transformation / quantization unit 104 using a quantization matrix for quantizing the sub-blocks, and performs an inverse orthogonal transformation on the transform coefficients, thereby generating a prediction error.

[0030] The image regeneration unit 106 generates a predicted image based on the prediction information generated by the prediction unit 103 and the coded image data stored in the frame memory 107, and regenerates the image based on the predicted image and the prediction error generated by the inverse quantization / inverse conversion unit 105. The image regeneration unit 106 stores the regenerated image in the frame memory 107. The image data stored in the frame memory 107 is the image referenced by the prediction unit 103 when performing prediction (prediction processing) for images of other frames.

[0031] The in-loop filtering unit 108 performs in-loop filtering on the image stored in the frame memory 107, such as unblocking filtering or sample adaptive offset.

[0032] The filter intensity calculation unit 112 uses the prediction information output from the prediction unit 103 and the quantization coefficients output from the conversion / quantization unit 104 to calculate the intensity (bS value) of the deblocking filter processing to be performed on the boundary between adjacent sub-blocks.

[0033] One of two adjacent sub-blocks will be called sub-block P, and the other will be called sub-block Q. The bS value, representing the intensity of the deblocking filter processing performed on the boundary between sub-blocks P and Q, is calculated as follows.

[0034] • If at least one of sub-blocks P and Q is a sub-block that has applied intra-prediction or weighted intra / inter-prediction, then the bS value = 2.

[0035] • If there are non-zero orthogonal transformation coefficients in at least one of sub-blocks P and Q, then the value of bS = 1.

[0036] Otherwise, if the absolute value (amplitude) of the difference between the motion vector of sub-block P and the motion vector of sub-block Q is equal to or greater than a predetermined value (e.g., one or more pixels), then the bS value = 1.

[0037] Otherwise, if the reference images for motion compensation between sub-blocks P and Q are different, or if the number of motion vectors is different, then the value of bS = 1.

[0038] Otherwise, the value of bS is 0.

[0039] Here, when the bS value increases, deblocking filter processing using a higher intensity deblocking filter is performed. In this embodiment, when the bS value = 0, no deblocking filter processing is performed. When the bS value = 1, deblocking filter processing is performed only for the luminance component. When the bS value = 2, deblocking filter processing is performed for both the luminance and chrominance components. That is, in this embodiment, the bS value indicates whether deblocking filter processing is performed or indicates the type of signal (image component) such as the luminance component or chrominance component that is the object of deblocking filter processing. However, the present invention is not limited to this. The number of types of deblocking filter processing intensities can be larger or smaller. The processing content can also differ depending on the intensity of the deblocking filter processing.

[0040] For example, as in H.264 deblocking filter processing, the bS value can take five levels from 0 to 4. In this embodiment, the intensity of the deblocking filter processing for the boundaries between sub-blocks using weighted intra / inter-frame prediction has the same value as when the sub-block uses intra-frame prediction. This indicates that the bS value = 2, i.e., the intensity is maximum. However, the embodiment is not limited to this. In this embodiment, an intermediate bS value can be set between bS value = 1 and bS value = 2, and the intermediate bS value can be used if at least one of sub-blocks P and Q uses weighted intra / inter-frame prediction. In this case, deblocking filter processing similar to that in the normal case of bS value = 2 is performed on the luma component, and deblocking filter processing with a lower intensity of smoothing effect is performed on the chroma component compared to the case of bS value = 2. This makes it possible to perform deblocking filter processing with intermediate smoothness on the boundaries between sub-blocks using weighted intra / inter-frame prediction. The simple term "intensity of deblocking filter processing" as described above refers to changing the intensity of deblocking filter processing by altering the signal (luminance component or chrominance component) that is being processed by the deblocking filter, or by changing the intensity of the smoothing effect of the signal corrected by the deblocking filter.

[0041] As described above, the intensity of the deblocking filter processing to be performed on the boundaries between adjacent blocks in the regenerated image (decoded image) stored in the frame memory 107 is determined based on information obtained in the predictive coding processing of each block.

[0042] In this embodiment, a deblocking filter is applied to an 8-pixel × 8-pixel image region including the sub-block boundaries, thereby achieving deblocking filter processing for the image region.

[0043] This will be used Figure 7 The example shown is used to describe in detail the deblocking filter processing performed by the in-loop filter unit 108. Figure 7 In the middle, a sub-block Q with a size of 8 pixels × 8 pixels. Figure 7 Block Q in the middle is to the right of sub-block P, which has a size of 8 pixels × 8 pixels. Figure 7The sub-blocks P and Q are adjacent. Here, a deblocking filter is applied to the boundary portion between sub-blocks P and Q (an 8-pixel (horizontal) × 8-pixel (vertical) area formed by 4 pixels (horizontal) × 8 pixels (vertical) on the right side of sub-block P and 4 pixels (horizontal) × 8 pixels (vertical) on the left side of sub-block Q). At this time, the processing described below is performed on each of the upper and lower halves of the boundary portion, thereby realizing the deblocking filter processing for each of the upper and lower halves. Here, the upper half of the boundary portion refers to an 8-pixel (horizontal) × 4-pixel (vertical) pixel region formed by 4 pixels (horizontal) × 4 pixels (vertical) on the upper right side of sub-block P and 4 pixels (horizontal) × 4 pixels (vertical) on the upper left side of sub-block Q. Additionally, the lower half of the boundary indicates a pixel area of ​​8 pixels (horizontal) × 4 pixels (vertical), which is formed by 4 pixels (horizontal) × 4 pixels (vertical) on the lower right side of sub-block P and 4 pixels (horizontal) × 4 pixels (vertical) on the lower left side of sub-block Q.

[0044] The deblocking filter processing for the lower half of the boundary region will be described below. The deblocking filter processing described below is similarly applied to the upper half of the boundary region.

[0045] Figure 7 A rectangle from p00 to p33 is added to represent the pixels in the 4-pixel (horizontal) × 4-pixel (vertical) pixel region on the lower right side of the sub-block P, and p00 to p33 indicate the pixel value. Figure 7 In this context, q00 to q33 represent pixels in a 4-pixel (horizontal) × 4-pixel (vertical) pixel region on the lower left side of sub-block Q, and q00 to q33 indicate pixel values. First, regarding the luminance signal, if the bS value is ≥ 1, the following inequality is used to determine whether filtering is required.

[0046] |p20–2×p10+p00|+|p23-2×p13+p03|+|q20–2×q10+q00|+|q23–2×q13+q03|<β

[0047] Here, β is a value obtained by averaging the quantization parameters of sub-block P and sub-block Q. Deblocking filter processing is determined only if this inequality is satisfied. When deblocking filter processing is to be performed, it is then determined whether to use a strong filter or a weak filter with different smoothing effects. If all inequalities (1) to (6) shown below are satisfied, it is determined that a strong filter with high smoothing effect is used. Otherwise, a weak filter with a weaker smoothing effect than a strong filter is used.

[0048] (1) 2 × (|p20 – 2 × p10 + p00| + |q20 - 2 × q10 + q00|) < (β >> 2)

[0049] (2) 2 × (|p23 – 2 × p13 + p03| + |q23 - 2 × q13 + q03|) < (β >> 2)

[0050] (3) |p30 - p00| + |q00 - q30| < (β >> 3)

[0051] (4) |p33 - p03| + |q03 - q33| < (β >> 3)

[0052] (5) |p00 - q00| < ((5 × tc + 1) >> 1)

[0053] (6) |p03 - q03| < ((5 × tc + 1) >> 1)

[0054] Here, >>N (N = 1 to 3) means N-bit arithmetic right shift calculation, and tc is the value obtained based on the bS value, the quantization parameters of sub-block P, and the quantization parameters of sub-block Q.

[0055] Let p'0k, p'1k, p'2k, q'0k, q'1k, and q'2k (k = 0 to 3) be the filtered pixels. The strong filtering process with high smoothness for the luminance signal is expressed by the following formula:

[0056] p'0k=Clip3(p0k-2×tc,p0k+2×tc,(p2k+2×p1k+2×p0k+2×q0k+q1k+4)>>3)

[0057] p'1k=Clip3(p1k-2×tc,p1k+2×tc,(p2k+p1k+p0k+q0k+2)>>2)

[0058] p'2k=Clip3(p2k-2×tc,p2k+2×tc,(2×p3k+3×p2k+p1k+p0k+q0k+4)>>3)

[0059] q'0k=Clip3(q0k-2×tc,q0k+2×tc,(q2k+2×q1k+2×q0k+2×p0k+p1k+4)>>3)

[0060] q'1k=Clip3(q1k-2×tc,q1k+2×tc,(q2k+q1k+q0k+p0k+2)>>2)

[0061] q'2k=Clip3(q2k-2×tc,q2k+2×tc,(2×q3k+3×q2k+q1k+q0k+p0k+4)>>3)

[0062] Clip3(a, b, c) is a function used to perform clipping such that the range of c satisfies a ≤ b ≤ c. On the other hand, a weak filtering process with low smoothing effect is performed on the luminance signal using the following formula.

[0063] Δ=(9×(q0k-p0k)–3×(q1k-p1k)+8)>>4

[0064] |Δ|<10×tc

[0065] If this inequality is not satisfied, the deblocking filter process is not performed. If the inequality is satisfied, the processing described below is performed for pixel values ​​p0k and q0k.

[0066] Δ = Clip3(-tc, tc, Δ)

[0067] p'0k = Clip1Y(p0k + Δ)

[0068] q'0k=Clip1Y(q0k-Δ)

[0069] Where Clip1Y(a) is a function used for clipping such that the range of a satisfies 0 ≤ a ≤ (the maximum value that can be represented by the bit depth of the luminance signal). Furthermore, if the condition expressed by the following formula is satisfied,

[0070] |p20–2×p10+p00|+|p23-2×p13+p03|<(β+(β>>1))>>3)

[0071] |q20–2×q10+q00|+|q23-2×q13+q03|<(β+(β>>1))>>3)

[0072] The following filtering process is then performed on p1k and q1k.

[0073] Δp=Clip3(-(tc>>1),tc>>1,(((p2k+p0k+1)>>1)-p1k+Δ)>>1)

[0074] p'1k = Clip1Y(p1k + Δp)

[0075] Δq=Clip3(-(tc>>1),tc>>1,(((q2k+q0k+1)>>1)-q1k+Δ)>>1)

[0076] q'1k=Clip1Y(q1k+Δq)

[0077] Regarding the deblocking filter processing of the color difference signal, the following processing is performed only when the bS value = 2.

[0078] Δ=Clip3(-tc,tc,((((q0k-p0k)<<2)+p1k-q1k+4)>>3))

[0079] p'0k = Clip1C(p0k + Δ)

[0080] p'0k = Clip1C(p0k - Δ)

[0081] Where Clip1C(a) is a function used for clipping such that the range of a satisfies 0 ≤ a ≤ (the maximum value that can be represented by the bit depth of the color difference signal). In this embodiment, the bS value, representing the intensity of the deblocking filter processing, indicates the type of signal to which the deblocking filter processing is applied, and additionally, a strong filter with high smoothing effect and a weak filter with low smoothing effect are selectively used based on the conditions of the pixel values. However, the invention is not limited thereto. For example, the intensity of the smoothing effect can be determined not only based on the bS value. Alternatively, the intensity of the smoothing effect can be determined based only on the bS value, and the type of signal can be determined based on other conditions.

[0082] Encoding unit 109 encodes the quantization coefficients generated by conversion / quantization unit 104 and the prediction information generated by prediction unit 103, thereby generating encoded data. Synthesis encoding unit 110 generates and outputs a bit stream, which includes the encoded data generated by encoding unit 109 and header data, which includes information required for decoding the input image.

[0083] The operation of the image encoding device according to this embodiment will now be described. The block segmentation unit 102 segments the input image into multiple basic blocks and outputs each segmented basic block.

[0084] Using basic blocks as units, prediction unit 103 divides a basic block into multiple sub-blocks (segmented image). For each sub-block, prediction unit 103 determines which of the following prediction methods to use for encoding.

[0085] Intra-frame prediction, such as horizontal or vertical prediction.

[0086] • Inter-frame prediction used for motion compensation based on reference frames

[0087] Weighted intra / inter-frame prediction, which combines intra-frame prediction and inter-frame prediction.

[0088] The prediction method used in this embodiment will be described again. In intra-frame prediction, coded pixels spatially located around a block (coded object block) are used as coded objects to generate (derive) prediction pixels for the coded object block, and intra-frame prediction modes representing intra-frame prediction methods such as horizontal prediction, vertical prediction, or DC prediction are also generated.

[0089] In inter-frame prediction, predicted pixels for the coded object block are generated using coded pixels from frames that are temporally different from the coded object block. Additionally, motion information or motion vectors representing the frames to be referenced are also generated.

[0090] In weighted intra / inter-frame prediction, the pixel value of the predicted pixel of the coded object block is generated by obtaining the weighted average of the pixel values ​​generated by the intra-frame prediction and the pixel values ​​generated by the inter-frame prediction (using both). The pixel value of the predicted pixel is calculated using, for example, the following equation (1) (the equation in the case where the size of the basic block is 8 pixels × 8 pixels).

[0091] p[x][y]=(w×pInter[x][y]+(8-w)×pIntra[x][y])>>3)...(1)

[0092] ">>" indicates a rightward shift. In equation (1), p[x][y] is the pixel value of the predicted pixel calculated using weighted intra / inter-frame prediction for coordinates (x, y) in the coded object block. pInter[x][y] is the pixel value of the predicted pixel for coordinates (x, y) in the coded object block using inter-frame prediction, and pInter[x][y] is the pixel value of the predicted pixel for coordinates (x, y) in the coded object block using inter-frame prediction. w represents the weight value for the pixel value predicted for inter-frame and the pixel value predicted for intra-frame. In this embodiment, when w = 4, the weights for the pixel values ​​predicted for inter-frame and intra-frame become equal. In other words, if w > 4, the weight for the pixel value predicted for inter-frame increases. If w < 4, the weight for the pixel value predicted for intra-frame increases. The method for determining the weight value is not particularly limited, and the weight value is determined based on the magnitude of the motion vector of the intra-frame prediction mode or inter-frame prediction and the position of the coded object block, etc. In weighted intra / inter-frame prediction, predicted pixels for coded object blocks are generated in this manner, and intra-frame prediction modes and motion information used to generate the predicted pixels are also generated.

[0093] Then, prediction unit 103 generates a prediction image based on the determined prediction method and coded pixels, and generates a prediction error based on the sub-blocks and the prediction image. Prediction unit 103 also generates prediction information, which represents how the basic block is divided into sub-blocks, the prediction mode, and information required for prediction, such as motion vectors.

[0094] The conversion / quantization unit 104 performs an orthogonal transformation on the prediction error of each sub-block, thereby generating the transformation coefficients of each sub-block. The conversion / quantization unit 104 then quantizes the transformation coefficients of each sub-block, thereby generating the quantization coefficients of the sub-block.

[0095] The inverse quantization / inverse transformation unit 105 generates transform coefficients by inverse quantization of the quantization coefficients of each sub-block generated by the transformation / quantization unit 104 using a quantization matrix for quantizing the sub-blocks, and performs an inverse orthogonal transformation on the transform coefficients, thereby generating a prediction error.

[0096] The image regeneration unit 106 generates a predicted image based on the prediction information generated by the prediction unit 103 and the coded image data stored in the frame memory 107, and regenerates the image based on the predicted image and the prediction error generated by the inverse quantization / inverse conversion unit 105. The image regeneration unit 106 stores the regenerated image in the frame memory 107.

[0097] The filtering intensity calculation unit 112 performs the above-mentioned processing by using the prediction information output from the prediction unit 103 and the quantization coefficients output from the conversion / quantization unit 104 to calculate the intensity of the deblocking filter processing to be performed on the boundary between adjacent sub-blocks.

[0098] The in-loop filtering unit 108 performs in-loop filtering processing on the image stored in the frame memory 107, such as deblocking filtering or sample adaptive shifting. The deblocking filter processing to be performed by the in-loop filtering unit 108 is based on the intensity obtained by the filtering intensity calculation unit 112.

[0099] Encoding unit 109 performs entropy encoding on the quantization coefficients generated by conversion / quantization unit 104 and the prediction information generated by prediction unit 103, thereby generating encoded data. No specific entropy encoding method is specified; Golomb coding, arithmetic coding, or Huffman coding, etc., can be used.

[0100] The synthesis coding unit 110 generates a bit stream by multiplexing the encoded data and header data generated by the coding unit 109, and outputs the generated bit stream.

[0101] Reference Figure 3The flowchart describes the encoding process of the input image using the aforementioned image encoding device. In step S301, the synthesis encoding unit 110 encodes the header required for image encoding, thereby generating header data (encoded data).

[0102] In step S302, the block segmentation unit 102 segments the input image into multiple basic blocks. In step S303, the prediction unit 103 selects the unselected blocks from the basic blocks segmented in step S302 as the selected basic blocks. The prediction unit 103 determines a sub-block segmentation method (in this embodiment, one of intra-frame prediction, inter-frame prediction, and weighted intra / inter-frame prediction) and segments the selected basic blocks into multiple sub-blocks according to the determined sub-block segmentation method. Furthermore, the prediction unit 103 determines the prediction method on a sub-block basis. For each sub-block, the prediction unit 103 generates a prediction image by using the image in the frame memory 107 and performing prediction according to the determined prediction method, and obtains the difference between the sub-block and the prediction image as the prediction error. In addition, the prediction unit 103 generates prediction information representing the sub-block segmentation method, the prediction mode, and prediction-required information such as motion vectors.

[0103] In step S304, the conversion / quantization unit 104 performs an orthogonal transformation on the prediction error of each sub-block, thereby generating the transformation coefficients of each sub-block. The conversion / quantization unit 104 then quantizes the transformation coefficients of each sub-block, thereby generating the quantization coefficients of the sub-block.

[0104] In step S305, the inverse quantization / inverse conversion unit 105 generates transform coefficients by inverse quantization of the quantization coefficients of each sub-block generated in step S304 using a quantization matrix for quantizing the sub-blocks. Then, the inverse quantization / inverse conversion unit 105 performs an inverse orthogonal transform on the generated transform coefficients, thereby generating a prediction error.

[0105] In step S306, the image regeneration unit 106 generates a predicted image based on the prediction information generated by the prediction unit 103 and the encoded image data stored in the frame memory 107. Then, the image regeneration unit 106 regenerates the image based on the predicted image and the prediction error generated by the inverse quantization / inverse conversion unit 105. The image regeneration unit 106 stores the regenerated image in the frame memory 107.

[0106] In step S307, the encoding unit 109 performs entropy encoding on the quantization coefficients generated by the conversion / quantization unit 104 and the prediction information generated by the prediction unit 103, thereby generating encoded data. The synthesis encoding unit 110 multiplexes the encoded data generated by the encoding unit 109 and the header data, thereby generating a bit stream.

[0107] In step S308, the control unit 199 determines whether all basic blocks have been encoded. If all basic blocks have been encoded, the process proceeds to step S309. If there are still unencoded basic blocks, the process returns to step S303.

[0108] In step S309, the filter processing intensity calculation unit 112 performs the above processing by using the prediction information obtained by the prediction unit 103 and the quantization coefficients obtained by the conversion / quantization unit 104 to calculate the intensity of the deblocking filter processing to be performed on the boundary between adjacent sub-blocks.

[0109] In step S310, the in-loop filtering unit 108 performs in-loop filtering processing on the image stored in the frame memory 107, such as deblocking filtering or sample adaptive shifting. The deblocking filter processing to be performed on the boundary between adjacent sub-blocks is based on the intensity obtained by the filtering intensity calculation unit 112 for that boundary.

[0110] As described above, according to this embodiment, particularly in step S309, weighted intra / inter-frame prediction can be used to set a deblocking filter with high distortion correction effect for the boundaries between sub-blocks. This can suppress block distortion and improve subjective image quality. Furthermore, since no new operation is required to calculate the intensity of the deblocking filter processing, it does not increase the complexity of the implementation.

[0111] Furthermore, in this embodiment, the presence / absence of deblocking filter processing for block boundaries of luminance or chromatic aberration is changed based on the intensity (bS value) of the deblocking filter processing. However, the intensity of the smoothing effect of the filter itself can be changed by the intensity (bS value). For example, when the intensity (bS value) of the deblocking filter processing is high, a filter with a longer tap length and a high correction effect can be used. When the intensity (bS value) of the deblocking filter processing is low, a filter with a shorter tap length and a low correction effect can be used. This allows the intensity of the filter, i.e., the correction effect, to be adjusted by methods other than the presence / absence of the deblocking filter processing.

[0112] [Second Embodiment]

[0113] In the following embodiments, including this embodiment, the differences from the first embodiment will be described, and unless otherwise stated below, the remainder is the same as the first embodiment. In this embodiment, the image encoding device according to Figure 3 The flowchart is processed as follows.

[0114] Figure 6An example of the configuration of the bitstream according to this embodiment is shown. The image header stores filter_weight_threshold, which is the weight threshold (intensity weight threshold) for the intensity of the deblocking filter processing. This is the threshold used to determine whether a sub-block should be processed as a sub-block applying intra-frame prediction or a sub-block applying inter-frame prediction when calculating the bS value of the deblocking filter, using the weight value w in the weighted intra / inter-frame prediction.

[0115] In step S309, the filter processing intensity calculation unit 112 calculates the bS value. More specifically, the bS value is calculated by processing the weighted intra / inter-frame predicted sub-blocks into intra-frame predicted sub-blocks if the following inequality holds.

[0116] w <filter_weight_threshold

[0117] For example, if the value of `filter_weight_threshold` is 4, then all sub-blocks of weighted intra / inter-prediction with a weight value w less than 4 will be treated as intra-prediction sub-blocks. All sub-blocks of weighted intra / inter-prediction with a weight value w greater than 4 will be treated as inter-prediction sub-blocks. The following diagram illustrates this. Figure 7 The example shown illustrates the bS values ​​when sub-blocks P and Q are encoded using intra-frame prediction, inter-frame prediction, weighted intra / inter-frame prediction (w=3), and weighted intra / inter-frame prediction (w=5).

[0118] [Table 1]

[0119] Prediction methods for blocks P and Q and bS values

[0120]

[0121] In this table, filter_weight_threshold = 4. For example, if at least one of adjacent sub-blocks P and Q is encoded using intra-frame prediction or weighted intra / inter-frame prediction (w = 3), then that sub-block is treated as an intra-frame prediction block. Therefore, the bS value is set to 2. If both sub-blocks P and Q are encoded using inter-frame prediction or weighted intra / inter-frame prediction (w = 5), then both sub-blocks P and Q are treated as inter-frame prediction blocks. Therefore, the bS value is set to 0 or 1. As in the filter processing intensity calculation unit 112 according to the first embodiment, it is determined whether to set the bS value to 0 or 1. This is determined based on the presence / absence of non-zero orthogonal transform coefficients in sub-blocks P and Q, the difference in the number or magnitude of motion vectors, and the difference in the reference image, etc.

[0122] The filtering intensity calculation unit 112 determines the bS value using data from a reference table. Note that in this embodiment, filter_weight_threshold is stored in the image header. However, the storage destination is not limited to a specific storage destination, and filter_weight_threshold can be stored, for example, in the sequence header. Furthermore, in this embodiment, the intensity weight threshold is stored in the header as information used to determine whether a weighted intra-frame / inter-frame prediction sub-block should be processed as an intra-frame prediction sub-block or an inter-frame prediction sub-block when calculating the bS value. However, the invention is not limited to this. It is possible to store marker information indicating that a sub-block should always be processed as an intra-frame prediction sub-block, or it is possible to store marker information indicating that a sub-block should always be processed as an inter-frame prediction sub-block. Alternatively, the intensity weight threshold filter_weight_threshold_minus4 can be set as the value obtained by subtracting 4 from the value of filter_weight_threshold. Since the probability of filter_weight_threshold_minus4 being set to 0 or a value close to 0 increases, the amount of code in the information itself can be reduced by using Golomb coding or the like.

[0123] As described above, according to this embodiment, the intensity of the deblocking filter processing for sub-blocks of weighted intra / inter-frame prediction can be determined without complex processing. Furthermore, the user can freely adjust the intensity of the deblocking filter processing for blocks of weighted intra / inter-frame prediction.

[0124] Furthermore, in this embodiment, information used to determine the strength of the filter is output to the header. However, the invention is not limited thereto. Whether a weighted intra / inter-frame prediction sub-block should be processed as an intra-frame prediction sub-block or as an inter-frame prediction sub-block can be uniquely determined in advance by the value w. Alternatively, a deblocking filter that smoothly increases in strength as the weight of intra-frame prediction increases can be applied independently of the bS value. Therefore, the amount of code corresponding to the strength weight threshold can be saved, and the implementation complexity can be reduced by using a fixed prediction mode for the deblocking filter processing.

[0125] [Third Embodiment]

[0126] In this embodiment, an image decoding apparatus for decoding an input image encoded by the image encoding apparatus according to the first embodiment will be described. Reference will be made to... Figure 2 The block diagram describes an example of the functional configuration of the image decoding device according to this embodiment.

[0127] Control unit 299 controls the overall operation of the image decoding device. Demultiplexing / decoding unit 202 acquires the bit stream generated by the image encoding device, demultiplexes information related to decoding processing and encoded data related to coefficients from the bit stream, and decodes the encoded data present in the header of the bit stream. In this embodiment, demultiplexing / decoding unit 202 performs the opposite operation to that of the synthesis encoding unit 110 described above.

[0128] Decoding unit 203 decodes the encoded data demultiplexed from the bitstream by demultiplexing / decoding unit 202, thereby obtaining quantization coefficients and prediction information. Inverse quantization / inverse conversion unit 204 performs operations similar to those of the inverse quantization / inverse conversion unit 105 provided in the aforementioned image encoding device. Inverse quantization / inverse conversion unit 204 obtains transform coefficients by inverse quantization of the quantization coefficients, and performs an inverse orthogonal transform on these transform coefficients, thereby obtaining the prediction error.

[0129] The image regeneration unit 205 generates a predicted image based on the prediction information decoded by the decoding unit 203 and by referring to the image stored in the frame memory 206. The image regeneration unit 205 uses the generated predicted image and the prediction error obtained by the inverse quantization / inverse conversion unit 204 to generate a regenerated image and stores the generated regenerated image in the frame memory 206.

[0130] Like the filtering intensity calculation unit 112, the filtering intensity calculation unit 209 uses the prediction information and quantization coefficients decoded by the decoding unit 203 to determine the bS value of the intensity of the deblocking filter processing for the boundary between adjacent sub-blocks.

[0131] Like the in-loop filtering unit 108, the in-loop filtering unit 207 performs in-loop filtering processing, such as deblocking filtering, on the regenerated image stored in the frame memory 206. The deblocking filter processing of the in-loop filtering unit 207 is the deblocking filter processing corresponding to the bS value obtained by the filtering intensity calculation unit 209.

[0132] The reproduced image stored in frame memory 206 is appropriately output by control unit 299. The output destination of the reproduced image is not limited to a specific output destination. For example, the reproduced image can be displayed on the display screen of a display device such as a monitor, or the reproduced image can be output to a projection device such as a projector.

[0133] The operation (bitstream decoding process) of the image decoding device with the above configuration will now be described. In this embodiment, the bitstream input to the demultiplexing / decoding unit 202 is the bitstream of each frame of a moving image. However, the bitstream input to the demultiplexing / decoding unit 202 can be the bitstream of a still image.

[0134] The demultiplexing / decoding unit 202 acquires the bitstream of a frame generated by the image encoding device, demultiplexes information related to decoding processing and coded data related to coefficients from the bitstream, and decodes the coded data present in the header of the bitstream. Furthermore, the demultiplexing / decoding unit 202 outputs the coded data of each basic block of the image data to the decoding unit 203.

[0135] Decoding unit 203 decodes the encoded data demultiplexed from the bitstream by demultiplexing / decoding unit 202, thereby obtaining quantization coefficients and prediction information. The prediction information includes information indicating which of the following prediction methods is used to encode each sub-block.

[0136] Intra-frame prediction, such as horizontal or vertical prediction.

[0137] • Inter-frame prediction used for motion compensation based on reference frames

[0138] Weighted intra / inter-frame prediction that combines intra-frame prediction and inter-frame prediction

[0139] The inverse quantization / inverse conversion unit 204 obtains the transformation coefficients by inverse quantization of the quantization coefficients of each sub-block, and performs inverse orthogonal transformation on the transformation coefficients to obtain the prediction error.

[0140] The image regeneration unit 205 generates a predicted image based on the prediction information decoded by the decoding unit 203 and by referring to the image stored in the frame memory 206. The image regeneration unit 205 uses the generated predicted image and the prediction error obtained by the inverse quantization / inverse conversion unit 204 to generate a regenerated image and stores the generated regenerated image in the frame memory 206.

[0141] Like the filtering intensity calculation unit 112, the filtering intensity calculation unit 209 uses the prediction information and quantization coefficients decoded by the decoding unit 203 to determine the bS value of the intensity of the deblocking filter processing for the boundary between adjacent sub-blocks.

[0142] Like the in-loop filtering unit 108, the in-loop filtering unit 207 performs in-loop filtering processing, such as deblocking filtering, on the regenerated image stored in the frame memory 206. The deblocking filter processing of the in-loop filtering unit 207 is the deblocking filter processing corresponding to the bS value obtained by the filtering intensity calculation unit 209.

[0143] In this embodiment, as in the first embodiment, the bS value indicates the type of signal (image component) to be processed as a deblocking filter. Furthermore, strong filters with high smoothing effects and weak filters with low smoothing effects are selectively used based on pixel values. However, the invention is not limited to this. For example, the bS value can determine not only the signal type but also the intensity of the smoothing effect. Alternatively, the intensity of the smoothing effect can be determined based only on the bS value, and the signal type can be determined based on other conditions.

[0144] Reference Figure 4 The flowchart describes the decoding process performed by the image decoding device on a bitstream corresponding to a frame. In step S401, the demultiplexing / decoding unit 202 acquires the bitstream corresponding to a frame generated by the image encoding device. Then, the demultiplexing / decoding unit 202 demultiplexes the information related to the decoding process and the coded data related to the coefficients from the bitstream, and decodes the coded data present in the header of the bitstream.

[0145] Steps S402 to S405 are performed on each basic block of the input image. In step S402, the decoding unit 203 decodes the encoded data demultiplexed from the bit stream by the demultiplexing / decoding unit 202, thereby obtaining quantization coefficients and prediction information.

[0146] In step S403, the inverse quantization / inverse conversion unit 204 obtains the transform coefficients by inverse quantization of the quantization coefficients of the sub-block that is the object of decoding, and performs inverse orthogonal transformation on the transform coefficients to obtain the prediction error.

[0147] In step S404, the image regeneration unit 205 generates a predicted image based on the prediction information decoded by the decoding unit 203, by referring to the image stored in the frame memory 206. The image regeneration unit 205 uses the generated predicted image and the prediction error obtained by the inverse quantization / inverse conversion unit 204 to generate a regenerated image, and stores the generated regenerated image in the frame memory 206.

[0148] In step S405, the control unit 299 determines whether the decoding of all basic blocks included in the bit stream is complete. As a result of the determination, if the decoding of all basic blocks included in the bit stream is complete, the process proceeds to step S406. On the other hand, if there are still basic blocks among all basic blocks included in the bit stream whose decoding is not yet complete, the process from step S402 is repeated for the basic blocks whose decoding is not yet complete.

[0149] In step S406, the filter processing intensity calculation unit 209, like the filter processing intensity calculation unit 112, uses the prediction information and quantization coefficients decoded by the decoding unit 203 to determine the bS value of the intensity of the deblocking filter processing for the boundary between adjacent sub-blocks.

[0150] In step S407, the in-loop filtering unit 207, like the in-loop filtering unit 108, performs in-loop filtering processing, such as deblocking filtering, on the regenerated image stored in the frame memory 206 in step S404. The deblocking filter processing of the in-loop filtering unit 207 is the deblocking filter processing corresponding to the bS value obtained by the filtering intensity calculation unit 209 in step S406.

[0151] As described above, according to this embodiment, a bitstream generated by an image coding device according to the first embodiment can be decoded, wherein an appropriate deblocking filter is applied to sub-blocks encoded by weighted intra / inter-frame prediction.

[0152] Furthermore, in this embodiment, the presence / absence of filters targeting block boundaries for luminance or chromatic aberration is varied based on the intensity (bS value) of the deblocking filter processing. However, the strength of the smoothing effect of the filter itself can be altered by the intensity (bS value) of the deblocking filter processing. For example, when the intensity (bS value) of the deblocking filter processing is high, a filter with a longer tap length and high correction effect can be used. When the intensity (bS value) of the deblocking filter processing is low, a filter with a shorter tap length and low correction effect can be used. This enables the decoding of bitstreams whose filter intensity (i.e., correction effect) has been adjusted by methods other than the presence / absence of filters.

[0153] [Fourth Embodiment]

[0154] In this embodiment, an image decoding device will be described that decodes an input image encoded by an image encoding device according to the second embodiment. In this embodiment, in accordance with... Figure 4 In the processing of the flowchart, the following aspects differ from those in the third embodiment.

[0155] In step S401, the demultiplexing / decoding unit 202 from Figure 6 The bitstream demultiplexing and decoding process shown includes information related to the decoding and the encoded data related to the coefficients, and the encoded data present in the bitstream header is decoded. In this decoding, the filter_weight_threshold, which serves as the intensity weight threshold in the image header, is decoded.

[0156] In step S406, the filter processing intensity calculation unit 209 calculates the bS value. Note that in this embodiment, as with the filter processing intensity calculation unit 209 according to the third embodiment, it is determined whether to set the bS value to 0 or 1.

[0157] In this embodiment, `filter_weight_threshold` is present in the image header. However, the invention is not limited to this, and `filter_weight_threshold` may be present, for example, in the sequence header. Furthermore, in this embodiment, the intensity weight threshold is decoded as information used to determine, when calculating the bS value, whether a weighted intra / inter-prediction block should be treated as an intra-prediction sub-block or an inter-prediction sub-block. However, the invention is not limited to this. It is possible to decode marker information indicating that a sub-block should always be treated as an intra-prediction sub-block, or it is possible to decode marker information indicating that a sub-block should always be treated as an inter-prediction sub-block. Alternatively, the value obtained by subtracting 4 from the value of `filter_weight_threshold` can be decoded as the intensity weight threshold `filter_weight_threshold_minus4`. Since the value of `filter_weight_threshold_minus4` is more likely to be set to 0 or a value close to 0, it is possible to decode bitstreams with a small amount of code.

[0158] As described above, according to this embodiment, the strength of the deblocking filter processing for sub-blocks of weighted intra / inter-frame prediction can be determined without complex processing. Furthermore, bitstreams for which the user has freely adjusted the strength of the deblocking filter processing for sub-blocks of weighted intra / inter-frame prediction can be decoded.

[0159] Furthermore, in this embodiment, information used to determine the strength of the filter is decoded from the header. However, the invention is not limited thereto. Whether a weighted intra / inter-prediction sub-block should be processed as an intra-prediction sub-block or an inter-prediction sub-block can be uniquely determined in advance by the value w. Alternatively, a deblocking filter that smoothly increases in strength as the weight of intra-prediction increases can be applied independently of the bS value. Therefore, bitstreams that save the amount of code corresponding to the strength weight threshold can be decoded, and implementation complexity can be reduced by using a prediction mode-fixed deblocking filter processing.

[0160] [Fifth Embodiment]

[0161] Figure 1 and 2All functional units shown can be implemented in hardware, and some can be implemented in software (computer programs). In this case, a computer device including frame memory 107 or frame memory 206 as a memory device and capable of executing computer programs can be applied to the aforementioned image encoding or image decoding device. (Refer to...) Figure 5 The block diagram description is an example of the hardware configuration of a computer device that can be applied to the aforementioned image encoding or image decoding device.

[0162] CPU 501 performs various processes using computer programs and data stored in RAM 502 or ROM 503. Therefore, CPU 501 performs overall operational control of the computer device and executes or controls the aforementioned processes as those to be performed by the image encoding or image decoding device.

[0163] RAM 502 has areas for storing computer programs and data loaded from ROM 503 or external storage device 506, as well as data received from the outside via I / F (interface) 507 (e.g., data of the aforementioned moving or still images). RAM 502 also has working areas used by CPU 501 to perform various processes. Therefore, RAM 502 can be appropriately provided with various areas. The computer device's setup data and startup program are stored in ROM 503.

[0164] Operation unit 504 is a user interface such as a keyboard, mouse, or touch panel. By operating operation unit 504, the user can input various instructions to CPU 501.

[0165] The display unit 505 is formed by a liquid crystal screen or a touch panel screen, and can display the processing results of the CPU 501 as images or characters. For example, a reproduced image decoded by the image decoding device described above can be displayed on the display unit 505. Note that the display unit 505 can be a projection device, such as a projector that projects images or characters.

[0166] External storage device 506 is a high-capacity information storage device such as a hard disk drive. External storage device 506 stores an OS (operating system) and computer programs and data that enable CPU 501 to execute or control the various processes described above (processes executed by the image encoding or decoding device). The computer programs stored in external storage device 506 include programs for enabling CPU 501 to perform the functions of functional units other than frame memory 107 and frame memory 206. Furthermore, the data stored in external storage device 506 includes various information required for encoding or decoding, such as information already described above as known information (intensity weight thresholds and data in Table 1, etc.).

[0167] Computer programs and data stored in external storage device 506 are appropriately loaded into RAM 502 under the control of CPU 501 and processed by CPU 501. Note that the aforementioned frame memory 107 or frame memory 206 can be implemented by a memory device such as RAM 502 or external storage device 506.

[0168] I / F 507 serves as an interface configured to communicate data with external devices. For example, moving or still images can be acquired from an external server device or image capturing device and stored in RAM 502 or external storage device 506 via I / F 507.

[0169] CPU 501, RAM 502, ROM 503, operation unit 504, display unit 505, external storage device 506, and I / F 507 are connected to bus 508. Note that... Figure 5 The configuration shown is merely an example of the hardware configuration of the computer equipment that can be applied to the aforementioned image encoding or image decoding equipment, and various changes / modifications can be made.

[0170] [Sixth Embodiment]

[0171] In the above embodiments, sub-blocks are used as the encoding unit. However, the encoding unit is not limited to sub-blocks; for example, basic blocks can be used as the encoding unit. Furthermore, the numerical values ​​used in the above description are for detailed description only and are not intended to limit the above embodiments to the numerical values ​​used. For example, the values ​​w, intensity weight threshold, and block size used in the above description are merely examples and are not limited to the numerical values ​​described above.

[0172] In the above embodiments, the image decoding device is described as a device distinct from the image encoding device. However, the image decoding device and the image encoding device can be combined into a single device. In this case, the device can encode an input image and decode the encoded input image as needed.

[0173] Furthermore, the object to which the deblocking filter is applied is not limited to the boundaries between sub-blocks, but can also be, for example, the boundaries between transform units. In the above embodiment, the size of the deblocking filter is 8 pixels × 8 pixels. However, the size is not limited to this. The size of the transform unit can be the same as or different from that of the sub-block.

[0174] Some or all of the above embodiments can be used in combination as appropriate. Alternatively, some or all of the above embodiments can be used selectively.

[0175] (Other embodiments)

[0176] This invention can be implemented by providing a program for implementing one or more functions of the above embodiments to a system or device via a network or storage medium, and by causing one or more processors in the computer of the system or device to read and execute the program. This invention can also be implemented by circuitry (e.g., an ASIC) for implementing one or more functions.

[0177] This invention is not limited to the embodiments described above, and various changes and modifications can be made within the spirit and scope of this invention. Therefore, the appended claims are made to inform the public of the scope of this invention.

[0178] This application claims priority to Japanese Patent Application 2018-235911, filed on December 17, 2018, which is incorporated herein by reference.

[0179] List of reference numerals

[0180] 102: Block segmentation unit; 103: Prediction unit; 104: Conversion / quantization unit; 105: Inverse quantization / inverse conversion unit; 106: Image regeneration unit; 107: Frame memory; 108: In-loop filtering unit; 109: Coding unit; 110: Synthesis coding unit; 112: Filtering intensity calculation unit; 202: Demultiplexing / decoding unit; 203: Decoding unit; 204: Inverse quantization / inverse conversion unit; 205: Image regeneration unit; 206: Frame memory; 207: In-loop filtering unit; 209: Filtering intensity calculation unit; 299: Control unit.

Claims

1. An image encoding device, comprising: The encoding unit is configured to encode the image by performing prediction processing on each block; A determining unit is configured to determine a bS value for unblocking filter processing to be performed on the boundary between the first block and the second block based on at least one of the modes used in the prediction processing in the first block and the modes used in the prediction processing in the second block adjacent to the first block, wherein the bS value corresponds to the intensity of the unblocking filter processing. as well as The processing unit is configured to perform deblocking filter processing on the boundary based on a tc value derived by using a first quantization parameter of the first block, a second quantization parameter of the second block, and a bS value determined by the determining unit. In the case where a first mode is used in the prediction processing for the object block to be encoded, the predicted pixels of the object block are derived by using pixels in an image that includes the object block. In the case where the second mode is used in the prediction processing for the object block, the predicted pixels of the object block are derived by using pixels from an image different from the image including the object block. In the case where a third mode is used in the prediction processing for the object block, the predicted pixels of the object block are derived by using both pixels in the image including the object block and pixels in an image different from the image including the object block. Specifically, the bS value to be used for the deblocking filter processing at the boundary between the first block and the second block when the third mode is used in the first block and the second mode is used in the second block is the same as the bS value to be used when the third mode is used in the first block and the first mode is used in the second block. Wherein, 2 is set as the bS value of the deblocking filter processing to be performed on the boundary between the first block and the second block when the first mode is used in the first block and the second mode is used in the second block, and Wherein, when the third mode is used in at least one of the first block and the second block, the bS value of the deblocking filter processing for the boundary between the first block and the second block is higher than the bS value used when the second mode is used in both the first block and the second block.

2. The image encoding device according to claim 1, wherein, The processing unit performs deblocking filter processing based on the bS value by selecting object components from the luminance and chrominance components in the block as objects to be processed by the deblocking filter.

3. The image encoding device according to claim 1 or 2, wherein, The processing unit performs deblocking filter processing based on the bS value by selecting whether to perform deblocking filter processing.

4. The image encoding device according to claim 1, in, The processing unit determines whether to perform deblocking filter processing based on the β value derived from using both the first quantization parameter and the second quantization parameter. When the processing unit determines that it needs to perform deblocking filter processing, it performs deblocking filter processing on the boundary.

5. The image encoding device according to claim 4, wherein, The β value is derived by using the average of the first quantization parameter and the second quantization parameter.

6. An image decoding device for decoding encoded image data for each block, comprising: The decoding unit is configured to decode the image by performing prediction processing on each block; A determining unit is configured to determine a bS value for unblocking filter processing to be performed on the boundary between the first block and the second block based on at least one of the modes used in the prediction processing in the first block and the modes used in the prediction processing in the second block adjacent to the first block, wherein the bS value corresponds to the intensity of the unblocking filter processing. as well as The processing unit is configured to perform deblocking filter processing on the boundary based on a tc value derived by using a first quantization parameter of the first block, a second quantization parameter of the second block, and a bS value determined by the determining unit. In the case where a first mode is used in the prediction processing for the object block to be decoded, the predicted pixels of the object block are derived by using pixels in the image including the object block. In the case where the second mode is used in the prediction processing for the object block, the predicted pixels of the object block are derived by using pixels from an image different from the image including the object block. In the case where a third mode is used in the prediction processing for the object block, the predicted pixels for the object block are generated by using both pixels from the image including the object block and pixels from an image different from the image including the object block. Specifically, the bS value to be used for the deblocking filter processing at the boundary between the first block and the second block when the third mode is used in the first block and the second mode is used in the second block is the same as the bS value to be used when the third mode is used in the first block and the first mode is used in the second block. Wherein, 2 is set as the bS value of the deblocking filter processing to be performed on the boundary between the first block and the second block when the first mode is used in the first block and the second mode is used in the second block, and Wherein, when the third mode is used in at least one of the first block and the second block, the bS value of the deblocking filter processing for the boundary between the first block and the second block is higher than the bS value used when the second mode is used in both the first block and the second block.

7. The image decoding device according to claim 6, wherein, The processing unit performs deblocking filter processing based on the bS value by selecting object components from the luminance and chrominance components in the block as objects to be processed by the deblocking filter.

8. The image decoding device according to claim 6, wherein, The processing unit performs deblocking filter processing based on the bS value by selecting whether to perform deblocking filter processing.

9. The image decoding device according to claim 6, in, The processing unit determines whether to perform deblocking filter processing based on the β value derived from using both the first quantization parameter and the second quantization parameter. When the processing unit determines that it needs to perform deblocking filter processing, it performs deblocking filter processing on the boundary.

10. The image decoding device according to claim 9, wherein, The β value is derived by using the average of the first quantization parameter and the second quantization parameter.

11. An image encoding method, comprising: The image is encoded by performing prediction processing on each block; Based on at least one of the patterns used in the prediction processing in the first block and the patterns used in the prediction processing in the second block adjacent to the first block, a bS value is determined for the unblocking filter processing to be performed on the boundary between the first block and the second block, wherein the bS value corresponds to the intensity of the unblocking filter processing; as well as Based on the tc value derived by using the first quantization parameter of the first block, the second quantization parameter of the second block, and the bS value, a deblocking filter is applied to the boundary. In the case where a first mode is used in the prediction processing for the object block to be encoded, the predicted pixels of the object block are derived by using pixels in an image that includes the object block. In the case where the second mode is used in the prediction processing for the object block, the predicted pixels of the object block are derived by using pixels from an image different from the image including the object block. In the case where a third mode is used in the prediction processing for the object block, the predicted pixels of the object block are derived by using both pixels in the image including the object block and pixels in an image different from the image including the object block. Specifically, the bS value to be used for the deblocking filter processing at the boundary between the first block and the second block when the third mode is used in the first block and the second mode is used in the second block is the same as the bS value to be used when the third mode is used in the first block and the first mode is used in the second block. Wherein, 2 is set as the bS value of the deblocking filter processing to be performed on the boundary between the first block and the second block when the first mode is used in the first block and the second mode is used in the second block, and Wherein, when the third mode is used in at least one of the first block and the second block, the bS value of the deblocking filter processing for the boundary between the first block and the second block is higher than the bS value used when the second mode is used in both the first block and the second block.

12. An image decoding method for decoding encoded image data for each block, comprising: The image is decoded by performing prediction processing on each block; Based on at least one of the patterns used in the prediction processing in the first block and the patterns used in the prediction processing in the second block adjacent to the first block, a bS value is determined for the unblocking filter processing to be performed on the boundary between the first block and the second block, wherein the bS value corresponds to the intensity of the unblocking filter processing; as well as Based on the tc value derived by using the first quantization parameter of the first block, the second quantization parameter of the second block, and the determined bS value, a deblocking filter is applied to the boundary. In the case where a first mode is used in the prediction processing for the object block to be decoded, the predicted pixels of the object block are derived by using pixels in the image including the object block. In the case where the second mode is used in the prediction processing for the object block, the predicted pixels of the object block are derived by using pixels from an image different from the image including the object block. In the case where a third mode is used in the prediction processing for the object block, the predicted pixels for the object block are generated by using both pixels from the image including the object block and pixels from an image different from the image including the object block. Specifically, the bS value to be used for the deblocking filter processing at the boundary between the first block and the second block when the third mode is used in the first block and the second mode is used in the second block is the same as the bS value to be used when the third mode is used in the first block and the first mode is used in the second block. Wherein, 2 is set as the bS value of the deblocking filter processing to be performed on the boundary between the first block and the second block when the first mode is used in the first block and the second mode is used in the second block, and Wherein, when the third mode is used in at least one of the first block and the second block, the bS value of the deblocking filter processing for the boundary between the first block and the second block is higher than the bS value used when the second mode is used in both the first block and the second block.

13. A computer-readable storage medium storing a computer program that, when executed by a computer, causes the computer to perform the method according to claim 11 or 12.

14. A computer program product comprising a computer program that, when executed by a computer, causes the computer to perform the method according to claim 11 or 12.

Images