Prediction image generation device, video decoding device, video encoding device, and prediction image generation method

Abstract

To provide a prediction image generation apparatus and generation method, a video image decoding apparatus, and an encoding apparatus for improving the accuracy of DIMD methods for intra prediction.SOLUTION: In a video decoding device, a DIMD prediction unit of an intra prediction image generation unit modifies the size of a filter to be used for deriving a gradient, according to the size of a target block. The filter is split into a 2×2 filter and a 3×3 filter. The target blocks are divided into small target blocks and large target blocks. Target blocks having a length and width of 16 or smaller are defined as small target blocks, and blocks having other sizes are defined as large target blocks. The accuracy of DIMD is improved by using the 2×2 filter for calculating a gradient for small target blocks, and using the 3×3 filter for calculating a gradient for large target blocks.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a predicted image generating device, a video decoding device, a video encoding device, and a predicted image generating method. [Background technology]

[0002] A video encoding device that generates coded data by encoding video, and a video decoding device that generates decoded images by decoding the coded data, are used for efficient transmission or recording of video.

[0003] For example, specific video coding formats include H.264 / AVC, High-Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC).

[0004] In such a video coding method, images (pictures) constituting a video are managed in a hierarchical structure including slices obtained by dividing images, coding tree units (CTUs) obtained by dividing slices, coding units (called coding units (CUs)) obtained by dividing coding tree units, and transform units (TUs) obtained by dividing coding units, and are coded / decoded for each CU.

[0005] In such video coding methods, a predicted image is typically generated based on a local decoded image obtained by encoding / decoding an input image (source image), and a prediction error component (sometimes called a "difference image" or "residual image") obtained by subtracting the predicted image from the input image is coded. Methods for generating predicted images include inter-frame prediction (inter-prediction) and intra-frame prediction (intra-prediction).

[0006] In recent video coding and decoding techniques, Non-Patent Document 1 proposes a decoder-side intra mode derivation (DIMD) prediction method in which a decoder derives an intra angular prediction mode using pixels in a neighboring region, thereby deriving a predicted image. Furthermore, Non-Patent Document 2 proposes an improvement to the DIMD method that uses different reference regions for prediction. [Prior art documents] [Non-patent literature]

[0007] Non-Patent Document 1: M. Abdoli, T. Guionnet, E. Mora et al., "Non-CE3: Decoder-side Intra Mode Derivation with Prediction Fusion Using Planar," JVET-O0449, Gothenburg, July 2019. Non-patent document 2: Z.Fan, Y.YASUGI, T.IKAI, "Non-EE2:Adaptive reference region DIMD", JVET-AB0065, Mainz, DE, 20-28 October 2022 Summary of the Invention

[0008] technical challenges In Non-Patent Document 2, the intra mode is derived on the decoding unit side using the gradient of pixel values ​​of different adjacent regions, but there is a problem that the prediction angle of the adjacent block may not match the prediction angle of the current block.

[0009] The present invention aims to improve accuracy in intra mode derivation on the decoding unit side by switching the filter used to calculate the gradient depending on the width and height of the current block. Means to solve the problem

[0010] The adaptive reference region DIMD (Non-Patent Document 2) method allows the selection of a reference region using the following three patterns, named DIMD_TL, DIMD_T, and DIMD_L, respectively: 1: Uses the upper left, upper, and left adjacent regions; 2: Uses the upper left, upper, and upper right adjacent regions; 3: Uses the upper left, left, and lower left adjacent regions.

[0011] The filter selection unit 3104611 is included in the gradient derivation unit 310461. The filter selection unit 3104611 selects a filter based on the width and height of the target block or dimd_mode.

[0012] The filter selection unit 3104611 is characterized by using a large filter for a target block having a large area, and a small filter for a target block having a small area. [Effects of the Invention]

[0013] According to one aspect of the present invention, it is possible to reduce the amount of calculation required to derive intra prediction modes while limiting coding loss. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a schematic diagram showing the configuration of an image transmission system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing a hierarchical structure of coded stream data. [Figure 3] FIG. 10 is a schematic diagram showing types of intra-prediction modes (mode numbers). [Figure 4] FIG. 2 is a schematic diagram of a video decoding unit. [Figure 5] This is an example of DIMD syntax. [Figure 6] FIG. 31 is a diagram showing the binarization of the dimd_mode syntax used in the DIMD prediction unit 31046. [Figure 7] Another example of DIMD syntax: [Figure 8]3 shows the structure of the intra-prediction image generation unit 310. [Figure 9] FIG. 31 is a diagram showing details of a DIMD prediction unit 31046. [Figure 10] An example of a reference area referred to by the DIMD prediction unit 31046 is shown below. [Figure 11] 1 shows an example of a set of spatial filters. [Figure 12] 1 shows an example of a target pixel for gradient derivation when using a 3×3 filter. [Figure 13] 1 shows an example of a pixel of interest for gradient derivation when using a 2×2 filter. [Figure 14] FIG. 1 is a block diagram showing the relationship between gradients and regions. [Figure 15] FIG. 2 is a block diagram showing the structure of an angle derivation unit. [Figure 16] 10 shows an example of a reference region used for gradient derivation in the DIMD prediction unit 31046. [Figure 17] FIG. 2 is a block diagram showing an example of the configuration of an inverse quantization and inverse transform processing unit 311. [Figure 18] FIG. 1 is a block diagram showing the structure of a video encoding system. DETAILED DESCRIPTION OF THE INVENTION

[0015] First embodiment Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0016] FIG. 1 is a schematic diagram showing the configuration of an image transmission system 1 according to this embodiment.

[0017] The image transmission system 1 is a system in which an encoded stream obtained by encoding an image to be encoded is transmitted, the transmitted encoded stream is decoded, and an image is displayed. The image transmission system 1 includes a video encoding device (image encoding device) 11, a network 21, a video decoding device (image decoding device) 31, and a video display device (image display device) 41.

[0018] An image T is input to the video encoding device 11.

[0019] The network 21 transmits the coded stream Te generated by the video coding device 11 to the video decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination thereof. The network 21 is not necessarily limited to a bidirectional communication network, but may be a unidirectional communication network configured to transmit broadcast waves such as terrestrial digital television broadcasting and satellite broadcasting. Furthermore, the network 21 may be replaced by a storage medium on which the coded stream Te is recorded, such as a Digital Versatile Disc (DVD) or a Blu-ray Disc (BD).

[0020] The video decoding device 31 decodes each of the coded streams Te transmitted from the network 21, and generates one or more decoded images Td.

[0021] The video display device 41 displays all or part of one or more decoded images Td generated by the video decoding device 31. Examples of the video display device 41 include display devices such as a liquid crystal display and an organic electroluminescence (EL: Electro-Luminescence) display. Display types include stationary types, mobile types, and HMD types. In addition, if the video decoding device 31 has high processing power, it displays high-quality images, and if the device has lower processing power, it displays images that do not require high processing power and display power.

[0022] operator The operators used in this specification are explained below.

[0023] >> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR, |= is an OR assignment operator, and || represents a logical OR.

[0024] x?y:z is a ternary operator that takes y when x is true (non-zero) and z when x is false (0).

[0025] Clip3(a, b, c) is a function that clips c to a value between a and b, returning a if c is less than a (c < a), b if c is greater than b (c > b), and c otherwise (provided a is less than or equal to b (a <= b)).

[0026] abs(a) is a function that returns the absolute value of a.

[0027] Int(a) is a function that returns the integer value of a.

[0028] floor(a) is a function that returns the largest integer less than or equal to a.

[0029] ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0030] a / d represents the division of a by d (truncating the decimal part).

[0031] Structure of the encoded stream Te Prior to the detailed description of the moving image encoding device 11 and the moving image decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the moving image encoding device 11 and decoded by the moving image decoding device 31 will be described.

[0032] Fig. 4 is a diagram showing the hierarchical structure of a coded stream Te. The coded stream Te illustratively includes one sequence and multiple pictures that make up the sequence. (a) to (f) of Fig. 4 respectively show a coded video sequence that defines the sequence SEQ, coded pictures that specify pictures PICT, coded slices that specify slices S, coded slice data that specify slice data, coding tree units included in the coded slice data, and coding units (CUs) included in each coding tree unit.

[0033] coded video sequence The coded video sequence defines a set of data that the video decoding device 31 refers to in order to decode the sequence SEQ to be processed. As shown in (a) of Fig. 4, the sequence SEQ includes a video parameter set (Video Parameter Set), a sequence parameter set (SPS), a picture parameter set (PPS), a picture PICT, and supplemental enhancement information (Supplemental Enhancement Information) SEI.

[0034] The video parameter set VPS defines a set of coding parameters common to multiple videos including multiple layers, and a set of coding parameters associated with multiple layers and individual layers included in the video.

[0035] The sequence parameter set SPS defines a set of coding parameters that the video decoding device 31 references to decode the target sequence. For example, the width and height of a picture are defined. Note that there may be multiple SPSs. In this case, one of the multiple SPSs is selected from the PPS.

[0036] The picture parameter set PPS defines a set of coding parameters that the video decoding device 31 references to decode each picture in the current sequence. For example, the picture parameter set PPS includes a reference value of the quantization width used in decoding the picture (pic_init_qp_minus26) and a flag indicating the application of weighted prediction (weighted_pred_flag). Note that there may be multiple PPSs. In this case, one of the multiple PPSs is selected for each picture in the current sequence.

[0037] Encoded Picture The coded picture defines a set of data that the video decoding device 31 refers to in order to decode the picture PICT to be processed. As shown in Fig. 2, the picture PICT includes slices 0 to NS-1 (NS is the total number of slices included in the picture PICT).

[0038] Please note that in the following, when there is no need to distinguish between slices 0 to NS-1, the subscripts of the codes may be omitted. This also applies to other subscripted data included in the coded stream Te described below.

[0039] Encoded Slice A coded slice defines a set of data that is referenced by the video decoding device 31 in order to decode a target slice S. As shown in Fig. 2, a slice includes a slice header and slice data.

[0040] The slice header includes a group of coding parameters that the video decoding device 31 refers to in order to determine a decoding method for the current slice. Slice type designation information (slice_type) indicating the slice type is an example of a coding parameter included in the slice header.

[0041] Examples of slice types that can be specified by the slice type specification information include (1) an I slice that uses only intra prediction during encoding, (2) a P slice that uses unidirectional prediction or intra prediction during encoding, and (3) a B slice that uses unidirectional prediction, bidirectional prediction, or intra prediction during encoding. It should be noted that inter prediction is not limited to uni-prediction and bi-prediction, and a predicted image can be generated by using a larger number of reference pictures. Hereinafter, when a slice is referred to as a P slice or a B slice, the slice refers to a slice that includes blocks that can use inter prediction.

[0042] Note that the slice header may contain a reference to a picture parameter set PPS (pic_parameter_set_id).

[0043] Encoded slice data The coded slice data defines a set of data that the video decoding device 31 refers to in order to decode the slice data to be processed. The slice data includes a CTU, as shown in Fig. 2. A CTU is a block of a fixed size (e.g., 64x64) that constitutes a slice, and is sometimes called a Largest Coding Unit (LCU).

[0044] Coding Tree Unit 2 defines a set of data that the video decoding device 31 references in order to decode a CTU to be processed. The CTU is divided into coding units (CUs), which are basic units of encoding processing, by recursive quad tree division (QT (Quad Tree) division), binary tree division (BT (Binary Tree) division), or ternary tree division (TT (Ternary Tree) division). BT division and TT division are collectively referred to as multi tree division (MT (Multi Tree) division). Nodes in the tree structure obtained by recursive quad tree division are called coding nodes. Intermediate nodes in the quad tree, binary tree, and ternary tree are coding nodes, and the CTU itself is also defined as the top-level coding node.

[0045] Encoding Unit 2, a set of data to be referenced by the video decoding device 31 in order to decode a coding unit to be processed is defined. Specifically, the CU includes a CU header (CUH), prediction parameters, transformation parameters, quantization transformation coefficients, etc. The CU header defines a prediction mode, etc.

[0046] The prediction process may be performed on a CU basis, or on a sub-CU basis obtained by further dividing the CU. When the size of the CU and the size of the sub-CU are equal to each other, the number of sub-CUs in the CU is one. When the size of the CU is larger than the size of the sub-CU, the CU is divided into sub-CUs. For example, when the size of the CU is 8x8 and the size of the sub-CU is 4x4, the CU is divided into four sub-CUs, including two horizontal divisions and two vertical divisions.

[0047] There are two types of prediction (prediction modes): intra prediction and inter prediction. Intra prediction refers to prediction within the same picture, while inter prediction is a prediction process performed between different pictures (for example, between pictures with different display times).

[0048] The transform and quantization processes are performed in units of CUs, but entropy coding may be performed on the quantized transform coefficients in units of sub-blocks such as 4×4.

[0049] Forecast Parameters The predicted image is derived from prediction parameters associated with the block, including intra-prediction and inter-prediction parameters.

[0050] The prediction parameters of intra prediction are described below. The intra prediction parameters include a luminance prediction mode IntraPredModeY and a color difference prediction mode IntraPredModeC. FIG. 6 is a schematic diagram showing types of intra prediction modes (mode numbers). As shown in the schematic diagram, there are, for example, 67 types (0 to 66) of intra prediction modes. For example, there are planar prediction (0), DC prediction (1), and angular prediction (2 to 66). Furthermore, for color difference, CCLM (Cross Component Linear Model) prediction modes (81 to 83), MMLM (Multi Mode Linear Model) prediction modes, and LM (Linear Model) prediction modes may be added.

[0051] Configuration of video decoding device The configuration of a video decoding device 31 (FIG. 4) according to this embodiment will be described.

[0052] The video decoding device 31 includes an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit 308, an inverse quantization and inverse transform processing unit 311, an adder 312, and a prediction parameter derivation unit 320. It should be noted that according to the video coding device 11 described later, the video decoding device 31 may also have a configuration in which the loop filter 305 is not included.

[0053] The parameter decoding unit 302 further includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 further includes a TU decoding unit 3024. These may be collectively referred to as a decoding module. The header decoding unit 3020 decodes parameter set information such as the VPS, SPS, and PPS, as well as slice headers (slice information), from the coded data. The CT information decoding unit 3021 decodes the CT from the coded data. The CU decoding unit 3022 decodes the CU from the coded data. If the TU includes a prediction error, the TU decoding unit 3024 decodes QP update information (quantization correction value) and quantized prediction error (residual_coding) from the coded data.

[0054] Furthermore, although an example in which CTUs and CUs are used as processing units will be described below, the processing is not limited to this example, and processing may be performed in units of sub-CUs. Alternatively, processing may be performed in units of blocks or sub-blocks by replacing CTUs and CUs with blocks and sub-CUs with sub-blocks.

[0055] The entropy decoding unit 301 performs entropy decoding on the externally input coded stream Te, separating and decoding individual codes (syntax elements). The separated codes include prediction information for generating a predicted image, prediction errors for generating a differential image, etc. Entropy coding can be performed in two ways: variable-length coding of syntax elements according to a context (probability model) that is adaptively selected depending on the type of syntax element and the surrounding circumstances, or variable-length coding of syntax elements using a predetermined table or formula.

[0056] In the entropy decoding unit 301, in the case of adaptive reference region DIMD (Non-Patent Document 2), the syntax element is named dimd_mode. dimd_mode is a parameter for selecting the reference region of the DIMD method. dimd_mode includes DIMD_MODE_TOP_LEFT mode, DIMD_MODE_TOP mode, and DIMD_MODE_LEFT mode. These three modes are represented by 0, 1, and 2, respectively.

[0057] An example of binarization of dimd_mode is shown in Figure 6. In Figure 6, binId is a variable indicating bit position bin0 (binidx == 0), and the syntax element Bin1 (binidx == 1) refers to the first bit and the next bit.

[0058] Bin0 is a flag used to indicate whether DIMD_MODE_TOP_LEFT is selected. If Bin0 is 0, the DIMD_MODE_TOP_LEFT mode is selected, and if Bin0 is 1, the DIMD_MODE_TOP_LEFT mode is not selected.

[0059] Bin1 is a flag used to indicate whether DIMD_MODE_TOP or DIMD_MODE_LEFT mode is selected. If Bin1 is 0, DIMD_MODE_TOP mode is selected, and if Bin1 is 1, DIMD_MODE_LEFT mode is selected.

[0060] In addition, it is worth noting that Bin0 and Bin1 are not used to form one syntax element, but are used to assign syntax elements to Bin0 and Bin1, respectively. Therefore, dimd_mode can see through two syntax elements. Here, the syntax element assigned to Bin0 is named dimd_mode_flag, and the syntax element assigned to Bin1 is named dimd_mode_dir (shown in FIG. 7). In this case, the entropy decoding unit 301 can calculate dimd_mode from dimd_mode_flag and dimd_mode_dir by the following formula: Also, when dimd_mode_flag is 0, dimd_mode_dir is also set to 0.

[0061] dimd_mode=((dimd_mode_flag==0)?0:1)+dimd_mode_dir In this example, DIMD_MODE_TOP_LEFT is represented by one bit (such as "0"), and one bit after the "0" is assigned to represent DIMD_MODE_TOP and DIMD_MODE_LEFT. When binarizing dimd_mode, for the DIMD_MODE_TOP_LEFT mode, which has a high selection rate, shorter bits are used than for selecting the DIMD_MODE_TOP mode and DIMD_MODE_LEFT mode. This shortens the average code length, improving coding efficiency.

[0062] The parameter decoding unit 302 notifies the entropy decoding unit 301 which syntax elements need to be decoded. The entropy decoding unit 301 outputs the syntax elements to the prediction parameter derivation unit 320.

[0063] Configuration of prediction parameter derivation unit 320 The prediction parameter derivation unit 320 can derive prediction parameters based on the output of the parameter decoding unit 302 and prediction parameters stored in the prediction parameter memory 307. The derived prediction parameters are output to the prediction image generation unit 308 and also stored in the prediction parameter memory 307. The prediction parameter derivation unit 320 can derive different prediction modes for luma (luminance) prediction and chroma (color difference) prediction.

[0064] The loop filter 305 is a filter provided in the encoding loop, which removes blocking noise and ringing noise and improves image quality. The loop filter 305 applies filters such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adder 312.

[0065] The reference picture memory 306 stores the decoded image of the CU generated by the adder 312 for each current picture and current CU in a predetermined location.

[0066] The prediction parameter memory 307 stores prediction parameters in a predetermined location for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores parameters derived by the prediction parameter derivation unit 320, a prediction mode predMode separated by the entropy decoding unit 301, and the like.

[0067] The predicted image generation unit 308 receives input such as prediction parameters derived by the prediction parameter deviation unit 320. In addition, the predicted image generation unit 308 reads a reference picture from the reference picture memory 306. The predicted image generation unit 308 generates a predicted image of a block or sub-block using the prediction parameters and the read reference picture (reference picture block) in a prediction mode indicated by the prediction mode predMode. Here, the reference picture block refers to a set of pixels (usually rectangular, hence called a block) on the reference picture, and is an area referenced to generate a predicted image.

[0068] Predicted image generation unit 310 If the prediction mode predMode indicates an intra prediction mode, the intra prediction image generation unit 310 performs intra prediction by using the intra prediction parameters input from the intra prediction parameter derivation unit 320 and the reference pixels read from the reference picture memory 306.

[0069] Specifically, the intra-prediction image generation unit 310 reads neighboring blocks within a predetermined range from a current block on a current picture from the reference picture memory 306. The predetermined range includes neighboring blocks to the left, upper left, upper, and upper right of the current block, and the referenced area differs depending on the intra-prediction mode.

[0070] The intra-prediction image generation unit 310 generates a prediction image of the current block by referring to the read decoded pixel values ​​and the prediction mode indicated by IntraPredMode. The intra-prediction image generation unit 310 outputs the generated prediction image of the block to the addition unit 312.

[0071] Generation of a predicted image based on an intra prediction mode will be described below. In planar prediction, DC prediction, and angular prediction, a decoded surrounding area adjacent (close to) a block to be predicted is configured as a reference area R. Then, a predicted image is generated by extrapolating pixels in the reference area R in a specific direction. For example, the reference area R may be configured as an L-shaped area including the left and top (or even the upper left, upper right, and lower left) of the block to be predicted.

[0072] Details of the predicted image generation section Next, the configuration of the intra-predicted image generation unit 310 will be described in detail with reference to Fig. 8. The predicted image generation unit 308 includes a filtered reference image setting unit 3103 (second reference image setting unit), an intra-prediction unit 3104, and a predicted image corrector 3105 (predicted image correction unit, filter switching unit, and weighting coefficient changing unit).

[0073] The prediction unit 3104 generates a predicted image of the current block based on each reference pixel (unfiltered reference image) in the reference region R, a filtered reference image generated by applying the reference pixel filter (first filter), and the intra prediction mode, and outputs the generated image to the predicted image corrector 3105. The predicted image corrector 3105 corrects the predicted image according to the intra prediction mode, and outputs a corrected predicted image.

[0074] Each unit included in the intra-prediction image generation unit 310 will be described below.

[0075] Filtered reference image setting unit 3103 The filtered reference image setting unit 3103 applies a reference pixel filter (first filter) to the unfiltered reference image in accordance with the intra prediction mode to derive a filtered reference image s[x][y] at each position (x, y) on the reference region R. Specifically, a low-pass filter is applied to the unfiltered reference image at each position (x, y) and its surroundings to derive a filtered reference image. It should be noted that a low-pass filter does not necessarily need to be applied in all intra prediction modes, and a low-pass filter may be applied in some intra prediction modes. It should be noted that the filter applied to the unfiltered reference image on the reference region R by the filtered reference image setting unit 3103 is called a "reference pixel filter (first filter)," while the filter that corrects the predicted image in the predicted image corrector 3105, which will be described later, is called a "boundary filter (second filter)."

[0076] Configuration of the intra prediction unit 3104 The intra prediction unit 3104 generates a prediction image (predicted pixel value, uncorrected predicted image) of the block to be predicted based on the intra prediction mode, the unfiltered reference image, and the filtered reference pixel value, and outputs the generated image to the prediction image corrector 3105. The prediction unit 3104 internally includes a planar prediction unit 31041, a DC prediction unit 31042, an angular prediction unit 31043, an LM prediction unit 31044, a matrix-based intra prediction (MIP) prediction unit 31045, and a decoder-side intra mode derivation (DIMD) prediction unit 31046. The predictor 3104 selects a specific predictor according to the intra prediction mode, and inputs the unfiltered reference image and the filtered reference image thereto. The relationship between the intra prediction mode and the corresponding predictor is as follows: ·Planar Prediction···Planar Prediction Section 31041 DC Prediction DC Prediction Unit 31042 ·Angular Prediction··Angular Prediction Part 31043 ·LM Prediction··LM Prediction Unit 31044 ·MIP prediction···MIP prediction unit 31045 ·DIMD Prediction··DIMD Prediction Unit 31046

[0077] Planar forecast The planar prediction unit 31041 generates a predicted image q[x][y] by linearly adding multiple filtered reference images s[x][y] according to the distance between the pixel position to be predicted and the reference pixel position, and outputs the generated image to the predicted image corrector 3105.

[0078] DC forecast The DC prediction unit 31042 derives a DC predicted value corresponding to the average value of the filtered reference image s[x][y], and outputs a predicted image q[x][y] that takes the DC predicted value as a pixel value.

[0079] Angular Prediction The angular prediction unit 31043 generates a predicted image q[x][y] using a filtered reference image s[x][y] in the prediction direction (reference direction) indicated by the intra prediction mode, and outputs the generated image to the predicted image corrector 3105.

[0080] LM forecast The LM prediction unit 31044 predicts chrominance pixel values ​​based on luminance pixel values. More specifically, a linear model is used to generate a predicted chrominance image (Cb, Cr) based on the decoded luminance image. An example of LM prediction is CCLM (Cross Component Linear Model Prediction). CCLM prediction is a prediction method that uses a linear model to predict chrominance from luminance for the same block.

[0081] MIP prediction The MIP prediction unit 31045 generates a predicted image q[x][y] by performing a product-sum operation on the reference sample s[x][y] and a weighting matrix derived from the surrounding area, and outputs the predicted image q[x][y] to the predicted image correction unit 3105.

[0082] DIMD forecast The dimd prediction unit 31046 generates a predicted image using an intra prediction mode adapted to the current block. An intra prediction mode suitable for the current block is derived using information about the neighboring region, and the dimd prediction unit 31046 generates a predicted image using this intra prediction mode. Details will be described later.

[0083] Configuration of the predicted image corrector 3105 The predicted image corrector 3105 corrects the predicted image output from the predictor 3104 according to the intra prediction mode. Specifically, the predicted image corrector 3105 derives a predicted image (corrected predicted image) Pred in which the predicted image has been corrected by performing weighted addition (weighted averaging) on ​​the unfiltered reference image and the predicted image for each pixel of the predicted image according to the distance between the reference region R and the target predicted pixel. It should be noted that in some intra prediction modes (e.g., planar prediction, DC prediction, etc.), the predicted image corrector 3105 may not correct the predicted image, and the output of the predictor 3104 may be used as the predicted image.

[0084] Application Examples 9 shows the configuration of the DIMD prediction unit 31046 in this embodiment. The DIMD prediction unit 31046 includes a reference sample derivation unit 310460, an angle derivation unit 310465, an angle mode selection unit 310463, and a predicted image generation unit 310464. The angle derivation unit 310465 includes a gradient derivation unit 310461 and an angle mode derivation unit 310462. The gradient derivation unit 310461 includes a filter selection unit 3104611. The angle mode derivation unit 310462 may include the angle mode selection unit 310463 in some circumstances.

[0085] 5 shows an example of encoded data related to the DIMD method. For each current block, the prediction parameter derivation unit 320 decodes a flag named dimd_flag, which is used to indicate whether the current block uses the DIMD method. When dimd_flag is 1, some syntax elements related to intra prediction (intra_mip_flag, intra_luma_mpm_flag, intra_luma_mpm_idx, intra_luma_mpm_reminder) are not decoded by the parameter decoding unit 302. intra_mip_flag indicates whether MIP prediction is used. intra_luma_mpm_flag is a flag indicating whether Most Probable Mode (MPM) is used. intra_luma_mpm_idx is an index indicating MPM candidates when MPM is used. intra_luma_mpm_reminder is an index for selecting an intra prediction mode from the remaining modes when MPM is not used. When dimd_flag is 0, intra_luma_mpm_flag is decoded, and when intra_luma_mpm_flag is 0, intra_luma_mpm_reminder is decoded. When dimd_flag of the current block is 1, the dimd_mode flag of the current block is decoded. The dimd_mode flag is a flag for indicating a reference region used to derive an intra prediction mode. The dimd_mode flag can be set to the following values.

[0086] dimd_mode=0 DIMD_MODE_TOP_LEFT (use top adjacent reference area and left adjacent reference area) dimd_mode=1 DIMD_MODE_LEFT (use left adjacent reference area) dimd_mode=2 DIMD_MODE_TOP (use top adjacent reference area) When dimd_flag is 1, the DIMD predictor 31046 derives an angular mode indicating the texture direction of the neighboring region indicated by the pixel value. This angular mode is used to generate an intra-predicted image. More specifically, (step 1) derives a pixel gradient using the pixel value at a given location, (step 2) converts the derived pixel gradient to an angular prediction mode, and (step 3) constructs a histogram using all angular prediction modes obtained from steps 1 and 2. (step 4) selects one or more angular prediction modes from the histogram and uses them to generate a predicted image. Figure 9 shows the DIMD predictor 31046, and each part of the DIMD predictor 31046 and the operation of each part are described in more detail below.

[0087] Reference sample derivation part The reference sample derivation unit 310460 derives the reference sample refUnit from previously decoded pixels recSamples that neighbor the current block. Note that the reference sample derivation unit 310460 may be included in the reference sample filter unit 3103. Figure 10 shows an example of the reference region of the dimd prediction unit 31046. The reference sample derivation unit 310460 stores recSamples that neighbor the current block and are used by the gradient derivation unit 310461 and the predicted image generation unit 308 in the reference sample refUnit.

[0088] Example of derivation of reference region selection based on dimd_mode When dimd_mode==DIMD_MODE_TOP_LEFT, the reference sample derivation unit 310460 derives reference samples refUnit from the left and top regions of the current block as follows.

[0089] First, the following process is executed at the position (x, y) where the left region of the target block (hereinafter, the left region will be referred to as RL) is located. refUnit[x][y]=recSamples[xC+x][yC+y] Here, RL has a range of (x=-1-refIdxW...-1, y=0...refH-1). (xC, yC) are the top-left coordinates of the target block, and refIdxW is a constant indicating the width of the reference area to the left of the target block. refIdxW may be 2, and refH is equal to the height (i.e., bH) of the target block.

[0090] Second, the following process is performed in the RT, which means the upper region of the target block. refUnit[x][y]=recSamples[xC+x][yC+y] Here, RT has a range of (x=0...refW-1, y=-1-refIdxH...-1). refIdxH is a constant indicating the height of the reference area at the top of the target block. refIdxH may be 2, and refW is equal to the width (i.e., bW) of the target block.

[0091] Third, to derive a reference sample in the upper left region of the target block, the following process is performed: refUnit[x][y]=recSamples[xC+x][yC+y] where x = -1-refIdxW...-1, y = -1-refIdxH...-1 (x and y cannot be equal to -1 at the same time). The RTL is an area that includes all three of the above areas.

[0092] When dimd_mode==DIMD_MODE_LEFT, the reference sample derivation unit 310460 derives the left adjacent region and the bottom left adjacent region of the current block. refUnit[x][y]=recSamples[xC+x][yC+y] where x=-1-refIdxW...-1, y=-1-refIdxH...refH * The score is 2-1. In this case, refIdxW and refIdxH are constants that respectively indicate the width and height of the reference area on the left side of the target block. refIdxW and refIdxH may be equal to 3, and refH is equal to the height (i.e., bH) of the target block.

[0093] When dimd_mode==DIMD_MODE_TOP, the reference sample derivation unit 310460 derives the top neighboring region and the upper right neighboring region of the current block. refUnit[x][y]=recSamples[xC+x][yC+y] where x=-1-refIdxW...refW * 2-1,y=-1-refIdxH...-1. In this case, refIdxW and refIdxH are constants that respectively indicate the width and height of the reference region on top of the target block. refIdxW and refIdxH may be 3, and refW is equal to the width (i.e., bW) of the target block.

[0094] Gradient derivation part The gradient derivation unit 310461 derives gradient values ​​in two or more specific directions (e.g., Dx and Dy) based on pixel values ​​of blocks adjacent to the target block, and derives angle parameters representing the orientation of the texture pattern from the gradient values. The accuracy of the angle parameters is, for example, approximately 90 / 16 degrees.

[0095] A spatial filter is used to derive the gradient. The spatial filter may be a 3x3 pixel filter or a 2x2 pixel filter corresponding to the horizontal and vertical directions, as shown in Figure 11 (a, b, e, f). The gradient derivation unit 310461 derives the gradient value of a point P[x][y] (hereinafter simply referred to as P) in the reference sample refUnit[x][y] derived by the reference sample derivation unit 310460. Instead of the point refUnit[x][y], the point recSamples[xC+x][yC+y] can be used as the point P.

[0096] 12 and 13 show examples of gradient target images for a current block of 8x8 pixels for a 2x2 filter and a 3x3 filter, respectively. The pixels in the gradient target image are used to calculate the gradient. When the angle mode derivation unit 310462 is used for intra prediction, the mesh image in the neighboring region of the current block is the gradient target image of the current block (shown in FIGS. 12 and 13). The gradient target image is a luminance image of the current block. The number of pixels used for gradient derivation to select the angle mode (modeVal) can be changed depending on the size of the current block (bW, bH). The number of pixels used for gradient derivation is determined by the spatial filter. Therefore, the number of taps of the spatial filter (spatial filter size) can be changed depending on the size of the current block.

[0097] The filter selection unit 3104611 determines the size of the filter (derives the parameter filterIdx) based on the size (bW, bH) of the target block. The variable filterIdx is defined to indicate the currently selected filter. The value of filterIdx is calculated using the following formula: filterIdx=(bW<=TW1&&bH<=TH1)?0:1 In the formula, TW1=16 and TH1=16. In other words, the filter selector 3104611 determines the size of the filter (e.g., whether to use a 2x2 filter or a 3x3 filter) according to the size of the target block. A smaller 2x2 filter is used for small target blocks whose width and height are 16 or less, and a larger 3x3 filter is used for non-small (larger) target blocks having other sizes. The gradient derivation unit 310461 derives the horizontal and vertical gradients Dx and Dy for each point P based on filterIdx as follows: Thus, the gradient derivation unit 310461 derives Dx and Dy according to the size of the target block, with larger filters being used for larger blocks.

[0098] If filterIdx==0 (for small target blocks), a 2x2 filter is applied as follows: Dx=P[x][y]+P[x][y+1]-P[x+1][y]-P[x+1][y+1] Dy=P[x][y]+P[x+1][y]-P[x][y+1]-P[x+1][y+1] (For other target blocks) If filterIdx==1, a 3x3 filter is applied as follows: Dx=P[x-1][y+1]+2 * P[x][y+1]+P[x+1][y+1]-P[x-1][y-1]-2 * P[x][y-1]-P[x+1][y-1] Dy=P[x-1][y-1]+2 * P[x-1][y]+P[x-1][y+1]-P[x+1][y-1]-2 * P[x+1][y]-P[x+1][y+1] Note that a 2x2 filter can use four positions: (x,y), (x+1,y), (x,y+1), (x+1,y+1), whereas a 3x3 filter can use nine positions: (x-1,y-1), (x,y-1), (x+1,y-1), (x-1,y), (x,y), (x+1,y), (x-1,y+1), (x,y+1), (x+1,y+1). A 4x4 filter can use 16 positions: (x-1,y-1), (x,y-1), (x+1,y-1), (x+2,y-1), (x-1,y), (x,y), (x+1,y), (x+2,y), (x-1,y+1), (x,y+1), (x+1,y+1), (x+2,y+1), (x-1,y+2), (x,y+2), (x+1,y+2), (x+2,y+2). Similarly, 2x3 and 3x2 filters can use six positions: (x,y-1), (x+1,y-1), (x,y), (x+1,y), (x,y+1), (x+1,y+1), and (x-1,y), (x,y), (x+1,y), (x-1,y), (x,y), (x+1,y), (x-1,y+1), (x,y+1), (x+1,y+1), respectively. In general, MxN filters can use (x+xx,y+yy), xx=-(M-1) / 2..M / 2, yy=-(N-1) / 2..N / 2, where / denotes integer division with the result truncated towards zero. The signs of each component in the above equation can be reversed, i.e., filter (a, b, e, f) in Figure 11 can be rotated 180 degrees to become filter (c, d, g, h) in Figure 11, which can be used to derive the gradient. In this case, Dx and Dy are derived using the following equations: If filterIdx==0 (for small target blocks), a 2x2 filter is applied as follows: Dx=P[x+1][y]+P[x+1][y+1]-P[x][y]-P[x][y+1] Dy=P[x][y+1]+P[x+1][y+1]-P[x][y]-P[x+1][y] (For other target blocks) If filterIdx==1, a 3x3 filter is applied as follows: Dx=P[x-1][y-1]+2* P[x][y-1]+P[x+1][y-1]-P[x-1][y+1]-2 * P[x][y+1]-P[x+1][y+1] Dy=P[x+1][y-1]+2 * P[x+1][y]+P[x+1][y+1]-P[x-1][y-1]-2 * P[x-1][y]-P[x-1][y+1]

[0099] The method of deriving the gradient is not limited to the above, and other methods can also be used. Other threshold examples The thresholds (TW1, TH1) may be set equal to (32, 32), (16, 32), (32, 16), and so on. "<=" can be replaced with "<" as follows: filterIdx=(bW <TW1&&bH<TH1)?0:1 The target block size can be determined based on bW+bH. filterIdx=(bW+bH<=TWH1)?0:1 The threshold TWH1 may be set equal to 16, 32, 64, and so on. The target block size is determined by bW * This can be done based on bH. filterIdx=(bW * bH<=TWH2)?0:1 The threshold TWH2 may be set equal to 256, 512, 1024, etc. The target block size determination can be based on log2(bW)+log2(bH). filterIdx=(log2(bW)+log2(bH)<=TWH3)?0:1 The threshold TWH3 may be equal to 8, 9, 10, etc. It should be noted that the threshold value is not limited to the above values.

[0100] Three filters The filter selection unit 3104611 can use two thresholds to select a filter from three types: large, medium, and small, depending on the size of the target block. The filter selection unit 3104611 selects a 4x4 filter, a 3x3 filter, and a 2x2 filter for large, medium, and small target blocks, respectively. The filter size can be set as follows: filterIdx=(bW <TW1&&bH<TH1)?0:(bW<TW2&&bH<TH2)?1:2 wherein TW1, TH1, TW2, and TH2 can be 16, 16, 32, and 32.

[0101] Rectangular Filter In addition to square filters, rectangular (non-square) filters can also be used. Figure 11 shows two groups of rectangular filters (i, j, m, l) as examples. Take filter (i, j) shown in Figure 11 as an example. This is a rectangular filter with a width of 3 and a height of 2 (a 3x2 filter). The corresponding Dx and Dy can be calculated according to the following formula: Dx=P[x-1][y+1]+2 * P[x][y+1]+P[x+1][y+1]-P[x-1][y]-2 * P[x][y]-P[x+1][y] Dy=2 * P[x-1][y]+2 * P[x-1][y+1]-2 * P[x+1][y-1]-2 * P[x+1][y+1] Similarly, the formula for deriving the gradient of the filter (2×3 filter) (m, l) shown in FIG. 11 is as follows: Dx=2 * P[x][y-1]+2 * P[x+1][y-1]-2 * P[x][y+1]-2 * P[x+1][y+1] Dy=P[x][y-1]+2 * P[x][y]+P[x][y+1]-P[x+1][y-1]-2 * P[x+1][y]-P[x+1][y+1] The rectangular filter is not limited to only the above two types, and other types of rectangular filters may also be used. Based on the block size, the filter selection unit 3104611 determines the size of the filter according to the size (bW, bH) of the target block. filterIdx=(bW<=8&&bH<=8)?0:((bW<=16&&bH<=16)?(bW>bH?3:4):1) The gradient derivation unit 310461 derives Dx and Dy according to the derived filterIdx. When filterIdx==0 (small target block, for example, bW<=8&&bH<=8), a 2×2 filter is applied. Dx=P[x][y]+P[x][y+1]-P[x+1][y]-P[x+1][y+1] Dy=P[x][y]+P[x+1][y]-P[x][y+1]-P[x+1][y+1] Otherwise, when filterIdx==3 (rectangular block longer in the horizontal direction, bW>bH&&bW<=16&&bH<=16), a 3×2 filter is applied. Dx=P[x-1][y+1]+2 * P[x][y+1]+P[x+1][y+1]-P[x-1][y]-2 * P[x][y]-P[x+1][y] Dy=2 * P[x-1][y]+2 * P[x-1][y+1]-2 * P[x+1][y-1]-2 * P[x+1][y+1] Otherwise, when filterIdx==4 (rectangular block longer in the vertical direction, bW<bH&&bW<=16&&bH<=16), a 2×3 filter is applied. Dx=2 * P[x][y-1]+2 * P[x+1][y-1]-2 * P[x][y+1]-2 * P[x+1][y+1] Dy=P[x][y-1]+2 *P[x][y]+P[x][y+1]-P[x+1][y-1]-2 * P[x+1][y]-P[x+1][y+1] Otherwise, if filterIdx==1 (other blocks), a 3x3 filter is applied. Dx=P[x-1][y+1]+2 * P[x][y+1]+P[x+1][y+1]-P[x-1][y-1]-2 * P[x][y-1]-P[x+1][y-1] Dy=P[x-1][y-1]+2 * P[x-1][y]+P[x-1][y+1]-P[x+1][y-1]-2 * P[x+1][y]-P[x+1][y+1]

[0102] The filter selector 3104611 can determine the size of the filter based on the dimd_mode. If both the top neighboring block and the left neighboring block are referenced, a larger filter (e.g., 3x3) is used; otherwise, a smaller filter (e.g., 2x2) is used. filterIdx=(dimd_mode!=DIMD_MODE_TOP_LEFT)?0:1 The gradient derivation unit 310461 derives Dx and Dy according to the derived filterIdx.

[0103] To reduce complexity, the filter selector 3104611 can determine the size of the filter based on the dimd_mode in another way: using a smaller block in the case of a reference image (e.g., DIMD_MODE_TOP_LEFT). If both the top and left neighboring blocks are referenced, a larger filter (e.g., 2x2) is used; otherwise, a smaller filter (e.g., 3x3) is used. filterIdx=(dimd_mode==DIMD_MODE_TOP_LEFT)?0:1 The gradient derivation unit 310461 derives Dx and Dy according to the derived filterIdx.

[0104] The filter selector 3104611 can determine the size of a filter including a rectangular kernel (e.g., 3x2, 2x3) based on the dim_mode. If both the top and left neighboring blocks are referenced, a larger filter (e.g., 2x2) is used; otherwise, a smaller filter (e.g., 3x2) is used for DIMD_MODE_LEFT, and another smaller filter (e.g., 2x3) is used for DIMD_MODE_TOP. filterIdx=(dimd_mode==DIMD_MODE_TOP_LEFT)?0:(dimd_mode==DIMD_MODE_LEFT)?3:4

[0105] The gradient derivation unit 310461 derives angle information consisting of an angle quadrant of the texture of the current block and an angle within this quadrant according to the relationship between Dx and Dy. Using the quadrant makes it possible to combine processing in several directions having a rotationally symmetric or line-symmetric relationship. However, the angle information is not limited to the quadrant and the angle within the quadrant. For example, the angle information can be set as an angle only, and the quadrant is derived from this angle as needed. In addition, in this embodiment, the derived intra-directional prediction mode is limited to the direction from the bottom left to the top right (2 to 66 in FIG. 3), and the intra-directional prediction mode in a 180-degree rotationally symmetric direction is the same.

[0106] 14(a) is a table showing the relationship between the signs (signx, signy) of Dx and Dy, the size relationship xgty, and the quadrants (Ra to Rd). 14(b) shows the quadrants Ra to Rd. The gradient derivation unit 310461 derives signx, signy, and xgty as follows: absx=abs(Dx) absy=abs(Dy) signx=Dx<0?1:0 signy=Dy<0?1:0 xgty=absx>absy?1:0 Here, the inequality signs (>, <) can be replaced by (>=, <=). The angle information can be derived from signx, signy, and xgty.

[0107] The gradient derivation unit 310461 derives the quadrant from signx, signy, and xgty using the following calculation or table reference: The gradient derivation unit 310461 may derive the quadrant by referring to the table in FIG. 14(a). The gradient derivation unit 310461 may also derive the quadrant using the following formula: quadrant=xgty?((signx^signy)?1:0):((signx^signy)?2:3) Here, ^ is an XOR operation. The quadrants are represented by values ​​from 0 to 3: {Ra, Rb, Rc, Rd} = {0, 1, 2, 3}. The values ​​of the quadrants are not limited to the above.

[0108] Angle mode derivation section The angular mode derivation unit 310462 derives an angular mode (a prediction mode corresponding to the gradient information, for example, an intra prediction mode) based on the gradient information of the point P[x][y]. FIG. 15 is a block diagram showing one configuration of the angular mode derivation unit 310462. As shown in FIG. 15, the angular mode (mode_delta or modeVal) is derived using a first gradient, a second gradient, and two tables.

[0109] The angle mode derivation unit 310462 includes an angle coefficient derivation unit 310466 and a mode conversion unit 310467. The angle coefficient derivation unit 310466 derives an angle coefficient iRatio (or v) based on two gradients. Here, the gradient iRatio (=absy÷absx) is derived based on the absolute value of the first gradient (absx) and the absolute value of the second gradient (absy). iRatio can be approximately expressed by ratio and R_UNIT as follows: iRatio=int(R_UNIT * absy / absx)=ratio * R_UNIT The R_UNIT is a power of 2 (1<<shiftR), for example, when shiftR = 16, R_UNIT = 65536.

[0110] The method for deriving iRatio will be described below, but the derivation method is not limited to this example.

Table 1

[0111] x is the integer part of the logarithm value (absx or absy) of the third gradient value s1 of the pixel. By performing a shift operation on x using the third gradient s1, norm_s1 is derived. The angular coefficient v is determined by using norm_s1 and the reference table gradDivTable. Furthermore, iRatio is derived by multiplying v and the fourth gradient s0 using a shift operation based on x. The angular mode mode_delta is output using iRatio and angTable. To ensure that iRatio does not exceed the range of the number of entries in the table LUT, the following clip operation is performed. iRatio = min((s0 * v) << 3 >> x, N_LUT - 1) In addition, this is also appropriate to ensure that the value of s0 * v does not exceed a predetermined value KK. For example, it does not exceed 32 bits. At this time, s0 * v = (min(s0* v,KK)<<3)>>x, and KK=(1<<(31-3))-1=268435455. Note that iRatio may be derived by reversing the definitions of s0 and s1.

[0112] The mode conversion unit 310467 uses mode_delta to derive and output the angle mode modeVal. modeVal=base_mode[quadrant]+direction[quadrant] * mode_delta The number of occurrences of modeVal is recorded in a histogram (HistMode[]), which can be calculated by adding 1 to the occurrence value corresponding to modeVal (remaining in this specification, this operation will be referred to as "histogram count"). HistMode[modeVal]+=1

[0113] Angle mode selection section The angle mode selector 310463 derives one or more angle modes dimdModeVal (dimdmodev0, dimdmodeva1, ...) by using a histogram derived from modeVal (modeVal) calculated using multiple points P belonging to the gradient derivation target image. In this embodiment, dimdModeVal is an estimate of the dominant texture direction of the target block. dimdModeVal is derived by finding the most frequent value (mode) in the histogram. In the histogram, a first mode dimdModeVal0 is derived by selecting the mode with the highest frequency, and a second mode dimdModeVal1 is derived by selecting the mode with the second highest frequency. More specifically, HistMode[x] is scanned to find the x with the maximum value of HistMode to determine dimdModeVal0 (=argmax(HistMode)), and the x with the second largest value of HistMode to determine dimdModeVal1.

[0114] [Table 2] The method of deriving dimdModeVal0 or dimdModeVal1 is not limited to the histogram. For example, the angle mode selection unit 310463 can set the average value of modeVal to dimdModeVar0 or dimdModeVal1.

[0115] The angle mode selector 310463 sets a predetermined mode, dimdModeVal2, as the third mode. In this embodiment, the third mode is set equal to the planar mode (0), but is not limited to the planar mode. Another mode may be adaptively set as the third mode, or the third mode may not be used.

[0116] The angular mode selection unit 310463 may derive weights for the above three modes to be used in the predicted image generation unit 310464. The overall weight is set to 64, a weight for the third mode is assigned (W2=21), and the remainder is distributed to weights W0 and W1 according to the ratio of the frequency of the first mode to the frequency of the second mode in the histogram. The weighting of the first mode, second mode, and third mode is not limited to this, and the weights W0, W1, and W2 of the first mode, second mode, and third mode may be changed. For example, W1 may be increased or decreased. Note that when the angular mode selection section sets a weight to 0, it means that the mode corresponding to this weight is not used.

[0117] Configuration of adaptive gradient derivation unit 310461 and angle mode derivation unit 310465 As described above, in this embodiment, the reference region of the reference image used to derive the intra prediction mode is changed according to dimd_mode. Specifically, the position of point P used in the gradient derivation unit 310461, the angle mode derivation unit 310462, and the angle mode selection unit 310463 is changed according to dimd_mode.

[0118] The position range (x, y) used for the histogram counts to derive the gradient and angle modes is located within the reference regions RL, RT, and RTL. For a 3x3 filter, the starting point is generated by increasing the range of the gradient derivation by 1, and the end point is generated by decreasing it by 1. For a 2x2 filter (filterIdx==0), the starting point is generated without any change in the range of the gradient derivation, and the end point is generated by decreasing it by 1. That is, for a 3x3 filter (filterIdx==1), if the reference range of the dimd prediction is x=x0..x1, y=y0..y1, the range of the gradient derivation can be x0+1..x1-1, y=y0+1..y1-1. For a 2x2 filter (filterIdx==0), if the reference range for dimd prediction is x=x0..x1, y=y0..y1, then the range for gradient derivation can be x=x0..x1-1, y=y0..y1-1. The ranges (x,y) for gradient derivation corresponding to RL, RT and RTL are called RDL, RDT, RDTL.

[0119] Example of selecting a reference area based on dimd_mide 16(b) shows an example of a reference range used in the gradient derivation process for dimd prediction. In this case, a 3×3 filter (filterIdx==1) is used.

[0120] When dimd_mode==DIMD_MODE_TOP_LEFT, the angle mode derivation unit 310462 derives Dx and Dy from each point P in the left region RDL of the target block, derives modeVal, and performs histogram count calculation. Next, it derives Dx and Dy from each point P in the region RDT of the target block, derives modeVal, and performs histogram count. The area of ​​the RDL is x=-refIdxW..-2, y=-refIedxH..refH-2. The domain of the RDT is x=-refIdxW..refW-2, y=-refIdxH..-2. RDTL is a region that combines RDL and RDT.

[0121] When dimd_mode==DIMD_MODE_LEFT, the angle mode derivation unit 310462 uses the extended left region RDL_EXT of the target block, for example, Dx and Dy from RDL_EXT, to derive and calculate modeVal. The area of ​​RDL_EXT is x=-refIdxW..-2, y=-refIedxH..refH * The score is 2-2. When mode==DIMD_MODE_TOP, the angle mode derivation unit 310462 derives modeVal from the extended upper region RDT_EXT of the target block and performs histogram counting. The RDT_EXT area is x=-refIdxW..refW * 2-2,y=-refIdxH..-2.

[0122] Here, refIdxW=2, refIedxH=2, refH=bH (height of the target block), and refW=bW (width of the target block).

[0123] Similarly, when using a 2x2 filter, the reference region for gradient derivation is shown in Figure 16(a). When using a 2x2 filter: The area of ​​the RDL is x=-refIdxW..-2, y=-refIedxH..refH-2. The domain of the RDT is x=-refIdxW..refW-2, y=-refIdxH..-2. RDTL is a region that combines RDL and RDT. The area of ​​RDL_EXT is x=-refIdxW..-2, y=-refIedxH..refH * The score is 2-2. The RDT_EXT area is x=-refIdxW..refW * 2-2,y=-refIdxH..-2. Here, refIdxW=2, refIedxH=2, refH=bH (height of the target block), and refW=bW (width of the target block).

[0124] The angle mode selection unit 310463 has a function of selecting an angle mode from multiple angle modes derived from the pixels of the gradient derivation target image, and therefore can obtain an angle mode with higher accuracy. As described above, the angle mode selection unit 310463 selects an angle mode estimated from the gradient and outputs it using a weight corresponding to each angle mode.

[0125] Predicted image generation unit 310464 The predicted image generation unit 310464 generates a predicted image using two or three intra prediction modes, setting one of the intra prediction modes to planar mode, and the other one or two intra prediction modes to angle modes input from the angle mode selection unit 310463. First, it generates predicted images (pred0, pred1, pred2) according to each intra prediction mode. Second, it synthesizes these predicted images using corresponding weights (w0, w1, w2) and outputs them as a predicted image q[x][y]. The predicted image q[x][y] is derived as follows. q[x][y]=(w0 * pred0[x][y]+w1 * pred1[x][y]+w2 * pred2[x][y])>>6 However, if the frequency of the third mode is 0, or if it is not a directional prediction mode (for example, DC mode (the representative number is 1)), the predicted image q[x][y] is generated by the first mode and the second mode as follows: q[x][y]=(w0 * pred0[x][y]+w1 * pred1[x][y])>>6 Here, the first and second modes are angular modes, and the third mode is a planar mode. w1 is 21, the sum of w0 and w2 is 43, and when w2 is 0, w0 is 43.

[0126] Configuring dimd mode decoding The inverse quantization and inverse transform processing unit 311 performs inverse quantization on the quantized transform coefficients input from the prediction parameter derivation unit 320 to calculate transform coefficients. These quantized transform coefficients are coefficients obtained by performing frequency transforms such as a discrete cosine transform (DCT) or a discrete sine transform (DST) on prediction errors in order to quantize them in the encoding process. The inverse quantization and inverse transform processing unit 311 performs inverse frequency transforms such as an inverse DCT or an inverse DST on the calculated transform coefficients to calculate prediction errors. The inverse quantization and inverse transform processing unit 311 outputs the prediction errors to the adder 312.

[0127] 17 is a block diagram showing the configuration of the inverse quantization and inverse transform processing unit 311 of this embodiment. The inverse quantization and inverse transform processing unit 311 includes a scaling unit 31111, an inverse non-separable transform unit 31121, and an inverse separable transform unit 31123. The inverse quantization and inverse transform unit 311 transforms transform coefficients decoded from encoded data by using an angle mode derived by an angle derivation device 310465.

[0128] The inverse quantization and inverse transform unit 311 obtains the transform coefficients d[][] by using a scaling unit 31111 that scales the quantized transform coefficients qd[][] input from the prediction parameter derivation unit 320. The quantized transform coefficients qd[][] are coefficients obtained by the quantization process. The quantization operation is performed by performing a transform operation, such as a DCT (Discrete Cosine Transform) or a DST (Discrete Sine Transform), on the prediction error in the encoding process. In some cases, a non-separable transform may be performed to obtain the quantized transform coefficients qd[][]. If the non-separable transform flag Ifnst_idx != 0, the inverse quantization and inverse transform unit 311 performs an inverse non-separable transform using the inverse non-separable transform unit 31121. Furthermore, the inverse quantization and inverse transform unit 311 performs an inverse frequency transform, such as an inverse DCT or an inverse DST, by referring to the transform coefficients to calculate the prediction error. When the non-separable transform flag Ifnst_idx !=0, inverse frequency transforms such as inverse DCT and inverse DST are calculated directly, and prediction errors are calculated without using the inverse non-separable transform unit 31121. The inverse quantization and inverse transform unit 311 outputs the prediction errors to the adder 312.

[0129] The adder 312 generates a decoded image for the block by adding, for each pixel, the predicted image of the block input from the predicted image generation unit 308 and the prediction error input from the inverse quantization and inverse transform processing unit 311. The adder 312 stores the decoded image of the block in the reference picture memory 306 and outputs the image to the loop filter 305.

[0130] Configuration of video encoding device Next, the configuration of the video encoding device 11 according to this embodiment will be described. Fig. 18 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 is configured to include a prediction image generation unit 101, a subtraction unit 102, a transformation and quantization unit 103, an inverse quantization and inverse transformation processing unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, a coding parameter determination unit 110, a parameter coding unit 111, a prediction parameter derivation unit 120, and an entropy coding unit 104.

[0131] The predicted image generation unit 101 generates a predicted image for each CU, which is an area obtained by dividing each picture of the image T. The operation of the predicted image generation unit 101 is the same as that of the predicted image generation unit 308 already described, and therefore a description thereof will be omitted.

[0132] The subtraction unit 102 generates a prediction error by subtracting pixel values ​​of the predicted image of the block input from the predicted image generation unit 101 from pixel values ​​of the image T. The subtraction unit 102 outputs the prediction error to the transformation and quantization unit 103.

[0133] The transform / quantization unit 103 calculates transform coefficients by performing frequency transform on the prediction errors input from the subtraction unit 102, and derives quantized transform coefficients by quantization. The transform / quantization unit 103 outputs the quantized transform coefficients to the entropy coding unit 104 and the inverse quantization / inverse transform processing unit 105.

[0134] The inverse quantization and inverse transform processing unit 105 is the same as the inverse quantization and inverse transform processing unit 311 (FIG. 4) in the video decoding device 31, and a description thereof will be omitted. The calculated prediction error is output to the adder .

[0135] The entropy coding unit 104 receives the quantized transform coefficients from the transform / quantization unit 103 and the coding parameters from the parameter coding unit 111. The entropy coding unit 104 performs entropy coding on the division information, prediction parameters, quantized transform coefficients, etc., to generate and output a coded stream Te.

[0136] The parameter coding unit 111 instructs the entropy coding unit 104 to code the prediction parameters and quantization coefficients derived from the prediction parameter derivation unit 120 .

[0137] The prediction parameter derivation unit 120 derives syntax elements from the parameters input from the coding parameter determination unit 110. Some parts of the prediction parameter derivation unit 120 have the same structure as the prediction parameter derivation unit 320.

[0138] The adder 106 generates a decoded image by adding, for each pixel, the pixel value of the predicted image of the block input from the predicted image generation unit 101 and the prediction error input from the inverse quantization and inverse transform processing unit 105. The adder 106 stores the generated decoded image in the reference picture memory 109.

[0139] The loop filter 107 applies a deblocking filter, SAO, and ALF to the decoded image generated by the adder 106. It should be noted that the loop filter 107 does not necessarily need to include the above-mentioned three types of filters, and may, for example, have a configuration of only a deblocking filter.

[0140] The prediction parameter memory 108 stores the prediction parameters generated by the prediction parameter derivation unit 120 in a predetermined location for each current picture and CU. The prediction parameter memory may also store transform coefficients generated by the transform / quantization unit 103.

[0141] The reference picture memory 109 stores the decoded image generated by the loop filter 107 in a predetermined location for each current picture and CU.

[0142] The coding parameter determination unit 110 selects one set of coding parameters from among multiple sets of coding parameters. The coding parameters are the above-mentioned QT, BT, or TT division information, prediction parameters, or parameters to be coded that are generated in relation to them. The predicted image generation unit 101 generates a predicted image by using these coding parameters.

[0143] The coding parameter determination unit 110 calculates an RD cost value indicating the magnitude of the information amount and the coding error for each of the multiple sets. The RD cost value is, for example, the sum of the code amount and a value obtained by multiplying the squared error by a coefficient of 1. The coding parameter determination unit 110 selects the set of coding parameters for which the calculated cost value is the smallest. In this configuration, the entropy coding unit 104 outputs the selected set of coding parameters as a coded stream Te. The coding parameter determination unit 110 outputs the determined coding parameters to the parameter coding unit 111, the prediction parameter derivation unit 120, and the predicted image generation unit 101.

[0144] It should be noted that some of the components of the video encoding device 11 and the video decoding device 31 in the above-described embodiments, such as the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generation unit 308, the inverse quantization and inverse transform processing unit 311, the addition unit 312, the prediction parameter derivation unit 320, the predicted image generation unit 101, the subtraction unit 102, the transform and quantization unit 103, the entropy coding unit 104, the inverse quantization and inverse transform processing unit 105, the loop filter 107, the coding parameter determination unit 110, the parameter coding unit 111, and the prediction parameter derivation unit 120, may be implemented by a computer. In this case, this configuration may be realized by recording a program for implementing such control functions on a computer-readable recording medium and reading the program recorded on the recording medium for execution by a computer system. The term "computer system" used herein refers to a computer system incorporated into either the video encoding device 11 or the video decoding device 31, and it should be noted that the computer system is expected to include hardware components such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks incorporated into computer systems. Furthermore, the term "computer-readable recording medium" may include media that dynamically store programs for a short period of time, such as communication lines when transmitting programs via a network such as the Internet or via communication lines such as telephone lines, and may also include media that store programs for a fixed period of time, such as volatile memory included in a computer system functioning as a server or client in such cases. Furthermore, the above-mentioned program may be a program for implementing some of the above-mentioned functions, or may be a program that can be combined with a program already stored in a computer system to implement the above-mentioned functions.

[0145] Furthermore, part or all of the video encoding device 11 and video decoding device 31 in the above-described embodiments may be realized as integrated circuits such as large-scale integrated circuits (LSIs). Each functional block of the video encoding device 11 and video decoding device 31 may be realized individually as a processor, or part or all of the functional blocks may be integrated into a processor. Furthermore, circuit integration technology is not limited to LSIs; integrated circuits related to functional blocks may be realized as dedicated circuits or multi-purpose processors. If a circuit integration technology that replaces LSIs emerges due to advances in semiconductor technology, integrated circuits based on that technology may be used.

[0146] Although the embodiments of the present disclosure have been described in detail above with reference to the drawings, the specific configuration is not limited to the above embodiments and includes modifications to the design within the scope that does not deviate from the spirit of the present disclosure.

[0147] An embodiment of the present invention can be applied to a video decoding device that decodes encoded data of image data, and a video encoding device that generates encoded data from image data, and in which the data structure of the encoded data is generated by the video encoding device and referenced by the video decoding device. [Explanation of symbols]

[0148] 31 Image decoding device 301 Entropy Decoding Unit 302 Parameter Decoding Unit 308 Prediction Image Generation Unit 3104 Intra prediction unit 31046 dimd prediction section 310460 Reference sample derivation part 310465 Angle derivation part 310461 Gradient derivation part 310462 Angle mode derivation part 310463 Angle mode selection section 310464 Prediction image generation unit 311 Inverse quantization and inverse transformation processing unit 312 Addition section 11 Image encoding device 101 Prediction image generation unit 102 Subtraction section 103 Transformation and Quantization Unit 104 Entropy coding unit 105 Inverse quantization and inverse transformation processing unit 107 Loop Filter 110 Encoding parameter determination unit 111 Parameter Encoding Unit

Claims

1. An image decoding device comprising: a decoding circuit configured to decode a dimd_mode; a reference sample derivation circuit configured to select a neighboring image of a current block according to the dimd_mode; a gradient derivation circuit configured to derive a gradient from a filter selected based on the pixel and block size of the selected neighboring image; and an angle mode selection circuit configured to derive an intra prediction mode from the gradient.

2. The image decoding device according to claim 1 , wherein the gradient derivation circuit changes the size of the filter to be used according to the width and height of the current block.

3. The image decoding device according to claim 1 , wherein the gradient derivation circuit determines a smaller filter for a smaller current block.

4. The image decoding device according to claim 1 , wherein the gradient derivation circuit changes the size of the filter to be used in accordance with the derived dimd_mode.

5. The image decoding device of claim 1 , wherein the gradient derivation circuit determines the smaller filter when dimd_mode indicates that both a top neighboring block and a left neighboring block are used.

6. An image encoding device comprising: a reference sample derivation circuit configured to select a neighboring image of a current block according to a dimd_mode; a gradient derivation circuit for deriving gradients from pixels of the selected neighboring image and a filter selected by the dimd_mode; and an angle mode selection circuit configured to derive an intra prediction mode from the gradient.

Images