Moving image decoding device and moving image encoding device

Abstract

To provide a moving image encoding and decoding device capable of improving encoding efficiency by non-separation transformation.SOLUTION: A moving image decoding device according to one aspect of an invention comprises: a prediction image generation unit for generating a prediction image; and an inverse non-separation transformation unit for performing inverse non-separation transformation. A non-separation transformation unit does not change a specific frequency component of a transformation coefficient.SELECTED DRAWING: Figure 15

Description

[Technical field]

[0001] An embodiment of the present invention relates to a video decoding device and a video encoding device. [Background technology]

[0002] In order to efficiently transmit or record moving images, a moving image encoding device is used that generates encoded data by encoding moving images, and a moving image decoding device is used that generates a decoded image by decoding the encoded data.

[0003] Specific examples of video encoding methods include H.264 / AVC and High-Efficiency Video Coding (HEVC).

[0004] In such a video coding method, images (pictures) constituting a video are managed in a hierarchical structure consisting of slices obtained by dividing images, coding tree units (CTUs) obtained by dividing slices, coding units (sometimes called coding units: CUs) obtained by dividing coding tree units, and transform units (TUs) obtained by dividing coding units, and are coded / decoded for each CU.

[0005] In such a video coding method, a predicted image is usually generated based on a locally decoded image obtained by encoding / decoding an input image, and a prediction error (sometimes called a "difference image" or "residual image") obtained by subtracting the predicted image from the input image (original image) is coded. Methods for generating a predicted image include inter-prediction and intra-prediction.

[0006] Non-Patent Document 1 also discloses a GPM (Geometric partition mode) mode in which a target block is divided into shapes other than a rectangle and different inter predictions are performed for each region. By dividing the target block into shapes other than a rectangle in this way, it is possible to make more accurate predictions even for complex textures, and improve coding efficiency. Non-Patent Document 1 also discloses an image coding device that performs non-separable transformation in order to further concentrate coefficients obtained by separating and transforming prediction errors for each transform unit in a low-frequency region. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] ITU-T Rec. H.266 Summary of the Invention [Problem to be solved by the invention]

[0008] In Non-Patent Document 1, a non-separable transform technique that is effective for prediction residuals with directionality is applied only to intra-prediction blocks that are highly correlated with adjacent pixels and are unlikely to experience luminance changes, and is not adapted to inter-prediction blocks that are likely to experience luminance changes. The present invention aims to provide a video decoding device and a video encoding device that can suitably apply inverse non-separable transform even when there is a luminance change. [Means for solving the problem]

[0009] In order to solve the above problems, a video decoding device according to one aspect of the present invention is a video decoding device including a predicted image generating unit that generates a predicted image, and an inverse non-separable transform unit that performs inverse non-separable transform. The non-separable transform unit does not change a specific frequency component of the transform coefficient.

[0010] The inverse non-separable transform unit is characterized in that it does not change the DC component, which is the upper left component.

[0011] The inverse non-separable transform unit is characterized in that, in the case of inter prediction, the DC component, which is the upper left component, is not changed.

[0012] The inverse non-separable transform unit is characterized in that it derives a one-dimensional array by removing the DC component, and performs transformation into the one-dimensional array.

[0013] The inverse non-separable transform unit is characterized in that it uses a transform matrix in which the leftmost component, excluding the upper left component, and the uppermost component are zero.

[0014] A video decoding device including a prediction image generation unit that generates a prediction image and an inverse non-separable transform unit that performs an inverse non-separable transform, the non-separable transform unit being characterized in that it does not change specific frequency components of transform coefficients.

[0015] A video decoding device characterized in that, when a predicted image is derived by a weighted sum of an intra-predicted image and an inter-predicted image, a transform matrix for a non-separable transform is selected using an intra-prediction mode used to derive the intra-predicted image, and a non-separable transform is performed.

[0016] 8. The video decoding device according to claim 7, wherein the mode in which the predicted image is used is a CIIP mode in which the weights are derived depending on the states of adjacent blocks.

[0017] 8. The video decoding device according to claim 7, wherein the mode using the predicted image is a geometry mode in which weights are derived from parameters derived from encoded data.

[0018] A video coding device characterized in that, when deriving a predicted image by a weighted sum of an intra-predicted image and an inter-predicted image, a transform matrix for a non-separable transform is selected using an intra-prediction mode used to derive the intra-predicted image, and a non-separable transform is performed. Effect of the Invention

[0019] According to an aspect of the present invention, in video encoding / decoding processing, encoding efficiency can be improved by performing non-separable transform suitable for inter prediction. [Brief description of the drawings]

[0020] [Figure 1] 1 is a schematic diagram showing a configuration of an image transmission system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. [Diagram 3] FIG. 1 is a schematic diagram showing a configuration of a video decoding device. [Figure 4] FIG. 13 is a schematic diagram showing a configuration of an inter-prediction image generating unit. [Diagram 5] FIG. 13 is a schematic diagram showing a configuration of an inter-prediction parameter derivation unit. [Figure 6] 11 is a flowchart illustrating a schematic operation of the video decoding device. [Figure 7] FIG. 13 is a syntax diagram illustrating coding parameters for GPM prediction. [Figure 8] FIG. 1 is a diagram for explaining GPM prediction. [Figure 9] 1 is a diagram showing a structure of a table showing the correspondence between merge_gpm_partition_idx, angleIdx, and distanceIdx in GPM prediction. FIG. 2 is a diagram showing a structure of a table showing the correspondence between idx and DisLut[idx] used in a weighting coefficient derivation process and a motion vector storage process in GPM prediction. [Figure 10] FIG. 13 is a schematic diagram showing types of intra-prediction modes (mode numbers). [Figure 11] FIG. 2 is a functional block diagram showing an example of the configuration of an inverse quantization and inverse transform unit. [Figure 12] 13 is an example of a syntax table showing flags of a non-separable transform. [Figure 13] 13 is an example of a syntax table showing flags of a non-separable transform. [Figure 14] 13 is a table used for selecting a transformation matrix for non-separable transformation in intra prediction mode. [Figure 15] 13 is a flowchart showing a process of switching non-separable transformation depending on a prediction mode. [Figure 16] FIG. 13 illustrates one configuration for the operation of a DC-preserving inverse non-separable transform. [Figure 17] FIG. 13 illustrates another configuration of the operation of the inverse non-separable transform that preserves the DC transform. [Figure 18] FIG. 13 illustrates a DC-preserving transformation (non-separable transformation). [Figure 19] 13 is a flowchart of a process for performing LFNST in the CIIP mode. [Figure 20] 13 is a flowchart showing a configuration for performing LFNST processing when a current block is inter-predicted (CuPredMode != MODE_INTRA). [Figure 21] 13 is a flowchart showing a configuration for performing LFNST processing when a current block is inter-predicted (CuPredMode != MODE_INTRA). [Figure 22] FIG. 1 is a block diagram showing a configuration of a video encoding device. [Figure 23] 13 is a schematic diagram showing a configuration of an inter-prediction parameter encoding unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] (First embodiment) Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0022] FIG. 1 is a schematic diagram showing the configuration of an image transmission system 1 according to this embodiment.

[0023] The image transmission system 1 is a system that transmits an encoded stream obtained by encoding a target image, decodes the transmitted encoded stream, and displays an image. The image transmission system 1 includes a video encoding device (image encoding device) 11, a network 21, a video decoding device (image decoding device) 31, and a video display device (image display device) 41.

[0024] An image T is input to the video encoding device 11 .

[0025] The network 21 transmits the encoded stream Te generated by the video encoding device 11 to the video decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may also be replaced by a storage medium on which the encoded stream Te is recorded, such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blue-ray Disc: registered trademark).

[0026] The video decoding device 31 decodes each of the coded streams Te transmitted by the network 21, and generates one or more decoded images Td.

[0027] The video display device 41 displays all or part of one or more decoded images Td generated by the video decoding device 31. The video display device 41 includes a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. The display may be in the form of a stationary display, a mobile display, an HMD, or the like. When the video decoding device 31 has high processing power, it displays high quality images, and when it has only low processing power, it displays images that do not require high processing power or display power.

[0028] <operator> The operators used in this specification are listed below.

[0029] >> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR, |= is the OR assignment operator, and || indicates logical OR.

[0030] x?y:z is a ternary operator that takes y when x is true (non-zero) and z when x is false (0).

[0031] Clip3(a, b, c) is a function that clips c to a value between a and b (inclusive), returning a if c < a, b if c > b, and c otherwise (where a <= b).

[0032] abs(a) is a function that returns the absolute value of a.

[0033] Int(a) is a function that returns the integer value of a.

[0034] floor(a) is a function that returns the largest integer less than or equal to a.

[0035] ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0036] a / d represents the division of a by d, rounded down to the nearest integer.

[0037] <Structure of the Encoded Stream Te> Prior to the detailed description of the moving image encoding apparatus 11 and the moving image decoding apparatus 31 according to the present embodiment, the data structure of the encoded stream Te generated by the moving image encoding apparatus 11 and decoded by the moving image decoding apparatus 31 will be described.

[0038] FIG. 2 is a diagram showing the hierarchical structure of data in the encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. FIGS. 2(a) to 2(f) respectively show an encoded video sequence that defines the sequence SEQ, an encoded picture that defines the picture PICT, an encoded slice that defines the slice S, encoded slice data that defines the slice data, an encoded tree unit included in the encoded slice data, and an encoded unit included in the encoded tree unit.

[0039] (Coded Video Sequence) The coded video sequence defines a set of data that the video decoding device 31 refers to in order to decode the sequence SEQ to be processed. As shown in Fig. 2, the sequence SEQ includes a video parameter set, a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), an adaptation parameter set (APS), a picture PICT, and supplemental enhancement information SEI (Supplemental Enhancement Information).

[0040] The video parameter set VPS specifies a set of coding parameters common to multiple videos composed of multiple layers, as well as a set of coding parameters related to multiple layers and each individual layer included in the video.

[0041] The sequence parameter set SPS specifies a set of coding parameters that the video decoding device 31 refers to in order to decode the target sequence. For example, the width and height of a picture are specified. Note that there may be multiple SPSs. In that case, one of the multiple SPSs is selected from the PPS.

[0042] The picture parameter set PPS specifies the number of pictures to be decoded for each picture in the target sequence. A set of coding parameters to be referred to by the video decoding device 31 is specified. For example, the set includes a reference value of the quantization width used in decoding a picture (pic_init_qp_minus26) and a flag indicating application of weighted prediction (weighted_pred_flag). Note that there may be multiple PPSs. In that case, one of the multiple PPSs is selected from each picture in the target sequence.

[0043] (Encoded Picture) A coded picture defines a set of data to be referenced by the video decoding device 31 in order to decode a picture PICT to be processed. As shown in FIG. 2, the picture PICT includes slices 0 to NS-1 (NS is the total number of slices included in the picture PICT).

[0044] In the following description, when there is no need to distinguish between slices 0 to NS-1, the subscripts of the symbols may be omitted. The same applies to other data that are included in the coded stream Te and that are to be described below and that are to be given subscripts.

[0045] (Coded Slice) A coded slice defines a set of data to be referenced by the video decoding device 31 in order to decode a target slice S. As shown in Fig. 2, a slice includes a slice header and slice data.

[0046] The slice header includes a group of coding parameters to be referred to by the video decoding device 31 in order to determine a decoding method for the current slice. Slice type designation information (slice_type) that designates the slice type is an example of a coding parameter included in the slice header.

[0047] Slice types that can be specified by the slice type specification information include (1) an I slice that uses only intra prediction when encoding, (2) a P slice that uses unidirectional prediction or intra prediction when encoding, and (3) a B slice that uses unidirectional prediction, bidirectional prediction, or intra prediction when encoding. Note that inter prediction is not limited to uni-prediction or bi-prediction, and a predicted image may be generated using more reference pictures. Hereinafter, when referring to P or B slice, it refers to a slice including a block that can use inter prediction.

[0048] In addition, the slice header may include a reference to a picture parameter set PPS (pic_parameter_set_id).

[0049] (Encoded slice data) The coded slice data specifies a set of data to be referenced by the video decoding device 31 in order to decode the slice data to be processed. As shown in Fig. 2(d), the slice data includes a CTU. A CTU is a block of a fixed size (e.g., 64x64) that constitutes a slice, and is also called a Largest Coding Unit (LCU).

[0050] (coding tree unit) 2 specifies a set of data that the video decoding device 31 refers to in order to decode a CTU to be processed. The CTU is divided into coding units CU, which are basic units of coding processing, by recursive quad tree division (QT (Quad Tree) division), binary tree division (BT (Binary Tree) division), or ternary tree division (TT (Ternary Tree) division). BT division and TT division are collectively called multi tree division (MT (Multi Tree) division). A node of a tree structure obtained by recursive quad tree division is called a coding node. Intermediate nodes of the quad tree, binary tree, and ternary tree are coding nodes, and the CTU itself is specified as the top coding node. The lowest coding node is specified as a coding unit.

[0051] (Encoding Unit) 2 specifies a set of data to be referenced by the video decoding device 31 in order to decode a coding unit to be processed. Specifically, a CU is composed of a CU header CUH, prediction parameters, transformation parameters, quantization transformation coefficients, etc. The CU header specifies a prediction mode, etc.

[0052] The prediction process may be performed on a CU basis, or on a sub-CU basis by further dividing a CU. If the size of a CU and a sub-CU are the same, there is one sub-CU in the CU. If the size of a CU is larger than that of a sub-CU, the CU is divided into sub-CUs. For example, if the CU is 8x8 and the sub-CU is 4x4, the CU is divided into 2 parts horizontally and 2 parts vertically, into 4 sub-CUs.

[0053] Prediction types (prediction modes) include intra prediction (MODE_INTRA), inter prediction (MODE_INTER), and intra block copy (MODE_IBC). Intra prediction is a prediction within the same picture, and inter prediction refers to a prediction process performed between different pictures (for example, between display times or between layer images).

[0054] The transform and quantization processes are performed in units of CUs, but the quantized transform coefficients may be entropy coded in units of sub-blocks such as 4x4.

[0055] (Prediction parameters) The predicted image is derived from prediction parameters associated with the block, which include intra-prediction and inter-prediction parameters.

[0056] (Inter prediction parameters) The prediction parameters of inter prediction will be described. The inter prediction parameters are composed of prediction list use flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and motion vectors mvL0 and mvL1. predFlagL0 and predFlagL1 are flags indicating whether or not a reference picture list (L0 list, L1 list) is used, and when the value is 1, the corresponding reference picture list is used. Note that in this specification, when the "flag indicating whether or not XX" is written, a flag other than 0 (for example, 1) is XX, and 0 is not XX, and in logical negation, logical product, etc., 1 is treated as true and 0 is treated as false (similar below). However, in an actual device or method, other values ​​can also be used as true and false values.

[0057] Syntax elements for deriving inter prediction parameters include, for example, a merge flag merge_flag (general_merge_flag), a merge index merge_idx, merge_subblock_flag indicating whether to use subblock-based inter prediction, regulare_merge_flag, an intra-inter flag ciip_flag, GPM_flag indicating whether to use GPM mode (Geometric partitioning merge mode), merge_gpm_partition_idx indicating the partition shape of the GPM mode, merge_gpm_idx0 and merge_gpm_idx1 indicating GPM merge indices, an inter prediction identifier inter_pred_idc for selecting a reference picture to be used in AMVP mode, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx for deriving a motion vector, a difference vector mvdLX, a motion vector precision mode amvr_mode, and ciip_flag indicating CIIP mode (Combined intra inter prediction).

[0058] (Reference picture list) The reference picture list is a list of reference pictures stored in the reference picture memory 306. In each CU, refIdxLX is used to specify which picture in the reference picture list RefPicListX (X=0 or 1) is actually referenced. Note that LX is a description method used when there is no distinction between L0 prediction and L1 prediction, and hereinafter, parameters for the L0 list and parameters for the L1 list are distinguished by replacing LX with L0 or L1.

[0059] (Merge prediction and AMVP prediction) The prediction parameter decoding (encoding) method includes merge prediction mode (merge mode) and AMVP (Advanced Motion Vector Prediction, adaptive motion vector prediction) mode, and merge_flag is a flag for identifying these. The merge mode is a prediction mode in which some or all of the motion vector difference is omitted, and the prediction list usage flag predFlagLX, the reference picture index refIdxLX, and the motion vector mvLX are not included in the encoded data, but are derived from the prediction parameters of the neighboring blocks that have already been processed. The AMVP mode is a mode in which inter_pred_idc, refIdxLX, and mvLX are included in the encoded data. Note that mvLX is encoded as mvp_LX_idx, which identifies the prediction vector mvpLX, and the difference vector mvdLX. The general term for the prediction mode in which the motion vector difference is omitted or simplified is called the general merge mode, and the general merge mode and the AMVP prediction may be selected by the general_merge_flag. In addition, regular_merge_flag may be transmitted, and if regular_merge_flag is 1, normal merge mode or MMVD may be selected, and otherwise CIIP mode or GPM mode may be selected.

[0060] inter_pred_idc is a value indicating the type and number of reference pictures, and takes one of the values ​​PRED_L0, PRED_L1, and PRED_BI. PRED_L0 and PRED_L1 indicate uni-prediction using one reference picture managed in the L0 list and L1 list, respectively. PRED_BI indicates bi-prediction using two reference pictures managed in the L0 list and L1 list.

[0061] The merge_idx is an index indicating which prediction parameter is to be used as the prediction parameter of the current block among prediction parameter candidates (merge candidates) derived from blocks for which processing has been completed.

[0062] (Motion Vector) mvLX indicates the amount of shift between blocks on two different pictures. A prediction vector and a difference vector related to mvLX are called mvpLX and mvdLX, respectively.

[0063] (Inter prediction identifier inter_pred_idc and prediction list usage flag predFlagLX) The relationship between inter_pred_idc, predFlagL0, and predFlagL1 is as follows, and they are mutually convertible.

[0064] inter_pred_idc = (predFlagL1<<1)+predFlagL0 predFlagL0 = inter_pred_idc & 1 predFlagL1 = inter_pred_idc >> 1 In addition, the inter prediction parameter may use a prediction list usage flag or an inter prediction identifier. Furthermore, the determination using the prediction list usage flag may be replaced with a determination using the inter prediction identifier. Conversely, the determination using the inter prediction identifier may be replaced with a determination using the prediction list usage flag.

[0065] (Bi-predictive biPred decision) A flag biPred indicating whether or not the prediction is bi-predictive can be derived based on whether or not two prediction list usage flags are both 1.

[0066] Alternatively, biPred can be derived based on whether the inter-prediction identifier is a value indicating the use of two prediction lists (reference pictures).

[0067] (Configuration of a video decoding device) The configuration of a video decoding device 31 (FIG. 3) according to this embodiment will be described.

[0068] The video decoding device 31 includes an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generating unit (prediction image generating device) 308, an inverse quantization and inverse transform unit 311, an adder 312, and a prediction parameter derivation unit 320. Note that, in accordance with the video encoding device 11 described below, the video decoding device 31 may also be configured not to include the loop filter 305.

[0069] The parameter decoding unit 302 further includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 further includes a TU decoding unit 3024. These may be collectively referred to as a decoding module. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, PPS, and APS, and slice header (slice information) from the encoded data. The CT information decoding unit 3021 decodes the CT from the encoded data. The CU decoding unit 3022 decodes the CU from the encoded data. The TU decoding unit 3024 decodes QP update information (quantization correction value) and quantization prediction error (residual_coding) from the encoded data when a prediction error is included in the TU.

[0070] The header decoding unit 3020 decodes a flag sps_lfnst_enabled_flag indicating whether or not non-separable transform is used from the SPS. Furthermore, the header decoding unit 3020 decodes ph_lfnst_enabled_flag from the picture header (PH) when sps_lfnst_enabled_flag is 1. When ph_lfnst_enabled_flag does not appear, the header decoding unit 3020 infers ph_lfnst_enabled_flag as 0. Alternatively, when ph_lfnst_enabled_flag does not appear, the value of sps_lfnst_enabled_flag may be set as the value of ph_lfnst_enabled_flag.

[0071] The GPM non-separable transform in the present invention is a name for an inverse non-separable transform (and a non-separable transform) for inter prediction in GPM mode. In the following, a method for deriving a transform matrix or a set of transform matrices for a non-separable transform from a partition shape of GPM will be described.

[0072] sps_gpm_enabled_flag is notified by SPS and indicates whether GPM prediction mode is used in the target sequence. sps_max_num_merge_cand_minus_max_num_gpm_cand is notified by SPS. sps_max_num_merge_cand_minus_max_num_gpm_cand is a parameter used to derive MaxNumGpmMergeCand, the maximum number of merge candidates for GPM prediction. When MaxNumGpmMergeCand is 0, GPM prediction is prohibited.

[0073] The header decoding unit 3020 decodes the flag sps_gpm_lfnst_enabled_flag from the SPS. The sps_gpm_lfnst_enabled_flag is a flag indicating whether or not to use a non-separable transform (hereinafter, GPM non-separable transform) in the case of GPM prediction. When the sps_lfnst_enabled_flag is 1 and the sps_gpm_enabled_flag is 1, the header decoding unit 3020 decodes the sps_gpm_lfnst_enabled_flag.

[0074] The TU decoding unit 3024 decodes a parameter lfnst_idx indicating whether or not non-separable transform is used and a transform base from the encoded data. Specifically, the TU decoding unit 3024 decodes lfnst_idx when the width and height of the CU are 4 or more and the prediction mode is an intra prediction mode. Note that when lfnst_idx is 0, it indicates non-application of non-separable transform, when it is 1, it indicates one transform matrix of a set (pair) of transform matrices (transform bases) of non-separable transform, and when it is 2, it indicates the other transform matrix of the pair.

[0075] The predicted image generation unit 308 includes an inter predicted image generation unit 309 (FIG. 4) and an intra predicted image generation unit 310.

[0076] The prediction parameter derivation unit 320 includes the inter prediction parameter derivation unit 303 (FIG. 5) and an intra prediction parameter derivation unit.

[0077] In addition, although an example in which CTU and CU are used as processing units will be described below, the present invention is not limited to this example, and processing may be performed in sub-CU units. Alternatively, CTU and CU may be read as blocks, and sub-CU as sub-blocks, and processing may be performed in block or sub-block units.

[0078] The entropy decoding unit 301 performs entropy decoding on the externally input coded stream Te to decode individual codes (syntax elements). Entropy coding includes a method of variable-length coding the syntax elements using a context (probability model) adaptively selected according to the type of syntax element or surrounding circumstances, and a method of variable-length coding the syntax elements using a predetermined table or formula.

[0079] The entropy decoding unit 301 outputs the decoded code to the parameter decoding unit 302. The decoded code is, for example, a prediction mode predMode, merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX, amvr_mode, etc. Control of which code to decode is performed based on an instruction from the parameter decoding unit 302.

[0080] (Basic flow) FIG. 6 is a flowchart illustrating a schematic operation of the video decoding device 31.

[0081] (S1100: Decode Parameter Set Information) The header decoding unit 3020 decodes parameter set information such as the VPS, SPS, and PPS from the encoded data.

[0082] (S1200: Decode slice information) The header decoding unit 3020 decodes the slice header (slice information) from the encoded data.

[0083] Thereafter, the video decoding device 31 repeats the processes from S1300 to S5000 for each CTU included in the target picture to derive a decoded image of each CTU.

[0084] (S1300: Decode CTU Information) The CT information decoding unit 3021 decodes the CTU from the encoded data.

[0085] (S1400: Decode CT Information) The CT information decoding unit 3021 decodes the CT from the encoded data.

[0086] (S1500: CU Decoding) The CU decoding unit 3022 performs S1510 and S1520 to decode the CU from the encoded data.

[0087] (S1510: Decode CU Information) The CU decoding unit 3022 decodes the CU information, prediction information, TU division flag, CU residual flag, and the like from the encoded data.

[0088] (S1520: Decode TU information) When a prediction error is included in a TU, the TU decoding unit 3024 decodes the quantized prediction error, the transform index lfnst_idx, and the like from the encoded data.

[0089] (S2000: Generation of predicted image) The predicted image generation unit 308 generates a predicted image for each block included in the current CU based on the prediction information.

[0090] (S3000: Inverse Quantization and Inverse Transformation) The inverse quantization and inverse transform unit 311 executes inverse quantization and inverse transform processing for each TU included in the target CU.

[0091] (S4000: Generate decoded image) The adder 312 adds the predicted image supplied from the predicted image generation unit 308 and the prediction error supplied from the inverse quantization and inverse transform unit 311 to generate a decoded image of the current CU.

[0092] (S5000: Loop Filter) The loop filter 305 applies a loop filter such as a deblocking filter, SAO, or ALF to the decoded image to generate a decoded image.

[0093] The loop filter 305 is a filter provided in the encoding loop, which removes block distortion and ringing distortion to improve image quality. The loop filter 305 applies a filter such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adder 312.

[0094] The reference picture memory 306 stores the decoded image of the CU generated by the adder 312 in a location that is determined in advance for each current picture and current CU.

[0095] The prediction parameter memory 307 stores prediction parameters at a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301.

[0096] The prediction image generating unit 308 receives a prediction mode predMode, prediction parameters, and the like. The prediction image generating unit 308 also reads a reference picture from the reference picture memory 306. The prediction image generating unit 308 generates a prediction image of a block or sub-block using the prediction parameters and the read reference picture (reference picture block) in the prediction mode indicated by the prediction mode predMode. Here, the reference picture block is a set of pixels on the reference picture (usually rectangular, so called a block), and is an area referenced to generate a prediction image.

[0097] (Configuration of inter-prediction parameter derivation unit) 5, the inter prediction parameter derivation unit 303 derives inter prediction parameters by referring to prediction parameters stored in the prediction parameter memory 307 based on the syntax elements input from the parameter decoding unit 302. The inter prediction parameter derivation unit 303 also outputs the inter prediction parameters to the inter prediction image generation unit 309 and the prediction parameter memory 307. The inter prediction parameter derivation unit 303 and its internal elements, the AMVP prediction parameter derivation unit 3032, the merge prediction parameter derivation unit 3036, the GPM prediction unit 30377, and the MV addition unit 3038, are means common to the video encoding device and the video decoding device, and therefore may be collectively referred to as a motion vector derivation unit (motion vector derivation device).

[0098] When merge_flag is 1, that is, when it indicates the merge prediction mode, merge_idx is derived and output to the merge prediction parameter derivation unit 3036 .

[0099] When merge_flag is 0, that is, when it indicates the AMVP prediction mode, the AMVP prediction parameter derivation unit 3032 derives mvpLX from inter_pred_idc, refIdxLX, or mvp_LX_idx.

[0100] When GPM_flag is 1, that is, when the current block is encoded in the GPM prediction mode, the GPM prediction unit 30377 derives parameters for GPM prediction. The derivation of GPM_flag will be described later.

[0101] (MV addition section) The MV adder 3038 adds the derived mvpLX and mvdLX to derive mvLX.

[0102] (Merge prediction) The merge prediction parameter derivation unit 3036 includes a merge candidate derivation unit 30361 and a merge candidate selection unit 30362. Note that a merge candidate includes prediction parameters (predFlagLX, mvLX, refIdxLX) and is stored in a merge candidate list. An index is assigned to the merge candidates stored in the merge candidate list according to a predetermined rule.

[0103] The merging candidate derivation unit 30361 derives merging candidates by directly using the motion vectors and refIdxLX of the decoded adjacent blocks. Alternatively, the merging candidate derivation unit 30361 may apply a spatial merging candidate derivation process, a temporal merging candidate derivation process, or the like, which will be described later.

[0104] As a spatial merge candidate derivation process, the merge candidate derivation unit 30361 reads out prediction parameters stored in the prediction parameter memory 307 according to a predetermined rule, and sets them as merge candidates. For example, prediction parameters for the positions A1, B1, B0, A0, and B2 are read out.

[0105] A1: (xCb-1, yCb+cbHeight-1) B1: (xCb+cbWidth-1, yCb-1) B0: (xCb+cbWidth, yCb-1) A0: (xCb-1, yCb+cbHeight) B2: (xCb-1, yCb-1) The upper left coordinates of the target block are (xCb, yCb), its width is cbWidth, and its height is cbHeight.

[0106] As a temporal merge derivation process, the merge candidate derivation unit 30361 reads prediction parameters of the lower right CBR of the target block or block C in the reference image including the center coordinates from the prediction parameter memory 307, sets it as a merge candidate Col, and stores it in the merge candidate list mergeCandList[ ].

[0107] The order of storing the mergeCandList[] is, for example, spatial merge candidates (B1, A1, B0, A0, B2), then temporal merge candidates Col. Note that reference blocks that are unavailable (blocks that are intra-predicted, etc.) are not stored in the merge candidate list. i = 0 if(availableFlagB1) mergeCandList[i++] = B1 if(availableFlagA1) mergeCandList[i++] = A1 if(availableFlagB0) mergeCandList[i++] = B0 if(availableFlagA0) mergeCandList[i++] = A0 if(availableFlagB2) mergeCandList[i++] = B2 if(availableFlagCol) mergeCandList[i++] = Col Furthermore, the history merge candidate HmvpCand, the average candidate avgCand, and the zero merge candidate zeroCandm may be added to mergeCandList[] for use.

[0108] The merging candidate selection unit 30362 selects a merging candidate N indicated by merge_idx from among the merging candidates included in the merging candidate list, using the following formula.

[0109] N = mergeCandList[merge_idx] Here, N is a label indicating a merging candidate, and can be A1, B1, B0, A0, B2, Col, etc. The motion information of the merging candidate indicated by the label N is indicated by (mvLXN[0], mvLXN[1]), predFlagLXN, and refIdxLXN.

[0110] The merging candidate selection unit 30362 selects the selected (mvLXN[0], mvLXN[0]), predFlagLXN, and refIdxLXN as inter prediction parameters of the current block. The merging candidate selection unit 30362 stores the inter prediction parameters of the selected merging candidate in the prediction parameter memory 307, and also outputs them to the inter predicted image generation unit 309.

[0111] (GPM forecast) GPM prediction is used in GPM mode, and generates a predicted image using two non-rectangular prediction units that divide a target CU by a line segment. An example of a line segment is shown in FIG. 8(a). A line segment that spans a target CU is specified by an angle index angleIdx (angle variable) and a distance index distanceIdx (distance variable) shown in FIG. 8(b). angleIdx indicates the angle φ between a vertical line and the line segment. distanceIdx indicates the distance ρ from the center of the target CU to the line segment. angleIdx is an integer value assigned to the angle of the line segment. For example, when 360 degrees is roughly divided into 32 equal parts, the value shown in FIG. 8(c) is assigned. Note that a configuration in which 360 degrees is divided into 64 and even finer angle indexes are sent may be used.

[0112] In generating a predicted image for GPM prediction, two "rectangular" predicted images (temporary predicted images) including a non-rectangular prediction unit may be derived, and the two rectangular regions may be derived by weighting according to the shape of the prediction unit. The motion compensation unit 3091 derives two temporary predicted images for the target CU, and the GPM synthesis unit 30952 derives a predicted image by applying weighting processing to each pixel of the two temporary predicted images according to the pixel position. This processing is called a GPM synthesis processing. Processing other than prediction (e.g., transformation (inverse transformation) and quantization (inverse quantization)) is applied to the entire target CU.

[0113] The GPM prediction unit 30377 derives prediction parameters for two non-rectangular regions and supplies them to the inter-prediction image generation unit 309. To simplify the processing, the GPM prediction may be configured not to use bi-prediction. In this case, unidirectional inter-prediction parameters are derived for the non-rectangular regions.

[0114] (Decoding syntax in GPM prediction) 7 is a diagram showing the syntax configuration notified by merge_data(). The parameter decoding unit 302 decodes syntax elements in the encoded data, and the GPM prediction unit 30377 (inter prediction parameter derivation unit 303) derives parameters for GPM prediction.

[0115] general_merge_flag is a flag indicating whether or not to use the merge mode, which is notified when the current block is not in the skip mode. In the case of the skip mode, the inter prediction parameter derivation unit 303 sets general_merge_flag=1.

[0116] When ciip_flag is 0, the parameter decoding unit 302 (CU decoding unit 3022) decodes the syntax elements merge_gpm_partition_idx, merge_gpm_idx0, and merge_gpm_idx1 of the GPM prediction. Alternatively, merge_gpm_partition_idx and merge_gpm_idx0 may be decoded, and merge_gpm_idx1 may be decoded if MaxNumGpmMergeCand>2. merge_gpm_partition_idx is an index (partition index) indicating a division pattern of the GPM prediction mode. The partition index indicates a combination of angleIdx and distanceIdx that identify a straight line that spans the target block to divide the target block into two regions. The motion information used to generate a predicted image for the GPM prediction uses a merge candidate. merge_gpm_idx0 and merge_gpm_idx1 are indices of merge candidates that indicate the motion information of the two regions, respectively. The number of options for the partition index (number of division patterns) is NumGPMFull, and merge_gpm_partition_idx takes an integer value from 0 to NumGPMFull-1 (=63).

[0117] MergeGpmFlag is a flag indicating whether or not to perform GPM prediction on the target block. If at least the following conditions (GPM judgment conditions) are satisfied, the GPM prediction unit 30377 sets MergeGpmFlag=1 (GPM prediction on), and if not, the GPM prediction unit 30377 sets MergeGpmFlag=0. (GPM judgment conditions) sps_gpm_enabled_flag=1 slice_type is B slice general_merge_flag=1 - The target block is within a certain size range (for example, cbWidth>=8 and cbHeight>=8 and cbWidth<8*cbHeight and cbHeight<8*cbWidth) regular_merge_flag=0 merge_subblock_flag=0 ciip_flag=0 The GPM prediction unit 30377 outputs the derived parameters to the GPM synthesis unit 30952.

[0118] (Movement information derivation process in GPM prediction) The GPM predictor 30377 derives merge indexes m and n from syntax elements merge_gpm_idx0 and merge_gpm_idx1 indicating motion information of two non-rectangular regions as follows.

[0119] m = merge_gpm_idx0 n = merge_gpm_idx1 + (merge_gpm_idx1 >= m) ? 1 : 0 In the following, the merge candidate pointed to by merge index m is denoted as M, and the merge candidate pointed to by merge index n is denoted as N.

[0120] The merge prediction parameter derivation unit 3036 derives motion information (mvLXM, mvLXN, refIdxLXM, refIdxLXN, predFlagLXM, predFlagLXN, bcwIdx, mergeCandList, etc.) of merge candidates M and N by the method described in (Merge Prediction). The GPM prediction unit 30377 uses this motion information to set the motion vectors mvA and mvB, reference indexes refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB of merge_gpm_idx0 and merge_gpm_idx1 as follows.

[0121] mvA[0] = mvLXM[0] mvA[1] = mvLXM[1] refIdxA = refIdxLXM predListFlagA = X Here, the GPM prediction unit 30377 sets the lowest 1 bit of m to X (m & 0x01). Note that when predFlagLXM is 0, the GPM prediction unit 30377 sets X to (1-X).

[0122] mvB[0] = mvLXN[0] mvB[1] = mvLXN[1] refIdxB = refIdxLXN predListFlagB = X Here, the GPM prediction unit 30377 sets the lowest 1 bit of n to X (n & 0x01). Note that when predFlagLXN is 0, the GPM prediction unit 30377 sets X to (1-X).

[0123] These motion information are referenced to generate temporal prediction images for the two non-rectangular regions.

[0124] The GPM prediction unit 30377 derives angleIdx and distanceIdx corresponding to merge_gpm_partition_idx according to the table shown in FIG.

[0125] (Motion vector storage process in GPM prediction) The GPM prediction unit 30377 performs the following steps to obtain the motion vectors (mvA, mvB) of the non-rectangular areas A and B and the reference picture information (predFlagA, prefFlagB, refIdxLA, refIdxLB) are stored in memory in 4x4 sub-block units.

[0126] numSbX and numSbY are the numbers of 4x4 sub-blocks in the horizontal and vertical directions of the current block, respectively. The GPM prediction unit 30377 sets numSbX=cbWidth>>2 and numSbY=cbHeight>>2. The GPM prediction unit 30377 derives the variables displacementX, displacementY, isFlip, and shiftHor as follows.

[0127] displacementX = angleIdx displacementY = (angleIdx + 8) % 32 isFlip = (angleIdx>=13 && angleIdx<=27) ? 1 : 0 shiftHor = (angleIdx%16==8 || (angleIdx%16!=0 && cbHeight>=cbWidth)) ? 0 : 1 The GPM prediction unit 30377 derives offsetX and offsetY as follows:

[0128] if (shiftHor == 0) { offsetX = (-cbWidth) >> 1 offsetY = ((-cbHeight) >> 1) + ((angleIdx<16) ? (distanceIdx * cbHeight) >> 3 : -((distanceIdx * cbHeight) >> 3)) } else { / / shiftHor == 1 offsetX = ((-cbWidth) >> 1) + ((angleIdx<16) ? (distanceIdx * cbWidth) >> 3 : -((distanceIdx * cbWidth) >> 3)) offsetY = (-cbHeight) >> 1 } The GPM prediction unit 30377 executes the following process for each position (xSbIdx, ySbIdx) of a 4x4 sub-block where xSbIdx = 0..numSbX-1, ySbIdx = 0..numSbY-1.

[0129] The GPM prediction unit 30377 uses disLut shown in FIG. 9 to calculate motionIdx as follows.

[0130] motionIdx = (((4 * xSbIdx + offsetX) << 1) + 5) * disLut[displacementX] + (((4 * ySbIdx + offsetY) << 1) + 5) * disLut[displacementY] The GPM prediction unit 30377 derives sType as follows.

[0131] sType = (abs(motionIdx)<32) ? 2 : ((motionIdx<=0) ? (1-isFlip) : isFlip) If sType is 0, the GPM prediction unit 30377 performs the following:

[0132] If the prediction list flag of A is 0 (predListFlagA == 0), the GPM predictor 30377 stores the motion vector of A in L0 as unidirectional prediction. If the prediction list flag of A is not 0 (predListFlagA != 0), the GPM predictor 30377 stores the motion vector of A in L1 as unidirectional prediction.

[0133] predFlagL0 = (predListFlagA==0) ? 1 : 0 predFlagL1 = (predListFlagA==0) ? 0 : 1 refIdxL0 = (predListFlagA==0) ? refIdxA : -1 refIdxL1 = (predListFlagA==0) ? -1 : refIdxA mvL0[0] = (predListFlagA==0) ? mvA[0] : 0 mvL0[1] = (predListFlagA==0) ? mvA[1] : 0 mvL1[0] = (predListFlagA==0) ? 0 : mvA[0] mvL1[1] = (predListFlagA==0) ? 0 : mvA[1] Otherwise, if sType is 1, or sType is 2 and predListFlagA+predListFlagB is not 1, then the GPM predictor 30377 performs the following: Here, predListFlagA+predListFlagB is not 1 indicates that the reference picture lists of A and B are the same.

[0134] If the prediction list flag for B is 0 (predListFlagB==0), the GPM prediction unit 30377 If the prediction list flag of B is not 0 (predListFlagB!=0), the GPM predictor 30377 stores the motion vector of B in L1 as unidirectional prediction.

[0135] predFlagL0 = (predListFlagB==0) ? 1 : 0 predFlagL1 = (predListFlagB==0) ? 0 : 1 refIdxL0 = (predListFlagB==0) ? refIdxB : -1 refIdxL1 = (predListFlagB==0) ? -1 : refIdxB mvL0[0] = (predListFlagB==0) ? mvB[0] : 0 mvL0[1] = (predListFlagB==0) ? mvB[1] : 0 mvL1[0] = (predListFlagB==0) ? 0 : mvB[0] mvL1[1] = (predListFlagB==0) ? 0 : mvB[1] Otherwise (sType is 2 and predListFlagA+predListFlagB is 1), the GPM predictor 30377 performs the following: where predListFlagA+predListFlagB being 1 indicates that the reference picture lists of A and B are different.

[0136] When the prediction list flag for A is 0 (predListFlagA==0), the GPM predictor 30377 performs bidirectional prediction by storing the motion vector of A in L0 and the motion vector of B in L1. When the prediction list flag for A is not 0 (predListFlagA!=0), the GPM predictor 30377 performs bidirectional prediction by storing the motion vector of B in L0 and the motion vector of A in L1.

[0137] predFlagL0 = 1 predFlagL1 = 1 refIdxL0 = (predListFlagA==0) ? refIdxA : refIdxB refIdxL1 = (predListFlagA==0) ? refIdxB : refIdxA mvL0[0] = (predListFlagA==0) ? mvA[0] : mvB[0] mvL0[1] = (predListFlagA==0) ? mvA[1] : mvB[1] mvL1[0] = (predListFlagA==0) ? mvB[0] : mvA[0] mvL1[1] = (predListFlagA==0) ? mvB[1] : mvA[1] (Inter-prediction image generation unit 309) When predMode indicates inter prediction mode, the inter prediction image generation unit 309 generates a prediction image of a block or sub-block by inter prediction using the inter prediction parameters and reference picture input from the inter prediction parameter derivation unit 303.

[0138] 4 is a schematic diagram showing the configuration of the inter-prediction image generation unit 309 included in the prediction image generation unit 308 according to this embodiment. The inter-prediction image generation unit 309 includes a motion compensation unit (prediction image generation device) 3091 and a synthesis unit 3095. The synthesis unit 3095 includes an IntraInter synthesis unit 30951, a GPM synthesis unit 30952, a BIO unit 30954, and a weighted prediction unit 3094. Intra-prediction image The predicted image derived by the generation unit 310 is used to generate predicted images by the IntraInter synthesis unit 30951 and the GPM synthesis unit 30952.

[0139] (Motion Compensation) The motion compensation unit 3091 (interpolated image generation unit 3091) generates an interpolated image (motion compensated image) by reading a reference block from the reference picture memory 306 based on the inter prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter prediction parameter derivation unit 303. The reference block is a block at a position shifted by mvLX from the position of the target block on the reference picture RefPicLX specified by refIdxLX. Here, if mvLX is not integer precision, a filter for generating pixels at decimal positions called a motion compensation filter is applied to generate an interpolated image.

[0140] The motion compensation unit 3091 first calculates the integer position (xInt, yInt) corresponding to the coordinates (x, y) in the prediction block and and phase (xFrac, yFrac) are calculated using the following formula:

[0141] xInt = xPb+(mvLX[0]>>(log2(MVPREC)))+x xFrac = mvLX[0]&(MVPREC-1) yInt = yPb+(mvLX[1]>>(log2(MVPREC)))+y yFrac = mvLX[1]&(MVPREC-1) Here, (xPb, yPb) are the upper left coordinates of a block of size bW*bH, where x=0...bW-1, y=0...bH-1, and MVPREC indicates the accuracy of mvLX (1 / MVPREC pixel accuracy), e.g., MVPREC=16.

[0142] The motion compensation unit 3091 derives a temporary image temp[][] by performing horizontal interpolation processing on the reference picture refImg using an interpolation filter. In the following, Σ is the sum for k=0..NTAP-1, shift1 is a normalization parameter that adjusts the value range, and offset1=1<<(shift1-1).

[0143] temp[x][y] = (ΣmcFilter[xFrac][k]*refImg[xInt+k-NTAP / 2+1][yInt]+offset1)>>shift1 Next, the motion compensation unit 3091 derives the interpolated image Pred[][] by vertically interpolating the temporary image temp[][]. In the following, Σ is the sum for k=0..NTAP-1, shift2 is a normalization parameter that adjusts the value range, and offset2=1<<(shift2-1).

[0144] Pred[x][y] = (ΣmcFilter[yFrac][k]*temp[x][y+k-NTAP / 2+1]+offset2)>>shift2 In the case of bi-prediction, the above Pred[][] is used to derive interpolated images PredL0[][] and PredL1[][] for each of the L0 and L1 lists, and an interpolated image Pred[][] is generated from PredL0[][] and PredL1[][].

[0145] (GPM synthesis section 30952) The GPM synthesis unit 30952 derives a predicted image using the temporary predicted images predSamplesLA and predSamplesLB and weighting parameters derived from the angle variables. The temporary predicted images predSamplesLA and predSamplesLB are predicted images generated by the motion compensation unit 3091 using the motion information (mvX, refIdxX, predListFlagX, X=A or B).

[0146] The GPM synthesis unit 30952 derives the predicted pixel value pbSample in the following procedure: Here, nCbW=cbWidth, and nCbH=cbHeight.

[0147] Next, the GPM synthesis unit 30952 derives the variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor as follows:

[0148] nW = (cIdx==0) ? nCbW : nCbW * SubWidthC nH = (cIdx==0) ? nCbH : nCbH * SubHeightC shift1 = Max(5, 17 - BitDepth) offset1 = 1 << (shift1 - 1) displacementX = angleIdx displacementY = (angleIdx + 8) % 32 partFlip = (angleIdx>=13 && angleIdx<=27) ? 0 : 1 shiftHor = (angleIdx%16==8 || (angleIdx%16!=0 && nH>=nW)) ? 0 : 1 The GPM synthesis unit 30952 derives offsetX and offsetY as follows.

[0149] if (shiftHor == 0) { offsetX = (-nW) >> 1 offsetY = ((-nH) >> 1) + (angleIdx<16? (distanceIdx * nH) >> 3 : -((distanceIdx * nH) >> 3)) } else / / shiftHor ==1 { offsetX = ((-nW) >> 1) + (angleIdx<16? (distanceIdx * nW) >> 3 : -((distanceIdx * nW) >> 3)) offsetY = (-nH) >> 1 } The GPM synthesis unit 30952 derives the predicted pixels pbSamples according to the following steps.

[0150] 1) The GPM synthesis unit 30952 derives generalized coordinates (xL, yL) from coordinates (x, y) within the CU.

[0151] xL = (cIdx==0) ? x : x * SubWidthC yL = (cIdx==0) ? y : y * SubHeightC 2) The GPM synthesis unit 30952 uses disLut shown in FIG. 9B to calculate a weighting coefficient wValue according to the coordinates within the CU as follows:

[0152] weightIdx = (((xL + offsetX) << 1) + 1) * disLut[displacementX] + (((yL + offsetY) << 1) + 1) * disLut[displacementY] weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx wValue = Clip3(0, 8, (weightIdxL + 4) >> 3) 3) The GPM synthesis unit 30952 derives the value of pbSample as follows:

[0153] pbSamples[x][y] = Clip3(0, (1 << BitDepth) - 1, (predSamplesLA[x][y] * wValue + predSamplesLB[x][y] * (8 - wValue) + offset1) >> shift1) (GPM mode with intra prediction) In the GPM mode, the GPM synthesis unit 30952 may generate a predicted image by a weighted sum of an intra-predicted image and an inter-predicted image. That is, one of the predSamplesLA and predSamplesLB may be an intra-predicted image predSamplesIntra, and the other may be an inter-predicted image predSamplesInter.

[0154] pbSamples[x][y] = Clip3(0, (1 << BitDepth) - 1, (predSamplesIntra[x][y] * wValue + predSamplesIner[x][y] * (8 - wValue) + offset1) >> shift1) At this time, when ciip_flag[x0][y0] is 0, that is, when the GPM mode is used, the CU decoding unit 3022 may decode merge_gpm_intra_flag from the encoded data, and if merge_gpm_intra_flag is true, may further decode IntraPredMode from the encoded data. Alternatively, the two predicted images may be configured to allow intra prediction. In this case, the CU decoding unit 3022 decodes merge_gpm_intra_flag0 and merge_gpm_intra_flag1. The merge_gpm_intra_flag0 and merge_gpm_intra_flag1 are flags indicating whether or not the predicted images predSamplesLA and predSamplesLB are derived by intra prediction.

[0155] (IntraInter synthesis processing) When ciip_mode is 1, the IntraInter synthesis unit 30951 generates a predicted image predSamplesComb[x][y] as a CIIP mode predicted image by a weighted sum of an inter predicted image and an intra predicted image.

[0156] predSamplesComb[x][y] =(wIntra * predSamplesIntra[x][y] +(4 - wIntra)*predSamplesInter[x][y] + 2)>> 2 Here, predSamplesIntra[x][y] is an intra-prediction image, which may be planar prediction. predSamplesInter[x][y] is an inter-prediction image. The IntraInter synthesis unit 30951 may derive a weight wIntra of the intra-prediction image according to the prediction modes of the left adjacent block and the upper adjacent block of the current block.

[0157] wIntra = 3 (CuPredMode of left block and top block are both PRED_INTRA) wIntra = 2 (one of the CuPredModes of the left block and the top block is PRED_INTRA) wIntra = 1 (otherwise, both left and top CuPredMode are not PRED_INTRA) Furthermore, the IntraInter synthesis unit 30951 may derive wIntra according to the intra prediction mode of the current block and the position of the pixel within the current block.

[0158] predSamplesComb[x][y] =(wIntra * predSamplesIntra[x][y] +(8 - wIntra)*predSamplesInter[x][y] + 4)>> 3 if (IntraPredMode == INTRA_PLANAR || IntraPredMode == INTRA_DC || nTbW <= 4 || nTbH <= 4) / / Player prediction or DC prediction or block size is small { wIntra = 6 (CuPredMode of left and top blocks are both PRED_INTRA) wIntra = 4 (one of the CuPredModes of the left block and the top block is PRED_INTRA) wIntra = 2 (otherwise, both left and top CuPredMode are not PRED_INTRA) } else if (IntraPredMode < DIA_IDX) / / In case of intra prediction mode closer to horizontal than 45 degrees to the top left wIntra = 6 (xC < (nTbW>>2)) wIntra = 5 (otherwise, xC < (nTbW>1)) wIntra = 3 (otherwise, xC < 3*(nTbW>>2)) wIntra = 2 (else, xC >= 3*(nTbW>>2)) } else { / / In case of intra prediction mode closer to vertical than 45 degrees to the top left wIntra = 6 (yC < (nTbH>>2)) wIntra = 5 (otherwise, yC < (nTbH>1)) wIntra = 3 (otherwise, yC < 3*(nTbH>>2)) wIntra = 2 (else, yC >= 3*(nTbH>>2)) } Here, (xC, yC) is the position of the target pixel, and nTbW and nTbH are the width and height of the target block. In the intra prediction mode close to horizontal (< DIA_IDX), the IntraInter synthesis unit 30951 gradually reduces the size of wIntra as the horizontal xC of the target pixel increases, and in the intra prediction mode close to vertical (>= DIA_IDX), the IntraInter synthesis unit 30951 gradually reduces the size of wIntra as yC increases. In other cases, the weight is changed according to the prediction mode of the adjacent block. In other words, the weight of the intra prediction image is reduced and the weight of the inter prediction image is increased as the distance from the boundary increases based on xC and yC.

[0159] (BIO forecast) In bi-prediction mode, the BIO unit 30954 generates a predicted image by referring to two predicted images (a first predicted image and a second predicted image) and a gradient correction term.

[0160] (Weighted prediction) The weighted prediction unit 3094 generates a predicted image of the block by multiplying the interpolated image PredLX by a weighting coefficient.

[0161] The inter-prediction image generation unit 309 outputs the generated prediction image of the block to the addition unit 312 .

[0162] (Intra-prediction image generation unit) When predMode indicates an intra prediction mode, the intra prediction image generation unit performs intra prediction using the intra prediction parameters input from the intra prediction parameter derivation unit and reference pixels read from the reference picture memory 306 .

[0163] The inverse quantization and inverse transform unit 311 inversely quantizes the quantized transform coefficients input from the parameter decoding unit 302. Quantize the signal to obtain the conversion coefficient.

[0164] (inverse quantization and inverse transformation) 11 is a block diagram showing the configuration of the inverse quantization and inverse transform unit 311 of this embodiment. The inverse quantization and inverse transform unit 311 includes a scaling unit 31111, an inverse non-separable transform unit 31121, and an inverse separate transform unit 31123.

[0165] The inverse quantization and inverse transform unit 311 obtains the transform coefficients d[][] by scaling (inverse quantization) the quantized transform coefficients qd[][] input from the entropy decoding unit 301 using a scaling unit 31111. The quantized transform coefficients qd[][] are coefficients obtained by performing a transform such as DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform) on the prediction error in the encoding process and quantizing the transform, or coefficients obtained by further performing a non-separable transform on the transformed coefficients. When lfnst_idx!=0, the inverse quantization and inverse transform unit 3111 performs an inverse transform using an inverse non-separable transform unit 31121. Furthermore, the transform coefficients are subjected to an inverse frequency transform such as an inverse DCT or an inverse DST to calculate the prediction error. Also, when lfnst_idx==0, the inverse non-separable transform unit 31121 is not performed, and inverse transforms such as inverse DCT and inverse DST are performed on the transform coefficients scaled by the scaling unit 31111 to calculate a prediction error. The inverse quantization and inverse transform unit 311 outputs the prediction error to the adder 312.

[0166] In addition, since the inverse transformation and the transformation are paired processes, the transformation and the inverse transformation may be interpreted as being interchangeable. Alternatively, when the inverse transformation is called the transformation, the transformation may be called the forward transformation. For example, when the inverse non-separable transformation is called the non-separable transformation, the non-separable transformation may be called the forward non-separable transformation. Moreover, the separable transformation is simply called the transformation.

[0167] (Details of Scaling Part 31111) The scaling unit 31111 uses the quantization parameter and scaling factor derived in the parameter decoding unit 302 to scale the transform coefficients decoded by the TU decoding unit using coefficient-by-coefficient weighting.

[0168] The scaled transform coefficients d[x][y] are transmitted to the inverse non-separable transform unit 31123 or the inverse non-separable transform unit 31121. The inverse non-separable transform unit 31121 applies an inverse non-separable transform to the transform coefficients d[][] after inverse quantization and before the separable transform.

[0169] (Non-separable transformation) Inverse non-separable transform is applied to transform coefficients of a part or the entire region of a TU in the video decoding device 31. After the inverse non-separable transform is applied, an inverse non-separable transform (DCT2, DST7, etc.) is applied to the transform coefficients after the inverse non-separable transform. In addition, when a TU is divided into 4x4 sub-blocks, the non-separable transform and the inverse non-separable transform are applied only to a predetermined sub-block at the top left. Examples of the size of a TU in which one of the width W and height H of the TU is 4 include 4×4, 8×4, 4×8, L×4, and 4×L (L is a natural number equal to or greater than 16).

[0170] Also, the technology of transmitting only some low-frequency components among the conversion coefficients after the separation conversion is called RST (Reduced Secondary Transform) conversion or LFNST (Low Frequency Non-Separable-Transform). Specifically, when the number nonZeroSize of the conversion coefficients of the non-separable conversion to be transmitted is less than or equal to the size of the separation conversion ((1<<log2StSize) x (1<<log2StSize)), it is called LFNST. In this specification, the conversion applied to the difference between the predicted image generated by the GPM synthesis unit 30952 and the original image is called GPM non-separable conversion (or GPM_LFNST). Hereinafter, the process of the inverse non-separable conversion will be described in order.

[0171] (S2000: Decoding of Non-Separable Conversion Index) FIG. 12 and FIG. 13 are examples of syntax tables showing the flags of non-separable conversion. FIG. 12 shows an example of notifying lfnst_idx and mtx_idx in the CU, and FIG. 13 shows an example of notifying lfnst_idx and mtx_idx in the TU. lfnst_idx is an index indicating the use or non-use of non-separable conversion and the conversion matrix, and mts_idx is an index indicating the conversion matrix of separable conversion.

[0172] The TU decoding unit 3024 decodes lfnst_idx and mts_idx from the encoded data. It may be configured to decode mts_idx after lfnst_idx and decode mtx_idx only when lfnst_idx is 0. That is, a configuration may be used in which a conversion matrix other than DCT2 indicated by mtx_idx!=0 is used only when non-separable conversion is not used (lfnst_idx is 0).

[0173] As shown in the figure, the TU decoding unit 3024 decodes lfnst_idx according to the determination shown in the following syntax table. if(condA&&(condB||condC)) lfnst_idx Here, condA, condB, and condC are derived by the following formula. condA:sps_lfnst_enabled_flag==1 && (IntraSubPartitionsSplitType!=ISP_NO_SPLIT || LfnstDcOnly==0) && LfnstZeroOutSigCoeffFlag==1 condB: CuPredMode[chType][x0][y0]==MODE_INTRA condC: MergeGpmFlag==1 || ciip_flag B includes in its determination formula that transform coefficients are zeroed out (LfnstZeroOutSigCoeffFlag==1), that non-separable transform is available, and that intra prediction is performed.

[0174] C is a judgment of inter prediction (GPM prediction or CIIP mode). Note that, when regular_merge_flag==0 and the GPM mode or the CIIP mode is used, condC may be the following judgment. condC: regular_merge_mode==0 Note that when regular_merge_mode is not decoded in the intra prediction mode (in this case, general_merge_flag=0, merge_subblock_flag=0), the same determination is possible even without condB by utilizing the fact that regular_merge_mode=0 from regular_merge_mode=general_merge_flag && !merge_subblock_flag. That is, the TU decoding unit 3024 may decode lfnst_idx using the following determination. if(condA&& regular_merge_mode==0) Decode lfnst_idx That is, in the case of intra prediction, or the GPM mode or the CIIP mode, the TU decoding unit 3024 may decode lfnst_idx from the encoded data.

[0175] Furthermore, the TU decoding unit 3024 may derive the value of ApplyLfnstFlag using lfnst_idx in the case of intra prediction, and without using lfnst_idx in cases other than intra prediction.

[0176] ApplyLfnstFlag = ((MergeGpmFlag==1) || (CuPredMode==MODE_INTRA && lfnst_idx>0)) ? 1 : 0 The following is also fine.

[0177] ApplyLfnstFlag = (((CuPredMode!=MODE_INTRA && regular_merge_flag == 0) || lfnst_idx>0)) ? 1 : 0 (Deriving the transformation matrix) The inverse non-separable transform unit 31121 derives a transform matrix secTransMatrix[][] used in the transform process.

[0178] Specifically, when the prediction mode is intra prediction, the inverse non-separable transform unit 31121 derives lfnstTrSetId from the intra prediction mode IntraPredMode. When the prediction mode is GPM prediction, the inverse non-separable transform unit 31121 derives lfnstTrSetId from the GPM angle index angleIdx (or the GPM mode number merge_gpm_partition_idx). lfnstTrSetId is a number indicating a set of transform matrices for non-separable transform. Note that the transform matrix (or set of transform matrices) corresponding to lfnstTrSetId=K is hereinafter referred to as MK.

[0179] Fig. 14 is a table used for a transformation matrix of non-separable transformation in intra prediction mode. The inverse non-separable transformation unit 31121 may derive a matrix of non-separable transformation, here a set number (lfnstTrSetId) of a transformation matrix, from the intra prediction mode IntraPredMode using the table in Fig. 14. Alternatively, lfnstTrSetId may be derived using a lookup table nstSetTableIntra as follows.

[0180] lfnstTrSetIdx = nstSetTableIntra[IntraPredMode] nstSetTableIntra[] = {2, 2, 3, 3, 3, 3, 3, 2, 2, 2, 1, 1, 1, 1, 1, 2, 2, 2,, 3, 3, 3, 3, 3, 2, 2, 2, 1, 1, 1, 1, 1, 2} The non-separable transform can use a plurality of predetermined transform matrices, and a suitable transform can be achieved by selecting an appropriate transform matrix from the intra prediction mode or the GPM mode.

[0181] The following example describes the processing in the case of a lookup table.

[0182] The inverse non-separable transform unit 31121 derives the transform matrix secTransMatrix[][] from the non-separable transform set number (lfnstTrSetId), the lfnst_idx indicating the transform matrix of the non-separable transform, and the non-separable transform size nStSize (nTrS). For example, it may be derived as follows secTransMatrix[][] = secTransMatrixTbl[nTrS][lfnstTrSetId][lfnst_idx - 1][][] The following describes the non-separable transform process in order.

[0183] (S2201: Setting the transform size and the input / output size) The inverse non-separable transform unit 31121 derives the transform matrix size nStSize (= 1 << log2StSize), the logarithmic value log2StSize of the transform matrix size, the number nonZeroSize of input transform coefficients, and the number nStOutSize of output transform coefficients according to the size (width nTbW, height nTbH) of the TU.

[0184] For example, it may be derived by the following formula.

[0185] nStOutSize = (nTbW>=16 && nTbH>=16) ? 96 : (nTbW>=8 && nTbH>=8) ? 64 : 16 log2StSize = (nTbW>=16 && nTbH>=16) ? 4 : (nTbW>=8 && nTbH>=8) ? 3 : 2 It may also be derived using the following formula:

[0186] nStOutSize = (nTbW>=16 && nTbH>=16) ? 96 : (nTbW>=8 && nTbH>=8) ? 32 : 16 log2StSize = (nTbW>=16 && nTbH>=16) ? 4 : (nTbW>=8 && nTbH>=8) ? 3 : 2 It may also be derived using the following formula:

[0187] nStOutSize = ((nTbW>=8 && nTbH>=8) ? 64 : 16 log2StSize = ((nTbW>=8 && nTbH>=8) ? 3 : 2 The inverse non-separable transform unit 31121 may derive nonZeroSize as follows.

[0188] nonZeroSize = (nTbW>=16 && nTbH>=16) ? 32 : 16 nonZeroSize may be derived as follows:

[0189] nonZeroSize = (nTbW>=16 && nTbH>=16) ? 32 : (nTbW==8 && nTbH==8)) ? 8 : 16 nonZeroSize may be derived as follows:

[0190] nonZeroSize = (nTbW==4 && nTbH==4) || (nTbW==8 && nTbH==8)) ? 8 : 16 Note that nonZeroSize is not limited to 8 and 16. For example, it may be 12 or 32. nStOutSize is also not limited to 16 and 64. It may be 32, 48, 64, 96, or the like.

[0191] For example, the following would be acceptable: If both nTbW and nTbH are greater than or equal to 8, set log2StSize=3, nStOutSize=48. Otherwise, set log2StSize=2, nStOutSize=16. If nTbW and nTbH are both 4 or both 8, set nonZeroSize=8. Otherwise, set nonZeroSize=16. The above can be expressed as the following formula.

[0192] nStOutSize = ( nTbW >= 8 && nTbH >= 8 ) ? 48 : 16 nonZeroSize = ( ( nTbW == 4 && nTbH == 4 ) || (nTbW == 8 && nTbH == 8 ) ) ? 8 : 16 log2StSize = ( nTbW >= 8 && nTbH >= 8 ) ? 3 : 2 Furthermore, the number of sub-blocks to which the inverse non-separable transform is applied, numStX and numStY, may be derived according to the size of the TU. numStX = (nTbH==4 && nTbW>8) ? 2 : 1 numStY = (nTbW==4 && nTbH>8) ? 2 : 1 It is also possible to always set numStX=numStY without performing non-separable transformation on a plurality of sub-blocks.

[0193] (S2202: Rearrange into a one-dimensional array) The inverse non-separable transform unit 31121 rearranges some transform coefficients d[][] of the TU into a one-dimensional array u[]. Specifically, the inverse non-separable transform unit 31121 derives u[] from the two-dimensional transform coefficients d[][] of the target TU indicated by the region RU by referring to the transform coefficients of x=0..nonZeroSize-1. xC and yC are positions on the TU and are derived from the array DiagScanOrder indicating the scan order and the position x of the array u[].

[0194] xC = (xSbIdx< <log2StSize) + DiagScanOrder[log2StSize][log2StSize][x][0] yC = (ySbIdx< <log2StSize) + DiagScanOrder[log2StSize][log2StSize][x][1] u[x] = d[xC][yC] If you want to process only the top left subblock, xC = DiagScanOrder[log2StSize][log2StSize][x][0] yC = DiagScanOrder[log2StSize][log2StSize][x][1] u[x] = d[xC][yC] The range copied to the one-dimensional array is called an area RU.

[0195] (S2203: Applying conversion processing) The inverse non-separable transform unit 31121 performs transform on u[] of length nonZeroSize using the transform matrix secTransMatrix[][], and derives coefficients v[] of a one-dimensional array of length nStOutSize as output.

[0196] v[i] = Clip3(CoeffMin, CoeffMax,Σ(secTransMatrix[i][j]*u[j]+64)>>7) Here, Σ is the sum from j=0..nonZeroSize-1. Also, processing is performed for i=0..nStSize-1. CoeffMin and CoeffMax indicate the range of values ​​of the transform coefficients. As described later, secTransMatrixDC may be used as secTransMatrix.

[0197] (S2204: 2D arrangement of 1D array after conversion process) The inverse non-separable transform unit 31121 again places the transformed coefficients v[] of the one-dimensional array in a predetermined position in the TU (d[][]). Specifically, the following process is performed on x = 0..nStSize - 1, y = 0..nStSize - 1.

[0198] if (transposeFlag==0) d[x][y] = (y<4&&x<0) ? v[x+(y< <log2StSize)] : d[x][y]) else d[x][y] = (x<4&&y<0) ? v[y+(x< <log2StSize)] : d[x][y]) At this time, transposition processing is performed according to the symmetry used in deriving the transformation matrix. Whether or not to perform transposition processing is determined by the coefficient transposition processing flag transposeFlag.

[0199] (Transposition flag derivation in case of intra prediction) When the prediction mode is intra prediction, the inverse non-separable transform unit 31121 sets transposeFlag=1 when the intra prediction mode PredModeIntra>34, and sets transposeFlag=0 when PredModeIntra<=34. PredModeIntra is the prediction direction shown in FIG.

[0200] In the case of GPM prediction, the inverse non-separable transform unit 31121 sets the transposeFlag according to merge_gpm_partition_idx. For example, when merge_gpm_partition_idx <= 13 or merge_gpm_partition_idx > 46, transposeFlag is set to 1, and when 13 < merge_gpm_partition_idx <= 46, transposeFlag is set to 0. Also, it may be derived from angleIdx. transposeFlag = (0 < angleIdx - 4 && angleIdx - 4 <= 16)? 0 : 1 When nStOutSize == 64 (64 coefficients), the inverse non-separable transform unit 31121 may perform the following processing.

[0201] if (transposeFlag == 0) d[x][y] = (x < 8 && y < 8)? v[x+(y << log2StSize)] : d[x][y] else d[x][y] = (x < 8 && y < 8)? v[y+(x << log2StSize)] : d[x][y] When nStOutSize == 96 (96 coefficients), the inverse non-separable transform unit 31121 may perform the following processing.

[0202] if (transposeFlag == 0) d[x][y] = (y < 4)? v[x+(y << log2StSize)] : (y < 8)? v[x + 48+((y - 4) << 3)] : (y < 12) ? v[x + 80+((y - 8) << 2)] : d[x][y]) else d[x][y] = (x < 4)? v[y+(x << log2StSize)] : (x < 8)? v[y + 48+((x - 4) << 3] : (x < 12)? v[y + 80+((x - 8) << 2)] : d[x][y]) Furthermore, when nStOutSize==48 (48 coefficients), the inverse non-separable transform unit 31121 may perform the following processing.

[0203] if (transposeFlag==0) d[x][y] = (y<4) ? v[x+(y< <log2StSize)] : ((x<4) ? v[32+x+((y-4)<<2)] : d[x][y]) else d[x][y] = (x<4) ? v[y+(x< <log2StSize)] : ((y<4) ? v[32+y+((x-4)<<2)] : d[x][y]) <Configuration for switching non-separable transform depending on prediction mode> FIG. 15 is a flowchart showing the process of switching the non-separable transformation depending on the prediction mode.

[0204] (S2230) Depending on the prediction mode, transition is made to S2231 or S2232. In a prediction mode with a large number of transform coefficients such as intra prediction, transition is made to S2231, and in the case of inter prediction with a small number of transform coefficients, transition is preferably made to S2232. In addition, in inter prediction using pixels other than adjacent pixels of the target block as reference pixels, the correlation between DC components and AC components is low. In such a case, transition is made to S2232, and in other cases, transition is made to S2231. For example, as shown in the figure, transition is made to S2231 in the case of intra prediction, and transition is made to S2232 in other cases. Note that transition is made to S2231 in the case of intra prediction mode or CIIP mode, and transition is made to S2232 in other cases. Also, transition is made to S2231 in the case of intra prediction mode, CIIP mode, or GPM using intra prediction, and transition is made to S2232 in other cases.

[0205] (S2231) The inverse non-separable transform unit 31121 performs non-separable transform on transform coefficients including a specific frequency component (DC component).

[0206] (S2232) The inverse non-separable transform unit 31121 performs a non-separable transform on the transform coefficients so as to "preserve" the specific frequency component (DC component). Here, "preserving" means not changing the value. However, "preserving" may also include the case where the polarity is inverted. For example, a non-separable transform may be performed on the transform coefficients excluding the specific frequency component (DC component). For example, a transform matrix may be used that performs a non-separable transform on the transform coefficients including the specific frequency component (DC component) but does not change the DC component.

[0207] According to the above configuration, the non-separated transform unit 31121 is characterized in that it does not change a specific frequency component of the transform coefficient. Also, the inverse non-separated transform unit 31121 may be configured not to change the DC component, which is the upper left component. Also, as shown in the figure, the inverse non-separated transform unit 31121 may be configured not to change the DC component, which is the upper left component, when the prediction mode of the current block is inter prediction.

[0208] According to the above configuration, even when non-zero transform coefficients are concentrated in a specific frequency (for example, DC components) in frequency transforms such as DCT2 and DST7 for inter prediction, and the number of non-zero coefficients is small, the effect of non-separable transform can be obtained, and the effect of improving coding efficiency is achieved. That is, it is possible to prevent non-zero transform coefficients concentrated in a specific frequency from being dispersed to coefficients other than the specific frequency by non-separable transform (reducing the concentration degree conversely). In intra prediction, which performs prediction from adjacent pixels near the target block, there is a correlation between the residual of the DC coefficient and the residual of the AC coefficient, and non-separable transform between the DC coefficient and the AC coefficient is effective. However, in the case of inter prediction, which performs prediction from an image in a different time and layer, there is a change in luminance and chrominance called DC offset, and the correlation between the residual of the DC coefficient and the residual of the AC coefficient is indefinite. In such a case, the effect of improving coding efficiency is achieved by non-separable transform that preserves the DC component.

[0209] <Configuration for converting components excluding DC components in non-separable conversion> In the non-separable transform, the value of the DC component, which is the lowest-order transform coefficient, is output as it is, and a configuration for transforming components excluding the DC component (AC component) will be described. As configurations, there are a method of performing transformation excluding the DC coefficient and a configuration of performing transformation using a transformation matrix that does not change the DC coefficient. Also, the inverse non-separable transform unit 31121 may derive a one-dimensional array excluding the DC component and perform transformation on the one-dimensional array (Configuration 1, Configuration 2). The inverse non-separable transform unit 31121 may use a transformation matrix in which the leftmost component and the uppermost component excluding the upper-left component are 0 (Configuration 3). According to the following configuration, in the case of prediction from pixels at different times, or when the correlation between the residual of the DC coefficient and the residual of the AC coefficient is indeterminate, the correlation of the DC coefficient is not used. The non-separable transform that preserves the DC component has the effect of improving the coding efficiency.

[0210] <Configuration 1 of non-separable transform excluding DC component> FIG. 16 is a diagram showing one configuration of the operation of the inverse non-separable transform that preserves the DC transform.

[0211] As shown in the figure, the inverse non-separable transform unit 31121 performs inverse non-separable transform processing other than the DC component by changing S2202 and S2204 among S2201 to S2205 already described.

[0212] (S2202NonDC: Rearrange into one-dimensional array) The inverse non-separable transform unit 31121 rearranges some of the transform coefficients d[][] of the TU into a one-dimensional array u[]. Specifically, the inverse non-separable transform unit 31121 derives u[] from the two-dimensional transform coefficients d[][] of the target TU indicated by the region RU, referring to the transform coefficients for x = 1..nonZeroSize - 1 (excluding the component where the leading component x == 0 in the scan order). xC and yC are positions on the TU, and DiagSca is an array indicating the scan order Derive from nOrder and the position x of the array u[].

[0213] xC = (xSbIdx << log2StSize) + DiagScanOrder[log2StSize][log2StSize][x][0] yC = (ySbIdx< <log2StSize) + DiagScanOrder[log2StSize][log2StSize][x][1] u[x-1] = d[xC][yC] If you want to process only the top left subblock, xC = DiagScanOrder[log2StSize][log2StSize][x][0] yC = DiagScanOrder[log2StSize][log2StSize][x][1] u[x-1] = d[xC][yC] Here, the scan position is x=1..nonZeroSize-1, and a one-dimensional array u[x] is derived from the residual excluding the DC component. As shown in S2203, the inverse non-separable transform unit 31121 transforms u[x] to v[] using a transformation matrix.

[0214] (S2204NonDC: 2D arrangement of 1D array after conversion processing) The inverse non-separable transform unit 31121 places the transformed coefficient v[] of the one-dimensional array again at a predetermined position in the TU (d[][]). Specifically, the following process is performed for x = 0..nStSize-1, y = 0..nStSize-1. The feature is that the value of the DC position d[0][0] is not changed.

[0215] if (transposeFlag==0) d[x][y] = ((x!=0 || y!=0) && y<4 && x<0) ? v[x-1+(y< <log2StSize)] : d[x][y]) else d[x][y] = ((x!=0 || y!=0) && x<4 && y<0) ? v[y-1+(x<<log2StSize)] : d[x][y]). Here, when it is not the DC component (x!=0 || y!=0), v[] is selected, and in other cases (x==0 && y==0), d[x][y] is used as it is. The determination of (x!=0 || y!=0) may use (x + y!=0). Also, "!=0" may use ">0". Also, when assigning from the one-dimensional array v[] to the two-dimensional array d[][] with x = 0..nStSize - 1, y = 0..nStSize - 1, the assignment may be performed except for x = 0, y = 0 (scanning may be performed except for x = 0, y = 0).

[0216] <Configuration 2 of non-separable transform excluding DC component> Figure 17 is a diagram showing another configuration of the operation of the inverse non-separable transform that preserves the DC transform.

[0217] As shown in the figure, the inverse non-separable transform unit 31121 performs the inverse non-separable transform process for components other than the DC component by changing S2203 among S2201 to S2205 already described.

[0218] (S2203KeepDC: Application of transformation process) The inverse non-separable transform unit 31121 performs a transform on u[] with a length of nonZeroSize using the transform matrix secTransMatrix[][], and derives the coefficient v[] of a one-dimensional array with a length of nStOutSize as the output. It is characterized by not changing the value of the DC position v[0] (v[0]=u[0]).

[0219] v[i] = u[i] (i==0) v[i] = Clip3(CoeffMin, CoeffMax,Σ(secTransMatrix[i][j]*u[j]+64) >> 7) (i!=0) Here, Σ is the sum from j = 1 to nonZeroSize - 1. Also, i processes for i = 1 to nStSize - 1. It may also be the following (i = 0 to nStSize - 2, j = 0 to nonZeroSize - 2).

[0220] v[i + 1] = Clip3(CoeffMin, CoeffMax, Σ(secTransMatrix[i][j] * u[j + 1] + 64) >> 7) (i!= 0) In the above, at the position of DC (i == 0), a configuration is shown where the value of DC is not changed by performing a different operation from other positions. However, a matrix secTransMatrixDC that does not change the value of DC may also be used.

[0221] <Configuration 3 of non-separable transformation excluding DC component> FIG. 18 is a diagram showing a transformation that preserves DC transformation. As shown in the figure, the inverse non-separable transformation unit 31121 performs an inverse non-separable transformation that preserves the DC value by using, as the matrix used in S2203 among S2201 to S2205 already described, secTransMatrixDC having a predetermined characteristic.

[0222] Specifically, as shown in (a) of the figure, secTransMatrixDC is a transformation matrix in which the value of the upper left component is set to 1 << shiftVal (for example, shiftVal = 7), the components of the upper end column are set to 0, and the components of the left end column are set to 0. By using secTransMatrixDC, an Identical transformation can be applied to the leading (DC) component of the input vector in the forward and inverse transformations of the non-separable transformation.

[0223] secTransMatrixDC[0][0] = 1 << shiftVal secTransMatrixDC[i][0] = 0 (i = 0 to nStSize - 1) secTransMatrixDC[0][j] = 0 (j = 0 to nonZeroSize - 1) Here, the product of secTransMatrixDC and the transposed matrix of secTransMatrixDC may be an integer multiple of the identity matrix (1<<(shiftVal+1)).

[0224] Alternatively, as shown in (b) of the figure, secTransMatrixDC is a transformation matrix with the upper left set to (1<<shiftVal)-1 (for example, shiftVal = 7), the top row set to 0, and the leftmost column set to 0. By using secTransMatrixDC, in the forward and inverse non-separable transforms, an Identical transform can be applied to the leading (DC) component of the input vector.

[0225] secTransMatrixDC[0][0] = (1 << shiftVal)-1 secTransMatrixDC[i][0] = 0 (i=0..nStSize-1) secTransMatrixDC[0][j] = 0 (j=0..nonZeroSize-1) Normally, signed variables use a range such as -128 to 127 (-2^shiftVal to 2^shiftVal-1). Thus, by using a value of (1<<shiftVal)-1 (for example, 127) as above, the range of 1-bit values can be reduced, reducing the load of multiplication processing.

[0226] Also, as shown in (c) of the figure, a value of -(1<<shiftVal) (for example, -128) may be used. In this case, the sign of the DC value is inverted like v[0]=-u[0] before and after the non-separable transform and inverse non-separable transform processes, but since the coefficient with the inverted sign in the encoder is inverted again in the decoder to obtain the same value, there is no problem.

[0227] secTransMatrixDC[0][0] = -(1 << shiftVal) secTransMatrixDC[i][0] = 0 (i=0..nStSize-1) secTransMatrixDC[0][j] = 0 (j = 0..nonZeroSize - 1) Also in this case, the range of 1-bit values can be reduced, and the load of the multiplication process can be reduced.

[0228] (Details of the non-separable conversion process) (Example of performing LFNST conversion in the CIIP mode) Hereinafter, in an example of performing LFNST conversion in the CIIP mode (or the GPM mode using an intra-predicted image), a method for deriving lfnstTrSetId that defines the conversion matrix of the non-separable conversion will be described.

[0229] FIG. 19 shows a flowchart of the process of performing LFNST in the CIIP mode.

[0230] (S2101) When performing non-separable conversion (ApplyLfnstFlag!= 0), transition to S2102A. ApplyLfnstFlag is a flag indicating whether to perform LFNST. When ApplyLfnstFlag is 1 (when greater than 0, or when other than 0), perform LFNST (non-separable conversion), and otherwise, skip the non-separable conversion and transition to S2300 to perform separable conversion. Note that when performing non-separable conversion only on luminance, the determination of whether to perform non-separable conversion may be made using ApplyLfnstFlag.

[0231] if (treeType == SINGLE_TREE or DUAL_TREE_LUMA) ApplyLfnstFlag[0] = (lfnst_idx > 0)? 1 : 0 else ApplyLfnstFlag[cIdx] = (lfnst_idx > 0 && treeType == DUAL_TREE_CHROMA)? 1 : 0 Here, treeType is any one of SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA, and cIdx is a variable that is 0 for luminance and 1 or 2 for color difference.

[0232] (S2102A) In the case of intra prediction mode, the process transitions to S2103A.

[0233] (S2103A) In the case of intra prediction mode, lfnstTrSetId is derived from IntraPredMode. Also, the transformation matrix secTransMatrix[][] is derived from lfnstTrSetId, the block size, and lfnst_idx.

[0234] (S2102C) If the mode is not intra prediction mode, the process transitions to S2102C, and a determination is made as to whether the mode is CIIP mode (ciip_flag!=0). This determination may be a determination as to whether the mode is CIIP mode or GPM using intra prediction. If the mode is not CIIP mode (and not GPM using intra prediction), the non-separable transform is skipped, and the process transitions to S2300, where separate transform is performed.

[0235] (S2103C) Derive lfnstTrSetId from IntraPredMode. Also derive the transformation matrix secTransMatrix[][] from lfnstTrSetId, block size, and lfnst_idx. IntraPredMode is used to derive an intra-predicted image in the case of CIIP mode or GPM mode that uses intra prediction.

[0236] (S2200) The inverse non-separable transform unit 31121 performs non-separable transform using the transform matrix secTransMatrix[][]. Specifically, the processes from S2201 to S2204 described above may be performed. Different transform matrices may be used for non-separable transform of intra prediction and non-separable transform of inter prediction (CIIP mode or GPM mode). Non-separable transform that removes or preserves DC components (configuration 1, configuration 2, or configuration 3 described above) may be used for non-separable transform of CIIP mode or GPM mode.

[0237] (S2300) The inverse separation transform unit 31123 performs separation transform on transform coefficients that have been non-separately transformed or transform coefficients that have not been non-separately transformed.

[0238] According to the above configuration, since the non-separable conversion is performed even in the CIIP mode that generates a predicted image close to intra prediction, the effect of improving the coding efficiency is achieved. Furthermore, the non-separable conversion may be performed when the prediction mode is not limited to the CIIP mode, but is the CIIP mode or the GPM mode using intra prediction. Whether the prediction mode is the CIIP mode or the GPM mode using intra prediction may be determined by the following generalizedIntraInter. generalizedIntraInter = ciip_flag[x0][y0] || merge_gpm_intra_flag0 || merge_gpm_intra_flag1 Alternatively, it may be derived as follows. generalizedIntraInter = ciip_flag[x0][y0] || merge_gpm_intra_flag In the case of inter prediction, non-separate conversion processing may always be performed without decoding lfnst_idx. Fig. 20 is a flowchart showing a configuration for performing LFNST processing when the current block is inter predicted (CuPredMode!=MODE_INTRA).

[0239] (S3111) It is determined whether the prediction mode is the intra mode. If NO (the prediction mode is the inter mode), the process proceeds to S3112C. The intra mode may be determined by using CuPredMode!=MODE_INTRA or CuPredMode=MODE_INTER.

[0240] (S3112A) When the prediction mode is intra, it is determined whether the number of non-zero transform coefficients is not small (for example, a flag indicating that there are not only DC coefficients, LfnstDcOnly==0) and the values ​​of the transform coefficients in a predetermined high frequency region are all 0 (LfnstZeroOutSigCoeffFlag==1). If the determination is YES, the process transitions to S3113B.

[0241] (S3112B) When the prediction mode is intra mode and non-separable transform is performed, lfnst_idx is decoded from the encoded data.

[0242] (S3112C) When the prediction mode is inter mode, it is determined whether the number of non-zero transform coefficients is not small (for example, a flag indicating other than DC coefficients, LfnstDcOnly == 0) and the values ​​of transform coefficients in a predetermined high frequency region are all 0 (LfnstZeroOutSigCoeffFlag == 1). If the determination is YES, the process transitions to S3113. At this time, a non-zero value is inferred for lfnst_idx.

[0243] (S3113) The inverse non-separable transform unit 31121 performs non-separable transform. A method for deriving a transform matrix used in the non-separable transform will be described separately.

[0244] In (S3112C), the value of lfnst_idx is derived in the case of inter prediction without being derived from the encoded data, but ApplyLfnstFlag may be set to TRUE as follows.

[0245] ApplyLfnstFlag = (lfnstDcOnly==0 && LfnstZeroOutSigCoeffFlag==1) ? 1 : 0 Here, LfnstZeroOutSigCoeffFlag is a flag indicating the presence or absence of a non-zero transform coefficient in the zero-out region, and when it is TRUE, it indicates that there is no non-zero coefficient in the zero-out region.

[0246] Furthermore, lfnst_idx may also be decoded by inter prediction, and LFNST may be performed when lfnst_idx>0. In this case, it is derived as follows.

[0247] ApplyLfnstFlag = (lfnstDcOnly==0 && LfnstZeroOutSigCoeffFlag==1 && lfnst_idx>0) ? 1 : 0 Taking into account the case of intra prediction, ApplyLfnstFlag may be derived as follows.

[0248] ApplyLfnstFlag = (lfnstDcOnly==0 && LfnstZeroOutSigCoeffFlag==1) && (CuPredMode!=MODE_INTER || (CuPredMode!=MODE_INTRA && lfnst_idx>0)) ? 1 : 0 If lfnst_idx does not appear in the encoded data, lfnst_idx may be inferred as 0.

[0249] Furthermore, if lfnst_idx>0, LFNST may always be performed and ApplyLfnstFlag may be derived as follows:

[0250] ApplyLfnstFlag = (CuPredMode==MODE_INTER && LfnstZeroOutSigCoeffFlag==1 && lfnstDcOnly==0) || (lfnst_idx>0)) ? 1 : 0 Note that the above ApplyLfnstFlag = (CuPredMode == MODE_INTER && LfnstZeroOutSigCoeffFlag == 1 && lfnstDcOnly == 0) || lfnst_idx > 0) may be CuPredMode != MODE_INTRA.

[0251] FIG. 21 is a flowchart showing a configuration for performing LFNST processing when the current block is inter-predicted (CuPredMode !=MODE_INTRA).

[0252] (S3111) It is determined whether the prediction mode is the intra mode. If NO (if the prediction mode is the inter mode), the process proceeds to S3112. The intra mode may be determined using CuPredMode!=MODE_INTRA or CuPredMode=MODE_INTER.

[0253] (S3112) When the prediction mode is inter mode, it is determined whether the number of non-zero transform coefficients is not small (for example, a flag indicating other than DC coefficients LfnstDcOnly==0) and the values ​​of transform coefficients in a predetermined high frequency region are all 0 (LfnstZeroOutSigCoeffFlag==1). If the determination is YES, the process transitions to S3113. Otherwise, LFNST processing is not performed. Also, the determination in (S3112) may use (LfnstDcOnly==0) && (LfnstZeroOutSigCoeffFlag==1) && AddCondInterLfnst. AddCondInterLfnst is a flag indicating whether the block size of the target block is within a predetermined range, or whether it is luminance or not, etc. LfnstDcOnly is a flag indicating that a DC coefficient is not included.

[0254] (S3113) The inverse non-separable transform unit 31121 performs non-separable transform. A method for deriving a transform matrix used in the non-separable transform will be described separately.

[0255] According to the above configuration, LFNST is not performed when the prediction mode is inter mode and the transform coefficient is only DC. Therefore, by applying LFNST transform to the DC coefficient, the transform coefficients already concentrated at one coefficient position (DC) can be dispersed, and waste such as causing unnecessary frequency restrictions can be reduced.

[0256] Alternatively, the CU decoding unit 3022 and the CU encoding unit 1112 may be configured to derive ApplyLfnstFlag using the following conditional expression and determine whether to perform LFNST using ApplyLfnstFlag.

[0257] ApplyLfnstFlag = (CuPredMode==MODE_INTER && LfnstDcOnly == 0 && LfnstZeroOutSigCoeffFlag==1) || (lfnst_idx>0)) ? 1 : 0 Alternatively, as a process for when lfnst_idx does not appear in the encoded data, lfnst_idx may be derived and ApplyLfnstFlag may be derived from lfnst_idx as follows: When lfnst_idx does not appear in the encoded data, lfnst_idx is derived as follows.

[0258] lfnst_idx = (CuPredMode==MODE_INTER && LfnstDcOnly == 0 && LfnstZeroOutSigCoeffFlag==1) ? 1 : 0 (Configuration 2, which performs non-separable transformation depending on zero-out, and GPM forecast configuration) In the case of GPM prediction, a configuration will be described in which lfnst_idx is not decoded and non-separable transform processing is always performed. In this configuration, when the transform coefficients are zeroed out in GPM prediction, ApplyLfnstFlag is set to TRUE.

[0259] ApplyLfnstFlag = ((MergeGpmFlag==1 && LfnstZeroOutSigCoeffFlag==1) || (CuPredMode!=MODE_INTRA && lfnst_idx>0))) ? 1 : 0 It may also be derived as follows.

[0260] ApplyLfnstFlag = ((MergeGpmFlag==1 && LfnstZeroOutSigCoeffFlag==1) || lfnst_idx>0)) ? 1 : 0 If ApplyLfnstFlag is other than 0, the inverse non-separable transform unit performs the inverse non-separable transform.

[0261] The adder 312 generates a decoded image of the block by adding, for each pixel, the predicted image of the block input from the predicted image generation unit 308 and the prediction error input from the inverse quantization and inverse transform unit 311. The adder 312 stores the decoded image of the block in the reference picture memory 306, and also outputs it to the loop filter 305.

[0262] (Configuration of a video encoding device) Next, the configuration of the video encoding device 11 according to this embodiment will be described. Fig. 22 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 includes a predicted image generating unit 101, a subtraction unit 102, a transform / quantization unit 103, an inverse quantization / inverse transform unit 105, an addition unit 106, and a transform / quantization unit 107. 6, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, an encoding parameter determination unit 110, a parameter encoding unit 111, a prediction parameter derivation unit 120, and an entropy encoding unit 104.

[0263] The predicted image generation unit 101 generates a predicted image for each CU. The predicted image generation unit 101 includes the inter predicted image generation unit 309 and the intra predicted image generation unit already described, and therefore a description thereof will be omitted.

[0264] The subtraction unit 102 generates a prediction error by subtracting the pixel values ​​of the predicted image of the block input from the predicted image generation unit 101 from the pixel values ​​of the image T. The subtraction unit 102 outputs the prediction error to the transformation and quantization unit 103.

[0265] The transform / quantization unit 103 calculates transform coefficients by frequency transforming the prediction errors input from the subtraction unit 102, and derives quantized transform coefficients by quantizing the prediction errors. The transform / quantization unit 103 outputs the quantized transform coefficients to the parameter coding unit 111 and the inverse quantization / inverse transform unit 105.

[0266] The transform / quantization unit 103 includes a separation transform unit (first transform unit), a non-separation transform unit (second transform unit), and a scaling unit.

[0267] The separate transform unit applies a separate transform to the prediction error. If lfnst_idx is not 0, the non-separable transform unit performs a non-separable transform. The scaling unit scales the transform coefficients with a quantization matrix.

[0268] Specifically, the transformation and quantization unit 103 performs the following processes.

[0269] S1: The separation conversion unit performs separation conversion.

[0270] S2: The non-separable transform unit determines whether the non-separable transform is valid (lfnst_idx!=0).

[0271] S3: If the non-separable transform is valid, the non-separable transform unit performs a non-separable transform (forward non-separable transform).

[0272] S3B: If the non-separable transformation is not valid, the non-separable transformation unit does not perform the non-separable transformation (forward non-separable transformation).

[0273] S4: The scaling unit performs scaling on the transform coefficients using a quantization matrix.

[0274] The (forward) non-separable transform performs approximately the same processing as the inverse non-separable transform.

[0275] The inverse quantization and inverse transform unit 105 is the same as the inverse quantization and inverse transform unit 311 in the video decoding device 31, and a description thereof will be omitted.

[0276] The parameter coding unit 111 includes a header coding unit 1110, a CT information coding unit 1111, and a CU coding unit 1112 (prediction mode coding unit). The CU coding unit 1112 further includes a TU coding unit 1114. The following describes an outline of the operation of each module.

[0277] The header encoding unit 1110 performs encoding processing of parameters such as header information, division information, prediction information, and quantized transform coefficients.

[0278] The CT information encoding unit 1111 encodes the QT, MT (BT, TT) division information and the like.

[0279] The CU encoding unit 1112 encodes the CU information, prediction information, division information, and so on.

[0280] When a prediction error is included in a TU, the TU encoding unit 1114 encodes the QP update information and the quantized prediction error.

[0281] The CT information encoding unit 1111 and the CU encoding unit 1112 supply syntax elements such as inter prediction parameters (predMode, merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX), intra prediction parameters, and quantized transform coefficients to the parameter encoding unit 111.

[0282] The entropy coding unit 104 receives the quantized transform coefficients and coding parameters (division information, prediction parameters) from the parameter coding unit 111. The entropy coding unit 104 entropy codes these to generate and output a coded stream Te.

[0283] The prediction parameter derivation unit 120 is a means including the inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit, and derives intra prediction parameters and intra prediction parameters from the parameters input from the encoding parameter determination unit 110. The derived intra prediction parameters and intra prediction parameters are output to the parameter encoding unit 111.

[0284] (Configuration of the inter-prediction parameter encoding unit) 23, the inter prediction parameter encoding unit 112 includes a parameter encoding control unit 1121 and an inter prediction parameter derivation unit 303. The inter prediction parameter derivation unit 303 has the same configuration as the video decoding device. The parameter encoding control unit 1121 includes a merge index derivation unit 11211 and a vector candidate index derivation unit 11212.

[0285] The merge index derivation unit 11211 derives merge candidates and the like, and outputs them to the inter prediction parameter derivation unit 303. The vector candidate index derivation unit 11212 derives predictive vector candidates and the like, and outputs them to the inter prediction parameter derivation unit 303 and the parameter coding unit 111.

[0286] (Configuration of intra-prediction parameter encoding unit) The intra-prediction parameter coding unit includes a parameter coding control unit and an intra-prediction parameter derivation unit. The intra-prediction parameter derivation unit has a common configuration with the video decoding device.

[0287] However, unlike the video decoding device, the inputs to the inter prediction parameter derivation unit 303 and the intra prediction parameter derivation unit are the coding parameter determination unit 110 and the prediction parameter memory 108 , and they are output to the parameter coding unit 111 .

[0288] The adder 106 generates a decoded image by adding, for each pixel, the pixel value of the predicted block input from the predicted image generation unit 101 and the prediction error input from the inverse quantization and inverse transform unit 105. The adder 106 stores the generated decoded image in a reference picture memory 109.

[0289] The loop filter 107 performs deblocking filtering, SAO, and ALF on the decoded image generated by the adder 106. Note that the loop filter 107 does not necessarily have to include the above three types of filters, and may be configured, for example, as only a deblocking filter.

[0290] The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 in a predetermined location for each current picture and CU.

[0291] The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each current picture and CU.

[0292] The coding parameter determination unit 110 selects one set from among a plurality of sets of coding parameters. The coding parameters are the above-mentioned QT, BT or TT division information, prediction parameters, or parameters to be coded that are generated in relation to these. The predicted image generation unit 101 generates a predicted image using these coding parameters.

[0293] The coding parameter determination unit 110 calculates an RD cost value indicating the magnitude of the information amount and the coding error for each of the multiple sets. The RD cost value is, for example, the sum of the code amount and the value obtained by multiplying the squared error by a coefficient λ. The code amount is the information amount of the coding stream Te obtained by entropy coding the quantization error and the coding parameters. The squared error is the sum of the squares of the prediction errors calculated in the subtraction unit 102. The coefficient λ is a real number greater than zero that is set in advance. The coding parameter determination unit 110 selects a set of coding parameters that minimizes the calculated cost value. The coding parameter determination unit 110 outputs the determined coding parameters to the parameter coding unit 111 and the prediction parameter derivation unit 120.

[0294] In addition, a part of the video encoding device 11 and the video decoding device 31 in the above-mentioned embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generating unit 308, the inverse quantization and inverse transform unit 311, the addition unit 312, the prediction parameter derivation unit 320, the predicted image generating unit 101, the subtraction unit 102, the transform and quantization unit 103, the entropy coding unit 104, the inverse quantization and inverse transform unit 105, the loop filter 107, the coding parameter determination unit 110, the parameter coding unit 111, and the prediction parameter derivation unit 120 may be realized by a computer. In this case, a program for realizing this control function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into and executed by a computer system. In addition, the "computer system" referred to here is a computer system built into either the video encoding device 11 or the video decoding device 31, and includes hardware such as an OS and peripheral devices. In addition, "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording medium" may also include devices that dynamically hold a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and devices that hold a program for a certain period of time, such as volatile memory inside a computer system that serves as a server or client in such cases. Furthermore, the above program may be one that realizes part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0295] In addition, a part or the whole of the video encoding device 11 and the video decoding device 31 in the above-mentioned embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the video encoding device 11 and the video decoding device 31 may be individually made into a processor, or a part or the whole may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Furthermore, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, an integrated circuit based on that technology may be used.

[0296] Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention.

[0297] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. In other words, the technical scope of the present invention also includes embodiments obtained by combining technical means that are appropriately modified within the scope of the claims. [Industrial Applicability]

[0298] The embodiments of the present invention can be suitably applied to a video decoding device that decodes coded data in which image data is coded, and a video coding device that generates coded data in which image data is coded, and can also be suitably applied to the data structure of coded data that is generated by a video coding device and referenced by the video decoding device. [Explanation of symbols]

[0299] 31 Image Decoding Device 301 Entropy Decoding Unit 302 Parameter Decoding Unit 303 Inter-prediction parameter derivation unit 30377 GPM Forecasting Division 305, 107 Loop Filter 306, 109 Reference Picture Memory 307, 108 Prediction parameter memory 308, 101 Prediction image generation unit 309 Inter-prediction image generation unit 30952 GPM synthesis section 311, 105 Inverse quantization and inverse transformation unit 312, 106 Addition section 320 Prediction Parameter Derivation Unit 11 Image encoding device 102 Subtraction section 103 Transformation and Quantization Section 104 Entropy coding unit 110 Encoding parameter determination unit 111 Parameter Encoding Unit 112 Inter-prediction parameter coding unit 120 Prediction parameter derivation part

Claims

1. a predicted image generating unit that generates a predicted image; A video decoding device including an inverse non-separable transform unit that performs an inverse non-separable transform, The video decoding device, wherein the non-separable transform unit does not change specific frequency components of the transform coefficients.

2. 2. The video decoding device according to claim 1, wherein the inverse non-separable transform unit does not change a DC component which is an upper left component.

3. 3. The video decoding device according to claim 2, wherein the inverse non-separable transform unit does not change a DC component that is an upper left component in the case of inter prediction.

4. 3. The video decoding device according to claim 2, wherein the inverse non-separable transform unit derives a one-dimensional array by excluding the DC component, and performs transformation into the one-dimensional array.

5. 2. The video decoding device according to claim 1, wherein the inverse non-separable transform section uses a transform matrix in which the leftmost component excluding the upper left component and the uppermost component are 0.

6. a predicted image generating unit that generates a predicted image; A video decoding device including an inverse non-separable transform unit that performs an inverse non-separable transform, The video coding device, wherein the non-separable transform section does not change a specific frequency component of the transform coefficient.

7. A video decoding device characterized in that, when a predicted image is derived by a weighted sum of an intra-predicted image and an inter-predicted image, a transform matrix for a non-separable transform is selected using an intra-prediction mode used to derive the intra-predicted image, and a non-separable transform is performed.

8. 8. The video decoding device according to claim 7, wherein the mode in which the predicted image is used is a CIIP mode in which the weights are derived depending on the states of adjacent blocks.

9. 8. The video decoding device according to claim 7, wherein the mode using the predicted image is a geometry mode in which weights are derived from parameters derived from encoded data.

10. A video coding device characterized in that, when deriving a predicted image by a weighted sum of an intra-predicted image and an inter-predicted image, a transform matrix for a non-separable transform is selected using an intra-prediction mode used to derive the intra-predicted image, and a non-separable transform is performed.

Images