Moving picture decoding device

Abstract

Provided is a moving image encoding / decoding device capable of improving encoding efficiency. A moving image decoding device is provided with: an MMVD prediction unit that obtains a motion vector by adding a predicted motion vector of a target block to a difference vector of a predetermined distance and direction; and a parameter decoding unit that decodes an index for specifying a difference vector from the MMVD candidate list, the moving image decoding device being characterized in that the MMVD prediction unit derives a difference vector in a specific distance direction from the prescribed distance and direction, derives a template matching cost from the difference vector, and performs a search; a differential vector candidate is inserted according to the cost, a list of MMVD candidates is derived, and in the search, a subset of the first direction is searched only among the distances included in a prescribed distance set, and a subset of the second direction is searched only among the distances included in a distance set other than the prescribed distance set.

Description

Technical Field

[0001] Embodiments of the present invention relate to a moving picture decoding device, etc. This application claims priority based on Japanese Patent Application No. 2022-101647 filed in Japan on June 24, 2022, and the contents are incorporated herein by reference. Background Art

[0002] In order to efficiently transmit or record moving images, a moving image encoding device that generates encoded data by encoding the moving image and a moving image decoding device that generates a decoded image by decoding the encoded data are used.

[0003] Specific moving picture coding methods include, for example, H.264 / AVC and HEVC (High-Efficiency Video Coding).

[0004] In such a motion image coding method, images (pictures) constituting the motion image are managed through a hierarchical structure and encoded / decoded for each CU. The hierarchical structure includes slices obtained by dividing the image, coding tree units (CTU: Coding Tree Unit) obtained by dividing the slices, coding units (sometimes also called coding units (Coding Unit: CU)) obtained by dividing the coding tree units, and transform units (TU: Transform Unit) obtained by dividing the coding units.

[0005] In such a moving picture coding method, a prediction image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and a prediction error (sometimes also referred to as a "difference image" or "residual image") obtained by subtracting the prediction image from the input image (original image) is encoded. Examples of methods for generating a prediction image include inter-picture prediction (inter-frame prediction) and intra-picture prediction (intra-frame prediction).

[0006] Furthermore, VVC / H.266 discloses an MMVD prediction in which a motion vector is obtained by adding a differential vector having a predetermined distance and a predetermined direction to a motion vector.

[0007] Furthermore, Non-Patent Document 1 discloses a technique in which a template matching cost is calculated for all MMVD difference vector candidates, and they are sorted according to the cost, thereby deriving a difference vector with a small amount of code. Prior art literature Non-patent literature

[0008] Non-patent document 1: "Non-EE2: Template Matching-based Reordering for ExtendedMMVD Design", JVET-X0085, Joint Video Experts Team (JVET) of ITU-T SG 16WP 3andISO / IEC JTC 1 / SC29 24th Meeting, by teleconference Summary of the invention Problem that the invention aims to solve

[0009] In the method described in Non-Patent Document 1, the template matching cost is derived for all MMVD difference vector candidates, so there is a problem of large amount of calculation. Technical Solution

[0010] In order to solve the above-mentioned problem, a motion image decoding device of one scheme of the present invention comprises: an MMVD prediction unit, which obtains a motion vector by adding a differential vector of a specified distance and a specified direction to a predicted motion vector of an object block; and a parameter decoding unit, which decodes, from the encoded data, an index for specifying a differential vector from an MMVD candidate list. The motion image decoding device is characterized in that the MMVD prediction unit is characterized in that a differential vector of a specific distance and direction is derived from the above-mentioned specified distance and specified direction, a template matching cost in the above-mentioned differential vector is derived, a search is performed, differential vector candidates are inserted according to the above-mentioned cost, and the MMVD candidate list is derived, and in the search, only a subset of the first direction (DIR1) is searched within the specified distance, and only a subset of the second direction (DIR2) is searched within the distance other than the specified distance. Beneficial Effects

[0011] According to the aspect of the present invention, it is possible to reduce the amount of calculation related to the prediction of the MMVD in the moving picture encoding / decoding process. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic diagram showing the configuration of the image transmission system according to the present embodiment. Figure 2 This is a diagram showing the hierarchical structure of the data of the coded stream. Figure 3 This is a schematic diagram showing the structure of a moving picture decoding device. Figure 4 This is a schematic diagram showing the structure of an inter-frame prediction image generation unit. Figure 5 This is a schematic diagram showing the structure of an inter-frame prediction parameter derivation unit. Figure 6 This is a flowchart for explaining the schematic operation of the moving picture decoding device. Figure 7 is a diagram showing an example of an index used in the MMVD mode. Figure 7 (a) is a diagram showing an example of the index mmvd_cand_idx of the MMVD candidate of the MMVD candidate list. Figure 7 (b) is a diagram showing an example of the position of a block adjacent to the target block. Figure 7 (c) is a diagram showing an example of mmvd_distance_idx. Figure 7 (d) is a diagram showing an example of mmvd_direction_idx. Figure 8 This is a diagram showing an example of the number of candidates for the search distance and the number of candidates for the derivation direction in the MMVD mode. Fig. 9 It is the syntax diagram in merged prediction and MMVD prediction. Fig.10 is a syntax diagram related to another component of MMVD prediction. Fig.11 This is a flowchart showing the flow of processing in another configuration of MMVD prediction. Fig.12 This is a flowchart showing the flow of processing in another configuration of the MMVD candidate list derivation processing. Fig.13 This is another flowchart showing the flow of processing in another configuration of the MMVD candidate list derivation processing. Fig.14 This is a diagram showing an example of candidates in 16 directions among MMVD candidates. Fig.15 This is a diagram showing four types of distances using candidates in 16 directions in the MMVD candidate. Fig.16 This is a block diagram showing the structure of a moving picture encoding device. Fig.17 This is a schematic diagram showing the structure of the inter-frame prediction parameter encoding unit. DETAILED DESCRIPTION (First Embodiment)

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0014] Figure 1 It is a schematic diagram showing the configuration of the image transmission system 1 according to the present embodiment.

[0015] The image transmission system 1 is a system for transmitting a coded stream obtained by encoding a target image, decoding the transmitted coded stream and displaying the image. The image transmission system 1 is configured to include: a moving image encoding device (image encoding device) 11, a network 21, a moving image decoding device (image decoding device) 31, and a moving image display device (image display device) 41.

[0016] The video image encoding device 11 receives the image T as input.

[0017] The network 21 transmits the coded stream Te generated by the motion image coding device 11 to the motion image decoding device 31. The network 21 is the Internet, a wide area network (WAN), a small network (LAN) or a combination thereof. The network 21 is not necessarily limited to a two-way communication network, but may also be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. In addition, the network 21 may also be replaced by a storage medium such as a DVD (Digital Versatile Disc: Digital Versatile Disc, registered trademark) or a BD (Blue-ray Disc: Blue-ray Disc, registered trademark) that records the coded stream Te.

[0018] The moving picture decoding device 31 decodes each of the coded streams Te transmitted through the network 21 , and generates one or more decoded pictures Td after decoding.

[0019] The moving image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31. The moving image display device 41 includes, for example, a liquid crystal display, an organic EL (Electro-luminescence) display, and other display devices. Examples of the display type include a fixed type, a mobile type, and an HMD (Head Mounted Display). In addition, when the moving image decoding device 31 has high processing power, it displays images with high image quality, and when it has only low processing power, it displays images that do not require high processing power or high display power. <operator>

[0020] The operators used in this specification are described below.

[0021] >> is a right shift, << is a left shift, & is a bitwise AND, | is a bitwise OR, |= is an OR substitution operator, and || represents a logical AND.

[0022] x?y:z is a three-term operator that takes y when x is true (other than 0) and takes z when x is false (0).

[0023] Clip3(a, b, c) is a function that clips c to a value greater than or equal to a and less than or equal to b, and returns a when c < a, returns b when c > b, and returns c in other cases (where a <= b).

[0024] abs(a) is a function that returns the absolute value of a.

[0025] Int(a) is a function that returns the integer value of a.

[0026] floor(a) is a function that returns the largest integer less than or equal to a.

[0027] ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0028] a / d means a divided by d (discarding the decimal part). <Structure of the coded stream Te>

[0029] Before explaining the moving image encoding device 11 and the moving image decoding device 31 of the present embodiment in detail, the data structure of the coded stream Te generated by the moving image encoding device 11 and decoded by the moving image decoding device 31 will be described.

[0030] Figure 2 It is a diagram showing the hierarchical structure of the data in the coded stream Te. The coded stream Te exemplarily includes a sequence and multiple pictures constituting the sequence. Figure 2 The (a) - (f) of it are respectively diagrams showing the coded video sequence of a given sequence SEQ, the coded picture of a specified picture PICT, the coded slice of a specified slice S, the coded slice data of the specified slice data, the coded tree unit included in the coded slice data, and the coded unit included in the coded tree unit. (Coded video sequence)

[0031] In the coded video sequence, a set of data for the moving image decoding device 31 to refer to for decoding the sequence SEQ to be processed is specified. As Figure 2 shown, the sequence SEQ includes a Video Parameter Set, a Sequence Parameter Set SPS, a Picture Parameter Set PPS, an Adaptation Parameter Set (APS), a picture PICT, and Supplemental Enhancement Information SEI.

[0032] The video parameter set VPS defines a set of coding parameters common to a plurality of moving images, and a set of coding parameters associated with a plurality of layers included in the moving image and each layer, in the moving image.

[0033] In the sequence parameter set SPS, a set of coding parameters is specified for reference by the motion image decoding device 31 to decode the object sequence. For example, the width and height of the picture are specified. It should be noted that there can be multiple SPSs. In this case, any one of the multiple SPSs is selected from the PPS.

[0034] In the picture parameter set PPS, a set of coding parameters is defined for reference by the video decoding device 31 to decode each picture in the target sequence. For example, it includes a reference value (pic_init_qp_minus26) of the quantization width for decoding of the picture and a flag (weighted_pred_flag) indicating the application of weighted prediction. It should be noted that there can be multiple PPSs. In this case, any one of the multiple PPSs is selected from each picture in the target sequence. (Encoded image)

[0035] In the coded picture, a set of data is defined for reference by the video decoding device 31 in order to decode the picture PICT to be processed. Figure 2 As shown, it includes slice 0 to slice NS-1 (NS is the total number of slices included in the picture PICT).

[0036] It should be noted that, in the following, when there is no need to distinguish between slices 0 to NS-1, the code subscript may be omitted for description. The same is true for data included in the coded stream Te described below, that is, other data with a suffix. (Encoding slice)

[0037] In the coded slice, a set of data is defined for reference by the video decoding device 31 to decode the slice S to be processed. Figure 2 The shown includes a slice header and slice data.

[0038] The slice header includes a coding parameter group that the moving picture decoding device 31 refers to in order to determine a decoding method of a target slice. Slice type designation information (slice_type) that designates a slice type is one example of a coding parameter included in the slice header.

[0039] Examples of slice types that can be specified by the slice type specification information include: (1) I slices that use only intra-frame prediction when encoding, (2) P slices that use unidirectional prediction or intra-frame prediction when encoding, and (3) B slices that use unidirectional prediction, bidirectional prediction, or intra-frame prediction when encoding. It should be noted that inter-frame prediction is not limited to unidirectional prediction and bidirectional prediction, and more reference pictures can be used to generate predicted images. Hereinafter, the cases referred to as P and B slices refer to slices that include blocks that can use inter-frame prediction.

[0040] It should be noted that the slice header may also include a reference to the picture parameter set PPS (pic_parameter_set_id). (Encoded slice data)

[0041] The coded slice data specifies a set of data that the video decoding device 31 refers to in order to decode the slice data to be processed. Figure 2 (d) includes CTU. CTU is a block of a fixed size (for example, 64×64) constituting a slice, and is also called the largest coding unit (LCU). (Coding Tree Unit)

[0042] exist Figure 2 In the specification, there is a set of data for the motion picture decoding device 31 to refer to in order to decode the CTU of the processing object. The CTU is divided into coding units CU which are the basic units of the encoding process by recursive quadtree partitioning (QT (Quad Tree) partitioning), binary tree partitioning (BT (Binary Tree) partitioning) or ternary tree partitioning (TT (Ternary Tree) partitioning). BT partitioning and TT partitioning are collectively referred to as multi-tree partitioning (MT (Multi Tree) partitioning). The nodes of the tree structure obtained by recursive quadtree partitioning are called coding nodes (Coding Node). The middle nodes of the quadtree, binary tree and ternary tree are coding nodes, and the CTU itself is also specified as the top-level coding node. In addition, the bottom-level coding node is specified as a coding unit. (Coding unit)

[0043] exist Figure 2 In the CU header, a set of data is specified for reference by the video decoding device 31 to decode the coding unit to be processed. Specifically, the CU is composed of a CU header CUH, prediction parameters, transformation parameters, quantized transformation coefficients, etc. The prediction mode, etc. are specified in the CU header.

[0044] The prediction process may be performed in units of CU or in units of sub-CUs obtained by further dividing the CU. When the size of the CU and the sub-CU are equal, there is one sub-CU in the CU. When the size of the CU is larger than the size of the sub-CU, the CU is divided into sub-CUs. For example, when the CU is 8×8 and the sub-CU is 4×4, the CU is divided into 4 sub-CUs, including two parts divided horizontally and two parts divided vertically.

[0045] The prediction types (prediction modes) include intra prediction (MODE_INTRA), inter prediction (MODE_INTER), and intra block copy (MODE_IBC). Intra prediction is prediction within the same picture, while inter prediction is prediction between different pictures (e.g., between display times, between layer images).

[0046] The transform / quantization process is performed in units of CUs, but the quantized transform coefficients may be entropy-coded in units of 4×4 or other sub-blocks. (Prediction parameters)

[0047] The predicted image is derived from the prediction parameters attached to the block. The prediction parameters include prediction parameters for intra-frame prediction and inter-frame prediction. (Prediction parameters for inter-frame prediction)

[0048] The prediction parameters of inter-frame prediction are explained. The inter-frame prediction parameters are composed of the prediction list utilization flags predFlagL0 and predFlagL1, the reference picture indexes refIdxL0 and refIdxL1, and the motion vectors mvL0 and mvL1. predFlagL0 and predFlagL1 are flags indicating whether to use the reference picture list (L0 list, L1 list). When the value is 1, the corresponding reference picture list is used. It should be noted that in the case of "flag indicating whether it is ××" in this specification, the flag is set to be ×× when it is other than 0 (for example, 1), and the flag is set to be not ×× when it is 0. In logical NOT, logical product, etc., 1 is regarded as true and 0 is regarded as false (the same applies below). However, other values ​​can also be used as true values ​​and false values ​​in actual devices and methods.

[0049] The syntax elements used to derive inter-frame prediction parameters include, for example, merge flag merge_flag (general_merge_flag), merge index merge_idx, merge_subblock_flag, regulare_merge_flag, ciip_flag, merge_gpm_partition_idx, merge_gpm_idx0, merge_gpm_idx1, inter_pred_idc, reference picture index refIdxLX, mvp_LX_idx, differential vector mvdLX, and motion vector accuracy mode amvr_mode. merge_subblock_flag is a flag indicating whether to use inter-frame prediction in sub-block units. regulare_merge_flag is a flag indicating whether to use the normal merge mode or MMVD. ciip_flag is a flag indicating whether to use the IIP (combined inter-picture merge and intra-picture prediction) mode or the GPM mode (Geometric partitioning merge mode). merge_gpm_partition_idx is an index indicating the partition shape of the GPM mode. merge_gpm_idx0 and merge_gpm_idx1 are indexes indicating the merge index of the GPM mode. inter_pred_idc is an inter-frame prediction identifier for selecting a reference picture used in the AMVP mode. mvp_LX_idx is a prediction vector index for deriving a motion vector. (See picture list)

[0050] The reference picture list is a list of reference pictures stored in the reference picture memory 306. In each CU, refIdxLX specifies which picture in the reference picture list RefPicListX (X = 0 or 1) is actually referenced. It should be noted that LX is a description method used when L0 prediction and L1 prediction are not distinguished. In the following, LX is replaced by L0 and L1 to distinguish the parameters for the L0 list and the parameters for the L1 list. (Combined forecast and AMVP forecast)

[0051] The decoding (encoding) method of the prediction parameters includes the merge prediction (merge) mode (merge mode) and the AMVP (Advanced Motion Vector Prediction: Advanced Motion Vector Prediction, Adaptive Motion Vector Prediction) mode, and general_merge_flag is a flag used to identify them. The merge mode is a prediction mode that omits part or all of the motion vector difference. It is a mode that does not include the prediction list using the flag predFlagLX, the reference picture index refIdxLX, and the motion vector mvLX in the encoded data, but derives it from the prediction parameters of the processed nearby blocks. The AMVP mode is a mode that includes inter_pred_idc, refIdxLX, and mvLX in the encoded data. It should be noted that mvLX is encoded as mvp_LX_idx that identifies the prediction vector mvpLX and the differential vector mvdLX. In addition, the general term for the prediction mode that omits / simplifies the motion vector difference can also be called the general merge mode (general merge mode), and the general merge mode and AMVP prediction are selected according to the general_merge_flag.

[0052] When general_merge_flag is 1, the transmission Fig.10 merge_data() shown in FIG. 14 , therefore, regular_merge_flag may also be transmitted. Alternatively, when regular_merge_flag is 1, the normal merge mode or MMVD is selected, and in other cases, the CIIP mode or the GPM mode is selected. The CIIP mode generates a predicted image by a weighted sum of an inter-frame predicted image and an intra-frame predicted image. The GPM mode generates a predicted image by treating the target CU as two non-rectangular prediction units divided by a line segment.

[0053] inter_pred_idc is a value indicating the type and number of reference pictures, and takes any value of PRED_L0, PRED_L1, or PRED_BI. PRED_L0 and PRED_L1 respectively indicate unidirectional prediction using a reference picture managed in the L0 list and the L1 list. PRED_BI indicates bidirectional prediction using two reference pictures managed in the L0 list and the L1 list.

[0054] merge_idx is an index indicating whether any prediction parameter among prediction parameter candidates (merge candidates) derived from a block for which processing has been completed is to be used as a prediction parameter of the target block. (Motion Vector)

[0055] mvLX represents the amount of shift between blocks in two different pictures. The prediction vector and difference vector related to mvLX are referred to as mvpLX and mvdLX, respectively. (Inter prediction identifier inter_pred_idc and prediction list use flag predFlagLX)

[0056] The relationship between inter_pred_idc and predFlagL0 and predFlagL1 can be converted to each other as follows.

[0057] inter_pred_idc=(predFlagL1<<1)+predFlagL0 predFlagL0=inter_pred_idc_&1 predFlagL1=inter_pred_idc>>1 It should be noted that the inter-frame prediction parameter may use a prediction list utilization flag or an inter-frame prediction identifier. In addition, the determination using the prediction list utilization flag may be replaced by the determination using the inter-frame prediction identifier. Conversely, the determination using the inter-frame prediction identifier may also be replaced by the determination using the prediction list utilization flag. (Bidirectional prediction biPred determination)

[0058] The flag biPred indicating whether bidirectional prediction is used can be derived based on whether the flags of the two prediction lists are both 1.

[0059] Alternatively, biPred can also be derived based on whether the inter-frame prediction identifier is a value indicating the use of two prediction lists (reference pictures). (Configuration of Moving Image Decoding Device)

[0060] The video decoding device 31 ( Figure 3 ) is described below.

[0061] The moving picture decoding device 31 is configured to include an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, an inverse quantization / inverse transformation unit 311 and an addition unit 312, and a prediction parameter derivation unit 320. It should be noted that, according to the moving picture encoding device 11 described later, there is also a configuration in which the loop filter 305 is not included in the moving picture decoding device 31.

[0062] The parameter decoding unit 302 also includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit). The CU decoding unit 3022 also includes a TU decoding unit 3024. They can also be collectively referred to as decoding modules. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, PPS, APS, and slice header (slice information) from the encoded data. The CT information decoding unit 3021 decodes CT from the encoded data. The CU decoding unit 3022 decodes CU from the encoded data. The TU decoding unit 3024 decodes CU from the encoded data.

[0063] The TU decoding unit 3024 decodes the QP update information and the quantized transform coefficient from the encoded data when the prediction error is included in the TU. Multiple modes (such as the RRC mode and the TSRC mode) may also be provided in the derivation of the quantized transform coefficient. RRC (Regular Residual Coding) is a decoding mode of prediction error using a transform, and TSRC (Transform Skip Residual Coding) is a decoding mode of prediction error in a transform skip mode in which a transform is not implemented. The TU decoding unit 3024 may also decode the LAST position of the transform coefficient in the RRC mode, but not decode the LAST position in the TSRC mode. The QP update information is a differential value from the quantization parameter prediction value qPpred which is a prediction value of the quantization parameter QP.

[0064] The prediction image generation unit 308 is configured to include an inter-frame prediction image generation unit 309 ( Figure 4 ) and the intra-frame prediction image generation unit 310.

[0065] The prediction parameter derivation unit 320 is configured to include an inter-frame prediction parameter derivation unit 303 ( Figure 5 ) and intra-frame prediction parameter derivation unit.

[0066] In addition, the following describes an example of using CTU and CU as processing units, but the example is not limited thereto, and processing may be performed in sub-CU units. Alternatively, CTU and CU may be replaced with blocks, and sub-CU may be replaced with sub-blocks, and processing may be performed in blocks or sub-blocks.

[0067] The entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside, and decodes each code (syntax element). There are the following methods in entropy coding: a method of variable-length coding the syntax element using a context (probability model) appropriately selected according to the type of syntax element and the surrounding conditions; and a method of variable-length coding the syntax element using a predetermined table or calculation formula.

[0068] The entropy decoding unit 301 outputs the decoded code to the parameter decoding unit 302. The decoded code is, for example, prediction mode predMode, general_merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX, amvr_mode, etc. Based on the instruction of the parameter decoding unit 302, control is performed on which code is decoded. (Basic process)

[0069] Figure 6 This is a flowchart for explaining the schematic operation of the moving picture decoding device 31 .

[0070] (S1100: Parameter set information decoding) The header decoding unit 3020 decodes parameter set information such as VPS, SPS, and PPS from the encoded data.

[0071] (S1200: Slice Information Decoding) The header decoding unit 3020 decodes a slice header (slice information) from the encoded data.

[0072] Thereafter, the video decoding device 31 repeatedly performs the processes from S1300 to S5000 for each CTU included in the target picture, thereby deriving a decoded image of each CTU.

[0073] (S1300: CTU Information Decoding) The CT information decoding unit 3021 decodes CTU from the encoded data.

[0074] (S1400: CT Information Decoding) The CT information decoding unit 3021 decodes CT from the encoded data.

[0075] (S1500: CU Decoding) The CU decoding unit 3022 implements S1510 and S1520 to decode the CU from the encoded data.

[0076] (S1510: CU Information Decoding) The CU decoding unit 3022 decodes CU information, prediction information, TU partition flag, CU residual flag, etc. from the encoded data.

[0077] (S1520: TU information decoding) When a prediction error is included in a TU, the TU decoding unit 3024 decodes the quantized prediction error and the like from the encoded data.

[0078] (S2000: Prediction Image Generation) The prediction image generation unit 308 generates a prediction image for each block included in the target CU based on the prediction information.

[0079] (S3000: Inverse Quantization / Inverse Transformation) The inverse quantization / inverse transformation unit 311 performs an inverse quantization / inverse transformation process on each TU included in the target CU.

[0080] (S4000: Decoded Image Generation) The addition unit 312 generates a decoded image of the target CU by adding the predicted image supplied from the predicted image generation unit 308 and the prediction error supplied from the inverse quantization / inverse transformation unit 311 .

[0081] (S5000: Loop Filtering) The loop filter 305 applies loop filtering such as deblocking filtering, SAO (Sample Adaptive Offset), and ALF (Adaptive Loop Filter) to the decoded image to generate a decoded image.

[0082] The loop filter 305 is a filter provided in the encoding loop, and is a filter for improving image quality by removing block distortion and ringing distortion. The loop filter 305 performs filtering such as deblocking filtering, sample adaptive offset (SAO), and adaptive loop filtering (ALF) on the decoded image of the CU generated by the adding unit 312.

[0083] The reference picture memory 306 stores the decoded image of the CU generated by the adding unit 312 at a predetermined position for each target picture and target CU.

[0084] The prediction parameter memory 307 stores the prediction parameter at a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301 .

[0085] The prediction image generation unit 308 receives the prediction mode predMode, prediction parameters, and the like as input. In addition, the prediction image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of a block or a sub-block using the prediction parameters and the read reference picture (reference picture block) in the prediction mode indicated by the prediction mode predMode. Here, the reference picture block refers to a set of pixels on the reference picture (usually rectangular, so it is called a block), which is a region referenced to generate a prediction image. (Configuration of Inter-frame Prediction Parameter Derivation Unit)

[0086] like Figure 5As shown, the inter-frame prediction parameter derivation unit 303 derivates the inter-frame prediction parameters based on the syntax elements input from the parameter decoding unit 302, referring to the prediction parameters stored in the prediction parameter memory 307. In addition, the inter-frame prediction parameters are output to the inter-frame prediction image generation unit 309 and the prediction parameter memory 307. The inter-frame prediction parameter derivation unit 303 and its internal elements, such as the AMVP prediction parameter derivation unit 3032, the merged prediction parameter derivation unit 3036, the GPM prediction unit 30377, and the MV addition unit 3038, are common units in the motion picture encoding device and the motion picture decoding device, and therefore, they may also be collectively referred to as a motion vector derivation unit (motion vector derivation device).

[0087] When general_merge_flag is 1, indicating the merge prediction mode, merge_idx is derived and output to the merge prediction parameter derivation unit 3036 .

[0088] When general_merge_flag is 0, indicating the AMVP prediction mode, the AMVP prediction parameter derivation unit 3032 derives mvpLX from inter_pred_idc, refIdxLX or mvp_LX_idx. (MV addition)

[0089] The derived mvpLX and mvdLX are added in the MV adding unit 3038 to derive mvLX. (Combined forecast)

[0090] The merge prediction parameter derivation unit 3036 includes a merge candidate derivation unit 30361 and a merge candidate selection unit 30362. It should be noted that the merge candidate is composed of prediction parameters (predFlagLX, mvLX, refIdxLX) and stored in the merge candidate list. Indexes are assigned to the merge candidates stored in the merge candidate list according to a prescribed rule.

[0091] The merge candidate derivation unit 30361 directly uses the decoded motion vector of the adjacent block and refIdxLX to derive the merge candidate. In addition, the merge candidate derivation unit 30361 can apply the spatial merge candidate derivation process, the temporal merge candidate derivation process, etc. described later.

[0092] As a spatial merge candidate derivation process, the merge candidate derivation unit 30361 reads the prediction parameters stored in the prediction parameter memory 307 according to a prescribed rule and sets them as merge candidates. Figure 7 The prediction parameters of the following positions A1, B1, B0, A0, B2 are shown in (b).

[0093] A1:(xCb-1,yCb+cbHeight-1) B1:(xCb+cbWidth-1,yCb-1) B0:(xCb+cbWidth,yCb-1) A0:(xCb-1,yCb+cbHeight) B2:(xCb-1,yCb-1) The upper left coordinate of the target block is set to (xCb, yCb), the width is cbWidth, and the height is cbHeight.

[0094] As a temporal merge derivation process, the merge candidate derivation unit 30361 may read the prediction parameter of the lower right CBR of the object block or the block C in the reference image including the central coordinates from the prediction parameter memory 307 and set it as the merge candidate Col, and store it in the merge candidate list mergeCandList[].

[0095] The order stored in mergeCandList[] is, for example, spatial merge candidates (B1, A1, B0, A0, B2) and temporal merge candidates Col. It should be noted that unusable reference blocks (blocks that are intra-predicted, etc.) are not stored in the merge candidate list. Furthermore, the historical merge candidate HmvpCand, the pairwise average candidate avgCand, and the zero merge candidate zeroCandm may be added to mergeCandList[] for use.

[0096] The merge candidate selection unit 30362 selects a merge candidate N indicated by merge_idx from among the merge candidates included in the merge candidate list using the following formula.

[0097] N = mergeCandList[merge_idx] Here, N is a label indicating a merge candidate, and may be A1, B1, B0, A0, B2, Col, etc. The motion information of the merge candidate indicated by label N is represented by (mvLXN[0], mvLXN[1]), predFlagLXN, and refIdxLXN.

[0098] The merge candidate selection unit 30362 selects (mvLXN[0], mvLXN[0]), predFlagLXN, and refIdxLXN as the inter prediction parameters of the target block according to merge_idx. The merge candidate selection unit 30362 stores the inter prediction parameters of the selected merge candidate in the prediction parameter memory 307 and outputs them to the inter prediction image generation unit 309. (MMVD Prediction Department 30376)

[0099] The MMVD prediction unit 30376 performs processing in the MMVD (Merge with Motion Vector Difference) mode. In merge prediction, the motion vector obtained from the adjacent block is used as the motion vector of the merge candidate. The MMVD mode is a mode for obtaining a fine motion vector by adding a differential vector of a specified distance and a specified direction to the motion vector of the merge candidate. In the MMVD mode, the MMVD prediction unit 30376 uses the merge candidate and limits the value range of the differential vector to a specified distance (for example, 6 types, 8 types, etc.) and a specified direction (for example, 4 directions, 8 directions, 16 directions, etc.), thereby efficiently deriving the motion vector. Figure 8 Examples of 4 directions are shown in FIG.

[0100] The MMVD prediction unit 30376 uses the merge candidate mergeCandList[] and the syntax mmvd_cand_flag, mmvd_direction_idx, mmvd_distance_idx to derive the motion vector mvLX[]. These are syntaxes decoded from or encoded into the coded data. In addition, the MMVD prediction unit 30376 can encode or decode the syntax distance_list_idx for selecting the distance table for use.

[0101] When regular_merge_flag is 1 (generally indicating the application of merge mode or MMVD mode), the MMVD prediction unit 30376 decodes the MMVD flag (mmvd_merge_flag) for the target CU. Furthermore, when the MMVD flag indicates the application of MMVD mode (mmvd_merge_flag = 1), the MMVD prediction unit 30376 applies the MMVD mode and encodes or decodes mmvd_cand_flag, mmvd_distance_idx, and mmvd_direction_idx.

[0102] The MMVD prediction unit 30376 generates an MMVD candidate list using any one of the two prediction vectors from the starting point of the merge candidate list mergeCandList[] and the difference vector (MVD: motion vector difference) with the prediction vector to derive the motion vector. The difference vector is divided into direction and distance for encoding or decoding. Moreover, the MMVD prediction unit 30376 derives the motion vector from the prediction vector and the difference vector.

[0103] Figure 8Indicates a candidate for the difference vector refineMvLX derived in the MMVD prediction unit 30376. Figure 8 In the example shown, the central black dot is the position pointed to by the prediction vector mvLXN (center vector).

[0104] Figure 7 (a) shows the relationship between the index mmvd_cand_flag of mergeCandList[] and mvLXN. The motion vector of mergeCandList[mmvd_cand_flag] is set for mvLXN. The position pointed to by the center vector ( Figure 8 The difference between the black dot (shown in the figure) and the actual motion vector is the differential vector refineMvLX.

[0105] Figure 7 (b) is a diagram showing an example of a block adjacent to the target block. For example, in the case of mergeCandList[] = {A1, B1, B0, A0, B2}, when the decoded mmvd_cand_flag indicates 0, the MMVD prediction unit 30376 selects Figure 7 The motion vector of block A1 shown in (b) is used as the center vector mvLXN. In addition, when the decoded mmvd_cand_flag indicates 1, the MMVD prediction unit 30376 selects Figure 7 The motion vector of the block B1 shown in (b) is taken as the center vector mvLXN. It should be noted that when mmvd_cand_flag is not notified in the coded data, it can also be estimated that mmvd_cand_flag=0.

[0106] Furthermore, the MMVD prediction unit 30376 derives refineMvLX using the index mmvd_distance_idx indicating the length of the difference vector refineMvLX and the index mmvd_direction_idx indicating the direction of refineMvLX.

[0107] Figure 7 (c) is a diagram showing an example of mmvd_distance_idx. Figure 7 As shown in (c) of FIG. 8 , in mmvd_distance_idx, the values ​​of 0, 1, 2, 3, 4, 5, 6, and 7 correspond to eight distances (lengths) of 1 / 4pel, 1 / 2pel, 1pel, 2pel, 4pel, 8pel, 16pel, and 32pel, respectively.

[0108] Figure 7 (d) is a diagram showing an example of mmvd_direction_idx. Figure 7 As shown in (d), in mmvd_direction_idx, the values ​​of 0, 1, 2, and 3 correspond to the positive direction of the x-axis, the negative direction of the x-axis, the positive direction of the y-axis, and the negative direction of the y-axis, respectively. The MMVD prediction unit 30376 refers to the direction table DirectionTable to derive the basic motion vector (mvdUnit[0], mvdUnit[1]) from mmvd_direction_idx. (mvdUnit[0], mvdUnit[1]) can also be expressed as (sign[0], sign[1]).

[0109] In addition, the MMVD prediction unit 30376 derives the size of the difference vector DistFromBaseMV (=MmvdDistance) from the distance DistanceTable[mmvd_distance_idx] indicated by mmvd_distance_idx in the distance table DistanceTable through the following formula.

[0110] DistFromBaseMV=DistanceTable[mmvd_distance_idx] In addition, you can also select DistanceTable from the following two according to the flag.

[0111] DistanceTable[]={1,2,4,8,16,32,64,128} DistanceTable[]={4,8,16,32,64,128,256,512} Alternatively, the accuracy of the motion vector (eg, 1 / 16) may be taken into consideration, and the center vector and the differential vector may be shifted to the left to make them consistent in size.

[0112] DistFromBaseMV=DistFromBaseMV<<2 (Other than 4 directions) The above description is for the case where the basic motion vector (mvdUnit[0], mvdUnit[1]) is in four directions, namely, up, down, left, and right. However, the direction is not limited to four directions and can be eight directions. The following is an example of the x component dir_table_x[] and y component dir_table_y[] of the direction table DirectionTable when the basic motion vector is set to eight directions.

[0113] dir_table_x[]={8,-8,0,0,6,-6,-6,6} dir_table_y[]={0,0,8,-8,6,-6,6,-6} It should be noted that the size and order of the direction table may be other than those described above.

[0114] The MMVD prediction unit 30376 refers to the DirectionTable and derives the basic motion vector (mvdUnit[0], mvdUnit[1]) from mmvd_direction_idx.

[0115] mvdUnit[0]=dir_table_x[mmvd_direction__idx] mvdUnit[1]=dir_table_y[mmvd_direction_idx] In addition, for example, the direction table shown below can be used to set the direction to 4, 6, 12, or 16 directions. · Case of 6 directions (numDir=6) dir_table_x[]={8,-8,2,-2,-2,2} dir_table_y[]={0, 0, 4, -4, 4, -4} or dir_table_x[]={8,-8,3,-3-3,3} dir_table_y[]={0, 0, 6, -6, 6, -6} · Case of 12 directions (numDir=12) dir_table_x[]={8,-8,0,0,4,2,-4,-2,-2,-4,2,4} dir_table_y[]={0, 0, 8, -8, 2, 4, -2, -4, 4, 2, -4, -2} or dir_table_x[]={8,-8,0,0,6,3,-6,-3,-3,-6,3,6} dir_table_y[]={0, 0, 8, -8, 3, 6, -3, -6, 6, 3, -6, -3} · Case of 16 directions (numDir=16) dir_table_x[]={8,-8,0,0,4,-4,-4,4,6,2,-6,-2,-2,-6,2,6} dir_table_y[]={0, 0, 8, -8, 4, -4, 4, -4, 2, 6, -2, -6, 6, 2, -6, -2} or dir_table_x[]={1,-1,0,0,1,-1,1,-1,2,-2,2,-2,1,1,-1,-1} dir_table_y[]={0, 0, 1, -1, 1, -1, -1, 1, 1, 1, -2, -1, 2, -2, 2, -2} Case of 4 directions (numDir=4) dir_table_x[]={1,-1,0,0} dir_table_y[]={0, 0, 1, -1} It should be noted that the size and order of the direction table may be other than those described above. (Multiple distance tables)

[0116] In addition, the distance table is not limited to one, and may be multiple. For example, the MMVD prediction unit 30376 uses the DistanceTable[] indicated by the distance_list_idx decoded or derived from the encoded data to derive the length of the differential vector DistFromBaseMV. With reference to these, DistanceFromBaseMV can be derived from the first distance table DistanceTable1[] and the second distance table DistanceTable2[] as described below.

[0117] DistanceTable1[]={1,2,3,5} DistanceTable2[]={4,8,16,32} DistanceTable=DistanceTable1(distance_list_idx==0) DistanceTable=DistanceTable2(distance_list_idx==1) DistFromBaseMV=DistanceTable[mmvd_distance_idx] In addition, the MMVD prediction unit 30376 can use the two-dimensional table DistanceTable2d to switch between the two distance tables.

[0118] DistanceTable2d[]={{1,2,3,5},{4,8,16,32}} DistFromBaseMV=DistanceTable2d[distance_list_idx][mmvd_distance_idx] (Derivation of differential vector) The MMVD prediction unit 30376 derives the difference vector refineMvLX from the size of the basic motion vector and the difference vector DistFromBaseMV. When the merge candidate N related to the center vector is a unidirectional prediction based on the L0 reference picture (predFlagL0N=1, predFlagL1N=0), the MMVD prediction unit 30376 derives the difference vector refineMvL0 of L0 from the size of the basic motion vector and the difference vector DistFromBaseMV.

[0119] refineMvL0[0]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[0] refineMvL0[1]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[1] refineMvL1[0]=0 refineMvL1[1]=0 Here, shiftMMVD is a value that adjusts the size of the differential vector to match the accuracy MVPREC of the motion vector in the motion compensation unit 3091 (interpolation unit). For example, when MVPREC is 16, that is, when the motion vector accuracy is 1 / 16 pixels and there are 4 directions, that is, when mvdUnit[0] and mvdUnit[1] are 0 or 1, it is appropriate to use 2. In addition, the shift direction of shiftMMVD is not limited to left shifting. For example, in 6, 8, 12, 16 directions, etc., when mvdUnit[0] and mvdUnit[1] use values ​​other than 0 or 1 (for example, 8), the MMVD prediction unit 30376 can perform a right shift to normalize. For example, the MMVD prediction unit 30376 can right shift after multiplying the basic motion vector (mvdUnit[0], mvdUnit[1]) as shown below.

[0120] refineMvL0[0]=(DistFromBaseMV*mvdUnit[0])>>shiftMMVD refineMvL0[1]=(DistFromBaseMV*mvdUnit[1])>>shiftMMVD In addition, the MMVD prediction unit 30376 can be divided into the size and code of the motion vector for calculation. Afterwards, the same is true in other derivation methods of the differential vector.

[0121] refineMvL0[0]=((DistFromBaseMV*abs(mvdUnit[0]))>>shiftMMVD)*sign(mvdUnit[0]) refineMvL0[1]=((DistFromBaseMV*abs(mvdUnit[1]))>>shiftMMVD)*sign(mvdUnit[1]) In addition to the above, when the merge candidate N related to the center vector is a unidirectional prediction based on the L1 reference picture (predFlagL0N=0, predFlagL1N=1), the MMVD prediction unit 30376 derives the L1 differential vector refineMvL1 from the size of the basic motion vector and the differential vector DistFromBaseMV.

[0122] refineMvL0[0]=0 refineMvL0[1]=0 refineMvL1[0]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[0] refineMvL1[1]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[1] or refineMvL1[0]=(DistFromBaseMV*mvdUnit[0])>>shiftMMVD refineMvL1[1]=(DistFromBaseMV*mvdUnit[1])>>shiftMMVD In addition to the above, when the merge candidate N related to the center vector is bidirectionally predicted (predFlagL0N=1, predFlagL1N=1), the MMVD prediction unit 30376 derives the first differential vector firstMv from the size of the basic motion vector and the differential vector DistFromBaseMV.

[0123] firstMv[0]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[0] firstMv[1]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[1] or firstMv=(DistFromBaseMV*mvdUnit[0])>>shiftMMVD firstMv=(DistFromBaseMV*mvdUnit[1])>>shiftMMVD Here, firstMv corresponds to the difference vector of the reference picture with the larger POC distance (POC difference) between the target picture and the reference picture. That is, the reference picture with the larger POC distance (POC difference) between the target picture and the reference picture in the reference picture list L0 and the reference picture list L1 is used as the reference picture of the reference picture list LX. firstMv is the difference vector between the reference block of the reference picture in the list LX with the larger POC distance (POC difference) and the target block on the target picture.

[0124] Next, the MMVD prediction unit 30376 can derive the second motion vector secondMv of the other reference picture (reference list LY (Y=1-X)) by scaling firstMv. secondMv is a difference vector with respect to the reference picture in list LY having a small POC distance.

[0125] For example, when the distance between the target picture currPic and the L0 picture RefPicList0[refIdxLN0] is greater than the distance between the target picture and the L1 picture RefPicList1[refIdxLN1], firstMv corresponds to the L0 difference vector refineMvL0. Furthermore, the MMVD prediction unit 30376 may also scale firstMv to derive the L1 difference vector refineMvL1.

[0126] refineMvL0[0]=firstMv[0] refineMvL0[1] = firstMv[1] refineMvL1[0]=Clip3(-32768,32767,Sign(distScaleFactor*firstMv[0])*((Abs(distScaleFactor*refineMvL0[0])+127)>>8)) refineMvL1[1]=Clip3(-32768,32767,Sign(distScaleFactor*firstMv[1])*((Abs(distScaleFactor*refineMvL0[1])+127)>>8)) Here, the MMVD prediction unit 30376 derives the scaling value distScaleFactor from the POC difference between currPic and the L0 reference picture and the POC difference between currPic and the L1 reference picture as shown below. distScaleFactor = Clip3 (-4096, 4095, (tb*tx+32) >> 6) (equation scale - 1) tx=(16384+(Abs(td)>>1)) / td td=Clip3(-128,127,DiffPicOrderCnt(currPic,RefPicList0[refIdxLN0])) tb=Clip3(-128,127,DiffPicOrderCnt(currPic,RefPicList1[refIdxLN1])) DiffPicOrderCnt(currPic, RefPicList0[refIdxLN0]) is the POC difference between currPic and the L0 reference picture, and DiffPicOrderCnt(currPic, RefPicList1[refIdxLN1]) is the POC difference between currPic and the L1 reference picture.

[0127] In addition to the above, when the distance between the object picture currPic and the L0 picture RefPicList0[refIdxLN0] is less than the distance between the object picture and the L1 picture RefPicList1[refIdxLN1], the first vector firstMv corresponds to the L1 differential vector refineMvL1. In this case, the MMVD prediction unit 30376 can scale the first vector firstMv to derive the L0 differential vector refineMvL0.

[0128] refineMvL0[0]=Clip3(-32768,32767,Sign(distScaleFactor*firstMv[0])*((Abs(distScaleFactor*firstMv[0])+127)>>8)) refineMvL0[1]=Clip3(-32768,32767,Sign(distScaleFactor*firstMv[1])*((Abs(distScaleFactor*firstMv[1])+127)>>8)) refineMvL1[0] = firstMv[0] refineMvL1[1]=firstMv[1] distScaleFactor is derived from the above (equation scale-1), but is calculated using td and tb. td=Clip3(-128,127,DiffPicOrderCnt(currPic,RefPicList1[refIdxLN1])) tb=Clip3(-128,127,DiffPicOrderCnt(currPic,RefPicList0[refIdxLN0])) It should be noted that the above-mentioned bifurcation of "above the distance" and "other than the above, less than the distance..." may also be the bifurcation of "greater than the distance" and "other than the above, less than the distance...". It should be noted that when the distance between the object picture currPic and the L0 picture RefPicList0[refIdxLN0] is equal to the distance between the object picture and the L1 picture RefPicList1[refIdxLN1], the MMVD prediction unit 30376 may set refineMvLX[] by the following processing (processing A or processing B) without scaling firstMv[]. Process A: refineMvL0[0]=firstMv[0] refineMvL0[1] = firstMv[1] refineMvL1[0]=-firstMv[0] refineMvL1[1]=-firstMv[1] Treatment B: refineMvL0[0]=firstMv[0] refineMvL0[1] = firstMv[1] refineMvL1[0] = firstMv[0] refineMvL1[1]=firstMv[1] More specifically, the MMVD prediction unit 30376 derives refineMvLX[] through processing A when the L0 reference picture, the target picture currPic, and the L1 target picture are arranged in chronological order, and derives refineMvLX[] through processing B in other cases.

[0129] It should be noted that the case of chronological arrangement refers to the case where (POC_L0-POC_curr)*(POC_L1-POC_curr)<0, that is, the following formula. DiffPicOrderCnt(RefPicList0[refIdxLN0],currPic)*DiffPicOrderCnt(currPic,RefPicList1[refIdxLN1])>0 Here, POC_L0, POC_L1, and POC_curr represent the picture order counts (Picture Order Count: POC) of the L0 reference picture, the L1 reference picture, and the target picture, respectively.

[0130] The opposite case (opposite to the time sequence) refers to the case where (POC_L0-POC_curr)*(POC_L1-POC_curr)>0, that is, the following formula. DiffPicOrderCnt(RefPicList0[refIdxLN0],currPic)*DiffPicOrderCnt(currPic,RefPicList1[refIdxLN1])<0 It should be noted that even when the distances between POCs are different, the MMVD prediction unit 30376 can derive the final refineMvLX[] based on the POC distance between the reference image and the object image using refineMvLX[] as a scale after deriving refineMvLX[] through processing A or processing B. (Addition of center vector and difference vector)

[0131] Finally, the MMVD prediction unit 30376 derives the motion vector of the MMVD merge candidate from the differential vector refineMv[ ] and the center vector mvLXN[ ] (mvpLX[ ]) as described below. mvL0[0]=mvL0N[0]+refineMvL0[0] mvL0[1]=mvL0N[1]+refineMvL0[1] mvL1[0]=mvL1N[0]+refineMvL1[0] mvL1[1]=mvL1N[1]+refineMvL1[1] It should be noted that, when a temporary motion vector is derived, the above mvLX[] is recorded as tempMvLX[]. (Summarize)

[0132] In this way, even if the center vector is bidirectionally predicted, the MMVD prediction unit 30376 only notifies the information of the motion vector of one side (mmvd_direction_idx, mmvd_distance_idx). Then, two motion vectors are derived from this information. The MMVD prediction unit 30376 scales the motion vector according to the difference between the POC of each of the two reference pictures and the POC of the target picture as needed. The differential vector between the reference block of the reference picture with the larger POC distance (POC difference) and the target block on the target picture is the first differential vector (firstMv).

[0133] firstMv[0]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[0] firstMv[1]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[1] The MMVD prediction unit 30376 derives the motion vector mvdLY(secondMv), LY(Y=1-X) of the reference picture with the smaller POC distance by scaling it according to the ratio of the POC distances between the pictures (POCS / POCL).

[0134] secondMv[0]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[0]*POCS / POCL secondMv[1]=(DistFromBaseMV< <shiftMMVD)*mvdUnit[1]*POCS / POCL It should be noted that the reference picture with a smaller POC distance corresponds to the one with a smaller POC distance (POC difference) between the object picture and the reference picture. Here, POCS is the difference between the POC difference between the reference picture close to the object picture and the object picture, and POCL is the difference between the POC difference between the reference picture far from the object picture and the object picture.

[0135] As described above, the MMVD prediction unit 30376 derives mvpLX[] (mvLXN[]) and refineMvLX[], and uses these to derive the motion vector mvLX[] of the target block.

[0136] mvLX[0]=mvpLX[0]+refineMvLX[0] mvLX[1]=mvpLX[1]+refineMvLX[1] (Rounding of motion vector) The MMVD prediction unit 30376 may perform correction when the size of the difference vector DistFromBaseMV added to the center vector is greater than a predetermined threshold value so that the motion vector mvLX of the object block represents an integer pixel position. For example, the MMVD prediction unit 30376 may perform integerization (rounding) when DistFromBaseMV is greater than a predetermined threshold value of 16.

[0137] In addition, the MMVD prediction unit 30376 may perform integerization when distance_list_idx is a specific distance table (e.g., DistanceTable2) and mmvd_distance_idx is a specific range (e.g., mmvd_distance_idx is 2, 3). distance_list_idx is an index to select a distance table, and mmvd_distance_idx is an index to select an element of the distance table (a coefficient of the selected distance). For example, the MMVD prediction unit 30376 may modify mvLX according to the following formula when distance_list_idx == 1 and mmvd_distance_idx>=2.

[0138] mvLX[0]=(mvLX[0] / MVPREC)*MVPREC mvLX[1]=(mvLX[1] / MVPREC)*MVPREC In addition, the MMVD prediction unit 30376 can also use shifting to derive mvLX.

[0139] mvLX[0]=(mvLX[0]>>MVBIT)< <MVBIT mvLX[1]=(mvLX[1]>>MVBIT)< <MVBIT Here, MVBIT=log2(MVPREC). For example, 4. In addition, the positive and negative values ​​can be considered and derived as follows.

[0140] mvLX[0]=mvLX[0]>=0? (mvLX[0]>>MVBIT)<<MVBIT:-((-mvLX[0]> >MVBIT) <MVBIT) mvLX[1]=mvLX[1]>=0? (mvLX[1]>>MVBIT)<<MVBIT:-((-mvLX[1]> >MVBIT) <MVBIT) In this way, the amount of calculation required for generating a predicted image can be reduced by rounding the motion vector. (grammar)

[0141] Fig. 9 This is an example of the syntax of merge_data() notified when merge prediction is turned on (general_merge_flag==1) in the target block. general_merge_flag is a flag notified when the target block is not in skip mode, indicating whether the prediction parameters of the target block are derived from adjacent blocks. In other words, it is a flag indicating whether the merge mode is used. In the case of skip mode, the inter-frame prediction parameter derivation unit 303 sets general_merge_flag=1. merge_data() is a syntax structure for notifying the parameters of merge prediction.

[0142] merge_subblock_flag is a flag indicating whether to derive parameters of inter-frame prediction based on subblocks of the target block from adjacent blocks. When merge_subblock_flag is 1, it indicates that inter-frame prediction based on subblocks is used. When merge_subblock_flag is 0, it indicates that inter-frame prediction based on subblocks is not used.

[0143] regular_merge_flag is a flag indicating whether to use the normal merge mode or the merge mode using differential motion vector (MMVD) in the target block. When regular_merge_flag is 1, it indicates that the normal merge mode or MMVD is used. When regular_merge_flag is 0, the normal merge mode and MMVD are not used.

[0144] mmvd_merge_flag is a flag indicating whether MMVD is used in the object block. When mmvd_merge_flag is 1, it indicates that MMVD is used in the object block. The parameter decoding unit 302 decodes mmvd_cand_flag, mmvd_distance_idx, and mmvd_direction_idx. When mmvd_merge_flag is 0, it indicates that the normal merge mode is used in the object block. Moreover, mmvd_cand_idx is derived from mmvd_distance_idx and mmvd_direction_idx. mmvd_cand_idx is an index indicating an MMVD candidate included in the MMVD candidate list. The MMVD candidate list is a list storing MMVD candidates.

[0145] mmvd_cand_idx=mmvd_distance_idx*numDir+mmvd_direction_idx Here, mmvd_distance_idx=0..numDist-1, mmvd_direction_idx=0..numDir-1. numDir indicates the number of predetermined directions of the difference vector in MMVD (for example, if there are 16 directions, numDir=16). numDist indicates the number of predetermined distances of the difference vector in MMVD (for example, if there are 6 distances, numDist=6). Hereinafter, mmvd_distance_idx and mmvd_direction_idx may be abbreviated as dist and dir.

[0146] Fig.10 This is another example of the syntax table of the MMVD prediction unit 30376. When it is indicated that MMVD is used in the target block (mmvd_merge_flag==1), the parameter decoding unit 302 decodes mmvd_cand_flag and mmvd_cand_idx. mmvd_cand_idx may take a value from 0 to maxNumMmvdLUT-1 (eg, 11). Fig.10 Instead of using the signal to indicate mmvd_cand_idx Fig. 9 The parameter decoding unit 302 can perform encoding and decoding using TR (Truncated Rice) binary or TB (Truncated Binary) with mmvd_cand_idx set to cMax=11 and Rice parameter set to 1. cMax is the upper limit of the possible values ​​of the syntax element. (Another configuration of the MMVD prediction unit 30376)

[0147] Fig.11 This is a flowchart showing the processing of another configuration of the MMVD prediction unit.

[0148] Another configuration of the MMVD prediction unit 30376 is described. In the present embodiment, when the MMVD prediction is used (mmvd_merge_flag is 1), the MMVD prediction unit 30376 derives the MMVD candidate list mmvdLUT (S301), and uses the index mmvd_cand_idx representing the MMVD candidate to derive the difference vector (MVD) (S302). The MMVD prediction unit 30376 selects the MMVD candidate using mmvdLUT and mmvd_cand_idx, and derives the motion vector of the target block. For example, the MMVD prediction unit 30376 may also select the MMVD candidate through mmvdLUT [mmvd_cand_idx].

[0149] The MMVD candidates may also be the motion vectors of L0 and L1 (MvL0[0], MvL0[1], MvL0[0], MvL0[1]). In this case, the mmvdLUT is as follows.

[0150] mmvdLUT[][]={(MvL0[0],MvL0[1],MvL0[0],MvL0[1]),(MvL0[0],MvL0[1],MvL0[0],MvL0[1]),…} The MMVD prediction unit 30376 selects an MMVD candidate using mmvdLUT and mmvd_cand_idx (=tempIdx), and derives a motion vector. mvL0[0](=tempMvL0[0])=mvL0[0] of mmvdLUT[mmvd_cand_idx](formula MMVD-1) mvL0[1](=tempMvL0[1])=mvL0[1] of mmvdLUT[mmvd_cand_idx] mvL1[0](=tempMvL1[0])=mvL1[0] of mmvdLUT[mmvd_cand_idx] mvL1[1](=tempMvL1[1])=mvL1[1] of mmvdLUT[mmvd_cand_idx] The above can also be expressed as follows.

[0151] mvL0[0](=tempMvL0[0])=mmvdLUT[mmvd_cand_idx][0] mvL0[1](=tempMvL0[1])=mmvdLUT[mmvd_cand_idx][1] mvL1[0](=tempMvL1[0])=mmvdLUT[mmvd_cand_idx][2] mvL1[1](=tempMvL1[1])=mmvdLUT[mmvd_cand_idx][3] In addition, the MMVD candidate can also be a differential vector (firstMv[0], firstMv[1]).

[0152] mmvdLUT[][]={(fistMv[0], fisetMv[1]), (fistMv[0], fisetMv[1]), (fistMv[0],fisetMv[1]),…} The MMVD prediction unit 30376 selects an MMVD candidate using mmvdLUT and mmvd_cand_idx (=tempIdx), derives firstMv, and derives a motion vector mvLX (=tempMvLX) from firstMv.

[0153] firstMv[0]=firstMv[0] of mmvdLUT[mmvd_cand_idx] firstMv[1]=firstMv[1] of mmvdLUT[mmvd_cand_idx] The above can also be expressed as follows.

[0154] firstMv[0]=mmvdLUT[mmvd_cand_idx][0] firstMv[1]=mmvdLUT[mmvd_cand_idx][1] The method of deriving the motion vector tempMvLX from firstMv has been described.

[0155] Alternatively, the MMVD candidate may be the position of a differential vector (the direction and distance from the center vector).

[0156] mmvdLUT[][]={(mmvd_direction_idx,mmvd_distance_idx),(mmvd_direction_idx,mmvd_distance_idx),(mmvd_direction_idx,mmvd_distance_idx),…} The MMVD prediction unit 30376 may select the MMVD candidate (mmvd_direction_idx, mmvd_distance_idx) using the mmvdLUT and mmvd_cand_idx (=tempIdx).

[0157] mmvd_direction_idx=mmvd_direction_idx of mmvdLUT[mmvd_cand_idx] mmvd_distance_idx=mmvd_distance_idx of mmvdLUT[mmvd_cand_idx] The above can also be expressed as follows.

[0158] mmvd_direction_idx=mmvdLUT[mmvd_cand_idx][0] mmvd_distance_idx=mmvdLUT[mmvd_cand_idx][1] mmvd_cand_idx=mmvd_distance_idx*numDir+mmvd_direction_idx The motion vector mvLX (=tempMvLX) can also be derived from mmvd_cand_idx according to (Formula MMVD-1).

[0159] In addition, the MMVD candidate may be only the number of the MMVD candidate.

[0160] mmvdLUT[][]={0, 1, 2, 3,…} In this case, the following firstMv[] may be used to derive a motion vector from the best number mmvdLUT[mmvd_cand_idx] or mmvdLUT[tempIdx].

[0161] dist = tempIdx / numDir; dir = tempIdx - dist * numDir; firstMv[0]=(dist< <shiftMMVD)*dir_table_x[dir] firstMv[1]=(dist< <shiftMMVD)*dir table y[dir] In this embodiment, by using the mmvd candidate list arranged in ascending order of cost, the amount of code of the information (mmvd_distance_idx, mmvd_direction_idx or mmvd_cand_idx) required for differential vector derivation is reduced, thereby improving the coding efficiency. (MMVD candidate list derivation process)

[0162] The MMVD prediction unit 30376 calculates the cost tempCost (template matching cost) for each MMVD candidate, and derives the final MMVD candidate list mmvdLUT by sorting the MMVD candidates in ascending order according to the cost. It is also possible to first derive all the elements of the mmvdLUT in advance, and use the cost to exchange the candidates of the mmvdLUT with each other, thereby deriving mmvdLUT1. In addition, the MMVD prediction unit 30376 may also set the empty state of the MMVD candidate list mmvdLUT as the initial state, and add the MMVD candidates to the list in order from small to large cost, thereby deriving the MMVD candidate list mmvdLUT. In addition, when adding MMVD candidates to the list, if the additional position exceeds the maximum number of candidates maxNumMmvdLUT (for example, 12) in the list, the MMVD prediction unit 30376 may also skip the addition of MMVD candidates.

[0163] Here, Fig.11 The details of the flowchart are described below.

[0164] First, the MMVD prediction unit 30376 initializes the MMVD candidate list mmvdLUT (S3011). It is also possible to derive all candidate motion vectors as elements of mmvdLUT. For example, tempIdx=0..numDir*numDist-1 is derived.

[0165] dist = tempIdx / numDir; dir = tempIdx - dist * numDir; Using the above formula, it is also possible to use mmvdLUT with {dir, dist} as an element. It is also possible to use mmvdLUT with {firstMv[0], firstMv[1]} derived from {dir, dist} as an element. Furthermore, it is also possible to use a table with {mvLX[0], mvLX[1]} derived from firstMv as an element. Here, the MMVD prediction unit 30376 may obtain the direction (dir) and the distance (dist) from the integer (index) tempIdx representing each mmvdMergeCand.

[0166] Use dist and dir to deduce firstMv.

[0167] firstMv[0]=(dist< <shiftMMVD)*dir_table_x[dir] firstMv[1]=(dist< <shiftMMVD)*dir_table_y[dir] Then, refineMvL0 and refineMvL1 are derived from firstMv by the same method as (derivation of differential vector), and tempMvLX[] (X=0, 1) is derived.

[0168] tempMvLX[0]=mvpLX[0]+refineMvLX[0] tempMvLX[1]=mvpLX[1]+refineMvLX[1] The derived tempMvLX is stored as the element mmvdLUT[tempIdx] of mmvdLUT.

[0169] Next, the MMVD prediction unit 30376 loops according to the required number of candidates (eg, NumMmvdCand=numDir*numDist) to derive the cost tempCost of the MMVD candidate (S3012).

[0170] The MMVD prediction unit 30376 obtains the direction (dir) and distance (dist) for each MMVD candidate (mmvdMergeCand). Then, the MMVD prediction unit 30376 obtains the motion vector tempMvLX of the MMVD candidate (S3013).

[0171] Next, the MMVD prediction unit 30376 performs template matching using tempMvLX and calculates the cost (S3014). The pixels to the left and above the object block are used as the template (rec template) and the pixels of the reference block represented by tempMvLx (ref template). The MMVD prediction unit 30376 may also perform template matching to derive the ref template (and the corresponding differential vector tempMvLx) that minimizes the difference (cost, tempCost) between the rec template and the ref template. Alternatively, the sum of absolute differences (SAD: Sum of Absolute Difference) or the sum of squared differences (SSD: Sum of Squared Distance) may be used as tempCost.

[0172] For example, the sum of the absolute difference between the left neighboring area of ​​the reference block and the left neighboring area of ​​the target block and the sum of the absolute difference between the upper neighboring area of ​​the reference block refSamplesLX and the upper neighboring area of ​​the target block may be set as tempCost.

[0173] tempCost=∑abs(refSamplesLX[xC+iL+tempMvLX[0]][yC+jL+tempMvLX[1]]-recSamples[xC+i][xC +j])+∑abs(refSamplesLX[xC+iT+tempMvLX[0]][yC+jT+tempMvLX[1]]-recSamples[xC+i][xC+jT]) Here, X=0 or 1.

[0174] Here, refSamples is the reference picture, recSamples is the target picture, tempMvLX[2] is the motion vector of the MMVD candidate, xC, yC are the upper left coordinates of the target block, nW, nH are the width and height of the block. Σ is the sum related to iL=-1, jL=0..nH-1, iT=0..nW-1, jT=-1. It should be noted that the above processing may be performed after the decimal precision is corrected to integer precision.

[0175] tempMvLX[0]=(abs(tempMvLX[0]>>shiftMMVD)*sign(tempMvLX[0]) tempMvLX[1]=(abs(tempMvLX[1]>>shiftMMVD)*sign(tempMvLX[1]) In the case of 1 / 16 accuracy, shiftMMVD=4. In order to derive firstMv with integer precision, firstMv may also be derived in advance as follows.

[0176] firstMv[0]=dist*dir_table_x[dir] firstMv[1]=dist*dir_table_y[dir] Then, the MMVD prediction unit 30376 updates the MMVD candidate list according to the cost obtained in S3014 (S3015). The MMVD prediction unit 30376 may also sort the MMVD candidate list in ascending order of cost. It should be noted that, as shown below, the position insertPos for inserting mmvdLUT may also be derived, and the table below insertPos may be updated. insertPos=0 while(insertPos <maxNumMmvdLUT&&tempCost<candCostList[endIdx-1-insertPos]){ insertPos++; } if (insertPos!=0) { for(i=1;i <insertPos;i++) { mmvdLUT[endIdx-i]=mmvdLUT[endIdx-1-i] candCostList[endIdx-i]=candCostList[endIdx-1-i] } mmvdLUT[endIdx-insertPos]=tempIdx candCostList[endIdx-insertPos]=tempCost } Here, while(x){process} is a loop process that repeatedly performs process while x is true. mmvdLUT is a list storing tempIdx. endIdx is the last index in the group, for example, endIdx = (tempIdx / grpSize)*grpSize+grpSize. Where grpSize = any one of 1, 2, 4, and 8. candCostList[idx] is a list of tempCost of the MMVD candidate represented by idx.

[0177] The MMVD prediction unit 30376 performs the processes of S3013 to S3015 according to the number of MMVD candidates. Through the above processes, the MMVD prediction unit 30376 derives an MMVD candidate list.

[0178] According to this embodiment, the MMVD prediction unit 30376 performs template matching on all MMVD candidates and performs cost calculation. For example, when there are 16 directions and 6 distances of MMVD, 96 template matchings (cost derivation and ranking processing) are performed. (Another example of the MMVD candidate list derivation process)

[0179] In this embodiment, the MMVD candidates for cost calculation are adaptively selected, thereby reducing the amount of calculation without reducing the coding efficiency.

[0180] Fig.12 303 is a flowchart showing the process of determining the direction and distance of the differential vector in this embodiment. It should be noted that the process of selecting the MMVD candidate (S302) by the MMVD prediction unit 30376 using mmvdLUT and mmvd_cand_idx is similar to the process of Fig.11 Same description as .

[0181] First, the MMVD prediction unit 30376 initializes the MMVD candidate list (S4011). The MMVD candidate represents the position of the differential vector (the direction and distance from the center vector). The MMVD candidate list is a list that stores MMVD candidates. Next, the MMVD prediction unit 30376 loops through the cost derivation process for the number of candidates (numDir*numDist) that can become differential vectors (S4012).

[0182] The MMVD prediction unit 30376 derives the direction (dir) and distance (dist) corresponding to each MMVD candidate (mmvdMergeCand) (S4013).

[0183] The MMVD prediction unit 30376 determines the search direction (dir) of mmvdMergeCand in the current distance (dist) (S4014). The MMVD prediction unit 30376 determines whether it is a derived search direction for each mmvdMergeCand (S4015), and if it is a search direction (Y in S4015), the cost is derived. Hereinafter, searching XX means deriving tempCost using XX as a search candidate.

[0184] When it is determined that it is not a search direction (N in S4015), the MMVD prediction unit 30376 does not derive the cost for mmvdMergeCand, and moves to the cost derivation of the next mmvdMergeCand.

[0185] In the case of a search direction (Y in S4015), the MMVD prediction unit 30376 obtains the current temporary difference vector (firstMv) based on the direction and distance of the search candidate (S4016).

[0186] Next, the MMVD prediction unit 30376 performs template matching using tempMvLX derived by the same method as in (MMVD candidate list derivation process) to calculate the cost tempCost (S4017). Then, the MMVD prediction unit 30376 updates the MMVD candidate list in ascending order of cost based on the cost obtained in S4017 (S4018).

[0187] The MMVD prediction unit 30376 performs the processes of S4013 to S4018 according to the number of MMVD candidates. Through the above processes, the MMVD prediction unit 30376 derives an MMVD candidate list.

[0188] also, Fig.13 is another flowchart showing the process of determining the direction and distance of the differential vector in this embodiment. It should be noted that the process of the MMVD prediction unit 30376 selecting the MMVD candidate (S302) using mmvdLUT and mmvd_cand_idx is similar to the process of Fig.11 same.

[0189] First, the MMVD prediction unit 30376 initializes the MMVD candidate list (S5011). Next, the MMVD prediction unit 30376 loops for the number of candidates (numDist) that can be the distance of the differential vector (S5012). Here, the MMVD prediction unit 30376 determines the direction (di) of the differential vector to be searched in the current distance (dist). r )(S5013). The details will be described later. Next, the MMVD prediction unit 30376 loops for the number of candidates determined as the direction to be searched in S5013 (S5014).

[0190] The MMVD prediction unit 30376 derives the motion vector of the MMVD candidate based on the direction and distance of the differential vector (S5016). Next, the MMVD prediction unit 30376 uses the motion vector of the MMVD candidate to perform template matching and calculate the cost (S5017). Then, the MMVD prediction unit 30376 updates the MMVD candidate list in ascending order of cost based on the cost calculated in S5017 (S4018).

[0191] The MMVD prediction unit 30376 performs the processes of S5016 to S5018 for the MMVD candidates according to the number of candidates in the search direction. In addition, the MMVD prediction unit 30376 performs the processes of S5013 to S5018 for the MMVD candidates according to the number of candidates in the distance. Through the above processes, the MMVD prediction unit 30376 derives the MMVD candidate list. (Details of the method for determining the search direction)

[0192] A method of determining the search direction based on the distance (dist) of mmvdMergeCand, which corresponds to processing S4014 and S5013, is described. <Method 1 for determining the search direction>

[0193] The MMVD prediction unit 30376 can determine the search direction based on the following: The following describes a case where numDir=16 (16 directions). 1: When the distance is 0 (dist==0), only the directions included in the direction group DIR1 are searched. 2: In the case of other distances (dist!=0), only the directions included in the direction group DIR2 are searched.

[0194] Here, direction groups DIR1 and DIR2 are subsets (parts) of directions of MMVD candidates, respectively. DIR1 may include 1 / 2 of all directions. For example, it may also be a direction dir of dir=0…numDir / 2-1. For example, DIR1 may include 8 directions: horizontal, vertical, and inclined 45 degrees. As shown below, it can be represented by an integer representing the direction or by a label.

[0195] DIR1 = {0, 1, 2, 3, 4, 5, 6, 7} DIR1 = {up, upper right, right, lower right, lower, lower left, left, upper left} The direction group DIR2 may include a total of three directions, namely, the direction of the minimum cost in the template matching cost of the search number obtained in the previous distance dist and the directions adjacent thereto. Alternatively, the direction group DIR2 may also include two directions of the minimum cost in the template matching cost of the search number obtained in the previous dist and the directions adjacent thereto at most six directions respectively. The directions included in the direction group DIR2 may also be updated for each dist.

[0196] Below, use Fig.14 , Fig.15 , according to Fig.12 , Fig.13 The example of limiting the search direction by distance shown in FIG. 1 is used to illustrate the difference vector candidates in MMVD. The difference vector candidates in MMVD are set to 96 in total, with 6 distances and 16 directions. The relationship between the direction and the index is as follows: Fig.14 as shown in (a) and (b). Fig.15 The search candidates until the distance dist at this time is 3 are shown. The following shows the case where the MMVD prediction unit 30376 searches for MMVD candidates. In the following example, the MMVD prediction unit 30376 searches for 23 (8+3*5) candidates.

[0197] Hereinafter, the operation of “adding the searched candidate to the MMVD candidate list and updating the MMVD candidate list in ascending order of cost” is described as “updating mmvdLUT for the searched candidate”. <Step 1>: Action when dist = 0 Use the direction group DIR1 = {0, 1, 2, 3, 4, 5, 6, 7} to search for the eight directions of horizontal, vertical, and 45 degree tilt. Update mmvdLUT for the searched candidates. The search result shows that dir = 5 is the minimum cost. <Step 2>: Action when dist = 1 In step 1, select the three directions of dir=5 and adjacent dir=11 and 15 that provide the minimum cost, and derive the direction group DIR2={5, 11, 15} for step 2. Use DIR2={5, 11, 15} to search for three directions. Update mmvdLUT for the searched candidates. The result of the search is that dir=5 has the minimum cost. <Step 3>: Action when dist = 2 Based on the result of step 2, deduce DIR2 = {5, 11, 15} of step 3. Use DIR2 = {5, 11, 15} to search 3 directions. Update mmvdLUT for the searched candidates. The result of the search is that dir = 15 is the minimum cost. <Step 4>: Action when dist = 3 Based on the result of step 3, deduce DIR2={3, 5, 15} of step 4. Use DIR2={3, 5, 15} to search 3 directions. Update mmvdLUT for the searched candidates. The result of the search is that dir=3 is the minimum cost. <Step 5>: Action when dist = 4 Based on the result of step 4, deduce DIR2={3, 13, 15} of step 5. Use DIR2={3, 13, 15} to search 3 directions. Update mmvdLUT for the searched candidates. The result of the search is that dir=3 is the minimum cost. <Step 6>: Action when dist = 5 According to the result of step 5, deduce DIR2 = {3, 13, 15} of step 6. Use DIR2 = {3, 13, 15} to search 3 directions. Update mmvdLUT for the searched candidates.

[0198] According to the above steps, the MMVD prediction unit 30376 derives the MMVD candidate list.

[0199] The following shows an example in which the direction group DIR2 is composed of a total of six directions, namely, two directions providing the minimum cost and directions adjacent to the two directions. <Step 1>: Action when dist = 0 Use the direction group DIR1 = {0, 1, 2, 3, 4, 5, 6, 7} to search for the eight directions of horizontal, vertical, and 45 degree tilt. Update mmvdLUT for the searched candidates. The search result shows that dir = 5 has the minimum cost and dir = 0 has the second minimum cost. <Step 2>: Action when dist = 1 According to the result of step 1, use the direction group DIR2 = {5, 11, 15, 0, 8, 10} to search 6 directions. Update mmvdLUT for the searched candidates. The result of the search is that dir = 10 has the minimum cost and dir = 11 has the second minimum cost. <Step 3>: Action when dist = 2 According to the result of step 2, use the direction group DIR2 = {0, 6, 10, 1, 5, 11} to search 6 directions. Update mmvdLUT for the searched candidates. The result of the search is that dir = 10 has the minimum cost and dir = 0 has the second minimum cost. <Step 4>: Action when dist = 3 Based on the result of step 3, 4 directions are searched using direction group DIR2 = {0, 6, 8, 10}. When the directions providing low costs are adjacent, sometimes there are not 6 directions as in this example. The mmvdLUT is updated for the searched candidates. The subsequent processing is omitted, but the MMVD prediction unit 30376 performs the same processing until <step 6>.

[0200] According to the above steps, the MMVD prediction unit 30376 derives the MMVD candidate list.

[0201] In this embodiment, in the MMVD candidate search, a selection is made according to each distance of mmvdMergeCand for cost calculation. In particular, it is characterized in that in the search, only a subset of the first direction (DIR1) is searched within a specified distance, and only a subset of the second direction (DIR2) is searched within a distance other than the specified distance. It is characterized in that in the search, the directions included in the subset of the second direction (DIR2) are searched for each distance candidate, and the directions included in the subset of the second direction (DIR2) are updated for each distance candidate.

[0202] This reduces the amount of calculation without decreasing coding efficiency. <Method 2 for determining the search direction>

[0203] The MMVD prediction unit 30376 has a plurality of distance sets. At this time, the MMVD prediction unit 30376 may determine the direction of the search based on the following. 1: In the case of distances included in the distance set DIST1, only directions included in the direction set DIR1 are searched. 2: In the case of distances included in other distance sets, only directions included in direction group DIR2 are searched.

[0204] Here, the direction group DIR1 may be as shown in <Method 1 for determining the search direction>.

[0205] The direction group DIR2 may include a total of 3 directions, which are the direction with the minimum cost in the template matching cost of the search number obtained in the previous distance set and the directions adjacent to it. Alternatively, the direction group DIR2 may also include 2 directions with the minimum cost in the template matching cost of the search number obtained in the previous distance set and the directions adjacent to the minimum cost at most 4 directions. Alternatively, the direction group DIR2 may also include 2 directions with the minimum cost in the template matching cost of the search number obtained in the previous distance set and the directions adjacent to them at most 6 directions.

[0206] The number of directions included in the direction group DIR2 may also be updated for each distance set. In addition, the method for selecting the directions included in the direction group DIR2 may also be different for each distance set. For example, the number of directions searched in the distance set DIST2 may be 6, and the number of directions searched in the distance set DIST3 may be 4 or 3.

[0207] The distance set DIST1 may only include dist=0. At this time, the distance set DIST2 may include dist=1, 2, and the distance set DIST3 may include dist=3, 4, . . . .

[0208] Alternatively, the distance set DIST1 may include dist=0, 1. In this case, the distance set DIST2 may include dist=2, 3, and the distance set DIST3 may include dist=4, 5, ... . . . . .

[0209] The following describes how the MMVD prediction unit 30376 searches for MMVD candidates. <Step 1>: Action when distance set DIST1 = {0} The direction group DIR1 = {0, 1, 2, 3, 4, 5, 6, 7} is used to search for eight directions: horizontal, vertical, and 45-degree inclined. The mmvdLUT is updated for the searched candidates. In addition, the search result shows that dir = 5 is the minimum cost. <Step 2>: Action when distance set DIST2 = {1, 2} According to the result of step 1, the direction group DIR2 = {5, 11, 15} is used to search for the three directions of dir = 5 and adjacent dir = 11 and 15 that provide the minimum cost. The mmvdLUT is updated for the searched candidates. In addition, the search result shows that dir = 5 has the minimum cost. <Step 3>: Action when distance set DIST3 = {3, 4, 5} According to the result of step 2, three directions are searched using the direction group DIR2 = {5, 11, 15}, and mmvdLUT is updated for the searched candidates.

[0210] According to the above steps, the MMVD prediction unit 30376 derives the MMVD candidate list.

[0211] In addition, an example is shown in which the method of selecting directions included in the direction group DIR2 is different for each distance set. The following is an example in which the number of directions to be searched in the distance set DIST2 is 6, and the number of directions to be searched in the distance set DIST3 is 3. <Step 1>: Action when distance set DIST1 = {0} Use the direction group DIR1 = {0, 1, 2, 3, 4, 5, 6, 7} to search for the eight directions of horizontal, vertical, and 45 degree tilt. Update mmvdLUT for the searched candidates. In addition, the search result shows that dir = 5 has the smallest cost and dir = 0 has the second smallest cost. <Step 2>: Action when distance set DIST2 = {1, 2} According to the result of step 1, the direction group DIR2 = {5, 11, 15, 0, 8, 10} is used to search for 6 directions. The mmvdLUT is updated for the searched candidates. In addition, the result of the search is that dir = 10 is the minimum cost. <Step 3>: Action when distance set DIST3 = {3, 4, 5} According to the result of step 2, three directions are searched using the direction group DIR2 = {0, 6, 10}, and mmvdLUT is updated for the searched candidates.

[0212] According to the above steps, the MMVD prediction unit 30376 derives the MMVD candidate list.

[0213] In this embodiment, the mmvdMergeCand for cost calculation is selected for each distance in the MMVD candidate search. In particular, it is characterized in that only a subset (DIR1) of the first direction is searched among the distances included in the specified distance set, and only a subset (DIR2) of the second direction is searched among the distances included in the distance set other than the specified distance set. In addition, it is characterized in that the directions included in the subset (DIR2) of the second direction are searched for the distance set including multiple distance candidates, and the directions included in the subset (DIR2) of the second direction are updated for each distance set including multiple distance candidates. In addition, it is characterized in that in the search, the number of directions included in the subset (DIR2) of the second direction varies according to the distance set.

[0214] In addition, although the case of 16 directions is described above, in the case of 8 directions (numDir=8), dir=0...numDir-1 and DIR1=0...numDir / 2-1 may be set to determine the search direction according to the distance. In addition, in the case of 4 directions (numDir=4), dir=0...numDir-1 and DIR1=0...numDir / 2-1 may be set to determine the search direction according to the distance.

[0215] This reduces the amount of calculation without reducing coding efficiency. In addition, by selecting the search direction for each of the plurality of distances, it is possible to reduce dependency, improve parallelism, and perform calculations efficiently. (Search terminated early)

[0216] The MMVD prediction unit 30376 calculates the template matching cost for each MMVD candidate mmvdMergeCand. At this time, if the above cost meets the following conditions, the MMVD prediction unit 30376 can end the search. <Search early termination condition 1>

[0217] The cost is less than a predetermined threshold value TH1. Here, TH1 can be set as shown below using the width width and height height of the target block. TH1=C*width*height C is a constant (eg, 5). That is, when the cost per pixel becomes less than C, the MMVD prediction unit 30376 ends the search. <Search early termination condition 2>

[0218] The cost is greater than a predetermined constant TH2 times the minimum cost. Here, TH2 is, for example, 1.05 or 1.1. That is, when sufficient cost reduction cannot be achieved for a candidate with a large distance, the MMVD prediction unit 30376 does not search for a candidate with a larger distance.

[0219] By terminating the search early as described above, the amount of calculation can be reduced without reducing the coding efficiency. (Inter-frame prediction image generation unit 309)

[0220] When predMode indicates inter prediction, the inter prediction image generation unit 309 generates a prediction image of a block or a subblock by inter prediction using the inter prediction parameters input from the inter prediction parameter derivation unit 303 and a reference picture.

[0221] Figure 4 1 is a schematic diagram showing the structure of the inter-frame prediction image generation unit 309 included in the prediction image generation unit 308 of the present embodiment. The inter-frame prediction image generation unit 309 is configured to include a motion compensation unit (prediction image generation device) 3091 and a synthesis unit 3095. The synthesis unit 3095 is configured to include an intra-frame inter-frame (IntraInter) synthesis unit 30951, a GPM synthesis unit 30952, a BIO unit 30954, and a weight prediction unit 3094. (Motion Compensation)

[0222] The motion compensation unit 3091 (interpolation image generation unit 3091) generates an interpolation image (motion compensation image) by reading a reference block from the reference picture memory 306 based on the inter prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter prediction parameter derivation unit 303. The reference block is a block that is shifted from the position of the target block by mvLX on the reference picture RefPicLX specified by refIdxLX. Here, when mvLX is not an integer precision, filtering for generating pixels at decimal positions, called motion compensation filtering, is performed to generate an interpolation image.

[0223] The motion compensation unit 3091 first derives the integer position (xInt, yInt) and phase (xFrac, yFrac) corresponding to the coordinates (x, y) in the prediction block through the following formula.

[0224] xInt=xPb+(mvLX[0]>>(log2(MVPREC)))+x xFrac = mvLX[0] & (MVPREC - 1) yInt=yPb+(mvLX[1]>>(log2(MVPREC)))+y yFrac=mvLX[1]&(MVPREC-1) Here, (xPb, yPb) is the upper left coordinate of the block of size bW*bH, x=0...bW-1, y=0...bH-1, and MVPREC represents the precision of mvLX (1 / MVPREC pixel precision). For example, MVPREC=16.

[0225] The motion compensation unit 3091 performs horizontal interpolation processing on the reference picture refImg using an interpolation filter, thereby deriving a temporary image temp[][]. Hereinafter, Σ is a sum related to k of k=0..NTAP-1, shift1 is a normalization parameter of the adjustment value interval, and offset1=1<<(shift1-1).

[0226] temp[x][y]=(∑mcFilter[xFrac][k]*refImg[xInt+k-NTAP / 2+1][yInt]+offset1)>>shift1 Next, the motion compensation unit 3091 performs vertical interpolation processing on the temporary image temp[][] to derive an interpolated image Pred[][]. Hereinafter, Σ is a sum related to k of k=0..NTAP-1, shift2 is a normalization parameter of the interval of the adjustment value, and offset2=1<<(shift2-1).

[0227] Pred[x][y]=(∑mcFilter[yFrac][k]*temp[x][y+k-NTAP / 2+1]+offset2)>>shift2 It should be noted that, in the case of bidirectional prediction, for the above-mentioned Pred[][], the interpolated images PredL0[][] and PredL1[][] are derived according to each L0 list and L1 list, and the interpolated image Pred[][] is generated based on PredL0[][] and PredL1[][]. (GPM synthesis processing)

[0228] When ciip_mode is 0, the GPM synthesis unit 30952 generates a prediction image in the GPM mode by taking the weighted sum of a plurality of inter-frame prediction images. (Intra-frame and inter-frame synthesis processing)

[0229] When ciip_mode is 1, the intra-frame inter-frame synthesis unit 30951 generates a prediction image in the CIIP mode by performing a weighted sum of the inter-frame prediction image and the intra-frame prediction image. (BIO forecast)

[0230] In the bidirectional prediction mode, the BIO unit 30954 generates a predicted image by referring to two predicted images (a first predicted image and a second predicted image) and a gradient correction item. (Weighted prediction)

[0231] The weight prediction unit 3094 generates a predicted image of the block by multiplying the interpolation image PredLX by the weight coefficient.

[0232] The inter-frame prediction image generation unit 309 outputs the generated prediction image of the block to the addition unit 312 . (Configuration of Moving Image Coding Device)

[0233] Next, the configuration of the moving picture encoding device 11 according to this embodiment will be described. Fig.16 1 is a block diagram showing the structure of the video coding device 11 of the present embodiment. The video coding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a transformation / quantization unit 103, an inverse quantization / inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference picture storage unit, frame memory) 109, a coding parameter determination unit 110, a parameter coding unit 111, a prediction parameter derivation unit 120, and an entropy coding unit 104.

[0234] The predicted image generation unit 101 generates a predicted image for each CU. The predicted image generation unit 101 includes the inter-prediction image generation unit 309 and the intra-prediction image generation unit which have been described, and the description thereof will be omitted.

[0235] The subtraction unit 102 generates a prediction error by subtracting the pixel value of the predicted image of the block input from the predicted image generation unit 101 from the pixel value of the image T. The subtraction unit 102 outputs the prediction error to the transformation / quantization unit 103 .

[0236] The transform / quantization unit 103 calculates transform coefficients by frequency transforming the prediction error input from the subtraction unit 102 , and derives quantized transform coefficients by quantization. The transform / quantization unit 103 outputs the quantized transform coefficients to the parameter encoding unit 111 and the inverse quantization / inverse transform unit 105 .

[0237] The transform / quantization unit 103 includes a separate transform unit (first transform unit), a non-separable transform unit (second transform unit), and a scaling unit.

[0238] The separation transform unit applies separation transform to the prediction error, and the scaling unit performs scaling on the transform coefficient based on the quantization matrix.

[0239] The inverse quantization / inverse transformation unit 105 is the same as the inverse quantization / inverse transformation unit 311 in the moving picture decoding device 31 , and a description thereof is omitted. The calculated prediction error is output to the addition unit 106 .

[0240] The parameter coding unit 111 includes a header coding unit 1110, a CT information coding unit 1111, and a CU coding unit 1112 (prediction mode coding unit). The CU coding unit 1112 further includes a TU coding unit 1114. The following describes the schematic operation of each module.

[0241] The header coding unit 1110 performs coding processing on parameters such as header information, partition information, prediction information, and quantized transform coefficients.

[0242] The CT information encoding unit 1111 encodes QT, MT (BT, TT) division information and the like.

[0243] The CU encoding unit 1112 encodes CU information, prediction information, partition information, and the like.

[0244] When a prediction error is included in a TU, the TU encoding unit 1114 encodes the QP update information and the quantized prediction error.

[0245] The CT information encoding unit 1111 and the CU encoding unit 1112 supply syntax elements such as inter prediction parameters (predMode, general_merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX), intra prediction parameters, and quantized transform coefficients to the parameter encoding unit 111 .

[0246] The entropy coding unit 104 receives the quantized transform coefficients and coding parameters (division information, prediction parameters) from the parameter coding unit 111. The entropy coding unit 104 performs entropy coding on these, and generates and outputs a coded stream Te.

[0247] The prediction parameter derivation unit 120 includes an inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit, and derives intra prediction parameters and inter prediction parameters based on the parameters input from the encoding parameter determination unit 110. The derived intra prediction parameters and inter prediction parameters are output to the parameter encoding unit 111. (Configuration of Inter-frame Prediction Parameter Encoding Unit)

[0248] like Fig.17 As shown, the inter-frame prediction parameter coding unit 112 includes a parameter coding control unit 1121 and an inter-frame prediction parameter derivation unit 303. The inter-frame prediction parameter derivation unit 303 is a common structure with the moving picture decoding device. The parameter coding control unit 1121 includes a merge index derivation unit 11211 and a vector candidate index derivation unit 11212.

[0249] The merge index derivation unit 11211 derives merge candidates and the like, and outputs them to the inter prediction parameter derivation unit 303. The vector candidate index derivation unit 11212 derives prediction vector candidates and the like, and outputs them to the inter prediction parameter derivation unit 303 and the parameter encoding unit 111. (Configuration of Intra-frame Prediction Parameter Coding Unit)

[0250] The intra prediction parameter coding unit includes a parameter coding control unit and an intra prediction parameter derivation unit. The intra prediction parameter derivation unit is a common configuration of the moving picture decoding apparatus.

[0251] However, unlike the moving picture decoding device, the inputs to the inter-frame prediction parameter derivation unit 303 and the intra-frame prediction parameter derivation unit are the encoding parameter determination unit 110 and the prediction parameter memory 108 , and the outputs are to the parameter encoding unit 111 .

[0252] The adder 106 generates a decoded image by adding the pixel value of the prediction block input from the prediction image generator 101 and the prediction error input from the inverse quantization / inverse transform unit 105 for each pixel. The adder 106 stores the generated decoded image in the reference picture memory 109.

[0253] The loop filter 107 applies a deblocking filter, SAO, and ALF to the decoded image generated by the adding unit 106. It should be noted that the loop filter 107 does not necessarily include the above three filters, and may include only a deblocking filter, for example.

[0254] The prediction parameter memory 108 stores the prediction parameter generated by the encoding parameter determination unit 110 in a predetermined position for each target picture and CU.

[0255] The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each target picture and CU.

[0256] The coding parameter determination unit 110 selects one of a plurality of sets of coding parameters. The coding parameters refer to the QT, BT or TT partition information, prediction parameters or parameters generated in association with them as the coding target. The predicted image generation unit 101 generates a predicted image using these coding parameters.

[0257] The coding parameter determination unit 110 calculates the RD cost value representing the size of the information amount and the coding error for each of the multiple sets. The RD cost value is, for example, the sum of the code amount and the value obtained by multiplying the square error by the coefficient λ. The code amount is the amount of information of the coded stream Te obtained by entropy encoding the quantization error and the coding parameter. The square error is the sum of the squares of the prediction error calculated in the subtraction unit 102. The coefficient λ is a real number greater than a preset zero. The coding parameter determination unit 110 selects a set of coding parameters with the smallest calculated cost value. The coding parameter determination unit 110 outputs the determined coding parameters to the parameter encoding unit 111 and the prediction parameter derivation unit 120.

[0258] It should be noted that a part of the moving picture encoding device 11 and the moving picture decoding device 31 in the above-mentioned embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generation unit 308, the inverse quantization / inverse transformation unit 311, the addition unit 312, the prediction parameter derivation unit 320, the predicted image generation unit 101, the subtraction unit 102, the transformation / quantization unit 103, the entropy encoding unit 104, the inverse quantization / inverse transformation unit 105, the loop filter 107, the coding parameter determination unit 110, the parameter encoding unit 111 and the prediction parameter derivation unit 120 can be implemented by a computer. In this case, the program for realizing the control function can be recorded in a computer-readable recording medium, and the computer system can read the program recorded in the recording medium and execute it. It should be noted that the "computer system" mentioned here refers to a computer system built into any one of the moving picture encoding device 11 and the moving picture decoding device 31, and a computer system including hardware such as an OS and peripheral devices is used. In addition, "computer-readable recording medium" refers to removable media such as floppy disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into computer systems. Moreover, "computer-readable recording medium" may also include: recording media that dynamically store programs for a short period of time, such as communication lines when sending programs via networks such as the Internet or communication lines such as telephone lines; and recording media that store programs for a fixed period of time, such as volatile memories inside computer systems that serve as servers or clients in this case. In addition, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, or a program that can realize the above-mentioned functions by combining with programs already recorded in the computer system.

[0259] In addition, part or all of the moving picture encoding device 11 and the moving picture decoding device 31 in the above-mentioned embodiment can also be implemented as an integrated circuit such as LSI (Large Scale Integration). Each functional block of the moving picture encoding device 11 and the moving picture decoding device 31 can be individually processor-based, or part or all of them can be integrated to be processor-based. In addition, the method of integrated circuitization is not limited to LSI, and it can also be implemented by a dedicated circuit or a general-purpose processor. In addition, when a technology for integrated circuitization that replaces LSI appears with the advancement of semiconductor technology, an integrated circuit based on this technology can also be used.

[0260] As mentioned above, one embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to the above embodiment, and various design changes and the like can be made within the scope not departing from the gist of the present invention.

[0261] The embodiments of the present invention are not limited to the above-mentioned embodiments, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical solutions appropriately modified within the scope of the claims are also included in the technical scope of the present invention. Industrial Applicability

[0262] The embodiments of the present invention can be preferably applied to a motion picture decoding device that decodes coded data obtained by encoding image data, and a motion picture encoding device that generates coded data obtained by encoding image data. In addition, it can be appropriately applied to the data structure of the coded data generated by the motion picture encoding device and referred to by the motion picture decoding device. Description of Reference Numerals 31: Motion picture decoding device 301: Entropy decoding unit 302: Parameter decoding unit 3022: CU decoding unit 3024: TU decoding unit 303: Inter-frame prediction parameter derivation unit 30376: MMVD Forecasting Department 305, 107: Loop filter 306, 109: Reference image memory 307, 108: Prediction parameter memory 308, 101: Prediction image generation unit 309: Inter-frame prediction image generation unit 311, 105: Inverse quantization / inverse transformation unit 312, 106: Addition Department 320: Prediction parameter derivation department 11: Motion picture encoding device 102: Subtraction Department 103: Transformation / Quantization Unit 104: Entropy coding unit 110: Coding parameter determination unit 111: Parameter encoding unit 112: Inter-frame prediction parameter encoding unit 120: Prediction parameter derivation department

Claims

1. A moving picture decoding device, comprising: an MMVD prediction unit for obtaining a motion vector by adding a differential vector of a predetermined distance and a predetermined direction to a predicted motion vector of a target block; and a parameter decoding unit for decoding, from coded data, an index for specifying the differential vector from an MMVD candidate list, wherein: The MMVD prediction unit is characterized in that a differential vector of a specific distance and direction is derived from the specified distance and the specified direction, a template matching cost in the differential vector is derived, and a search is performed thereby, and a differential vector candidate is inserted according to the cost, thereby deriving an MMVD candidate list. In the search, a subset DIR1 of the first direction is searched only within a predetermined distance, and a subset DIR2 of the second direction is searched only within a distance other than the predetermined distance.

2. The moving picture decoding device according to claim 1, characterized in that The first direction subset DIR1 includes eight directions: horizontal, vertical, and 45-degree inclined directions.

3. The moving picture decoding device according to claim 1, characterized in that The second direction subset DIR2 includes a total of three directions, namely, the direction having the minimum cost among the template matching costs providing the number of searches obtained in the previous distance and directions adjacent thereto.

4. The moving picture decoding device according to claim 1, characterized in that: The second direction subset DIR2 includes two directions with the minimum cost among the template matching costs of the search number obtained in the previous distance and a maximum of six directions adjacent to the two directions.

5. The moving picture decoding device according to claim 1, characterized in that: In the search, the directions included in the subset DIR2 of the second directions are searched for each distance candidate, and the directions included in the subset DIR2 of the second directions are updated for each distance candidate.

6. A moving picture decoding device, comprising: an MMVD prediction unit for obtaining a motion vector by adding a differential vector of a predetermined distance and a predetermined direction to a predicted motion vector of a target block; and a parameter decoding unit for decoding, from coded data, an index for specifying the differential vector from an MMVD candidate list, wherein: The MMVD prediction unit is characterized in that a differential vector of a specific distance and direction is derived from the specified distance and the specified direction, a template matching cost in the differential vector is derived, and a search is performed thereby, and a differential vector candidate is inserted according to the cost, thereby deriving an MMVD candidate list. In the search, a subset DIR1 of the first direction is searched only in distances included in a prescribed distance set, and a subset DIR2 of the second direction is searched only in distances included in a distance set other than the prescribed distance set.

7. The moving picture decoding device according to claim 6, characterized in that: In the search, directions included in the subset DIR2 of second directions are searched for distance sets including a plurality of distance candidates, and directions included in the subset DIR2 of second directions are updated for each distance set including a plurality of distance candidates.

8. The moving picture decoding device according to claim 7, characterized in that: In the search, the number of directions included in the subset DIR2 of second directions differs depending on the distance set.

Images