Video encoding device, video encoding method, and video encoding program

Abstract

To provide a technique that improves encoding efficiency by performing block division suitable for image encoding and decoding.SOLUTION: A video encoding device includes a merging candidate list construction unit that constructs a merging candidate list including a spatial merging candidate, a normal merging candidate selection unit that selects a uni-prediction or bi-prediction normal merging candidate from the merging candidate list, and a triangle merging candidate selection unit that selects a first triangular merging candidate that is a uni-prediction and a second triangular merging candidate that is a uni-prediction from the merging candidate list, and the triangle merging candidate selection unit makes a priority order of motion information equal for a first triangle merging candidate and a second triangle merging candidate.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image encoding and decoding technique for dividing an image into blocks and performing prediction. [Background technology]

[0002] In image encoding and decoding, the image to be processed is divided into blocks, which are groups of a predetermined number of pixels. Divide into appropriate blocks and process in blocks. By appropriately setting the inter-frame prediction and frame prediction, the coding efficiency is improved. do.

[0003] In video coding and decoding, inter-prediction is used to predict from previously coded or decoded pictures. In Patent Document 1, affine transformation is used in inter prediction, which improves coding efficiency. The technology to be applied is described. In moving images, objects undergo transformations such as enlargement, reduction, and rotation. It is not uncommon for this to happen, and by applying the technology of Patent Document 1, efficient coding This becomes possible. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 9-172644 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology in Patent Document 1 involves image conversion, which places a heavy processing load on the user. In view of the above, the present invention provides an efficient encoding technique with a low load. . [Means for solving the problem]

[0006] In one aspect of the present invention that solves the above problem, a merge candidate list including spatial merge candidates is a merge candidate list construction unit that constructs a merge candidate list to be uni-predictive or bi-predictive from the merge candidate list; a normal merging candidate selection unit for selecting a normal merging candidate; and A triangle merger that selects a first triangle merge candidate that is uni-predictive and a second triangle merge candidate that is uni-predictive. a candidate selection unit, the triangle merge mode selection unit being configured to select the first triangle merge candidate and a previous The second triangle merging candidates have the same priority order of motion information. Effect of the Invention

[0007] According to the present invention, highly efficient image encoding / decoding processing can be realized with a low load. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram of an image encoding device according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram of an image decoding device according to an embodiment of the present invention. [Diagram 3] 13 is a flowchart for explaining an operation of dividing a tree block. [Figure 4] FIG. 13 is a diagram showing how an input image is divided into tree blocks. [Diagram 5] FIG. 1 is a diagram illustrating a z-scan. [Figure 6] FIG. 13 is a diagram showing the divided shapes of blocks. [Figure 7] 13 is a flowchart for explaining an operation of dividing a block into four. [Figure 8] 13 is a flowchart for explaining an operation of dividing a block into two or three. [Figure 9] This is a syntax for expressing the shape of block division. [Figure 10] FIG. 1 is a diagram illustrating intra prediction. [Figure 11] FIG. 13 is a diagram for explaining reference blocks for inter prediction. [Figure 12] This is a syntax for expressing a coding block prediction mode. [Figure 13] FIG. 11 is a diagram showing the correspondence between syntax elements and modes relating to inter prediction. [Figure 14] FIG. 13 is a diagram for explaining affine transformation motion compensation of two control points. [Figure 15] FIG. 13 is a diagram for explaining affine transformation motion compensation of three control points. [Figure 16] FIG. 2 is a block diagram showing a detailed configuration of the inter prediction unit 102 in FIG. [Figure 17] 17 is a block diagram showing a detailed configuration of a normal predicted motion vector mode derivation unit 301 in FIG. 16. [Figure 18] 17 is a block diagram showing a detailed configuration of a normal merge mode derivation unit 302 in FIG. 16. [Figure 19] 17 is a flowchart for explaining a normal predicted motion vector mode derivation process of the normal predicted motion vector mode derivation unit 301 of FIG. 16. [Figure 20] 13 is a flowchart showing a processing procedure of a normal predicted motion vector mode derivation process. [Figure 21] 13 is a flowchart illustrating a procedure for a normal merge mode derivation process. [Figure 22] FIG. 3 is a block diagram showing a detailed configuration of an inter prediction unit 203 in FIG. 2. [Diagram 23] 23 is a block diagram showing a detailed configuration of a normal predicted motion vector mode derivation unit 401 in FIG. 22. [Figure 24] 23 is a block diagram showing a detailed configuration of a normal merge mode derivation unit 402 in FIG. 22. [Diagram 25] 23 is a flowchart for explaining a normal predicted motion vector mode derivation process of the normal predicted motion vector mode derivation unit 401 in FIG. 22. [Figure 26]17 is a block diagram showing a detailed configuration of a sub-block predictive motion vector mode derivation unit 303 in FIG. 16. [Figure 27] 23 is a block diagram showing a detailed configuration of a sub-block predictive motion vector mode derivation unit 403 in FIG. 22. [Figure 28] 17 is a block diagram showing a detailed configuration of a sub-block merge mode derivation unit 304 in FIG. 16. [Figure 29] 23 is a block diagram showing a detailed configuration of a sub-block merge mode derivation unit 404 in FIG. 22. [Diagram 30] FIG. 13 is a diagram for explaining derivation of affine inheritance predicted motion vector candidates. [Diagram 31] FIG. 13 is a diagram for explaining derivation of affine constructed predicted motion vector candidates. [Diagram 32] FIG. 13 is a diagram for explaining affine inheritance merge candidate derivation. [Diagram 33] FIG. 13 is a diagram illustrating affine construction merge candidate derivation. [Diagram 34] 13 is a flowchart of deriving an affine inheritance predicted motion vector candidate. [Diagram 35] 13 is a flowchart of deriving an affine constructed predicted motion vector candidate. [Diagram 36] 13 is a flowchart of affine inheritance merge candidate derivation. [Figure 37] 13 is a flowchart of affine construct merge candidate derivation. [Figure 38] 13 is a flowchart illustrating a procedure of a history motion vector predictor candidate list initialization / update process. [Figure 39] 13 is a flowchart of a procedure of a same element confirmation process in the procedure of a process of deriving a history predicted motion vector candidate. [Diagram 40] 13 is a flowchart of an element shifting process procedure in the history motion vector predictor candidate derivation process procedure. [Diagram 41] 13 is a flowchart illustrating a procedure of a process of deriving a historical predicted motion vector candidate. [Diagram 42] 13 is a flowchart illustrating a history merging candidate derivation process procedure. [Diagram 43] 11 is a diagram illustrating an example of a history motion vector predictor candidate list update process. FIG. [Diagram 44] 13 is a flowchart for explaining the operation of the sub-block temporal merge candidate derivation unit 381. [Diagram 45] 13 is a flowchart illustrating a process of deriving adjacent motion information of a block. [Figure 46] 13 is a flowchart illustrating a process of deriving a temporal motion vector. [Figure 47] 13 is a flowchart illustrating derivation of inter prediction information. [Figure 48] 13 is a flowchart illustrating a process of deriving sub-block motion information. [Figure 49] FIG. 2 is a diagram for explaining the temporal relationship between pictures. [Figure 50] 13 is a flowchart for explaining a process of deriving a temporal motion vector predictor candidate in a normal motion vector predictor mode derivation unit 301. [Figure 51] 13 is a flowchart for explaining the derivation process of ColPic in the derivation process of a temporal motion vector predictor candidate in the normal motion vector predictor mode derivation unit 301. [Figure 52] 13 is a flowchart for explaining a process of deriving coding information of ColPic in a process of derivation of a temporal motion vector predictor candidate in the normal motion vector predictor mode derivation unit 301. [Diagram 53] 13 is a flowchart illustrating a process of deriving inter prediction information. [Figure 54] 13 is a flowchart showing the procedure of a process of deriving inter prediction information of a coding block when the inter prediction mode of the coding block colCb is bi-prediction (Pred_BI). [Figure 55] 13 is a flowchart illustrating a procedure of a motion vector scaling calculation process. [Figure 56] 13 is a flowchart illustrating a process of deriving a temporal merge candidate. [Figure 57] FIG. 13 is a diagram for explaining motion compensation prediction in the case of L0 prediction in which an L0 reference picture (RefL0Pic) is located at a time earlier than a current picture (CurPic) to be processed. [Figure 58] 13 is a diagram for explaining motion compensation prediction in the case where L0 prediction is performed and a reference picture for L0 prediction is located at a later time than the current picture to be processed. FIG. [Figure 59] FIG. 13 is a diagram for explaining motion compensation prediction in a bi-predictive case where a reference picture for L0 prediction is located at a time earlier than a current picture to be processed and a reference picture for L1 prediction is located at a time later than the current picture to be processed. [Figure 60] 13 is a diagram for explaining the prediction direction of motion compensation prediction in bi-prediction where the reference picture for L0 prediction and the reference picture for L1 prediction are located at a time earlier than the current picture to be processed. FIG. [Figure 61] 13 is a diagram for explaining the prediction direction of motion compensation prediction in bi-prediction where the reference picture for L0 prediction and the reference picture for L1 prediction are at a time later than the current picture. FIG. [Figure 62] 13 is a flowchart illustrating an average merging candidate derivation process; [Figure 63] 11 is a table showing information regarding merge motion vector difference. [Figure 64] FIG. 13 is a diagram illustrating the derivation of a merge differential motion vector. [Figure 65] FIG. 13 is a diagram illustrating a prediction of a triangle merge mode. [Figure 66] 13 is a flowchart illustrating derivation of triangle merge candidates. [Figure 67] 13 is a flowchart illustrating the derivation of uni-prediction motion information for merged triangular partitions. [Figure 68] FIG. 1 is a diagram illustrating a merging triangular index and a merging triangular partition. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The technologies and technical terms used in this embodiment will be defined below.

[0010] <Tree Block> In this embodiment, the image to be encoded / decoded is divided equally into a predetermined size. As shown in FIG. 4, in this embodiment, the subtree block is The size of the tree block is set to 128x128 pixels. Instead, any size can be set. , which corresponds to the decoding target in the decoding process. That is, the order is from left to right and from top to bottom. The inside of each tree block is further reordered. Recursive division is possible. Blocks to be coded and decoded after tree block division We define a coding block as a tree block. We also refer to the tree block and coding block as a block. By dividing the image into blocks appropriately, efficient coding is possible. The size of the block can be a fixed value pre-agreed between the encoding device and the decoding device. The size of the tree block determined by the encoding device is transmitted to the decoding device. It can also be taken.

[0011] <Prediction mode> The processed image (coded block in the coding process) is A decoded image, image signal, tree block, block, coding block, etc. In the decoding process, the decoded image, image signal, tree block, block, etc. are used. Intra prediction (MO) is used to predict the surrounding image signals. DE_INTRA), and inter prediction (MODE_INTER) which predicts from the image signal of the processed image. Switch between intra prediction (MODE_INTRA) and inter prediction (MODE_INTER) The prediction mode (PredMode) is defined as an intra prediction mode ( It has the values ​​MODE_INTRA, or inter prediction (MODE_INTER).

[0012] <Inter Prediction> Inter-prediction, which predicts from the image signal of a processed image, refers to multiple processed images. It can be used as a reference picture. To manage multiple reference pictures, L0 (reference link) Two types of reference lists, L0 (reference list 0) and L1 (reference list 1), are defined, and the reference indexes are The L0 prediction (Pred_L0) is available for P slices. For B slices, L0 prediction (Pred_L0), L1 prediction (Pred_L1), and bi-prediction (Pred_BI ) is available. L0 prediction (Pred_L0) refers to the reference picture managed by L0. Inter prediction is used to predict the frame rate of a frame, and L1 prediction (Pred_L1) uses a reference picture managed by L1. This is the inter prediction to refer to. Bi-prediction (Pred_BI) is performed with both L0 prediction and L1 prediction, Inter prediction refers to one reference picture managed in L0 and one in L1. Information specifying L0 prediction, L1 prediction, and bi-prediction is defined as an inter prediction mode. In the following processing, the constants and variables with the subscript LX in the output are L0, L1 It is assumed that processing is performed for each

[0013] <Predictive motion vector mode> The predicted motion vector mode is an index for specifying a predicted motion vector, Transmits the inter prediction vector, inter prediction mode, and reference index, and the index of the block to be processed. The predicted motion vector is determined based on the motion vector of the block adjacent to the current block. A processed block that belongs to the same image as the target block, or a block that belongs to the processed image and has the same position as the target block A candidate predicted motion vector is derived from a block located at or near the block. The measured motion vector is derived from an index for identifying the measured motion vector.

[0014] <Merge mode> In the merge mode, the difference motion vector and reference index are not transmitted, and the target block is merged. The processed blocks adjacent to the block to be processed or blocks belonging to the processed image The inter prediction information of the block located at the same position as the block or in its vicinity (neighborhood) is used for processing. This is a mode for deriving inter prediction information for the current block.

[0015] The processed blocks adjacent to the block to be processed and the inter-blocks of the processed blocks The spatial merge candidate is defined as a block belonging to the processed image. Blocks that are in the same location as the block or in its vicinity (neighborhood), and the index of that block Inter prediction information derived from the super prediction information is defined as a temporal merge candidate. The complement is registered in the merge candidate list, and the merge index is used to predict the block to be processed. Identify which merge candidates to use.

[0016] <Adjacent block> FIG. 11 shows the procedure for deriving inter prediction information in the predicted motion vector mode and the merge mode. A0, A1, A2, B0, B1, B2, B3 is a processed block adjacent to the block to be processed. T0 is a processed image. The block to be processed is located at the same position as the coding block to be processed in the image to be processed or in its vicinity (nearby It is a block located next to

[0017] A1 and A2 are located to the left of the coding block to be processed and are adjacent to the coding block to be processed. B1 and B3 are adjacent blocks located above the coding block to be processed. A0, B0, and B2 are the blocks adjacent to the target coding block. These are the blocks located at the bottom left, top right, and top left of the encryption block.

[0018] Details of how adjacent blocks are treated in predictive motion vector mode and merge mode The details will be described later.

[0019] <Affine transformation motion compensation> In the affine transformation motion compensation, a coding block is divided into sub-blocks of a predetermined unit, and each sub-block is The motion vector is set for each subblock and motion compensation is performed. The motion vector of the block is determined by the processed block adjacent to the current block or the processed block. A block belonging to the image that is located at the same position as the processing target block or in its vicinity (neighborhood) The control points are derived based on one or more control points derived from the inter prediction information of the block. In this embodiment, the size of the subblock is 4x4 pixels. This is not a limitation, and the motion vector may be derived on a pixel-by-pixel basis.

[0020] FIG. 14 shows an example of affine transformation motion compensation for two control points. Since the control point has two parameters, a horizontal component and a vertical component, the control point has two The affine transformation in this case is called a four-parameter affine transformation. are the control points. Figure 15 shows an example of affine transformation motion compensation when there are three control points. In the case of , the three control points have two parameters, a horizontal component and a vertical component, so An affine transformation with three control points is called a six-parameter affine transformation. CP1, CP2, and CP3 are control points.

[0021] Affine transformation motion compensation is available in both the predicted motion vector mode and the merge mode. It is also available in the predictive motion vector mode. The mode in which the affine transformation is performed is defined as the subblock prediction motion vector mode, and the merge mode is A mode in which motion compensation is applied is defined as a sub-block merging mode.

[0022] <Syntax of coding block> Using Fig. 12(a), Fig. 12(b), and Fig. 13, the prediction mode of the coding block is The syntax for expressing the above is explained below. The pred_mode_flag in FIG. 12(a) indicates inter-prediction. If pred_mode_flag is 0, it is inter prediction, and if pred_mode_flag is 1, it is inter prediction. If de_flag is 1, it is intra prediction. In the case of intra prediction, the intra prediction information i In the case of inter prediction, send ntra_pred_mode and in the case of inter prediction, send merge_flag. This is a flag indicating whether the motion vector mode is the predicted motion vector mode or the predicted motion vector mode. In case of tole mode (merge_flag=0), whether to apply sub-block prediction motion vector mode A flag inter_affine_flag is sent to indicate whether the sub-block prediction motion vector mode is applied. If you use affine (inter_affine_flag=1), send cu_affine_type_flag. g is a flag for determining the number of control points in the sub-block prediction motion vector mode. It is gu.

[0023] On the other hand, in the case of merge mode (merge_flag=1), the merge_subblock_flag in Fig. 12(b) is merge_subblock_flag is a flag that indicates whether to apply the subblock merge mode. In the case of subblock merge mode (merge_subblock_flag=1), the merge index If the subblock merge mode is not selected (merge_subblock_idx), subblock_flag=0), flag merge_triangle_fl indicating whether to apply triangle merge mode ag. If triangle merge mode is applied (merge_triangle_flag=1), divide the block. The direction of the split, merge_triangle_split_dir, and the merge On the other hand, the merge triangle indexes merge_triangle_idx0 and merge_triangle_idx1 are sent. If merge mode is not applied (merge_triangle_flag=0), the merge index is merge_idx. Send.

[0024] Figure 13 shows the values ​​of each syntax element and the corresponding prediction modes. flag=0,inter_affine_flag=0 corresponds to normal predicted motion vector mode (Inter Pred Mode) merge_flag=0,inter_affine_flag=1 corresponds to the sub-block predicted motion vector mode ( Inter Affine Mode). merge_flag=1,merge_subblock_flag=0,merge_trianlge _flag=0 corresponds to the normal merge mode. merge_flag=1,merge_subblock _flag=0,merge_trianlge_flag=1 corresponds to Triangle Merge Mode. merge_flag=1,merge_subblock_flag=1 is the subblock merge mode (Affine Merge Mode).

[0025] <poc> POC (Picture Order Count) is a variable associated with the picture to be coded, The value is set to a value that increases by 1 in the picture output order. It can also determine whether a picture is a frame or not, determine the order of pictures in the output order, and For example, we can calculate the distance between the POCs of two pictures. If the POC of two pictures is different, they are considered to be the same picture. The picture with the smaller POC value is the one that will be output first. The difference in POC between two pictures indicates the distance between the pictures along the time axis.

[0026] (First embodiment) Regarding the image encoding device 100 and the image decoding device 200 according to the first embodiment of the present invention, He explains.

[0027] FIG. 1 is a block diagram of an image coding device 100 according to a first embodiment. The video encoding device in this embodiment includes an image encoding device 100, a block division unit 101, and an inter prediction unit. unit 102, intra prediction unit 103, decoded image memory 104, prediction method determination unit 105, residual A signal generating unit 106, an orthogonal transform and quantization unit 107, a bit string encoding unit 108, and a dequantization and dequantization unit 109. The image processing apparatus includes an orthogonal transform unit 109, a decoded image signal superimposing unit 110, and an encoded information storage memory 111. can.

[0028] The block division unit 101 recursively divides an input image to generate coding blocks. The block division unit 101 divides the blocks to be divided into horizontal and vertical blocks. The quadrants to be divided and whether the block to be divided is to be divided horizontally or vertically. The generated image signal of the coding block to be processed is divided into two parts. The signal is supplied to the prediction unit 102, the intra prediction unit 103, and the residual signal generation unit 106. The block division unit 10 supplies information indicating the recursive division structure to the bit string encoding unit 108. The detailed operation of 1 will be described later.

[0029] The inter prediction unit 102 performs inter prediction on the coding block to be processed. Inter prediction information stored in the storage memory, A plurality of inter-prediction information candidates are derived from a decoded image signal, and an appropriate one is selected from the plurality of candidates. and selects the selected inter prediction mode and the selected inter prediction mode. The prediction method determination unit 105 supplies a prediction image signal corresponding to the inter prediction mode. The detailed configuration and operation of unit 102 will be described later.

[0030] The intra prediction unit 103 performs intra prediction on the coding block to be processed. A predicted image signal is generated by intra-prediction from the decoded image signal stored in the memory 104. a plurality of intra-prediction modes, and a suitable intra-prediction mode is selected from the plurality of intra-prediction modes. and a predicted image signal according to the selected intra-prediction mode. The prediction method is then supplied to the measurement method determination unit 105. FIG. 10 shows an example of intra prediction. The figure shows the correspondence between the prediction direction of intra prediction and the intra prediction mode number. Intra prediction mode 50 produces an intra predicted image by copying pixels vertically. Intra prediction mode 1 is a DC mode, in which all pixels of the processing target block are Intra prediction mode 0 is the Planar mode. This is a mode that creates a two-dimensional intra-prediction image from vertical and horizontal reference pixels. FIG. 10B shows a process for generating an intra-prediction image in the case of intra-prediction mode 40. For each pixel in the block to be processed, a reference image in the direction indicated by the intra prediction mode is If the reference pixel in the intra prediction mode is not an integer position, the surrounding The reference pixel value is determined by interpolation from the reference pixel values ​​at integer positions.

[0031] The decoded image memory 104 stores the decoded image generated by the decoded image signal superimposing unit 110 . The decoded image stored in the decoded image memory is predicted by the inter prediction unit 102 and the intra prediction unit 103. Supply to 03.

[0032] The prediction method determination unit 105 determines the amount of coding information and residual signal code, the predicted image, and The optimum prediction mode is determined by evaluating the distortion between the image signal and the image signal to be processed. In case of merge mode of inter prediction, The merge index, information indicating whether or not the sub-block merge mode is selected (sub-block merge The coding information of the prediction motion flag of the inter prediction is supplied to the bit string coding unit 108. In case of vector mode, inter prediction mode, predicted motion vector index, L0, L 1 reference index, differential motion vector, information indicating whether it is a sub-block mode (sub Block prediction motion vector flag) and other coding information are supplied to the bit stream coding unit 108. The determined coding information is supplied to the coding information storage memory 111.

[0033] The residual signal generating unit 106 subtracts the predicted image signal from the image signal to be processed to generate a residual signal. A difference signal is generated and supplied to the orthogonal transformation and quantization unit 107 .

[0034] The orthogonal transform and quantization unit 107 performs orthogonal transform and quantization on the residual signal in accordance with the quantization parameter. The orthogonal transform and quantization are performed to generate a residual signal, which is then inversely transmitted to the bit stream coding unit 108. The resulting signal is supplied to the quantization and inverse orthogonal transformation unit 109.

[0035] The bitstream coding unit 108 encodes the bitstream in units of sequences, pictures, slices, and coding blocks. In addition to the information, the prediction method determined by the prediction method determination unit 105 for each coding block is Specifically, the prediction mode PredMode for each coding block, In the case of partition mode PartMode and inter prediction (PRED_INTER), it is necessary to determine whether or not it is merge mode. flag to merge subblocks, subblock merge flag, merge index if merge mode, If not in inter prediction mode, the predicted motion vector index, differential motion vector, The coding information such as vector information, sub-block predicted motion vector flags, etc. are The bit sequence is encoded according to a predetermined syntax rule to generate a first encoded bit sequence. The bit stream coding unit 108 converts the orthogonally transformed and quantized residual signal into a bit stream in accordance with a prescribed syntax rule. Therefore, the first encoded bit string is entropy-encoded to generate a second encoded bit string. and the second coded bitstream are multiplexed according to a prescribed syntax rule to generate a bitstream Output.

[0036] The inverse quantization and inverse orthogonal transformation unit 109 performs the orthogonal transformation supplied from the orthogonal transformation and quantization unit 107. The quantized residual signal is inversely quantized and inversely orthogonally transformed to calculate the residual signal, and the decoded image signal is The signal is supplied to the superimposing unit 110.

[0037] The decoded image signal superimposing unit 110 outputs a predicted image signal according to the determination by the prediction method determining unit 105. The inverse quantization and inverse orthogonal transformation unit 109 superimposes and decodes the residual signal that has been inverse quantized and inverse orthogonal transformed. The decoded image is generated and stored in the decoded image memory 104. After performing filtering to reduce block distortion and other distortions, the decoded image memory 104 may be stored in

[0038] The coding information storage memory 111 stores the prediction mode (inference mode) determined by the prediction method determination unit 105. The coding information storage memory 111 stores coding information such as sub-prediction or intra-prediction. The coding information stored in the 3D prediction block includes a determined motion vector, a reference list, and In addition to the reference index, in case of inter prediction merge mode, the merge index, Encoding information of information indicating whether or not the sub-block merge mode is selected (sub-block merge flag) In the case of the predicted motion vector mode of the inter prediction, the inter prediction mode, the predicted motion vector L0 and L1 reference indices, differential motion vectors, sub-block motion vectors, In the case of intra prediction, The determined intra-prediction mode, etc. The construction of the complement list is described below.

[0039] FIG. 2 shows a configuration of a video decoding device according to an embodiment of the present invention, which corresponds to the video encoding device shown in FIG. 2 is a block diagram showing the configuration of the video decoding device according to the embodiment. A block division unit 202, an inter prediction unit 203, an intra prediction unit 204, and a coding information storage memory The memory 205, the inverse quantization and inverse orthogonal transformation unit 206, the decoded image signal superimposition unit 207, and the decoded image signal superimposition unit 208 are connected to the inverse quantization and inverse orthogonal transformation unit 206. The image memory 208 is provided.

[0040] The decoding process of the video decoding device of FIG. 2 is provided inside the video encoding device of FIG. Since this corresponds to the decoding process, the coding information storage memory 205 in FIG. The orthogonal transform unit 206, the decoded image signal superimposing unit 207, and the decoded image memory 208 are configured as follows: , the inverse quantization and inverse orthogonal transformation unit 109 and the decoded image signal superimposition unit 110 of the video encoding device of FIG. , the encoded information storage memory 111, and the decoded image memory 104. It has the function of

[0041] The bit stream supplied to the bit stream decoder 201 follows the rules of the prescribed syntax. The separated first coded bit stream is decoded to obtain a sequence, a picture, a slice, and a The information on the coding block unit and the coding information on the coding block unit are obtained. In general, prediction is done by inter prediction (PRED_INTER) or intra prediction (PRED_INTRA) on a per coding block basis. Prediction mode PredMode, partition mode PartMode, inter prediction (PRED_INTER) If , a flag to determine whether or not it is in merge mode, if it is, the merge index subblock merge flag, and if the predicted motion vector mode is inter prediction mode , predicted motion vector index, differential motion vector, sub-block predicted motion vector The encoded information regarding flags, etc. is decoded according to the syntax rules described below, and encoded The information is sent to the inter prediction unit 203 or the intra prediction unit 204, and to the coding information storage memory The separated second coded bit string is decoded and the orthogonally transformed and quantized residual is supplied to 205. A difference signal is calculated, and the orthogonally transformed and quantized residual signal is provided to an inverse quantization and inverse orthogonal transformation unit 206. Provide.

[0042] The inter prediction unit 203 detects that the prediction mode PredMode of the coding block to be processed is inter When the prediction (PRED_INTER) is in the predicted motion vector mode, the coding information storage memory 205 A plurality of predictive motion vectors are generated by using the coding information of the already decoded image signal stored in A candidate motion vector is derived and registered in a motion vector predictor candidate list described later. The bitstream decoding unit 201 selects a motion vector predictor candidate from among the multiple motion vector predictor candidates registered in the supplementary list. Selecting a predicted motion vector according to the decoded and supplied predicted motion vector index; The bitstream decoder 201 decodes the difference vector and the selected predicted motion vector to generate a motion vector. The vector is calculated and stored in the coded information storage memory 205 together with other coded information. The coding information of the coding block to be supplied and stored is the prediction mode PredMode, the partition mode Pa Flags indicating whether to use rtMode, L0 prediction, and L1 prediction: predFlagL0[xP][yP], pr edFlagL1[xP][yP], L0 and L1 reference indices refIdxL0[xP][yP], refIdxL1[xP][yP] , L0, L1 motion vectors mvL0[xP][yP], mvL1[xP][yP], etc., where xP and yP are the pixel The prediction mode is an index that indicates the position of the top left pixel of the coding block in the block. PredMode is inter prediction (MODE_INTER) and inter prediction mode is L0 prediction (Pred_L0) In this case, the flag predFlagL0 indicating whether to use L0 prediction is 1, and whether to use L1 prediction is 2. The flag predFlagL1 indicating whether the inter prediction mode is L1 prediction (Pred_L1) is set to 0. In this case, the flag predFlagL0 indicating whether to use L0 prediction is 0, and whether to use L1 prediction is The flag predFlagL1 indicating whether the inter prediction mode is bi-prediction (Pred_BI) is set to 1. , a flag predFlagL0 indicating whether to use L0 prediction, and a flag predFlagL0 indicating whether to use L1 prediction. The flags predFlagL1 and predFlagL2 are both set to 1. Furthermore, the prediction mode Pred When the mode is inter prediction (PRED_INTER) and merging mode is selected, merging candidates are derived. The coding information of the already decoded coding block stored in the coding information storage memory 205 is Using the above, multiple merge candidates are derived and registered in a merge candidate list described later. Among the multiple merge candidates registered in the list, the bit stream is decoded by the bit stream decoding unit 201 and supplied. Select a merge candidate that corresponds to the merge index of the selected merge candidate, and make an L0 prediction for the selected merge candidate. , and flags predFlagL0[xP][yP], predFlagL1[xP][yP] indicating whether to use L1 prediction , the reference indices of L0 and L1, refIdxL0[xP][yP], refIdxL1[xP][yP], the motion of L0 and L1 The inter-prediction information such as the vectors mvL0[xP][yP] and mvL1[xP][yP] is stored in the coding information storage memory 2. 05, where xP and yP are the positions of the top left pixel of the coding block in the picture. The detailed configuration and operation of the inter prediction unit will be described later.

[0043] The intra prediction unit 204 detects whether the prediction mode PredMode of the coding block to be processed is intra. In the prediction (PRED_INTRA) step, intra prediction is performed. The encoding information includes the intra-prediction mode, and the decoded image is A predicted image signal is generated by intra-prediction from the decoded image signal stored in the image memory 208. The intra prediction unit 20 generates a prediction image signal and supplies the prediction image signal to the decoded image signal superimposition unit 207. 4 corresponds to the intra prediction unit 103 of the image encoding device 100, The predictor 103 performs the same process as the predictor 103.

[0044] The inverse quantization and inverse orthogonal transformation unit 206 performs orthogonal transformation and quantization on the bit string decoded by the bit string decoding unit 201. The inverse orthogonal transform and inverse quantization are performed on the converted residual signal. Obtain the residual signal.

[0045] The decoded image signal superimposing unit 207 receives the predicted image signal inter-predicted by the inter prediction unit 203. 203, or a predicted image signal obtained by intra-prediction in the intra-prediction unit 204, and The decoded signal is obtained by superimposing the residual signal that has been inversely orthogonally transformed and inversely quantized by the orthogonal transform unit 206. The decoded image signal is decoded and stored in the decoded image memory 208. When encoding, a filtering process is performed on the decoded image to reduce block distortion caused by encoding. After this, the image may be stored in the decoded image memory 208.

[0046] Next, the operation of the block division unit 101 in the image coding device 100 will be described. Figure 3 shows the division of an image into treeblocks and the further division of each treeblock. First, the input image is divided into tree blocks of a given size. Each tree block is sorted in a predetermined order, i.e., in a raster order (step S1001). The tree block is scanned in the scan order (step S1002) and the inside of the tree block to be processed is divided. (Step S1003).

[0047] FIG. 7 is a flowchart showing the detailed operation of the division process in step S1003. Then, it is determined whether or not the block to be processed is to be divided into four (step S1101).

[0048] If it is determined that the processing block should be divided into four, the processing block is divided into four (step (Step S1102). For each block into which the processing target block is divided, That is, the upper left, upper right, lower left, and lower right are scanned in this order (step S1103). 6 is an example of a scan order, and 601 in FIG. 6 is an example in which the processing target block is divided into four. The numbers 0 to 3 in 601 indicate the order of processing. For each block, the flowchart in FIG. 7 is called recursively.

[0049] If it is determined that the processing target block is not to be divided into four, it is divided into two or three (step S1 105).

[0050] FIG. 8 is a flowchart showing the detailed operation of the 2-3 division process in step S1105. First, we decide whether to divide the block to be processed into 2 or 3 parts, i.e., how many parts to divide into 2 or 3 parts. It is determined whether or not to perform either of these (step S1201).

[0051] When it is not determined that the processing block should be divided into 2-3 blocks, i.e., when it is determined that the processing block should not be divided If so, the division ends (step S1211) and the process returns to the upper hierarchical block.

[0052] If it is determined that the block to be processed should be divided into 2-3, the block to be processed is further divided into 2 It is determined whether or not to divide (step S1202).

[0053] If it is determined that the processing block should be divided into two, the processing block is divided vertically. Based on the result, the processing target block is Divide the target block vertically (step S1204), or divide the target block horizontally ( Step S1205) As a result of step S1204, the processing target block is As shown in FIG. 12, the block is divided into two vertically. As a result of step S1205, the block to be processed is As shown in FIG. 604, it is divided into two parts horizontally.

[0054] In step S1202, if it is not determined that the block to be processed is to be divided into two In other words, if it is determined that the block should be divided into three, whether or not to divide the block vertically Based on the result, the processing target block is divided vertically. (step S1207), or divide the processing target block horizontally (step S 1208). As a result of step S1207, the processing target block is vertically divided into two blocks as shown in FIG. The block is divided into three parts in the vertical direction. As a result of step S1208, the block to be processed is as shown in FIG. As shown, it is divided into three horizontal sections.

[0055] After executing either step S1204 or step S1205, the processing target block is Each divided block is scanned from left to right and from top to bottom (step S1209 The numbers 0 to 3 in 602 to 605 in FIG. 6 indicate the order of processing. For each block, the flowchart in FIG. 8 is called recursively.

[0056] The recursive block splitting described here is done by dividing the number of times or the number of blocks to be processed. The necessity of division may be restricted depending on the size of the image. By making a prearrangement between the decoder and the decoder, the decoder can be realized without transmitting information. Alternatively, the encoding device may determine information that limits the necessity of division and record it in the encoded bit string. The signal may be transmitted to the decoding device by the above-mentioned method.

[0057] Here, when a block is divided, the block before the division is called the parent block, and the block after the division is called the parent block. Each block in is called a child block.

[0058] Next, the operation of the block division unit 202 in the image decoding device 200 will be described. The block division unit 202 performs the same processing procedure as the block division unit 101 of the image encoding device 100. However, the block division of the image coding device 100 is In the unit 101, an optimization method such as optimal shape estimation by image recognition and strain rate optimization is applied, The optimal block division shape is determined, whereas the block division in the image decoding device 200 is determined. The dividing unit 202 divides the blocks by decoding the block division information recorded in the encoded bit string. The difference is in determining the lock split shape.

[0059] Syntax for block division in the first embodiment (syntax rules for encoded bit strings) The coding_quadtree() syntax is shown in Figure 9. ,multi_type_tree() represents the syntax for dividing a block into two or three parts. qt_split is a flag that indicates whether to split a block into four. If you do not want to split into 4, set qt_split=1. If you do not want to split into 4, set qt_split=0. 1) For each block divided into four, recursively divide it into four (coding_quadtree(0), c coding_quadtree(1), coding_quadtree(2), coding_quadtree(3)). split=0) determines the subsequent split according to multi_type_tree(). mtt_split A flag that indicates whether to split the image further. If further splitting is performed (mtt_split=1), the image is split vertically. The flag mtt_split_vertical indicates whether to split the image horizontally or vertically, and the flag mtt_split_vertical indicates whether to split the image horizontally or vertically. Refer to mtt_split_binary, which is the flag that determines whether to split the vertical or horizontal axis into three parts. al=1 indicates a vertical split, mtt_split_vertical=0 indicates a horizontal split mtt_split_binary=1 indicates splitting into two parts, mtt_split_binary=0 indicates splitting into three parts. This indicates that the tree is split by recursively calling multi_type_tree until mtt_split=0 is reached. This allows for hierarchical block division.

[0060] <Inter Prediction> The inter prediction method according to the embodiment is performed by the inter prediction unit 10 of the video encoding device in FIG. 2 and is implemented in the inter prediction unit 203 of the video decoding device of FIG.

[0061] The inter prediction method according to the embodiment will be described with reference to the drawings. The method is performed in both encoding and decoding processes on a coding block basis.

[0062] <Description of the inter prediction unit 102 on the encoding side> FIG. 16 is a diagram showing a detailed configuration of the inter-prediction unit 102 of the video encoding device of FIG. The normal motion vector predictor mode derivation unit 301 derives a plurality of normal motion vector predictor candidates. A predicted motion vector is selected, and a difference vector between the detected motion vector and the predicted motion vector is calculated. Inter prediction mode, reference index, motion vector, and calculated difference vector This inter prediction information is the inter prediction information for the normal prediction motion vector mode. The normal predicted motion vector mode derivation unit 301 supplies the motion vector to the normal predicted motion vector mode determination unit 305. The detailed configuration and processing will be described later.

[0063] The normal merge mode derivation unit 302 derives a plurality of normal merge candidates and determines the normal merge candidates. Select the inter prediction information for normal merge mode to obtain the inter prediction information for normal merge mode. The prediction mode determination unit 305 receives the merge mode. The theory will be explained later.

[0064] The sub-block prediction motion vector mode derivation unit 303 determines a plurality of sub-block prediction motion vectors. Then, vector candidates are derived to select sub-block predicted motion vectors, and the detected motion vector and The difference vector of the detected inter prediction mode, reference index, motion vector, and The calculated difference vector is used as inter prediction information for normal prediction motion vector mode. This inter prediction information is supplied to the inter prediction mode determination unit 305. The detailed configuration and processing of the predictive motion vector mode derivation unit 303 will be described later.

[0065] The sub-block merging mode derivation unit 304 derives a plurality of sub-block merging candidates. Select sub-block merging candidates and obtain inter prediction information for sub-block merging mode This inter prediction information is supplied to the inter prediction mode determination unit 305. The detailed configuration and processing of the merge mode derivation unit 304 will be described later.

[0066] The inter prediction mode determination unit 305 determines whether the normal prediction motion vector mode is derived from the normal prediction motion vector mode derivation unit 301 or the normal A merge mode derivation unit 302, a sub-block predicted motion vector mode derivation unit 303, a sub-block Based on the inter prediction information supplied from the block merge mode derivation unit 304, The inter prediction mode is determined from the inter prediction mode determination unit 305. The prediction information is provided to a motion compensation prediction unit 306 .

[0067] The motion compensation prediction unit 306 stores the decoded image in the decoded image memory 1 based on the determined inter prediction information. Inter prediction is performed on the reference image signal stored in 04. For details on the configuration and processing, More details will be given later.

[0068] <Description of the Inter Prediction Unit 203 on the Decoding Side> FIG. 22 is a diagram showing a detailed configuration of the inter prediction unit 203 of the video decoding device of FIG.

[0069] The normal prediction motion vector mode derivation unit 401 derives a plurality of normal prediction motion vector candidates. A predicted motion vector is selected, and a difference vector between the detected motion vector and the predicted motion vector is calculated. The inter prediction mode, reference index, motion vector, and difference vector are normal prediction This inter prediction information is the motion vector mode inter prediction information. The normal prediction motion vector mode derivation unit 40 The detailed configuration and processing of 1 will be described later.

[0070] The normal merge mode derivation unit 402 derives a plurality of normal merge candidates and determines the normal merge candidates. Select and obtain inter prediction information for normal merge mode. This inter prediction information is switched The merge mode derivation unit 402 normally supplies the merge mode derivation signal to the motion compensation prediction unit 406 via the merge mode derivation unit 408. The detailed configuration and processing will be described later.

[0071] The sub-block prediction motion vector mode derivation unit 403 selects a plurality of sub-block prediction motion vectors. Then, vector candidates are derived to select sub-block predicted motion vectors, and the detected motion vector and The difference vector of the detected inter prediction mode, reference index, motion vector, and The calculated difference vector is used as inter prediction information for normal prediction motion vector mode. This inter prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the sub-block predicted motion vector mode derivation unit 403 will be described later. State.

[0072] The subblock merging mode derivation unit 404 derives multiple subblock merging candidates. Select sub-block merging candidates and obtain inter prediction information for sub-block merging mode This inter prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the sub-block merging mode derivation unit 404 will be described later.

[0073] The motion compensation prediction unit 406 stores the decoded image in the decoded image memory 2 based on the determined inter prediction information. Inter prediction is performed on the reference image signal stored in 08. The procedure is the same as for the encoding side.

[0074] <Normal predicted motion vector mode derivation part (normal AMVP)> The normal prediction motion vector mode derivation unit 301 in FIG. unit 321, a temporal predicted motion vector candidate derivation unit 322, and a historical predicted motion vector candidate derivation unit 3 23, a predicted motion vector candidate supplementation unit 325, a normal motion vector detection unit 326, a predicted motion vector candidate The image processing unit 320 includes a vector candidate selection unit 327 and a motion vector subtraction unit 328.

[0075] The normal prediction motion vector mode derivation unit 401 in FIG. unit 421, a temporal predicted motion vector candidate derivation unit 422, and a historical predicted motion vector candidate derivation unit 4 23, a motion vector predictor candidate supplementation unit 425, a motion vector predictor candidate selection unit 426, a motion vector The vector adder 427 is included.

[0076] The normal prediction motion vector mode derivation unit 301 on the encoding side and the normal prediction motion vector mode derivation unit 302 on the decoding side The processing procedure of the torque mode derivation unit 401 is shown in the flowcharts of FIG. 19 and FIG. 25. 19 is a diagram showing a normal motion vector mode derivation unit 301 on the encoding side. FIG. 25 is a flowchart showing a procedure for deriving a predicted motion vector mode. The normal motion vector mode derivation unit 401 performs a normal predicted motion vector mode derivation process. 6 is a flowchart.

[0077] <Normal prediction motion vector mode derivation part (normal AMVP): Explanation on the encoding side> The normal prediction motion vector mode derivation process procedure on the encoding side will be described with reference to FIG. In the description of the processing procedure of FIG. 19, the term motion vector in the specification and the normal motion vector in FIG. The term "vector" corresponds to the motion vector. A normal motion vector is detected for each center prediction mode and reference index (see the step in FIG. 19). Step S100).

[0078] Next, the spatial prediction motion vector candidate derivation unit 321 and the temporal prediction motion vector candidate derivation unit 3 22, a history prediction motion vector candidate derivation unit 323, a prediction motion vector candidate supplement unit 325, A predicted motion vector candidate selection unit 327 and a motion vector subtraction unit 328 select a normal predicted motion vector The difference motion vectors of the motion vectors used in the inter prediction of the mode are set for L0 and L1, respectively. Specifically, the prediction of the processing target block is calculated (steps S101 to S106 in FIG. 19). Prediction mode PredMode is inter prediction (MODE_INTER) and inter prediction mode is L0 prediction (Pr ed_L0), calculate the motion vector predictor candidate list mvpListL0 of L0 and Then, the differential motion vector mvdL0 of the motion vector mvL0 of L0 is calculated. If the inter prediction mode of the target block is L1 prediction (Pred_L1), the predicted motion vector of L1 is A motion vector candidate list mvpListL1 is calculated, a predicted motion vector mvpL1 is selected, and the motion vector of L1 is calculated. Calculate the differential motion vector mvdL1 of the to-be-processed block mvL1. When bi-prediction (Pred_BI) is used, both L0 and L1 predictions are performed, and the predicted motion vector of L0 is A motion vector candidate list mvpListL0 is calculated, a motion vector predictor mvpL0 of L0 is selected, and the motion vector predictor A differential motion vector mvdL0 of the vector mvL0 is calculated, and a predicted motion vector candidate of L1 is calculated. A complementary list mvpListL1 is calculated, a predicted motion vector mvpL1 of L1 is calculated, and the motion vector of L1 is calculated. Then, the differential motion vector mvdL1 of the corresponding to the tor mvL1 is calculated.

[0079] The differential motion vector calculation process is performed for each of L0 and L1. Therefore, in the following explanation, L0 and L1 are represented as a common LX. In the process of calculating the difference motion vector of L0, X is 0, and the difference motion vector of L1 is In the process of calculating the difference motion vector of LX, X is 1. If you want to refer to information from another list, not LX, use LY as the other list. represent.

[0080] When the motion vector mvLX of LX is used (step S102 in FIG. 19: YES), Calculate the candidate predicted motion vectors of LX and construct a candidate predicted motion vector list mvpListLX of LX. (Step S103 in FIG. 19). A spatial prediction motion vector candidate derivation unit 321, a temporal prediction motion vector candidate derivation unit 322, a history A motion vector predictor candidate derivation unit 323 and a motion vector predictor candidate supplement unit 325 generate a plurality of motion vector predictors. The motion vector predictor candidate list mvpListLX is constructed by deriving motion vector candidates. The detailed process of step S103 will be described later with reference to the flowchart of FIG. do.

[0081] Next, the motion vector predictor candidate selection unit 327 selects a motion vector predictor candidate list for LX. Select a predicted motion vector mvpLX for LX from the mvpListLX (step S104 in FIG. 19). The motion vector mvLX and each predicted motion vector stored in the predicted motion vector candidate list mvpListLX are The motion vector candidate mvpListLX[i] is used as the difference between the motion vector candidate mvpListLX[i] and each of the motion vector candidates mvpListLX[i]. The amount of code required to encode these differential motion vectors is estimated by the motion vector candidate list mvpLis Then, the motion vector predictor candidate list mvpListLX is calculated for each element of tLX. Among the elements, the candidate motion vector predictor that has the smallest amount of coding for each candidate motion vector predictor Select mvpListLX[i] as the predicted motion vector mvpLX and obtain its index i. The predicted motion vector with the smallest amount of generated code in the predicted motion vector candidate list mvpListLX is If there are multiple candidates, the index in the motion vector predictor candidate list mvpListLX is used. The candidate mvpListLX[i] of the predicted motion vector represented by the smallest number is used as the optimal predicted motion vector. mvpLX and obtain its index i.

[0082] Next, the motion vector subtraction unit 328 subtracts the motion vector mvLX of LX selected from the motion vector mvLX of LX. Subtract the predicted motion vector mvpLX, mvdLX = mvLX - mvpLX Then, the differential motion vector mvdLX of LX is calculated as (Step S105 in FIG. 19).

[0083] <Normal predicted motion vector mode derivation part (normal AMVP): Decoding side explanation> Next, a normal predicted motion vector mode process procedure on the decoding side will be described with reference to FIG. On the other hand, a spatial prediction motion vector candidate derivation unit 421 and a temporal prediction motion vector candidate derivation unit 4 22, the history prediction motion vector candidate derivation unit 423 and the prediction motion vector candidate supplementation unit 425, The motion vectors used in inter prediction in normal prediction motion vector mode are divided into L0 and L1. Specifically, the process of the processing block is performed as follows (steps S201 to S206 in FIG. 25). Prediction mode PredMode is inter prediction (MODE_INTER) and the processing target block is inter prediction When the mode is L0 prediction (Pred_L0), calculate the predicted motion vector candidate list mvpListL0 of L0. Then, the predicted motion vector mvpL0 is selected, and the motion vector mvL0 of L0 is calculated. If the inter prediction mode of the target block is L1 prediction (Pred_L1), the predicted motion vector of L1 is A motion vector candidate list mvpListL1 is calculated, a predicted motion vector mvpL1 is selected, and the motion vector of L1 is calculated. If the inter prediction mode of the processing target block is bi-predictive (Pred_BI), Then, both L0 prediction and L1 prediction are performed, and the L0 predicted motion vector candidate list mvpListL0 is calculated. Then, the predicted motion vector mvpL0 of L0 is selected, and the motion vector mvL0 of L0 is calculated. In both cases, a motion vector predictor candidate list mvpListL1 of L1 is calculated, and the motion vector predictor of L1 is Then, the motion vector mvpL1 for L1 is calculated, and the motion vector mvL1 for L2 is calculated.

[0084] As with the encoding side, the decoding side also performs motion vector calculation processing for each of L0 and L1. However, the same processing is performed for both L0 and L1. Therefore, in the following explanation, L1 is represented as a common LX. LX is used for inter prediction of the coding block being processed. It represents the inter prediction mode. In the process of calculating the motion vector of L0, X is 0, and in the process of calculating the motion vector of L1, In the process of calculating the motion vector of X, X is 1. Also, in the process of calculating the motion vector of LX, During processing, the LX is calculated by referencing information from another reference list, not the same reference list. When referring to one reference list, the other reference list is represented as LY.

[0085] When the motion vector mvLX of LX is used (step S202 in FIG. 25: YES), Calculate the candidate predicted motion vectors of LX and construct a candidate predicted motion vector list mvpListLX of LX. (Step S203 in FIG. 25). A spatial prediction motion vector candidate derivation unit 421, a temporal prediction motion vector candidate derivation unit 422, a history A motion vector predictor candidate derivation unit 423 and a motion vector predictor candidate supplement unit 425 generate a plurality of motion vector predictors. Then, the motion vector predictor candidate list mvpListLX is constructed. The detailed process of step S203 will be described later with reference to the flowchart of FIG. do.

[0086] Next, the motion vector predictor candidate selection unit 426 selects the motion vector predictor candidate list mvpListLX The bitstream decoder 201 decodes and supplies the predicted motion vector index mv The candidate predicted motion vector mvpListLX[mvpIdxLX] corresponding to pIdxLX is selected as the predicted motion vector. The vector mvpLX is extracted (step S204 in FIG. 25).

[0087] Then, the motion vector adder 427 decodes the bit stream and supplies it to the bit stream decoder 201. Add the differential motion vector mvdLX of LX and the predicted motion vector mvpLX of LX, mvLX = mvpLX + mvdLX Then, the motion vector mvLX of LX is calculated (step S205 in FIG. 25).

[0088] <Normal predicted motion vector mode derivation part (normal AMVP): Motion vector prediction method> FIG. 20 shows a normal prediction motion vector mode guide of a video encoding device according to an embodiment of the present invention. The motion vector mode derivation unit 301 and the normal prediction motion vector mode derivation unit 401 of the video decoding device are common to the motion vector mode derivation unit 301 and the normal prediction motion vector mode derivation unit 401 of the video decoding device. 1 is a flowchart showing a processing procedure of a normal predicted motion vector mode derivation process having a function. do.

[0089] The normal prediction motion vector mode derivation unit 301 and the normal prediction motion vector mode derivation unit 40 1, a motion vector predictor candidate list mvpListLX is provided. The mvpListLX has a list structure and is a prediction vector that indicates the location within the predicted motion vector candidate list. The motion vector index and the motion vector predictor candidate corresponding to the index are used as elements. The motion vector predictor index number starts from 0. The motion vector predictor candidate list mvpListLX is stored in the storage area of ​​the motion vector predictor candidate list mvpListLX. In this embodiment, the motion vector predictor candidate list mvpListLX is stored at least It is assumed that two predicted motion vector candidates (inter prediction information) can be registered. Furthermore, the motion vector predictor registered in the motion vector predictor candidate list mvpListLX is The variable numCurrMvpCand indicating the number of candidates is set to 0.

[0090] The spatial prediction motion vector candidate derivation units 321 and 421 derive the spatial prediction motion vector candidate from the adjacent block on the left. In this process, the candidate predicted motion vector is derived from the adjacent block on the left (A0 or A1) is a flag availableFlagLXA indicating whether the predicted motion vector candidate is available or not; Then, the motion vector mvLXA and the reference index refIdxA are derived, and mvLXA is used as the predicted motion vector candidate. Add it to the complement list mvpListLX (step S301 in FIG. 20). Note that when L0, X is 0. When L1, X is set to 1 (similarly below). Next, the spatial prediction motion vector candidate derivation unit 32 1 and 421 are the predicted motion vectors from the adjacent blocks (B0, B1 or B2) on the upper side. In this process, the motion vector candidate of the adjacent block on the upper side is derived. A flag availableFlagLXB indicating whether the motion vector mvLXB and the reference index are available or not. If mvLXA and mvLXB are not equal, mvLXB is used as the predicted motion vector candidate. Add it to the complement list mvpListLX (Step S302 in FIG. 20). The processes in S301 and S302 are the same except for the positions and numbers of adjacent blocks to be referenced. A flag indicating whether a motion vector predictor candidate for the coding block is available. , and the motion vector mvLxN, the reference index refIdxN (N is A or B, and so on). Put out.

[0091] Next, the temporal motion vector predictor candidate derivation units 322 and 422 calculate the current processing target picture The motion vector prediction candidates are derived from coding blocks in a picture that is different in time from the image. In this process, the predicted motion vectors of coding blocks in pictures of different times are obtained. A flag indicating whether a motion candidate is available or not, and a motion vector mvLXCol , derive the reference index refIdxCol and the reference list listCol, and use mvLXCol as the predicted motion vector. In step S303 of FIG. 20, the VP is added to the rule candidate list mvpListLX. The derivation process of 3 will be explained in detail later.

[0092] Note that the temporal prediction motion vector is calculated in units of sequence (SPS), picture (PPS) or slice. It is assumed that the processing of the torque candidate derivation units 322 and 422 can be omitted.

[0093] Next, the history prediction motion vector candidate derivation units 323 and 423 calculate the history prediction motion vector candidate. The historical predicted motion vector candidates registered in the auxiliary list HmvpCandList are used as the predicted motion vector candidates. The mvpListLX is added to the auxiliary list (step S304 in FIG. 20). The registration process will be described in detail later with reference to the flowchart of FIG.

[0094] Next, the motion vector predictor candidate supplementation units 325 and 425 generate the motion vector predictor candidate list mv Add a motion vector of a given value, such as (0,0), until pListLX is satisfied (S3 in FIG. 20). 05).

[0095] <Normal merge mode derivation part (normal merge)> The normal merge mode derivation unit 302 in FIG. 18 includes a spatial merge candidate derivation unit 341, a temporal merge candidate derivation unit 342, and a A candidate derivation unit 342, an average merge candidate derivation unit 344, a history merge candidate derivation unit 345, and a merge It includes a candidate supplementation unit 346 and a merge candidate selection unit 347 .

[0096] The normal merge mode derivation unit 402 in FIG. 24 includes a spatial merge candidate derivation unit 441, a temporal merge candidate derivation unit 442, and a A candidate derivation unit 442, an average merge candidate derivation unit 444, a history merge candidate derivation unit 445, and a merge It includes a candidate supplementation unit 446 and a merge candidate selection unit 447 .

[0097] FIG. 21 shows the normal merge mode derivation unit 302 of the video encoding device according to the embodiment of the present invention. A normal merge mode derivation unit 402 of the video decoding device has a common function. 11 is a flowchart illustrating a procedure for a dimode derivation process.

[0098] The steps will be explained below in order. The slice type slice_type is explained for the B slice, but for the P slice, However, if the slice type slice_type is P slice, the inter prediction model can be applied. There is only L0 prediction (Pred_L0) as a mode, L1 prediction (Pred_L1) and bi-prediction (Pred_BI) are also supported. Since there is no L1, processing related to L1 can be omitted.

[0099] In the normal merge mode derivation unit 302 and the normal merge mode derivation unit 402, The merge candidate list mergeCandList has a list structure, A merge index that indicates the location within the merge candidate list, and the merge A memory area is provided to store merge candidates as elements. The merge index number is 0. The merge candidates are stored in the merge candidate list (mergeCandList) starting from In the following process, the merge index i registered in the merge candidate list mergeCandList is The merge candidates are represented by mergeCandList[i]. The candidate list mergeCandList must contain at least six merge candidates (inter prediction information). In addition, the merge candidates registered in the mergeCandList can be The variable numCurrMergeCand, which indicates the number of merge candidates, is set to 0.

[0100] In the spatial merging candidate derivation unit 341 and the spatial merging candidate derivation unit 441, The encoded information storage memory 111 or the encoded information storage memory 205 of the video decoding device is stored in the From the stored coding information, the blocks adjacent to the left and above the block to be processed are Derive spatial merge candidates A and B, and create a merge candidate list, mergeC, of ​​the derived spatial merge candidates. The spatial merge candidates A and B are registered in the andList (step S401 in FIG. 21). Define N, which indicates one of the temporal merge candidates Col. Inter prediction information of block N A flag indicating whether a spatial merge candidate N can be used as a spatial merge candidate. The reference index refIdxL0N of L0 and the reference index refIdxL1N of L1 of the complement N, A L0 prediction flag predFlagL0N indicates whether L0 prediction is performed or not, and a L1 prediction flag predFlagL0N indicates whether L1 prediction is performed or not. The L1 prediction flag predFlagL1N indicates the motion vector mvL0N of L0, and the motion vector mvL1 of L1 indicates the motion vector mvL1 of L1. However, in this embodiment, N is derived from the block including the coding block to be processed. Since the merge candidates are derived without referring to other coding blocks included in the block, Spatial merging candidates contained in blocks that contain elephant coding blocks are not derived.

[0101] Next, the temporal merge candidate derivation unit 342 and the temporal merge candidate derivation unit 442 determine different time Then, a temporal merge candidate is derived from the pictures between the first and second pictures, and the derived temporal merge candidate is called a merge candidate. The time merge candidates are registered in the list mergeCandList (step S402 in FIG. 21). availableFlagCol, a flag indicating whether L0 prediction of temporal merge candidates is performed The L0 prediction flag predFlagL0Col indicates whether L0 prediction is performed, and the L1 prediction flag predFlagL0Col indicates whether L1 prediction is performed. The flag predFlagL1Col, the motion vector mvL0Col of L0, and the motion vector mvL1Col of L1 are The detailed process of step S402 will be described later with reference to FIG. explain.

[0102] Note that the temporal merge candidate derivation is performed in units of sequences (SPS), pictures (PPS), or slices. It is assumed that the processing of the extraction unit 342 and the temporal merge candidate derivation unit 442 can be omitted.

[0103] Next, the history merging candidate derivation unit 345 and the history merging candidate derivation unit 445 perform history prediction Mark the historical predicted motion vector candidates registered in the motion vector candidate list HmvpCandList. The mergeCandList is added to the merge candidate list (step S403 in FIG. 21). The detailed process of step 3 will be explained later with reference to the flowchart of FIG.

[0104] Next, the average merge candidate derivation unit 344 and the average merge candidate derivation unit 444 calculate the merge candidates. Derive average merge candidates from the complement list mergeCandList and merge the derived average merge candidates. The merge candidate list is registered in mergeCandList (step S404 in FIG. 21). The detailed process of step 04 will be explained later using the flowchart in FIG. .

[0105] Next, the merge candidate supplementation unit 346 and the merge candidate supplementation unit 446 create a merge candidate list The number of merge candidates registered in mergeCandList, numCurrMergeCand, is less than the maximum number of merge candidates, M If it is smaller than axNumMergeCand, the merge candidate list mergeCandList The number of merge candidates numCurrMergeCand is the maximum number of merge candidates MaxNumMergeCand. The merge candidates are derived and registered in the merge candidate list mergeCandList (step S4 in FIG. 21). 05). In the P slice, the maximum number of merge candidates is limited to MaxNumMergeCand. The prediction mode with the motion vector value (0,0) in the index is L0 prediction (Pred_L0) Add merge candidates for B slices. For B slices, the motion vectors are ( 0,0) is added as a merge candidate whose prediction mode is bi-predictive (Pred_BI).

[0106] Next, the merging candidate selection unit 347 and the merging candidate selection unit 447 select the merging candidate list Select a merge candidate from the merge candidates registered in mergeCandList. The merge candidate selection unit 347 selects merge candidates by calculating the code amount and the distortion amount. A merge index indicating the selected merge candidate is generated. On the other hand, the decoding side merge candidate selection unit 447 supplies the decoded Based on the merge index, a merge candidate is selected, and the selected merge candidate is motion-compensated. The result is supplied to the compensation prediction unit 406.

[0107] The normal merge mode derivation unit 302 and the normal merge mode derivation unit 402 are If the size of a block (the product of width and height) is less than 32, the parent block of that coding block Then, for every child block, merge candidates are derived using the The merge candidates are used, provided that the parent block size is 32 or more and fits within the screen. Only if

[0108] <Sub-block prediction motion vector mode derivation> Sub-block predicted motion vector mode derivation will now be described.

[0109] FIG. 26 shows a sub-block prediction motion vector mode guide in the encoding device according to the present embodiment. 3 is a block diagram of an output unit 303.

[0110] First, the affine inheritance predicted motion vector candidate derivation unit 361 derives an affine inheritance predicted motion vector candidate. The details of affine inheritance prediction motion vector candidate derivation will be described later. State.

[0111] Next, the affine construction prediction motion vector candidate derivation unit 362 performs affine construction prediction Derive motion vector candidates. For details on deriving affine construction prediction motion vector candidates, see This will be explained later.

[0112] Next, the affine-identical prediction motion vector candidate derivation unit 363 performs affine-identical prediction Derive motion vector candidates. For details on deriving affine identical predictor motion vector candidates, see This will be explained later.

[0113] The sub-block motion vector detector 366 is adapted to the sub-block prediction motion vector mode. A sub-block motion vector corresponding to the sub-block is detected, and the detected vector is used as a sub-block predicted motion vector. The result is supplied to a torque candidate selection unit 367 and a difference calculation unit 368.

[0114] The sub-block predicted motion vector candidate selection unit 367 selects the affine inheritance predicted motion vector candidate Affine construction predicted motion vector candidate derivation unit 361, an affine construction predicted motion vector candidate derivation unit 362, an affine same predicted motion vector candidate derivation unit 363, an affine construction predicted motion vector candidate derivation unit 364, an affine same predicted motion vector candidate derivation unit 365, an affine construction predicted motion vector candidate derivation unit 366, an affine same predicted motion vector candidate derivation unit 367, an Among the sub-block predicted motion vector candidates derived in the motion vector candidate derivation unit 363, Based on the motion vector supplied from the sub-block motion vector detection unit 366, A sub-block predicted motion vector candidate is selected, and the selected sub-block predicted motion vector Information about the candidates is supplied to the inter prediction mode determination unit 305 and the difference calculation unit 368 .

[0115] The difference calculation unit 368 calculates the motion vector supplied from the sub-block motion vector detection unit 366. The sub-block predicted motion vector candidate selection unit 367 selects the sub-block predicted motion vector candidate from the motion vector. The inter prediction mode determination unit 102 determines the difference prediction motion vector obtained by subtracting the inter prediction motion vector from the inter prediction mode prediction motion vector. Supply to 305.

[0116] FIG. 27 shows a sub-block prediction motion vector mode derivation method in the decoding device according to the present embodiment. 4 is a block diagram of unit 403.

[0117] First, the affine inheritance predicted motion vector candidate derivation unit 461 derives an affine inheritance predicted motion vector candidate. The process of the affine inheritance predicted motion vector candidate derivation unit 461 is as follows: Processing of the affine inheritance predicted motion vector candidate derivation unit 361 in the encoding device according to the embodiment is equivalent to.

[0118] Next, the affine construction prediction motion vector candidate derivation unit 462 performs affine construction prediction The affine construction prediction motion vector candidate derivation unit 462 performs the following process: The processing of the affine construction predictor motion vector candidate derivation unit 362 in the encoding device of this embodiment It is identical with the principle.

[0119] Next, the affine-identical prediction motion vector candidate derivation unit 463 performs affine-identical prediction The affine identical predictor motion vector candidate derivation unit 463 performs the following process: The processing of the affine identical predictor motion vector candidate derivation unit 363 in the encoding device of this embodiment It is identical with the principle.

[0120] The sub-block predicted motion vector candidate selection unit 466 selects the affine inheritance predicted motion vector candidate Affine construction predicted motion vector candidate derivation unit 461, an affine construction predicted motion vector candidate derivation unit 462, an affine same predicted motion vector Among the sub-block predicted motion vector candidates derived in the motion vector candidate derivation unit 463, Then, based on the predicted motion vector index transmitted from the encoding device and decoded, a sub-block predicted motion vector candidate is selected, and the selected sub-block predicted motion vector candidate is The information on the complement is supplied to the motion compensation prediction unit 406 and the addition operation unit 467 .

[0121] The addition operation unit 467 adds the sub-block predicted motion vector candidate selected by the sub-block predicted motion vector candidate selection unit 466. The differential motion vector transmitted from the encoding device and decoded is used as the subblock predicted motion vector. The motion vector generated by the addition is supplied to the motion compensation prediction unit 406 .

[0122] <Derivation of affine inheritance predicted motion vector candidates> The affine successive prediction motion vector candidate derivation unit 361 will be described. The motion vector candidate derivation unit 461 is also the same as the affine inheritance prediction motion vector candidate derivation unit 36 Same as 1.

[0123] The affine inheritance motion vector predictor candidate inherits the motion vector information of the control point.

[0124] FIG. 30 is a diagram for explaining derivation of affine inheritance predicted motion vector candidates.

[0125] The affine inheritance motion vector candidate predictor is a vector that is derived from the spatially adjacent encoded and decoded blocks. The motion vectors of the control points are obtained by searching for the motion vectors of the control points.

[0126] Specifically, the block (A0, A1) adjacent to the left of the block to be processed and the block to be processed Up to one block from each of the blocks (B0, B1, B2) adjacent to the top of the elephant block An affine mode is searched for and an affine inheritance predicted motion vector is obtained.

[0127] FIG. 34 is a flowchart showing the derivation of affine inheritance predicted motion vector candidates.

[0128] First, the block (A0, A1) adjacent to the left of the block to be processed is set as the left group. (Step S3101) The block including A0 is a block using affine transformation motion compensation. It is determined whether A0 is in affine mode (step S3102). If it is (step S3102: YES), obtain the affine mode used by A0. (Step S3103), and the process moves to the adjacent block on the upper side. If the affine inheritance prediction motion vector candidate is not the affine inheritance prediction motion vector candidate (step S3102: NO), The target is A0->A1, and an attempt is made to obtain the affine mode from the block containing A1.

[0129] Next, the blocks (B0, B1, B2) adjacent to the upper side of the block to be processed are grouped together. Then, the block including B0 is checked to see if it is in affine mode (step S3104). If B0 is in affine mode (step S310 5: YES), obtain the affine mode used by B0 (step S3106), and execute the process. If B0 is not in affine mode (step S3105: NO), the affine inheritance is The target of the prediction motion vector candidate derivation is B0->B1, and the affiliate is selected from the block including B1. Furthermore, if B1 is not an affine mode (step S310 5:NO), the target of affine inheritance predicted motion vector candidate derivation is B1->B2, and B2 is Attempts to get the affine mode from the containing block.

[0130] In this way, we divided the group into the left block and the upper block, and for the left block, searches for affine modes in the order from the bottom left to the top left block, and for the left block, By searching for affine modes in the order from the top right to the top left block, we find two affine modes that are as different as possible. The affine mode of the affine prediction motion vector can be obtained by It is possible to derive an affine predicted motion vector candidate that has a smaller error vector.

[0131] <Derivation of affine-constructed motion vector candidates> The affine construction prediction motion vector candidate derivation unit 362 will now be described. The motion vector candidate derivation unit 462 is also the same as the affine construction prediction motion vector candidate derivation unit 36 Same as 2.

[0132] Affine construction motion vector candidates are controlled from motion information of spatially adjacent blocks. Construct motion vector information for the points.

[0133] FIG. 31 is a diagram for explaining derivation of affine constructed predicted motion vector candidates.

[0134] The affine construction motion vector candidate predictor is a vector of spatially adjacent coded and decoded blocks. The motion vectors are then combined to construct a new affine mode.

[0135] Specifically, the blocks adjacent to the upper left of the target block (B2, B3, A2) The motion vector of the upper left control point CP0 is derived from the block to be processed. The motion vector of the upper right control point CP1 is derived from the block (B1, B0) and the target block is The motion vector of the lower left control point CP2 is derived from the block (A1, A0) adjacent to the lower left of the block. Put out.

[0136] FIG. 35 is a flowchart showing the derivation of affine constructed motion vector predictor candidates.

[0137] First, the upper left control point CP0, the upper right control point CP1, and the lower left control point CP2 are derived (step The upper left control point CP0 is the reference block that has the same reference image as the block to be processed. The lock is calculated by searching the B2, B3, A2 reference blocks in that order. The control point CP1 selects reference blocks B1, B0, and The lower left control point CP2 is calculated by searching the reference block in the order of priority. Search for reference blocks with the same reference image as the lock, in the order of A1, A0 reference blocks. It is calculated by

[0138] When selecting the three control points mode as the affine construction predicted motion vector (step S 3202:YES), whether all three control points (CP0, CP1, CP2) have been derived It is determined whether all three control points (CP0, CP1, CP2) are If the control points are derived from the three control points (CP0, CP1, C P2) is used as the affine-constructed predicted motion vector (step S32 04). If the three control points mode is not selected and the two control points mode is selected (step S3 202:NO), and judge whether or not the two control points (CP0, CP1) have all been derived. (Step S3205). When both control points (CP0, CP1) have been derived (Step Step S3205: YES), Affine model using two control points (CP0, CP1) is set as an affine constructed predicted motion vector (step S3206).

[0139] <Derivation of affine identical predictive motion vector candidates> The affine co-prediction motion vector candidate derivation unit 363 will be described. The motion vector candidate derivation unit 463 is also the same as the affine identical predictive motion vector candidate derivation unit 36 Same as 3.

[0140] The affine identical predictor motion vector candidate derives the same motion vector at each control point. is obtained.

[0141] Specifically, similarly to the affine construction motion vector candidate derivation units 362 and 462, each control By deriving the control point information and setting all control points to the same value between CP0 and CP2, In addition, all the temporal motion vectors derived in the same way as in the normal predicted motion vector mode are can also be obtained by setting all control points.

[0142] <Sub-block merge mode derivation> Sub-block merge mode derivation will now be described.

[0143] FIG. 28 shows the sub-block merge mode derivation unit 304 in the encoding device of this embodiment. The sub-block merging mode derivation unit 304 determines the sub-block merging candidate. The sublist subblockMergeCandList is provided by the normal merge mode derivation unit 302. The merge candidate list in The merge index indicates the location of the candidate list and the subblock corresponding to the index. A storage area is provided for storing the merge candidates as elements. The subblockMergeCandList contains at least five merge candidates. Inter prediction information can be registered. However, each merge candidate can be registered as follows: In addition, it has motion vector information for each subblock, or motion vector information for control points. Have information.

[0144] First, the sub-block time merge candidate derivation unit 381 determines the sub-block time merge candidate The details of the derivation of sub-block temporal merging candidates will be described later.

[0145] Next, the affine inheritance merge candidate derivation unit 382 derives affine inheritance merge candidates. The details of affine inheritance merge candidate derivation will be described later.

[0146] Next, the affine construction merge candidate derivation unit 383 derives affine construction merge candidates. The details of deriving affine construction merge candidates will be described later.

[0147] Next, the affine-fixed merge candidate derivation unit 385 derives affine-fixed merge candidates. The details of affine fixed merge candidate derivation will be described later.

[0148] The sub-block merging candidate selection unit 386 includes the sub-block temporal merging candidate derivation unit 381, An affine inheritance merge candidate derivation unit 382, ​​an affine construction merge candidate derivation unit 383, an affine Among the sub-block merging candidates derived by the fixed merging candidate derivation unit 385, Select a block merging candidate and display information about the selected sub-block merging candidate in the The prediction mode is supplied to the prediction mode determination unit 305.

[0149] FIG. 29 shows the sub-block merging mode derivation unit 404 in the decoding device according to this embodiment. 1 is a block diagram of a sub-block merging mode derivation unit 404. It has a list subblockMergeCandList, which is used to derive the subblock merge mode. It is the same as 304.

[0150] First, the sub-block time merge candidate derivation unit 481 determines the sub-block time merge candidate The process of the sub-block time merging candidate derivation unit 481 derives the sub-block time merging candidate. This is the same as the processing of the candidate derivation unit 381.

[0151] Next, the affine inheritance merge candidate derivation unit 482 derives affine inheritance merge candidates. The processing of the affine inheritance merge candidate derivation unit 482 is the same as that of the affine inheritance merge candidate derivation unit 3 This is the same as the process in 82.

[0152] Next, the affine construction merge candidate derivation unit 483 derives affine construction merge candidates. The processing of the affine construction merge candidate derivation unit 483 is the same as that of the affine construction merge candidate derivation unit 3 This is the same as the process in 83.

[0153] Next, the affine-fixed merge candidate derivation unit 485 derives affine-fixed merge candidates. The processing of the affine fixed merge candidate derivation unit 485 is the same as that of the affine fixed merge candidate derivation unit 485. This is the same as the process in 85.

[0154] The sub-block merging candidate selection unit 486 includes the sub-block temporal merging candidate derivation unit 481, An affine inheritance merge candidate derivation unit 482, an affine construction merge candidate derivation unit 483, an affine Among the sub-block merging candidates derived by the fixed merging candidate derivation unit 485, Selecting sub-block merging candidates based on the index transmitted and decoded from the decoder and supplies information about the selected sub-block merge candidates to the motion compensation prediction unit 406. .

[0155] The sub-block merge mode derivation unit 304 and the sub-block merge mode derivation unit 404 If the size (product of width and height) of a coding block is less than 32, Subblock merge candidates are derived for the parent block of . Then, for all child blocks In this case, we use the subblock merge candidates derived in the parent block. This applies only if the block size is 32 or more and fits on the screen.

[0156] <Sub-block time merge candidate derivation> The operation of the sub-block temporal merging candidate derivation unit 381 will be described later.

[0157] <Affine inheritance merge candidate derivation> The affine inheritance merge candidate derivation unit 382 will be described. The affine inheritance merge candidate derivation unit 482 is similar to the affine inheritance merge candidate derivation unit 382 .

[0158] Affine inheritance merge candidates are constrained by the affine models of spatially adjacent blocks. It inherits the affine model of the points.

[0159] FIG. 32 is a diagram for explaining the derivation of affine inheritance merge candidates. The derivation of the affine inheritance prediction motion vector is similar to the derivation of the affine inheritance prediction motion vector. The motion vectors are obtained by searching for the control points of the coded and decoded blocks.

[0160] Specifically, the block (A0, A1) adjacent to the left of the block to be processed and the block to be processed Up to one block from each of the blocks (B0, B1, B2) adjacent to the top of the elephant block Find an affine mode and use it for the affine merge mode.

[0161] FIG. 36 is a flowchart of affine inheritance merge candidate derivation.

[0162] First, the block (A0, A1) adjacent to the left of the block to be processed is set as the left group. (Step S3301) Determine whether the block including A0 is in affine mode. (Step S3302). If A0 is in affine mode (Step S3102: YES), S), obtain the affine model used by A0 (step S3303), and The process proceeds to block processing. If A0 is not in affine mode (step S3302: NO), The affine inheritance merge candidate derivation target is A0->A1, and the affine inheritance merge candidate is derived from the block including A1. Attempt to obtain the login mode.

[0163] Next, the blocks (B0, B1, B2) adjacent to the upper side of the block to be processed are grouped together. Then, the block including B0 is checked to see if it is in affine mode (step S3304). If B0 is in affine mode (step S3305), 5: YES), obtain the affine model used by B0 (step S3306), and execute the process. If B0 is not in affine mode (step S3305: NO), the affine inheritance is The target for deriving merge candidates is B0->B1, and the affine mode is Furthermore, if B1 is not in affine mode (step S3305: NO), The target of affine inheritance merge candidate derivation is B1->B2, and the affine inheritance merge candidate is derived from the block including B2. Attempt to obtain the login mode.

[0164] <Affine construction merge candidate derivation> The affine construction merge candidate derivation unit 383 will be described. The affine construction merge candidate derivation unit 483 is similar to the affine construction merge candidate derivation unit 383 .

[0165] FIG. 33 is a diagram for explaining derivation of affine construction merge candidates. The control points are calculated based on the motion information of spatially adjacent blocks and the time-coded blocks. Build the fin model.

[0166] Specifically, the blocks adjacent to the upper left of the target block (B2, B3, A2) The motion vector of the upper left control point CP0 is derived from the block to be processed. The motion vector of the upper right control point CP1 is derived from the block (B1, B0) and the target block is The motion vector of the lower left control point CP2 is derived from the block (A1, A0) adjacent to the lower left of the block. Then, the lower right control point is calculated from the coding block (T0) adjacent to the lower right of the block to be processed. Derive the motion vector for CP3.

[0167] FIG. 37 is a flowchart of affine construct merge candidate derivation.

[0168] First, the upper left control point CP0, the upper right control point CP1, the lower left control point CP2, and the lower right control point CP3 The upper left control point CP0 is derived as follows (step S3401). It is calculated by searching the B2, B3, and A2 blocks in that order. The upper right control point CP1 is It is calculated by searching for blocks with motion information in the order of priority, B1, B0 blocks. The lower left control point CP2 searches for blocks with motion information in the order of priority of A1 and A0 blocks. The lower right control point CP3 is calculated by searching the motion information of the time block. It is calculated as follows.

[0169] Next, an affine model is created using the three control points CP0, CP1, and CP2. It is determined whether construction is possible (step S3402), and if construction is possible (step S3402:YES), 3-control-point affine model using CP0, CP1, and CP2 The selected object is determined as a merge candidate (step S3403).

[0170] Next, an affine model is created using the three control points CP0, CP1, and CP3. It is determined whether construction is possible (step S3404), and if construction is possible (step S3404:YES), 3-control-point affine model using CP0, CP1, and CP3 The selected object is then determined as a merge candidate (step S3405).

[0171] Next, an affine model is created using the three control points CP0, CP2, and CP3. It is determined whether construction is possible (step S3406), and if construction is possible (step S3406:YES), 3-control-point affine model using CP0, CP2, and CP3 The selected object is determined as a merge candidate (step S3407).

[0172] Next, an affine model is created using the three control points CP1, CP2, and CP3. It is determined whether construction is possible (step S3408), and if construction is possible (step S3408:YES), 3-control-point affine model using CP1, CP2, and CP3 The selected object is determined as a merge candidate (step S3409).

[0173] Next, an affine model can be constructed using the derived CP0 and CP1. If it is possible to construct the system (step S3410), 0: YES), the two-control-point affine model by CP0 and CP1 is considered as an affine merge candidate. (step S3411).

[0174] Next, an affine model can be constructed using the derived CP0 and CP2 control points. If it is possible to construct the system (step S341), 2: YES), the two-control-point affine model by CP0 and CP2 is selected as an affine merging candidate. (step S3413).

[0175] Here, whether or not an affine model can be constructed depends on whether or not all the control points are referenced. The condition is that the reference images are the same (affine transformation is possible). Three-control-point affine model using CP2, two-control-point affine model using CP0 and CP1 For affine models other than the one with three control axes, CP0, CP1, and CP For the three-control-point affine model by 2, for the two-control-point affine model, CP0,C Convert to a two-control-point affine model using P1.

[0176] <Affine fixed merge candidate derivation> The affine fixed merge candidate derivation unit 385 will be described. The affine fixed merge candidate derivation unit 485 is similar to the affine fixed merge candidate derivation unit 385 .

[0177] The affine fixed merge candidate fixes the motion information of the control points with the fixed motion information.

[0178] Specifically, the motion vector of each control point is fixed to (0,0).

[0179] <Temporal prediction motion vector derivation> Before describing the temporal prediction motion vector, the temporal relationship of pictures is explained with reference to FIG. FIG. 49(a) shows a coding block to be processed and a picture to be processed. The relationship between previously coded pictures differs in time. We define a specific coded picture referred to in the JPEG20 ... It is identified by:

[0180] FIG. 49(b) shows that the same position as the coding block to be processed and The blocks in the block diagram are the blocks in the block diagram and the blocks in the block diagram. The T0 and T1 coding blocks shown are schematic, and their actual positions and sizes are not shown in this figure. For the coding block to be processed, the position is (xCb, yCb) and the width is cbWidt. Let h be the height and cbHeight be the height. And xColBr = xCb + cbWidth yColBr = yCb + cbHeight Calculate the coding block on ColPic that contains the position ((xColBr >> 3) << 3, (yColBr >> 3) << 3). The lock is T0. Also, xColCtr = xCb + (cbWidth >> 1) yColCtr = yCb + (cbHeight >> 1) Calculate the encoding on ColPic containing position ((xColCtr >> 3) << 3, (yColCtr >> 3) << 3). The block becomes T1.

[0181] The above explanation of the temporal relationship of pictures is for the encoding process, but the same applies for the decoding process. In other words, when decoding, the above explanation is replaced with "decoding" and the same explanation is applied. It will be revealed.

[0182] Temporal motion vector predictor candidate in the normal motion vector predictor mode derivation unit 301 in FIG. The operation of the derivation unit 322 will be described with reference to FIG.

[0183] First, ColPic is derived (step S4201). For the derivation of ColPic, see FIG. This will be explained in light of the above.

[0184] If the slice type slice_type is a B slice and the flag collocated_from_l0_flag is 0, If so (step S4211: YES, step S4212: YES), pictures at different times are The ColPic is the picture RefPicList1[0] whose reference index in the reference list L1 is 0 ( Step S4213). If not, that is, if the slice type slice_type is B slice If the flag collocated_from_l0_flag is set to 1 (step S4211: YES, Step S4212: NO), or if the slice type slice_type is a P slice ( Step S4211: NO, Step S4214: YES), is the picture RefPicList0[0] whose reference index in the reference list L0 is 0 (step If slice_type is not a P slice (step S4214: NO), the process Exit.

[0185] Refer again to FIG. 50. After deriving ColPic, derive the coding block colCb and This process will be described with reference to FIG. do.

[0186] First, in a picture ColPic at a different time, the right side of the coding block to be coded is The coding block including the lower position is set as a coding block colCb of a different time (step S 4221). An example of this coding block is shown in coding block T0 in FIG.

[0187] Next, the coding information of the coding block colCb at a different time is obtained (step S422 2) PredMode of coding block colCb of different time is not available or coding block of different time is not available. When the prediction mode PredMode of the coding block colCb is intra prediction (MODE_INTRA) ( Step S4223: NO, Step S4224: YES), The coding block including the same central lower right position as the coding block to be processed in the The coding block for time is colCb (step S4225). An example of this coding block is , as shown in coding block T1 in FIG.

[0188] Refer again to FIG. 50. Next, for each reference list, inter prediction information is derived (step Here, for the coding block colCb, The motion vector mvLXCol for each frame and the flag availableFlagLXCol indicating whether the coding information is available or not LX indicates the reference list, and in the derivation of reference list 0, LX becomes L0, and In the derivation of bit 1, LX is L1. The derivation of inter prediction information will be described with reference to FIG. do.

[0189] When a coding block colCb of a different time is not available (Step S4231: NO) or when the prediction mode PredMode is intra prediction (MODE_INTRA) (step S4232 :NO), and set both the flag availableFlagLXCol and the flag predFlagLXCol to 0 (step S 4233), and the motion vector mvLXCol is set to (0,0) (step S4234). Finish.

[0190] If the coding block colCb is available (step S4231: YES), and the prediction mode PredMod If e is not intra prediction (MODE_INTRA) (step S4232: YES), the following procedure is performed. Calculate mvCol, refIdxCol, and availableFlagCol in that order.

[0191] A flag PredFlagL0[xPCo If [yPCol] is 0 (step S4235: YES), the prediction mode of the coding block colCb is Since the current block is Pred_L1, the motion vector mvCol is the motion vector of L1 of the coding block colCb. The reference index is set to the same value as MvL1[xPCol][yPCol] (step S4236). The index refIdxCol is set to the same value as the reference index RefIdxL1[xPCol][yPCol] of L1 ( Step S4237), and the reference list listCol is set to L1 (Step S4238). Here, xPCol and yPCol are the left and right pixels of the coding block colCb in the picture ColPic at different times. The index indicating the position of the pixel above.

[0192] On the other hand, if the L0 prediction flag PredFlagL0[xPCol][yPCol] of the coding block colCb is not 0, If so (step S4235: NO), the L1 prediction flag PredFlagL1[xP Col][yPCol] is checked to see if it is 0. If [PCol] [yPCol] is 0 (step S4239: YES), the motion vector mvCol is encoded. It is set to the same value as MvL0[xPCol][yPCol], the motion vector of L0 of block colCb (step Step S4240) The reference index refIdxCol is the reference index RefIdxL0[xP Col][yPCol] is set to the same value as L0 (step S4241), and the reference list listCol is set to L0. It is set (step S4242).

[0193] L0 prediction flag PredFlagL0[xPCol][yPCol] of coding block colCb and coding block col When both of the L1 prediction flags PredFlagL1[xPCol][yPCol] of Cb are not 0 (step S4235 :NO, and S4239:NO), the inter prediction mode of the coding block colCb is bi-predictive Since the motion vector is (Pred_BI), one of the two motion vectors L0 and L1 is selected (step Top S4243).

[0194] FIG. 54 shows the code when the inter prediction mode of the coding block colCb is bi-prediction (Pred_BI). 13 is a flowchart showing a procedure for deriving inter prediction information of a coded block.

[0195] First, the POCs of all pictures registered in all reference lists are checked against the current processing target. It is determined whether the POC is smaller than the POC of the picture (step S4251). P of all pictures registered in L0 and L1, which are all reference lists of colCb If OC is smaller than the POC of the current picture to be processed (step S4251: YES), ), LX is L0, that is, the predicted vector candidate of the motion vector of L0 of the coding block to be processed. If the complement has been derived (step S4252: YES), The inter prediction information of the L1 block is selected, and LX is the motion vector of the L1 block of the coding block to be processed. If a predicted vector candidate for the vector is derived (step S4252: NO), On the other hand, the L1 inter prediction information of the coding block colCb is selected. At least one POC of a picture registered in all reference lists L0 and L1 of If the POC is greater than the POC of the current picture being processed (step S4251: NO), If the flag collocated_from_l0_flag is 0 (step S4253: YES), Select the L0 inter prediction information of colCb and set the flag collocated_from_l0_flag to 1. If NO in step S4253, the inter prediction information in the L1 direction of the coding block colCb is Select the information.

[0196] When selecting inter prediction information for L0 of coding block colCb (step S42 52: YES, or step S4253: YES), the motion vector mvCol is MvL0[xPCol] [yPCol] is set to the same value (step S4254), and the reference index refIdxCol is set to RefI The listCol is set to the same value as dxL0[xPCol][yPCol] (step S4255), and the list (step S4256).

[0197] When selecting inter prediction information for L1 of the coding block colCb (step S42 52: NO, or step S4253: NO), the motion vector mvCol is MvL1[xPCol][yPC ol] (step S4257), and the reference index refIdxCol is set to the same value as RefIdxL1 [xPCol][yPCol] are set to the same value (step S4258), and the list listCol is set to L1. The value is set (step S4259).

[0198] Returning to FIG. 53, when inter prediction information is obtained from the coding block colCb, the flag ava Both ilableFlagLXCol and flag predFlagLXCol are set to 1 (step S4244).

[0199] Next, the motion vector mvCol is scaled to obtain the motion vector mvLXCol (step The procedure for scaling calculation of this motion vector mvLXCol is shown in FIG. He explains.

[0200] The list of coding blocks colCb from the POC of the picture ColPic at different times is referred to as listCol. The POC of the reference picture corresponding to the reference index refIdxCol that The distance td is td = [POC of pictures ColPic at different times] - [List of coding blocks colCb POC of the reference picture referenced in listCol] (Step S4261). If the POC of the reference picture referenced in the list listCol of the lock colCb is earlier in the display order The inter-picture distance td is a positive value, and the coding block is closer to the picture ColPic at a different time. If the POC of the reference picture referenced in the list listCol of colCb is later in the display order, The inter-clutch distance td is a negative value.

[0201] Next, the list LX of the current picture to be processed is retrieved from the POC of the current picture to be processed. The inter-picture distance tb is calculated by subtracting the POC of the reference picture tb = [POC of current picture] - [Reference index of LX of temporal merge candidate] POC of the reference picture corresponding to the index] (Step S4262). If the reference picture in the list of pictures LX is earlier in the display order, The distance tb is a positive value, and is the reference picture referenced in the list LX of the current picture to be processed. is later in the display order, the inter-picture distance tb will be a negative value.

[0202] Next, the inter-picture distances td and tb are compared (step S4263). If td is equal to tb (step S4263: YES), the motion vector mvLXCol is mvLXCol = mvCol (step S4264), and this scaling calculation process ends.

[0203] On the other hand, if the inter-picture distances td and tb are not equal (step S4263: NO), Number of tx, tx = ( 16384 + Abs( td ) >> 1 ) / td (step S4265). Next, the scaling coefficient distScaleFactor is calculated as follows: distScaleFactor = Clip3( -4096, 4095, ( tb * tx + 32 ) >> 6 ) (step S4266). Here, Clip3(x, y, z) is calculated by taking the minimum value of x and This is a function that limits the maximum value to y. Next, the motion vector mvLXCol is mvLXCol = Clip3( -32768, 32767, Sign( distScaleFactor * mvLXCol ) * ((Abs( distScaleFactor * mvLXCol ) + 127 ) >> 8 ) ) (step S4267), and this scaling calculation process is terminated. (x) is a function that returns the sign of the value x, and Abs(x) is a function that returns the absolute value of the value x.

[0204] Again, refer to FIG. 50. Then, the motion vector mvL0Col of L0 is calculated by the normal prediction motion vector mvL0Col. The motion vector predictor candidate list mvpListLX in the motion vector mode derivation unit 301 is added as a candidate. However, this addition is not possible for the coding block in reference list 0. This is only possible when the flag availableFlagL0Col=1 indicates whether the L1 The motion vector mvL1Col is calculated by the prediction in the normal prediction motion vector mode derivation unit 301. The candidate is added to the motion vector candidate list mvpListLX (step S4205). This addition is made by the flag availabl, which indicates whether the coding block colCb in the reference list 1 is available or not. This is only the case where eFlagL1Col=1. End processing.

[0205] The above description of the normal predicted motion vector mode derivation unit 301 is for the encoding process. That is, in the normal predicted motion vector mode derivation unit 401 in FIG. The operation of the temporal motion vector predictor candidate derivation unit 422 in this embodiment is the same as that of the encoding described above. and is explained in the same manner.

[0206] <Deriving Temporal Merge Candidates> In the operation of the temporal merge candidate derivation unit 342 in the normal merge mode derivation unit 302 in FIG. This will be described with reference to FIG.

[0207] First, ColPic is derived (step S4301). Next, the coding block colCb is derived. Then, the encoding information is obtained (step S4302). The above process is a process for deriving time prediction information. Since these steps are the same as steps S4201 to S4204 in the vector candidate derivation unit 322, the explanation will be omitted. is omitted.

[0208] Next, a flag, availableFlagCol, indicating whether the coding block colCb is available or not is calculated ( Step S4305). If the flag availableFlagL0Col or the flag availableFlagL1Col is If it is 1, availableFlagCol will be 1. Otherwise availableFlagCol will be 0.

[0209] Then, the motion vector mvL0Col of L0 and the motion vector mvL1Col of L1 are calculated by the above-mentioned normal matrix. The merge candidate list mergeCandList in the merge mode derivation unit 302 is added as a candidate. (Step S4306). However, this addition does not indicate whether the coding block colCb is valid or not. The time merge candidate derivation unit 34 can determine whether the time merge candidate is available or not only when the flag availableFlagCol=1. End process 2.

[0210] The above explanation of the temporal merge candidate derivation unit 342 is for encoding, but the same applies for decoding. That is, the temporal merge candidate derivation in the normal merge mode derivation unit 402 in FIG. The operation of unit 442 is similarly described, substituting decoding for encoding in the above description.

[0211] <Update of historical motion vector prediction candidate list> Next, the coding information storage memory 111 on the coding side and the coding information storage memory 20 on the decoding side For details on how to initialize and update the historical motion vector prediction candidate list HmvpCandList in preparation for version 5, FIG. 38 is a flowchart for explaining the procedure for initializing and updating the history predicted motion vector candidate list. 1 is a flowchart showing the process of

[0212] In this embodiment, the history motion vector predictor candidate list HmvpCandList is updated based on the coding information. The information storage memory 111 and the encoded information storage memory 205 are assumed to be implemented. A history candidate list update unit is provided in the prediction unit 102 and the inter-prediction unit 203 to perform history prediction. An update of the motion vector candidate list HmvpCandList may be performed.

[0213] At the beginning of the slice, the historical motion vector prediction candidate list HmvpCandList is initialized. On the encoding side, the prediction method decision unit 105 selects the normal prediction vector mode or the normal merge mode. When the selected candidate is selected, the history motion vector predictor candidate list HmvpCandList is updated. The inter prediction mode decoded by the bit sequence decoding unit 201 is a normal prediction vector mode or In the normal merge mode, the historical motion vector predictor candidate list HmvpCandList is updated.

[0214] Inter prediction mode used for inter prediction in normal prediction vector mode or normal merge mode. The inter prediction information is used as the inter prediction information candidate hMvpCand, and the historical predicted motion vector candidate list is Register it in HmvpCandList. The inter-prediction information candidate hMvpCand contains the reference index of L0. refIdxL0 and L1 reference index refIdxL1, L0 prediction indicating whether L0 prediction is performed a prediction flag predFlagL0 indicating whether or not L1 prediction is performed; and a L1 prediction flag predFlagL1 indicating whether or not L1 prediction is performed. The motion vector mvL0 of L0 and the motion vector mvL1 of L1 are included. The motion vector history prediction memory 111 and the coding information storage memory 205 on the decoding side Among the elements (i.e., inter prediction information) registered in the candidate list HmvpCandList, If there is inter-prediction information with the same value as the inter-prediction information candidate hMvpCand, the history prediction The element is deleted from the motion vector candidate list HmvpCandList. If there is no inter prediction information with the same value as the candidate hMvpCand, the history prediction motion vector candidate The first element of the auxiliary list HmvpCandList is deleted, and the history motion vector prediction candidate list HmvpCand The inter-prediction information candidate hMvpCand is added to the end of List.

[0215] The encoding information storage memory 111 on the encoding side and the encoding information storage memory 2 on the decoding side according to the present invention The number of elements in the historical motion vector predictor candidate list HmvpCandList for 05 is six.

[0216] First, initialize the historical motion vector predictor candidate list HmvpCandList in slice units. (Step S2101 in FIG. 38). At the beginning of the slice, the history predicted motion vector candidate list Hm All elements of vpCandList are emptied and registered in the historical motion vector prediction candidate list HmvpCandList. The value of the number of recorded historical motion vector predictor candidates NumHmvpCand is set to 0.

[0217] Note that the initialization of the historical predicted motion vector candidate list HmvpCandList is performed on a slice-by-slice basis ( However, it was decided to implement it on a picture-by-picture, tile-by-tile or tree block basis. This may be done on a row-by-row basis.

[0218] Next, for each coding block in the slice, the following historical motion vector predictor candidate list Hmvp The process of updating the CandList is repeated (steps S2102 to S2107 in FIG. 38).

[0219] First, the initial setting is performed for each coding block. Set the value of the tag "identicalCandExist" to FALSE and set the index to be deleted, "removeIdx", to 0. (Step S2103 in FIG. 38).

[0220] Inter prediction information candidate hMvp to be registered in the history prediction motion vector candidate list HmvpCandList It is determined whether Cand exists (step S2104 in FIG. 38). When the determination unit 105 determines that the normal predicted motion vector mode or the normal merge mode is selected, Or, the bitstream decoding unit 201 on the decoding side uses the normal predicted motion vector mode or the normal merge motion vector mode. If the inter prediction mode is decoded as the encoding mode, the inter prediction mode is set to hMvpCand. The method determining unit 105 determines whether the intra prediction mode, the sub-block prediction motion vector mode, or the sub-block prediction motion vector mode is an intra prediction mode, When the subblock merge mode is determined, or when the bitstream decoding unit 201 on the decoding side determines that the in- tra prediction mode, sub-block predicted motion vector mode or sub-block merge mode When the motion vector prediction candidate list HmvpCandList is decoded as Therefore, the inter prediction information candidate hMvpCand to be registered does not exist. If the information candidate hMvpCand does not exist, steps S2105 to S2106 are skipped (see FIG. 38: NO in step S2104). If so, the process proceeds to step S2105 and thereafter (step S2104 in FIG. 38: YES). .

[0221] Next, the input motion vector to be registered is added to each element of the history motion vector candidate list HmvpCandList. It is determined whether an element identical to the target prediction information candidate hMvpCand exists (step S2105). Figure 39 is a flow chart of the same element confirmation process. If the value of the number of candidate vectors NumHmvpCand is 0 (step S2121 in FIG. 39: NO), The historical motion vector predictor candidate list HmvpCandList is empty and there is no identical candidate. Steps S2122 to S2125 are skipped, and the same element confirmation process ends. When the value of the number of historical motion vector predictor candidates NumHmvpCand is greater than 0 (step S2 in FIG. 39), 121:YES), the historical predicted motion vector index hMvpIdx is between 0 and NumHmvpCand-1. Then, the process of step S2123 is repeated (steps S2122 to S2125 in FIG. 39). First, the hMvpIdx-th element HmvpCandList, counting from 0 in the history motion vector predictor candidate list, t[hMvpIdx] is compared with the inter-prediction information candidate hMvpCand to see if it is the same as the candidate hMvpCand (step If they are the same (step S2123 in FIG. 39: YES), the same candidate exists. Set the value of TRUE to the flag identicalCandExist, which indicates whether the index is deleted. Set the value of hMVpIndex to the removeIdx box and end this identical element confirmation process. If so (step S2123 in FIG. 39: NO), hMvpIdx is incremented by 1, and the history predicted motion If the vector index hMvpIdx is equal to or smaller than NumHmvpCand-1, the process proceeds to step S2123. Processing is performed (steps S2122 to S2125 in FIG. 39).

[0222] Returning to the flowchart of FIG. 38, the historical motion vector predictor candidate list HmvpCandList The element shift and addition process is performed (step S2106 in FIG. 38). Step S2106: Shifting / adding elements of the history prediction motion vector candidate list HmvpCandList 1 is a flowchart of a processing procedure. First, in the historical motion vector predictor candidate list HmvpCandList, Either remove the stored elements and then add the new elements, or add the new elements without removing any elements. Specifically, the flag "identicalC" indicates whether or not an identical candidate exists. A comparison is made to see whether andExist is TRUE or NumHmvpCand is 6 (step S21 in FIG. 40). 41). The flag identicalCandExist, which indicates whether an identical candidate exists, is TRUE or If NumHmvpCand satisfies any of the 6 conditions (Step S2141 in FIG. 40: YES) Then, the elements stored in the history motion vector predictor candidate list HmvpCandList are removed, and the new Add an element that is not a member of the current element. Set the initial value of index i to the value of removeIdx + 1. The element shift process in step S2143 is repeated from NumHmvpCand to NumHmvpCand (see step S2143 in FIG. 40). (Steps S2142 to S2144). HMVPCandList[ i - 1 ] is assigned the element of HMVPCandList[ i ]. Shift the elements forward by copying (step S2143 in FIG. 40) and change i by 1 in. (Steps S2142 to S2144 in FIG. 40). When the element shift process in step S2143 is completed, the history prediction motion vector is The inter-prediction information candidate hMvpCand is added to the end of the list of inter-prediction information candidates (step S2 in FIG. 40). 145). Here, the end of the history motion vector predictor candidate list is the end of the history motion vector predictor candidate list, counting from 0 (NumHmvp This is the historical predicted motion vector candidate HMVPCandList[NumHmvpCand-1]. Shifting and adding elements of the complement list HMVPCandList is completed. Meanwhile, whether the same candidate exists is checked. Flag indicating whether or not identicalCandExist is TRUE and NumHmvpCand is 6 If neither of the above conditions is satisfied (step S2141 in FIG. 40: NO), the history of the motion vector predictor candidate list is The elements in the history motion vector predictor candidate list are not removed from the HmvpCandList. Finally, the inter-prediction information candidate hMvpCand is added (step S2146 in FIG. 40). The end of the history motion vector candidate list is the NumHmvpCand-th HMVPC andList[NumHmvpCand]. In addition, NumHmvpCand is incremented by 1 to obtain the current historical prediction. The element shift / addition process of the motion vector candidate list HMVPCandList ends.

[0223] FIG. 43 is a diagram for explaining an example of a process of updating a history prediction motion vector list. Six elements (inter prediction information) are registered in the motion vector candidate list HMVPCandList. When adding new inter prediction information, the history prediction motion vector candidate list HMVP Each element of CandList is compared with the new inter prediction information from the front (Figure 43(a)). The inter prediction information is the third one from the top of the history prediction motion vector candidate list HMVPCandList. If the value is the same as that of the element HMVP2, the element HMV P2 is deleted and the elements HMVP3 to HMVP5 are shifted forward one by one. New inter prediction information is added to the end of the vector candidate list HMVPCandList (Figure 43). (b)), and then the update of the historical motion vector predictor candidate list HMVPCandList is completed (FIG. 43(c) )).

[0224] <Historical Prediction Motion Vector Candidate Derivation Process> Next, the history of predicted motion vector candidates in the normal predicted motion vector mode derivation unit 301 on the encoding side is The prediction motion vector mode derivation unit 323 and the history prediction motion vector mode derivation unit 401 on the decoding side The process of step S304 in FIG. 20 is common to all the torque candidate derivation units 423. Method for deriving historical predicted motion vector candidates from the historical predicted motion vector candidate list HMVPCandList FIG. 41 is a flowchart for explaining a procedure for deriving a historical motion vector predictor candidate. This is a low chart.

[0225] The number of current motion vector predictor candidates numCurrMvpCand is the motion vector predictor candidate list mvpList t The maximum number of elements in LX (here, 2) or more, or the number of history motion vector predictor candidates NumHmvp If the value of Cand is 0 (step S2201 in FIG. 41: NO), The process of steps S2202 to S2209 is omitted, and the procedure of the historical motion vector predictor candidate derivation process is terminated. The number of current motion vector predictor candidates numCurrMvpCand is the motion vector predictor candidate list mvpListL If the number of elements in X is less than 2, and the number of historical motion vector predictor candidates NumHmvpCand If the value is greater than 0 (step S2201 in FIG. 41: YES), step S Processing from 2202 to S2209 is carried out.

[0226] Next, the index i goes from 0 to 3 and the number of history motion vector predictor candidates NumHmvpCand The process of steps S2203 to S2208 in FIG. 41 is repeated until the smaller value is reached ( (Steps S2202 to S2209 in FIG. 41) The current number of motion vector predictor candidates, numCurrM When vpCand is equal to or greater than 2, which is the maximum number of elements in the motion vector predictor candidate list mvpListLX (see FIG. 4), 1, step S2203: NO), and the processing of steps S2204 to S2209 in FIG. The process of deriving the historical motion vector predictor candidate is omitted and the current motion vector predictor candidate deriving process is terminated. The number of candidates, numCurrMvpCand, is the maximum number of elements in the motion vector predictor candidate list, mvpListLX, 2 If it is smaller (step S2203 in FIG. 41: YES), step S2204 in FIG. 41 The following process is carried out.

[0227] Next, the process from step S2205 to S2207 is performed when Y is 0 and 1 (L0 and L1). The process is repeated for each of the current predicted motion vectors (steps S2204 to S2208 in FIG. 41). The number of motion vector predictor candidates, numCurrMvpCand, is the maximum number of elements in the motion vector predictor candidate list, mvpListLX. If the number of the first pair is two or more (step S2205 in FIG. 41: NO), the process proceeds from step S2206 in FIG. Therefore, the process of S2209 is omitted, and the process of deriving the historical motion vector predictor candidate is terminated. The number of current motion vector predictor candidates numCurrMvpCand is the motion vector predictor candidate list mvpListLX If the number of elements is smaller than 2, which is the maximum number of elements in (step S2205 in FIG. 41: YES), The process proceeds to step S2206 and thereafter.

[0228] Next, the reference index of LY in the history motion vector predictor candidate list HmvpCandList[i] is When the reference index refIdxLX of the motion vector to be coded / decoded is the same as the reference index refIdxLX of the motion vector to be coded / decoded (step S100 in FIG. 41), Step S2206: YES) and the predicted motion vector is added as the last element of the predicted motion vector candidate list. numCurrMvpCand-th element of the vector candidate list, mvpListLX[numCurrMvpCand ], the motion vector of LY in the history motion vector predictor candidate HmvpCandList[i] is added (see the step in Figure 41). (Step S2207) Increment the number of current motion vector predictor candidates, numCurrMvpCand, by 1. The reference index of LY in the history motion vector predictor candidate list HmvpCandList[i] is When the reference index refIdxLX of the motion vector to be coded / decoded is not the same as the reference index refIdxLX of the motion vector to be coded / decoded (see FIG. 41), Step S2206: NO), and skips the additional process of step S2207.

[0229] The above steps S2205 to S2207 in FIG. 41 are performed on both L0 and L1. (Steps S2204 to S2208 in FIG. 41).

[0230] The index i is incremented by 1, and the index i becomes 3 and the history predicted motion vector candidate is If the number of complements NumHmvpCand is smaller than the smaller value, the process returns to step S2203. (Steps S2202 to S2209 in FIG. 41).

[0231] <History merge candidate derivation process> Next, the history merge candidate derivation unit 345 of the normal merge mode derivation unit 302 on the encoding side, This is a common process between the normal merge mode derivation unit 402 and the history merge candidate derivation unit 445. The history from the history merge candidate list HmvpCandList, which is the processing procedure of step S404 in The method of deriving merge candidates will now be described in detail. Figure 42 shows the history merge candidate derivation process. 1 is a flowchart for explaining the process.

[0232] First, initialization is performed (step S2301 in FIG. 42). Set the value of FALSE for each (rrMergeCand -1)th element and set the variable numOrigMergeCand to the current Sets numCurrMergeCand, the number of elements currently in the merge candidate list.

[0233] Next, set the initial value of the index hMvpIdx to 1, and then run the Then, the additional processing from step S2303 to step S2310 in FIG. 42 is repeated (FIG. 4 Steps S2302 to S2311 in 2. Elements registered in the current merge candidate list If the number of merge candidates, numCurrMergeCand, is not less than (MaxNumMergeCand-1), Since merge candidates have been added to all elements in the candidate list, the process of deriving merge candidates in this history is The process ends (step S2303 in FIG. 42: NO). If the number of elements in the merge candidate list, numCurrMergeCand, is less than or equal to (Maximum number of merge candidates, MaxNumMergeCand-1) (see Figure 42, step S2303: YES), and then the processing from step S2304 onwards is carried out.

[0234] First, the value FALSE is set to sameMotion (step S2304 in FIG. 42). The initial value of index i is set to 0, and the index i is increased from this initial value to 1 in step S2 of FIG. The process goes to steps S2306 and S2307 (S2305 to S2308 in FIG. 42).

[0235] Next, counting from 0 in the history motion vector prediction candidate list, Element HmvpCandList[NumHmvpCand-hMvpIdx] and the i-th element in the merge candidate list, counting from 0 It is compared whether the element mergeCandList[i] has the same value or not (step S2306 in FIG. 42). A merge candidate has the same value if all components of the merge candidate (inter prediction mode, The values ​​of the reference index and motion vector are the same. The processing of S2306 is performed if hMvpIdx is greater than NumHmvpCand-2 and mergeCandList[i] is not a spatial Only if it is a merge candidate and isPruned[i] is FALSE. Step S2306: YES, set both sameMotion and isPruned[i] to TRUE. (Step S2307 in FIG. 42). If the values ​​are not the same (Step S2306 in FIG. 39: O), skip the process of step S2307. When the loop process up to step S2308 is completed, check whether sameMotion is FALSE. If sameMotion is FALSE (step S2309 in FIG. 42), (P2309: YES), mergeCandList[nu mCurrMergeCand], counting from 0 in the history prediction motion vector candidate list (NumHmvpCand - hMvp Add the (HmvpCandList[NumHmvpCand - hMvpIdx])th element and set numCurrMergeCand by 1. (Step S2310 in FIG. 42). Increment the index hMvpIdx by 1. (Step S2302 in FIG. 42), and steps S2302 to S2311 in FIG. Repeat the process.

[0236] All elements in the history motion vector prediction candidate list are checked or the merge candidate list is Once all elements in the history have been added with merge candidates, the process of deriving merge candidates for this history is complete. .

[0237] <Average merge candidate derivation process> Next, the average merge candidate derivation unit 344 of the normal merge mode derivation unit 302 on the encoding side This is a common process between the average merge candidate derivation unit 444 of the normal merge mode derivation unit 402 on the side. A detailed description of the method for deriving average merging candidates, which is the processing procedure in step S403 of 21, is provided below. FIG. 62 is a flowchart illustrating the procedure of the average merging candidate derivation process.

[0238] First, an initialization process is performed (step S1301 in FIG. 62). Sets numCurrMergeCand, the number of elements currently in the merge candidate list.

[0239] Next, the merge candidate list is scanned from the top to determine two pieces of motion information. Let the index i=0 indicating the first motion information, and the index j=1 indicating the second motion information. Steps S1302 to S1303 in FIG. 62. If the number of elements, numCurrMergeCand, is not less than (the maximum number of merge candidates, MaxNumMergeCand-1), Since merge candidates have been added to all elements in the history candidate list, the history merge candidate derivation process The process is then terminated (step S1304 in FIG. 62). If the number of elements to be merged is numCurrMergeCand (the maximum number of merge candidates, MaxNumMergeCand-1), the The processing from step S1305 onwards is carried out.

[0240] The i-th motion information of the merge candidate list, mergeCandList[i], and the j-th motion information of the merge candidate list, It is determined whether or not both of the pieces of information mergeCandList[j] are invalid (step S130 in FIG. 62). 5) If both are invalid, the average merge candidates of mergeCandList[i] and mergeCandList[j] If mergeCandList[i] and mergeCandList[j] are both invalid, If not, the following process is repeated with X set to 0 and 1 (steps S1306 to S1309 in FIG. 62). 1314).

[0241] Determine whether the LX prediction of mergeCandList[i] is valid (step S1307 in FIG. 62). If the LX prediction of mergeCandList[i] is valid, then the LX prediction of mergeCandList[j] is valid. It is determined whether the LX prediction of mergeCandList[j] is valid (Step S1308 in FIG. 62). That is, if the LX prediction of mergeCandList[i] and the LX prediction of mergeCandList[j] are both valid, If so, the motion vector of the LX prediction in mergeCandList[i] and the motion vector of the LX prediction in mergeCandList[j] are The motion vector of the LX prediction obtained by averaging the motion vectors and the reference index of the LX prediction in mergeCandList[i] The average merge candidate of the LX prediction with the averaging function is calculated and set to the LX prediction of averageCand. The LX prediction of geCand is validated (step S1309 in FIG. 62). In step 08, if the LX prediction of mergeCandList[j] is not valid, i.e., the LX prediction of mergeCandList[i] If the prediction is valid and the LX prediction of mergeCandList[j] is invalid, Derive the average merge candidate of the LX prediction with the motion vector and reference index of the prediction, and The LX prediction of ageCand is set to the LX prediction of averageCand, and the LX prediction of averageCand is enabled (step S13 in FIG. 62). 10). In step S1307 of FIG. 62, if the LX prediction of mergeCandList[i] is not valid, Determine whether the LX prediction of mergeCandList[j] is valid (step S1311 in FIG. 62). ) If the LX prediction of mergeCandList[j] is valid, i.e., the LX prediction of mergeCandList[i] is If invalid and the LX prediction of mergeCandList[j] is valid, the LX prediction of mergeCandList[j] The average merge candidate of LX prediction with the motion vector and reference index of and sets the LX prediction of averageCand to valid (step S1312 in FIG. 62). In step S1311 of FIG. 62, if the LX prediction of mergeCandList[j] is not valid, i.e. That is, if both the LX prediction of mergeCandList[i] and the LX prediction of mergeCandList[j] are invalid, The LX prediction of averageCand is disabled (step S1312 in FIG. 62).

[0242] Here, LX prediction is valid when the reference index refIdxLX is 0 or greater. If LX prediction is disabled, i.e., does not exist, the reference index refIdxLX is set to -1.

[0243] The average merge candidate averageCand of the L0 prediction, L1 prediction, or BI prediction generated as above is , and add it to mergeCandList[numCurrMergeCand], the numCurrMergeCand-th merge candidate in the merge candidate list. Then, numCurrMergeCand is incremented by 1 (step S1315 in FIG. 62). , completing the process of deriving the average merging candidates.

[0244] The average merge candidate is calculated by dividing the horizontal and vertical components of the motion vector. It is averaged.

[0245] <Sub-block time merge candidate derivation> Sub-block temporal merging candidates in the sub-block merging mode derivation unit 304 in FIG. The operation of the derivation unit 381 will be described with reference to FIG.

[0246] First, it is determined whether the coding block is smaller than 8x8 pixels (step S4002).

[0247] If the coding block is smaller than 8x8 pixels (step S4002: YES), The flag indicating the presence of a temporal merge candidate is set to availableFlagSbCol=0 (step S400 3) The process of the sub-block temporal merge candidate derivation unit is completed. Here, When retemporal motion vector prediction is disabled or sub-block temporal merging is disabled, If it is prohibited, when the coding block is smaller than 8x8 pixels (step S4002: YES) will be processed the same way.

[0248] On the other hand, if the coding block is 8x8 pixels or more (step S4002: NO), Next, neighboring motion information of the coding block in the frame is derived (step S4004).

[0249] The process of deriving adjacent motion information of a coding block will be described with reference to FIG. The process of deriving adjacent motion information corresponds to the process of the spatial prediction motion vector candidate derivation unit 321 described above. However, the order of searching the adjacent blocks is A0, B0, B1, A1, and B2 is not searched. First, the coding information is obtained for adjacent block n=A0 (step S4052). The coding information includes a flag, availableFlagN, indicating whether adjacent blocks are available, a reference link, The reference index refIdxLXN and the motion vector mvLXN for each block are shown.

[0250] Next, it is determined whether the adjacent block n is valid or invalid (step S4054). If availableFlagN=1, the flag indicates whether the network is available or not. Otherwise, the network is invalid. do.

[0251] If the adjacent block n is valid (step S4054: YES), the reference index ref IdxLXN is set as the reference index refIdxLXn of the adjacent block n (step S4056). Also, the motion vector mvLXN is set as the motion vector mvLXn of the adjacent block n (step S4 056), the process of deriving adjacent motion information for the block is completed.

[0252] On the other hand, if the adjacent block n is invalid (step S4054: NO), the adjacent block n=B 0, obtain the coding information (step S4052), and determine whether adjacent block n is valid or invalid. A determination is made (step S4054). Thereafter, the same process is repeated to loop through B1 and A1 in that order. The process of deriving neighboring motion information loops until the neighboring blocks are valid, and all neighboring If blocks A0, B0, B1, and A1 are invalid, the process of deriving adjacent motion information of blocks is terminated. do.

[0253] Again, refer to FIG. 44. After the adjacent motion information is derived (step S4004), the tempo Then, a primary motion vector is derived (step S4006).

[0254] The process of deriving a temporal motion vector will be described with reference to FIG. A temporal motion vector tempMv is initialized as (0,0) (step S4062).

[0255] Next, it is determined whether the adjacent motion information is valid or invalid (step S4064). Flag indicating whether it can be used. If availableFlagN=1, it is valid, otherwise it is invalid. If the adjacent motion information is invalid (step S4064: NO), the temporal motion vector is The derivation process is completed.

[0256] On the other hand, if the adjacent motion information is valid (step S4064: YES), In the step, it is determined whether a flag predFlagL1N indicating whether L1 prediction is used is 1 or not (step If predFlagL1N=0 (step S4066: NO), the next process ( If predFlagL1N=1 (step S4066: YES), The POCs of all pictures registered in all reference lists are the same as the current picture to be processed. It is determined whether the amount is equal to or less than the POC of the chat (step S4068). If the answer is "YES" at step S4068, the process proceeds to the next step (step S4070).

[0257] If the slice type slice_type is a B slice and the flag collocated_from_l0_flag is 0, If (step S4070: YES, step S4072: YES), ColPic and the reference Picture RefPicList1[refIdxL1N] (picture with reference index refIdxL1N in reference list L1) If the determination is true (step S4074), 4074: YES), and the temporal motion vector tempMv is set to mvL1N (step S4076 If this determination is false (step S4074: NO), the next process (step S4078 ) The slice type slice_type is not a B slice and the flag collocated_from_l0_f If lag is not 0 (step S4070: NO or step S4072: NO), Proceed to the next process (step S4078).

[0258] A flag predFlagL0N indicating whether L0 prediction is used in the adjacent block N It is determined whether predFlagL0N=1 (step S4078). 78:YES), ColPic and reference picture RefPicList0[refIdxL0N] (reference of reference list L0) It is determined whether the picture with the index refIdxL0N is the same (step S4080). If this determination is true (step S4080: YES), the temporal motion vector tempMv=m vL0N (step S4082). If this determination is false (step S4080: NO), Then, the process of deriving the temporal motion vector is completed.

[0259] Again, refer to FIG. 44. Next, ColPic is derived (step S4016). This process Since this is the same as S4201 in the temporal motion vector predictor candidate derivation unit 322, is omitted.

[0260] Then, the coding block colCb of a different time is set (step S4017). is located at the bottom right of the center of the same position as the coding block to be processed in a different time picture ColPic. The coding block in which the pixel is located is set as colCb. This corresponds to the coding block T1 of 49.

[0261] Next, the position where the temporal motion vector tempMv is added to the coding block colCb is newly Let the upper left position of the coding block colCb be (xCo lCb, yColCb), and the temporal motion vector tempMv with 1 / 16 pixel accuracy (tempMv[0], tempMv[1]) And, xColCb = Clip3( xCtb, xCtb + CtbSizeY + 3, xColCb + ( tempMv[0] >> 4 ) ) yColCb = Clip3( yCtb, yCtb + CtbSizeY - 1, yColCb + ( tempMv[1] >> 4 ) ) Here, the top left position of the tree block is (xCtb, yCtb), and the top left position of the tree block is (xCtb, yCtb). The size is CtbSizeY. On ColPic containing position ((xColCb >> 3) << 3, (yColCb >> 3) << 3). The coding block of t becomes the new colCb. As shown in the above formula, the position after adding tempMv is Corrected to a range of about the size of a tree block so that it does not deviate significantly from before empMv addition If this position is outside the screen, it is corrected to be within the screen.

[0262] The prediction mode PredMode of this coding block colCb is inter prediction (MODE_INTER It is determined whether the prediction mode of colCb is not inter prediction (step S4020). If so (step S4020: NO), a flag av Set ailableFlagSbCol=0 (step S4003) and perform sub-block time merge candidate derivation. The output processing is then terminated.

[0263] On the other hand, if the prediction mode of colCb is inter prediction (step S4020: YES), Inter prediction information is derived for each reference list (steps S4022 and S4023). Now, for colCb, we use the central motion vector ctrMvLX for each reference list and LX prediction. The flag ctrPredFlagLX indicating whether the reference list is included or not is derived. LX indicates the reference list. In the derivation of reference list 0, LX is L0, and in the derivation of reference list 1, LX is L1. The derivation will be explained with reference to FIG.

[0264] When a coding block colCb of a different time is not available (Step S4112: NO) or when the prediction mode PredMode is intra prediction (MODE_INTRA) (step S4114 :NO), and set both the flag availableFlagLXCol and the flag predFlagLXCol to 0 (step S 4116), and the motion vector mvCol is set to (0,0) (step S4118). The process of deriving prediction information is then completed.

[0265] If the coding block colCb is available (step S4112: YES), and the prediction mode PredMod If e is not intra prediction (MODE_INTRA) (step S4114: YES), the following procedure is performed. Calculate mvCol, refIdxCol, and availableFlagCol in that order.

[0266] Flag PredFlagLX[xPCo If [yPCol] is 1 (step S4120: YES), the motion vector mvCol is The motion vector MvLX[xPCol][yPCol] of LX in block colCb is set to the same value (step Step S4122), the reference index refIdxCol is the reference index RefIdxLX[xPCol] of LX. [yPCol] is set to the same value (step S4124), and the list listCol is set to LX. (Step S4126). Here, xPCol and yPCol are the codes in the picture ColPic at different times. The index indicates the position of the top left pixel of the coding block colCb.

[0267] On the other hand, a flag PredFlagL indicating whether LX prediction of the coding block colCb is used or not If X[xPCol][yPCol] is 0 (step S4120: NO), the following process is performed. The POC of all pictures registered in all reference lists is the same as the current picture to be processed. It is determined whether the POC of the colCb is less than or equal to the POC of the colCb (step S4128). Determine whether the flag PredFlagLY[xPCol][yPCol] indicating whether the Here, we define LY prediction as a reference list that is different from LX prediction. , when LX=L0, LY=L1, and when LX=L1, LY=L0.

[0268] If this determination is true (step S4128: YES), the motion vector mvCol is The motion vector MvLY[xPCol][yPCol] of the LY of the locked colCb is set to the same value (step The reference index refIdxCol is the reference index RefIdxLY[xPCo [yPCol] is set to the same value as [yPCol] (step S4132), and the list listCol is set to LX. (step S4134).

[0269] On the other hand, if this determination is false (step S4128: NO), the flag availableFlagLXCol and flag predFlagLXCol are both set to 0 (step S4116), and the motion vector mvCol is set to (0 , 0) (step S4118), and the process of deriving inter prediction information ends.

[0270] When inter prediction information is obtained from the coding block colCb, the flag availableFlagLXCo Both l and the flag predFlagLXCol are set to 1 (step S4136).

[0271] Next, the motion vector mvCol is scaled to obtain the motion vector mvLXCol (step This process is performed in step S424 in the temporal motion vector predictor candidate derivation unit 322. 5, so the explanation is omitted.

[0272] Refer again to FIG. 44. After deriving the inter prediction information for each reference list, The calculated motion vector mvLXCol is the center motion vector ctrMvLX, and the calculated flag predFlagLXCol is The flag is set to ctrPredFlagLX (steps S4022 and S4023).

[0273] Then, it is determined whether the central motion vector is valid or invalid (step S4024). If dFlagL0=0 and ctrPredFlagL1=0, it is invalid. Otherwise, it is invalid. If the tolerant is invalid (step S4024: NO), the existence of a sub-block temporal merge candidate is The flag indicating the sub-block time is set to availableFlagSbCol=0 (step S4003). The processing of the merging candidate derivation unit is completed.

[0274] On the other hand, if the central motion vector is valid (step S4024: YES), the subblock The flag indicating the presence of a temporal merge candidate is set to availableFlagSbCol=1 (step S402 5) Derive sub-block motion information (step S4026). Please refer to 48 for further explanation.

[0275] First, the number of sub-blocks in the width direction is calculated from the width cbWidth and height cBheight of the coding block colCb. The number of sub-blocks numSbX and the number of sub-blocks numSbY in the height direction are calculated (step S4152). , refIdxLXSbCol=0 (step S4152). Repeat the process in units of olSb. This repetition starts with the height index ySbIdx set to 0 The process is performed while changing the width index xSbIdx from 0 to numSbX. do.

[0276] If the top left position of the coding block colCb is (xCb, yCb), then the left The position (xSb, ySb) above is xSb = xCb + xSbIdx * sbWidth ySb = yCb + ySbIdx * sbHeight Next, the temporal motion vector tempMv is added to the prediction sub-block colSb. The position thus determined is set as the new colSb (step S4154). The position is (xColSb, yColSb), and the temporal motion vector tempMv is (tempMv[0], tempMv[1]), the top left position of the new colSb is xColSb = Clip3( xCtb, xCtb + CtbSizeY + 3, xSb + ( tempMv[0] >> 4 ) ) yColSb = Clip3( yCtb, yCtb + CtbSizeY - 1, ySb + ( tempMv[1] >> 4 ) ) Here, the top left position of the tree block is (xCtb, yCtb), and the size of the tree block is As shown in the above formula, the position after adding tempMv is larger than before adding tempMv. In order to prevent the tree from collapsing, the position is corrected to within the range of the tree block size. If this position is If it is off-screen, it is corrected to be on-screen.

[0277] Then, inter prediction information is derived for each reference list (steps S4156 and S41 58). Here, for the prediction subblock colSb, the reference list is calculated for each subblock. A motion vector mvLXSbCol and a flag availableFl indicating whether the prediction sub-block is available or not Derive agLXSbCol. LX indicates the reference list. In the derivation of reference list 0, LX becomes L0. In the derivation of the reference list 1, LX is L1. The derivation of the inter prediction information is performed in S4022 of FIG. Since this is the same as S4023, the explanation will be omitted.

[0278] After deriving the inter prediction information (steps S4156 and S4158), the prediction sub-block col It is determined whether Sb is valid (step S4160). If bleFlagL1SbCol=0, colSb is invalid, otherwise it is valid. If colSb is invalid (Step S4160: NO), the motion vector mvLXSbCol is set to the center motion vector ctrMvLX. In addition, a flag predFl indicating whether or not LX prediction is used is set (step S4162). agLXSbCol is set as the flag ctrPredFlagLX for the central motion vector (step S416 2) This completes the derivation of sub-block motion information.

[0279] Again, refer to FIG. 44. Then, the motion vector mvL0SbCol of L0 and the motion vector of L1 The tor mvL1SbCol is calculated by the sub-block merge mode derivation unit 304. Add the candidate to the merge candidate list subblockMergeCandList (step S4028) However, this addition does not include the flag availableSbCol which indicates the presence of sub-block time merge candidates. = 1. With the above, the process of the temporal merge candidate derivation unit 342 ends.

[0280] The above description of the sub-block temporal merge candidate derivation unit 381 is for the encoding process. The same applies to the decoding process. The operation of the sub-block temporal merge candidate derivation unit 481 is the same as the above-described coding and decoding. and is explained in the same manner.

[0281] <Motion compensation prediction processing> The motion compensation prediction unit 306 predicts the current block being predicted during coding. The motion compensation prediction unit 306 obtains the position and size of the inter prediction information. The inter prediction mode is obtained from the inter prediction mode determination unit 305. The reference image memory is used to derive the reference coordinates and motion vectors specified by the reference index. The reference picture is moved from the same position as the image signal of the prediction block by the amount of the motion vector. After acquiring the image signal at the position, a prediction signal is generated.

[0282] In inter prediction, the inter prediction mode is a single reference picture, such as L0 prediction or L1 prediction. In the case of prediction from a reference picture, the prediction signal obtained from one reference picture is called the motion compensated prediction signal. The prediction mode is a prediction mode from two reference pictures, such as BI prediction. In the case of prediction, the weighted average of the prediction signals obtained from two reference pictures is used as the motion vector. The bi-predictive prediction signal is supplied to the prediction method determination unit. The weighted average ratio is 1:1, but other ratios may be used to perform the weighted average. For example, the closer the picture interval between the picture to be predicted and the reference picture, the The weighting ratio may be increased. Alternatively, a correspondence table of interval combinations and weighting ratios may be used.

[0283] The motion compensation prediction unit 406 has the same function as the motion compensation prediction unit 306 on the encoding side. The compensation prediction unit 406 outputs the inter prediction information to the normal prediction motion vector mode derivation unit 401, A normal merge mode derivation unit 402, a sub-block predicted motion vector mode derivation unit 403, The block merge mode is obtained from the subblock merge mode derivation unit 404 via the switch 408.

[0284] The motion compensation prediction unit 406 outputs the obtained motion compensation prediction signal to the decoded image signal superimposition unit 207. Supply.

[0285] <Inter prediction mode> The process of making predictions from a single reference picture is defined as uni-prediction, and in the case of uni-prediction, it is called L0 prediction. Or L1 prediction, which is a prediction using one of the two reference pictures registered in the reference lists L0 and L1. L0 prediction and L1 prediction are forward predictions (using the forward reference image) It can be a backward prediction (prediction that refers to a backward reference image) or a backward prediction (prediction that refers to a backward reference image). 57 and 58 are diagrams for explaining motion compensation prediction in L0 prediction (uni-prediction).

[0286] FIG. 57 shows an example in which the inter prediction mode is L0 prediction and the reference picture of L0 (RefL0Pi c) is located before the picture to be processed (CurPic). indicates that the L0 prediction is performed and the L0 reference picture is located after the current picture. Similarly, the reference pictures for L0 prediction in Figures 57 and 58 are used as reference pictures for L1 prediction. It can also be replaced with Cha (RefL1Pic) for uniprediction.

[0287] The process of making predictions from two reference pictures is defined as bi-prediction, and in the case of bi-prediction, it is called L0 prediction. 59 to 61 show the motion compensation prediction in bi-prediction. FIG. 59 is a diagram for explaining the measurement. The reference picture for L1 prediction is before the target picture and the reference picture for L1 prediction is after the target picture. FIG. 60 shows a case where the L0 prediction reference picture and the L1 prediction reference picture are bi-predictive. FIG. 61 shows a case where the reference picture for the current frame is located earlier than the picture being processed. Prediction, and the reference picture for L0 prediction and the reference picture for L1 prediction are closer to the processing target picture 4 shows the case where the time is later.

[0288] In this way, the relationship between the prediction type and time of L0 / L1 is as follows: L0 is forward prediction (forward reference image) L1 is backward prediction (prediction that references a backward reference image) and L2 is backward prediction (prediction that references a backward reference image). In the case of bi-prediction, the same reference picture can be used for L0 prediction. In addition, the motion compensation prediction may be performed in a uni-predictive or bi-predictive manner. The decision as to whether to use L0 prediction or L1 prediction is made based on, for example, whether to use L0 prediction or L1 prediction. The decision is based on the information (e.g., flags) that is displayed.

[0289] <About reference indexes> In the embodiment of the present invention, in order to improve the accuracy of the motion compensation prediction, This makes it possible to select the most suitable reference picture from among a number of reference pictures. The reference picture used in the motion compensation prediction is used as a reference index, and The index is encoded into the encoded stream along with the encoded vector.

[0290] <Motion compensation process based on normal predicted motion vector mode> The motion compensation prediction unit 306 is also shown in the inter prediction unit 102 on the encoding side in FIG. In this way, the inter prediction mode determination unit 305 performs normal prediction motion vector mode derivation. When the inter prediction information by the unit 301 is selected, the inter prediction information is - The inter prediction mode of the block currently being processed is obtained from the prediction mode determination unit 305. The prediction mode, reference index, and motion vector are derived, and a motion compensation prediction signal is generated. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0291] Similarly, the motion compensation prediction unit 406 also performs the same processing as the inter prediction unit 203 on the decoding side in FIG. As shown, during the decoding process, a switch 408 is turned on to the normal predicted motion vector mode derivation unit 40. When connected to 1, inter prediction by the normal prediction motion vector mode derivation unit 401 The information is obtained and the inter prediction mode and reference index of the currently processed block are The generated motion compensation prediction signal is then used to derive a motion vector. The signal is supplied to a decoded image signal superimposing unit 207.

[0292] <Motion compensation processing based on normal merge mode> The motion compensation prediction unit 306 is also shown in the inter prediction unit 102 on the encoding side in FIG. In this way, the inter prediction mode determination unit 305 determines whether the normal merge mode is When inter prediction information by the inter prediction mode is selected, the inter prediction information is the inter prediction mode of the block currently being processed, obtained from the inter prediction mode determination unit 305; A reference index and a motion vector are derived, and a motion compensation prediction signal is generated. The compensated prediction signal is supplied to a prediction method determination unit 105.

[0293] Similarly, the motion compensation prediction unit 406 also performs the same processing as the inter prediction unit 203 on the decoding side in FIG. As shown, during the decoding process, the switch 408 is normally connected to the merge mode derivation unit 402. If the merge mode is not set, the inter prediction information is obtained from the normal merge mode derivation unit 402, and the current process is performed. The inter prediction mode, reference index, and motion vector of the block being processed are The generated motion compensation prediction signal is multiplied by the decoded image signal. The signal is supplied to the folding unit 207.

[0294] <Motion compensation process based on sub-block predicted motion vector mode> The motion compensation prediction unit 306 is also shown in the inter prediction unit 102 on the encoding side in FIG. In this way, the inter prediction mode determination unit 305 determines the sub-block predicted motion vector model. When the inter prediction information by the code derivation unit 303 is selected, this inter prediction information The inter prediction mode determination unit 305 obtains the inter prediction mode of the block currently being processed. Derives the center prediction mode, reference index, and motion vector, and generates a motion compensated prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0295] Similarly, the motion compensation prediction unit 406 also performs the same processing as the inter prediction unit 203 on the decoding side in FIG. As shown, during the decoding process, a switch 408 controls the subblock prediction motion vector mode. When the sub-block prediction motion vector mode derivation unit 403 is connected to the The inter prediction information is obtained by the current block being processed. The generated code, reference index, and motion vector are derived to generate a motion compensation prediction signal. The resulting motion compensation prediction signal is supplied to a decoded image signal superimposing unit 207 .

[0296] <Motion compensation processing based on sub-block merge mode> The motion compensation prediction unit 306 is also shown in the inter prediction unit 102 on the encoding side in FIG. In this way, the inter prediction mode determination unit 305 includes a sub-block merge mode derivation unit When the inter prediction information by 304 is selected, the inter prediction information is Inter prediction of the block currently being processed obtained from the prediction mode determination unit 305 A mode, a reference index, and a motion vector are derived, and a motion compensation prediction signal is generated. The motion compensation prediction signal is supplied to the prediction method determination unit 105 .

[0297] Similarly, the motion compensation prediction unit 406 also performs the same processing as the inter prediction unit 203 on the decoding side in FIG. As shown, during the decoding process, a switch 408 controls the sub-block merge mode derivation unit 404 to , the inter prediction information by the subblock merge mode derivation unit 404 The inter prediction mode and reference index of the currently processed block are obtained. , derive a motion vector, and generate a motion compensation prediction signal. The generated motion compensation prediction signal is , and is supplied to the decoded image signal superimposing unit 207.

[0298] <Affine mode based motion compensation processing> In this embodiment, motion compensation based on an affine model can be used. In motion compensation by the 3D filter, two to four corners of the coding block are set as control points, and the motion vector of the control points is calculated. A motion vector for each sub-block is derived from the motion vector, and motion compensation is performed on a sub-block basis.

[0299] The following flags are determined by the inter prediction mode determination unit 305 in the encoding process. The following flags are reflected based on the inter prediction conditions and are coded into the coding stream. In the decoding process, the affine model is selected based on the following flags in the encoded stream: Specifies whether to perform motion compensation using the delta.

[0300] sps_affine_enabled_flag is the flag for motion compensation using the affine model in inter prediction. If sps_affine_enabled_flag is 0, it is possible to use The motion compensation is suppressed so that it is not based on the affine model. cu_affine_type_flag specifies the CU (coding unit) syntax of the coded video sequence. If sps_affine_enabled_flag is set to 1, the encoded video sequence is not transmitted. In this case, affine model-based motion compensation can be used.

[0301] sps_affine_type_flag is the 6-parameter affine mode for inter prediction. Indicates whether motion compensation is available. The six-parameter affine model is a function of three control points. The motion vector of each subblock is calculated from the six parameters of the horizontal and vertical components of each motion vector. In this mode, the motion compensation is performed on a subblock basis. A vector is derived, but a common reference index is derived for each coding block.

[0302] If sps_affine_type_flag is 0, the 6-parameter affine model is used for motion compensation. In addition, cu_affine_type_flag is suppressed to prevent CUs in a coded video sequence from Not transmitted in the syntax. If sps_affine_type_flag is 1, A six-parameter affine model of motion compensation is available for sequences.

[0303] If sps_affine_type_flag is not present, it shall be set to 0.

[0304] When decoding a P or B slice, the current CU If r_affine_flag is 1, generate a motion compensation prediction signal for the CU currently being processed. To achieve this, affine model motion compensation is used.

[0305] If inter_affine_flag is 0, the affine model is not included in the currently processed CU. Not used.

[0306] If inter_affine_flag is not present, it shall be set to 0.

[0307] When decoding a P or B slice, in the currently processed CU, If ffine_type_flag is 1, the motion compensation prediction signal for the currently processed CU is generated. To achieve this, motion compensation using a six-parameter affine model is used.

[0308] If cu_affine_type_flag is 0, the motion compensation prediction signal of the currently processed CU is To generate the signal, a four-parameter affine model of motion compensation is used. The data affine model is the horizontal and vertical components of the motion vectors of the two control points. The motion vector of a subblock is derived from the four parameters, and the motion compensation is performed on a subblock basis. This is the mode in which compensation is made.

[0309] <Merged Motion Vector Difference (MMVD)> The top two merge candidates (those with merge index 0 and 1 in the merge candidates list) A differential motion vector can be added to the motion vector of the candidate image. The resulting vector is called the merged differential motion vector.

[0310] When the merging motion vector difference is added in the merging candidate selection unit 347 on the encoding side, The motion vector to which the merge differential motion vector is added is determined by the inter prediction mode determination unit 305. The bitstream coding unit 108 also supplies the merged data to the motion compensation prediction unit 306 via the merged data. Information about the merged differential motion vector is encoded. , an index mmvd_distance_idx indicating the distance to be added to the motion vector, and a motion vector The index mmvd_direction_idx indicates the direction in which to add the value. These indexes are , are defined as shown in the tables in Figure 63(a) and Figure 63(b). The x and y components of the vector offset MmvdOffset are set to MmvdOffset[0] and MmvdOffset[1], respectively. This can be expressed as: MmvdOffset[0] = ( MmvdDistance << 2 ) * MmvdSign[0] MmvdOffset[1] = ( MmvdDistance << 2 ) * MmvdSign[1] The merged motion vector difference is the merged motion vector offset MmvdOffs The details of deriving the merged differential motion vector are given below for the decoding side. I will explain this in more detail.

[0311] On the decoding side, if a merged motion vector difference exists, it is provided to the bitstream decoding unit 201. Separating information about merge difference motion vectors from the bitstream provided, The merge candidate selection unit 447 also derives a motion vector offset MmvdOffset. A merge differential motion vector is derived from the coded merge differential motion vector offset. This merged difference motion vector is added to the motion vector, and then the motion vector is converted to the motion vector. The result is supplied to the compensation prediction unit 406.

[0312] The derivation of the merge differential motion vector mMvdLX in the merge candidate selection unit 447 is shown in FIG. First, the inter prediction model of the coding block is It is determined whether the mode is bi-prediction (PRED_BI) (S4402). S4402: No), and it is determined whether it is L0 prediction (PRED_L0) (S4404). In the case of (S4404: Yes), mMvdL0 = MmvdOffset mMvdL1 = 0 As a result, the process of deriving the merged motion vector difference is completed (S4406). (S4404:No), mMvdL0 = 0 mMvdL1 = MmvdOffset This completes the process of deriving a merge differential motion vector (S4408).

[0313] On the other hand, in the case of bi-prediction (S4402: Yes), the current picture currPic and the reference picture The POC difference is calculated for each reference list and designated as currPocDiffL0 and currPocDiffL1, respectively ( S4410). Here, the difference in POC between picA and picB, DiffPicOrderCnt(picA, picB), is DiffPicOrderCnt( picA, picB ) = [POC of picA] - [POC of picB] Also, the reference picture RefPicList0[refIdxL0] indicates the reference index of the reference list L0. Similarly, the reference picture RefPicList1[refIdxL1] is , which is the picture indicated by the reference index refIdxL1 in the reference list L1.

[0314] Next, it is determined whether -currPocDiffL0 * currPocDiffL1 >= 0 (step S4412 If this determination is true (step S4412: Yes), mMvdL0 = MmvdOffset mMvdL1 = -MmvdOffset This is the end of the process of deriving the merge motion vector difference (step S4414). On the other hand, if this determination is false (step S4412: No), mMvdL0 = MmvdOffset mMvdL1 = MmvdOffset Next, the absolute value of the difference in POC from the reference list L0 is calculated based on the reference list L1. If the absolute value of the difference between the POC of the first L1 and the POC of the second L2 is not less than the absolute value (step S4418), If so (step S4418: Yes), set X=0 and Y=1 (step S4420), and The differential motion vector mMvdL1 is scaled (step S4424). indicates that when Y=0, it is mMvdL0, and when Y=1, it is mMvdL1. On the other hand, if this judgment is false, If so (step S4418: No), set X=1 and Y=0 (step S4422), and The motion vector mMvdL0 is scaled (step S4424). The scaling of the mMvdLY is as shown in Figure 64(b). td = Clip3( -128, 127, currPocDiffLX ) tb = Clip3( -128, 127, currPocDiffLY ) tx = ( 16384 + Abs( td ) >> 1 ) / td distScaleFactor = Clip3( -4096, 4095, ( tb * tx + 32 ) >> 6 ) mMvdLY = Clip3( -32768, 32767, Sign( distScaleFactor * mMvdLY ) * ((Abs( distScaleFactor * mMvdLY ) + 127 ) >> 8 ) ) Here, currPocDiffLX is derived as currPocDiffL0 when X=0, and cu when X=1. Similarly, when Y=0, currPocDiffLY is currPocDiffL0. Y=1 indicates that it is currPocDiffL1. Clip3(x,y,z) is the minimum for the value z. A function that limits the value to x and the maximum value to y. Sign(x) is a function that returns the sign of the value x, and Abs(x) is a function that returns the absolute value of the value x. will end.

[0315] The merge differential motion vector is calculated based on the top two motion vectors of the subblock merge candidates. In this case, the index mmvd_dis indicating the distance to be added to the motion vector is tance_idx is defined as shown in the table in Figure 63(c). The operation of the merging candidate selection unit 386 is the same as that of the merging candidate selection unit 347, and therefore a description thereof will be omitted. The operation of the sub-block merging candidate selection unit 486 is the same as that of the merging candidate selection unit 447. Therefore, the description will be omitted.

[0316] As mentioned above, MmvdDistance is defined as shown in the tables in Figure 63(a) and Figure 63(c). These tables are defined at quarter-pixel accuracy, so the merged differential motion vectors generated are The rules may contain fractional pixel precision. However, the pixel precision in these tables is assumed to be 1. The merged motion vector difference is generated by encoding / decoding the flag indicating the merged motion vector difference on a slice-by-slice basis. The vector can be modified to not include fractional pixel precision.

[0317] <Adaptive Motion Vector Resolution (AMVR)> The resolution of the differential motion vector can be adaptively changed for each coding block. This resolution is called the adaptive motion vector resolution.

[0318] We will now explain the use of adaptive motion vector resolution for normal motion vector prediction mode. In this case, the spatial prediction motion vector candidate derivation units 321 and 421 and the temporal prediction motion vector candidate derivation units 422 and 423 are The motion vector candidate derivation units 322 and 422, the historical motion vector candidate derivation units 323 and In step 423, the derived candidate motion vectors are rounded according to the resolution. You can select from 1 / 4, 1, or 4 pixel accuracy. If you do not change the resolution, the accuracy will be 1 / 4 pixel. The processing is performed according to the resolution of the motion vector in the coding block being processed. That is, the derived candidate motion vector mvX is rightShift = leftShift = MvShift + 2 offset = 1 << ( rightShift - 1 ) mvX = ( mvX >= 0 ? ( mvX + offset ) >> rightShift : - ( ( - mvX + offset ) >> rightShift ) ) << leftShift Here, the resolution of the motion vector in the coding block to be processed is In the case of 1 / 4 pixel resolution, MvShift = 0. Similarly, in the case of motion vector resolution of 1 pixel resolution, If the resolution of the motion vector is 4 pixels, MvShift=4. The formula processes each of the x and y components of mvX.

[0319] The adaptive motion vector resolution is used for the sub-block predicted motion vector mode. In this case, only the resolution is different from the above normal predicted motion vector mode. That is, the affine inheritance predicted motion vector candidate derivation units 361 and 461 and the affine Constructed predictor motion vector candidate derivation units 362 and 462 and affine identical predictor motion vectors In the candidate derivation units 363 and 463, the motion vectors of the derived candidates are determined according to the resolution. The resolution can be selected from 1 / 16, 1 / 4, or 1 pixel accuracy. If you do not want to change the resolution, the resolution is rounded to 1 / The rounding process reduces the resolution of the motion vectors in the coding block being processed to 16 pixels. In other words, the derived candidate motion vector mvX is calculated by the above formula: Here, the resolution of the motion vector in the coding block to be processed is rounded off. In the case of 4-pixel resolution, MvShift = 0. Similarly, in the case of 1-pixel resolution of motion vectors, In this case, MvShift = 2. The above formula processes each of the x and y components of mvX.

[0320] <Triangular merge mode> The triangular merge mode is a type of merge mode that merges diagonally within the coding / decoding block. This is a mode in which motion compensation prediction is performed by dividing the image into partitions.

[0321] The triangle merge mode will be explained using Figure 65. Figure 65 shows a 16x16 triangle merge. The following shows the prediction of the encoding and decoding blocks in the triangle merge mode. The code block is divided into 4x4 subblocks, and each subblock is a uni-predictive partition. Partition 0 (UNI0), uni-predictive partition 1 (UNI1), bi-predictive partition 2 (BI) are assigned to three partitions. Here, the sub partitions above the diagonal The block goes into partition 0, and the subblock below the diagonal goes into partition 1. , and assign the diagonal subblocks to partition 2. If plit_dir is 0, partitions are allocated as shown in Figure 65(a) and merge_tr If iangle_split_dir is 1, partitions are assigned as shown in Figure 65(b). .

[0322] For motion compensation prediction of partition 0, uni-prediction is specified by merging triangle index 0. The motion information of partition 1 is used for the motion compensation prediction. The uni-prediction motion information specified in partition 1 is used for the motion compensation prediction in partition 2. , uni-prediction motion information specified by merging triangle index 0 and merging triangle index 1 The bi-predictive motion information is used by combining the uni-predictive motion information specified by

[0323] Here, the uni-prediction motion information is a set of a motion vector and a reference index, and the bi-prediction motion information is a set of a motion vector and a reference index. The motion information is composed of two pairs of motion vectors and reference indices. refers to uni-predictive motion information or bi-predictive motion information.

[0324] The merge candidate selection units 347 and 447 select the derived merge candidate list, mergeCandList is used as the triangle merge candidate list triangleMergeCandList.

[0325] The flowchart of FIG. 66 regarding derivation of triangle merging candidates will now be described.

[0326] First, we create a triangle merge candidate list called mergeCandList. Use List (step S3501). The number of candidates in the triangle merge candidate list, numTriangleMerge Cand is set to the same value as the number of merge candidates, numCurrMergeCand.

[0327] Next, derive uni-prediction motion information for the merged triangular partition (step S3502). .

[0328] FIG. 67 illustrates the derivation of uni-prediction motion information for merged triangular partitions in this embodiment. 1 is a flowchart showing the process of

[0329] In this embodiment, the merge triangular partition 0 and the merge triangular partition 1 are the same. The method derives uni-prediction motion information in the order of priority, thereby reducing the processing load.

[0330] First, for the Mth candidate in the derived merge candidate list, mergeCandList, It is determined whether the candidate has motion information in the motion information list L0 (step S3601). If M has motion information in the motion information list L0, the motion information in the motion information list L0 of the candidate M is The information is treated as a triangle merge candidate (step S3602).

[0331] Next, for the M-th candidate in the derived merge candidate list, mergeCandList, It is determined whether or not M has motion information in the motion information list L1 (step S3603). If candidate M has motion information in the motion information list L1, then candidate M's motion information in the motion information list L1 The information is treated as a triangle merge candidate (step S3604). Candidate M (M=numMergeCand-1,... , 1, 0), in descending order, step S3601, step S3602, step S36 03. Perform step S3604 to additionally derive triangle merging candidates.

[0332] FIG. 68 is a diagram illustrating an example of motion information of triangle merging candidates according to this embodiment.

[0333] Figure 68(a) shows an example of a merge candidate list. The merge candidate with merge index 0 is The inter prediction mode is bi-prediction (Pred-BI), and the motion information in the motion information list L0 is MV0 The motion information list L1 is MV0_L1. The supplementary note is that the inter prediction mode is uni-prediction (Pred-L0), and the motion information in the motion information list L0 is The merge candidate of the merge index 2 does not have the motion information of the motion information list L1. The supplementary note is that the inter prediction mode is uni-prediction (Pred-L1), and the motion information in the motion information list L0 is The motion information list L1 does not have a merge index of 3, and the motion information list L1 has a motion information of MV2_L1. The inter prediction mode of the candidate frame is bi-prediction (Pred-BI), and the motion information list is L0. The information in the motion information list L1 is MV3_L1. The merge candidate has an inter prediction mode of uni-prediction (Pred-L0) and motion information list L0. The motion information is MV4_L0, but there is no motion information in the motion information list L1.

[0334] FIG. 68(b) shows the merge triangle index for the example of the merge candidate list in FIG. 68(a). FIG. 13 is a diagram illustrating the relationship between the partition and the merge triangular partition.

[0335] The uni-prediction motion information candidates for merge triangle partition 0 are MV0_L0, MV0_L1, MV1_L0, MV2_ The motion information is composed in the order of L1, MV3_L0.

[0336] Similarly, the uni-prediction motion information candidates for merge triangle partition 1 are MV0_L0, MV0_L1, and MV1_L0. The motion information consists of the following order: MV2_L1, MV3_L0, (MV3_L1), but the merge triangular partition The motion information of merge triangle partition 0 and the motion information of merge triangle partition 1 are not identical. The uni-prediction motion information selected by merge_triangle_idx0 is removed. , derive merge triangle index 1 (merge_triangle_idx1).

[0337] In this way, the uni-prediction of merge triangle partition 0 and merge triangle partition 1 is By prioritizing the candidate merge list in the same way as the candidate merge list, an efficient triangulation is achieved. The merge mode can be transmitted with a small amount of code. The motion information of the merge triangle partition 1 is not the same as the motion information of the merge triangle partition 2. Transmitting index 0 and merge triangle index 1 will switch to triangle merge mode. The motion information of merge triangle partition 0 and the motion information of merge triangle partition 1 are not required. This eliminates redundancy, which results in identical information being transmitted, and enables the transmission of triangle merge mode with a small amount of code. .

[0338] In all the embodiments described above, the encoded bitstream output by the image encoding device is The stream is specified so that it can be decoded according to the encoding method used in the embodiment. The encoded bit stream is stored on HDD, SSD, flash memory, and other storage devices. Provided by recording it on a computer-readable recording medium such as a flash memory or optical disk. Alternatively, the information may be provided from a server via a wired or wireless network. Therefore, the image decoding device corresponding to this image coding device can reproduce this specific data regardless of the providing means. The present invention is capable of decoding encoded bitstreams in the data format.

[0339] In order to exchange encoded bitstreams between an image encoding device and an image decoding device, When a wired or wireless network is used, the data format appropriate for the transmission mode of the communication path In this case, the image coding device may convert the output bit stream into The encoded bit stream is converted into encoded data in a data format suitable for the transmission mode of the communication channel. a transmitting device that converts the encoded data into a digital signal and transmits it to a network; A receiving device is provided for restoring the encoded bit stream to an image decoding device. The receiving device includes a memory for buffering the coded bit stream output by the image coding device, A packet processing unit that packetizes the encoded bit stream and transmits the packets via a network. and a transmitting unit for transmitting the coded data. a receiving unit for receiving packetized coded data and a buffer for buffering the received coded data; The memory for storing the encoded data is a memory for processing packets to generate an encoded bit stream and for displaying the image. and a packet processing unit for providing the packet to an image decoding device.

[0340] In order to exchange encoded bitstreams between an image encoding device and an image decoding device, When a wired or wireless network is used, in addition to a transmitting device and a receiving device, Even if a relay device is provided to receive the encoded data transmitted by the transmitting device and supply it to the receiving device, The relay device includes a receiving section for receiving packetized encoded data transmitted from the transmitting device. a memory for buffering the received coded data; and a transmitting unit for transmitting the packetized coded data to the network. a reception packet processing unit that processes the received data into packets to generate an encoded bit stream; A recording medium for storing the encoded bit stream and a transmission device for packetizing the encoded bit stream. The communication packet processing unit may include a communication packet processing unit.

[0341] In addition, by adding a display unit for displaying the image decoded by the image decoding device to the configuration, In this case, the display unit may display the decoded image signal generated by the decoded image signal superimposing unit 207. The decoded image signal stored in the decoded image memory 208 is read out and displayed on the screen.

[0342] In addition, an imaging unit is added to the configuration, and the captured image is input to the image encoding device. In this case, the imaging unit inputs an image signal captured by the imaging unit to the block division unit 101. To exert effort.

[0343] The above coding and decoding processes are carried out by hardware-based transmission, storage, and reception devices. It is of course possible to realize it by using ROM (read only memory) or flash memory. Firmware stored in memory, etc., and software executed by a computer, etc. The firmware program and software program may be executed on a computer. The information may be provided by recording it on a recording medium that can be read by a computer or by a wired or wireless network. It may be provided from a server via the Internet, or it may be provided as terrestrial or satellite digital broadcasting data. It may also be provided as a broadcast.

[0344] The present invention has been described above based on the embodiments. The embodiments are merely examples, and each of the structures The combination of components and processes can be modified in many different ways, and these modifications It will be appreciated by those skilled in the art that the embodiments are within the scope of the present invention. [Explanation of symbols]

[0345] 100 image encoding device, 101 block division unit, 102 inter prediction unit, 103 intra prediction unit, 104 decoded image memory, 105 prediction method determination unit, 10 6 residual signal generation unit, 107 orthogonal transformation and quantization unit, 108 bit string encoding unit, 109 inverse quantization and inverse orthogonal transformation unit, 110 decoded image signal superposition unit, 111 coding information information storage memory, 200 image decoding device, 201 bit string decoding unit, 202 block A block division unit, 203 an inter prediction unit, 204 an intra prediction unit, and 205 coding information Storage memory, 206 inverse quantization and inverse orthogonal transformation unit, 207 decoded image signal superposition unit, 2 08 Decoded image memory.< / poc>

Claims

1. A video encoding device using a merge mode, comprising: a merge candidate list builder for building a merge candidate list including spatial merge candidates; A normal merge candidate selection unit that selects a normal merge candidate that is uni-predictive or bi-predictive from the merge candidate list; a triangle merge candidate selection unit that selects a first triangle merge candidate that is uni-predictive and a second triangle merge candidate that is uni-predictive from the merge candidate list; an encoding unit that encodes a first index that identifies the first triangle merge candidate and a second index that identifies the second triangle merge candidate; Equipped with the triangle merge candidate selection unit derives the first triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the first index, and derives the second triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the second index; the triangle merge candidate selection unit uses the first index and the second index to first determine whether the candidates in the merge candidate list include L0 motion information, and if the candidates include L0 motion information, sets the L0 motion information as a triangle merge candidate, and then determines whether the candidates in the merge candidate list include L1 motion information, and if the candidates include L1 motion information, sets the L1 motion information as a triangle merge candidate, thereby selecting the first triangle merge candidate and the second triangle merge candidate. A video encoding device comprising:

2. 1. A video encoding method using a merge mode, comprising: a merge candidate list construction step of constructing a merge candidate list including spatial merge candidates; A normal merge candidate selection step of selecting a normal merge candidate that is uni-predictive or bi-predictive from the merge candidate list; a triangle merge candidate selection step of selecting a first triangle merge candidate that is uni-predictive and a second triangle merge candidate that is uni-predictive from the merge candidate list; an encoding step of encoding a first index identifying the first triangle merge candidate and a second index identifying the second triangle merge candidate; Equipped with The triangle merge candidate selection step includes deriving the first triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the first index, and deriving the second triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the second index; The triangle merge candidate selection step uses the first index and the second index to first determine whether the candidates in the merge candidate list include L0 motion information, and if the L0 motion information is included, select the L0 motion information as a triangle merge candidate, and then determine whether the candidates in the merge candidate list include L1 motion information, and if the L1 motion information is included, select the L1 motion information as a triangle merge candidate, thereby selecting the first triangle merge candidate and the second triangle merge candidate. A video encoding method comprising:

3. A video encoding program using a merge mode, comprising: a merge candidate list construction step of constructing a merge candidate list including spatial merge candidates; A normal merge candidate selection step of selecting a normal merge candidate that is uni-predictive or bi-predictive from the merge candidate list; a triangle merge candidate selection step of selecting a first triangle merge candidate that is uni-predictive and a second triangle merge candidate that is uni-predictive from the merge candidate list; an encoding step of encoding a first index identifying the first triangle merge candidate and a second index identifying the second triangle merge candidate; Equipped with The triangle merge candidate selection step includes deriving the first triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the first index, and deriving the second triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the second index; The triangle merge candidate selection step uses the first index and the second index to first determine whether the candidates in the merge candidate list include L0 motion information, and if the L0 motion information is included, select the L0 motion information as a triangle merge candidate, and then determine whether the candidates in the merge candidate list include L1 motion information, and if the L1 motion information is included, select the L1 motion information as a triangle merge candidate, thereby selecting the first triangle merge candidate and the second triangle merge candidate. A video encoding program comprising:

4. A video decoding device using a merge mode, comprising: a decoder for decoding a first index identifying a first triangle merging candidate and a second index identifying a second triangle merging candidate; a merge candidate list builder for building a merge candidate list including spatial merge candidates; A normal merge candidate selection unit that selects a normal merge candidate that is uni-predictive or bi-predictive from the merge candidate list; a triangle merge candidate selection unit that selects the first triangle merge candidate that is uni-predictive and the second triangle merge candidate that is uni-predictive from the merge candidate list; Equipped with the triangle merge candidate selection unit derives the first triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the first index, and derives the second triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the second index; the triangle merge candidate selection unit uses the first index and the second index to first determine whether the candidates in the merge candidate list include L0 motion information, and if the candidates include L0 motion information, sets the L0 motion information as a triangle merge candidate, and then determines whether the candidates in the merge candidate list include L1 motion information, and if the candidates include L1 motion information, sets the L1 motion information as a triangle merge candidate, thereby selecting the first triangle merge candidate and the second triangle merge candidate. A video decoding device comprising:

5. A video decoding method using a merge mode, comprising the steps of: a decoding step of decoding a first index identifying a first triangle merge candidate and a second index identifying a second triangle merge candidate; a merge candidate list construction step of constructing a merge candidate list including spatial merge candidates; A normal merge candidate selection step of selecting a normal merge candidate that is uni-predictive or bi-predictive from the merge candidate list; a triangle merge candidate selection step of selecting a first triangle merge candidate that is uni-predictive and a second triangle merge candidate that is uni-predictive from the merge candidate list; Equipped with The triangle merge candidate selection step includes deriving the first triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the first index, and deriving the second triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the second index; The triangle merge candidate selection step uses the first index and the second index to first determine whether the candidates in the merge candidate list include L0 motion information, and if the L0 motion information is included, select the L0 motion information as a triangle merge candidate, and then determine whether the candidates in the merge candidate list include L1 motion information, and if the L1 motion information is included, select the L1 motion information as a triangle merge candidate, thereby selecting the first triangle merge candidate and the second triangle merge candidate. A video decoding method comprising:

6. A video decoding program using a merge mode, comprising: a decoding step of decoding a first index identifying a first triangle merge candidate and a second index identifying a second triangle merge candidate; a merge candidate list construction step of constructing a merge candidate list including spatial merge candidates; A normal merge candidate selection step of selecting a normal merge candidate that is uni-predictive or bi-predictive from the merge candidate list; a triangle merge candidate selection step of selecting a first triangle merge candidate that is uni-predictive and a second triangle merge candidate that is uni-predictive from the merge candidate list; Equipped with The triangle merge candidate selection step includes deriving the first triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the first index, and deriving the second triangle merge candidate from the merge candidate list including bi-predictive merge candidates using the second index; The triangle merge candidate selection step uses the first index and the second index to first determine whether the candidates in the merge candidate list include L0 motion information, and if the L0 motion information is included, select the L0 motion information as a triangle merge candidate, and then determine whether the candidates in the merge candidate list include L1 motion information, and if the L1 motion information is included, select the L1 motion information as a triangle merge candidate, thereby selecting the first triangle merge candidate and the second triangle merge candidate. A video decoding program comprising:

7. A method for storing a bit stream generated by the moving image encoding method according to claim 2 on a recording medium.

8. A transmission method for transmitting a bit stream generated by the moving image encoding method according to claim 2.

Images