Dynamic-image encoding device, dynamic-image encoding method, dynamic-image encoding program, dynamic-image decoding device, dynamic-image decoding method, and dynamic-image decoding program

Abstract

To provide a technique capable of improving coding efficiency by performing block division suitable for image encoding and decoding.SOLUTION: There is provided a technique capable of improving coding efficiency by performing block division suitable for image encoding and decoding. A dynamic-image encoding device performing affine transform in units of encoding blocks includes: an affine inheriting merge candidate deriving unit which derives an affine inheriting merge candidate inheriting an affine model that a block spatially proximate to a block to be encoded has; and an affine fixing merge candidate deriving unit which derives an affine fixing merge candidate in which motion information pertaining to an affine control point is fixed, the affine fixing merge candidate fixing the motion vector of each affine control point to (0,0).SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image encoding and decoding technique that divides an image into blocks and performs prediction. [Background technology]

[0002] In image encoding and decoding, the image to be processed is divided into blocks, which are groups of a predetermined number of pixels. Divide into appropriate blocks and process in blocks. By appropriately setting the inter-frame prediction and frame prediction, the coding efficiency is improved. do.

[0003] In video coding and decoding, inter-prediction is used to predict from previously coded or decoded pictures. In Patent Document 1, affine transformation is used in inter prediction, which improves coding efficiency. The technology to be applied is described. In moving images, objects undergo transformations such as enlargement, reduction, and rotation. It is not uncommon for this to happen, and by applying the technology of Patent Document 1, efficient coding This becomes possible. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 9-172644 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology in Patent Document 1 involves image conversion, which places a heavy processing load on the user. In view of the above, the present invention provides an efficient encoding technique with a low load. . [Means for solving the problem]

[0006] The embodiments described below may relate to the following aspects, for example. A first aspect is a video coding device that performs an affine transformation on a coding block basis to form a bitstream, and includes a sub-block temporal merge candidate derivation unit that derives sub-block temporal merge candidates having a motion vector for each reference list and a flag indicating whether a predicted sub-block is valid or not on a sub-block basis, an affine inheritance merge candidate derivation unit that derives affine inheritance merge candidates that inherit an affine model of a block spatially adjacent to the coding target block, and an affine construction merge candidate derivation unit that derives affine construction merge candidates from a plurality of motion information of blocks spatially or temporally adjacent to the coding target block. the affine-fixed merge candidate derivation unit that derives affine-fixed merge candidates with fixed motion information of affine control points; a sub-block merge candidate selection unit that selects a sub-block merge candidate from the sub-block temporal merge candidates, the affine inheritance merge candidates, the affine construction merge candidates, and the affine-fixed merge candidates; and an encoding unit that encodes information indicating whether or not a sub-block unit merge mode is selected and a merge index indicating the selected merge candidate, wherein the affine-fixed merge candidate derivation unit fixes the motion vectors of each affine control point of the affine-fixed merge candidates to (0,0).

[0007] A second aspect is a video coding method for performing affine transformation on a coding block basis to form a bitstream, the method including: a sub-block temporal merge candidate derivation step of deriving, on a sub-block basis, a sub-block temporal merge candidate having a motion vector for each reference list and a flag indicating whether a predicted sub-block is valid or not; an affine inheritance merge candidate derivation step of deriving an affine inheritance merge candidate that inherits an affine model of a block spatially adjacent to the coding target block; and an affine construction merge candidate derivation step of deriving an affine construction merge candidate from a plurality of motion information of blocks spatially or temporally adjacent to the coding target block. the affine-fixed merge candidate derivation step of deriving an affine-fixed merge candidate having fixed motion information of an affine control point; a sub-block merge candidate selection step of selecting a sub-block merge candidate from the sub-block temporal merge candidates, the affine inheritance merge candidates, the affine construction merge candidates, and the affine-fixed merge candidates; and an encoding step of encoding information indicating whether or not a sub-block unit merge mode is selected, and a merge index indicating the selected merge candidate, wherein the affine-fixed merge candidate derivation step fixes the motion vectors of each affine control point of the affine-fixed merge candidate to (0,0).

[0008] A third aspect is a video coding program for performing an affine transformation on a coding block basis to form a bitstream, the program including: a sub-block temporal merge candidate derivation step for deriving, on a sub-block basis, a sub-block temporal merge candidate having a motion vector for each reference list and a flag indicating whether a predicted sub-block is valid or not; an affine inheritance merge candidate derivation step for deriving an affine inheritance merge candidate that inherits an affine model of a block spatially adjacent to the coding target block; and an affine construction merge candidate derivation step for deriving an affine construction merge candidate from a plurality of motion information of blocks spatially or temporally adjacent to the coding target block. The video coding program includes an affine fixed merge candidate derivation step for deriving an affine fixed merge candidate having fixed motion information of an affine control point; a sub-block merge candidate selection step for selecting a sub-block merge candidate from the sub-block temporal merge candidates, the affine inheritance merge candidates, the affine construction merge candidates, and the affine fixed merge candidates; and an encoding step for encoding information indicating whether or not a sub-block unit merge mode is selected and a merge index indicating the selected merge candidate, wherein the affine fixed merge candidate derivation step fixes the motion vectors of each affine control point of the affine fixed merge candidate to (0,0).

[0009] A fourth aspect is a video decoding device that performs an affine transformation on a decoded block basis, the video decoding device including: a decoding unit that decodes information indicating whether or not a sub-block unit merge mode is in effect, and a merge index indicating a merge candidate; a sub-block temporal merge candidate derivation unit that derives, on a sub-block unit basis, sub-block temporal merge candidates having a motion vector for each reference list and a flag indicating whether or not a predictive sub-block is valid; an affine inheritance merge candidate derivation unit that derives affine inheritance merge candidates that inherit an affine model of a block spatially adjacent to the block to be decoded; The video decoding device includes an affine construction merge candidate derivation unit that derives affine construction merge candidates from multiple motion information of adjacent blocks, an affine fixed merge candidate derivation unit that derives affine fixed merge candidates whose motion information of affine control points is fixed, and a sub-block merge candidate selection unit that selects sub-block merge candidates from the sub-block temporal merge candidates, the affine inheritance merge candidates, the affine construction merge candidates, and the affine fixed merge candidates, wherein the affine fixed merge candidate derivation unit fixes the motion vectors of each affine control point of the affine fixed merge candidates to (0,0).

[0010] A fifth aspect is a video decoding method for performing an affine transformation on a decoding block basis, the method including: a decoding step of decoding information indicating whether or not a sub-block unit merge mode is in effect, and a merge index indicating a merge candidate; a sub-block temporal merge candidate derivation step of deriving, on a sub-block unit basis, a sub-block temporal merge candidate having a motion vector for each reference list and a flag indicating whether or not a predictive sub-block is valid; an affine inheritance merge candidate derivation step of deriving an affine inheritance merge candidate that inherits an affine model of a block spatially adjacent to the decoding target block; the affine construction merge candidate derivation step of deriving affine construction merge candidates from multiple motion information of blocks to be decoded; an affine fixed merge candidate derivation step of deriving affine fixed merge candidates having fixed motion information of affine control points; and a sub-block merge candidate selection step of selecting sub-block merge candidates from the sub-block temporal merge candidates, the affine inheritance merge candidates, the affine construction merge candidates, and the affine fixed merge candidates, wherein the affine fixed merge candidate derivation step fixes the motion vectors of each affine control point of the affine fixed merge candidates to (0,0).

[0011] A sixth aspect is a video decoding program for performing an affine transformation on a decoding block basis, the video decoding program including: a decoding step of decoding information indicating whether or not a sub-block unit merge mode is in effect, and a merge index indicating a merge candidate; a sub-block temporal merge candidate derivation step of deriving, on a sub-block unit basis, a sub-block temporal merge candidate having a motion vector for each reference list and a flag indicating whether or not a prediction sub-block is valid; an affine construction merge candidate derivation step of deriving an affine construction merge candidate from a plurality of motion information of blocks spatially or temporally adjacent to the decoding target block; The video decoding program includes an affine inheritance merge candidate derivation step of deriving an affine inheritance merge candidate that inherits an affine model held by a lock, an affine fixed merge candidate derivation step of deriving an affine fixed merge candidate in which motion information of an affine control point is fixed, and a sub-block merge candidate selection step of selecting a sub-block merge candidate from the sub-block temporal merge candidates, the affine inheritance merge candidates, the affine construction merge candidates, and the affine fixed merge candidates, and the affine fixed merge candidate derivation step fixes the motion vector of each affine control point of the affine fixed merge candidate to (0,0). Note that these descriptions are merely examples. The scope of the present application and the present invention is not limited or restricted by these descriptions. It should be understood that the description "the present invention" in this specification does not limit the scope of the present application and the present application, but is used for illustrative purposes. Effect of the Invention

[0012] According to the present invention, highly efficient image encoding / decoding processing can be realized with a low load. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram of an image encoding device according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram of an image decoding device according to an embodiment of the present invention. [Diagram 3]13 is a flowchart for explaining an operation of dividing a tree block. [Figure 4] FIG. 13 is a diagram showing how an input image is divided into tree blocks. [Diagram 5] FIG. 1 is a diagram illustrating a z-scan. [Figure 6A] FIG. 13 is a diagram showing the divided shapes of blocks. [Figure 6B] FIG. 13 is a diagram showing the divided shapes of blocks. [Figure 6C] FIG. 13 is a diagram showing the divided shapes of blocks. [Figure 6D] FIG. 13 is a diagram showing the divided shapes of blocks. [Figure 6E] FIG. 13 is a diagram showing the divided shapes of blocks. [Figure 7] 13 is a flowchart for explaining an operation of dividing a block into four. [Figure 8] 13 is a flowchart for explaining an operation of dividing a block into two or three. [Figure 9] This is a syntax for expressing the shape of block division. [Figure 10A] FIG. 1 is a diagram illustrating intra prediction. [Figure 10B] FIG. 1 is a diagram illustrating intra prediction. [Figure 11] FIG. 13 is a diagram for explaining reference blocks for inter prediction. [Figure 12] This is a syntax for expressing a coding block prediction mode. [Figure 13] FIG. 11 is a diagram showing the correspondence between syntax elements and modes relating to inter prediction. [Figure 14] FIG. 13 is a diagram for explaining affine transformation motion compensation of two control points. [Figure 15] FIG. 13 is a diagram for explaining affine transformation motion compensation of three control points. [Figure 16] FIG. 2 is a block diagram showing a detailed configuration of the inter prediction unit 102 in FIG. [Figure 17]17 is a block diagram showing a detailed configuration of a normal predicted motion vector mode derivation unit 301 in FIG. 16. [Figure 18] 17 is a block diagram showing a detailed configuration of a normal merge mode derivation unit 302 in FIG. 16. [Figure 19] 17 is a flowchart for explaining a normal predicted motion vector mode derivation process of the normal predicted motion vector mode derivation unit 301 of FIG. 16. [Figure 20] 13 is a flowchart showing a processing procedure of a normal predicted motion vector mode derivation process. [Figure 21] 13 is a flowchart illustrating a processing procedure for a normal merge mode derivation process. [Figure 22] FIG. 3 is a block diagram showing a detailed configuration of an inter prediction unit 203 in FIG. 2. [Figure 23] 23 is a block diagram showing a detailed configuration of a normal predicted motion vector mode derivation unit 401 in FIG. 22. [Figure 24] 23 is a block diagram showing a detailed configuration of a normal merge mode derivation unit 402 in FIG. 22. [Diagram 25] 23 is a flowchart for explaining a normal predicted motion vector mode derivation process of the normal predicted motion vector mode derivation unit 401 in FIG. 22. [Figure 26] FIG. 13 is a diagram illustrating a procedure for initializing and updating a history motion vector predictor candidate list. [Figure 27] 13 is a flowchart of a procedure of a process of checking identical elements in the procedure of a process of initializing and updating a history motion vector predictor candidate list. [Figure 28] 13 is a flowchart of an element shifting process procedure in the history motion vector predictor candidate list initialization / update process procedure. [Figure 29] 13 is a flowchart illustrating a procedure of a process of deriving a history predicted motion vector candidate. [Diagram 30] 13 is a flowchart illustrating a history merging candidate derivation process procedure. [Figure 31A] FIG. 11 is a diagram for explaining an example of a history motion vector predictor candidate list update process. [Figure 31B]FIG. 11 is a diagram for explaining an example of a history motion vector predictor candidate list update process. [Figure 31C] FIG. 11 is a diagram for explaining an example of a history motion vector predictor candidate list update process. [Diagram 32] FIG. 13 is a diagram illustrating motion compensation prediction in the case of L0 prediction in which an L0 reference picture (RefL0Pic) is located at a time earlier than a current picture (CurPic). [Diagram 33] 13 is a diagram for explaining motion compensation prediction in the case where L0 prediction is performed and a reference picture for L0 prediction is located at a later time than the current picture to be processed. FIG. [Diagram 34] FIG. 1 is a diagram for explaining the prediction direction of motion compensation prediction in a bi-predictive case where a reference picture for L0 prediction is at a time earlier than a current picture to be processed and a reference picture for L1 prediction is at a time later than a current picture to be processed. [Diagram 35] In the present embodiment, the prediction direction of the motion compensation prediction in the case of the bi-prediction video decoding method will be described. [Diagram 36] 13 is a diagram for explaining the prediction direction of motion compensation prediction in bi-prediction where the reference picture for L0 prediction and the reference picture for L1 prediction are at a time later than the current picture. FIG. [Figure 37] 1 is a diagram illustrating an example of a hardware configuration of a coding / decoding device according to an embodiment of the present invention; [Figure 38] 3 is a block diagram of a sub-block prediction motion vector mode derivation unit 303 in the encoding device of the present application. FIG. [Figure 39] 4 is a block diagram of a sub-block prediction motion vector mode derivation unit 403 in the decoding device of the present application. FIG. [Diagram 40] FIG. 3 is a block diagram of a sub-block merge mode derivation unit 304 in the encoding device of the present application. [Diagram 41] FIG. 4 is a block diagram of a sub-block merging mode derivation unit 404 in the decoding device of the present application. [Diagram 42] FIG. 13 is a diagram for explaining derivation of affine inheritance predicted motion vector candidates. [Diagram 43]FIG. 13 is a diagram for explaining derivation of affine constructed predicted motion vector candidates. [Diagram 44] FIG. 13 is a diagram for explaining affine inheritance merge candidate derivation. [Diagram 45] FIG. 13 is a diagram illustrating affine construction merge candidate derivation. [Figure 46] 13 is a flowchart of deriving an affine inheritance predicted motion vector candidate. [Figure 47] 13 is a flowchart of deriving an affine constructed predicted motion vector candidate. [Figure 48] 13 is a flowchart of affine inheritance merge candidate derivation. [Figure 49] 13 is a flowchart of affine construct merge candidate derivation. [Figure 50] 13 is a flowchart for explaining the operation of the sub-block temporal merge candidate derivation unit 381. [Figure 51] 13 is a flowchart illustrating a process of deriving adjacent motion information of a block. [Figure 52] 13 is a flowchart illustrating a process of deriving a temporal motion vector. [Figure 53] 13 is a flowchart illustrating derivation of inter prediction information. [Figure 54] 13 is a flowchart illustrating a process of deriving sub-block motion information. [Figure 55] FIG. 2 is a diagram for explaining the temporal relationship between pictures. [Figure 56] 13 is a flowchart for explaining a process of deriving a temporal motion vector predictor candidate in a normal motion vector predictor mode derivation unit 301. [Figure 57] 13 is a flowchart for explaining the derivation process of ColPic in the derivation process of a temporal motion vector predictor candidate in the normal motion vector predictor mode derivation unit 301. [Figure 58] 13 is a flowchart for explaining a process of deriving coding information of ColPic in a process of derivation of a temporal motion vector predictor candidate in the normal motion vector predictor mode derivation unit 301. [Figure 59] 13 is a flowchart illustrating a process of deriving inter prediction information. [Figure 60] 13 is a flowchart showing the procedure of a process of deriving inter prediction information of a coding block when the inter prediction mode of the coding block colCb is bi-prediction (Pred_BI). [Figure 61] 13 is a flowchart illustrating a procedure of a motion vector scaling calculation process. [Figure 62] 13 is a flowchart illustrating a process of deriving a temporal merge candidate. [Figure 63] 13 is a diagram showing a comparison target of motion information in deriving affine construction merge candidates; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The technologies and technical terms used in this embodiment will be defined below.

[0015] <Tree Block> In this embodiment, the image to be encoded / decoded is divided equally into a predetermined size. In Figure 4, the size of the tree block is 128x128 pixels. However, the size of the tree block is not limited to this and can be any size. The processing target (the encoding target in the encoding process, and the decoding target in the decoding process) may be set. The tree blocks of the tree are sorted in raster scan order, i.e., from left to right and from top to bottom. The order is as follows: Inside each tree block, further recursive partitioning is possible. After recursively dividing the tree block, the block to be coded and decoded is called the coding block. In addition, the tree block and the coding block are collectively defined as blocks. By dividing the blocks appropriately, efficient coding becomes possible. The size of the code can be a fixed value previously agreed upon between the coding device and the decoding device, or The size of the tree block determined by the encoding device may be transmitted to the decoding device. Here, the maximum size of the tree block is set to 128x128 pixels, and the tree block size is set to The minimum size of the coding block is 16x16 pixels. The maximum size of the coding block is 64x64. The minimum size of a pixel,coding block is set to 4x4 pixels.

[0016] <Prediction mode> An image prediction algorithm that predicts the processed image signal of the image to be processed on a per-block basis. Inter prediction (MODE_INTRA), which predicts from the image signal of the processed image, and Inter prediction (MODE_M DE_INTER). In the coding process, a processed image is an image obtained by decoded a signal that has been coded. It is used for codes, tree blocks, blocks, coding blocks, etc., and in the decoding process, is used for completed images, image signals, tree blocks, blocks, coding blocks, etc. . This predicts the mode to distinguish between intra prediction (MODE_INTRA) and inter prediction (MODE_INTER). The prediction mode (PredMode) is defined as the intra prediction (MODE_INTRA ), or inter prediction (MODE_INTER).

[0017] <Inter Prediction> Inter-prediction, which predicts from the image signal of a processed image, refers to multiple processed images. It can be used as a reference picture. To manage multiple reference pictures, L0 (reference link) Two types of reference lists, L0 (reference list 0) and L1 (reference list 1), are defined, and the reference indexes are The L0 prediction (Pred_L0) is available for P slices. For B slices, L0 prediction (Pred_L0), L1 prediction (Pred_L1), and bi-prediction (Pred_BI ) is available. L0 prediction (Pred_L0) refers to the reference picture managed by L0. Inter prediction is used to predict the frame rate of a frame, and L1 prediction (Pred_L1) uses a reference picture managed by L1. This is the inter prediction to refer to. Bi-prediction (Pred_BI) is performed with both L0 prediction and L1 prediction, Inter prediction refers to one reference picture managed in L0 and one in L1. Information specifying L0 prediction, L1 prediction, and bi-prediction is defined as an inter prediction mode. In the following processing, the constants and variables with the subscript LX in the output are L0, L1 It is assumed that processing is performed for each

[0018] <Predictive motion vector mode> The predicted motion vector mode is an index for specifying a predicted motion vector, Transmits the inter prediction vector, inter prediction mode, and reference index, and the index of the block to be processed. The predicted motion vector is determined based on the motion vector of the block adjacent to the current block. A processed block that belongs to the same image as the target block, or a block that belongs to the processed image and has the same position as the target block A candidate predicted motion vector is derived from a block located at or near the block. The measured motion vector is derived from an index for identifying the measured motion vector.

[0019] <Merge mode> In the merge mode, the difference motion vector and reference index are not transmitted, and the target block is merged. The processed blocks adjacent to the block to be processed or blocks belonging to the processed image The inter prediction information of the block located at the same position as the block or in its vicinity (neighborhood) is used for processing. This is a mode for deriving inter prediction information for the current block.

[0020] The processed blocks adjacent to the block to be processed and the inter-blocks of the processed blocks The spatial merge candidate is defined as a block belonging to the processed image. Blocks that are in the same location as the block or in its vicinity (neighborhood), and the index of that block Inter prediction information derived from the super prediction information is defined as a temporal merge candidate. The complement is registered in the merge candidate list, and the prediction of the target block is determined by the merge index. Identify merge candidates to use in

[0021] <Adjacent block> FIG. 11 shows the procedure for deriving inter prediction information in the predicted motion vector mode and the merge mode. A0, A1, A2, B0, B1, B2, B3 is a processed block adjacent to the block to be processed. T0 is the processed image. The block belongs to the same position as the processing target block in the processing target image or its vicinity ( It is a block located in the neighborhood.

[0022] A1 and A2 are located to the left of the coding block to be processed and are adjacent to the coding block to be processed. B1 and B3 are adjacent blocks located above the coding block to be processed. A0, B0, and B2 are the blocks adjacent to the target coding block. These are the blocks located at the bottom left, top right, and top left of the encryption block.

[0023] Details of how adjacent blocks are treated in predictive motion vector mode and merge mode The details will be described later.

[0024] <Affine transformation motion compensation> In the affine transformation motion compensation, a coding block is divided into sub-blocks of a predetermined unit, and the divided sub-blocks are The motion vector is determined individually for each sub-block and motion compensation is performed. The motion vector of each sub-block is determined based on the processed blocks adjacent to the current block, or is a block that belongs to the processed image and is located at the same position as the block to be processed or in its vicinity (neighborhood) based on one or more control points derived from inter prediction information of a block located at In this embodiment, the size of the sub-block is set to 4x4 pixels. The size is not limited to this, and the motion vector may be derived on a pixel-by-pixel basis.

[0025] FIG. 14 shows an example of affine transformation motion compensation for two control points. A control point has two parameters, a horizontal component and a vertical component. Therefore, The affine transformation in these two cases is called a four-parameter affine transformation. CP2 is the control point. FIG. 15 shows an example of affine transformation motion compensation for three control points. A control point has two parameters, a horizontal component and a vertical component. Therefore, The affine transformation in the three cases is called a six-parameter affine transformation. CP2 and CP3 are control points.

[0026] Affine transformation motion compensation is available in both the predicted motion vector mode and the merge mode. It is also available in the predictive motion vector mode. The mode in which the affine transformation is performed is defined as the subblock prediction motion vector mode, and the merge mode is A mode in which motion compensation is applied is defined as a sub-block merging mode.

[0027] <Inter prediction syntax> The syntax relating to inter prediction will be described with reference to FIGS. The merge_flag in FIG. 12 indicates whether the current coding block is to be processed in merge mode or in predictive motion vector mode. The merge_affine_flag is a flag that indicates whether to use the merge mode. This is a flag indicating whether or not to apply the sub-block merging mode to the current coding block. The inter_affine_flag is a sub-block of the coding block to be processed in the predicted motion vector mode. A flag indicating whether to apply the affine motion vector prediction mode. is a flag for determining the number of control points in the sub-block predicted motion vector mode. It is. Figure 13 shows the values ​​of each syntax element and the corresponding prediction method. ag=1,merge_affine_flag=0 corresponds to the normal merge mode. merge_flag=1,merge_affine_flag=1 is a merge mode that is not a subblock merge. merge_flag=0,inter_affine_flag=0 corresponds to the normal prediction mode. The normal predicted motion vector mode corresponds to the sub-block predicted motion vector mode. merge_flag=0,inter_affine_flag=1 corresponds to the sub-block predicted motion vector mode. merge_flag=0,inter_affine_flag If =1, then cu_affine_type_flag is also transmitted to determine the number of control points.

[0028] <poc> POC (Picture Order Count) is a variable associated with the picture being coded. The value is set to a value that increases by 1 according to the picture output order. It can also determine whether a picture is the same as another picture, determine the order of pictures in the output, and For example, if two pictures have the same POC, we can calculate the distance between them. If the POCs of two pictures have different values, they are considered to be the same picture. If it does, the picture with the smaller POC value is determined to be the picture to be output first. The difference between the POCs of two pictures indicates the distance between the pictures along the time axis.

[0029] (First embodiment) Regarding the image encoding device 100 and the image decoding device 200 according to the first embodiment of the present invention, He explains.

[0030] FIG. 1 is a block diagram of an image coding device 100 according to a first embodiment. The image encoding device 100 in this embodiment includes a block division unit 101, an inter prediction unit 102, an intra prediction unit 103, and an A prediction unit 103, a decoded image memory 104, a prediction method determination unit 105, a residual generation unit 106, and an orthogonal A transform / quantization unit 107, a bit string encoding unit 108, an inverse quantization / inverse orthogonal transformation unit 109, and a decoding unit 109. The image signal superimposing unit 110 and the encoded information storage memory 111 are provided.

[0031] The block division unit 101 recursively divides an input image to generate coding blocks. The block division unit 101 divides the blocks to be divided into horizontal and vertical blocks. The four divisions into which the blocks are divided and the direction in which the blocks are divided, either horizontally or vertically. The block division unit 101 processes the generated coding blocks. The image signal of the processing target coding block is input to the inter prediction unit. 102, the intra prediction unit 103, and the residual generation unit 106. The unit 101 supplies information indicating the determined recursive division structure to the bit string encoding unit 108. The detailed operation of the lock division unit 101 will be described later.

[0032] The inter prediction unit 102 performs inter prediction on the coding block to be processed. The measurement unit 102 receives the inter-prediction information stored in the encoded information storage memory 111 and the A plurality of inter-prediction information are obtained from the decoded image signal stored in the image memory 104. and selecting an appropriate inter prediction mode from among the derived candidates. and a prediction image signal according to the selected inter prediction mode. The inter prediction unit 102 supplies the prediction method to the prediction method determination unit 105. The detailed configuration and operation of the inter prediction unit 102 will be described later. .

[0033] The intra prediction unit 103 performs intra prediction on the coding block to be processed. The measurement unit 103 measures the decoded image signal stored in the decoded image memory 104 as a reference pixel. The coding information storage memory 111 stores the intra-prediction mode and other codes. A predicted image signal is generated by intra prediction based on the encoding information. The intra prediction unit 103 selects an appropriate intra prediction mode from among a plurality of intra prediction modes. Then, the selected intra prediction mode and a predicted image according to the selected intra prediction mode are The image signal is supplied to a prediction method determination unit 105 . 10A and 10B show examples of intra prediction. For example, the intra prediction mode 5 0 generates an intra-predicted image by copying reference pixels vertically. Prediction mode 1 is a DC mode, in which all pixel values ​​of the processing target block are compared with the reference pixels. Intra prediction mode 0 is a planar mode, and vertical This mode creates a two-dimensional intra-prediction image from reference pixels in the vertical and horizontal directions. 10B is an example of generating an intra-prediction image in the case of intra-prediction mode 40. The intra prediction unit 103 performs intra prediction on each pixel of the processing target block in a direction indicated by the intra prediction mode. The intra prediction unit 103 copies the values ​​of the reference pixels in the intra prediction mode. If the pixel position is not an integer, the reference pixel value is determined by interpolation from the reference pixel values ​​of the neighboring integer positions. Determine.

[0034] The decoded image memory 104 stores the decoded image generated by the decoded image signal superimposing unit 110 . The decoded image memory 104 transfers the stored decoded images to the inter prediction unit 102 and the intra prediction unit 103. The measurement unit 103 supplies the signal.

[0035] The prediction method determination unit 105 determines the coding information for each of the intra prediction and the inter prediction. The evaluation is performed using the amount of coding information and residuals, the amount of distortion between the predicted image signal and the image signal to be processed, etc. In the case of intra prediction, the prediction method determination unit 10 5 is a bitstream encoding unit 1 which encodes intra prediction information such as an intra prediction mode as encoding information. In the case of the inter prediction merge mode, the prediction method determination unit 105 supplies the merge sub-block merge mode (sub-block merge flag) The inter prediction information such as the inter-frame coding information is supplied to the bit stream coding unit 108 as coding information. In the case of the predicted motion vector mode of the inter prediction, the prediction method determination unit 105 selects the inter prediction mode. code, predicted motion vector index, L0, L1 reference index, differential motion vector information indicating whether the mode is a sub-block predicted motion vector mode (sub-block predicted motion vector mode) The inter prediction information such as the bit stream (transform flag) is supplied to the bit stream coding unit 108 as coding information. Furthermore, the prediction method determination unit 105 stores the determined coding information in the coding information storage memory 11. The prediction method determination unit 105 outputs the residual signal to the residual generation unit 106 and the prediction image signal to the decoded image 1. The signal is supplied to the signal superimposing unit 110.

[0036] The residual generation unit 106 generates a residual by subtracting a predicted image signal from an image signal to be processed. The generated signal is supplied to the orthogonal transform and quantization unit 107.

[0037] The orthogonal transform and quantization unit 107 performs orthogonal transform and quantization on the residual in accordance with the quantization parameter. The residual is then subjected to orthogonal transformation and quantization, and the generated residual is then input to a bit string coding unit 10. 8 and the inverse quantization and inverse orthogonal transformation unit 109.

[0038] The bitstream coding unit 108 encodes the bitstream in units of sequences, pictures, slices, and coding blocks. In addition to the information, the prediction method determined by the prediction method determination unit 105 for each coding block is Specifically, the bit string coding unit 108 encodes the coding information according to the coding block. The prediction mode PredMode for each block is coded. If the prediction mode is inter prediction (MODE_INTER), In this case, the bitstream encoding unit 108 sets a flag for determining whether or not the merge mode is selected, merge flag, merge index if in merge mode, index if not in merge mode Prediction mode, predicted motion vector index, information on differential motion vector, Block prediction motion vector flags and other coding information (inter prediction information) are stored in a specified syntax. The first bitstream is encoded according to the bitstream syntax rules. If MODE_INTRA is intra prediction, the coding information such as intra prediction mode (intra prediction The first bit is encoded according to a prescribed syntax (syntax rules for bit strings). The bit string coding unit 108 also generates a bit string by defining the orthogonal transformed and quantized residual. The bit string is then entropy coded according to the syntax: The encoding unit 108 multiplexes the first bit string and the second bit string according to a specified syntax. and outputs the bitstream.

[0039] The inverse quantization and inverse orthogonal transformation unit 109 performs the orthogonal transformation supplied from the orthogonal transformation and quantization unit 107. The quantized residual is inversely quantized and inversely orthogonally transformed to calculate the residual, and the calculated residual is converted into a decoded image. The image signal is supplied to the image signal superimposing unit 110.

[0040] The decoded image signal superimposing unit 110 outputs a predicted image signal according to the determination by the prediction method determining unit 105. The residuals that have been inverse quantized and inverse orthogonally transformed by the inverse quantization and inverse orthogonal transformation unit 109 are superimposed to form a decoded image. The decoded image signal superimposing unit 110 generates a decoded image and stores it in the decoded image memory 104. The image is filtered to reduce block distortion and other distortions caused by coding. The decoded image may then be stored in the decoded image memory 104 .

[0041] The coding information storage memory 111 stores the prediction mode (inference mode) determined by the prediction method determination unit 105. In the case of inter prediction, the code is stored. The coding information stored in the coding information storage memory 111 includes the determined motion vector, the reference list, Inter prediction information such as L0 and L1 reference indexes, and historical prediction motion vector candidate list In the case of the merge mode of the inter prediction, the coding information storage memory 111 includes The encoded information to be stored includes the merge index, sub-block markers, and the above information. Inter prediction information is included, such as information indicating whether the sub-block is in the multi-mode or not (sub-block merge flag). In the case of the inter-prediction predicted motion vector mode, the coding information storage memory 111 The stored coding information includes the above information as well as inter prediction mode, predicted motion vector, etc. An index, a differential motion vector, and information indicating whether or not the sub-block prediction motion vector mode is used. It includes inter prediction information such as intra block prediction information (sub-block prediction motion vector flag). In the case of prediction, the coding information stored in the coding information storage memory 111 includes the determined input. The JPEG 2000 includes intra prediction information such as intra prediction mode.

[0042] FIG. 2 shows the configuration of an image decoding device according to an embodiment of the present invention, which corresponds to the image encoding device of FIG. The image decoding device according to the embodiment includes a bit stream decoding unit 201, a block diagram showing a configuration of the image decoding device. A block division unit 202, an inter prediction unit 203, an intra prediction unit 204, and an encoding information storage memory 205, an inverse quantization and inverse orthogonal transformation unit 206, a decoded image signal superimposition unit 207, and a decoded image memory unit 208. Equipped with 208 harpoons.

[0043] The decoding process of the image decoding device of FIG. 2 is performed by the decoding device provided inside the image encoding device of FIG. Since this corresponds to the processing, the coding information storage memory 205 in FIG. The configurations of the conversion unit 206, the decoded image signal superimposition unit 207, and the decoded image memory 208 are shown in FIG. The image encoding device includes an encoding information storage memory 111, an inverse quantization and inverse orthogonal transformation unit 109, and a decoding unit 114. The configurations of the decoded image signal superimposing unit 110 and the decoded image memory 104 and their corresponding functions has.

[0044] The bit stream supplied to the bit stream decoder 201 conforms to the prescribed syntax rules. The bit stream decoding unit 201 decodes the separated first bit stream and Sequence, picture, slice, coding block unit information, and coding block unit Specifically, the bit stream decoding unit 201 obtains the coding information by inter-coding the coding block unit. -Prediction mode PredMode that determines whether prediction is (MODE_INTER) or intra prediction (MODE_INTRA) When the prediction mode is inter prediction (MODE_INTER), the bitstream decoding unit 201 Flag to determine whether or not the mode is merged. If it is merged, the merge index and sub Block merge flag, Inter prediction mode if predictive motion vector mode, prediction Motion vector index, differential motion vector, sub-block predicted motion vector flag, etc. The coding information (inter prediction information) related to is decoded according to a specified syntax and coded Information (inter prediction information) is sent to the inter prediction unit 203 and the block division unit 202. The prediction mode is intra prediction (MODE_INTRA). In this case, the coding information (intra prediction information) such as intra prediction mode is specified in the specified syntax. Therefore, the encoded information (intra prediction information) is decoded and fed to the inter prediction unit 203 or the intra prediction unit 204. The signal is supplied to the coding information storage memory 205 via the prediction unit 204 and the block division unit 202. The bit stream decoding unit 201 decodes the separated second bit stream and converts it into an orthogonal transformed and quantized bit stream. The residual that has been orthogonally transformed and quantized is supplied to the inverse quantization and inverse orthogonal transformation unit 206. do.

[0045] The inter prediction unit 203 detects that the prediction mode PredMode of the coding block to be processed is inter When the prediction motion vector mode is set to prediction (MODE_INTER), the coding information storage memory 205 A plurality of predictive motion vectors are generated by using the coding information of the already decoded image signal stored in The motion vector predictor candidates are derived, and the derived motion vector predictor candidates are used as the motion vector predictor candidates described later. The inter prediction unit 203 registers the motion vector predicted candidate in the motion vector predicted candidate list. Among the multiple motion vector predictor candidates recorded in the bitstream, the bitstream decoder 201 decodes and supplies A motion vector predictor corresponding to the motion vector predictor index is selected, and a bit stream is decoded. A motion vector is calculated from the differential motion vector decoded by the unit 201 and the selected predicted motion vector. The calculated motion vector is stored in the coded information storage memory 205 together with other coded information. The coding information of the coding block to be supplied and stored here is the prediction mode PredMode , flags predFlagL0[xP][yP], predFlag indicating whether to use L0 prediction and L1 prediction L1[xP][yP], L0, L1 reference indices refIdxL0[xP][yP], refIdxL1[xP][yP], L0 , L1 motion vectors mvL0[xP][yP], mvL1[xP][yP], etc., where xP and yP are the The Prediction Mode PredMo is an index indicating the position of the top left pixel of the coding block in the Prediction Mode PredMo. When de is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0) A flag predFlagL0 indicating whether to use L0 prediction is 1, and a flag predFlagL0 indicating whether to use L1 prediction is 2. The flag predFlagL1 indicating this is 0. When the inter prediction mode is L1 prediction (Pred_L1), The flag predFlagL0, which indicates whether to use L0 prediction, is set to 0, and the flag predFlagL0, which indicates whether to use L1 prediction, is set to 1. The flag predFlagL1 is set to 1. When the inter prediction mode is bi-prediction (Pred_BI), Flag indicating whether prediction is used. predFlag: Flag indicating whether L0, L1 prediction is used. predFlagL1 and predFlagL2 are both set to 1. Furthermore, the prediction mode PredMode of the coding block to be processed is Derives merge candidates when in merge mode with inter prediction (MODE_INTER). Using the coding information of the already decoded coding block stored in the storage memory 205 , multiple merge candidates are derived and registered in a merge candidate list described later. The merging candidates are decoded by the bitstream decoding unit 201 and supplied from among the multiple merging candidates registered in the Select a merging candidate corresponding to the merging index, and perform L0 prediction of the selected merging candidate. Flags predFlagL0[xP][yP], predFlagL1[xP][yP], and L0 , the reference indices of L1 refIdxL0[xP][yP], refIdxL1[xP][yP], the motion vectors of L0 and L1 Inter prediction information such as mvL0[xP][yP] and mvL1[xP][yP] is stored in the coding information storage memory 205. Here, xP and yP are the indices that indicate the position of the top left pixel of the coding block in the picture. The detailed configuration and operation of the inter prediction unit 203 will be described later.

[0046] The intra prediction unit 204 detects whether the prediction mode PredMode of the coding block to be processed is intra. In the case of prediction (MODE_INTRA), intra prediction is performed. The encoding information includes an intra-prediction mode. The intra-prediction unit 204 performs bitstream decoding. The decoded image memory 202 is generated according to the intra-prediction mode included in the encoded information decoded by the decoding unit 201. A predicted image signal is generated by intra-prediction from the decoded image signal stored in the memory 208. The intra prediction unit 206 generates a prediction image signal and supplies the generated prediction image signal to the decoded image signal superimposition unit 207. 204 corresponds to the intra prediction unit 103 of the image encoding device 100. The same processing as that of the intra-prediction unit 103 is performed.

[0047] The inverse quantization and inverse orthogonal transformation unit 206 performs orthogonal transformation and quantization on the bit string decoded by the bit string decoding unit 201. The inverse orthogonal transform and inverse quantization are performed on the inverse orthogonal transformed and inverse quantized residual. get.

[0048] The decoded image signal superimposing unit 207 receives the predicted image signal inter-predicted by the inter prediction unit 203. 203, or a predicted image signal obtained by intra-prediction in the intra-prediction unit 204, and The decoded image is obtained by superimposing the residual that has been inversely orthogonally transformed and inversely quantized by the inverse transform unit 206. The decoded image signal is stored in the decoded image memory 208. When storing the decoded image in the memory 208, the decoded image signal superimposing unit 207 performs coding on the decoded image. After filtering to reduce block distortion, the decoded image is stored in the decoded image memory 208. It may be stored.

[0049] Next, the operation of the block division unit 101 in the image coding device 100 will be described. Figure 3 shows the division of an image into treeblocks and the further division of each treeblock. First, the input image is divided into tree blocks of a given size. Each tree block is sorted in a predetermined order, i.e., in a raster order (step S1001). The tree block is scanned in the scan order (step S1002) and the inside of the tree block to be processed is divided. (Step S1003).

[0050] FIG. 7 is a flowchart showing the detailed operation of the division process in step S1003. Then, it is determined whether or not the block to be processed is to be divided into four (step S1101).

[0051] If it is determined that the processing block should be divided into four, the processing block is divided into four (step (Step S1102). For each block into which the processing target block is divided, That is, the upper left, upper right, lower left, and lower right are scanned in this order (step S1103). FIG. 6A shows an example of a scan order in which the processing target block is divided into four. The numbers 0 to 3 in 601 of A indicate the order of processing. For each divided block, the division process of FIG. 7 is recursively performed (step S1104 ).

[0052] If it is determined that the processing target block is not to be divided into four, it is divided into two or three (step S1 105).

[0053] FIG. 8 is a flowchart showing the detailed operation of the 2-3 division process in step S1105. First, we decide whether to divide the block to be processed into 2 or 3 parts, i.e., how many parts to divide into 2 or 3 parts. It is determined whether or not to perform either of these (step S1201).

[0054] When it is not determined that the processing block should be divided into 2-3 blocks, i.e., when it is determined that the processing block should not be divided In other words, the division is terminated by the recursive division process (step S1211). No further recursive division processing is performed on the blocks that have been divided.

[0055] If it is determined that the block to be processed should be divided into 2-3, the block to be processed is further divided into 2 It is determined whether or not to divide (step S1202).

[0056] If it is determined that the processing block should be divided into two, the processing block is divided into upper and lower parts (vertical ) (step S1203), and based on the result, Either divide the block vertically into two (step S1204), or Divide the image into two on the right (horizontal direction) (step S1205). The target block is divided into two parts, top and bottom (vertical direction), as shown in 602 of FIG. 6B, and the steps are As a result of the step S1205, the processing target block is divided into left and right (horizontal) blocks as shown in 604 of FIG. 6D. (direction) is divided into two parts.

[0057] In step S1202, if it is not determined that the block to be processed is to be divided into two In other words, if it is determined that the block should be divided into three parts, the block to be processed is divided into upper, middle and lower parts (vertically). Based on the result of the determination, the processing target block is moved up or down. Divide the target block into three vertically (step S1207), or divide the target block into left, center, right, As a result of step S1207, the processing object is divided into three parts (horizontal direction) (step S1208). The block is divided into three parts, top, middle and bottom (vertical direction), as shown in 603 of FIG. 6C. As a result of step S1208, the processing target block is divided into four blocks, left, center, and right (horizontally) as shown in 605 of FIG. 6E. (Directional) is divided into three parts.

[0058] Step S1204, Step S1205, Step S1207, Step S1208 After performing either of the above, for each block into which the processing target block is divided, The numbers 0 to 605 in FIG. 2 shows the order of processing. For each divided block, the 2-3 division process in FIG. The process is then recursively executed (step S1210).

[0059] The recursive block splitting described here is done by dividing the number of times or the number of blocks to be processed. The necessity of division may be restricted depending on the size of the image. By making a prearrangement between the decoder and the decoder, the decoder can be realized without transmitting information. Alternatively, the encoding device may determine information that limits the necessity of division and record it in the bit string. , may be transmitted to a decoding device.

[0060] When a block is divided, the block before the division is called the parent block, and each block after the division is called the parent block. The blocks are called child blocks.

[0061] Next, the operation of the block division unit 202 in the image decoding device 200 will be described. The block division unit 202 performs the same processing procedure as the block division unit 101 of the image encoding device 100. However, the block division of the image coding device 100 is In the unit 101, an optimization method such as optimal shape estimation by image recognition and strain rate optimization is applied, The optimal block division shape is determined, whereas the block division in the image decoding device 200 is determined. The dividing unit 202 divides the blocks by decoding the block division information recorded in the bit string. The difference is in determining the division shape.

[0062] The syntax (syntax rules for bit strings) related to block division in the first embodiment is shown in FIG. coding_quadtree() represents the syntax for dividing a block into four parts. _type_tree() represents the syntax for splitting a block into two or three parts. it is a flag that indicates whether to divide the block into four parts. To divide the block into four parts, use qt If you do not want to split the image into four, set qt_split=0. If you want to split the image into four (qt_split=1), For each block, recursively divide it into four (coding_quadtree(0), codin g_quadtree(1), coding_quadtree(2), coding_quadtree(3), arguments 0 to 3 are the 6 in Figure 6A. (This corresponds to the number 01.) If you do not split into four (qt_split=0), the following is true: , determines the subsequent split. mtt_split is a flag indicating whether to perform further splitting. If further splitting is done (mtt_split=1), indicates whether to split vertically or horizontally. The flag mtt_split_vertical determines whether to split the image into two or three parts. mtt_split_binary is transmitted to split the binary data. mtt_split_vertical=1 indicates splitting in the vertical direction. mtt_split_vertical=0 indicates splitting horizontally. mtt_split_binary=1 indicates splitting horizontally. mtt_split_binary=0 indicates splitting into 2, mtt_split_binary=0 indicates splitting into 3. tt_split_binary=1), recursively split each block into two (multi_t ype_tree(0), multi_type_tree(1), arguments 0 to 1 are 602 or 604 in Figure 6B to D (The number corresponds to the number.) When dividing into 3 parts (mtt_split_binary=0), and recursively split the tree (multi_type_tree(0), multi_type_tree(1), multi_type_ tree(2), 0 to 2 correspond to numbers 603 in FIG. 6B or 605 in FIG. 6E.) mtt_spli Hierarchical block partitioning is performed by recursively calling multi_type_tree until t=0. Do the following.

[0063] <Inter Prediction> The inter prediction method according to the embodiment is performed by the inter prediction unit 102 of the image encoding device shown in FIG. This is carried out in the inter prediction unit 203 of the image decoding device in FIG.

[0064] The inter prediction method according to the embodiment will be described with reference to the drawings. The method is performed in both encoding and decoding processes on a coding block basis.

[0065] <Description of the inter prediction unit 102 on the encoding side> FIG. 16 is a diagram showing a detailed configuration of the inter prediction unit 102 in the image encoding device of FIG. The normal motion vector predictor mode derivation unit 301 derives a plurality of normal motion vector predictor candidates. A predicted motion vector is selected by the motion vector prediction method, and the selected predicted motion vector and the detected motion vector are compared. The difference motion vector of the detected inter prediction mode, reference index, and motion vector are calculated. The calculated differential motion vector is the inter prediction motion vector in normal prediction motion vector mode. This inter prediction information is supplied to the inter prediction mode determination unit 305. The detailed configuration and processing of the predictive motion vector mode derivation unit 301 will be described later.

[0066] The normal merge mode derivation unit 302 derives a plurality of normal merge candidates and determines the normal merge candidates. Select the inter prediction information for normal merge mode to obtain the inter prediction information for normal merge mode. The prediction mode determination unit 305 receives the merge mode. The theory will be explained later.

[0067] The sub-block prediction motion vector mode derivation unit 303 determines a plurality of sub-block prediction motion vectors. A vector candidate is derived to select a sub-block predictive motion vector, and the selected sub-block predictive motion vector is A difference motion vector is calculated between the estimated motion vector and the detected motion vector. Inter prediction mode, reference index, motion vector, and calculated differential motion vector are This is the inter prediction information of the sub-block prediction motion vector mode. is supplied to the inter prediction mode determination unit 305.

[0068] The sub-block merging mode derivation unit 304 derives a plurality of sub-block merging candidates. Select sub-block merging candidates and obtain inter prediction information for sub-block merging mode This inter prediction information is supplied to the inter prediction mode determination unit 305.

[0069] The inter prediction mode determination unit 305 determines whether the normal prediction motion vector mode is derived from the normal prediction motion vector mode derivation unit 301 or the normal A merge mode derivation unit 302, a sub-block predicted motion vector mode derivation unit 303, a sub-block Based on the inter prediction information supplied from the block merge mode derivation unit 304, The inter prediction mode determination unit 305 determines the prediction information. The measurement information is supplied to a motion compensation prediction unit 306 .

[0070] The motion compensation prediction unit 306 stores the decoded image in the decoded image memory 1 based on the determined inter prediction information. The motion compensation prediction unit 306 performs inter prediction on the reference image signal stored in the motion compensation prediction unit 306. The detailed configuration and processing will be described later.

[0071] <Description of the Inter Prediction Unit 203 on the Decoding Side> FIG. 22 is a diagram showing a detailed configuration of the inter prediction unit 203 of the image decoding device in FIG.

[0072] The normal prediction motion vector mode derivation unit 401 derives a plurality of normal prediction motion vector candidates. A predicted motion vector is selected by the above-mentioned method, and the selected predicted motion vector and the decoded differential motion vector are The decoded inter prediction mode, reference index, and In this case, the motion vector is the inter prediction information in the normal prediction motion vector mode. The center prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the motion vector measurement mode derivation unit 401 will be described later.

[0073] The normal merge mode derivation unit 402 derives a plurality of normal merge candidates and determines the normal merge candidates. Select and obtain inter prediction information for normal merge mode. This inter prediction information is switched The merge mode derivation unit 402 normally supplies the merge mode derivation signal to the motion compensation prediction unit 406 via the merge mode derivation unit 408. The detailed configuration and processing will be described later.

[0074] The sub-block prediction motion vector mode derivation unit 403 selects a plurality of sub-block prediction motion vectors. A vector candidate is derived to select a sub-block predictive motion vector, and the selected sub-block predictive motion vector is The sum of the estimated motion vector and the decoded differential motion vector is calculated and used as the motion vector. The decoded inter prediction mode, reference index, and motion vector are used for sub-block prediction. This inter prediction information is the motion vector mode inter prediction information. The signal is then supplied to the motion compensation prediction unit 406 via

[0075] The subblock merging mode derivation unit 404 derives multiple subblock merging candidates. Select sub-block merging candidates and obtain inter prediction information for sub-block merging mode This inter prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. .

[0076] The motion compensation prediction unit 406 stores the decoded image in the decoded image memory 2 based on the determined inter prediction information. 08. The motion compensation prediction unit 406 performs inter prediction on the reference image signal stored in the motion compensation prediction unit 406. The detailed configuration and processing are the same as those of the motion compensation prediction unit 306 on the encoding side.

[0077] <Normal predicted motion vector mode derivation part (normal AMVP)> The normal prediction motion vector mode derivation unit 301 in FIG. unit 321, a temporal predicted motion vector candidate derivation unit 322, and a historical predicted motion vector candidate derivation unit 3 23, a predicted motion vector candidate supplementation unit 325, a normal motion vector detection unit 326, a predicted motion vector candidate The image processing unit 320 includes a vector candidate selection unit 327 and a motion vector subtraction unit 328.

[0078] The normal prediction motion vector mode derivation unit 401 in FIG. unit 421, a temporal predicted motion vector candidate derivation unit 422, and a historical predicted motion vector candidate derivation unit 4 23, a motion vector predictor candidate supplementation unit 425, a motion vector predictor candidate selection unit 426, a motion vector The vector adder 427 is included.

[0079] The normal prediction motion vector mode derivation unit 301 on the encoding side and the normal prediction motion vector mode derivation unit 302 on the decoding side The processing procedure of the torque mode derivation unit 401 is shown in the flowcharts of FIG. 19 and FIG. 25. 19 is a diagram showing a normal motion vector mode derivation unit 301 on the encoding side. FIG. 25 is a flowchart showing a procedure for deriving a predicted motion vector mode. The normal motion vector mode derivation unit 401 performs a normal predicted motion vector mode derivation process. 6 is a flowchart.

[0080] <Normal prediction motion vector mode derivation part (normal AMVP): Explanation on the encoding side> The normal prediction motion vector mode derivation process procedure on the encoding side will be described with reference to FIG. In the explanation of the processing procedure of FIG. 19, the word "normal" shown in FIG. 19 may be omitted.

[0081] First, the normal motion vector detection unit 326 detects the inter prediction mode and the reference index. First, a normal motion vector is detected (step S100 in FIG. 19).

[0082] Next, the spatial prediction motion vector candidate derivation unit 321 and the temporal prediction motion vector candidate derivation unit 3 22, a history prediction motion vector candidate derivation unit 323, a prediction motion vector candidate supplement unit 325, A predicted motion vector candidate selection unit 327 and a motion vector subtraction unit 328 select a normal predicted motion vector The difference motion vectors of the motion vectors used in the inter prediction of the mode are set for L0 and L1, respectively. Specifically, the prediction of the processing target block is calculated (steps S101 to S106 in FIG. 19). Prediction mode PredMode is inter prediction (MODE_INTER) and inter prediction mode is L0 prediction (Pr ed_L0), calculate the motion vector predictor candidate list mvpListL0 of L0 and Then, the differential motion vector mvdL0 of the motion vector mvL0 of L0 is calculated. If the inter prediction mode of the target block is L1 prediction (Pred_L1), the predicted motion vector of L1 is A motion vector candidate list mvpListL1 is calculated, a predicted motion vector mvpL1 is selected, and the motion vector of L1 is calculated. Calculate the differential motion vector mvdL1 of the to-be-processed block mvL1. When bi-prediction (Pred_BI) is used, both L0 and L1 predictions are performed, and the predicted motion vector of L0 is A motion vector candidate list mvpListL0 is calculated, a motion vector predictor mvpL0 of L0 is selected, and the motion vector predictor A differential motion vector mvdL0 of the vector mvL0 is calculated, and a predicted motion vector candidate of L1 is calculated. A complementary list mvpListL1 is calculated, a predicted motion vector mvpL1 of L1 is calculated, and the motion vector of L1 is calculated. Then, the differential motion vector mvdL1 of the corresponding to the tor mvL1 is calculated.

[0083] The differential motion vector calculation process is performed for each of L0 and L1. Therefore, in the following explanation, L0 and L1 are represented as a common LX. In the process of calculating the difference motion vector of L0, X of LX is 0, and the difference motion vector of L1 is In the process of calculating the motion vector of LX, X of LX is 1. If you refer to information in the other list instead of the LX during the process of is expressed as LY.

[0084] When the motion vector mvLX of LX is used (step S102 in FIG. 19: YES), Calculate the candidate predicted motion vectors of LX and construct a candidate predicted motion vector list mvpListLX of LX. (Step S103 in FIG. 19). A spatial prediction motion vector candidate derivation unit 321, a temporal prediction motion vector candidate derivation unit 322, a history A motion vector predictor candidate derivation unit 323 and a motion vector predictor candidate supplement unit 325 generate a plurality of motion vector predictors. The motion vector predictor candidate list mvpListLX is constructed by deriving motion vector candidates. The detailed process of step S103 will be described later with reference to the flowchart of FIG. do.

[0085] Next, the motion vector predictor candidate selection unit 327 selects a motion vector predictor candidate list for LX. Select a predicted motion vector mvpLX for LX from the mvpListLX (step S104 in FIG. 19). Here, in the motion vector predictor candidate list mvpListLX, a certain element (counting from 0) The motion vector mvLX and the motion vector predictor candidate are expressed as mvpListLX[i]. The difference between each predicted motion vector candidate mvpListLX[i] stored in the list mvpListLX is The differential motion vectors are calculated for each of the two frames. The coding amount is calculated for each element (predictor motion vector candidate) of the predictor motion vector candidate list mvpListLX. Then, among the elements registered in the motion vector predictor candidate list mvpListLX, The candidate motion vector predictor mvpListLX[i] that has the smallest coding amount for each candidate motion vector predictor is The motion vector predictor mvpLX is selected as the motion vector predictor mvpLX, and its index i is obtained. There are multiple candidates for the predicted motion vector with the smallest generated code amount in the candidate list mvpListLX. In this case, the index i in the motion vector predictor candidate list mvpListLX is a small number. The candidate motion vector predictor mvpListLX[i] represented by the following is set as the optimal motion vector predictor mvpLX. Select it and get its index i.

[0086] Next, the motion vector subtraction unit 328 subtracts the motion vector mvLX of LX selected from the motion vector mvLX of LX. Subtract the predicted motion vector mvpLX, mvdLX = mvLX - mvpLX Then, the differential motion vector mvdLX of LX is calculated as (Step S105 in FIG. 19).

[0087] <Normal predicted motion vector mode derivation part (normal AMVP): Decoding side explanation> Next, a normal predicted motion vector mode process procedure on the decoding side will be described with reference to FIG. On the other hand, a spatial prediction motion vector candidate derivation unit 421 and a temporal prediction motion vector candidate derivation unit 4 22, the history prediction motion vector candidate derivation unit 423 and the prediction motion vector candidate supplementation unit 425, The motion vectors used in inter prediction in normal prediction motion vector mode are divided into L0 and L1. Specifically, the process of the processing block is performed as follows (steps S201 to S206 in FIG. 25). Prediction mode PredMode is inter prediction (MODE_INTER) and the processing target block is inter prediction When the mode is L0 prediction (Pred_L0), calculate the predicted motion vector candidate list mvpListL0 of L0. Then, the predicted motion vector mvpL0 is selected, and the motion vector mvL0 of L0 is calculated. If the inter prediction mode of the target block is L1 prediction (Pred_L1), the predicted motion vector of L1 is A motion vector candidate list mvpListL1 is calculated, a predicted motion vector mvpL1 is selected, and the motion vector of L1 is calculated. If the inter prediction mode of the processing target block is bi-predictive (Pred_BI), Then, both L0 prediction and L1 prediction are performed, and the L0 predicted motion vector candidate list mvpListL0 is calculated. Then, the predicted motion vector mvpL0 of L0 is selected, and the motion vector mvL0 of L0 is calculated. In both cases, a motion vector predictor candidate list mvpListL1 of L1 is calculated, and the motion vector predictor of L1 is Then, the motion vector mvpL1 for L1 is calculated, and the motion vector mvL1 for L2 is calculated.

[0088] As with the encoding side, the decoding side also performs motion vector calculation processing for each of L0 and L1. However, the same processing is performed for both L0 and L1. Therefore, in the following explanation, L1 is represented as a common LX. LX is used for inter prediction of the coding block being processed. It represents the inter prediction mode. In the process of calculating the motion vector of L0, X is 0, and in the process of calculating the motion vector of L1, In the process of calculating the motion vector of X, X is 1. Also, in the process of calculating the motion vector of LX, During processing, the LX is calculated by referencing information from another reference list, not the same reference list. When referring to one reference list, the other reference list is represented as LY.

[0089] When the motion vector mvLX of LX is used (step S202 in FIG. 25: YES), Calculate the candidate predicted motion vectors of LX and construct a candidate predicted motion vector list mvpListLX of LX. (Step S203 in FIG. 25). A spatial prediction motion vector candidate derivation unit 421, a temporal prediction motion vector candidate derivation unit 422, a history A motion vector predictor candidate derivation unit 423 and a motion vector predictor candidate supplement unit 425 generate a plurality of motion vector predictors. Then, the motion vector predictor candidate list mvpListLX is constructed. The detailed process of step S203 will be described later with reference to the flowchart of FIG. do.

[0090] Next, the motion vector predictor candidate selection unit 426 selects the motion vector predictor candidate list mvpListLX The bitstream decoder 201 decodes and supplies the predicted motion vector index mv The candidate predicted motion vector mvpListLX[mvpIdxLX] corresponding to pIdxLX is selected as the predicted motion vector. The vector mvpLX is extracted (step S204 in FIG. 25).

[0091] Then, the motion vector adder 427 decodes the bit stream and supplies it to the bit stream decoder 201. Add the differential motion vector mvdLX of LX and the predicted motion vector mvpLX of LX, mvLX = mvpLX + mvdLX Then, the motion vector mvLX of LX is calculated (step S205 in FIG. 25).

[0092] <Normal predicted motion vector mode derivation part (normal AMVP): Motion vector prediction method> FIG. 20 shows a normal prediction motion vector mode derivation process of the image coding apparatus according to the embodiment of the present invention. Functions common to the motion vector mode derivation unit 301 and the normal prediction motion vector mode derivation unit 401 of the image decoding device 13 is a flowchart showing a processing procedure of a normal predicted motion vector mode derivation process having the above-mentioned steps.

[0093] The normal prediction motion vector mode derivation unit 301 and the normal prediction motion vector mode derivation unit 40 1, a motion vector predictor candidate list mvpListLX is provided. The mvpListLX has a list structure and is a prediction vector that indicates the location within the predicted motion vector candidate list. A motion vector index and a motion vector predictor candidate corresponding to the index are used as elements. The predicted motion vector index number starts from 0. The motion vector predictor candidate list mvpListLX is stored in a storage area of ​​the motion vector predictor candidate list mvpListLX. In this embodiment, the motion vector predictor candidate list mvpListLX stores at least At least two predicted motion vector candidates (inter prediction information) can be registered. In addition, the motion vector predictor registered in the motion vector predictor candidate list mvpListLX is The variable numCurrMvpCand, which indicates the number of candidates, is set to 0.

[0094] The spatial prediction motion vector candidate derivation units 321 and 421 derive the spatial prediction motion vector candidate from the adjacent block on the left. In this process, a candidate predicted motion vector is derived from the adjacent block on the left ( Whether inter prediction information (A0 or A1), i.e., whether a predicted motion vector candidate is available A flag indicating whether the motion vector is a predicted motion vector mv LXA is derived, and the derived mvLXA is added to the motion vector predictor candidate list mvpListLX (FIG. 20). In step S301, X is set to 0 for L0 prediction and 1 for L1 prediction. Next, the spatial prediction motion vector candidate derivation units 321 and 421 select the adjacent spatial prediction motion vector candidate In this process, the candidate predicted motion vectors are derived from the adjacent block on the upper side. Inter prediction information of the block (B0, B1, or B2 in FIG. 11), i.e., predicted motion vector A flag indicating whether a candidate is available or not, and a reference of motion vectors, reference indexes, etc. The predicted motion vector mvLXB is derived based on the calculated motion vector mvLXA and mvLXB. If so, mvLXB is added to the motion vector predictor candidate list mvpListLX (step S3 in FIG. 20). 02). The processes in steps S301 and S302 in FIG. 20 refer to the position and number of adjacent blocks. The difference is whether the predicted motion vector candidate of the coding block can be used. A flag indicating whether the motion vector is available or not, a motion vector mvLXN, and a reference index refIdxN( N indicates A or B, and so on).

[0095] Next, the temporal motion vector predictor candidate derivation units 322 and 422 calculate the current processing target picture A candidate motion vector predictor is derived from a block in a picture that is different in time from the image. In this process, motion vector prediction candidates of coding blocks of pictures at different times are used. availableFlagLXCol, a flag indicating whether the motion vector mvLXCol and the reference index The refIdxCol and the reference list listCol are derived, and mvLXCol is used to create the predicted motion vector candidate list m vpListLX (step S303 in FIG. 20).

[0096] Note that temporal prediction can be performed on a sequence (SPS), picture (PPS), or slice basis. It is assumed that the processing of the motion vector candidate derivation units 322 and 422 can be omitted.

[0097] Next, the history prediction motion vector candidate derivation units 323 and 423 calculate the history prediction motion vector candidate. The historical predicted motion vector candidates registered in the auxiliary list HmvpCandList are used as the predicted motion vector candidates. The mvpListLX is added to the auxiliary list (step S304 in FIG. 20). The registration process will be described in detail later with reference to the flowchart of FIG.

[0098] Next, the motion vector predictor candidate supplementation units 325 and 425 generate the motion vector predictor candidate list mv Add a motion vector predictor candidate with a predetermined value, such as (0,0), until pListLX is satisfied ( S305 in Figure 20).

[0099] <Normal merge mode derivation part (normal merge)> The normal merge mode derivation unit 302 in FIG. 18 includes a spatial merge candidate derivation unit 341, a temporal merge candidate derivation unit 342, and a A candidate derivation unit 342, an average merge candidate derivation unit 344, a history merge candidate derivation unit 345, and a merge It includes a candidate supplementation unit 346 and a merge candidate selection unit 347 .

[0100] The normal merge mode derivation unit 402 in FIG. 24 includes a spatial merge candidate derivation unit 441, a temporal merge candidate derivation unit 442, and a A candidate derivation unit 442, an average merge candidate derivation unit 444, a history merge candidate derivation unit 445, and a merge It includes a candidate supplementation unit 446 and a merge candidate selection unit 447 .

[0101] FIG. 21 shows the normal merge mode derivation unit 302 and A normal merge mode having a function common to the normal merge mode derivation unit 402 of the image decoding device and the normal merge mode derivation unit 402 of the image decoding device. 11 is a flowchart illustrating the procedure of a code derivation process.

[0102] The steps will be explained below in order. The slice type slice_type is explained for the B slice, but for the P slice, However, if the slice type slice_type is P slice, the inter prediction model can be applied. There is only L0 prediction (Pred_L0) as a mode, L1 prediction (Pred_L1) and bi-prediction (Pred_BI) are also supported. Since there is no L1, processing related to L1 can be omitted.

[0103] In the normal merge mode derivation unit 302 and the normal merge mode derivation unit 402, The merge candidate list mergeCandList has a list structure, A merge index that indicates the location within the merge candidate list, and the merge A memory area is provided to store merge candidates as elements. The merge index number is 0. The merge candidates are stored in the merge candidate list (mergeCandList) starting from In the following process, the merge index i registered in the merge candidate list mergeCandList is The merge candidates are represented by mergeCandList[i]. The candidate list mergeCandList must contain at least six merge candidates (inter prediction information). In addition, the merge candidates registered in the mergeCandList can be The variable numCurrMergeCand, which indicates the number of merge candidates, is set to 0.

[0104] The spatial merge candidate derivation unit 341 and the spatial merge candidate derivation unit 441 are The encoded information stored in the encoded information storage memory 111 or the encoded information storage memory 205 of the image decoding device From the coding information stored in the memory, each block adjacent to the processing target block (B in FIG. 11) is 1, A1, B0, A0, B2) to B1, A1, B0, A0, B2 The spatial merge candidates are derived in order and registered in the merge candidate list mergeCandList. (Step S401 in FIG. 21). Here, B1, A1, B0, A0, B2 or time marker We define N, which indicates one of the candidate blocks Col. A flag indicating whether the candidate can be used as a spatial merge candidate, availableFlagN, is Reference index refIdxL0N and L1 reference index refIdxL1N, L0 prediction is performed A L0 prediction flag predFlagL0N indicates whether L1 prediction is performed or not, and a L1 prediction flag predFlagL0N indicates whether L1 prediction is performed or not. The measurement flag predFlagL1N, the motion vector mvL0N for L0, and the motion vector mvL1N for L1 are derived. However, in this embodiment, the blocks included in the coding block to be processed Since the merging candidates are derived without referring to the inter prediction information, the coding block to be processed , no spatial merge candidates using inter prediction information of blocks included in

[0105] Next, the temporal merge candidate derivation unit 342 and the temporal merge candidate derivation unit 442 determine different time Then, a temporal merge candidate is derived from the pictures between the first and second pictures, and the derived temporal merge candidate is called a merge candidate. The time merge candidates are registered in the list mergeCandList (step S402 in FIG. 21). availableFlagCol, a flag indicating whether L0 prediction of temporal merge candidates is performed L0 prediction flag predFlagL0Col indicating whether L0 prediction is performed and L1 prediction flag predFlagL0Col indicating whether L1 prediction is performed The flag predFlagL1Col, the motion vector mvL0Col of L0, and the motion vector mvL1Col of L1 are Derive.

[0106] Note that time markers can be set in units of sequences (SPS), pictures (PPS), or slices. The processing of the merge candidate derivation unit 342 and the time merge candidate derivation unit 442 can be omitted. Let us assume that.

[0107] Next, the history merging candidate derivation unit 345 and the history merging candidate derivation unit 445 perform history prediction Mark the historical predicted motion vector candidates registered in the motion vector candidate list HmvpCandList. The candidate list is registered in mergeCandList (step S403 in FIG. 21). The number of merge candidates registered in the merge candidate list mergeCandList is numCurrMergeC If and is smaller than the maximum number of merge candidates, MaxNumMergeCand, the merge candidate list, mergeCandL The number of merge candidates registered in ist is numCurrMergeCand, which is the maximum number of merge candidates, MaxNumMergeCand. History merge candidates are derived with nd as the upper limit and registered in the merge candidate list mergeCandList. can be.

[0108] Next, the average merge candidate derivation unit 344 and the average merge candidate derivation unit 444 calculate the merge candidates. Derive average merge candidates from the complement list mergeCandList and merge the derived average merge candidates. The merge candidate list mergeCandList is then added to the merge candidate list mergeCandList (step S404 in FIG. 21). The number of merge candidates registered in the merge candidate list mergeCandList is numCurrMergeC If and is smaller than the maximum number of merge candidates, MaxNumMergeCand, the merge candidate list, mergeCandL The number of merge candidates registered in ist is numCurrMergeCand, which is the maximum number of merge candidates, MaxNumMergeCand. The average merge candidates are derived with nd as the upper limit and registered in the merge candidate list mergeCandList. can be. Here, the average merge candidate is the first merge candidate registered in the merge candidate list mergeCandList. The motion vectors of the merge candidate and the second merge candidate are averaged for L0 prediction and L1 prediction. The new merge candidate has the motion vector obtained by

[0109] Next, the merge candidate supplementation unit 346 and the merge candidate supplementation unit 446 create a merge candidate list The number of merge candidates registered in mergeCandList, numCurrMergeCand, is less than the maximum number of merge candidates, M If it is smaller than axNumMergeCand, the merge candidate list mergeCandList The number of merge candidates numCurrMergeCand is the maximum number of merge candidates MaxNumMergeCand. The merge candidates are derived and registered in the merge candidate list mergeCandList (step S4 in FIG. 21). 05). In P slices, the motion vector is A merge candidate having a prediction mode of L0 prediction (Pred_L0) with a value of (0,0) is added. In B slices, the prediction mode with a motion vector of (0,0) is bi-predictive (Pred_BI). Add a merge candidate. The reference index for adding a merge candidate is the same as the index of the The reference index is different from the one specified.

[0110] Next, the merging candidate selection unit 347 and the merging candidate selection unit 447 select the merging candidate list Select a merge candidate from the merge candidates registered in mergeCandList. The merge candidate selection unit 347 selects merge candidates by calculating the code amount and the distortion amount. , a merge index indicating the selected merge candidate, inter prediction information of the merge candidate, The inter-prediction mode determination unit 305 supplies the inter-prediction mode to the motion compensation prediction unit 306. The merge candidate selection unit 447 on the side selects merge candidates based on the decoded merge index. The motion compensation prediction unit 406 selects a merge candidate from the list of merge candidates and supplies the selected merge candidate to the motion compensation prediction unit 406.

[0111] <Sub-block prediction motion vector mode derivation> Sub-block predicted motion vector mode derivation will now be described. FIG. 38 shows the sub-block prediction motion vector mode derivation unit 30 in the encoding device of the present application. FIG. 3 is a block diagram of FIG.

[0112] First, the affine inheritance predicted motion vector candidate derivation unit 361 derives an affine inheritance predicted motion vector candidate. The details of affine inheritance prediction motion vector candidate derivation will be described later. State.

[0113] Next, the affine construction prediction motion vector candidate derivation unit 362 performs affine construction prediction Derive motion vector candidates. For details on deriving affine construction prediction motion vector candidates, see This will be explained later.

[0114] Next, the affine-identical prediction motion vector candidate derivation unit 363 performs affine-identical prediction Derive motion vector candidates. For details on deriving affine identical predictor motion vector candidates, see This will be explained later.

[0115] The sub-block motion vector detector 366 is adapted to the sub-block prediction motion vector mode. A sub-block motion vector corresponding to the sub-block is detected, and the detected vector is used as a sub-block predicted motion vector. The result is supplied to a torque candidate selection unit 367 and a difference calculation unit 368.

[0116] The sub-block predicted motion vector candidate selection unit 367 selects the affine inheritance predicted motion vector candidate Affine construction predicted motion vector candidate derivation unit 361, an affine construction predicted motion vector candidate derivation unit 362, an affine same predicted motion vector candidate derivation unit 363, an affine construction predicted motion vector candidate derivation unit 364, an affine same predicted motion vector candidate derivation unit 365, an affine construction predicted motion vector candidate derivation unit 366, an affine same predicted motion vector candidate derivation unit 367, an Among the sub-block predicted motion vector candidates derived in the motion vector candidate derivation unit 363, Based on the motion vector supplied from the sub-block motion vector detection unit 366, A sub-block predicted motion vector candidate is selected, and the selected sub-block predicted motion vector Information about the candidates is supplied to the inter prediction mode determination unit 305 and the difference calculation unit 368 .

[0117] The difference calculation unit 368 calculates the motion vector supplied from the sub-block motion vector detection unit 366. The sub-block predicted motion vector candidate selection unit 367 selects the sub-block predicted motion vector candidate from the motion vector. The inter prediction mode determination unit 102 determines the difference prediction motion vector obtained by subtracting the inter prediction motion vector from the inter prediction mode prediction motion vector. Supply to 305.

[0118] FIG. 39 shows the sub-block prediction motion vector mode derivation unit 403 in the decoding device of the present application. FIG.

[0119] First, the affine inheritance predicted motion vector candidate derivation unit 461 derives an affine inheritance predicted motion vector candidate. The process of the affine inheritance predicted motion vector candidate derivation unit 461 is as follows: The processing is the same as that of the affine inheritance predicted motion vector candidate derivation unit 361 in the encoding device of the present application. be.

[0120] Next, the affine construction prediction motion vector candidate derivation unit 462 performs affine construction prediction The affine construction prediction motion vector candidate derivation unit 462 performs the following process: The same processing as that of the affine construction predicted motion vector candidate derivation unit 362 in the encoding device of the present application It is.

[0121] Next, the affine-identical prediction motion vector candidate derivation unit 463 performs affine-identical prediction The affine identical predictor motion vector candidate derivation unit 463 performs the following process: The same processing as that of the affine-identical predictor motion vector candidate derivation unit 363 in the encoding device of the present application It is.

[0122] The sub-block predicted motion vector candidate selection unit 467 selects the affine inheritance predicted motion vector candidate Affine construction predicted motion vector candidate derivation unit 461, an affine construction predicted motion vector candidate derivation unit 462, an affine same predicted motion vector Among the sub-block predicted motion vector candidates derived in the motion vector candidate derivation unit 463, Then, based on the predicted motion vector index transmitted from the encoding device and decoded, a sub-block predicted motion vector candidate is selected, and the selected sub-block predicted motion vector candidate is The information on the complement is supplied to the motion compensation prediction unit 406 and the addition operation unit 467 .

[0123] The addition operation unit 467 adds the sub-block predicted motion vector candidate selected by the sub-block predicted motion vector candidate selection unit 466. The differential motion vector transmitted from the encoding device and decoded is used as the subblock predicted motion vector. The motion vector generated by the addition is supplied to the motion compensation prediction unit 406 .

[0124] <Derivation of affine inheritance predicted motion vector candidates> The affine successive prediction motion vector candidate derivation unit 361 will be described. The motion vector candidate derivation unit 461 is also the same as the affine inheritance prediction motion vector candidate derivation unit 36 Same as 1.

[0125] The affine inherited predicted motion vector candidate inherits the motion vector information of the affine control point. FIG. 42 is a diagram for explaining derivation of affine inheritance predicted motion vector candidates.

[0126] The affine inheritance motion vector candidate predictor is a vector that is derived from the spatially adjacent encoded and decoded blocks. The motion vectors are obtained by searching for affine control points having the same vector.

[0127] Specifically, the block (A0, A1) adjacent to the left of the block to be coded and decoded, From the blocks (B0, B1, B2) adjacent to the upper side of the block to be coded / decoded, A maximum of one affine mode is searched for and set as the affine inheritance predicted motion vector.

[0128] FIG. 46 is a flowchart showing the derivation of affine inheritance predicted motion vector candidates. First, the block (A0, A1) adjacent to the left of the block to be coded / decoded is treated as the left group. The block containing A0 is treated as a block using affine guarantee (S3101). If A0 is in affine mode (S3102), 3102: YES), obtain the affine model used by A0 (S3103), and The process moves to the adjacent block. If A0 is not in affine mode (S3102: NO), The affine inheritance predicted motion vector candidate is derived from A0 to A1, and the block including A1 is Attempts to obtain an affine mode from

[0129] Next, the blocks (B0, B1, B2) adjacent to the upper side of the block to be coded / decoded are Then, the block including B0 is classified as the upper group (S3104), and it is judged whether the block including B0 is in the affine mode or not. If B0 is in the affine mode (S3105: YES), The affine model used by B0 is acquired (S3106), and the process ends. If the affine inheritance prediction motion vector candidate is not the prediction mode (S3105: NO), the affine inheritance prediction motion vector candidate is not derived. B0->B1, and try to get the affine mode from the block containing B1. If 1 is not the affine mode (S3105: NO), the affine inheritance predicted motion vector candidate The target of the supplementary derivation is B1->B2, and an attempt is made to obtain an affine mode from the block including B2. do.

[0130] In this way, we divided the group into the left block and the upper block, and for the left block, searches the affine models from the bottom left to the top left block, and for the left block, By searching the affine models in the order from the top right to the top left block, we can find two blocks that are as different as possible. Then, we can obtain an affine model of the affine predicted motion vectors. It is possible to derive an affine predicted motion vector candidate that has a smaller error vector.

[0131] <Derivation of affine-constructed motion vector candidates> The affine construction prediction motion vector candidate derivation unit 362 will now be described. The motion vector candidate derivation unit 462 is also the same as the affine construction prediction motion vector candidate derivation unit 36 Same as 2.

[0132] The affine constructed motion vector candidate is derived from the motion information of spatially adjacent blocks. The motion vector information of the control points is constructed.

[0133] FIG. 43 is a diagram for explaining derivation of affine constructed predicted motion vector candidates. The affine construction motion vector candidate predictor is a vector of spatially adjacent coded and decoded blocks. The motion vectors are then combined to construct a new affine model. Specifically, the blocks adjacent to the upper left of the block to be coded and decoded (B2, B3, A2 ) to derive the motion vector of the upper left affine control point CP0, and The motion vector from the upper right adjacent block (B1, B0) to the upper right affine control point CP1 Then, the block (A1, A0) adjacent to the lower left of the block to be coded / decoded is A motion vector for the lower affine control point CP2 is derived.

[0134] FIG. 47 is a flowchart showing the process of deriving affine constructed motion vector predictor candidates. First, derive the upper left control point CP0, the upper right control point CP1, and the lower left affine control point CP2 ( The upper left affine control point CP0 is the same as the reference image of the block to be coded or decoded. The image is calculated by searching the B2, B3, and A2 reference blocks in that order. The upper right affine control point CP1 has the same reference image as the block to be coded and decoded. The calculation is done by searching the reference blocks B1, B0 in the order of priority. The affine control point CP2 is a reference block that has the same reference image as the block to be coded or decoded. The address is calculated by searching the A1, A0 reference blocks in that order.

[0135] When selecting the three affine control points mode as the affine construction predicted motion vector (S 3202:YES), all three affine control points (CP0, CP1, CP2) are derived. It is determined whether the three affine control points (CP0, CP1, CP2 ) are derived (S3203: YES), the three affine control points (CP0,C P1, CP2) is used as the affine construction predicted motion vector (S32 04). If you do not select the 3 affine control points mode and select the 2 affine control points mode, If (S3202:NO), both affine control points (CP0, CP1) are derived. It is determined whether or not the two affine control points (CP0, CP1) are all derived (S3205). If the command is issued (S3205: YES), the two affine control points (CP0, CP1) are used. The obtained affine model is used as an affine constructed predicted motion vector (S3206).

[0136] <Derivation of affine identical predictive motion vector candidates> The affine co-prediction motion vector candidate derivation unit 363 will be described. The motion vector candidate derivation unit 463 is also the same as the affine identical predictive motion vector candidate derivation unit 36 Same as 3.

[0137] The affine identical predicted motion vector candidate derives the same motion vector at each affine control point. This can be obtained by doing the following.

[0138] Specifically, similarly to the affine construction motion vector candidate derivation units 362 and 462, Derive the fin control point information and set all affine control points to the same one of CP0 to CP2. In addition, the time vectors derived in the same way as in the normal predicted motion vector mode are Alternatively, we can set the vector to all the affine control points.

[0139] <Sub-block merge mode derivation> Sub-block merge mode derivation will now be described.

[0140] FIG. 40 shows a block diagram of the sub-block merge mode derivation unit 304 in the encoding device of the present application. The sub-block merging mode derivation unit 304 determines the sub-block merging candidate list This is the same as in the normal merge mode derivation unit 302. This is similar to the merge candidate list, mergeCandList, and is a list of candidates that differs on a subblock basis. The only difference is that it becomes a strike.

[0141] First, the sub-block time merge candidate derivation unit 381 determines the sub-block time merge candidate The details of the derivation of sub-block temporal merging candidates will be described later.

[0142] Next, the affine inheritance merge candidate derivation unit 382 derives affine inheritance merge candidates. The details of affine inheritance merge candidate derivation will be described later.

[0143] Next, the affine construction merge candidate derivation unit 383 derives affine construction merge candidates. The details of deriving affine construction merge candidates will be described later.

[0144] Next, the affine-fixed merge candidate derivation unit 384 derives affine-fixed merge candidates. The details of affine fixed merge candidate derivation will be described later.

[0145] The sub-block merging candidate selection unit 386 includes the sub-block temporal merging candidate derivation unit 381, An affine inheritance merge candidate derivation unit 382, ​​an affine construction merge candidate derivation unit 383, an affine Among the sub-block merging candidates derived by the fixed merging candidate derivation unit 384, Select a block merging candidate and display information about the selected sub-block merging candidate in the The prediction mode is supplied to the prediction mode determination unit 305.

[0146] FIG. 41 shows the block diagram of the sub-block merge mode derivation unit 404 in the decoding device of the present application. The sub-block merging mode derivation unit 404 generates the sub-block merging candidate list su This is the sub-block merge mode derivation unit 304 and It's the same thing.

[0147] First, the sub-block time merge candidate derivation unit 481 determines the sub-block time merge candidate The process of the sub-block time merging candidate derivation unit 481 derives the sub-block time merging candidate. This is the same as the processing of the candidate derivation unit 381.

[0148] Next, the affine inheritance merge candidate derivation unit 482 derives affine inheritance merge candidates. The processing of the affine inheritance merge candidate derivation unit 482 is the same as that of the affine inheritance merge candidate derivation unit 3 This is the same as the process in 82.

[0149] Next, the affine construction merge candidate derivation unit 483 derives affine construction merge candidates. The processing of the affine construction merge candidate derivation unit 483 is the same as that of the affine construction merge candidate derivation unit 3 This is the same as the process in 83.

[0150] Next, the affine-fixed merge candidate derivation unit 485 derives affine-fixed merge candidates. The processing of the affine fixed merge candidate derivation unit 485 is the same as that of the affine fixed merge candidate derivation unit 485. This is the same as the process in 85.

[0151] The sub-block merging candidate selection unit 486 includes the sub-block temporal merging candidate derivation unit 481, An affine inheritance merge candidate derivation unit 482, an affine construction merge candidate derivation unit 483, an affine Among the sub-block merging candidates derived by the fixed merging candidate derivation unit 484, Selecting sub-block merging candidates based on the index transmitted and decoded from the decoder and supplies information about the selected sub-block merge candidates to the motion compensation prediction unit 406. .

[0152] <Sub-block temporal merge candidate derivation> The operation of the sub-block temporal merging candidate derivation unit 381 will be described later.

[0153] <Affine inheritance merge candidate derivation> The affine inheritance merge candidate derivation unit 382 will be described. The affine inheritance merge candidate derivation unit 482 is similar to the affine inheritance merge candidate derivation unit 382 .

[0154] Affine inheritance merge candidates are affine inherited from the affine models of spatially adjacent blocks. The affine model of the finite control points is inherited. It is determined by the size of the object and the motion information of the affine control points.

[0155] FIG. 44 is a diagram for explaining the derivation of affine inheritance merge candidates. The derivation of the affine inheritance prediction motion vector is similar to the derivation of the affine inheritance prediction motion vector. The motion vectors of the affine control points of the coded and decoded blocks are searched for. can be done.

[0156] Specifically, the block (A0, A1) adjacent to the left of the block to be coded and decoded, From the blocks (B0, B1, B2) adjacent to the upper side of the block to be coded / decoded, At most one affine mode is searched for and used for the affine merge mode.

[0157] FIG. 48 is a flowchart of affine inheritance merge candidate derivation. First, the block (A0, A1) adjacent to the left of the block to be coded / decoded is treated as the left group. Then, it is determined whether the block including A0 is in affine mode (S3301). S3302). If A0 is in affine mode (S3102: YES), A0 uses The affine model is obtained (S3303), and the process moves to the adjacent block on the upper side. If is not in affine mode (S3302: NO), it is the target for deriving affine inheritance merge candidates. Let A0->A1, and try to obtain the affine mode from the block containing A1.

[0158] Next, the blocks (B0, B1, B2) adjacent to the upper side of the block to be coded / decoded are Then, the block including B0 is classified as the upper group (S3304), and it is judged whether the block including B0 is in the affine mode or not. If B0 is in affine mode (S3305: YES), The affine model used by B0 is obtained (S3306), and the process ends. If it is not (S3305:NO), the affine inheritance merge candidate derivation target is B0->B 1 and try to get the affine mode from the block containing B1. If the merge mode is not selected (S3305: NO), the affine inheritance merge candidate derivation target is set to B1- >B2 and try to get the affine mode from the block containing B2.

[0159] <Affine construction merge candidate derivation> The affine construction merge candidate derivation unit 383 will be described. The affine construction merge candidate derivation unit 483 is similar to the affine construction merge candidate derivation unit 383 .

[0160] FIG. 45 is a diagram for explaining derivation of affine construction merge candidates. The affine control is derived from the motion information of spatially adjacent blocks and the temporal coding block. Construct an affine model of the points.

[0161] Specifically, the blocks adjacent to the upper left of the block to be coded and decoded (B2, B3, A 2) The motion vector of the upper left affine control point CP0 is derived from The motion vector of the upper right affine control point CP1 from the block (B1,B0) adjacent to the upper right side of The code is derived from the block (A1, A0) adjacent to the lower left of the block to be coded / decoded. The motion vector of the lower-left affine control point CP2 is derived, and the lower-right side of the block to be coded / decoded is The motion vector of the lower right affine control point CP3 from the adjacent temporal coding block (T0) is Derive.

[0162] FIG. 49 is a flowchart of affine construct merge candidate derivation. First, the upper left affine control point CP0, the upper right affine control point CP1, and the lower left affine control point C P2, the lower right affine control point CP3 is derived (S3401). The upper left affine control point CP0 is obtained by searching for blocks with motion information in the order of priority: B2, B3, A2 blocks. The upper right affine control point CP1 is used to calculate the motion information of the block B1 and B0. The lower left affine control point CP2 is calculated by searching in the order of lock priority. The calculation is done by searching the blocks with the same address, in the order of A1, A0, etc. The control point CP3 is calculated by searching the motion information of the time block.

[0163] Next, the derived upper left affine control point CP0, upper right affine control point CP1, and lower left affine control point CP2 are Is it possible to construct an affine model using three affine control points using the affine control point CP2? If it is possible to construct it (S3402: YES), it is called an upper-left affine Three affine control points, CP0, CP1, and CP2, are used. The control point affine model is treated as an affine merge candidate (S3403).

[0164] Next, the derived upper left affine control point CP0, upper right affine control point CP1, and lower right affine control point CP2 are Is it possible to construct an affine model using three affine control points using the affine control point CP3? If it is possible to construct it (S3404: YES), Three affine control points: CP0, CP1, and CP3. The control point affine model is treated as an affine merge candidate (S3405).

[0165] Next, the derived upper left affine control point CP0, lower left affine control point CP2, and lower right affine control point CP3 are Is it possible to construct an affine model using three affine control points using the affine control point CP3? If it is possible to construct it (S3406: YES), it is called an upper-left affine Three affine control points CP0, CP2, and CP3 The affine control point affine model is treated as an affine merge candidate (S3407).

[0166] Next, two lines are drawn using the derived upper left affine control point CP0 and upper right affine control point CP1. It is determined whether or not an affine model can be constructed using affine control points (S3408); If construction is possible (S3408: YES), the upper-left affine control point CP0, the upper-right affine The two-point affine model with control point CP1 is considered as an affine merge candidate (S 3409).

[0167] Next, two lines are drawn using the derived upper left affine control point CP0 and lower left affine control point CP2. It is determined whether or not an affine model can be constructed using affine control points (S3410); If construction is possible (S3410: YES), the upper-left affine control point CP0, the lower-left affine The two-point affine model with control point CP2 is considered as an affine merge candidate (S 3411).

[0168] Here, whether or not an affine model is constructed is determined based on the following conditions. 1. The reference images for all affine control points are the same. (Affine transformation is possible) 2. At least one affine control point has a different motion vector. (It can be expressed by a translation.) (I can't)

[0169] In this way, in the first embodiment, the upper left affine construct merge candidate is The top-left affine control point CP0 is the top-left affine control point CP0. When processing in the raster scan order below, there is a possibility that there are already coded and decoded blocks. Highest, ie, most likely that motion information is present.

[0170] Using the upper-left affine control point CP0, we derive affine construction merge candidates to The amount of processing required to determine whether the reference images are the same when combining control points is reduced. In addition, the upper right affine control point CP1, the lower left affine control point CP2, and the lower right affine control point C In the derivation of the affine control point of P3 (S3401), the upper left affine control point is the same as CP0. In this case, it is possible to search for the same reference image. The affine construction merge candidates are derived, and the reference images are the same and valid affine construction merge candidates are selected. It is possible to derive affine merge candidates with high coding efficiency.

[0171] Here, we will explain in more detail how to determine the reference image based on the upper left affine control point CP0. explain.

[0172] Figure 62 shows the combination of each derived affine control point in the affine construction merge candidate derivation. This is a diagram showing what motion information is compared when deriving affine construction merge candidates by combining them. be.

[0173] FIG. 62(A) shows the upper left affine control point CP0, the upper right affine control point CP1, and the lower left affine control point CP2. This is a diagram showing a case where a three-control-point affine model is constructed using the affine control point CP2. In this case, the motion information of the upper left affine control point CP0 and the upper right affine control point CP1 are compared, Whether the reference images of the upper-left affine control point CP0 and the upper-right affine control point CP1 are the same, and The motion vectors of the upper left affine control point CP0 and the upper right affine control point CP1 are compared to see whether they are different. Similarly, the motion information of the upper-left affine control point CP0 and the lower-left affine control point CP2 is To be compared.

[0174] FIG. 62(B) shows the upper left affine control point CP0, the lower left affine control point CP2, and the lower right affine control point CP3. This is a diagram showing a case where a three-point affine control point affine model is constructed using the affine control point CP3. In this case, the motion information of the upper left affine control point CP0 and the lower left affine control point CP2 are compared, The motion information of the upper left affine control point CP0 and the lower right affine control point CP3 are compared.

[0175] FIG. 62(C) shows the upper left affine control point CP0, the upper right affine control point CP1, and the lower right affine control point CP2. This is a diagram showing a case where a three-point affine control point affine model is constructed using the affine control point CP3. In this case, the motion information of the upper left affine control point CP0 and the upper right affine control point CP1 are compared, The motion information of the upper left affine control point CP0 and the lower right affine control point CP3 are compared.

[0176] Figure 62(D) shows a two-line affine control point with the upper left affine control point CP0 and the upper right affine control point CP1. This is a diagram showing the case where an affine control point affine model is constructed. In this case, the upper left affine control point The motion information of CP0 and the upper right affine control point CP1 is compared.

[0177] Figure 62(E) shows a two-line affine control point with an upper left affine control point CP0 and a lower left affine control point CP2. This is a diagram showing the case where an affine control point affine model is constructed. In this case, the upper left affine control point The motion information of CP0 and the lower-left affine control point CP2 is compared.

[0178] In this way, in all combinations of affine construction merge candidate derivations, the upper-left affine The motion information of the control point CP0 and each affine control point CPx (x=1,2,3) is compared. It is determined whether the reference images are identical and affine transformation is possible.

[0179] <Affine fixed merge candidate derivation> The affine fixed merge candidate derivation unit 385 will be described. The affine fixed merge candidate derivation unit 485 is similar to the affine fixed merge candidate derivation unit 385. The candidate page fixes the motion information of the affine control points with the fixed motion information. Fix the motion vector of each affine control point to (0,0).

[0180] <Temporal motion vector prediction> Before describing the temporal prediction motion vectors, the temporal relationship of pictures will be described. FIG. 55(a) shows a case where the current coding block and the current coding picture are different in time. The relationship between the currently coded picture and the coded picture is shown. A specific coded picture is defined as ColPic. ColPic is specified by the syntax. can be.

[0181] FIG. 55(b) shows a block in ColPic that is located at the same position as the coding block to be coded, and These coding blocks T 0 and T1 are at almost the same position in a picture that is different in time from the picture to be coded. is a coding block.

[0182] The above explanation of the temporal relationship of pictures is for the encoding process, but the same applies for the decoding process. In other words, when decoding, the above explanation is replaced with "decoding" and the same explanation is applied. It will be revealed.

[0183] The temporal motion vector predictor candidate derivation in the normal motion vector predictor mode derivation unit 301 in FIG. The operation of the output unit 322 will be described with reference to FIG.

[0184] First, ColPic is derived (step S4201). For the derivation of ColPic, see FIG. This will be explained in light of the above.

[0185] If the slice type slice_type is a B slice and the flag collocated_from_l0_flag is 0, If (step S4211: YES, step S4212: YES), RefPicList1[0], That is, the picture with reference index 0 in the reference list L1 is a picture colPic at a different time. If not, that is, if the slice type slice_type is B slice, If the flag collocated_from_l0_flag is 1 (step S4211: YE S, step S4212: NO), or if the slice type slice_type is a P slice (Step S4211: NO, Step S4214: YES), RefPicList0[0], that is, That is, the picture with reference index 0 in reference list L0 is the picture colPic at a different time. If slice_type is not a P slice (step S4214: NO), terminate the process.

[0186] Refer again to FIG. 56. After deriving ColPic, derive the coding block colCb and This process will be described with reference to FIG. do.

[0187] First, in a picture colPic at a different time, the bottom right corner ( The coding block located outside is defined as a coding block colCb at a different time (step S4221). This coding block corresponds to coding block T0 in FIG.

[0188] Next, the coding information of the coding block colCb at a different time is obtained (step S422 2) PredMode of coding block colCb of different time is not available or coding block of different time is not available. When the prediction mode PredMode of the coding block colCb is intra prediction (MODE_INTRA) ( Step S4223: NO, Step S4224: YES), in the picture colPic at a different time The coding block located at the same position as the coding block to be processed is located at the lower right of the center at a different time. The coding block between the first and second pixels is colCb (step S4225). This corresponds to the coding block T1.

[0189] Again, see FIG. 56. Next, for each reference list, inter prediction information is derived (S Here, for the coding block colCb, the motion vector for each reference list is Derive the mvLXCol vector and the availableFlagLXCol flag indicating whether the coding information is available or not. LX indicates the reference list. In the derivation of reference list 0, LX becomes L0, and in the derivation of reference list 1, Then, LX becomes L1. The derivation of the inter prediction information will be described with reference to FIG.

[0190] When a coding block colCb of a different time is not available (S4231S4231:NO) Or, if the prediction mode PredMode is intra prediction (MODE_INTRA) (S4232:NO) Then, the flag availableFlagLXCol and the flag predFlagLXCol are both set to 0 (step S4233 ), the motion vector mvLXCol is set to (0,0) (S4234), and the process ends.

[0191] The coding block colCb is available (S4231: YES) and the prediction mode PredMode is If it is not a trajectory prediction (MODE_INTRA) (S4232: YES), use the following procedure to check mvCol, refIdx Calculate Col and availableFlagCol.

[0192] Flag PredFlagL0[xP If [Col][yPCol] is 0 (S4235: YES), the prediction mode of the coding block colCb is Pr ed_L1, the motion vector mvCol is the L1 motion vector of the coding block colCb. The reference index refIdxCol is set to the same value as the MvL1[xPCol][yPCol] (S4236). is set to the same value as the reference index RefIdxL1[xPCol][yPCol] of L1 (S4237), The list ListCol is set to L1 (S4238). Here, xPCol and yPCol are different time The index indicates the position of the top left pixel of the coding block colCb in the picture colPic. be.

[0193] On the other hand, if the L0 prediction flag PredFlagL0[xPCol][yPCol] of the coding block colCb is not 0, If (S4235:NO), the L1 prediction flag PredFlagL1[xPCol][yPC ol] is 0. The L1 prediction flag PredFlagL1[xPCol][ If yPCol is 0 (S4239: YES), the motion vector mvCol is the It is set to the same value as the motion vector MvL0[xPCol][yPCol] of L0 (S4240). The index refIdxCol is set to the same value as the reference index RefIdxL0[xPCol][yPCol] of L0. The list ListCol is set to L0 (S4242).

[0194] L0 prediction flag PredFlagL0[xPCol][yPCol] of coding block colCb and coding block col If neither of the L1 prediction flags PredFlagL1[xPCol][yPCol] of Cb is 0 (S4235:NO, S4239:NO), the inter prediction mode of the coding block colCb is bi-predictive (Pred_BI). Therefore, one of the two motion vectors L0 and L1 is selected (S4243). 0 is the coding block when the inter prediction mode of the coding block colCb is bi-prediction (Pred_BI). 13 is a flowchart showing a procedure for deriving inter prediction information of a locked block.

[0195] First, the POCs of all pictures in all reference lists are checked against the current encoding. It is determined whether the POC is smaller than the POC of the target picture (S4251). POC of all pictures registered in L0 and L1, which are all reference lists of b If LX is smaller than the POC of the current picture to be coded (S4251: YES), Derive predicted vector candidates for L0, i.e., the motion vector of the L0 coding block to be coded If the L0 prediction is performed (S4252: YES), inter prediction is performed for the L0 side of the coding block colCb. The information is selected, and LX is L1, that is, the prediction of the motion vector of L1 of the coding block to be coded. If a candidate measurement vector is derived (S4252: NO), the L1 On the other hand, all the reference list of the coding block colCb is selected. At least one of the POCs of the pictures registered in L0 and L1 is the current picture to be coded. If the POC is larger than the current POC (S4251: NO), the flag collocated_from_l0_fla When g is 0 (S4253: YES), inter prediction information for L0 of coding block colCb If the flag collocated_from_l0_flag is 1 (S4253: NO), Select the inter prediction information for L1 of lock colCb.

[0196] When selecting inter prediction information for L0 of coding block colCb (S4252:Y ES, S4253:YES), the motion vector mvCol is set to the same value as MvL0[xPCol][yPCol] (S4254), and the reference index refIdxCol is set to the same value as RefIdxL0[xPCol][yPCol]. The list ListCol is set to L0 (S4256).

[0197] When selecting inter prediction information for L1 of coding block colCb (S4252:N O, S4253:NO), the motion vector mvCol is set to the same value as MvL1[xPCol][yPCol]. (S4257), the reference index refIdxCol is set to the same value as RefIdxL1[xPCol][yPCol] (S4258), and the list ListCol is set to L1 (S4259).

[0198] Returning to Figure 59, when inter prediction information is obtained from the coding block colCb, the flag ava Both ilableFlagLXCol and flag predFlagLXCol are set to 1 (S4244).

[0199] Next, the motion vector mvCol is scaled to obtain a motion vector mvLXCol (S42 The procedure for scaling calculation of this motion vector mvLXCol is shown in FIG. I will explain this in more detail.

[0200] The POC of the picture colPic at different times is referenced by a list ListCol of coding blocks colCb. The POC of the reference picture corresponding to the reference index refIdxCol that The distance td between the two pictures is calculated (S4261). If the POC of the reference picture referenced in the list ListCol of block colCb comes before the other in the display order, In this case, the inter-picture distance td is a positive value, and the coding block is closer to the picture colPic at a different time. If the POC of the reference picture referenced in the list ListCol of the check colCb is later in the display order, The inter-picture distance td is a negative value. td = POC of pictures colPic at different times - list of coding blocks colCb ListCol POC of the reference picture to be referenced in

[0201] The list of pictures currently to be coded LX is referenced from the POC of the currently to be coded picture. The POC of the corresponding reference picture is subtracted from the POC to calculate the inter-picture distance tb (S4262). , and the picture to be coded is referred to in the list LX of the picture to be coded rather than the picture to be coded. If the reference picture is earlier in display order, the inter-picture distance tb is a positive value and the current code If the reference picture in the list of pictures to be coded LX is later in the display order, The inter-cha distance tb is a negative value.

[0202] tb = POC of the current picture to be coded / decoded - LX reference index of the temporal merge candidate POC of the reference picture corresponding to the index

[0203] Next, the inter-picture distances td and tb are compared (S4263). If b is equal (S4263: YES), the motion vector mvLXCol is calculated using the following formula ( S4264), this scaling calculation process is terminated.

[0204] mvLXCol = mvCol

[0205] On the other hand, if the inter-picture distances td and tb are not equal (S4263: NO), the following formula is used: The variable tx is calculated (S4265).

[0206] tx = ( 16384 + Abs( td ) >> 1 ) / td

[0207] Next, the scaling coefficient distScaleFactor is calculated using the following equation (S4266). distScaleFactor = Clip3( -4096, 4095, ( tb * tx + 32 ) >> 6 ) Here, Clip3(x,y,z) is a function that limits the value z to a minimum value of x and a maximum value of y. Continued Then, the motion vector mvLXCol is calculated by the following formula (S4267), and the scaling calculation is performed. End processing.

[0208] mvLXCol = Clip3( -32768, 32767, Sign( distScaleFactor * mvLXCol ) * ( (Abs( distScaleFactor * mvLXCol ) + 127 ) >> 8 ) )

[0209] Here, Sign(x) is a function that returns the sign of the value x, and Abs(x) is a function that returns the absolute value of the value x. .

[0210] Again, refer to FIG. 56. Then, the motion vector mvL0Col of L0 is calculated by the normal prediction motion vector mvL0Col. The motion vector predictor candidate list mvpListLXN in the motion vector mode derivation unit 301 is added as a candidate. However, this addition is performed for the coding block colCb This is only possible when the flag indicating whether L1 is available or not is availableFlagL0Col=1. The vector mvL1Col is calculated based on the predicted motion vector in the normal predicted motion vector mode derivation unit 301. The candidate is added to the list of candidates mvpListLXN (S4205). Flag indicating whether the coding block colCb of reference list 1 is available or not: availableFlagL1Col=1 This is the only step. The process of the temporal motion vector predictor candidate derivation unit 322 is then completed. The above description of the normal predicted motion vector mode derivation unit 301 is for the encoding process. The same applies during decoding. That is, in the normal predicted motion vector mode derivation unit 401 in FIG. The operation of the temporal motion vector predictor candidate derivation unit 422 is the same as that of the encoding in the above description. In place, they are explained similarly.

[0211] <Time Merge> In the operation of the temporal merge candidate derivation unit 342 in the normal merge mode derivation unit 302 in FIG. This will be described with reference to FIG.

[0212] First, ColPic is derived (step S4301). Next, the coding block colCb is derived. Then, the encoding information is obtained (step S4302). The above process derives the time prediction motion vector (S4303, S4304). Since these steps are the same as steps S4201 to S4204 in the rule candidate derivation unit 322, their explanations will be omitted. do.

[0213] Next, a flag, availableFlagCol, indicating whether the coding block colCb is available or not is calculated ( S4305). When the flag availableFlagL0Col or the flag availableFlagL1Col is 1 availableFlagCol will be 1. Otherwise availableFlagCol will be 0.

[0214] Then, the motion vector mvL0Col of L0 and the motion vector mvL1Col of L1 are calculated by the above-mentioned normal matrix. The merge candidate list mergeCandList in the merge mode derivation unit 302 is added as a candidate. (S4306). However, this addition is performed by using the flag a, which indicates whether the coding block colCb is valid or not. This is only the case when validFlagCol=1. Finish.

[0215] The above explanation of the temporal merge candidate derivation unit 342 is for encoding, but the same applies for decoding. That is, the temporal merge candidate derivation in the normal merge mode derivation unit 402 in FIG. The operation of unit 442 is similarly described, substituting decoding for encoding in the above description.

[0216] <Update of historical motion vector prediction candidate list> Next, the coding information storage memory 111 on the coding side and the coding information storage memory 20 on the decoding side Methods for initializing and updating the historical motion vector predictor candidate list (HmvpCandList) in preparation for 5 FIG. 26 shows the procedure for initializing and updating the history predicted motion vector candidate list. 1 is a flowchart illustrating the above.

[0217] In this embodiment, the history motion vector predictor candidate list HmvpCandList is updated based on the coding information. The information storage memory 111 and the encoded information storage memory 205 are assumed to be implemented. A history prediction motion vector candidate list update unit is included in the prediction unit 102 and the inter prediction unit 203. It may be set to update the historical motion vector predictor candidate list HmvpCandList.

[0218] At the beginning of the slice, the historical motion vector prediction candidate list HmvpCandList is initialized. On the encoding side, the prediction method decision unit 105 decides whether the normal predicted motion vector mode or the normal merge mode is selected. When the selected motion vector prediction candidate list HmvpCandList is updated, The prediction information decoded by the bit stream decoding unit 201 is in the normal predicted motion vector mode or the normal In the case of merge mode, the historical motion vector predictor candidate list HmvpCandList is updated.

[0219] Used when performing inter prediction in normal prediction motion vector mode or normal merge mode. The inter prediction information is stored as the inter prediction information candidate hMvpCand in the history prediction motion vector candidate list. The inter prediction information candidate hMvpCand is registered in the HmvpCandList. The reference index for L0 and L1, refIdxL1, indicates whether L0 prediction is performed. An L0 prediction flag predFlagL0 and an L1 prediction flag predFl indicating whether or not L1 prediction is performed agL1, the motion vector mvL0 for L0, and the motion vector mvL1 for L1 are included.

[0220] The encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side are provided The elements registered in the history motion vector predictor candidate list HmvpCandList (i.e., Among the inter prediction information candidates, there is inter prediction information with the same value as the inter prediction information candidate hMvpCand. If it exists, remove the element from the historical motion vector predictor candidate list HmvpCandList. On the other hand, if there is no inter-prediction information with the same value as the inter-prediction information candidate hMvpCand, Then, the first element of the history prediction motion vector candidate list HmvpCandList is deleted, and the history prediction motion vector The inter-prediction information candidate hMvpCand is added to the end of the vector candidate list HmvpCandList.

[0221] The encoding information storage memory 111 on the encoding side and the encoding information storage memory 2 on the decoding side according to the present invention The number of elements in the historical motion vector predictor candidate list HmvpCandList for 05 is six.

[0222] First, initialize the historical motion vector predictor candidate list HmvpCandList in slice units. (Step S2101 in FIG. 26). At the beginning of the slice, the history predicted motion vector candidate list Hm All elements of vpCandList are emptied and registered in the historical motion vector prediction candidate list HmvpCandList. The number of recorded history motion vector predictor candidates (current number of candidates). The value of NumHmvpCand is set to 0. do.

[0223] Note that the initialization of the historical predicted motion vector candidate list HmvpCandList is performed on a slice-by-slice basis ( However, it was decided to implement it on a picture-by-picture, tile-by-tile or tree block basis. This may be done on a row-by-row basis.

[0224] Next, for each coding block in the slice, the following historical motion vector predictor candidate list Hmvp The process of updating the CandList is repeated (steps S2102 to S2107 in FIG. 26).

[0225] First, the initial setting is performed for each coding block. Set the value of the field "identicalCandExist" to FALSE, and then delete the target index that indicates the candidate for deletion. The index removeIdx is set to 0 (step S2103 in FIG. 26).

[0226] It is determined whether or not there is an inter-prediction information candidate hMvpCand to be registered (step S20 in FIG. 26). In the prediction method determination unit 105 on the encoding side, the normal prediction motion vector mode or When the normal merge mode is determined, or when the bitstream decoding unit 201 on the decoding side performs normal prediction, When decoded as a motion vector mode or normal merge mode, the inter prediction information The information is set as the inter prediction information candidate hMvpCand to be registered. Intra prediction mode, sub-block prediction motion vector mode or sub-block merge mode, or when the bitstream decoding unit 201 on the decoding side determines that the bitstream is an intra-prediction mode, Decoded as sub-block predicted motion vector mode or sub-block merge mode In this case, the historical motion vector predictor candidate list HmvpCandList is not updated, and the registered object There is no inter-prediction information candidate hMvpCand. If it does not exist, steps S2105 to S2106 are skipped (step S in FIG. 26). 2104: NO). If there is an inter-prediction information candidate hMvpCand to be registered, The processes from S2105 onwards are carried out (step S2104 in FIG. 26: YES).

[0227] Next, the input motion vector to be registered is added to each element of the history motion vector candidate list HmvpCandList. The elements (inter prediction information) with the same value as the target prediction information candidate hMvpCand, i.e., the same elements It is determined whether or not the same element exists (step S2105 in FIG. 26). 10 is a flowchart of a processing procedure. (Step S2121 in FIG. 27: NO), the historical motion vector predictor candidate list HmvpCa Since the ndList is empty and there are no identical candidates, steps S2122 to S2125 in FIG. 27 are skipped. The number of historical motion vector predictor candidates NumHmvpC is If the value of and is greater than 0 (step S2121 in FIG. 27: YES), the history predicted motion vector The process of step S2123 is repeated for the memory index hMvpIdx from 0 to NumHmvpCand-1. First, the history of the motion vector predictor candidate list is returned (steps S2122 to S2125 in FIG. 27). The hMvpIdx-th element HmvpCandList[hMvpIdx], counting from 0 in the list, is the inter prediction information candidate hM It is compared with vpCand to see if it is the same (step S2123 in FIG. 27). Step S2123: YES), a flag identicalCandE indicating whether an identical candidate exists or not Set the value of TRUE in xist and the deletion target index remo, which indicates the position of the element to be deleted. Set the current value of the history predicted motion vector index hMvpIdx to veIdx and check the same element. If they are not the same (step S2123 in FIG. 27: NO), increment hMvpIdx by 1. If the history prediction motion vector index hMvpIdx is less than or equal to NumHmvpCand-1, If so, the process proceeds to step S2123 and thereafter.

[0228] Returning to the flowchart of FIG. 26, the historical motion vector predictor candidate list HmvpCandList The element shift and addition process is performed (step S2106 in FIG. 26). Step S2106: Shifting / adding elements of the history prediction motion vector candidate list HmvpCandList 1 is a flowchart of a processing procedure. First, in the historical motion vector predictor candidate list HmvpCandList, Either remove the stored elements and then add new elements, or add new elements without removing any elements. Specifically, the flag "identicalCandE" indicates whether or not an identical candidate exists. A comparison is made with xist to see if it is TRUE or if NumHmvpCand is 6 (step S2141 in FIG. 28). ). The flag identicalCandExist indicates whether an identical candidate exists or not. When the number of candidates NumHmvpCand is 6 (step S2141 in FIG. 28: YES), except for the elements stored in the historical motion vector predictor candidate list HmvpCandList. Add a new element from . Set the initial value of index i to the value of removeIdx + 1. The element shift process in step S2143 is repeated from the initial value of NumHmvpCand to NumHmvpCand. Steps S2142 to S2144 of 28. HmvpCandList[i - 1] is set to HmvpCandList[i] Shift the elements forward by copying the elements of i (step S2143 in FIG. 28), and set i to Increment by 1 (steps S2142 to S2144 in FIG. 28). The (NumHmvpCand-1)th HmvpCandList, counting from 0, which corresponds to the end of the motion vector candidate list. The inter-prediction information candidate hMvpCand is added to t[NumHmvpCand-1] (step S214 in FIG. 28). 5) Finish the element shift and addition process of this historical predicted motion vector candidate list HmvpCandList. On the other hand, the flag identicalCandExist, which indicates whether an identical candidate exists, is TRUE. If neither of the conditions of NumHmvpCand and NumHmvpCand is 6 is satisfied (step S2141 in FIG. 28: NO), ), without excluding the elements stored in the history motion vector predictor candidate list HmvpCandList, Add the inter prediction information candidate hMvpCand to the end of the historical prediction motion vector candidate list (Figure 2). 8, step S2146). Here, the end of the history motion vector predictor candidate list is a list starting from 0. It is the NumHmvpCand-th HmvpCandList[NumHmvpCand] counting from the beginning. Increment the element shift and Then the addition process is completed.

[0229] FIG. 31 is a diagram for explaining an example of a process of updating the history predicted motion vector candidate list. The elements (inter prediction information) of the registered historical predicted motion vector candidate list HmvpCandList When adding a new element to the history motion vector predictor candidate list HmvpCandList, The new inter prediction information is compared with the previous prediction information (Figure 31A). If the value is the same as the third element HMVP2 from the top of the vector candidate list HmvpCandList, The element HMVP2 is deleted from the motion vector predictor candidate list HmvpCandList, and the following elements HMVP3 to HM Shift (copy) VP5 forward one by one and create a history prediction motion vector candidate list HmvpCandList t (FIG. 31B) and the history predicted motion vector candidate list HmvpCan The update of dList is then completed (Figure 31C).

[0230] <Historical Prediction Motion Vector Candidate Derivation Process> Next, the history of predicted motion vector candidates in the normal predicted motion vector mode derivation unit 301 on the encoding side is The prediction motion vector mode derivation unit 323 and the history prediction motion vector mode derivation unit 401 on the decoding side The process of step S304 in FIG. 20 is common to all the torque candidate derivation units 423. A method for deriving a historical predicted motion vector candidate from a historical predicted motion vector candidate list HmvpCandList FIG. 29 is a flowchart for explaining a procedure for deriving a historical motion vector predictor candidate. This is a low chart.

[0231] The number of current motion vector predictor candidates numCurrMvpCand is the motion vector predictor candidate list mvpList The number of elements of tLX is equal to or greater than the maximum number of elements (2 in this example) or the number of history predicted motion vector candidates is equal to or greater than NumHm If the value of vpCand is 0 (step S2201 in FIG. 29: NO), step S22 in FIG. The process from S2202 to S2209 is omitted, and the procedure for deriving the historical predicted motion vector candidate is terminated. The current number of motion vector predictor candidates numCurrMvpCand is the motion vector predictor candidate list mvpList If the number of elements of tLX is smaller than 2, and the number of history motion vector predictor candidates NumHmvpCa If the value of nd is greater than 0 (step S2201 in FIG. 29: YES), Processes S2202 to S2209 are carried out.

[0232] Next, the index i goes from 1 to 4 and the number of historical motion vector predictor candidates numCheckedHMVP The process of steps S2203 to S2208 in FIG. 29 is repeated until the smaller value of The current number of motion vector predictor candidates, nu, is returned (steps S2202 to S2209 in FIG. 29). When mCurrMvpCand is equal to or greater than 2, which is the maximum number of elements in the motion vector predictor candidate list mvpListLX. (Step S2203 in FIG. 29: NO), Steps S2204 to S2209 in FIG. 29 The process is omitted, and the procedure for deriving the historical motion vector predictor candidate is terminated. The number of vector candidates, numCurrMvpCand, is the maximum number of elements in the motion vector predictor candidate list, mvpListLX. If the value is smaller than 2 (step S2203 in FIG. 29: YES), step S2 Process 204 and onwards.

[0233] Next, the process from step S2205 to S2207 is performed when Y is 0 and 1 (L0 and L1). The process is repeated for each of the current predicted motion vectors (steps S2204 to S2208 in FIG. 29). The number of motion vector predictor candidates, numCurrMvpCand, is the maximum number of elements in the motion vector predictor candidate list, mvpListLX. If the number of the first pair is two or more (step S2205 in FIG. 29: NO), the process proceeds from step S2206 in FIG. Therefore, the process of S2209 is omitted, and the process of deriving the historical motion vector predictor candidate is terminated. The number of current motion vector predictor candidates numCurrMvpCand is the motion vector predictor candidate list mvpListLX If the number of elements is smaller than 2, which is the maximum number of elements in (step S2205 in FIG. 29: YES), The process proceeds to step S2206 and thereafter.

[0234] Next, the motion vector to be encoded / decoded is added to the history prediction motion vector candidate list HmvpCandList. The element of the same reference index as the vector reference index refIdxLX. If the element is different from any element in the vector list mvpListLX (step S2206 in FIG. 29), : YES), the numCurrMvpCand-th element mvpL of the motion vector predictor candidate list, counting from 0 istLX[numCurrMvpCand] is the LY of the history predicted motion vector candidate HmvpCandList[NumHmvpCand - i] (Step S2207 in FIG. 29) to the current predicted motion vector candidate Increment the number of historical motion vector predictor candidates, numCurrMvpCand, by 1. In the list, there is a reference index that is the same as the reference index refIdxLX of the motion vector to be coded / decoded. An element of the motion vector predictor list mvpListLX that is different from any element of the motion vector predictor list mvpListLX If not (step S2207 in FIG. 29: NO), skip the additional process of step S2207. Upload.

[0235] The above steps S2205 to S2207 in FIG. 29 are performed on both L0 and L1. (Steps S2204 to S2208 in FIG. 29). Increment the index i by 1. , the index i is the smaller value between 4 and the number of historical motion vector predictor candidates NumHmvpCand In the following cases, the process from step S2203 onward is performed again (steps S2202 to S2204 in FIG. 29). S2209).

[0236] <History merge candidate derivation process> Next, the history merge candidate derivation unit 345 of the normal merge mode derivation unit 302 on the encoding side, This is a common process between the normal merge mode derivation unit 402 and the history merge candidate derivation unit 445. The history from the history merge candidate list HmvpCandList, which is the processing procedure of step S404 in The method of deriving merge candidates will now be described in detail. Figure 30 shows the history merge candidate derivation process. 1 is a flowchart for explaining the process.

[0237] First, an initialization process is performed (step S2301 in FIG. 30). Set the value of FALSE for each (rrMergeCand -1)th element and set the variable numOrigMergeCand to the current Sets numCurrMergeCand, the number of elements currently in the merge candidate list.

[0238] Next, set the initial value of the index hMvpIdx to 1, and then run the Then, the additional processing from step S2303 to step S2310 in FIG. 30 is repeated (FIG. 3 0 steps S2302 to S2311). Elements registered in the current merge candidate list If the number of merge candidates, numCurrMergeCand, is not less than (MaxNumMergeCand-1), Since merge candidates have been added to all elements in the candidate list, the process of deriving merge candidates in this history is The process ends (step S2303 in FIG. 30: NO). If the number of elements in the merge candidate list, numCurrMergeCand, is less than or equal to (MaxNumMergeCand-1), the merge candidate list is The process from step S2304 onwards is carried out. Set the value of sameMotion to FALSE (see Figure 30). (Step S2304). Next, the initial value of index i is set to 0, and The processes in steps S2306 and S2307 in FIG. 30 are repeated up to numOrigMergeCand-1 (FIG. 30 S2305 to S2308 of the previous motion vector prediction candidate list. Counting from 0 in the history motion vector prediction candidate list (NumHmvp Cand - hMvpIdx)th element HmvpCandList[NumHmvpCand- hMvpIdx] is the merge candidate list. Counting from 0, compare whether the value is the same as the i-th element mergeCandList[i] (step in Figure 30). (P2306).

[0239] The same value for a merge candidate means that all components (inter prediction mode, reference If the values ​​of the merge candidates (reference index, motion vector) are the same, the merge candidates are considered to have the same value. If the complements are the same and isPruned[i] is FALSE (step S2306 in FIG. 30: YES), Both sameMotion and isPruned[i] are set to TRUE (step S2307 in FIG. 30). If the values ​​are not the same (step S2306 in FIG. 30: NO), the process proceeds to step S2307. The process from step S2305 to step S2308 in FIG. When the process is completed, compare whether sameMotion is FALSE (step S230 in FIG. 30). 9) If sameMotion is FALSE (step S2309 in FIG. 30: YES), that is, That is, the (NumHmvpCand - hMvpIdx)th element in the history motion vector candidate list, counting from 0 The element HmvpCandList[NumHmvpCand - hMvpIdx] does not exist in mergeCandList, so it is not a merge candidate. The history prediction movement vector is stored in the numCurrMergeCand-th mergeCandList[numCurrMergeCand] of the list. The (NumHmvpCand - hMvpIdx)th element of the candidate list, counting from 0, is HmvpCandList[NumHmv pCand - hMvpIdx] and increment numCurrMergeCand by 1 (step 30 in Figure 30). The index hMvpIdx is incremented by 1 (step S23 in FIG. 30). 02), steps S2302 to S2311 in FIG. 30 are repeated. All elements in the history motion vector prediction candidate list are checked or the merge candidate list is Once all elements in the history have been added with merge candidates, the process of deriving merge candidates for this history is complete. .

[0240] <Sub-block temporal merge candidate derivation> Sub-block temporal merging candidates in the sub-block merging mode derivation unit 304 in FIG. The operation of the derivation unit 381 will be described with reference to FIG.

[0241] First, it is determined whether the coding block is smaller than 8x8 pixels (S4002). If the coding block is less than 8x8 pixels (S4002: YES), the sub-block time marker The flag indicating the presence of a candidate block is set to availableFlagSbCol=0 (S4003), and the subblock is The process of the time-varying merge candidate derivation unit is completed. Here, the temporal motion is calculated by the syntax. When vector prediction is inhibited or sub-block temporal merging is inhibited, In this case, the same process as when the coding block is less than 8x8 pixels (S4002: YES) is performed. .

[0242] On the other hand, if the coding block is 8x8 pixels or more (S4002: NO), Next, adjacent motion information of the coding block in the pixel is derived (S4004).

[0243] The process of deriving adjacent motion information of a coding block will be described with reference to FIG. The process of deriving adjacent motion information corresponds to the process of the spatial prediction motion vector candidate derivation unit 321 described above. However, the order of searching the adjacent blocks is A0, B0, B1, A1, and B2 is not searched. First, the coding information is obtained for adjacent block n=A0 (S4052). The information includes a flag, availableFlagN, indicating whether adjacent blocks are available, and , and a motion vector mvLXN.

[0244] Next, it is determined whether the adjacent block n is valid or invalid (S4054). If the flag indicating whether or not the request can be made is "availableFlagN=1", the request is valid; otherwise, the request is invalid. If adjacent block n is valid (S4054: YES), the reference index refIdxLXN is The reference index of the adjacent block n is set as refIdxLXn (S4056). The motion vector mvLXN of the adjacent block n is set as the motion vector mvLXn of the adjacent block n (S4056). The process of deriving motion information is then completed.

[0245] On the other hand, if adjacent block n is invalid (S4106: NO), adjacent block n=B0, The encoding information is obtained (S4104). The same process is then repeated for B1 and A1 in the order shown in the loop. The process of deriving adjacent motion information loops until the adjacent blocks are valid. If the adjacent blocks A0, B0, B1, and A1 are invalid, the process of deriving the adjacent motion information of the blocks is terminated. Complete.

[0246] Referring again to FIG. 50, after the adjacent motion information is derived (S4004), the temporal motion A vector is derived (S4006). The process of deriving a temporal motion vector will be described with reference to FIG. The temporal motion vector tempMv is initialized as (0,0) (S4062).

[0247] Next, it is determined whether the adjacent motion information is valid or invalid (S4064). If the flag indicating whether the neighboring motion is available or not is set to availableFlagN=1, it is valid, otherwise it is invalid. If the information is invalid (S4064: NO), the process of deriving the temporal motion vector is terminated. Complete.

[0248] On the other hand, if the adjacent motion information is valid (S4064: YES), the L1 prediction is performed for the adjacent block N. It is determined whether the flag predFlagL1N indicating whether the measurement is being used is 1 or not (S4066). If predFlagL1N=0 (S4066: NO), proceed to the next process (S4078). When L1N=1 (S4066:YES), all the pictures registered in all the reference lists are It is determined whether the POC of the current picture is equal to or smaller than the POC of the current picture to be coded (S4068). If this determination is true (S4068: YES), the process proceeds to the next step (S4070).

[0249] If the slice type slice_type is a B slice and the flag collocated_from_l0_flag is 0, If (S4070: YES and S4072: YES), ColPic and reference picture RefPicList1[ refIdxL1N] (picture with reference index refIdxL1N in reference list L1) is the same If this determination is true (S4074: YES), the temporal movement Let the vector tempMv = mvL1N (S4076). If this determination is false (S4074: NO), The process proceeds to step S4078. If llocated_from_l0_flag is not 0 (S4070: NO, or S4072: NO), Proceed to the next process (S4078).

[0250] A flag predFlagL0N indicating whether L0 prediction is used in the adjacent block N It is determined whether predFlagL0N is 1 (S4078). If predFlagL0N=1 (S4078: YES), Co lPic and reference picture RefPicList0[refIdxL0N] (reference index refIdx in reference list L0) If the picture in the L0N column is the same as the picture in the L1 column (S4080), the process determines whether the picture in the L0N column is the same as the picture in the L1 column (S4080). 80: YES), and the temporal motion vector tempMv=mvL0N (S4082). If it is false (S4080: NO), the process of deriving the temporal motion vector ends. Again, refer to FIG. 50. Next, ColPic is derived (S4016). This process is performed using time prediction. This is the same as S4201 in the motion vector candidate derivation unit 322, so the explanation will be omitted. .

[0251] Then, a coding block colCb for a different time is set (S4017). The block is located at the same position as the target coding block in the picture ColPic at the same time. The coding block is set as colCb. This coding block is the code in FIG. This corresponds to encryption block T1.

[0252] Next, the position where the temporal motion vector tempMv is added to the coding block colCb is newly The upper left position of the coding block colCb is (xColCb, yColCb) (S4018). If the temporal motion vector tempMv is (tempMv[0], tempMv[1]) with 1 / 16 pixel accuracy, The top left position of colCb is as follows:

[0253] xColCb = Clip3( xCtb, xCtb + CtbSizeY + 3, xcolCb + ( tempMv[0] >> 4 ) ) yColCb = Clip3( yCtb, yCtb + CtbSizeY - 1, ycolCb + ( tempMv[1] >> 4 ) )

[0254] Here, the top left position of the tree block is (xCtb, yCtb), and the size of the tree block is CtbS. As shown in the above formula, the position after adding tempMv is significantly different from the position before adding tempMv. The position is corrected to within the range of the tree block size so that it does not get out of the screen. If this occurs, the image will be corrected within the screen.

[0255] The prediction mode PredMode of this coding block colCb is inter prediction (MODE_INTER If the prediction mode of colCb is not inter prediction (S4020), 4020: NO), flag indicating the presence of sub-block time merge candidates availableFlagSbCol= It is set to 0 (S4003), and the processing of the sub-block temporal merge candidate derivation unit is terminated.

[0256] On the other hand, if the prediction mode of colCb is inter prediction (S4020: YES), Inter prediction information is derived for each of the colCb and colCb (S4022, S4023). , the central motion vector ctrMvLX for each reference list, and a flag indicating whether LX prediction is used or not. Derive the lag ctrPredFlagLX. LX indicates the reference list. In the derivation of reference list 0, LX is L0. In the derivation of reference list 1, LX is L1. Regarding the derivation of inter prediction information, This will be explained with reference to 3.

[0257] If the coding block colCb of a different time is not available (S4112: NO), or if If the prediction mode PredMode is intra prediction (MODE_INTRA) (S4114:NO), the flag ava The ilableFlagLXCol and flag predFlagLXCol are both set to 0 (step S4116). The tor mvCol is set to (0,0) (S4118), and the process of deriving inter prediction information is terminated. .

[0258] If the coding block colCb is available (S4112: YES) and the prediction mode PredMode is If it is not a trajectory prediction (MODE_INTRA) (S4114: YES), use the following procedure to set mvCol, refIdx Calculate Col and availableFlagCol.

[0259] Flag PredFlagLX[xP If [yPCol] is 1 (S4120: YES), the motion vector mvCol is the coding block col It is set to the same value as MvLX[xPCol][yPCol], which is the motion vector of LX of Cb (S4122). The reference index refIdxCol is the same as the reference index RefIdxLX[xPCol][yPCol] of LX. (S4124), and the list listCol is set to LX (S4126). , xPCol, yPCol are the upper left pixel of the coding block colCb in the picture colPic at different times. is an index indicating the position of

[0260] On the other hand, a flag PredFla indicating whether LX prediction of the coding block colCb is used If gLX[xPCol][yPCol] is 0 (S4120: NO), the following process is performed. First, The POC of all pictures registered in the reference list is the same as that of the current picture to be coded. It is judged whether it is below POC or not (S4128). And, LY prediction of colCb is used. It is determined whether the flag PredFlagLY[xPCol][yPCol] indicating whether the poling address is 1 or not (S4128). Here, we define LY prediction as a reference list different from LX prediction. That is, when LX=L0, LY=L1, When LX=L1, LY=L0.

[0261] If this determination is true (S4128: YES), the motion vector mvCol is It is set to the same value as MvLY[xPCol][yPCol], which is the LY motion vector of Cb (S4130). The reference index refIdxCol is the same as the reference index RefIdxLY[xPCol][yPCol] of LY. (S4132), and the list listCol is set to LX (S4134).

[0262] On the other hand, if this judgment is false (S4128: NO), the flag availableFlagLXCol and the flag Both predFlagL, X, and Col are set to 0 (step S4116), and the motion vector mvCol is set to (0, 0). Then, (S4118), the process of deriving inter prediction information ends. When inter prediction information is obtained from the coding block colCb, the flag availableFlagLXCol and flag predFlagLXCol are both set to 1 (S4136).

[0263] Next, the motion vector mvCol is scaled to obtain a motion vector mvLXCol (S41 38). This process is the same as S4245 in the temporal motion vector predictor candidate derivation unit 322. Therefore, the description will be omitted.

[0264] Refer again to FIG. 50. After deriving the inter prediction information for each reference list, The calculated motion vector mvLXCol is the center motion vector ctrMvLX, and the calculated flag predFlagLXCol is The flag is set to ctrPredFlagLX (S4022, S4023).

[0265] Then, it is determined whether the central motion vector is valid or invalid (S4024). If 0 and ctrPredFlagL1=0, it is invalid. Otherwise, it is invalid. If the subblock time merge candidate is available (S4024: NO), the flag available is set to Set leFlagSbCol=0 (S4003) and execute the process of the sub-block time merge candidate derivation part. Finish.

[0266] On the other hand, if the central motion vector is valid (S4024: YES), the sub-block time marker The flag indicating the presence of a candidate block is set to availableFlagSbCol=1 (S4025), and the subblock is This process will be described with reference to FIG.

[0267] First, the number of sub-blocks in the width direction is calculated from the width cbWidth and height cBheight of the coding block colCb. numSbX and the number of sub-blocks in the height direction numSbY are calculated (S4152). LXSbCol=0 (S4152). After this process, the process is repeated in units of the prediction sub-block colSb. This iteration repeats the process of changing the height index ySbIdx from 0 to numSbY. Processing is performed while changing the width direction index xSbIdx from 0 to numSbX.

[0268] If the top left position of the coding block colCb is (xCb, yCb), then the left The top position (xSb, ySb) is calculated as follows:

[0269] xSb = xCb + xSbIdx * sbWidth ySb = yCb + ySbIdx * sbHeight

[0270] Next, the position where the temporal motion vector tempMv is added to the predicted sub-block colSb is The top left position of the predicted sub-block colSb is (xColSb, yColSb). If the temporal motion vector tempMv is (tempMv[0], tempMv[1]) with 1 / 16 pixel accuracy, The top left position of colSb is as follows:

[0271] xColSb = Clip3( xCtb, xCtb + CtbSizeY + 3, xSb + ( tempMv[0] >> 4 ) ) yColSb = Clip3( yCtb, yCtb + CtbSizeY - 1, ySb + ( tempMv[1] >> 4 ) )

[0272] Here, the top left position of the tree block is (xCtb, yCtb), and the size of the tree block is CtbS. As shown in the above formula, the position after adding tempMv is significantly different from the position before adding tempMv. The position is corrected to within the range of the tree block size so that it does not get out of the screen. If this occurs, the image will be corrected within the screen.

[0273] Then, inter prediction information is derived for each reference list (S4156, S4158). Here, for the prediction subblock colSb, the movement of each reference list is calculated for each subblock. A vector mvLXSbCol and a flag availableFlagLXSbCo indicating whether the prediction subblock is available or not LX indicates the reference list, and in the derivation of reference list 0, LX becomes L0, and In the derivation of step 1, LX becomes L1. The derivation of the inter prediction information is performed in steps S4022 and S4023 of FIG. 3, so the explanation will be omitted.

[0274] After deriving the inter prediction information (S4156, S4158), the prediction sub-block colSb is valid. It is determined whether availableFlagL0SbCol=0 and availableFlagL1SbCol=0 (S4160). If colSb is invalid (S4160:N O), the motion vector mvLXSbCol is set as the central motion vector ctrMvLX (S4162). Then, a flag predFlagLXSbCol indicating whether or not LX prediction is used is added to the central motion vector. The flag in the sub-block is set to ctrPredFlagLX (S4162). This completes the derivation of

[0275] Again, refer to FIG. 50. Then, the motion vector mvL0SbCol of L0 and the motion vector of L1 The tor mvL1SbCol is calculated by the sub-block merge mode derivation unit 304. The block is added as a candidate to the merge candidate list subblockMergeCandList (S4028). This addition is done when the flag availableSbCol=1 indicates the presence of a subblock time merge candidate. This is the only step that is performed by the temporal merge candidate derivation unit 342.

[0276] The above description of the sub-block temporal merge candidate derivation unit 381 is for the encoding process. The same applies to the decoding process. The operation of the sub-block temporal merge candidate derivation unit 481 is the same as the above-described coding and decoding. and is explained in the same manner.

[0277] <Motion compensation prediction processing> The motion compensation prediction unit 306 predicts the current block being predicted during coding. The motion compensation prediction unit 306 obtains the position and size of the inter prediction information. The inter prediction mode is obtained from the inter prediction mode determination unit 305. The pixel value and the motion vector are derived and stored in the decoded image memory 104. The reference picture to be used is moved from the same position as the image signal of the prediction block by the amount of the motion vector. After acquiring the image signal at the position where the image has been shifted, a prediction signal is generated.

[0278] In inter prediction, the inter prediction mode is a single reference picture, such as L0 prediction or L1 prediction. In the case of prediction from a reference picture, the prediction signal obtained from one reference picture is called the motion compensated prediction signal. The prediction mode is a prediction mode from two reference pictures, such as BI prediction. In the case of prediction, the weighted average of the prediction signals obtained from two reference pictures is used as the motion vector. The bidirectional motion compensation prediction signal is supplied to the prediction method determination unit 105. The weighted average of the predictions is set to a ratio of 1:1, but weighting can be done using other ratios. For example, the picture interval between the picture to be predicted and the reference picture is close. The weighting ratio may be set to be larger for a larger number of pixels. This may be done using a correspondence table of combinations of channel intervals and weighting ratios.

[0279] The motion compensation prediction unit 406 has the same function as the motion compensation prediction unit 306 on the encoding side. The compensation prediction unit 406 outputs the inter prediction information to the normal prediction motion vector mode derivation unit 401, A normal merge mode derivation unit 402, a sub-block predicted motion vector mode derivation unit 403, The block merge mode is obtained from the block merge mode derivation unit 404 via the switch 408. The prediction unit 406 supplies the obtained motion compensation prediction signal to the decoded image signal superimposition unit 207 .

[0280] <Inter prediction mode> The process of making predictions from a single reference picture is defined as uni-prediction, and in the case of uni-prediction, it is called L0 prediction. Or L1 prediction, which is a prediction using one of the two reference pictures registered in the reference lists L0 and L1. A prediction is made using either one of them.

[0281] FIG. 32 shows a uni-prediction model in which the L0 reference picture (RefL0Pic) is the picture to be processed. FIG. 33 shows a case where the time is before the CurPic (CurPic). In the example shown in FIG. 1, the reference picture is located after the current picture. 32 and 33, the reference picture for L0 prediction is changed to the reference picture for L1 prediction (RefL1Pi It is also possible to replace it with c) and perform uni-prediction.

[0282] The process of making predictions from two reference pictures is defined as bi-prediction, and in the case of bi-prediction, it is called L0 prediction. The bi-prediction is expressed as BI prediction using both L0 and L1 prediction. The reference picture for L1 prediction is located before the current picture and the reference picture for L1 prediction is the current picture. FIG. 35 shows a case where the picture is bi-predictive and has a reference for L0 prediction. This indicates that the reference picture for L1 prediction and the reference picture for L2 prediction are located at a time earlier than the picture being processed. FIG. 36 shows a bi-prediction example in which a reference picture for L0 prediction and a reference picture for L1 prediction are processed. This shows the case where the picture is located after the picture to be processed.

[0283] In this way, the relationship between the L0 / L1 prediction type and time is that L0 is the past direction and L1 is the future direction. In the case of bi-prediction, the same reference picture can be used. Alternatively, the motion compensation prediction may be performed in a uni-predictive manner. The decision as to whether to use L0 prediction or bi-prediction is made based on, for example, whether to use L1 prediction or not. The determination is made based on information (for example, a flag) indicating whether or not the

[0284] <About reference indexes> In the embodiment of the present invention, in order to improve the accuracy of the motion compensation prediction, This makes it possible to select the most suitable reference picture from among a number of reference pictures. The reference picture used in the motion compensation prediction is used as a reference index, and The index is coded into the bitstream together with the differential motion vector.

[0285] <Motion compensation process based on normal predicted motion vector mode> The motion compensation prediction unit 306 is also shown in the inter prediction unit 102 on the encoding side in FIG. In this way, the inter prediction mode determination unit 305 performs normal prediction motion vector mode derivation. When the inter prediction information by the unit 301 is selected, the inter prediction information is - The inter prediction mode of the block currently being processed is obtained from the prediction mode determination unit 305. The prediction mode, reference index, and motion vector are derived, and a motion compensation prediction signal is generated. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0286] Similarly, the motion compensation prediction unit 406 also performs the same processing as the inter prediction unit 203 on the decoding side in FIG. As shown, during the decoding process, a switch 408 is turned on to the normal predicted motion vector mode derivation unit 40. When connected to 1, inter prediction by the normal prediction motion vector mode derivation unit 401 The information is obtained and the inter prediction mode and reference index of the currently processed block are The generated motion compensation prediction signal is then used to derive a motion vector. The signal is supplied to a decoded image signal superimposing unit 207.

[0287] <Motion compensation processing based on normal merge mode> The motion compensation prediction unit 306 is also shown in the inter prediction unit 102 on the encoding side in FIG. In this way, the inter prediction mode determination unit 305 determines whether the normal merge mode is When inter prediction information by the inter prediction mode is selected, the inter prediction information is the inter prediction mode of the block currently being processed, obtained from the inter prediction mode determination unit 305; A reference index and a motion vector are derived, and a motion compensation prediction signal is generated. The compensated prediction signal is supplied to a prediction method determination unit 105.

[0288] Similarly, the motion compensation prediction unit 406 also performs the same processing as the inter prediction unit 203 on the decoding side in FIG. As shown, during the decoding process, the switch 408 is normally connected to the merge mode derivation unit 402. If the merge mode is not set, the inter prediction information is obtained from the normal merge mode derivation unit 402, and the current process is performed. The inter prediction mode, reference index, and motion vector of the block being processed are The generated motion compensation prediction signal is multiplied by the decoded image signal. The signal is supplied to the folding unit 207.

[0289] <Motion compensation process based on sub-block predicted motion vector mode> The motion compensation prediction unit 306 is also shown in the inter prediction unit 102 on the encoding side in FIG. In this way, the inter prediction mode determination unit 305 determines the sub-block predicted motion vector model. When the inter prediction information is selected by the code derivation unit 303, this inter prediction information The inter prediction mode determination unit 305 obtains the inter prediction mode of the block currently being processed. Derives the center prediction mode, reference index, and motion vector, and generates a motion compensated prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0290] Similarly, the motion compensation prediction unit 406 also performs the same processing as the inter prediction unit 203 on the decoding side in FIG. As shown, during the decoding process, a switch 408 controls the subblock prediction motion vector mode. When the sub-block prediction motion vector mode derivation unit 403 is connected to the The inter prediction information is obtained by the current block being processed. The generated code, reference index, and motion vector are derived to generate a motion compensation prediction signal. The resulting motion compensation prediction signal is supplied to a decoded image signal superimposing unit 207 .

[0291] <Motion compensation processing based on sub-block merge mode> The motion compensation prediction unit 306 is also shown in the inter prediction unit 102 on the encoding side in FIG. In this way, the inter prediction mode determination unit 305 includes a sub-block merge mode derivation unit When the inter prediction information by 304 is selected, the inter prediction information is Inter prediction of the block currently being processed obtained from the prediction mode determination unit 305 A mode, a reference index, and a motion vector are derived, and a motion compensation prediction signal is generated. The motion compensation prediction signal is supplied to the prediction method determination unit 105 .

[0292] Similarly, the motion compensation prediction unit 406 also performs the same processing as the inter prediction unit 203 on the decoding side in FIG. As shown, during the decoding process, a switch 408 controls the sub-block merge mode derivation unit 404 to , the inter prediction information by the subblock merge mode derivation unit 404 The inter prediction mode and reference index of the currently processed block are obtained. , derive a motion vector, and generate a motion compensation prediction signal. The generated motion compensation prediction signal is , and is supplied to the decoded image signal superimposing unit 207.

[0293] <Motion compensation processing based on affine transformation prediction> In normal motion vector prediction mode and normal merge mode, the following flags are used: The following flags are used to indicate whether the affine model is used in the encoding process. Based on the inter prediction conditions determined by the inter prediction mode determination unit 305, the following flags are The bitstream is then coded into the bitstream. Determine whether to perform affine model motion compensation based on the following flags in the program: .

[0294] sps_affine_enabled_flag is the flag for motion compensation using the affine model in inter prediction. If sps_affine_enabled_flag is 0, it is possible to use The motion compensation is suppressed so that it is not based on the affine model. cu_affine_type_flag specifies the CU (coded block) syntax of the coded video sequence. If sps_affine_enabled_flag is set to 1, the encoded video sequence is not transmitted. In this case, affine model-based motion compensation can be used.

[0295] sps_affine_type_flag is the 6-parameter affine model for inter prediction. Indicates whether motion compensation is available. If sps_affine_type_flag is 0, then 6 parameters The motion compensation is suppressed so that it is not based on the affine model. is not conveyed in the CU syntax of a coded video sequence. If e_flag is 1, the 6-parameter affine model is used in the coded video sequence. If sps_affine_type_flag is not present, it is assumed to be 0. do.

[0296] When decoding a P or B slice, the current CU If r_affine_flag is 1, generate a motion compensation prediction signal for the CU currently being processed. To achieve this, affine motion compensation is used. If set, no affine model is used for the currently processed CU. If lag is not present, it is assumed to be 0.

[0297] When decoding a P or B slice, in the currently processed CU, If ffine_type_flag is 1, the motion compensation prediction signal for the currently processed CU is generated. To achieve this, a 6-parameter affine model of motion compensation is used. If e_flag is 0, the motion compensation prediction signal for the currently processed CU is generated. In this paper, motion compensation using a four-parameter affine model is used.

[0298] In the affine model, the reference index and the motion vector are calculated for each subblock. Since the tor is derived, the reference index that is processed on a subblock basis and a motion vector are used to generate a motion compensation prediction signal.

[0299] The four-parameter affine model is the horizontal and vertical components of the motion vectors of the two control points. The motion vector of a subblock is derived from the four parameters of the vertical component. This is a mode in which motion compensation is performed at the pixel position.

[0300] (Second embodiment) In the second embodiment, the sub-block merging mode derivation units 304 and 404 When deriving affine construction merge candidates, each affine control point (CP0, CP1, CP2, CP3 ) Is it possible to fix the reference image to be compared with the reference image in (a) and perform an affine transformation? For example, the reference image with the reference image index 0 is fixed, and each affine Affine transformation is possible with each combination of control points (CP0, CP1, CP2, CP3). Determine whether or not this is the case.

[0301] The upper-left affine control point is calculated by comparing the reference image of each affine control point with a fixed reference image. The upper right affine control point CP1, the lower left affine control point CP2, and the lower right affine control point CP3 are not used. A three-point affine control point affine model can be constructed using the fin control point CP3. This increases the possibility of deriving affine construction merge candidates, and the affine merge mode This improves prediction accuracy and coding efficiency.

[0302] All of the above-described embodiments may be combined in multiple forms.

[0303] In all the embodiments described above, the bit stream output by the image coding device The specific data is encoded so that it can be decoded according to the encoding method used in the embodiment. The image decoding device corresponding to this image coding device has the following data format. The bitstream can be decoded in a particular data format.

[0304] In order to exchange bitstreams between the image coding device and the image decoding device, a wired or When a wireless network is used, the data is bit-coded in a data format suitable for the transmission mode of the communication path. In that case, the bit stream output by the image coding device may be converted and transmitted. The stream is converted into coded data in a format suitable for the transmission mode of the communication path and transmitted over the network. a transmitting device that transmits encoded data to a network and receives the encoded data from the network and decodes the data into a bit stream; The image encoding device includes a receiving device that transmits the image to the image decoding device. Memory to buffer the output bitstream and packetize the bitstream A packet processing unit and a transmission unit for transmitting packetized encoded data via a network. The receiving device receives packetized encoded data via a network. a receiving unit for receiving the encoded data, a memory for buffering the received encoded data, and a packetizing unit for packetizing the encoded data. and a packet processing unit that processes the received signal to generate a bitstream and provides the bitstream to the image decoding device.

[0305] In addition, by adding a display unit for displaying the image decoded by the image decoding device to the configuration, In this case, the display unit may display the decoded image signal generated by the decoded image signal superimposing unit 207. The decoded image signal stored in the decoded image memory 208 is read out and displayed on the screen.

[0306] In addition, an imaging unit is added to the configuration, and the captured image is input to the image encoding device. In this case, the imaging unit inputs an image signal captured by the imaging unit to the block division unit 101. To exert effort.

[0307] FIG. 37 shows an example of a hardware configuration of the encoding / decoding device according to the present embodiment. The image encoding device includes the configuration of an image encoding device and an image decoding device according to an embodiment of the present invention. The encoding / decoding device 9000 includes a CPU 9001, a codec IC 9002, an I / O O interface 9003, memory 9004, optical disk drive 9005, network The network interface 9006 and the video interface 9009 are connected via a bus. Connected by 9010.

[0308] The image encoding unit 9007 and the image decoding unit 9008 are typically a codec IC 9002 and The image encoding process of the image encoding device according to the embodiment of the present invention is implemented as an image encoding The image decoding process is executed by the decoding unit 9007 in the image decoding device according to the embodiment of the present invention. The decoding process is performed by the image decoding unit 9008. The I / O interface 9003 For example, a USB interface is used to connect an external keyboard 9104 and mouse 91 The CPU 9001 receives input via the I / O interface 9003. The encoding / decoding device 90 is controlled to execute an operation desired by the user based on the user's operation. 00. The user's operations using the keyboard 9104, mouse 9105, etc. are as follows: , select whether to perform encoding or decoding functions, set the encoding quality, and There are input and output destinations for programs, and input and output destinations for images, etc.

[0309] When the user desires to play back the images recorded on the disk recording medium 9100 The optical disk drive 9005 reads the bitstream from the inserted disk recording medium 9100. The read bit stream is then sent to the codec I via bus 9010. The image decoder 9008 of the C9002 receives the bit stream and sends it to the image decoder 9008. The image decoding process is performed on the image data in the image decoding device according to the embodiment of the present invention. The image is sent to an external monitor 9103 via a video interface 9009. The encoding / decoding device 9000 includes a network interface 9006. Through the network 9101, it is possible to connect to an external distribution server 9106 and a mobile terminal 9107. The user changes the image recorded on the disk recording medium 9100 to the image distribution server 9106. If you want to play back images stored in the mobile terminal 9107, you will need to connect to the network. The interface 9006 receives a bit stream from the input disk recording medium 9100. Instead of reading out the bit stream, the bit stream is acquired from the network 9101. When the user wishes to play back an image stored in memory 9004, 004, an image decoding apparatus according to an embodiment of the present invention The image decoding process is then performed.

[0310] The user captures an image using an external camera 9102, encodes the image, and records it in memory 9004. When an operation is desired, the video interface 9009 receives an image from the camera 9102. The image data is inputted and sent to the image encoding unit 9007 of the codec IC 9002 via the bus 9010. The image encoding unit 9007 performs the following on the image input via the video interface 9009: An image encoding process is performed in the image encoding device according to the embodiment of the present invention, and a bitstream is The bit stream is then sent to the memory 9004 via the bus 9010. The user changes the memory 9004 and writes the bit stream to the disk recording medium 9100. If a user wishes to record a program, the optical disk drive 9005 reads the inserted disk. The bit stream is written to the disk recording medium 9100.

[0311] A hardware configuration that has an image encoding device but does not have an image decoding device, or a hardware configuration that has an image decoding device It is also possible to realize a hardware configuration that does not include an image coding device. The hardware configuration is, for example, a codec IC 9002, an image encoding unit 9007, or This is realized by replacing the image decoding unit 9008 with the image decoding unit 9009.

[0312] The above coding and decoding processes are carried out by hardware-based transmission, storage, and reception devices. It is of course possible to realize it by using ROM (read only memory) or flash memory. Firmware stored in memory, etc., and software executed by a computer, etc. The firmware program and software program may be executed on a computer. The information may be provided by recording it on a recording medium that can be read by a computer or by a wired or wireless network. It may be provided from a server via the Internet, or it may be provided as terrestrial or satellite digital broadcasting data. It may also be provided as a broadcast.

[0313] The present invention has been described above based on the embodiments. The embodiments are merely examples, and each of the structures The combination of components and processes can be modified in many different ways, and these modifications It will be appreciated by those skilled in the art that the embodiments are within the scope of the present invention. [Explanation of symbols]

[0314] 100 image encoding device, 101 block division unit, 102 inter prediction unit, 103 intra prediction unit, 104 decoded image memory, 105 prediction method determination unit, 10 6 residual generation unit, 107 orthogonal transformation and quantization unit, 108 bit string encoding unit, 10 9 inverse quantization and inverse orthogonal transformation unit, 110 decoded image signal superposition unit, 111 coding information storage unit A memory for storing data, 200 an image decoding device, 201 a bit string decoding unit, 202 a block Division unit, 203 inter prediction unit, 204 intra prediction unit, 205 coding information storage Memory 206 inverse quantization and inverse orthogonal transformation unit, 207 decoded image signal superposition unit, 208 Decoded image memory.< / poc>

Claims

1. A video encoding device that performs affine transformation on a coding block basis to form a bit stream, comprising: a sub-block temporal merge candidate derivation unit that derives sub-block temporal merge candidates having a motion vector for each reference list and a flag indicating whether a predictive sub-block is valid or not, for each sub-block; an affine inheritance merge candidate derivation unit that derives affine inheritance merge candidates that inherit an affine model of a block spatially adjacent to the encoding target block; an affine construction merge candidate derivation unit that derives affine construction merge candidates from a plurality of pieces of motion information of blocks spatially or temporally adjacent to the encoding target block; an affine-fixed merge candidate derivation unit that derives affine-fixed merge candidates in which motion information of affine control points is fixed; a sub-block merging candidate selection unit for selecting a sub-block merging candidate from the sub-block temporal merging candidates, the affine inheritance merging candidates, the affine construction merging candidates, and the affine fixed merging candidates; an encoding unit that encodes information indicating whether or not the mode is a sub-block unit merge mode and a merge index indicating a selected merge candidate; Equipped with The affine-fixed merge candidate derivation unit fixes a motion vector of each affine control point of the affine-fixed merge candidate to (0, 0). A video encoding device comprising:

2. A video coding method for forming a bitstream by performing affine transformation on a coding block basis, comprising the steps of: a sub-block temporal merging candidate derivation step of deriving sub-block temporal merging candidates having a motion vector for each reference list and a flag indicating whether a predictive sub-block is valid or not, for each sub-block; an affine inheritance merge candidate derivation step of deriving an affine inheritance merge candidate that inherits an affine model of a block spatially adjacent to the encoding target block; an affine construction merge candidate derivation step of deriving affine construction merge candidates from a plurality of pieces of motion information of blocks spatially or temporally adjacent to the encoding target block; an affine-fixed merge candidate derivation step of deriving an affine-fixed merge candidate in which motion information of an affine control point is fixed; a sub-block merging candidate selection step of selecting a sub-block merging candidate from the sub-block temporal merging candidates, the affine inheritance merging candidates, the affine construction merging candidates, and the affine fixed merging candidates; an encoding step of encoding information indicating whether or not the mode is a sub-block unit merge mode and a merge index indicating a selected merge candidate; having The affine-fixed merge candidate derivation step fixes a motion vector of each affine control point of the affine-fixed merge candidate to (0, 0). A video encoding method comprising:

3. A video encoding program that performs affine transformation on a coding block basis to form a bit stream, a sub-block temporal merging candidate derivation step of deriving sub-block temporal merging candidates having a motion vector for each reference list and a flag indicating whether a predictive sub-block is valid or not, for each sub-block; an affine inheritance merge candidate derivation step of deriving an affine inheritance merge candidate that inherits an affine model of a block spatially adjacent to the encoding target block; an affine construction merge candidate derivation step of deriving affine construction merge candidates from a plurality of pieces of motion information of blocks spatially or temporally adjacent to the encoding target block; an affine-fixed merge candidate derivation step of deriving an affine-fixed merge candidate in which motion information of an affine control point is fixed; a sub-block merging candidate selection step of selecting a sub-block merging candidate from the sub-block temporal merging candidates, the affine inheritance merging candidates, the affine construction merging candidates, and the affine fixed merging candidates; an encoding step of encoding information indicating whether or not the mode is a sub-block unit merge mode and a merge index indicating a selected merge candidate; having The affine-fixed merge candidate derivation step fixes a motion vector of each affine control point of the affine-fixed merge candidate to (0, 0).

4. A video decoding device that performs affine transformation on a decoding block basis, a decoding unit that decodes information indicating whether or not a sub-block unit merge mode is used and a merge index indicating a merge candidate; a sub-block temporal merge candidate derivation unit that derives sub-block temporal merge candidates having a motion vector for each reference list and a flag indicating whether a predictive sub-block is valid or not, for each sub-block; an affine inheritance merge candidate derivation unit that derives affine inheritance merge candidates that inherit an affine model of a block spatially adjacent to the decoding target block; an affine construction merge candidate derivation unit that derives affine construction merge candidates from a plurality of pieces of motion information of blocks spatially or temporally adjacent to the decoding target block; an affine-fixed merge candidate derivation unit that derives affine-fixed merge candidates in which motion information of affine control points is fixed; a sub-block merging candidate selection unit for selecting a sub-block merging candidate from the sub-block temporal merging candidates, the affine inheritance merging candidates, the affine construction merging candidates, and the affine fixed merging candidates; Equipped with The affine-fixed merge candidate derivation unit fixes a motion vector of each affine control point of the affine-fixed merge candidate to (0, 0). A video decoding device comprising:

5. A video decoding method for performing affine transformation on a decoding block basis, comprising the steps of: a decoding step of decoding information indicating whether or not the mode is a sub-block unit merge mode and a merge index indicating a merge candidate; a sub-block temporal merging candidate derivation step of deriving sub-block temporal merging candidates having a motion vector for each reference list and a flag indicating whether a predictive sub-block is valid or not, for each sub-block; an affine inheritance merge candidate derivation step of deriving an affine inheritance merge candidate that inherits an affine model of a block spatially adjacent to the decoding target block; an affine construction merge candidate derivation step of deriving affine construction merge candidates from a plurality of pieces of motion information of blocks spatially or temporally adjacent to the decoding target block; an affine-fixed merge candidate derivation step of deriving an affine-fixed merge candidate in which motion information of an affine control point is fixed; a sub-block merging candidate selection step of selecting a sub-block merging candidate from the sub-block temporal merging candidates, the affine inheritance merging candidates, the affine construction merging candidates, and the affine fixed merging candidates; having The affine-fixed merge candidate derivation step fixes a motion vector of each affine control point of the affine-fixed merge candidate to (0, 0). A video decoding method comprising:

6. A video decoding program that performs affine transformation on a decoding block basis, a decoding step of decoding information indicating whether or not the mode is a sub-block unit merge mode and a merge index indicating a merge candidate; a sub-block temporal merging candidate derivation step of deriving sub-block temporal merging candidates having a motion vector for each reference list and a flag indicating whether a predictive sub-block is valid or not, for each sub-block; an affine construction merge candidate derivation step of deriving affine construction merge candidates from a plurality of pieces of motion information of blocks spatially or temporally adjacent to the decoding target block; an affine inheritance merge candidate derivation step of deriving an affine inheritance merge candidate that inherits an affine model of a block spatially adjacent to the decoding target block; an affine-fixed merge candidate derivation step of deriving an affine-fixed merge candidate in which motion information of an affine control point is fixed; a sub-block merging candidate selection step of selecting a sub-block merging candidate from the sub-block temporal merging candidates, the affine inheritance merging candidates, the affine construction merging candidates, and the affine fixed merging candidates; having The affine-fixed merge candidate derivation step fixes a motion vector of each affine control point of the affine-fixed merge candidate to (0, 0). A video decoding program comprising:

7. 3. A method for storing a bit stream generated according to the moving picture coding method according to claim 2 on a recording medium.

8. A transmission method for transmitting a bit stream generated according to the video coding method according to claim 2.

Images