Picture decoding device, picture decoding method, and picture decoding program

Abstract

Technology for improving coding efficiency by performing a block split suitable for picture coding and decoding is provided. A picture decoding device (200) includes a spatial candidate derivation unit configured to derive a spatial candidate from inter prediction information of a block neighboring a decoding target block and register the derived spatial candidate as a candidate in a first candidate list, a history-based candidate derivation unit configured to generate a second candidate list by adding a history-based candidate included in a history-based candidate list as a candidate to the first candidate list, a candidate selection unit configured to select a selection candidate from candidates included in the second candidate list; and an inter prediction unit (203) configured to perform inter prediction using the selection candidate. The history-based candidate derivation unit switches between whether or not a history-based candidate overlapping a candidate included in the first candidate list is added in accordance with a prediction mode.

Description

Image decoding device, image decoding method, and image decoding program

[0001] The present invention relates to image encoding and decoding technologies that divide an image into blocks and perform prediction.

[0002] In image encoding and decoding, the image to be processed is divided into blocks, which are sets of a predetermined number of pixels, and processing is performed in block units. By dividing the image into appropriate blocks and appropriately setting intra prediction (intra-frame prediction) and inter prediction (inter-frame prediction), the encoding efficiency is improved.

[0003] In the encoding and decoding of moving images, the encoding efficiency is improved by inter prediction that predicts from encoded and decoded pictures. Patent Document 1 describes a technique of applying an affine transformation during inter prediction. In moving images, it is not uncommon for an object to undergo deformations such as enlargement, reduction, and rotation. By applying the technique of Patent Document 1, efficient encoding becomes possible.

[0004] Japanese Patent Application Laid-Open No. 9-172644

[0005] However, since the technique of Patent Document 1 involves image transformation, there is a problem of a large processing load. In view of the above problems, the present invention provides an encoding technique with a low load and high efficiency.

[0006] The following embodiments disclose, for example, the following aspects.

[0007] A spatial candidate derivation unit that derives a spatial candidate from inter prediction information of blocks adjacent to the decoding target block and registers it in a first candidate list as a candidate, a history candidate derivation unit that adds a history candidate included in a history candidate list to the first candidate list as a candidate to form a second candidate list, a candidate selection unit that selects a selected candidate from the candidates included in the second candidate list, and an inter prediction unit that performs inter prediction using the selected candidate. The history candidate derivation unit switches whether to add a history candidate that overlaps with the candidates included in the first candidate list according to the prediction mode. An image decoding device is disclosed.

[0008] The present invention discloses an image decoding device in which the prediction modes are merge mode and predictive motion vector mode, the candidate when the prediction mode is merge mode is motion information, and the candidate when the prediction mode is predictive motion vector mode is a motion vector.

[0009] The image decoding device disclosed above includes a history candidate derivation unit which, when the prediction mode is merge mode, adds the history candidate to the first candidate list as a candidate if there is no overlap between the history candidate and the candidates included in the first candidate list, and when the prediction mode is prediction motion vector mode, adds the history candidate to the first candidate list as a candidate regardless of whether there is any overlap between the history candidate and the candidates included in the first candidate list.

[0010] The image decoding device further comprises: a history prediction motion vector candidate list update unit that, when the prediction mode is merge mode, updates the history prediction motion vector candidate list with selected candidates so that the history prediction motion vector candidate list does not contain duplicate candidates; and when the prediction mode is prediction motion vector mode, updates the history prediction motion vector candidate list with motion information including at least selected candidates and a reference index indicating the picture referenced by the selected candidate so that the history prediction motion vector candidate list does not contain duplicate candidates.

[0011] The above-described image decoding device is disclosed, wherein the maximum number of candidates included in the candidate list when the prediction mode is merge mode is greater than the maximum number of candidates included in the candidate list when the prediction mode is predicted motion vector mode.

[0012] The image decoding device disclosed above includes a history prediction motion vector candidate derivation unit which, when the prediction mode is the prediction motion vector mode, adds the history candidate to the first candidate list as a candidate, regardless of whether the history candidate and the candidates included in the first candidate list overlap, if the reference index of the history candidate and the reference index of the picture to be decoded are the same.

[0013] The present invention discloses an image decoding method for an image decoding device, comprising the steps of: deriving spatial candidates from inter-prediction information of blocks adjacent to the block to be decoded, and registering the derived spatial candidates as candidates in a first candidate list; adding history candidates included in a history candidate list to the first candidate list as candidates to form a second candidate list, wherein the method switches whether or not to add history candidates that overlap with candidates included in the first candidate list, depending on the prediction mode; selecting candidates from the candidates included in the second candidate list; and performing inter-prediction using the selected candidates.

[0014] The present invention discloses a computer program stored in a computer-readable non-temporary storage medium for an image decoding device, comprising instructions to cause the computer of the image decoding device to execute the following steps: deriving spatial candidates from inter prediction information of blocks adjacent to the block to be decoded, registering the derived spatial candidates as candidates in a first candidate list; adding history candidates included in a history candidate list to the first candidate list as candidates to form a second candidate list, switching whether or not to add history candidates that overlap with candidates included in the first candidate list depending on the prediction mode; selecting candidates from the candidates included in the second candidate list; and performing inter prediction using the selected candidates.

[0015] The present invention discloses an image encoding device comprising: a spatial candidate derivation unit that derives spatial candidates from inter prediction information of blocks adjacent to the block to be encoded and registers them as candidates in a first candidate list; a history candidate derivation unit that adds history candidates included in a history candidate list to the first candidate list as candidates to form a second candidate list; a candidate selection unit that selects candidates from the candidates included in the second candidate list; and an inter prediction unit that performs inter prediction using the selected candidates, wherein the history candidate derivation unit switches whether or not to add history candidates that overlap with candidates included in the first candidate list depending on the prediction mode, the prediction mode is a merge mode and a predicted motion vector mode, the candidates when the prediction mode is a merge mode are motion information and the candidates when the prediction mode is a predicted motion vector mode are motion vectors.

[0016] The image encoding device disclosed above includes a history candidate derivation unit which, when the prediction mode is merge mode, adds the history candidate to the first candidate list as a candidate if there is no overlap between the history candidate and the candidates included in the first candidate list, and when the prediction mode is prediction motion vector mode, adds the history candidate to the first candidate list as a candidate regardless of whether there is any overlap between the history candidate and the candidates included in the first candidate list.

[0017] The image encoding device disclosed further includes a history prediction motion vector candidate list update unit that, when the prediction mode is merge mode, updates the history prediction motion vector candidate list with selected candidates so that the history prediction motion vector candidate list does not contain duplicate candidates, and when the prediction mode is prediction motion vector mode, updates the history prediction motion vector candidate list with motion information including at least selected candidates and a reference index indicating the picture referenced by the selected candidate, so that the history prediction motion vector candidate list does not contain duplicate candidates.

[0018] The above-described image encoding device is disclosed, wherein the maximum number of candidates included in the candidate list when the prediction mode is merge mode is greater than the maximum number of candidates included in the candidate list when the prediction mode is predicted motion vector mode.

[0019] The image encoding device disclosed above includes a history prediction motion vector candidate derivation unit which, when the prediction mode is the prediction motion vector mode, adds the history candidate to the first candidate list as a candidate, regardless of whether the history candidate and the candidates included in the first candidate list overlap, if the reference index of the history candidate and the reference index of the picture to be decoded are the same.

[0020] The present invention discloses an image encoding method comprising the steps of: deriving spatial candidates from inter-prediction information of blocks adjacent to the block to be encoded and registering them as candidates in a first candidate list; adding history candidates included in a history candidate list to the first candidate list as candidates to form a second candidate list, wherein the method switches whether or not to add history candidates that overlap with candidates included in the first candidate list, depending on the prediction mode; selecting candidates from the candidates included in the second candidate list; and performing inter-prediction using the selected candidates, wherein the prediction mode is a merge mode and a predicted motion vector mode, the candidates when the prediction mode is a merge mode are motion information, and the candidates when the prediction mode is a predicted motion vector mode are motion vectors.

[0021] A computer program stored in a computer-readable non-temporary storage medium in an image encoding device, comprising the steps of: the computer program deriving spatial candidates from interprediction information of blocks adjacent to the block to be encoded and registering them as candidates in a first candidate list; and adding history candidates included in the history candidate list to the first candidate list as candidates to form a second candidate list, wherein the program switches whether or not to add history candidates that overlap with candidates included in the first candidate list depending on the prediction mode, and if the prediction mode is merge mode, if the history candidates and candidates included in the first candidate list do not overlap, the history candidates are added to the first candidate list as candidates, and the prediction mode The present invention discloses a computer program stored in a computer-readable non-temporary storage medium, which includes instructions to cause the computer of an image encoding device to execute the following steps: when the prediction mode is merge mode, the candidates are motion information; when the prediction mode is predictive motion vector mode, the candidates are motion vectors; when the prediction mode is predictive motion vector mode, the candidates are motion vectors; and when the prediction mode is predictive motion vector mode, the instructions to cause the computer to execute the following steps: adding a history candidate to the first candidate list as a candidate regardless of whether the history candidate and the candidates included in the first candidate list overlap; selecting a candidate from the candidates included in the second candidate list; and performing an interpretation using the selected candidate.

[0022] The above description is illustrative. The scope of this application and the present invention is not limited or restricted by this description. Furthermore, it should be understood that the term "the present invention" in this specification is not intended to limit the scope of the present invention or this application, but is used for illustrative purposes only.

[0023] According to the present invention, highly efficient image encoding and decoding can be achieved with low overhead.

[0024] This is a block diagram of an image encoding device according to an embodiment of the present invention. This is a block diagram of an image decoding device according to an embodiment of the present invention. This is a flowchart for explaining the operation of dividing a tree block. This is a diagram showing how an input image is divided into tree blocks. This is a diagram explaining z-scanning. This is a diagram showing the division shape of a block. This is a diagram showing the division shape of a block. This is a diagram showing the division shape of a block. This is a diagram showing the division shape of a block. This is a flowchart for explaining the operation of dividing a block into four parts. This is a flowchart for explaining the operation of dividing a block into two or three parts. This is syntax for representing the shape of block division. This is a diagram for explaining intra-prediction. This is a diagram for explaining intra-prediction. This is a diagram for explaining the reference block of inter-prediction. This is syntax for representing the coding block prediction mode. This is a diagram showing the correspondence between syntax elements and modes related to inter-prediction. This is a diagram for explaining affine transform motion compensation of two control points. This is a diagram for explaining affine transform motion compensation of three control points. This is a block diagram of the detailed configuration of the inter-prediction unit 102 in Figure 1. This is a block diagram of the detailed configuration of the normal prediction motion vector mode derivation unit 301 in Figure 16. This is a block diagram of the detailed configuration of the normal merge mode derivation unit 302 in Figure 16. This is a flowchart for explaining the normal predicted motion vector mode derivation process of the normal predicted motion vector mode derivation unit 301 in Figure 16. This is a flowchart showing the processing steps of the normal predicted motion vector mode derivation process. This is a flowchart explaining the processing steps of the normal merge mode derivation process. This is a block diagram of the detailed configuration of the inter-prediction unit 203 in Figure 2. This is a block diagram of the detailed configuration of the normal predicted motion vector mode derivation unit 401 in Figure 22. This is a block diagram of the detailed configuration of the normal merge mode derivation unit 402 in Figure 22. This is a flowchart for explaining the normal predicted motion vector mode derivation process of the normal predicted motion vector mode derivation unit 401 in Figure 22. This is a diagram explaining the history predicted motion vector candidate list initialization / update processing procedure. This is a flowchart of the identical element confirmation processing procedure in the history predicted motion vector candidate list initialization / update processing procedure.This is a flowchart of the element shifting process in the initialization and update process of the history prediction motion vector candidate list. This is a flowchart explaining the history prediction motion vector candidate derivation process. This is a flowchart explaining the history merge candidate derivation process. This is a diagram to explain an example of the history prediction motion vector candidate list update process. This is a diagram to explain an example of the history prediction motion vector candidate list update process. This is a diagram to explain an example of the history prediction motion vector candidate list update process. This is a diagram to explain motion compensation prediction when it is an L0 prediction and the L0 reference picture (RefL0Pic) is at a time earlier than the processing target picture (CurPic). This is a diagram to explain motion compensation prediction when it is an L0 prediction and the L0 prediction reference picture is at a time later than the processing target picture. This is a diagram to explain the prediction direction of motion compensation prediction when it is a dual prediction and the L0 prediction reference picture is at a time earlier than the processing target picture and the L1 prediction reference picture is at a time later than the processing target picture. This is a diagram illustrating the prediction direction of motion compensation prediction when there is a dual prediction and the reference picture for L0 prediction and the reference picture for L1 prediction are at a time earlier than the picture to be processed. This is a diagram illustrating the prediction direction of motion compensation prediction when there is a dual prediction and the reference picture for L0 prediction and the reference picture for L1 prediction are at a time later than the picture to be processed. This is a diagram illustrating an example of the hardware configuration of the encoding / decoding device according to an embodiment of the present invention. This is a table showing an example of a history prediction motion vector candidate added by the initialization of the history prediction motion vector candidate list. This is a table showing another example of a history prediction motion vector candidate added by the initialization of the history prediction motion vector candidate list. This is a table showing another example of a history prediction motion vector candidate added by the initialization of the history prediction motion vector candidate list. This is a flowchart illustrating the history prediction motion vector candidate derivation process procedure of Modification 2 of the First Embodiment.

[0025] This section defines the technologies and technical terms used in this embodiment.

[0026] <Tree Blocks> In this embodiment, the image to be encoded and decoded is divided equally into units of a predetermined size. This unit is defined as a tree block. In Figure 4, the size of the tree block is set to 128 x 128 pixels, but the size of the tree block is not limited to this and can be set to any size. The tree blocks of the processing target (corresponding to the encoding target in encoding processing and the decoding target in decoding processing) are switched in the order of the raster scan, that is, from left to right and from top to bottom. The interior of each tree block can be further recursively divided. The blocks that are to be encoded and decoded after recursively dividing the tree block are defined as encoded blocks. Tree blocks and encoded blocks are collectively defined as blocks. Efficient encoding is possible by performing appropriate block division. The size of the tree block can be a fixed value predetermined by the encoding device and the decoding device, or the encoding device can transmit the determined tree block size to the decoding device. Here, the maximum size of the tree block is set to 128 x 128 pixels, and the minimum size of the tree block is set to 16 x 16 pixels. Furthermore, the maximum size of the encoded block is set to 64x64 pixels, and the minimum size is set to 4x4 pixels.

[0027] <Prediction Mode> For each encoding block to be processed, the system switches between intra-prediction (MODE_INTRA), which makes predictions from the processed image signal of the image to be processed, and inter-prediction (MODE_INTER), which makes predictions from the image signal of the processed image. In the encoding process, the processed image is used for the decoded image, image signal, tree block, block, encoding block, etc., of the encoded signal, and in the decoding process, it is used for the decoded image, image signal, tree block, block, encoding block, etc. The mode that distinguishes between intra-prediction (MODE_INTRA) and inter-prediction (MODE_INTER) is defined as the prediction mode (PredMode). The prediction mode (PredMode) has either intra-prediction (MODE_INTRA) or inter-prediction (MODE_INTER) as its value.

[0028] <Interpretation> Interpretation, which performs predictions from the image signals of processed images, can use multiple processed images as reference pictures. To manage multiple reference pictures, two types of reference lists, L0 (reference list 0) and L1 (reference list 1), are defined, and each uses a reference index to identify the reference picture. L0 prediction (Pred_L0) is available for P slices. L0 prediction (Pred_L0), L1 prediction (Pred_L1), and dual prediction (Pred_BI) are available for B slices. L0 prediction (Pred_L0) is an interpretation that refers to the reference picture managed in L0, and L1 prediction (Pred_L1) is an interpretation that refers to the reference picture managed in L1. Dual prediction (Pred_BI) is an interpretation that performs both L0 prediction and L1 prediction, referring to one reference picture each managed in L0 and L1. The information that identifies L0 prediction, L1 prediction, and dual prediction is defined as the interpretation mode. In subsequent processing, it is assumed that constants and variables with the subscript LX in their output will be processed separately for L0 and L1.

[0029] <Predicted Motion Vector Mode> The predicted motion vector mode transmits an index for identifying the predicted motion vector, a differential motion vector, an inter-prediction mode, and a reference index to determine the inter-prediction information of the block to be processed. The predicted motion vector is derived from candidate predicted motion vectors derived from processed blocks adjacent to the block to be processed, or from blocks belonging to the processed image that are located at the same position as or near the block to be processed, and from an index for identifying the predicted motion vector.

[0030] <Merge Mode> Merge mode is a mode in which, without transmitting differential motion vectors or reference indices, interpretation information for the target block is derived from interpretation information of processed blocks adjacent to the target block, or blocks belonging to the processed image that are located at the same position or near (near) the target block.

[0031] Processed blocks adjacent to the block to be processed, and the inter-prediction information of those processed blocks, are defined as spatial merge candidates. Blocks belonging to the processed image that are located at the same position as or near the block to be processed, and the inter-prediction information derived from the inter-prediction information of those blocks, are defined as temporal merge candidates. Each merge candidate is registered in the merge candidate list, and the merge index identifies the merge candidate to be used for prediction of the block to be processed.

[0032] <Adjacent Blocks> Figure 11 illustrates the reference blocks used to derive interprediction information in the predictive motion vector mode and merge mode. A0, A1, A2, B0, B1, B2, and B3 are processed blocks adjacent to the block to be processed. T0 is a block belonging to the processed image that is located at the same position as or near (neighboring) the block to be processed in the image to be processed.

[0033] Blocks A1 and A2 are located to the left of the coding block to be processed and are adjacent to it. Blocks B1 and B3 are located above the coding block to be processed and are adjacent to it. Blocks A0, B0, and B2 are located to the lower left, upper right, and upper left of the coding block to be processed, respectively.

[0034] Details on how adjacent blocks are handled in predictive motion vector mode and merge mode will be described later.

[0035] <Affine Transform Motion Compensation> Affine transform motion compensation divides an encoded block into predetermined subblocks and performs motion compensation by individually determining a motion vector for each divided subblock. The motion vector of each subblock is derived based on one or more control points derived from interprediction information of processed blocks adjacent to the block to be processed, or blocks belonging to the processed image that are located at the same position as or near the block to be processed. In this embodiment, the size of the subblock is set to 4x4 pixels, but the size of the subblock is not limited to this, and the motion vector may be derived on a pixel-by-pixel basis.

[0036] Figure 14 shows an example of affine transform motion compensation with two control points. In this case, the two control points have two parameters: a horizontal component and a vertical component. For this reason, the affine transform with two control points is called a four-parameter affine transform. CP1 and CP2 in Figure 14 are the control points. Figure 15 shows an example of affine transform motion compensation with three control points. In this case, the three control points have two parameters: a horizontal component and a vertical component. For this reason, the affine transform with three control points is called a six-parameter affine transform. CP1, CP2, and CP3 in Figure 15 are the control points.

[0037] Affine transformation motion compensation is available in both predictive motion vector mode and merge mode. The mode in which affine transformation motion compensation is applied in predictive motion vector mode is defined as subblock predictive motion vector mode, and the mode in which affine transformation motion compensation is applied in merge mode is defined as subblock merge mode.

[0038] <Inter-prediction syntax> The syntax for inter-prediction is explained using Figures 12 and 13. In Figure 12, merge_flag is a flag that indicates whether the coded block to be processed is in merge mode or predictive motion vector mode. merge_affine_flag is a flag that indicates whether or not to apply subblock merge mode to the coded block to be processed in merge mode. inter_affine_flag is a flag that indicates whether or not to apply subblock predictive motion vector mode to the coded block to be processed in predictive motion vector mode. cu_affine_type_flag is a flag for determining the number of control points in subblock predictive motion vector mode. Figure 13 shows the values ​​of each syntax element and the corresponding prediction methods. merge_flag=1, merge_affine_flag=0 corresponds to normal merge mode. Normal merge mode is a merge mode that is not subblock merge. merge_flag=1, merge_affine_flag=1 corresponds to subblock merge mode. merge_flag=0, inter_affine_flag=0 corresponds to normal predictive motion vector mode. The normal predictive motion vector mode is a predictive motion vector merge that is not a subblock predictive motion vector mode. merge_flag=0, inter_affine_flag=1 corresponds to the subblock predictive motion vector mode. In the case of merge_flag=0, inter_affine_flag=1, cu_affine_type_flag is also transmitted to determine the number of control points.

[0039] <POC> POC (Picture Order Count) is a variable associated with the picture being encoded, and is set to a value that increases by 1 according to the output order of the pictures. The value of POC can be used to determine if two pictures are the same, to determine the order in which pictures are output, and to derive the distance between pictures. For example, if two pictures have the same POC value, they can be determined to be the same picture. If two pictures have different POC values, the picture with the smaller POC value is determined to be the picture that is output first, and the difference in the POC values ​​of the two pictures indicates the distance between the pictures in the time axis direction.

[0040] (First Embodiment) An image encoding device 100 and an image decoding device 200 according to the first embodiment of the present invention will be described.

[0041] Figure 1 is a block diagram of an image encoding device 100 according to the first embodiment. The image encoding device 100 of this embodiment includes a block division unit 101, an inter-prediction unit 102, an intra-prediction unit 103, a decoded image memory 104, a prediction method determination unit 105, a residual generation unit 106, an orthogonal transformation / quantization unit 107, a bit string encoding unit 108, an inverse quantization / inverse orthogonal transformation unit 109, a decoded image signal superposition unit 110, and an encoded information storage memory 111.

[0042] The block division unit 101 recursively divides the input image to generate encoded blocks. The block division unit 101 includes a 4-division unit that divides the block to be divided horizontally and vertically, and a 2-3 division unit that divides the block to be divided horizontally or vertically. The block division unit 101 uses the generated encoded blocks as the encoded blocks to be processed and supplies the image signals of the encoded blocks to be processed to the inter-prediction unit 102, the intra-prediction unit 103, and the residual generation unit 106. The block division unit 101 also supplies information indicating the determined recursive division structure to the bit string encoding unit 108. The detailed operation of the block division unit 101 will be described later.

[0043] The inter prediction unit 102 performs inter prediction on the coding block to be processed. The inter prediction unit 102 derives a plurality of candidate inter prediction information from the inter prediction information stored in the coding information storage memory 111 and the decoded image signal stored in the decoded image memory 104, selects a suitable inter prediction mode from the plurality of derived candidates, and supplies the selected inter prediction mode and the prediction image signal corresponding to the selected inter prediction mode to the prediction method determination unit 105. The detailed configuration and operation of the inter prediction unit 102 will be described later.

[0044] The intra prediction unit 103 performs intra prediction on the coding block to be processed. The intra prediction unit 103 refers to the decoded image signal stored in the decoded image memory 104 as a reference pixel, and generates a prediction image signal by intra prediction based on coding information such as the intra prediction mode stored in the coding information storage memory 111. In intra prediction, the intra prediction unit 103 selects a suitable intra prediction mode from a plurality of intra prediction modes, and supplies the selected intra prediction mode and the prediction image signal corresponding to the selected intra prediction mode to the prediction method determination unit 105. Examples of intra prediction are shown in FIGS. 10A and 10B. FIG. 10A shows the correspondence between the prediction direction of intra prediction and the intra prediction mode number. For example, in intra prediction mode 50, an intra prediction image is generated by copying the reference pixel in the vertical direction. Intra prediction mode 1 is the DC mode, in which the pixel values of all the blocks to be processed are set to the average value of the reference pixels. Intra prediction mode 0 is the Planar mode, in which a two-dimensional intra prediction image is created from the reference pixels in the vertical and horizontal directions. FIG. 10B is an example of generating an intra prediction image in the case of intra prediction mode 40. The intra prediction unit 103 copies the value of the reference pixel in the direction indicated by the intra prediction mode for each pixel of the block to be processed. When the reference pixel of the intra prediction mode is not at an integer position, the intra prediction unit 103 determines the reference pixel value by interpolation from the reference pixel values at the surrounding integer positions.

[0045] The decoded image memory 104 stores the decoded image generated by the decoded image signal superposition unit 110. The decoded image memory 104 supplies the stored decoded image to the inter prediction unit 102 and the intra prediction unit 103.

[0046] The prediction method determination unit 105 evaluates each of the intra prediction and the inter prediction by using the encoded information, the coded amount of the residual, the distortion amount between the predicted image signal and the processing target image signal, etc., and determines the optimal prediction mode. In the case of intra prediction, the prediction method determination unit 105 supplies intra prediction information such as the intra prediction mode as encoded information to the bit sequence encoding unit 108. In the case of the merge mode of inter prediction, the prediction method determination unit 105 supplies inter prediction information such as the merge index and the information indicating whether it is a sub-block merge mode (sub-block merge flag) as encoded information to the bit sequence encoding unit 108. In the case of the predicted motion vector mode of inter prediction, the prediction method determination unit 105 supplies inter prediction information such as the inter prediction mode, the predicted motion vector index, the reference indices of L0 and L1, the differential motion vector, and the information indicating whether it is a sub-block predicted motion vector mode (sub-block predicted motion vector flag) as encoded information to the bit sequence encoding unit 108. Further, the prediction method determination unit 105 supplies the determined encoded information to the encoded information storage memory 111. The prediction method determination unit 105 supplies the residual generation unit 106 and the predicted image signal to the decoded image signal superposition unit 110.

[0047] The residual generation unit 106 generates a residual by subtracting the predicted image signal from the image signal to be processed, and supplies it to the orthogonal transform / quantization unit 107.

[0048] The orthogonal transform / quantization unit 107 performs orthogonal transform and quantization on the residual according to the quantization parameter to generate an orthogonally transformed / quantized residual, and supplies the generated residual to the bit sequence encoding unit 108 and the inverse quantization / inverse orthogonal transform unit 109.

[0049] The bit string encoding unit 108 encodes information for each sequence, picture, slice, and encoding block, as well as encoding information corresponding to the prediction method determined by the prediction method determination unit 105 for each encoding block. Specifically, the bit string encoding unit 108 encodes the prediction mode PredMode for each encoding block. When the prediction mode is inter prediction (MODE_INTER), the bit string encoding unit 108 encodes encoded information (inter prediction information) such as a flag to determine whether or not it is merge mode, a subblock merge flag, a merge index if it is merge mode, the inter prediction mode if it is not merge mode, the prediction motion vector index, information about the difference motion vector, and a subblock prediction motion vector flag, according to a specified syntax (syntax rules for bit strings), and generates a first bit string. When the prediction mode is intra prediction (MODE_INTRA), the bit string encoding information such as the intra prediction mode (intra prediction information) according to a specified syntax (syntax rules for bit strings), and generates a first bit string. Furthermore, the bit string encoding unit 108 entropy encodes the orthogonal transform and quantized residuals according to a specified syntax to generate a second bit string. The bit string encoding unit 108 multiplexes the first bit string and the second bit string according to a specified syntax and outputs a bitstream.

[0050] The inverse quantization / inverse orthogonal transformation unit 109 performs inverse quantization and inverse orthogonal transformation on the orthogonal transformed residual supplied from the orthogonal transformation / quantization unit 107 to calculate the residual, and supplies the calculated residual to the decoded image signal superimposition unit 110.

[0051] The decoded image signal superposition unit 110 superimposes the predicted image signal determined by the prediction method determination unit 105 with the residual obtained by inverse quantization and inverse orthogonal transformation by the inverse quantization / inverse orthogonal transformation unit 109 to generate a decoded image, which is then stored in the decoded image memory 104. Alternatively, the decoded image signal superposition unit 110 may apply a filtering process to the decoded image to reduce distortions such as block distortion caused by encoding before storing it in the decoded image memory 104.

[0052] The encoded information storage memory 111 stores encoded information such as the prediction mode (inter-prediction or intra-prediction) determined by the prediction method determination unit 105. In the case of inter-prediction, the encoded information stored in the encoded information storage memory 111 includes inter-prediction information such as the determined motion vector, the reference index of the reference lists L0 and L1, and the history prediction motion vector candidate list. In the case of inter-prediction merge mode, the encoded information stored in the encoded information storage memory 111 includes the above-mentioned information, as well as inter-prediction information such as the merge index and information indicating whether or not it is sub-block merge mode (sub-block merge flag). In the case of inter-prediction prediction motion vector mode, the encoded information stored in the encoded information storage memory 111 includes the above-mentioned information, as well as inter-prediction information such as the inter-prediction mode, prediction motion vector index, difference motion vector, and information indicating whether or not it is sub-block prediction motion vector mode (sub-block prediction motion vector flag). In the case of intra-prediction, the encoded information stored in the encoded information storage memory 111 includes intra-prediction information such as the determined intra-prediction mode.

[0053] Figure 2 is a block diagram showing the configuration of an image decoding device according to an embodiment of the present invention, corresponding to the image encoding device of Figure 1. The image decoding device of this embodiment includes a bit sequence decoding unit 201, a block division unit 202, an inter-prediction unit 203, an intra-prediction unit 204, an encoded information storage memory 205, an inverse quantization / inverse orthogonal transformation unit 206, a decoded image signal superimposition unit 207, and a decoded image memory 208.

[0054] Since the decoding process of the image decoding device in Figure 2 corresponds to the decoding process provided inside the image encoding device in Figure 1, the configurations of the encoding information storage memory 205, inverse quantization / inverse orthogonal transformation unit 206, decoded image signal superposition unit 207, and decoded image memory 208 in Figure 2 have functions that correspond to the configurations of the encoding information storage memory 111, inverse quantization / inverse orthogonal transformation unit 109, decoded image signal superposition unit 110, and decoded image memory 104 of the image encoding device in Figure 1.

[0055] The bitstream supplied to the bit string decoding unit 201 is separated according to the rules of a defined syntax. The bit string decoding unit 201 decodes the separated first bit string to obtain sequence, picture, slice, coded block unit information, and coded block unit coded information. Specifically, the bit string decoding unit 201 decodes the prediction mode PredMode, which determines whether the coded block unit is inter-prediction (MODE_INTER) or intra-prediction (MODE_INTRA). If the prediction mode is inter-prediction (MODE_INTER), the bit string decoding unit 201 decodes coded information (inter-prediction information) according to a defined syntax, including a flag that determines whether it is merge mode or not, a merge index and sub-block merge flag if it is merge mode, and if it is predicted motion vector mode, the inter-prediction mode, predicted motion vector index, differential motion vector, sub-block predicted motion vector flag, etc., and supplies the coded information (inter-prediction information) to the coded information storage memory 205 via the inter-prediction unit 203 and the block division unit 202. When the prediction mode is intra-prediction (MODE_INTRA), the encoded information (intra-prediction information), such as the intra-prediction mode, is decoded according to a specified syntax, and the encoded information (intra-prediction information) is supplied to the encoded information storage memory 205 via the inter-prediction unit 203 or intra-prediction unit 204 and the block division unit 202. The bit sequence decoding unit 201 decodes the separated second bit sequence, calculates the orthogonal-transformed and quantized residual, and supplies the orthogonal-transformed and quantized residual to the inverse quantization and inverse orthogonal-transformation unit 206.

[0056] The inter-prediction unit 203, when the prediction mode PredMode of the encoding block to be processed is inter-prediction (MODE_INTER) and in prediction motion vector mode, uses the encoding information of the already decoded image signal stored in the encoding information storage memory 205 to derive a plurality of candidate prediction motion vectors, and registers the derived plurality of candidate prediction motion vectors in the prediction motion vector candidate list described later. The inter-prediction unit 203 selects a prediction motion vector from among the plurality of candidate prediction motion vectors registered in the prediction motion vector candidate list that corresponds to the prediction motion vector index decoded and supplied by the bit string decoding unit 201, calculates a motion vector from the difference motion vector decoded by the bit string decoding unit 201 and the selected prediction motion vector, and stores the calculated motion vector together with other encoding information in the encoding information storage memory 205. The encoded information of the encoded block supplied and stored here includes the prediction mode PredMode, flags predFlagL0[xP][yP] and predFlagL1[xP][yP] indicating whether or not to use L0 prediction and L1 prediction, reference indices refIdxL0[xP][yP] and refIdxL1[xP][yP] for L0 and L1, and motion vectors mvL0[xP][yP] and mvL1[xP][yP] for L0 and L1, etc. Here, xP and yP are indices indicating the position of the top-left pixel of the encoded block within the picture. When the prediction mode PredMode is inter-prediction (MODE_INTER) and the inter-prediction mode is L0 prediction (Pred_L0), the flag predFlagL0 indicating whether or not to use L0 prediction is 1, and the flag predFlagL1 indicating whether or not to use L1 prediction is 0. When the inter-prediction mode is L1 prediction (Pred_L1), the flag predFlagL0, which indicates whether to use L0 prediction, is 0, and the flag predFlagL1, which indicates whether to use L1 prediction, is 1. When the inter-prediction mode is bi-prediction (Pred_BI), both the flag predFlagL0, which indicates whether to use L0 prediction, and the flag predFlagL1, which indicates whether to use L1 prediction, are 1. Furthermore, when the prediction mode PredMode of the coded block to be processed is inter-prediction (MODE_INTER) and in merge mode, merge candidates are derived.Using the encoded information of already decoded encoded blocks stored in the encoded information storage memory 205, multiple merge candidates are derived and registered in the merge candidate list described later. From the multiple merge candidates registered in the merge candidate list, a merge candidate corresponding to the merge index decoded and supplied by the bit sequence decoding unit 201 is selected, and inter prediction information such as flags predFlagL0[xP][yP], predFlagL1[xP][yP] indicating whether or not to use the L0 prediction and L1 prediction of the selected merge candidate, the reference indices of L0 and L1 refIdxL0[xP][yP], refIdxL1[xP][yP], and the motion vectors of L0 and L1 mvL0[xP][yP], mvL1[xP][yP] is stored in the encoded information storage memory 205. Here, xP and yP are indices indicating the position of the top-left pixel of the encoded block in the picture. The detailed configuration and operation of the inter prediction unit 203 will be described later.

[0057] The intra prediction unit 204 performs intra prediction when the prediction mode PredMode of the encoding block to be processed is intra prediction (MODE_INTRA). The encoding information decoded by the bit sequence decoding unit 201 includes the intra prediction mode. The intra prediction unit 204 generates a predicted image signal from the decoded image signal stored in the decoded image memory 208 using intra prediction, according to the intra prediction mode included in the encoding information decoded by the bit sequence decoding unit 201, and supplies the generated predicted image signal to the decoded image signal superposition unit 207. Since the intra prediction unit 204 corresponds to the intra prediction unit 103 of the image encoding device 100, it performs the same processing as the intra prediction unit 103.

[0058] The inverse quantization / inverse orthogonal transformation unit 206 performs an inverse orthogonal transformation and inverse quantization on the orthogonal transformation / quantized residual decoded by the bit sequence decoding unit 201 to obtain the inverse orthogonal transformation / inverse quantized residual.

[0059] The decoded image signal superposition unit 207 decodes the decoded image signal by superimposing the predicted image signal inter-predicted by the inter-prediction unit 203 or the predicted image signal intra-predicted by the intra-prediction unit 204 with the residual obtained by the inverse quantization / inverse orthogonal transformation unit 206, and stores the decoded image signal in the decoded image memory 208. When storing in the decoded image memory 208, the decoded image signal superposition unit 207 may perform a filtering process on the decoded image to reduce block distortion caused by encoding, etc., before storing it in the decoded image memory 208.

[0060] Next, the operation of the block division unit 101 in the image encoding device 100 will be described. Figure 3 is a flowchart showing the operation of dividing an image into tree blocks and further dividing each tree block. First, the input image is divided into tree blocks of a predetermined size (step S1001). Each tree block is scanned in a predetermined order, i.e., in the raster scan order (step S1002), and the inside of the tree block to be processed is divided (step S1003).

[0061] Figure 7 is a flowchart showing the detailed operation of the division process in step S1003. First, it is determined whether or not to divide the block to be processed into four parts (step S1101).

[0062] If it is determined that the block to be processed should be divided into four parts, the block to be processed is divided into four parts (step S1102). Each of the divided blocks is scanned in Z-scan order, i.e., top left, top right, bottom left, and bottom right (step S1103). Figure 5 is an example of the Z-scan order, and 601 in Figure 6A is an example of the block to be processed being divided into four parts. The numbers 0 to 3 in 601 in Figure 6A indicate the order of processing. Then, for each of the blocks divided in step S1101, the division process shown in Figure 7 is recursively executed (step S1104).

[0063] If it is determined that the block to be processed should not be divided into four parts, then it is divided into two or three parts (step S1105).

[0064] Figure 8 is a flowchart showing the detailed operation of the 2-3 division process in step S1105. First, it is determined whether or not to divide the block to be processed into 2-3 parts, that is, whether or not to perform a 2-part division or a 3-part division (step S1201).

[0065] If it is not determined that the block to be processed should be divided into 2-3 parts, that is, if it is determined not to divide it, the division is terminated (step S1211). In other words, no further recursive division is performed on a block that has been divided by a recursive division process.

[0066] If it is determined that the block to be processed should be divided into two or three parts, then it is determined whether or not to further divide the block to be processed into two parts (step S1202).

[0067] If it is determined that the block to be processed should be divided into two, it is determined whether or not to divide the block to be processed vertically (up and down) (step S1203), and based on the result, the block to be processed is divided into two vertically (up and down) (step S1204) or horizontally (left and right) (step S1205). As a result of step S1204, the block to be processed is divided into two vertically (up and down) as shown in 602 of Figure 6B, and as a result of step S1205, the block to be processed is divided into two horizontally (left and right) as shown in 604 of Figure 6D.

[0068] If, in step S1202, it is not determined to divide the block to be processed into two parts, that is, if it is determined to divide it into three parts, it is determined whether or not to divide the block to be processed into upper, middle, and lower (vertical) sections (step S1206). Based on the result, the block to be processed is divided into three parts vertically (upper, middle, and lower) (step S1207), or into three parts horizontally (left, middle, and right) (step S1208). As a result of step S1207, the block to be processed is divided into three parts vertically (upper, middle, and lower) as shown in 603 of Figure 6C, and as a result of step S1208, the block to be processed is divided into three parts horizontally (left, middle, and right) as shown in 605 of Figure 6E.

[0069] After executing one of steps S1204, S1205, S1207, or S1208, each of the divided blocks of the processing target is scanned from left to right and from top to bottom (step S1209). The numbers 0 to 2 in Figures 6B to E, 602 to 605, indicate the order of processing. For each divided block, the 2-3 division process shown in Figure 8 is recursively executed (step S1210).

[0070] The recursive block partitioning described here may restrict the necessity of partitioning based on the number of partitions or the size of the block to be processed. The information restricting the necessity of partitioning may be implemented in a configuration that does not involve the transmission of information, by pre-arranging between the encoding device and the decoding device, or it may be implemented in a configuration where the encoding device determines the information restricting the necessity of partitioning, records it in a bit string, and transmits it to the decoding device.

[0071] When a block is divided, the original block is called the parent block, and each of the resulting blocks is called a child block.

[0072] Next, the operation of the block division unit 202 in the image decoding device 200 will be described. The block division unit 202 divides tree blocks using the same processing procedure as the block division unit 101 of the image encoding device 100. However, while the block division unit 101 of the image encoding device 100 determines the optimal block division shape by applying optimization methods such as estimation of the optimal shape by image recognition and distortion rate optimization, the block division unit 202 of the image decoding device 200 differs in that it determines the block division shape by decoding the block division information recorded in a bit string.

[0073] Figure 9 shows the syntax (bit sequence syntax rules) for block partitioning in the first embodiment. coding_quadtree() represents the syntax for partitioning a block into four parts. multi_type_tree() represents the syntax for partitioning a block into two or three parts. qt_split is a flag indicating whether or not to partition the block into four parts. Set qt_split=1 if the block is to be partitioned into four parts, and qt_split=0 if it is not to be partitioned into four parts. If partitioned into four parts (qt_split=1), the partitioning process is recursively performed on each of the four partitioned blocks (coding_quadtree(0), coding_quadtree(1), coding_quadtree(2), coding_quadtree(3), the arguments 0 to 3 correspond to the numbers 601 in Figure 6A). If partitioned into four parts (qt_split=0), the subsequent partitioning is determined according to multi_type_tree(). mtt_split is a flag indicating whether or not to perform further partitioning. If further division is performed (mtt_split=1), the flag mtt_split_vertical, which indicates whether to divide vertically or horizontally, and the flag mtt_split_binary, which determines whether to divide into two or three parts, are transmitted. mtt_split_vertical=1 indicates vertical division, and mtt_split_vertical=0 indicates horizontal division. mtt_split_binary=1 indicates two divisions, and mtt_split_binary=0 indicates three divisions. If two divisions are performed (mtt_split_binary=1), the division process is recursively performed on each of the two divided blocks (multi_type_tree(0), multi_type_tree(1), the argument 0 to 1 corresponds to the numbers 602 or 604 in Figures 6B to D). When dividing into three parts (mtt_split_binary=0), the division process is performed recursively for each of the three divided blocks (multi_type_tree(0), multi_type_tree(1), multi_type_tree(2), where 0 to 2 correspond to numbers 603 in Figure 6B or 605 in Figure 6E).Hierarchical block partitioning is performed by recursively calling multi_type_tree until mtt_split=0.

[0074] <Interface Prediction> The interface prediction method according to the embodiment is implemented in the interface prediction unit 102 of the image encoding device in Figure 1 and the interface prediction unit 203 of the image decoding device in Figure 2.

[0075] An interpretation method according to an embodiment will be explained with reference to the drawings. The interpretation method is performed by either encoding or decoding on a coding block basis.

[0076] <Explanation of the Interpretation Unit 102 on the Encoding Side> Figure 16 shows a detailed configuration of the interpretation unit 102 of the image encoding device shown in Figure 1. The normal prediction motion vector mode derivation unit 301 derives a plurality of normal prediction motion vector candidates, selects a prediction motion vector, and calculates the difference motion vector between the selected prediction motion vector and the detected motion vector. The detected interpretation mode, reference index, motion vector, and calculated difference motion vector become the interpretation information for the normal prediction motion vector mode. This interpretation information is supplied to the interpretation mode determination unit 305. The detailed configuration and processing of the normal prediction motion vector mode derivation unit 301 will be described later.

[0077] The normal merge mode derivation unit 302 derives multiple normal merge candidates, selects a candidate, and obtains inter-prediction information for the normal merge mode. This inter-prediction information is supplied to the inter-prediction mode determination unit 305. The detailed configuration and processing of the normal merge mode derivation unit 302 will be described later.

[0078] The subblock predicted motion vector mode derivation unit 303 derives multiple subblock predicted motion vector candidates, selects a subblock predicted motion vector, and calculates the difference motion vector between the selected subblock predicted motion vector and the detected motion vector. The detected inter-prediction mode, reference index, motion vector, and calculated difference motion vector become the inter-prediction information for the subblock predicted motion vector mode. This inter-prediction information is supplied to the inter-prediction mode determination unit 305.

[0079] The subblock merge mode derivation unit 304 derives multiple subblock merge candidates, selects a candidate, and obtains inter-prediction information for the subblock merge mode. This inter-prediction information is supplied to the inter-prediction mode determination unit 305.

[0080] The inter-prediction mode determination unit 305 determines the inter-prediction information based on the inter-prediction information supplied from the normal prediction motion vector mode derivation unit 301, the normal merge mode derivation unit 302, the sub-block prediction motion vector mode derivation unit 303, and the sub-block merge mode derivation unit 304. The inter-prediction mode determination unit 305 supplies the inter-prediction information corresponding to the determination result to the motion compensation prediction unit 306.

[0081] The motion compensation prediction unit 306 performs inter prediction on the reference image signal stored in the decoded image memory 104 based on the determined inter prediction information. The detailed configuration and processing of the motion compensation prediction unit 306 will be described later.

[0082] <Explanation of the Interpretation Unit 203 on the Decoding Side> Figure 22 shows a detailed configuration of the Interpretation Unit 203 of the image decoding device shown in Figure 2.

[0083] The normal prediction motion vector mode derivation unit 401 derives a plurality of normal prediction motion vector candidates, selects a prediction motion vector, and calculates the sum of the selected prediction motion vector and the decoded difference motion vector to obtain the motion vector. The decoded inter-prediction mode, reference index, and motion vector become the inter-prediction information of the normal prediction motion vector mode. This inter-prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the normal prediction motion vector mode derivation unit 401 will be described later.

[0084] The normal merge mode derivation unit 402 derives multiple normal merge candidates, selects a candidate, and obtains inter-prediction information for the normal merge mode. This inter-prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the normal merge mode derivation unit 402 will be described later.

[0085] The subblock predicted motion vector mode derivation unit 403 derives multiple subblock predicted motion vector candidates, selects a subblock predicted motion vector, and calculates the sum of the selected subblock predicted motion vector and the decoded difference motion vector to obtain the motion vector. The decoded inter-prediction mode, reference index, and motion vector become the inter-prediction information for the subblock predicted motion vector mode. This inter-prediction information is supplied to the motion compensation prediction unit 406 via the switch 408.

[0086] The subblock merge mode derivation unit 404 derives multiple subblock merge candidates, selects a candidate, and obtains inter-prediction information for the subblock merge mode. This inter-prediction information is supplied to the motion compensation prediction unit 406 via the switch 408.

[0087] The motion compensation prediction unit 406 performs interpretation on the reference image signal stored in the decoded image memory 208 based on the determined interpretation prediction information. The detailed configuration and processing of the motion compensation prediction unit 406 are the same as those of the motion compensation prediction unit 306 on the encoding side.

[0088] <Normal Predicted Motion Vector Mode Derivation Unit (Normal AMVP)> The normal predicted motion vector mode derivation unit 301 in Figure 17 includes a spatial predicted motion vector candidate derivation unit 321, a time predicted motion vector candidate derivation unit 322, a history predicted motion vector candidate derivation unit 323, a predicted motion vector candidate supplementation unit 325, a normal motion vector detection unit 326, a predicted motion vector candidate selection unit 327, and a motion vector subtraction unit 328.

[0089] The normal predicted motion vector mode derivation unit 401 in Figure 23 includes a spatial predicted motion vector candidate derivation unit 421, a time predicted motion vector candidate derivation unit 422, a history predicted motion vector candidate derivation unit 423, a predicted motion vector candidate supplementation unit 425, a predicted motion vector candidate selection unit 426, and a motion vector addition unit 427.

[0090] The processing procedures of the encoding-side normal predicted motion vector mode derivation unit 301 and the decoding-side normal predicted motion vector mode derivation unit 401 will be explained using the flowcharts in Figures 19 and 25, respectively. Figure 19 is a flowchart showing the normal predicted motion vector mode derivation processing procedure by the encoding-side normal predicted motion vector mode derivation unit 301, and Figure 25 is a flowchart showing the normal predicted motion vector mode derivation processing procedure by the decoding-side normal predicted motion vector mode derivation unit 401.

[0091] <Explanation of the Normal Predicted Motion Vector Mode Derivation Unit (Normal AMVP): Encoding Side> The normal predicted motion vector mode derivation process on the encoding side will be explained with reference to Figure 19. In the explanation of the process in Figure 19, the word "normal" as shown in Figure 19 may be omitted.

[0092] First, the motion vector detection unit 326 detects the normal motion vector for each interpretation mode and reference index (step S100 in Figure 19).

[0093] Next, the spatial prediction motion vector candidate derivation unit 321, the time prediction motion vector candidate derivation unit 322, the history prediction motion vector candidate derivation unit 323, the prediction motion vector candidate supplementation unit 325, the prediction motion vector candidate selection unit 327, and the motion vector subtraction unit 328 calculate the difference motion vectors of the motion vectors used in inter prediction in the normal prediction motion vector mode for L0 and L1, respectively (steps S101 to S106 in Figure 19). Specifically, if the prediction mode PredMode of the block to be processed is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), the prediction motion vector candidate list mvpListL0 for L0 is calculated, the prediction motion vector mvpL0 is selected, and the difference motion vector mvdL0 of the L0 motion vector mvL0 is calculated. If the interpretation mode of the block to be processed is L1 prediction (Pred_L1), the L1 prediction motion vector candidate list mvpListL1 is calculated, the prediction motion vector mvpL1 is selected, and the difference motion vector mvdL1 of the L1 motion vector mvL1 is calculated. If the interpretation mode of the block to be processed is dual prediction (Pred_BI), both L0 and L1 predictions are performed, the L0 prediction motion vector candidate list mvpListL0 is calculated, the L0 prediction motion vector mvpL0 is selected, the difference motion vector mvdL0 of the L0 motion vector mvL0 is calculated, and the L1 prediction motion vector candidate list mvpListL1 is calculated, the L1 prediction motion vector mvpL1 is calculated, and the difference motion vector mvdL1 of the L1 motion vector mvL1 is calculated.

[0094] The differential motion vector calculation process is performed for both L0 and L1, but the process is common to both L0 and L1. Therefore, in the following explanation, L0 and L1 will be represented as a common LX. In the process of calculating the differential motion vector for L0, X in LX is 0, and in the process of calculating the differential motion vector for L1, X in LX is 1. Also, when the process of calculating the differential motion vector for LX refers to information from the other list instead of LX, the other list will be represented as LY.

[0095] When using the motion vector mvLX for LX (step S102 in Figure 19: YES), candidate predicted motion vectors for LX are calculated and a list of candidate predicted motion vectors for LX, mvpListLX, is constructed (step S103 in Figure 19). In the normal predicted motion vector mode derivation unit 301, multiple candidate predicted motion vectors are derived using the spatial predicted motion vector candidate derivation unit 321, the time predicted motion vector candidate derivation unit 322, the history predicted motion vector candidate derivation unit 323, and the predicted motion vector candidate supplementation unit 325 to construct the list of candidate predicted motion vectors, mvpListLX. A detailed processing procedure for step S103 in Figure 19 will be described later using the flowchart in Figure 20.

[0096] Next, the predicted motion vector candidate selection unit 327 selects the predicted motion vector mvpLX for LX from the predicted motion vector candidate list mvpListLX (step S104 in Figure 19). Here, one element (the i-th element counting from 0) in the predicted motion vector candidate list mvpListLX is represented as mvpListLX[i]. The difference between the motion vector mvLX and each predicted motion vector candidate mvpListLX[i] stored in the predicted motion vector candidate list mvpListLX is calculated, which is the difference between each motion vector. The code amount when these difference motion vectors are encoded is calculated for each element (predicted motion vector candidate) in the predicted motion vector candidate list mvpListLX. Then, among the elements registered in the predicted motion vector candidate list mvpListLX, the predicted motion vector candidate mvpListLX[i] that has the minimum code amount for each predicted motion vector candidate is selected as the predicted motion vector mvpLX, and its index i is obtained. If there are multiple candidates for the predicted motion vector with the smallest generated sign amount in the predicted motion vector candidate list mvpListLX, the candidate mvpListLX[i] represented by the smallest index i in the predicted motion vector candidate list mvpListLX is selected as the optimal predicted motion vector mvpLX, and its index i is obtained.

[0097] Next, the motion vector subtraction unit 328 subtracts the selected predicted motion vector mvpLX of LX from the motion vector mvLX of LX, and calculates the difference motion vector mvdLX of LX as mvdLX = mvLX - mvpLX (step S105 in Figure 19).

[0098] <Explanation of the Decoder Side: Derivation of the Normal Predicted Motion Vector Mode (Normal AMVP)> Next, the normal predicted motion vector mode processing procedure on the decoding side will be explained with reference to Figure 25. On the decoding side, the spatial predicted motion vector candidate derivation unit 421, the time predicted motion vector candidate derivation unit 422, the history predicted motion vector candidate derivation unit 423, and the predicted motion vector candidate supplementation unit 425 calculate the motion vectors used in the inter prediction of the normal predicted motion vector mode for L0 and L1 respectively (steps S201 to S206 in Figure 25). Specifically, if the prediction mode PredMode of the block to be processed is inter prediction (MODE_INTER) and the inter prediction mode of the block to be processed is L0 prediction (Pred_L0), the predicted motion vector candidate list mvpListL0 for L0 is calculated, the predicted motion vector mvpL0 is selected, and the motion vector mvL0 for L0 is calculated. If the interpretation mode of the block to be processed is L1 prediction (Pred_L1), the L1 prediction motion vector candidate list mvpListL1 is calculated, the prediction motion vector mvpL1 is selected, and the L1 motion vector mvL1 is calculated. If the interpretation mode of the block to be processed is dual prediction (Pred_BI), both L0 and L1 predictions are performed, the L0 prediction motion vector candidate list mvpListL0 is calculated, the L0 prediction motion vector mvpL0 is selected, and the L0 motion vector mvL0 is calculated, as well as the L1 prediction motion vector candidate list mvpListL1, the L1 prediction motion vector mvpL1 is calculated, and the L1 motion vector mvL1 is calculated.

[0099] Similar to the encoding side, the decoding side also performs motion vector calculation processing for L0 and L1 respectively, but the processing is common to both L0 and L1. Therefore, in the following explanation, L0 and L1 will be represented as a common LX. LX represents the inter prediction mode used for inter prediction of the encoded block being processed. In the process of calculating the motion vector of L0, X is 0, and in the process of calculating the motion vector of L1, X is 1. Also, if, during the process of calculating the motion vector of LX, information from a reference list other than the same reference list as the LX being calculated is referenced, the other reference list will be represented as LY.

[0100] When using the motion vector mvLX for LX (step S202 in Figure 25: YES), candidate predicted motion vectors for LX are calculated and a list of candidate predicted motion vectors for LX, mvpListLX, is constructed (step S203 in Figure 25). Normally, the spatial predicted motion vector candidate derivation unit 421, the time predicted motion vector candidate derivation unit 422, the history predicted motion vector candidate derivation unit 423, and the predicted motion vector candidate supplementation unit 425 within the predicted motion vector mode derivation unit 401 calculate multiple candidate predicted motion vectors and construct the list of candidate predicted motion vectors, mvpListLX. A detailed explanation of the processing procedure in step S203 in Figure 25 will be described later using the flowchart in Figure 20.

[0101] Next, the predicted motion vector candidate selection unit 426 extracts the candidate predicted motion vector mvpListLX[mvpIdxLX] corresponding to the predicted motion vector index mvpIdxLX that is decoded and supplied from the predicted motion vector candidate list mvpListLX by the bit sequence decoding unit 201, as the selected predicted motion vector mvpLX (step S204 in Figure 25).

[0102] Next, the motion vector addition unit 427 adds the difference motion vector mvdLX of LX, which is decoded and supplied by the bit sequence decoding unit 201, and the predicted motion vector mvpLX of LX, and calculates the motion vector mvLX of LX as mvLX = mvpLX + mvdLX (step S205 in Figure 25).

[0103] <Normal Predicted Motion Vector Mode Derivation Unit (Normal AMVP): Motion Vector Prediction Method> Figure 20 is a flowchart showing the processing procedure for the normal predicted motion vector mode derivation process, which has functions common to the normal predicted motion vector mode derivation unit 301 of the image coding device and the normal predicted motion vector mode derivation unit 401 of the image decoding device according to an embodiment of the present invention.

[0104] The normal predicted motion vector mode derivation unit 301 and the normal predicted motion vector mode derivation unit 401 are equipped with a predicted motion vector candidate list mvpListLX. The predicted motion vector candidate list mvpListLX has a list structure and is provided with a storage area that stores a predicted motion vector index indicating the location within the predicted motion vector candidate list and the predicted motion vector candidate corresponding to the index as elements. The predicted motion vector index starts from 0, and the predicted motion vector candidates are stored in the storage area of ​​the predicted motion vector candidate list mvpListLX. In this embodiment, the predicted motion vector candidate list mvpListLX can register at least two predicted motion vector candidates (inter-prediction information). Furthermore, the variable numCurrMvpCand, which indicates the number of predicted motion vector candidates registered in the predicted motion vector candidate list mvpListLX, is set to 0.

[0105] The spatial prediction motion vector candidate derivation units 321 and 421 derive candidate prediction motion vectors from the block adjacent to the left. In this process, the predicted motion vector mvLXA is derived by referring to the inter-prediction information of the block adjacent to the left (A0 or A1 in Figure 11), i.e., a flag indicating whether or not a candidate prediction motion vector is available, as well as the motion vector, reference index, etc., and the derived mvLXA is added to the prediction motion vector candidate list mvpListLX (step S301 in Figure 20). Note that X is 0 when predicting L0, and X is 1 when predicting L1 (the same applies hereafter). Subsequently, the spatial prediction motion vector candidate derivation units 321 and 421 derive candidate prediction motion vectors from the block adjacent to the upper side. In this process, the predicted motion vector mvLXB is derived by referring to the interpretation information of the upper adjacent block (B0, B1, or B2 in Figure 11), i.e., a flag indicating whether or not a predicted motion vector candidate is available, as well as the motion vector, reference index, etc. If the derived mvLXA and mvLXB are not equal, mvLXB is added to the predicted motion vector candidate list mvpListLX (step S302 in Figure 20). The processes in steps S301 and S302 in Figure 20 are the same except that the position and number of adjacent blocks referenced are different, and the flag availableFlagLXN indicating whether or not a predicted motion vector candidate for the encoded block is available, the motion vector mvLXN, and the reference index refIdxN (N represents A or B, and so on) are derived.

[0106] Next, the time-predicted motion vector candidate derivation units 322 and 422 derive candidate predicted motion vectors from blocks in pictures with different time zones than the currently processed picture. In this process, a flag availableFlagLXCol indicating whether or not candidate predicted motion vectors for encoded blocks in pictures with different time zones are available, along with the motion vector mvLXCol, reference index refIdxCol, and reference list listCol are derived, and mvLXCol is added to the predicted motion vector candidate list mvpListLX (step S303 in Figure 20).

[0107] Furthermore, the processing of time-predicted motion vector candidate derivation units 322 and 422 can be omitted in units of sequence (SPS), picture (PPS), or slice.

[0108] Next, the history prediction motion vector candidate derivation units 323 and 423 add the history prediction motion vector candidates registered in the history prediction motion vector candidate list HmvpCandList to the prediction motion vector candidate list mvpListLX. (Step S304 in Figure 20). Details of the registration process in step S304 will be described later using the flowchart in Figure 29.

[0109] Next, the predicted motion vector candidate replenishment units 325 and 425 add predicted motion vector candidates with predetermined values, such as (0,0), until the predicted motion vector candidate list mvpListLX is satisfied (S305 in Figure 20).

[0110] <Normal Merge Mode Derivation Unit (Normal Merge)> The normal merge mode derivation unit 302 in Figure 18 includes a spatial merge candidate derivation unit 341, a time merge candidate derivation unit 342, an average merge candidate derivation unit 344, a history merge candidate derivation unit 345, a merge candidate replenishment unit 346, and a merge candidate selection unit 347.

[0111] The normal merge mode derivation unit 402 in Figure 24 includes a spatial merge candidate derivation unit 441, a time merge candidate derivation unit 442, an average merge candidate derivation unit 444, a history merge candidate derivation unit 445, a merge candidate replenishment unit 446, and a merge candidate selection unit 447.

[0112] Figure 21 is a flowchart illustrating the procedure for a normal merge mode derivation process that has functions common to both the normal merge mode derivation unit 302 of the image encoding device and the normal merge mode derivation unit 402 of the image decoding device according to an embodiment of the present invention.

[0113] The following explains the various processes in order. Unless otherwise specified, the following explanation will describe the case where slice type slice_type is B slice, but it can also be applied to P slice. However, when slice type slice_type is P slice, there is only L0 prediction (Pred_L0) as the inter-prediction mode, and there is no L1 prediction (Pred_L1) or bi-prediction (Pred_BI), so the processing related to L1 can be omitted.

[0114] The normal merge mode derivation unit 302 and the normal merge mode derivation unit 402 are equipped with a merge candidate list mergeCandList. The merge candidate list mergeCandList has a list structure and is provided with a merge index that indicates the location within the merge candidate list and a storage area that stores the merge candidate corresponding to the index as an element. The merge index numbers start from 0, and merge candidates are stored in the storage area of ​​the merge candidate list mergeCandList. In subsequent processing, the merge candidate with merge index i registered in the merge candidate list mergeCandList will be represented by mergeCandList[i]. In this embodiment, the merge candidate list mergeCandList can register at least 6 merge candidates (inter-prediction information). Furthermore, the variable numCurrMergeCand, which indicates the number of merge candidates registered in the merge candidate list mergeCandList, is set to 0.

[0115] The spatial merge candidate derivation unit 341 and the spatial merge candidate derivation unit 441 derive spatial merge candidates A and B from blocks adjacent to the left and above the processing target block from the encoded information stored in the encoded information storage memory 111 of the image encoding device or the encoded information storage memory 205 of the image decoding device, and register the derived spatial merge candidates in the merge candidate list mergeCandList (step S401 in Figure 21). Here, N is defined to represent either spatial merge candidates A and B, or time merge candidate Col. A flag availableFlagN indicating whether the inter prediction information of block N can be used as a spatial merge candidate, the reference index refIdxL0N of L0 and the reference index refIdxL1N of L1 of spatial merge candidate N, an L0 prediction flag predFlagL0N indicating whether L0 prediction is performed and an L1 prediction flag predFlagL1N indicating whether L1 prediction is performed, the motion vector mvL0N of L0 and the motion vector mvL1N of L1 are derived. However, in this embodiment, merge candidates are derived without referring to the inter-prediction information of the blocks included in the encoding block to be processed; therefore, spatial merge candidates that use the inter-prediction information of the blocks included in the encoding block to be processed are not derived.

[0116] Next, the time merge candidate derivation unit 342 and the time merge candidate derivation unit 442 derive time merge candidates from pictures of different time periods and register the derived time merge candidates in the merge candidate list mergeCandList (step S402 in Figure 21). A flag availableFlagCol indicating whether a time merge candidate is available, an L0 prediction flag predFlagL0Col indicating whether an L0 prediction is performed for the time merge candidate, an L1 prediction flag predFlagL1Col indicating whether an L1 prediction is performed, and the L0 motion vector mvL0Col and L1 motion vector mvL1Col are derived.

[0117] Furthermore, the processing of the time merge candidate derivation unit 342 and the time merge candidate derivation unit 442 can be omitted in units of sequence (SPS), picture (PPS), or slice.

[0118] Next, the history merge candidate derivation unit 345 and the history merge candidate derivation unit 445 register the history predicted motion vector candidates registered in the history predicted motion vector candidate list HmvpCandList to the merge candidate list mergeCandList (step S403 in Figure 21). If the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, the history merge candidates are derived up to the maximum number of merge candidates MaxNumMergeCand, and registered in the merge candidate list mergeCandList.

[0119] Next, the average merge candidate derivation unit 344 and the average merge candidate derivation unit 444 derive average merge candidates from the merge candidate list mergeCandList and add the derived average merge candidates to the merge candidate list mergeCandList (step S404 in Figure 21). If the number of merge candidates registered in the merge candidate list mergeCandList, numCurrMergeCand, is smaller than the maximum number of merge candidates, MaxNumMergeCand, then the average merge candidates are derived with the number of merge candidates registered in the merge candidate list mergeCandList numCurrMergeCand being limited to the maximum number of merge candidates, MaxNumMergeCand, and registered in the merge candidate list mergeCandList. Here, the average merge candidate is a new merge candidate having a motion vector obtained by averaging the motion vectors of the first merge candidate and the second merge candidate registered in the merge candidate list mergeCandList for each L0 prediction and L1 prediction.

[0120] Next, in the merge candidate replenishment unit 346 and the merge candidate replenishment unit 446, if the number of merge candidates registered in the merge candidate list mergeCandList, numCurrMergeCand, is smaller than the maximum number of merge candidates, MaxNumMergeCand, then additional merge candidates are derived up to the maximum number of merge candidates, MaxNumMergeCand, and registered in the merge candidate list mergeCandList (step S405 in Figure 21). With the maximum number of merge candidates, MaxNumMergeCand, as the upper limit, in the P slice, merge candidates with a motion vector value of (0,0) and a prediction mode of L0 prediction (Pred_L0) are added. In the B slice, merge candidates with a motion vector value of (0,0) and a prediction mode of biprediction (Pred_BI) are added. The reference index used when adding merge candidates is different from the reference index already added.

[0121] Next, the merge candidate selection unit 347 and the merge candidate selection unit 447 select merge candidates from those registered in the merge candidate list mergeCandList. The encoding-side merge candidate selection unit 347 selects merge candidates by calculating the code amount and distortion amount, and supplies the merge index indicating the selected merge candidate and the inter-prediction information of the merge candidate to the motion compensation prediction unit 306 via the inter-prediction mode determination unit 305. On the other hand, the decoding-side merge candidate selection unit 447 selects merge candidates based on the decoded merge index and supplies the selected merge candidates to the motion compensation prediction unit 406.

[0122] <Updating the History Prediction Motion Vector Candidate List> Next, we will explain in detail how to initialize and update the history prediction motion vector candidate list HmvpCandList, which is stored in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side. Figure 26 is a flowchart illustrating the history prediction motion vector candidate list initialization and update process procedure.

[0123] In this embodiment, the updating of the history prediction motion vector candidate list HmvpCandList is performed in the encoded information storage memory 111 and the encoded information storage memory 205. Alternatively, a history prediction motion vector candidate list update unit may be installed in the interpretation unit 102 and the interpretation unit 203 to perform the updating of the history prediction motion vector candidate list HmvpCandList.

[0124] At the beginning of the slice, the history prediction motion vector candidate list HmvpCandList is initialized. On the encoding side, the prediction method determination unit 105 updates the history prediction motion vector candidate list HmvpCandList if the normal prediction motion vector mode or normal merge mode is selected. On the decoding side, the bit sequence decoding unit 201 updates the history prediction motion vector candidate list HmvpCandList if the decoded prediction information is in the normal prediction motion vector mode or normal merge mode.

[0125] Interpretation information used when performing interpretation in normal prediction motion vector mode or normal merge mode is registered as an interpretation information candidate hMvpCand in the historical prediction motion vector candidate list HmvpCandList. The interpretation information candidate hMvpCand includes the L0 reference index refIdxL0 and the L1 reference index refIdxL1, the L0 prediction flag predFlagL0 indicating whether or not L0 prediction is performed and the L1 prediction flag predFlagL1 indicating whether or not L1 prediction is performed, the L0 motion vector mvL0 and the L1 motion vector mvL1.

[0126] If an element (i.e., interpretation information) with the same value as the interpretation information candidate hMvpCand exists in the history prediction motion vector candidate list HmvpCandList, which is stored in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side, then that element is deleted from the history prediction motion vector candidate list HmvpCandList. On the other hand, if no interpretation information with the same value as the interpretation information candidate hMvpCand exists, then the first element of the history prediction motion vector candidate list HmvpCandList is deleted, and the interpretation information candidate hMvpCand is added to the end of the history prediction motion vector candidate list HmvpCandList.

[0127] The number of elements in the history prediction motion vector candidate list HmvpCandList provided in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side of the present invention is 6.

[0128] First, the history prediction motion vector candidate list HmvpCandList is initialized on a slice-by-slice basis. A history prediction motion vector candidate is added to all elements of the history prediction motion vector candidate list HmvpCandList at the beginning of the slice, and the value of NumHmvpCand, which is the number of history prediction motion vector candidates registered in the history prediction motion vector candidate list HmvpCandList, is set to 6 (step S2101 in Figure 26).

[0129] Here, we've assumed that the HmvpCandList, a list of candidate motion vectors for historical prediction, is initialized on a slice-by-slice basis (the first encoded block of a slice), but it can also be initialized on a picture-by-picture, tile-by-tile, or tree block-by-row basis.

[0130] Figure 38 is a table showing an example of historical prediction motion vector candidates added by the initialization of the historical prediction motion vector candidate list HmvpCandList. The example shows the case where the slice type is a B slice and the number of reference pictures is 4. Interpretation information with motion vector values ​​of (0, 0) according to the slice type, starting from (number of historical prediction motion vector candidates NumHmvpCand-1) down to 0, is added to the historical prediction motion vector candidate list HmvpCandList as historical prediction motion vector candidates, filling the historical prediction motion vector candidate list with historical candidates. At this time, the historical prediction motion vector index starts from (number of historical prediction motion vector candidates NumHmvpCand-1), and the reference index refIdxLX (where X is 0 or 1) is set to a value incremented by 1 from 0 to (number of reference pictures numRefIdx-1). After that, duplication of historical prediction motion vector candidates is allowed, and the value of refIdxLX is set to 0. By setting all values ​​to the number of historical prediction motion vector candidates, NumHmvpCand, and fixing the value of NumHmvpCand to a fixed value, invalid historical prediction motion vector candidates are eliminated. In this way, coding efficiency can be improved by assigning values ​​with small reference indices, which generally have a high selectivity, from candidates with large historical prediction motion vector indices that are more likely to be added to the prediction motion vector candidate list or merge candidate list.

[0131] Furthermore, by filling the history prediction motion vector candidate list with history prediction motion vector candidates on a slice-by-slice basis, the number of history prediction motion vector candidates can be treated as a fixed value, thus simplifying processes such as the history prediction motion vector candidate derivation process and the history merge candidate derivation process.

[0132] Here, the motion vector value is set to (0, 0), which generally has a high selection probability, but any predetermined value will suffice. For example, values ​​such as (4, 4), (0, 32), or (-128, 0) can be used to improve the encoding efficiency of the differential motion vector, or multiple predetermined values ​​can be set to improve the encoding efficiency of the differential motion vector.

[0133] Furthermore, while it was stated that the historical prediction motion vector index starts from (the number of historical prediction motion vector candidates NumHmvpCand-1) and the reference index refIdxLX (where X is 0 or 1) is set to a value incremented by 1 from 0 to (the number of reference pictures numRefIdx-1), it is also acceptable to start the historical prediction motion vector index from 0.

[0134] Figure 39 is a table showing another example of historical prediction motion vector candidates added by the initialization of the historical prediction motion vector candidate list HmvpCandList. This example shows the case where the slice type is B slice and the number of reference pictures is 2. In this example, the historical prediction motion vector candidate list HmvpCandList is filled by adding interpretation information that differs in either the reference index or the motion vector value as historical prediction motion vector candidates, so that there are no overlaps between historical prediction motion vector candidates for each element. At this time, the historical prediction motion vector index starts from (number of historical prediction motion vector candidates NumHmvpCand-1), and the reference index refIdxLX (X is 0 or 1) is set to a value that is incremented by 1 from 0 to (number of reference pictures numRefIdx-1). After that, motion vectors with different values ​​of 0 for refIdxLX are added as historical prediction motion vector candidates. By setting all values ​​to the number of historical prediction motion vector candidates NumHmvpCand and fixing the value of the number of historical prediction motion vector candidates NumHmvpCand to a fixed value, invalid historical prediction motion vector candidates are eliminated.

[0135] In this way, by filling the history prediction motion vector candidate list with non-overlapping history prediction motion vector candidates on a slice-by-slice basis, the processing of the merge candidate replenishment unit 346, which is performed on a coding block basis and follows the history merge candidate derivation unit 345 in the normal merge mode derivation unit 302 described later, can be omitted, thereby reducing the amount of processing required.

[0136] Here, the motion vector values ​​were set to small values ​​such as (0, 0) and (1, 0), but the motion vector values ​​can be increased as long as there is no overlap between the candidate motion vectors for historical prediction.

[0137] Furthermore, while it was stated that the historical prediction motion vector index starts from (the number of historical prediction motion vector candidates NumHmvpCand-1) and the reference index refIdxLX (where X is 0 or 1) is set to a value incremented by 1 from 0 to (the number of reference pictures numRefIdx-1), it is also acceptable to start the historical prediction motion vector index from 0.

[0138] Figure 40 is a table showing another example of historical prediction motion vector candidates added by the initialization of the historical prediction motion vector candidate list HmvpCandList.

[0139] This example shows the case where the slice type is B slice. In this example, to ensure that there are no duplicate historical prediction motion vector candidates in the historical prediction motion vector candidate list HmvpCandList, inter-prediction information with a reference index of 0 and different motion vector values ​​is added as a historical prediction motion vector candidate, thereby filling the historical prediction motion vector candidate list. At this time, the historical prediction motion vector index starts from (number of historical prediction motion vector candidates NumHmvpCand-1), and the reference index refIdxLX (X is 0 or 1) is set to 0. All values ​​are set to the number of historical prediction motion vector candidates NumHmvpCand, and invalid historical prediction motion vector candidates are eliminated by fixing the value of the number of historical prediction motion vector candidates NumHmvpCand to a fixed value.

[0140] In this way, by setting the reference index to 0, initialization can be performed without considering the number of referenced pictures, thus simplifying the process.

[0141] Here, the motion vector values ​​are set to multiples of 2, but other values ​​are also acceptable as long as the reference index is 0 and there is no overlap between candidate historically predicted motion vectors.

[0142] Furthermore, while it was stated that the historical prediction motion vector index starts from (the number of historical prediction motion vector candidates NumHmvpCand-1) and the reference index refIdxLX (where X is 0 or 1) is set to a value incremented by 1 from 0 to (the number of reference pictures numRefIdx-1), it is also acceptable to start the historical prediction motion vector index from 0.

[0143] Next, the following process of updating the history prediction motion vector candidate list HmvpCandList is repeatedly performed for each encoded block within the slice (steps S2102 to S2111 in Figure 26).

[0144] First, initial settings are performed on a per-encoded block basis. The flag `identicalCandExist`, which indicates whether or not identical candidates exist, is set to FALSE, and the index to be deleted, `removeIdx`, is set to 0 (step S2103 in Figure 26).

[0145] Although the initialization of the history prediction motion vector candidate list HmvpCandList is performed on a slice-by-slice basis (the first encoded block of a slice), it may also be performed on a picture-by-picture, tile-by-tile, or tree block-by-row basis.

[0146] It is determined whether or not a candidate inter-prediction information hMvpCand to be registered exists (step S2104 in Figure 26). If the prediction method determination unit 105 on the encoding side determines that it is in normal prediction motion vector mode or normal merge mode, or if the bit string decoding unit 201 on the decoding side decodes it as normal prediction motion vector mode or normal merge mode, then that inter-prediction information is set as the candidate inter-prediction information hMvpCand to be registered. If the prediction method determination unit 105 on the encoding side determines that it is in intra-prediction mode, sub-block prediction motion vector mode or sub-block merge mode, or if the bit string decoding unit 201 on the decoding side decodes it as intra-prediction mode, sub-block prediction motion vector mode or sub-block merge mode, then the history prediction motion vector candidate list HmvpCandList is not updated, and there is no candidate inter-prediction information hMvpCand to be registered. If there is no candidate inter-prediction information hMvpCand to be registered, steps S2105 to S2106 are skipped (step S2104 in Figure 26: NO). If a candidate for the interpretation information to be registered, hMvpCand, exists, the following steps are performed (step S2104 in Figure 26: YES).

[0147] Next, it is determined whether there is an element with the same value as the candidate inter-prediction information hMvpCand to be registered, i.e., an identical element, in the list of candidate historical motion vectors HmvpCandList (step S2105 in Figure 26). Figure 27 is a flowchart of this identical element verification process. If the value of the number of candidate historical motion vectors NumHmvpCand is 0 (step S2121 in Figure 27: NO), the list of candidate historical motion vectors HmvpCandList is empty and no identical candidates exist, so steps S2122 to S2125 in Figure 27 are skipped and this identical element verification process is terminated. If the value of the number of candidate historical motion vectors NumHmvpCand is greater than 0 (step S2121 in Figure 27: YES), the process in step S2123 is repeated for the history prediction motion vector index hMvpIdx from 0 to NumHmvpCand-1 (steps S2122 to S2125 in Figure 27). First, the system compares whether the element at position hMvpIdx (counting from 0) of the historical prediction motion vector candidate list, HmvpCandList[hMvpIdx], is identical to the inter-prediction information candidate hMvpCand (step S2123 in Figure 27). If they are identical (step S2123 in Figure 27: YES), the flag identicalCandExist, which indicates whether an identical candidate exists, is set to TRUE, and the value of the current historical prediction motion vector index hMvpIdx is set to the removal target index removeIdx, which indicates the position of the element to be removed, and the identical element verification process is terminated. If they are not identical (step S2123 in Figure 27: NO), hMvpIdx is incremented by 1, and if the historical prediction motion vector index hMvpIdx is less than or equal to NumHmvpCand-1, the process from step S2123 onward is performed.

[0148] Here, by filling the history prediction motion vector candidate list with history prediction motion vector candidates, step S2121 in Figure 27 can be omitted.

[0149] Returning to the flowchart in Figure 26, the shift and addition processes for elements in the history prediction motion vector candidate list HmvpCandList are performed (step S2106 in Figure 26). Figure 28 is a flowchart of the element shift / addition process for the history prediction motion vector candidate list HmvpCandList in step S2106 of Figure 26. First, it is determined whether to remove elements stored in the history prediction motion vector candidate list HmvpCandList and then add new elements, or to add new elements without removing any elements. Specifically, the flag identicalCandExist, which indicates whether an identical candidate exists, is checked to see if it is TRUE, or whether NumHmvpCand is 6 (step S2141 in Figure 28). If either the flag identicalCandExist, which indicates whether an identical candidate exists, is TRUE, or the current number of candidates NumHmvpCand is 6 is met (step S2141 in Figure 28: YES), the elements stored in the history prediction motion vector candidate list HmvpCandList are removed and then new elements are added. The initial value of index i is set to removeIdx + 1. The element shifting process in step S2143 is repeated from this initial value up to NumHmvpCand (steps S2142 to S2144 in Figure 28). The elements are shifted forward by copying the elements of HmvpCandList[i] to HmvpCandList[i - 1] (step S2143 in Figure 28), and i is incremented by 1 (steps S2142 to S2144 in Figure 28). Subsequently, the inter-prediction information candidate hMvpCand is added to the (NumHmvpCand-1)th HmvpCandList[NumHmvpCand-1], which corresponds to the end of the history prediction motion vector candidate list (step S2145 in Figure 28), and the element shifting and addition process of this history prediction motion vector candidate list HmvpCandList is completed.On the other hand, if neither the flag identicalCandExist (which indicates whether an identical candidate exists) is TRUE, nor is NumHmvpCand equal to 6, then the inter-prediction information candidate hMvpCand is added to the end of the history prediction motion vector candidate list HmvpCandList without removing any elements already stored in the list (step S2146 in Figure 28). Here, the end of the history prediction motion vector candidate list is the NumHmvpCand-th HmvpCandList[NumHmvpCand], counting from 0. Finally, NumHmvpCand is incremented by 1 to complete the element shifting and addition process of the history prediction motion vector candidate list HmvpCandList.

[0150] Here, the historical prediction motion vector candidate list is applied to both the prediction motion vector mode and the merge mode, but it may be applied to only one of them.

[0151] As described above, when updating the historical prediction motion vector candidate list, duplicate elements stored in the historical prediction motion vector candidate list are removed before new elements are added. Therefore, there are no duplicate elements in the historical prediction motion vector candidate list, and each historical prediction motion vector candidate list is composed of completely different elements.

[0152] Figure 31 illustrates an example of the update process for the historical prediction motion vector candidate list. When adding a new element to the historical prediction motion vector candidate list HmvpCandList, which already has six elements (inter-prediction information) registered, the new inter-prediction information is compared with the elements in the historical prediction motion vector candidate list HmvpCandList in order from the beginning (Figure 31A). If the new element has the same value as the third element from the beginning of the historical prediction motion vector candidate list HmvpCandList, HMVP2 is deleted from the historical prediction motion vector candidate list HmvpCandList, and the elements HMVP3 to HMVP5 are shifted (copied) one by one to the beginning. Finally, the new element is added to the end of the historical prediction motion vector candidate list HmvpCandList (Figure 31B), completing the update of the historical prediction motion vector candidate list HmvpCandList (Figure 31C).

[0153] <Processing for Deriving Candidate Historical Prediction Motion Vectors> Next, we will explain in detail the process of deriving candidate historical prediction motion vectors from the historical prediction motion vector candidate list HmvpCandList, which is a common process in the historical prediction motion vector candidate derivation unit 323 of the encoding-side normal prediction motion vector mode derivation unit 301 and the historical prediction motion vector candidate derivation unit 423 of the decoding-side normal prediction motion vector mode derivation unit 401. Figure 29 is a flowchart illustrating the historical prediction motion vector candidate derivation process.

[0154] If the current number of predicted motion vector candidates, numCurrMvpCand, is greater than or equal to the maximum number of elements in the predicted motion vector candidate list, mvpListLX (which is assumed to be 2), or if the number of historical predicted motion vector candidates is equal to 0 (NO in step S2201 of Figure 29), then steps S2202 to S2208 of Figure 29 are omitted, and the historical predicted motion vector candidate derivation procedure is terminated. If the current number of predicted motion vector candidates, numCurrMvpCand, is less than 2, which is the maximum number of elements in the predicted motion vector candidate list, mvpListLX, AND the number of historical predicted motion vector candidates, NumHmvpCand, is greater than 0 (YES in step S2201 of Figure 29), then steps S2202 to S2208 of Figure 29 are performed.

[0155] Next, the process from steps S2203 to S2207 in Figure 29 is repeated until the index i is 1 up to the smaller of 4 or the number of historically predicted motion vector candidates numCheckedHMVPCand (steps S2202 to S2208 in Figure 29). If the current number of predicted motion vector candidates numCurrMvpCand is 2 or greater than or equal to the maximum number of elements in the predicted motion vector candidate list mvpListLX (step S2203 in Figure 29: NO), the process from steps S2204 to S2208 in Figure 29 is omitted, and this historically predicted motion vector candidate derivation procedure is terminated. If the current number of predicted motion vector candidates numCurrMvpCand is less than 2 (step S2203 in Figure 29: YES), the process from step S2204 onwards in Figure 29 is performed.

[0156] Next, steps S2205 and S2206 are performed for Y = 0 and 1 (L0 and L1), respectively (steps S2204 to S2207 in Figure 29). If the current number of predicted motion vector candidates, numCurrMvpCand, is 2 or greater than or equal to the maximum number of elements in the predicted motion vector candidate list, mvpListLX (step S2205 in Figure 29: NO), steps S2206 to S2208 in Figure 29 are omitted, and this history predicted motion vector candidate derivation procedure is terminated. If the current number of predicted motion vector candidates, numCurrMvpCand, is less than 2 (step S2205 in Figure 29: YES), the processes from step S2206 onwards in Figure 29 are performed.

[0157] Next, the LY motion vector from the historical prediction motion vector candidate HmvpCandList[NumHmvpCand - i] is added to the numCurrMvpCand-th element mvpListLY[numCurrMvpCand ] of the LY prediction motion vector candidate list, and the current number of prediction motion vector candidates numCurrMvpCand is incremented by 1 (step S2206 in Figure 29).

[0158] The processes from steps S2205 to S2206 in Figure 29 are performed on both L0 and L1 (steps S2204 to S2207 in Figure 29). Index i is incremented by 1, and if index i is less than or equal to the smaller of 4 and the number of historical prediction motion vector candidates NumHmvpCand, the processes from step S2203 onwards are performed again (steps S2202 to S2208 in Figure 29).

[0159] In this embodiment, as described above, in the process of deriving candidate motion vectors for historical prediction, the motion vectors of the elements in the candidate motion vector list are added to the candidate motion vector list without comparing them with the motion vectors of the elements in the candidate motion vector list.

[0160] By adopting this configuration, if the number of historical prediction motion vector candidates is two or more, it is possible to guarantee that the predicted motion vector candidate list will reach the maximum number of elements after the historical prediction motion vector candidate list derivation process is completed. In addition, the processing load and circuit size required to check whether the motion vectors are identical can be reduced.

[0161] The normal predicted motion vector mode determines the motion information of the block to be processed using the predicted motion vector candidates and difference vectors included in the predicted motion vector candidate list. Because there is room to determine an appropriate motion vector using the difference vector, even if elements in the predicted motion vector candidate list inevitably overlap, the decrease in coding efficiency due to the reduction in choices can be kept to a minimum.

[0162] Furthermore, the normal prediction motion vector mode processes the prediction motion vector candidate list separately for L0 prediction and L1 prediction. Therefore, even if elements overlap in the prediction motion vector candidate list for L0 prediction, elements may not overlap in the prediction motion vector candidate list for L1 prediction.

[0163] Furthermore, the normal predictive motion vector mode processes the predicted motion vector candidates and the reference index separately. Therefore, even if there are duplicate elements in the predicted motion vector candidate list, it does not affect the reference index.

[0164] Furthermore, since the list of predicted motion vector candidates contains at most two elements, even if there are duplicate elements in the list of predicted motion vector candidates, the number of options that decrease is only one.

[0165] Furthermore, since the history prediction motion vector candidate list does not contain identical elements, a comparison between the elements of the history prediction motion vector candidate list and the elements of the prediction motion vector candidate list is only practically meaningful when only one element is generated in the prediction motion vector candidate list by the spatial prediction motion vector candidate derivation unit 421 and the time prediction motion vector candidate derivation unit 422, which is therefore an extremely infrequent occurrence. In addition, the normal prediction motion vector mode is generally selected when the movements of adjacent blocks are not similar, and the elements of the history prediction motion vector candidate list are unlikely to overlap with elements already added to the prediction motion vector candidate list.

[0166] For the reasons stated above, even if elements in the list of predicted motion vector candidates overlap, it is possible to reduce the comparison process between the motion vectors of the historical predicted motion vector candidates and the motion vectors of the predicted motion vector candidates while suppressing the decrease in coding efficiency due to the reduction in the number of choices.

[0167] Furthermore, by filling the history prediction motion vector candidate list with non-duplicate history prediction motion vector candidates, and adding elements from the history prediction motion vector candidate list to the prediction motion vector candidate list without comparing the elements of the history prediction motion vector candidate list with the elements of the prediction motion vector candidate list, the processing of the prediction motion vector supplementation unit 325 after the history prediction motion vector candidate derivation unit 323 in the normal prediction motion vector mode derivation unit 301 can be omitted.

[0168] <History Merge Candidate Derivation Process> Next, we will explain in detail the process of deriving history merge candidates from the history merge candidate list HmvpCandList, which is a common process in the history merge candidate derivation unit 345 of the encoding side's normal merge mode derivation unit 302 and the history merge candidate derivation unit 445 of the decoding side's normal merge mode derivation unit 402, as shown in step S404 of Figure 21. Figure 30 is a flowchart illustrating the history merge candidate derivation process procedure.

[0169] First, initialization is performed (step S2301 in Figure 30). The value FALSE is set for each element from 0 to (numCurrMergeCand - 1) in isPruned[i], and the variable numOrigMergeCand is set to the number of elements currently registered in the merge candidate list, numCurrMergeCand.

[0170] Next, the initial value of index hMvpIdx is set to 1, and the additional processing from step S2303 to step S2310 in Figure 30 is repeated from this initial value up to NumHmvpCand (steps S2302 to S2311 in Figure 30). If the number of elements currently registered in the merge candidate list, numCurrMergeCand, is not less than or equal to (maximum number of merge candidates MaxNumMergeCand-1), then merge candidates have been added to all elements in the merge candidate list, and this history merge candidate derivation process is terminated (NO in step S2303 in Figure 30). If the number of elements currently registered in the merge candidate list, numCurrMergeCand, is less than or equal to (maximum number of merge candidates MaxNumMergeCand-1), the processing from step S2304 onwards is performed. The value of sameMotion is set to FALSE (step S2304 in Figure 30). Next, the initial value of index i is set to 0, and the processes in steps S2306 and S2307 in Figure 30 are performed from this initial value up to numOrigMergeCand-1 (S2305 to S2308 in Figure 30). The (NumHmvpCand - hMvpIdx)th element HmvpCandList[NumHmvpCand- hMvpIdx] from the history motion vector prediction candidate list is compared to the i-th element mergeCandList[i] from the merge candidate list to determine if it has the same value (step S2306 in Figure 30).

[0171] A merge candidate is considered to have the same value if all of its constituent elements (interpretation mode, reference index, motion vector) have the same value. If the merge candidates have the same value and isPruned[i] is FALSE (YES in step S2306 of Figure 30), both sameMotion and isPruned[i] are set to TRUE (step S2307 of Figure 30). If the values ​​are not the same (NO in step S2306 of Figure 30), the process in step S2307 is skipped. Once the iterative processing from step S2305 to step S2308 in Figure 30 is complete, it is checked whether sameMotion is FALSE (step S2309 in Figure 30). If sameMotion is FALSE (YES in step S2309 in Figure 30), that is, the (NumHmvpCand - hMvpIdx)th element of the history prediction motion vector candidate list HmvpCandList[NumHmvpCand - hMvpIdx] counting from 0 does not exist in mergeCandList, so the numCurrMergeCandth mergeCandList[numCurrMergeCand] of the merge candidate list is changed to the (NumHmvpCand - hMvpIdx)th element of the history prediction motion vector candidate list HmvpCandList[NumHmvpCand - The index hMvpIdx is added, and numCurrMergeCand is incremented by 1 (step S2310 in Figure 30). The index hMvpIdx is incremented by 1 (step S2302 in Figure 30), and the process of steps S2302 to S2311 in Figure 30 is repeated. Once all elements of the history prediction motion vector candidate list have been checked, or once merge candidates have been added to all elements of the merge candidate list, the derivation process of this history merge candidate is completed.

[0172] In this embodiment, as described above, the history merge candidate derivation process compares the elements of the history prediction motion vector candidate list with the elements of the current merge candidate list, and adds only the elements of the history prediction motion vector candidate list that are not present in the current merge candidate list to the merge candidate list.

[0173] Unlike the normal predictive motion vector mode, the normal merge mode directly determines the motion information of the block to be processed without using a difference vector. Therefore, encoding efficiency can be improved by prohibiting the addition of elements from the history merge candidate list that overlap with elements from the current merge candidate list. Here, we have assumed that elements from the history predictive motion vector candidate list are compared with all candidates in the current merge candidate list, but this is not limited to this, as long as encoding efficiency can be improved by comparing at least the elements from the history predictive motion vector candidate list with the elements from the current merge candidate list.

[0174] For example, the number of elements in the list of historical prediction motion vector candidates to be compared may be limited to one or two. Similarly, the number of elements in the current merge candidate list to be compared may be limited to one or two. In normal merge mode, the merge candidate list includes both L0 prediction motion vectors and L1 prediction motion vectors. Therefore, unlike in normal prediction motion vector mode, it is not possible to adjust the L0 prediction motion vector and the L1 prediction motion vector separately.

[0175] Furthermore, in normal merge mode, the merge candidate list includes both the L0 prediction reference index and the L1 prediction reference index. Therefore, unlike in normal prediction motion vector mode, it is not possible to adjust the L0 prediction reference index and the L1 prediction reference index separately.

[0176] Furthermore, since the merge candidate list contains up to six elements, which is more than the predicted motion vector candidate list, adding duplicate elements to the merge candidate list increases the number of duplicate elements in the list, making it inefficient to use the merge candidate list.

[0177] Furthermore, the elements of the historical prediction motion vector candidate list that are added to the merge candidate list are the elements that were added most recently to the historical prediction motion vector candidate list. Therefore, the elements of the historical prediction motion vector candidate list that are added to the merge candidate list will be the motion information that is spatially closest to the encoded block being processed. Generally, the normal merge mode is selected when the motion of adjacent blocks is similar, and the elements of the historical prediction motion vector candidate list are likely to overlap with elements that have already been added to the merge candidate list.

[0178] To address the above challenges, encoding efficiency can be improved by increasing the number of valid selection elements by prohibiting the addition of elements to the history prediction motion vector candidate list that overlap with elements in the merge candidate list.

[0179] Furthermore, by making the maximum number of elements that can be included in the merge candidate list, which is 6, greater than the maximum number of elements that can be included in the predicted motion vector candidate list, which is 2, the probability of selecting the normal merge mode is increased. This reduces the comparison process between the motion vectors of the historical predicted motion vector candidates and the motion vectors of the predicted motion vector candidates, while suppressing the decrease in coding efficiency due to the duplication of elements in the predicted motion vector candidate list.

[0180] Furthermore, by including a variety of merge candidates, such as time merge candidates, historical merge candidates, average merge candidates, and zero merge candidates, in addition to spatial merge candidates, and increasing the selection probability of the normal merge mode, it is possible to reduce the comparison process between the motion vectors of historically predicted motion vector candidates and the motion vectors of predicted motion vector candidates while suppressing the decrease in coding efficiency due to the duplication of elements in the predicted motion vector candidate list.

[0181] <Motion Compensation Prediction Processing> The motion compensation prediction unit 306 obtains the position and size of the block currently being predicted during encoding. The motion compensation prediction unit 306 also obtains inter-prediction information from the inter-prediction mode determination unit 305. From the obtained inter-prediction information, it derives a reference index and a motion vector, and generates a prediction signal after obtaining an image signal of the reference picture identified by the reference index in the decoded image memory 104, moved by the amount of the motion vector from the same position as the image signal of the prediction block.

[0182] In interpretation, if the interpretation mode is L0 prediction or L1 prediction, which is prediction from a single reference picture, the prediction signal obtained from one reference picture is used as the motion-compensated prediction signal. If the interpretation mode is BI prediction, which is prediction from two reference pictures, the prediction signals obtained from the two reference pictures are weighted and averaged to obtain the motion-compensated prediction signal, and this motion-compensated prediction signal is supplied to the prediction method determination unit 105. Here, the weighted averaging ratio for the two predictions is set to 1:1, but other ratios may be used for weighted averaging. For example, the weighting ratio may be increased for pictures with close intervals between the picture to be predicted and the reference picture. Alternatively, the weighting ratio may be calculated using a correspondence table between picture interval combinations and weighting ratios.

[0183] The motion compensation prediction unit 406 has the same functions as the motion compensation prediction unit 306 on the encoding side. The motion compensation prediction unit 406 acquires interpretation information from the normal prediction motion vector mode derivation unit 401, the normal merge mode derivation unit 402, the subblock prediction motion vector mode derivation unit 403, and the subblock merge mode derivation unit 404 via the switch 408. The motion compensation prediction unit 406 supplies the obtained motion compensation prediction signal to the decoded image signal superimposition unit 207.

[0184] <About Interpretation Mode> The process of making a prediction from a single reference picture is defined as single prediction. In the case of single prediction, the prediction is made using either L0 prediction or L1 prediction, which is one of the two reference pictures registered in reference lists L0 and L1.

[0185] Figure 32 shows a single prediction where the L0 reference picture (RefL0Pic) is at a time earlier than the picture to be processed (CurPic). Figure 33 shows a single prediction where the L0 prediction reference picture is at a time later than the picture to be processed. Similarly, single prediction can also be performed by replacing the L0 prediction reference picture (RefL1Pic) in Figures 32 and 33 with the L1 prediction reference picture (RefL1Pic).

[0186] The process of making predictions from two reference pictures is defined as dual prediction, and in the case of dual prediction, both L0 prediction and L1 prediction are used and expressed as BI prediction. Figure 34 shows a case of dual prediction where the reference picture for L0 prediction is at a time earlier than the picture to be processed, and the reference picture for L1 prediction is at a time later than the picture to be processed. Figure 35 shows a case of dual prediction where both the reference picture for L0 prediction and the reference picture for L1 prediction are at a time earlier than the picture to be processed. Figure 36 shows a case of dual prediction where both the reference picture for L0 prediction and the reference picture for L1 prediction are at a time later than the picture to be processed.

[0187] Thus, the relationship between the L0 / L1 prediction type and time is not limited to L0 being in the past direction and L1 being in the future direction; it can be used in any way. Also, in the case of dual prediction, both L0 and L1 predictions may be performed using the same reference picture. The decision of whether to perform motion compensation prediction as a single prediction or a dual prediction is made based on information (e.g., a flag) indicating whether or not to use L0 prediction and whether or not to use L1 prediction.

[0188] <Regarding the Reference Index> In the embodiment of the present invention, in order to improve the accuracy of motion compensation prediction, it is possible to select the optimal reference picture from among multiple reference pictures in motion compensation prediction. For this purpose, the reference picture used in motion compensation prediction is used as the reference index, and the reference index is encoded in the bitstream together with the differential motion vector.

[0189] <Motion compensation processing based on normal predicted motion vector mode> As shown in the inter prediction unit 102 on the encoding side in Figure 16, the motion compensation prediction unit 306, when the inter prediction information from the normal predicted motion vector mode derivation unit 301 is selected in the inter prediction mode determination unit 305, obtains this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0190] Similarly, as shown in the inter-prediction unit 203 on the decoding side in Figure 22, when the switch 408 is connected to the normal predicted motion vector mode derivation unit 401 during the decoding process, the motion compensation prediction unit 406 acquires inter-prediction information from the normal predicted motion vector mode derivation unit 401, derives the inter-prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

[0191] <Motion compensation processing based on normal merge mode> As shown in the inter prediction unit 102 on the encoding side in Figure 16, if the inter prediction information from the normal merge mode derivation unit 302 is selected in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 obtains this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0192] Similarly, as shown in the inter-prediction unit 203 on the decoding side in Figure 22, when the switch 408 is connected to the normal merge mode derivation unit 402 during the decoding process, the motion compensation prediction unit 406 acquires inter-prediction information from the normal merge mode derivation unit 402, derives the inter-prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

[0193] <Motion compensation processing based on subblock predicted motion vector mode> As shown in the inter prediction unit 102 on the encoding side in Figure 16, the motion compensation prediction unit 306, when the inter prediction information from the subblock predicted motion vector mode derivation unit 303 is selected in the inter prediction mode determination unit 305, obtains this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0194] Similarly, as shown in the inter-prediction unit 203 on the decoding side in Figure 22, when the switch 408 is connected to the sub-block predicted motion vector mode derivation unit 403 during the decoding process, the motion compensation prediction unit 406 acquires inter-prediction information from the sub-block predicted motion vector mode derivation unit 403, derives the inter-prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

[0195] <Motion compensation processing based on subblock merge mode> As shown in the inter prediction unit 102 on the encoding side in Figure 16, if the inter prediction information from the subblock merge mode derivation unit 304 is selected in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 obtains this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0196] Similarly, as shown in the inter-prediction unit 203 on the decoding side in Figure 22, if the switch 408 is connected to the sub-block merge mode derivation unit 404 during the decoding process, the motion compensation prediction unit 406 acquires inter-prediction information from the sub-block merge mode derivation unit 404, derives the inter-prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

[0197] <Motion compensation processing based on affine transformation prediction> In normal prediction motion vector mode and normal merge mode, motion compensation using an affine model is available based on the following flags. The following flags are reflected in the following flags based on the inter-prediction conditions determined by the inter-prediction mode determination unit 305 during the encoding process and encoded in the bitstream. During the decoding process, it is determined whether or not to perform motion compensation using an affine model based on the following flags in the bitstream.

[0198] The sps_affine_enabled_flag indicates whether affine motion compensation is available in inter-prediction. If sps_affine_enabled_flag is 0, affine motion compensation is suppressed on a sequence-by-sequence basis. Also, inter_affine_flag and cu_affine_type_flag are not transmitted in the CU (encoded block) syntax of the encoded video sequence. If sps_affine_enabled_flag is 1, affine motion compensation can be used in the encoded video sequence.

[0199] The sps_affine_type_flag indicates whether motion compensation using a 6-parameter affine model is available in interpretation. If sps_affine_type_flag is 0, motion compensation using a 6-parameter affine model is suppressed. Also, cu_affine_type_flag is not transmitted in the CU syntax of the encoded video sequence. If sps_affine_type_flag is 1, motion compensation using a 6-parameter affine model can be used in the encoded video sequence. If sps_affine_type_flag does not exist, it is assumed to be 0.

[0200] When decoding a P or B slice, if inter_affine_flag is 1 for the currently processed CU, motion compensation using an affine model is used to generate the motion compensation prediction signal for the currently processed CU. If inter_affine_flag is 0, the affine model is not used for the currently processed CU. If inter_affine_flag does not exist, it is assumed to be 0.

[0201] When decoding a P or B slice, if cu_affine_type_flag is 1 for the currently processed CU, motion compensation using a 6-parameter affine model is used to generate the motion compensation prediction signal for the currently processed CU. If cu_affine_type_flag is 0, motion compensation using a 4-parameter affine model is used to generate the motion compensation prediction signal for the currently processed CU.

[0202] In motion compensation using the affine model, reference indices and motion vectors are derived at the sub-block level. Therefore, a motion compensation prediction signal is generated using the reference indices and motion vectors that are being processed at the sub-block level.

[0203] The 4-parameter affine model is a mode in which the motion vector of a subblock is derived from four parameters: the horizontal and vertical components of the motion vectors of two control points, and motion compensation is performed on a subblock basis.

[0204] <Modification 1> Modification 1 of this embodiment will be described. This modification differs from this embodiment in that a merge difference motion vector mode has been added, and only the differences from this embodiment will be explained. If umve_flag in Figure 12 is 1, it is in merge difference motion vector mode, and if umve_flag is 0, it is in normal merge mode.

[0205] Next, we will explain the operation of the merge difference motion vector mode. This mode allows you to add one merge difference motion vector to each of the L0 predicted motion vector and L1 predicted motion vector of one of the top two merge candidates (the merge candidates with merge indices of 0 and 1 in the merge candidate list).

[0206] In the case of merged-difference motion vector mode, the merged-difference motion vector is encoded by the bit string encoding unit 108 and decoded by the bit string decoding unit 201.

[0207] As described above, since the merge candidate list is also used in the merge difference motion vector mode, encoding efficiency can be improved by prohibiting the addition of elements to the history prediction motion vector candidate list that overlap with elements in the merge candidate list, thereby increasing the number of valid selection elements.

[0208] Furthermore, by adding a merged difference motion vector mode and reducing the selection probability of the normal prediction motion vector mode, it is possible to reduce the comparison process between the motion vectors of historical prediction motion vector candidates and the motion vectors of prediction motion vector candidates while suppressing the decrease in coding efficiency due to the duplication of elements in the prediction motion vector candidate list.

[0209] <Modification 2> Modification 2 of this embodiment will now be described. This modification differs from this embodiment only in the operation of the history prediction motion vector candidate derivation units 323 and 423 shown in Figure 29. Figure 41 is a flowchart illustrating the history prediction motion vector candidate derivation process procedure of Modification 2. Figure 41 differs from Figure 29 in that step S2209 is added. Figure 29 has the same operation as Figure 41 except for step S2209.

[0210] The process for the case where numCurrMvpCand is less than 2, which is the maximum number of elements in the predicted motion vector candidate list (YES in step S2205 of Figure 41) will be explained.

[0211] The system checks whether the reference index of the LY in the historical prediction motion vector candidate HmvpCandList[NumHmvpCand - i] is the same as the reference index of the LY in the coded block to be processed (step S2209 in Figure 41). If the reference index of the LY in the historical prediction motion vector candidate HmvpCandList[NumHmvpCand - i] is the same as the reference index of the LY in the coded block to be processed (YES in step S2209 in Figure 41), the system proceeds to S2206. If the reference index of the LY in the historical prediction motion vector candidate HmvpCandList[NumHmvpCand - i] is not the same as the reference index of the LY in the coded block to be processed (NO in step S2209 in Figure 41), the system proceeds to S2207.

[0212] As described above, when the reference index of the LY of the historical prediction motion vector candidate and the reference index of the LY of the encoded block to be processed are the same, by adding the motion vector of the historical prediction motion vector candidate to the predicted motion vector candidate list for LY prediction, a highly accurate historical prediction motion vector candidate can be added to the predicted motion vector candidate list without comparing the motion vector of the historical prediction motion vector candidate with the motion vector of the predicted motion vector candidate.

[0213] In this embodiment, in the history prediction motion vector candidate derivation process, elements of the history prediction motion vector candidate list are added to the prediction motion vector candidate list without comparing them with the elements of the prediction motion vector candidate list. On the other hand, in the history merge candidate derivation process, elements of the history merge candidate list are compared with the elements of the merge list, and only history merge candidate lists that are not present in the merge list are added to the merge list. By adopting the above configuration, the following effects can be obtained.

[0214] 1. In the process of deriving candidate lists for predicted motion vectors, there is no need to perform additional candidate derivation processing, which reduces the processing load and circuit size. Furthermore, even if elements of the predicted motion vector candidate list inevitably overlap, the decrease in coding efficiency due to the reduction in choices can be kept to a minimum.

[0215] 2. In the history merge candidate derivation process, by prohibiting the addition of elements to the history merge candidate list that overlap with elements in the merge list, an appropriate merge candidate list can be constructed in the normal merge mode, which determines the movement information of the target block without using difference vectors, thereby improving encoding efficiency.

[0216] 3. By filling the history prediction motion vector candidate list with history prediction motion vector candidates that do not overlap, the processing of the merge candidate replenishment unit 346 after the history merge candidate derivation unit 345 in the normal merge mode derivation unit 302 can be omitted, thereby reducing the amount of processing.

[0217] 4. By filling the history prediction motion vector candidate list with non-duplicate history prediction motion vector candidates, and by adding elements from the history prediction motion vector candidate list to the prediction motion vector candidate list without comparing the elements of the history prediction motion vector candidate list with the elements of the prediction motion vector candidate list, the processing of the prediction motion vector supplementation unit 325 after the history prediction motion vector candidate derivation unit 323 in the normal prediction motion vector mode derivation unit 301 can be omitted.

[0218] All of the embodiments described above may be combined in any way.

[0219] In all the embodiments described above, the bitstream output by the image encoding device has a specific data format so that it can be decoded according to the encoding method used in the embodiment. Furthermore, the image decoding device corresponding to this image encoding device can decode the bitstream of this specific data format.

[0220] When a wired or wireless network is used to exchange bitstreams between an image encoding device and an image decoding device, the bitstream may be converted to a data format suitable for the transmission mode of the communication channel before transmission. In this case, a transmitting device is provided that converts the bitstream output by the image encoding device into encoded data in a data format suitable for the transmission mode of the communication channel and transmits it to the network, and a receiving device is provided that receives the encoded data from the network, restores it to a bitstream, and supplies it to the image decoding device. The transmitting device includes a memory that buffers the bitstream output by the image encoding device, a packet processing unit that packets the bitstream, and a transmitting unit that transmits the packetized encoded data over the network. The receiving device includes a receiving unit that receives the packetized encoded data over the network, a memory that buffers the received encoded data, and a packet processing unit that packets the encoded data to generate a bitstream and provides it to the image decoding device.

[0221] Furthermore, by adding a display unit to the configuration that displays the image decoded by the image decoding device, it can also be used as a display device. In that case, the display unit reads the decoded image signal generated by the decoded image signal superimposition unit 207 and stored in the decoded image memory 208 and displays it on the screen.

[0222] Alternatively, by adding an imaging unit to the configuration and inputting the captured image into an image encoding device, it can also function as an imaging device. In that case, the imaging unit inputs the captured image signal into the block division unit 101.

[0223] Figure 37 shows an example of the hardware configuration of the encoding / decoding device of this embodiment. The encoding / decoding device encompasses the configuration of the image encoding device and the image decoding device according to the embodiment of the present invention. The encoding / decoding device 9000 includes a CPU 9001, a codec IC 9002, an I / O interface 9003, a memory 9004, an optical disk drive 9005, a network interface 9006, and a video interface 9009, and each part is connected by a bus 9010.

[0224] The image encoding unit 9007 and the image decoding unit 9008 are typically implemented as a codec IC 9002. In the image encoding device according to an embodiment of the present invention, the image encoding process is performed by the image encoding unit 9007, and in the image decoding device according to an embodiment of the present invention, the image decoding process is performed by the image decoding unit 9008. The I / O interface 9003 is implemented, for example, by a USB interface and connects to an external keyboard 9104, mouse 9105, etc. The CPU 9001 controls the encoding / decoding device 9000 to perform operations desired by the user based on user operations input via the I / O interface 9003. User operations via the keyboard 9104, mouse 9105, etc. include selecting whether to perform encoding or decoding, setting the encoding quality, the input / output destination of the bitstream, and the input / output destination of the image.

[0225] When a user desires to play back an image recorded on the disk recording medium 9100, the optical disk drive 9005 reads a bitstream from the inserted disk recording medium 9100 and sends the read bitstream to the image decoding unit 9008 of the codec IC 9002 via the bus 9010. The image decoding unit 9008 performs image decoding processing according to the embodiment of the present invention on the input bitstream and sends the decoded image to an external monitor 9103 via the video interface 9009. The encoding / decoding device 9000 also has a network interface 9006 and can connect to an external distribution server 9106 or a mobile terminal 9107 via the network 9101. When a user desires to play back an image recorded on the distribution server 9106 or a mobile terminal 9107 instead of an image recorded on the disk recording medium 9100, the network interface 9006 acquires a bitstream from the network 9101 instead of reading a bitstream from the input disk recording medium 9100. Furthermore, if the user wishes to play back an image recorded in memory 9004, the image decoding process in the image decoding device according to the embodiment of the present invention is performed on the bitstream recorded in memory 9004.

[0226] When a user desires to encode an image captured by an external camera 9102 and record it in memory 9004, the video interface 9009 receives the image from the camera 9102 and sends it via bus 9010 to the image encoding unit 9007 of the codec IC 9002. The image encoding unit 9007 performs image encoding processing according to an embodiment of the present invention on the image input via the video interface 9009 and creates a bitstream. The bitstream is then sent to memory 9004 via bus 9010. When a user desires to record the bitstream on a disk recording medium 9100 instead of memory 9004, the optical disk drive 9005 writes the bitstream to the inserted disk recording medium 9100.

[0227] It is also possible to implement hardware configurations that have an image encoding device but no image decoding device, or hardware configurations that have an image decoding device but no image encoding device. Such hardware configurations can be realized, for example, by replacing the codec IC 9002 with the image encoding unit 9007 or the image decoding unit 9008, respectively.

[0228] The above encoding and decoding processes may be implemented not only as hardware-based transmission, storage, and receiving devices, but also by firmware stored in ROM (read-only memory) or flash memory, or by software on a computer. The firmware program or software program may be recorded on a recording medium readable by a computer and provided, provided from a server via a wired or wireless network, or provided as data broadcasting on terrestrial or satellite digital broadcasting.

[0229] The present invention has been described above based on embodiments. The embodiments are illustrative, and it will be understood by those skilled in the art that various modifications are possible in combinations of their components and processing processes, and that such modifications also fall within the scope of the present invention.

[0230] 100 Image encoding device, 101 Block division unit, 102 Interpretation unit, 103 Intraprediction unit, 104 Decoded image memory, 105 Prediction method determination unit, 106 Residual generation unit, 107 Orthogonal transformation / quantization unit, 108 Bit sequence encoding unit, 109 Inverse quantization / inverse orthogonal transformation unit, 110 Decoded image signal superposition unit, 111 Encoded information storage memory, 200 Image decoding device, 201 Bit sequence decoding unit, 202 Block division unit, 203 Interpretation unit, 204 Intraprediction unit, 205 Encoded information storage memory, 206 Inverse quantization / inverse orthogonal transformation unit, 207 Decoded image signal superposition unit, 208 Decoded image memory.

Claims

1. An image decoding device comprising: a spatial candidate derivation unit that derives spatial candidates from inter prediction information of blocks adjacent to the block to be decoded and registers them as candidates in a first candidate list; a history candidate derivation unit that adds history candidates included in a history candidate list to the first candidate list as candidates to create a second candidate list; a candidate selection unit that selects a selection candidate from the candidates included in the second candidate list; and an inter prediction unit that performs inter prediction using the selected candidates, wherein the history candidate derivation unit switches whether or not to add history candidates that overlap with candidates included in the first candidate list depending on the prediction mode.

2. An image decoding device as described in claim 1, wherein the prediction mode is a merge mode and a predicted motion vector mode, and when the prediction mode is a merge mode, the candidate is motion information, and when the prediction mode is a predicted motion vector mode, the candidate is a motion vector.

3. The image decoding device of claim 2, wherein the history candidate derivation unit adds the history candidate to the first candidate list as a candidate if the history candidate does not overlap with a candidate included in the first candidate list when the prediction mode is a merge mode, and adds the history candidate to the first candidate list as a candidate when the prediction mode is a predicted motion vector mode, regardless of whether the history candidate overlaps with a candidate included in the first candidate list.

4. An image decoding device as described in claims 2 or 3, further comprising a history prediction motion vector candidate list update unit that, when the prediction mode is merge mode, updates the history prediction motion vector candidate list with the selected candidate so that the history prediction motion vector candidate list does not include overlapping candidates, and, when the prediction mode is predicted motion vector mode, updates the history prediction motion vector candidate list with motion information including at least the selected candidate and a reference index indicating a picture referenced by the selected candidate so that the history prediction motion vector candidate list does not include overlapping candidates.

5. An image decoding device as described in claims 2 to 4. The maximum number of candidates included in the candidate list when the prediction mode is merge mode is greater than the maximum number of candidates included in the candidate list when the prediction mode is predicted motion vector mode.

6. An image decoding device as described in claims 2 to 5, wherein the history prediction motion vector candidate derivation unit adds the history candidate to the first candidate list as a candidate when the prediction mode is a prediction motion vector mode and the reference index of the history candidate is the same as the reference index of the picture to be decoded, regardless of whether the history candidate overlaps with a candidate included in the first candidate list.

7. An image decoding method in an image decoding device, comprising: a step of deriving spatial candidates from inter prediction information of blocks adjacent to a block to be decoded and registering the derived spatial candidates as candidates in a first candidate list; a step of adding history candidates included in a history candidate list to the first candidate list as candidates to create a second candidate list, wherein the step of switching whether to add history candidates that overlap with candidates included in the first candidate list depending on a prediction mode; a step of selecting a selection candidate from candidates included in the second candidate list; and a step of performing inter prediction using the selected candidates.

8. A computer program stored on a computer-readable non-transitory storage medium for an image decoding device, the computer program comprising instructions to cause a computer of the image decoding device to execute the following steps: deriving spatial candidates from inter prediction information of blocks adjacent to a block to be decoded and registering the derived spatial candidates as candidates in a first candidate list; adding history candidates included in a history candidate list to the first candidate list as candidates to create a second candidate list, wherein the step of switching whether to add history candidates that overlap with candidates included in the first candidate list depending on a prediction mode; selecting a selection candidate from candidates included in the second candidate list; and performing inter prediction using the selected candidate.

9. An image encoding device comprising: a spatial candidate derivation unit that derives spatial candidates from inter prediction information of blocks adjacent to a block to be encoded and registers the candidates in a first candidate list; a history candidate derivation unit that adds history candidates included in a history candidate list as candidates to the first candidate list to create a second candidate list; a candidate selection unit that selects a selection candidate from the candidates included in the second candidate list; and an inter prediction unit that performs inter prediction using the selected candidate, wherein the history candidate derivation unit switches whether to add history candidates that overlap with candidates included in the first candidate list depending on a prediction mode, wherein the prediction mode is a merge mode and a predicted motion vector mode, and when the prediction mode is a merge mode, the candidate is motion information, and when the prediction mode is a predicted motion vector mode, the candidate is a motion vector.

10. The image encoding device of claim 9, wherein the history candidate derivation unit adds the history candidate to the first candidate list as a candidate when the prediction mode is a merge mode and the history candidate does not overlap with a candidate included in the first candidate list, and when the prediction mode is a predicted motion vector mode, adds the history candidate to the first candidate list as a candidate regardless of whether the history candidate overlaps with a candidate included in the first candidate list.

11. An image encoding device as described in claim 9 or 10, further comprising a history prediction motion vector candidate list update unit that, when the prediction mode is merge mode, updates the history prediction motion vector candidate list with the selected candidate so that the history prediction motion vector candidate list does not include overlapping candidates, and, when the prediction mode is predicted motion vector mode, updates the history prediction motion vector candidate list with motion information including at least the selected candidate and a reference index indicating a picture referenced by the selected candidate so that the history prediction motion vector candidate list does not include overlapping candidates.

12. An image encoding device as described in any one of claims 9 to 11, wherein the maximum number of candidates included in the candidate list when the prediction mode is merge mode is greater than the maximum number of candidates included in the candidate list when the prediction mode is predicted motion vector mode.

13. An image encoding device as described in claims 9 to 12, wherein, when the prediction mode is a predicted motion vector mode, if the reference index of the history candidate and the reference index of the picture to be decoded are the same, the history candidate is added to the first candidate list as a candidate regardless of whether the history candidate overlaps with a candidate included in the first candidate list.

14. An image encoding method comprising: a step of deriving spatial candidates from inter prediction information of blocks adjacent to the block to be encoded and registering them as candidates in a first candidate list; a step of adding history candidates included in a history candidate list to the first candidate list as candidates to create a second candidate list, wherein the step of switching whether to add history candidates that overlap with candidates included in the first candidate list depending on the prediction mode; a step of selecting a selection candidate from candidates included in the second candidate list; and a step of performing inter prediction using the selected candidate, wherein the prediction mode is a merge mode and a predicted motion vector mode, and the candidate is motion information when the prediction mode is the merge mode, and the candidate is a motion vector when the prediction mode is the predicted motion vector mode.

15. A computer program stored on a computer-readable non-transitory storage medium for an image encoding device, the computer program comprising: a step of deriving spatial candidates from inter prediction information of blocks adjacent to a current block to be encoded and registering the candidates in a first candidate list; a step of adding history candidates included in a history candidate list to the first candidate list as candidates to create a second candidate list, switching whether to add a history candidate that overlaps with a candidate included in the first candidate list according to a prediction mode, and when the prediction mode is a merge mode, adding the history candidate to the first candidate list as a candidate if the history candidate and a candidate included in the first candidate list do not overlap, the prediction mode being a merge mode and a predictive motion vector mode, the candidate when the prediction mode is the merge mode is motion information, and when the prediction mode is a predictive motion vector mode, the candidate is a motion vector, and when the prediction mode is the predictive motion vector mode, adding the history candidate to the first candidate list as a candidate regardless of whether the history candidate and a candidate included in the first candidate list overlap; a step of selecting a selection candidate from the candidates included in the second candidate list; and a step of performing inter prediction using the selected candidate. a computer program stored on the computer-readable non-transitory storage medium, the computer program comprising instructions for causing a computer of the image encoding device to execute the following: