Video encoding devices, video encoding methods, video encoding programs, video decoding devices, video decoding methods, and video decoding programs.

Abstract

DEPCT6503 / 03 / 2565 Picture coding devices include: unit converters and block options. Vector that transforms the vector block options of the target block in the target image from the information side. The encoding information is stored in memory; the selector selects the blocks. The vector is selected from the vector block selection; and the reference position editor unit is executed. Correction regarding the referenced blocks that are referenced by the selected vector blocks, thus their reference positions. The reference block was modified to refer to the inside of the referenced area, within which the decoded example is located. The target image is searched from image memory that has been decoded according to the block's predicted value. The goal is based on the reference position of the reference block. ----------------------------------------------------------- DEPCT65 Picture coding devices include: unit converters and block options. Vector that transforms the vector block options of the target block in the target image from the information side. The encoding information is stored in memory; the selector selects the blocks. The vector is selected from the vector block selection; and the reference position editor unit is executed. Correction regarding the referenced blocks that are referenced by the selected vector blocks, thus their reference positions. The reference block was modified to refer to the inside of the referenced area, within which the decoded example is located. The target image is searched from image memory that has been decoded according to the block's predicted value. The goal is based on the reference position of the reference block. -----------------------------------------------------------

Description

Video encoding device, video encoding method, video encoding program, video decoding device, video decoding method, and video decoding program

[0001] The present invention relates to an image encoding and decoding technique that divides an image into blocks and performs prediction.

[0002] In image encoding and decoding, the image to be processed is divided into blocks, which are sets of a predetermined number of pixels, and processed on a block-by-block basis. By dividing the image into appropriate blocks and appropriately setting intra-frame prediction (intra-prediction) and inter-frame prediction (inter-prediction), encoding efficiency can be improved.

[0003] Patent Document 1 discloses an intra-prediction technique for obtaining a predicted image using decoded pixels adjacent to a block to be coded and decoded.

[0004] JP 2009-246975 A

[0005] However, the technique of Patent Document 1 uses only decoded pixels adjacent to the block to be coded / decoded for prediction, resulting in poor prediction efficiency.

[0006] In one aspect of the present invention that solves the above problem, the present invention includes a block vector candidate derivation unit that derives block vector candidates for a block to be processed in a picture to be processed from coding information stored in a coding information storage memory, a selection unit that selects a selected block vector from the block vector candidates, and a reference position correction unit that corrects the reference position of a reference block referenced by the selected block vector so that the reference block references the inside of a referenceable area, and based on the reference position of the reference block, a decoded pixel in the picture to be processed is obtained from a decoded image memory as a predicted value of the block to be processed.

[0007] According to the present invention, highly efficient image encoding and decoding processes can be realized with a low load.

[0008] 1. FIG. 1 is a block diagram of an image encoding device according to an embodiment of the present invention. FIG. 2 is a block diagram of an image decoding device according to an embodiment of the present invention. FIG. 3 is a flowchart for explaining an operation of dividing a tree block. FIG. 4 is a diagram showing how an input image is divided into tree blocks. FIG. 5 is a diagram explaining z-scanning. FIG. 6 is a diagram showing a block division shape. FIG. 7 is a diagram showing a block division shape. FIG. 8 is a diagram showing a block division shape. FIG. 9 is a diagram showing a block division shape. FIG. 10 is a flowchart for explaining an operation of dividing a block into four. FIG. 11 is a flowchart for explaining an operation of dividing a block into two or three. FIG. 12 is a syntax for expressing a block division shape. FIG. 13 is a diagram for explaining intra prediction. FIG. 14 is a diagram for explaining intra prediction. FIG. 15 is a diagram for explaining reference blocks for inter prediction. FIG. 16 is a syntax for expressing a coding block prediction mode. FIG. 17 is a syntax for expressing a coding block prediction mode. FIG. 18 is a diagram showing the correspondence between syntax elements and modes related to inter prediction. FIG. 19 is a diagram for explaining affine transformation motion compensation for two control points. FIG. 20 is a diagram for explaining affine transformation motion compensation for three control points. FIG. 21 is a block diagram showing a detailed configuration of the inter prediction unit 102 of FIG. 1. 22. A block diagram of a detailed configuration of the normal predictor motion vector mode derivation unit 301 of Figure 16. A block diagram of a detailed configuration of the normal merge mode derivation unit 302 of Figure 16. A flowchart for explaining normal predictor motion vector mode derivation processing of the normal predictor motion vector mode derivation unit 301 of Figure 16. A flowchart showing the processing procedure of the normal predictor motion vector mode derivation processing. A flowchart for explaining the processing procedure of the normal merge mode derivation processing. A block diagram of a detailed configuration of the inter predictor 203 of Figure 2. A block diagram of a detailed configuration of the normal predictor motion vector mode derivation unit 401 of Figure 22. A block diagram of a detailed configuration of the normal merge mode derivation unit 402 of Figure 22. A flowchart for explaining normal predictor motion vector mode derivation processing of the normal predictor motion vector mode derivation unit 401 of Figure 22. A diagram for explaining the history predictor motion vector candidate list initialization / update processing procedure.1 is a flowchart of a same element confirmation process in a history prediction motion vector candidate list initialization and update process. 2 is a flowchart of an element shift process in a history prediction motion vector candidate list initialization and update process. 3 is a flowchart illustrating a history prediction motion vector candidate derivation process. 4 is a flowchart illustrating a history merge candidate derivation process. 5 is a diagram illustrating an example of a history prediction motion vector candidate list update process. 6 is a diagram illustrating an example of a history prediction motion vector candidate list update process. 7 is a diagram illustrating an example of a history prediction motion vector candidate list update process. 8 is a diagram illustrating motion compensation prediction in L0 prediction where the L0 reference picture (RefL0Pic) is located earlier than the current picture (CurPic). 9 is a diagram illustrating motion compensation prediction in L0 prediction where the reference picture for L0 prediction is located later than the current picture. 10 is a diagram illustrating the prediction direction of motion compensation prediction in bi-prediction where the reference picture for L0 prediction is located earlier than the current picture and the reference picture for L1 prediction is located later than the current picture. 2. A diagram for explaining the prediction direction of motion compensation prediction in bi-prediction where the reference picture for L0 prediction and the reference picture for L1 prediction are located at a time earlier than the current picture. A diagram for explaining the prediction direction of motion compensation prediction in bi-prediction where the reference picture for L0 prediction and the reference picture for L1 prediction are located at a time later than the current picture. A diagram for explaining an example of the hardware configuration of an encoding / decoding device according to an embodiment of the present invention. A flowchart for explaining an average merge candidate derivation process. A diagram for explaining a valid reference area for intra block copying. A diagram for explaining a valid reference area for intra block copying. A block diagram of a detailed configuration of the intra prediction unit 103 of FIG. 1. A block diagram of a detailed configuration of the intra prediction unit 204 of FIG. 2. A block diagram of an intra block copy prediction unit 352. A block diagram of an intra block copy prediction unit 362. A flowchart for explaining the predicted intra block copy process of the intra block copy prediction unit 352. A flowchart for explaining the predicted intra block copy process of the intra block copy prediction unit 362.1 is a flowchart for explaining merge intra block copy processing. FIG. 1 is a flowchart showing the processing steps of block vector mode derivation processing for predictive intra block copy. FIG. 2 is a diagram explaining the processing of the reference position correction unit 380 and the reference position correction unit 480. FIG. 3 is a diagram showing how a reference position is corrected. FIG. 4 is a diagram explaining the positions of the top left and bottom right when the referenceable area is rectangular. FIG. 5 is a diagram explaining the positions of the top left and bottom right when the referenceable area is rectangular. FIG. 6 is a diagram explaining the positions of the top left and bottom right when the referenceable area is rectangular. FIG. 7 is a diagram explaining the positions of the top left and bottom right when the referenceable area is rectangular. FIG. 8 is a diagram explaining processing for correcting the reference position of a portion of the referenceable area that is not rectangular. FIG. 9 is a diagram showing how a reference position is corrected. FIG. 10 is a diagram explaining the processing of the reference position correction unit 380 and the reference position correction unit 480. FIG. 11 is a diagram explaining how a referenceable area is decomposed into two. FIG. 12 is a diagram explaining how a referenceable area is decomposed into two. 10A and 10B are diagrams illustrating how a referenceable area is divided into two parts, and a process for dividing a referenceable area into two parts and correcting the reference positions of each part is described.

[0009] The technologies and technical terms used in this embodiment will be defined below.

[0010] <Tree Block> In this embodiment, the image to be encoded / decoded is equally divided into predetermined units. This unit is defined as a tree block. In FIG. 4, the size of the tree block is 128 x 128 pixels, but the size of the tree block is not limited to this and may be set to any size. The tree blocks to be processed (corresponding to the encoding target in the encoding process and the decoding target in the decoding process) are switched in raster scan order, i.e., from left to right and from top to bottom. Further recursive division within each tree block is possible. The block to be encoded / decoded after recursive division of the tree block is defined as an encoding block. Furthermore, tree blocks and encoding blocks are collectively defined as blocks. Appropriate block division enables efficient encoding. The size of the tree block can be a fixed value predetermined by the encoding device and the decoding device, or the tree block size determined by the encoding device can be transmitted to the decoding device. Here, the maximum size of the tree block is 128 x 128 pixels, and the minimum size of the tree block is 16 x 16 pixels. Also, the maximum size of a coding block is 64x64 pixels, and the minimum size of a coding block is 4x4 pixels.

[0011] <Prediction Mode> Switching is performed for each coding block to be processed between intra prediction (MODE_INTRA), which performs prediction from the processed image signal of the image to be processed, and inter prediction (MODE_INTER), which performs prediction from the image signal of the processed image. In the coding process, the processed image is used as an image, image signal, tree block, block, coding block, etc. obtained by decoded signals for which coding has been completed, and in the decoding process, it is used as an image, image signal, tree block, block, coding block, etc. for which decoding has been completed. A mode that distinguishes between intra prediction (MODE_INTRA) and inter prediction (MODE_INTER) is defined as the prediction mode (PredMode). The prediction mode (PredMode) has the values ​​intra prediction (MODE_INTRA) or inter prediction (MODE_INTER).

[0012] <Intra Block Copy Prediction> Intra block copy prediction is a process that encodes / decodes a current block by referencing decoded pixels in the current picture as predicted values. The distance from the current block to the referenced pixel is represented by a block vector. The block vector references the current picture, and the reference picture is uniquely determined, so no reference index is required. The difference between a block vector and a motion vector is whether the reference picture is the current picture or a previously processed picture. Furthermore, block vectors can be selected with 1-pixel or 4-pixel accuracy using adaptive motion vector resolution (AMVR). Intra block copy can be performed in two modes: predicted intra block copy mode and merged intra block copy mode. The predicted intra block copy mode determines the block vector of the current block from a predicted block vector derived from processed information and a differential block vector. The predicted block vector is derived from previously processed blocks adjacent to the current block and an index for identifying the predicted block vector. The index for identifying the prediction block vector and the differential block vector are transmitted in the bitstream. The merge intra block copy mode is a mode in which the intra block copy prediction information of the current block is derived from the intra block copy prediction information of a processed block adjacent to the current block, without transmitting the differential motion vector.

[0013] <Inter Prediction> In inter prediction, which performs prediction from the image signal of a processed image, multiple processed images can be used as reference pictures. To manage multiple reference pictures, two types of reference lists, L0 (reference list 0) and L1 (reference list 1), are defined, and each uses a reference index to identify a reference picture. L0 prediction (Pred_L0) is available for P slices. L0 prediction (Pred_L0), L1 prediction (Pred_L1), and bi-prediction (Pred_BI) are available for B slices. L0 prediction (Pred_L0) is inter prediction that references a reference picture managed by L0, while L1 prediction (Pred_L1) is inter prediction that references a reference picture managed by L1. Bi-prediction (Pred_BI) is inter prediction that performs both L0 prediction and L1 prediction, and references one reference picture managed by L0 and one reference picture managed by L1. Information specifying L0 prediction, L1 prediction, or bi-prediction is defined as the inter prediction mode. In the following processing, it is assumed that the constants and variables with the subscript LX in the output are processed for each of L0 and L1.

[0014] <Predicted motion vector mode> The predicted motion vector mode is a mode in which an index for identifying a predicted motion vector, a differential motion vector, an inter prediction mode, and a reference index are transmitted to determine inter prediction information for a block to be processed. The predicted motion vector is derived from a candidate predicted motion vector derived from a processed block adjacent to the block to be processed, or a block belonging to a processed image that is located at the same position as the block to be processed or in the vicinity (vicinity) thereof, and an index for identifying the predicted motion vector.

[0015] <Merge mode> Merge mode is a mode in which inter-prediction information for the block to be processed is derived from inter-prediction information for a processed block adjacent to the block to be processed, or a block belonging to a processed image that is located at the same position as the block to be processed or in the vicinity (vicinity) thereof, without transmitting a differential motion vector or a reference index.

[0016] Spatial merge candidates are defined as processed blocks adjacent to the current block and their inter-prediction information. Temporal merge candidates are defined as blocks in the processed image that are located at or near the same position as the current block and their inter-prediction information. Each merge candidate is registered in a merge candidate list, and a merge index identifies the merge candidate to be used in predicting the current block.

[0017] 11 is a diagram illustrating reference blocks referenced to derive inter prediction information in predicted motion vector mode and merge mode. A0, A1, A2, B0, B1, B2, and B3 are processed blocks adjacent to the target block. T0 is a block belonging to a processed image, and is located at the same position as the target block in the target image or in the vicinity (vicinity).

[0018] A1 and A2 are blocks located to the left of the current coding block and adjacent to the current coding block. B1 and B3 are blocks located above the current coding block and adjacent to the current coding block. A0, B0, and B2 are blocks located at the bottom left, top right, and top left of the current coding block, respectively.

[0019] How adjacent blocks are handled in the predicted motion vector mode and merge mode will be described in detail later.

[0020] <Affine Transform Motion Compensation> Affine transform motion compensation involves dividing a coding block into sub-blocks of a predetermined unit, determining a motion vector for each divided sub-block individually, and performing motion compensation. The motion vector for each sub-block is derived based on one or more control points derived from inter-prediction information of a processed block adjacent to the block to be processed, or a block belonging to the processed image that is located at the same position as the block to be processed or in the vicinity (vicinity). In this embodiment, the size of the sub-block is 4x4 pixels, but the size of the sub-block is not limited to this, and the motion vector may be derived on a pixel-by-pixel basis.

[0021] FIG. 14 shows an example of affine transformation motion compensation when there are two control points. In this case, the two control points have two parameters, a horizontal component and a vertical component. Therefore, the affine transformation when there are two control points is called a four-parameter affine transformation. CP1 and CP2 in FIG. 14 are the control points. FIG. 15 shows an example of affine transformation motion compensation when there are three control points. In this case, the three control points have two parameters, a horizontal component and a vertical component. Therefore, the affine transformation when there are three control points is called a six-parameter affine transformation. CP1, CP2, and CP3 in FIG. 15 are the control points.

[0022] Affine transform motion compensation can be used in both the predictive motion vector mode and the merge mode. The mode in which affine transform motion compensation is applied in the predictive motion vector mode is defined as a sub-block predictive motion vector mode, and the mode in which affine transform motion compensation is applied in the merge mode is defined as a sub-block merge mode.

[0023] <Syntax of Coding Block> The syntax for representing the prediction mode of a coding block will be described using Figures 12A, 12B, and 13. pred_mode_flag in Figure 12A is a flag indicating whether or not inter prediction is used. If pred_mode_flag is 0, it is inter prediction, and if pred_mode_flag is 1, it is intra prediction. In the case of intra prediction, pred_mode_ibc_flag, a flag indicating whether intra block copy prediction is used, is sent. If intra block copy prediction is used (pred_mode_ibc_flag = 1), merge_flag is sent. merge_flag is a flag indicating whether merge intra block copy mode or predicted intra block copy mode is used. If merge intra block copy mode is used (merge_flag = 1), a merge index merge_idx is sent. If intra block copy prediction is not used (pred_mode_ibc_flag = 0), normal intra prediction is used, and normal intra prediction information intra_pred_mode is sent. In the case of inter prediction, merge_flag is sent. merge_flag is a flag indicating whether to use merge mode or predicted motion vector mode. In the case of predicted motion vector mode (merge_flag=0), a flag inter_affine_flag indicating whether to apply sub-block predicted motion vector mode is sent. In the case of sub-block predicted motion vector mode (inter_affine_flag=1), cu_affine_type_flag is sent. cu_affine_type_flag is a flag for determining the number of control points in sub-block predicted motion vector mode. On the other hand, in the case of merge mode (merge_flag=1), merge_subblock_flag of FIG. 12B is sent. merge_subblock_flag is a flag indicating whether to apply sub-block merge mode. In the case of sub-block merge mode (merge_subblock_flag=1), a merge index merge_subblock_idx is sent.On the other hand, if the subblock merge mode is not used (merge_subblock_flag=0), a flag "merge_triangle_flag" indicating whether or not to apply the triangle merge mode is sent. If the triangle merge mode is used (merge_triangle_flag=1), a block split direction "merge_triangle_split_dir" and merge triangle indices "merge_triangle_idx0" and "merge_triangle_idx1" for each of the two partitions are sent. On the other hand, if the triangle merge mode is not used (merge_triangle_flag=0), a merge index "merge_idx" is sent. Figure 13 shows the values ​​of each syntax element for inter prediction and the corresponding prediction mode. merge_flag=0 and inter_affine_flag=0 correspond to normal prediction motion vector mode (Inter Pred Mode). merge_flag=0 and inter_affine_flag=1 correspond to subblock prediction motion vector mode (Inter Affine Mode). merge_flag=1,merge_subblock_flag=0,merge_trianlge_flag=0 corresponds to normal merge mode (Merge Mode). merge_flag=1,merge_subblock_flag=0,merge_trianlge_flag=1 corresponds to triangle merge mode (Triangle Merge Mode). merge_flag=1,merge_subblock_flag=1 corresponds to subblock merge mode (Affine Merge Mode).

[0024] <POC> POC (Picture Order Count) is a variable associated with a picture to be coded, and is set to a value that increases by one according to the picture's output order. The POC value can be used to determine whether the pictures are the same, determine the order of pictures in the output order, and derive the distance between pictures. For example, if two pictures have the same POC value, they can be determined to be the same picture. If two pictures have different POC values, the picture with the smaller POC value can be determined to be the picture that will be output first, and the difference between the POCs of the two pictures indicates the distance between the pictures along the time axis.

[0025] First Embodiment An image encoding device 100 and an image decoding device 200 according to a first embodiment of the present invention will be described.

[0026] 1 is a block diagram of an image coding device 100 according to a first embodiment. The image coding device 100 according to the embodiment includes a block division unit 101, an inter prediction unit 102, an intra prediction unit 103, a decoded image memory 104, a prediction method determination unit 105, a residual generation unit 106, an orthogonal transform and quantization unit 107, a bit string coding unit 108, an inverse quantization and inverse orthogonal transform unit 109, a decoded image signal superimposition unit 110, and a coding information storage memory 111.

[0027] The block division unit 101 recursively divides an input image to generate coding blocks. The block division unit 101 includes a 4-way division unit that divides the block to be divided horizontally and vertically, and a 2-3-way division unit that divides the block to be divided either horizontally or vertically. The block division unit 101 sets the generated coding block as a coding block to be processed, and supplies an image signal of the coding block to be processed to the inter prediction unit 102, the intra prediction unit 103, and the residual generation unit 106. The block division unit 101 also supplies information indicating the determined recursive division structure to the bit string coding unit 108. The detailed operation of the block division unit 101 will be described later.

[0028] The inter prediction unit 102 performs inter prediction on the coding block to be processed. The inter prediction unit 102 derives multiple inter prediction information candidates from the inter prediction information stored in the coding information storage memory 111 and the decoded image signal stored in the decoded image memory 104, selects an appropriate inter prediction mode from the multiple derived candidates, and supplies the selected inter prediction mode and a predicted image signal corresponding to the selected inter prediction mode to the prediction method determination unit 105. The detailed configuration and operation of the inter prediction unit 102 will be described later.

[0029] The intra prediction unit 103 performs intra prediction of the coding block to be processed. The intra prediction unit 103 references the decoded image signal stored in the decoded image memory 104 as reference pixels, and generates a predicted image signal by intra prediction based on coding information such as an intra prediction mode stored in the coding information storage memory 111. In intra prediction, the intra prediction unit 103 selects an appropriate intra prediction mode from a plurality of intra prediction modes, and supplies the selected intra prediction mode and a predicted image signal corresponding to the selected intra prediction mode to the prediction method determination unit 105. The detailed configuration and operation of the intra prediction unit 103 will be described later.

[0030] The decoded image memory 104 stores the decoded image generated by the decoded image signal superimposing unit 110. The decoded image memory 104 supplies the stored decoded image to the inter prediction unit 102 and the intra prediction unit 103.

[0031] The prediction method determination unit 105 determines the optimal prediction mode for each of intra prediction and inter prediction by evaluating the coding information, the coding amount of residuals, the amount of distortion between the predicted image signal and the image signal to be processed, etc. In the case of intra prediction, the prediction method determination unit 105 supplies intra prediction information such as the intra prediction mode as coding information to the bitstream coding unit 108. In the case of inter prediction merge mode, the prediction method determination unit 105 supplies inter prediction information such as a merge index and information indicating whether or not a sub-block merge mode (sub-block merge flag) as coding information to the bitstream coding unit 108. In the case of inter prediction predicted motion vector mode, the prediction method determination unit 105 supplies inter prediction information such as the inter prediction mode, predicted motion vector index, L0 and L1 reference indexes, differential motion vectors, and information indicating whether or not a sub-block predicted motion vector mode (sub-block predicted motion vector flag) as coding information to the bitstream coding unit 108. Furthermore, the prediction method determination unit 105 supplies the determined coding information to the coding information storage memory 111. The prediction method determination unit 105 supplies the residual generation unit 106 and the predicted image signal to the decoded image signal superimposition unit 110 .

[0032] The residual generation unit 106 generates a residual by subtracting the predicted image signal from the image signal to be processed, and supplies the residual to the orthogonal transformation and quantization unit 107 .

[0033] The orthogonal transform / quantization unit 107 performs orthogonal transform and quantization on the residual in accordance with the quantization parameter to generate an orthogonally transformed / quantized residual, and supplies the generated residual to the bit string coding unit 108 and the inverse quantization / inverse orthogonal transform unit 109.

[0034] The bitstream encoding unit 108 encodes encoding information according to the prediction method determined by the prediction method determination unit 105 for each coding block, in addition to information for each sequence, picture, slice, and coding block. Specifically, the bitstream encoding unit 108 encodes the prediction mode PredMode for each coding block. When the prediction mode is inter prediction (MODE_INTER), the bitstream encoding unit 108 encodes encoding information (inter prediction information) such as a flag for determining whether or not the prediction mode is a merge mode, a sub-block merge flag, a merge index for the merge mode, an inter prediction mode for the merge mode not, a predicted motion vector index, information about the differential motion vector, and a sub-block predicted motion vector flag according to a specified syntax (bitstream syntax rules) to generate a first bitstream. When the prediction mode is intra prediction (MODE_INTRA), the bitstream encoding unit 108 encodes a flag for determining whether or not the prediction mode is an intra block copy according to a specified syntax. In the case of intra block copy, coding information (intra prediction information) such as a merge index if the merge mode is selected, or a predicted block vector index and a differential block vector if the merge mode is not selected, is coded according to a specified syntax. If the intra block copy is not selected, coding information (intra prediction information) such as an intra prediction mode is coded according to a specified syntax. A first bitstream is generated by the above coding. Furthermore, the bitstream coding unit 108 entropy codes the orthogonally transformed and quantized residual according to a specified syntax to generate a second bitstream. The bitstream coding unit 108 multiplexes the first bitstream and the second bitstream according to the specified syntax and outputs a bitstream.

[0035] The inverse quantization and inverse orthogonal transform unit 109 inverse quantizes and inverse orthogonal transforms the orthogonally transformed and quantized residual supplied from the orthogonal transform and quantization unit 107 to calculate a residual, and supplies the calculated residual to the decoded image signal superimposition unit 110.

[0036] The decoded image signal superimposing unit 110 generates a decoded image by superimposing the predicted image signal determined by the prediction method determining unit 105 and the residual that has been inversely quantized and inversely orthogonal transformed by the inverse quantization and inverse orthogonal transform unit 109, and stores the decoded image in the decoded image memory 104. Note that the decoded image signal superimposing unit 110 may store the decoded image in the decoded image memory 104 after performing a filtering process on the decoded image to reduce distortions such as block distortions caused by encoding.

[0037] The coding information storage memory 111 stores coding information such as the prediction mode (inter prediction or intra prediction) determined by the prediction method determination unit 105. In the case of inter prediction, the coding information stored in the coding information storage memory 111 includes inter prediction information such as the determined motion vector, reference indexes of reference lists L0 and L1, and a historical predicted motion vector candidate list. In the case of inter prediction merge mode, the coding information stored in the coding information storage memory 111 includes, in addition to the above-mentioned information, inter prediction information such as a merge index and information indicating whether or not a sub-block merge mode is in effect (a sub-block merge flag). In the case of inter prediction predicted motion vector mode, the coding information stored in the coding information storage memory 111 includes, in addition to the above-mentioned information, inter prediction information such as an inter prediction mode, a predicted motion vector index, a differential motion vector, and information indicating whether or not a sub-block predicted motion vector mode is in effect (a sub-block predicted motion vector flag). In the case of intra prediction, the coding information stored in the coding information storage memory 111 includes, in addition to the above-mentioned information, intra prediction information such as the determined intra prediction mode.

[0038] Fig. 2 is a block diagram showing the configuration of an image decoding device according to an embodiment of the present invention, which corresponds to the image encoding device in Fig. 1. The image decoding device according to the embodiment includes a bitstream decoding unit 201, a block dividing unit 202, an inter prediction unit 203, an intra prediction unit 204, an encoding information storage memory 205, an inverse quantization and inverse orthogonal transform unit 206, a decoded image signal superimposition unit 207, and a decoded image memory 208.

[0039] The decoding process of the image decoding device of Figure 2 corresponds to the decoding process provided inside the image coding device of Figure 1, and therefore each of the components of the coding information storage memory 205, inverse quantization and inverse orthogonal transform unit 206, decoded image signal superimposition unit 207, and decoded image memory 208 of Figure 2 has functions corresponding to each of the components of the coding information storage memory 111, inverse quantization and inverse orthogonal transform unit 109, decoded image signal superimposition unit 110, and decoded image memory 104 of the image coding device of Figure 1.

[0040] The bitstream supplied to the bitstream decoding unit 201 is separated according to a specified syntax rule. The bitstream decoding unit 201 decodes the separated first bitstream to obtain information on a sequence, picture, slice, and coding block basis, and coding information on a coding block basis. Specifically, the bitstream decoding unit 201 decodes a prediction mode PredMode that determines whether the prediction mode is inter prediction (MODE_INTER) or intra prediction (MODE_INTRA) on a coding block basis. When the prediction mode is inter prediction (MODE_INTER), the bitstream decoding unit 201 decodes coding information (inter prediction information) related to a flag determining whether the prediction mode is a merge mode, a merge index and a sub-block merge flag in the merge mode, and an inter prediction mode, a predicted motion vector index, a differential motion vector, and a sub-block predicted motion vector flag in the predicted motion vector mode, according to a specified syntax, and supplies the coding information (inter prediction information) to the inter prediction unit 203 and the coding information storage memory 205 via the block division unit 202. If the prediction mode is intra prediction (MODE_INTRA), the bitstream decoding unit 201 decodes a flag that determines whether or not the prediction mode is intra block copy. In the case of intra block copy, the bitstream decoding unit 201 decodes coding information (intra prediction information) such as a merge index if the merge mode is selected, or a predicted block vector index and a difference block vector if the merge mode is not selected, according to a specified syntax. If the block copy is not intra block copy, the bitstream decoding unit 201 decodes coding information (intra prediction information) such as an intra prediction mode, according to a specified syntax. Through the above decoding, the coding information (intra prediction information) is supplied to the inter prediction unit 203 or the intra prediction unit 204 and to the coding information storage memory 205 via the block division unit 202. The bitstream decoding unit 201 decodes the separated second bitstream to calculate an orthogonally transformed and quantized residual, and supplies the orthogonally transformed and quantized residual to the inverse quantization and inverse orthogonal transformation unit 206.

[0041] When the prediction mode PredMode of the coding block to be processed is inter prediction (MODE_INTER) and the predicted motion vector mode, the inter prediction unit 203 derives multiple candidate predicted motion vectors using coding information of an already decoded image signal stored in the coding information storage memory 205, and registers the derived multiple candidate predicted motion vectors in a candidate predicted motion vector list (described later). The inter prediction unit 203 selects a predictive motion vector from the multiple candidate predicted motion vectors registered in the candidate predicted motion vector list according to a predictive motion vector index decoded and supplied by the bitstream decoding unit 201, calculates a motion vector from the differential motion vector decoded by the bitstream decoding unit 201 and the selected predictive motion vector, and stores the calculated motion vector in the coding information storage memory 205 together with other coding information. The coding information of the coding block supplied and stored here includes a prediction mode PredMode, flags predFlagL0[xP][yP] and predFlagL1[xP][yP] indicating whether to use L0 prediction and L1 prediction, L0 and L1 reference indices refIdxL0[xP][yP] and refIdxL1[xP][yP], and L0 and L1 motion vectors mvL0[xP][yP] and mvL1[xP][yP]. Here, xP and yP are indices indicating the position of the upper left pixel of the coding block within a picture. When the prediction mode PredMode is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), the flag predFlagL0 indicating whether to use L0 prediction is 1, and the flag predFlagL1 indicating whether to use L1 prediction is 0. When the inter prediction mode is L1 prediction (Pred_L1), the flag predFlagL0 indicating whether or not to use L0 prediction is 0, and the flag predFlagL1 indicating whether or not to use L1 prediction is 1. When the inter prediction mode is bi-prediction (Pred_BI), the flag predFlagL0 indicating whether or not to use L0 prediction and the flag predFlagL1 indicating whether or not to use L1 prediction are both 1. Furthermore, when the prediction mode PredMode of the coding block to be processed is inter prediction (MODE_INTER) and a merge mode, merge candidates are derived.Using the coding information of already decoded coding blocks stored in the coding information storage memory 205, multiple merge candidates are derived and registered in a merge candidate list (described later). From the multiple merge candidates registered in the merge candidate list, a merge candidate corresponding to a merge index decoded and supplied by the bitstream decoding unit 201 is selected. Inter-prediction information, such as flags predFlagL0[xP][yP] and predFlagL1[xP][yP] indicating whether L0 prediction and L1 prediction of the selected merge candidate are to be used, L0 and L1 reference indices refIdxL0[xP][yP] and refIdxL1[xP][yP], and L0 and L1 motion vectors mvL0[xP][yP] and mvL1[xP][yP], are stored in the coding information storage memory 205. Here, xP and yP are indices indicating the position of the upper left pixel of the coding block within a picture. The detailed configuration and operation of the inter prediction unit 203 will be described later.

[0042] The intra prediction unit 204 performs intra prediction when the prediction mode PredMode of the coding block to be processed is intra prediction (MODE_INTRA). The coding information decoded by the bitstream decoding unit 201 includes an intra prediction mode. The intra prediction unit 204 generates a predicted image signal by intra prediction from the decoded image signal stored in the decoded image memory 208, according to the intra prediction mode included in the coding information decoded by the bitstream decoding unit 201, and supplies the generated predicted image signal to the decoded image signal superimposition unit 207. The intra prediction unit 204 corresponds to the intra prediction unit 103 of the image coding device 100, and therefore performs the same processing as the intra prediction unit 103.

[0043] The inverse quantization and inverse orthogonal transformation unit 206 performs inverse orthogonal transformation and inverse quantization on the orthogonally transformed and quantized residual decoded by the bit stream decoding unit 201, thereby obtaining an inverse orthogonally transformed and inverse quantized residual.

[0044] The decoded image signal superimposing unit 207 decodes the decoded image signal by superimposing the predicted image signal inter-predicted by the inter prediction unit 203 or the predicted image signal intra-predicted by the intra prediction unit 204 on the residual that has been inverse orthogonally transformed and inverse quantized by the inverse quantization and inverse orthogonal transform unit 206, and stores the decoded decoded image signal in the decoded image memory 208. When storing the decoded image in the decoded image memory 208, the decoded image signal superimposing unit 207 may perform a filtering process on the decoded image to reduce block artifacts and the like caused by encoding, and then store the decoded image in the decoded image memory 208.

[0045] Next, the operation of the block division unit 101 in the image encoding device 100 will be described. Fig. 3 is a flowchart showing the operation of dividing an image into tree blocks and further dividing each tree block. First, the input image is divided into tree blocks of a predetermined size (step S1001). Each tree block is scanned in a predetermined order, i.e., raster scan order (step S1002), and the inside of the tree block to be processed is divided (step S1003).

[0046] 7 is a flowchart showing the detailed operation of the division process in step S1003. First, it is determined whether or not the block to be processed is to be divided into four (step S1101).

[0047] If it is determined that the processing target block should be divided into four, the processing target block is divided into four (step S1102). Each divided block of the processing target block is scanned in Z scan order, i.e., upper left, upper right, lower left, and lower right (step S1103). FIG. 5 shows an example of Z scan order, and 601 in FIG. 6A is an example of a processing target block divided into four. Numbers 0 to 3 in 601 in FIG. 6A indicate the order of processing. Then, the division process of FIG. 7 is recursively performed for each block divided in step S1101 (step S1104).

[0048] If it is determined that the block to be processed is not to be divided into four, it is divided into two or three (step S1105).

[0049] 8 is a flowchart showing the detailed operation of the 2-3 division process in step S1105. First, it is determined whether the block to be processed is to be divided into 2-3, that is, whether to divide into 2 or 3 (step S1201).

[0050] If it is not determined that the block to be processed should be divided into 2 or 3 parts, i.e., if it is determined that no division should be made, the division is terminated (step S1211). In other words, no further recursive division is made on the blocks that have been divided by the recursive division process.

[0051] If it is determined that the block to be processed should be divided into 2 or 3, it is further determined whether or not the block to be processed should be divided into 2 (step S1202).

[0052] If it is determined that the processing target block should be divided into two, it is determined whether or not to divide the processing target block vertically (step S1203), and based on the result, the processing target block is divided into two vertically (step S1204) or divided horizontally (step S1205). As a result of step S1204, the processing target block is divided into two vertically (as shown in 602 of Fig. 6B), and as a result of step S1205, the processing target block is divided into two horizontally (as shown in 604 of Fig. 6D).

[0053] If it is not determined in step S1202 that the block to be processed should be divided into two, i.e., if it is determined that the block to be processed should be divided into three, it is determined whether to divide the block to be processed into top, middle, and bottom (vertical) (step S1206), and based on the result, the block to be processed is divided into three parts into top, middle, and bottom (vertical) (step S1207) or left, middle, and right (horizontal) (step S1208). As a result of step S1207, the block to be processed is divided into three parts into top, middle, and bottom (vertical) as shown in 603 of Fig. 6C, and as a result of step S1208, the block to be processed is divided into three parts into left, middle, and right (horizontal) as shown in 605 of Fig. 6E.

[0054] After executing any one of steps S1204, S1205, S1207, and S1208, the target block is divided and each divided block is scanned from left to right and top to bottom (step S1209). The numbers 0 to 2 in 602 to 605 in Figures 6B to 6E indicate the order of processing. For each divided block, the 2-3 division process in Figure 8 is recursively executed (step S1210).

[0055] The recursive block division described here may limit the necessity of division depending on the number of divisions, the size of the block to be processed, etc. The information limiting the necessity of division may be realized by a configuration in which no information is transmitted by making a prior agreement between the encoding device and the decoding device, or may be realized by a configuration in which the encoding device determines the information limiting the necessity of division and records it in a bit string to transmit it to the decoding device.

[0056] When a block is divided, the block before the division is called a parent block, and each block after the division is called a child block.

[0057] Next, the operation of the block division unit 202 in the image decoding device 200 will be described. The block division unit 202 divides tree blocks using the same processing procedure as the block division unit 101 in the image encoding device 100. However, the difference is that the block division unit 101 in the image encoding device 100 determines the optimal block division shape by applying optimization techniques such as optimal shape estimation through image recognition and distortion rate optimization, whereas the block division unit 202 in the image decoding device 200 determines the block division shape by decoding block division information recorded in a bit string.

[0058] The syntax (bitstream syntax rules) for block division in the first embodiment is shown in Figure 9. coding_quadtree() represents the syntax for dividing a block into four. multi_type_tree() represents the syntax for dividing a block into two or three. qt_split is a flag indicating whether to divide a block into four. If the block is to be divided into four, set qt_split=1; if not, set qt_split=0. If the block is to be divided into four (qt_split=1), each block is recursively divided into four (coding_quadtree(0), coding_quadtree(1), coding_quadtree(2), coding_quadtree(3); arguments 0 to 3 correspond to number 601 in Figure 6A). If the block is not to be divided into four (qt_split=0), subsequent divisions are determined according to multi_type_tree(). mtt_split is a flag indicating whether further division is to be performed. If further division is required (mtt_split=1), mtt_split_vertical, a flag indicating whether to divide vertically or horizontally, and mtt_split_binary, a flag determining whether to divide into two or three, are transmitted. mtt_split_vertical=1 indicates vertical division, and mtt_split_vertical=0 indicates horizontal division. mtt_split_binary=1 indicates division into two, and mtt_split_binary=0 indicates division into three. If division into two is required (mtt_split_binary=1), recursive division processing is performed on each of the two divided blocks (multi_type_tree(0), multi_type_tree(1); the arguments 0 to 1 correspond to numbers 602 or 604 in Figures 6B to 6D). When dividing into three blocks (mtt_split_binary=0), the division process is performed recursively for each of the three divided blocks (multi_type_tree(0), multi_type_tree(1), multi_type_tree(2), where 0 to 2 correspond to numbers 603 in Figure 6B or 605 in Figure 6E).Hierarchical block division is performed by recursively calling multi_type_tree until mtt_split=0.

[0059] <Intra Prediction> The intra prediction method according to the embodiment is implemented in the intra prediction unit 103 of the image encoding device 100 in FIG. 1 and the intra prediction unit 204 of the image decoding device 200 in FIG. 2. The intra prediction method according to the embodiment will be described using the drawings. The intra prediction method is implemented in both encoding and decoding processes on a coding block-by-coding block basis. <Explanation of the Intra Prediction Unit 103 on the Encoding Side> FIG. 40 is a diagram showing a detailed configuration of the intra prediction unit 103 of the image encoding device 100 in FIG. 1. The normal intra prediction unit 351 generates a predicted image signal using normal intra prediction from decoded pixels adjacent to the coding block to be processed, selects an appropriate intra prediction mode from multiple intra prediction modes, and supplies the selected intra prediction mode and a predicted image signal corresponding to the selected intra prediction mode to the prediction method determination unit 105. FIGS. 10A and 10B show examples of intra prediction. FIG. 10A shows the correspondence between the prediction direction of normal intra prediction and the intra prediction mode number. For example, intra prediction mode 50 generates an intra predicted image by copying pixels vertically. Intra prediction mode 1 is DC mode, in which all pixel values ​​of the current block are set to the average value of reference pixels. Intra prediction mode 0 is Planar mode, in which a two-dimensional intra predicted image is created from reference pixels in the vertical and horizontal directions. FIG. 10B shows an example of generating an intra predicted image in intra prediction mode 40. For each pixel of the current block, the value of the reference pixel in the direction indicated by the intra prediction mode is copied. If the reference pixel in the intra prediction mode is not an integer position, the reference pixel value is determined by interpolation from the reference pixel values ​​of surrounding integer positions. The intra block copy prediction unit 352 obtains a decoded area of ​​the image signal that is the same as the current coding block from the decoded image memory 104, generates a predicted image signal through intra block copy processing, and supplies it to the prediction method determination unit 105. The detailed configuration and processing of the intra block copy prediction unit 352 will be described later. <Description of the Intra Prediction Unit 204 on the Decoding Side> FIG. 41 is a diagram showing a detailed configuration of the intra prediction unit 204 of the image decoding device 200 in FIG.The normal intra prediction unit 361 generates a predicted image signal by normal intra prediction from decoded pixels adjacent to the current coding block, selects an appropriate intra prediction mode from among multiple intra prediction modes, and obtains the selected intra prediction mode and a predicted image signal corresponding to the selected intra prediction mode. This predicted image signal is supplied to the decoded image signal superimposition unit 207 via a switch 364. The processing of the normal intra prediction unit 361 in Figure 41 corresponds to the normal intra prediction unit 351 in Figure 40, and therefore detailed description will be omitted. The intra block copy prediction unit 362 obtains a decoded region of the image signal that is the same as the current coding block from the decoded image memory 208, and obtains a predicted image signal by intra block copy processing. This predicted image signal is supplied to the decoded image signal superimposition unit 207 via the switch 364. The detailed configuration and processing of the intra block copy prediction unit 362 will be described later.

[0060] <Inter Prediction> The inter prediction method according to the embodiment is performed in the inter prediction unit 102 of the image encoding device in FIG. 1 and the inter prediction unit 203 of the image decoding device in FIG.

[0061] The inter prediction method according to the embodiment will be described with reference to the drawings. The inter prediction method is performed in both encoding and decoding processes in units of coding blocks.

[0062] <Explanation of the inter prediction unit 102 on the encoding side> Figure 16 is a diagram showing a detailed configuration of the inter prediction unit 102 of the image encoding device of Figure 1. The normal predicted motion vector mode derivation unit 301 derives a plurality of normal predicted motion vector candidates, selects a predicted motion vector, and calculates a differential motion vector between the selected predicted motion vector and the detected motion vector. The detected inter prediction mode, reference index, motion vector, and calculated differential motion vector become inter prediction information for the normal predicted motion vector mode. This inter prediction information is supplied to the inter prediction mode determination unit 305. The detailed configuration and processing of the normal predicted motion vector mode derivation unit 301 will be described later.

[0063] The normal merge mode derivation unit 302 derives multiple normal merge candidates, selects a normal merge candidate, and obtains inter prediction information for the normal merge mode. This inter prediction information is supplied to the inter prediction mode determination unit 305. The detailed configuration and processing of the normal merge mode derivation unit 302 will be described later.

[0064] The sub-block predicted motion vector mode derivation unit 303 derives multiple sub-block predicted motion vector candidates, selects a sub-block predicted motion vector, and calculates a differential motion vector between the selected sub-block predicted motion vector and the detected motion vector. The detected inter prediction mode, reference index, motion vector, and calculated differential motion vector become inter prediction information for the sub-block predicted motion vector mode. This inter prediction information is supplied to the inter prediction mode determination unit 305.

[0065] The subblock merging mode derivation unit 304 derives a plurality of subblock merging candidates, selects a subblock merging candidate, and obtains inter prediction information for the subblock merging mode. This inter prediction information is supplied to the inter prediction mode determination unit 305.

[0066] The inter prediction mode determination unit 305 determines inter prediction information based on the inter prediction information supplied from the normal prediction motion vector mode derivation unit 301, the normal merge mode derivation unit 302, the sub-block prediction motion vector mode derivation unit 303, and the sub-block merge mode derivation unit 304. The inter prediction information according to the determination result is supplied from the inter prediction mode determination unit 305 to the motion compensation prediction unit 306.

[0067] Based on the determined inter prediction information, the motion compensation prediction unit 306 performs inter prediction on the reference image signal stored in the decoded image memory 104. The detailed configuration and processing of the motion compensation prediction unit 306 will be described later.

[0068] <Description of Inter Prediction Unit 203 on Decoding Side> FIG. 22 is a diagram showing a detailed configuration of the inter prediction unit 203 of the image decoding device in FIG.

[0069] The normal predicted motion vector mode derivation unit 401 derives a plurality of normal predicted motion vector candidates, selects a predicted motion vector, and calculates the sum of the selected predicted motion vector and the decoded differential motion vector to obtain a motion vector. The decoded inter prediction mode, reference index, and motion vector become inter prediction information for the normal predicted motion vector mode. This inter prediction information is supplied to the motion compensation prediction unit 406 via a switch 408. The detailed configuration and processing of the normal predicted motion vector mode derivation unit 401 will be described later.

[0070] The normal merge mode derivation unit 402 derives multiple normal merge candidates, selects one, and obtains inter prediction information for the normal merge mode. This inter prediction information is supplied to the motion compensation prediction unit 406 via a switch 408. The detailed configuration and processing of the normal merge mode derivation unit 402 will be described later.

[0071] The sub-block predicted motion vector mode derivation unit 403 derives multiple sub-block predicted motion vector candidates, selects a sub-block predicted motion vector, and calculates the sum of the selected sub-block predicted motion vector and the decoded differential motion vector to obtain a motion vector. The decoded inter prediction mode, reference index, and motion vector become inter prediction information for the sub-block predicted motion vector mode. This inter prediction information is supplied to the motion compensation prediction unit 406 via a switch 408.

[0072] The subblock merging mode derivation unit 404 derives a plurality of subblock merging candidates, selects a subblock merging candidate, and obtains inter-prediction information for the subblock merging mode. This inter-prediction information is supplied to the motion compensation prediction unit 406 via a switch 408.

[0073] Based on the determined inter prediction information, the motion compensation prediction unit 406 performs inter prediction on the reference image signal stored in the decoded image memory 208. The detailed configuration and processing of the motion compensation prediction unit 406 are similar to those of the motion compensation prediction unit 306 on the encoding side.

[0074] <Normal prediction motion vector mode derivation unit (normal AMVP)> The normal prediction motion vector mode derivation unit 301 in Figure 17 includes a spatial prediction motion vector candidate derivation unit 321, a temporal prediction motion vector candidate derivation unit 322, a history prediction motion vector candidate derivation unit 323, a prediction motion vector candidate supplementation unit 325, a normal motion vector detection unit 326, a prediction motion vector candidate selection unit 327, and a motion vector subtraction unit 328.

[0075] The normal prediction motion vector mode derivation unit 401 in Figure 23 includes a spatial prediction motion vector candidate derivation unit 421, a temporal prediction motion vector candidate derivation unit 422, a history prediction motion vector candidate derivation unit 423, a prediction motion vector candidate supplementation unit 425, a prediction motion vector candidate selection unit 426, and a motion vector addition unit 427.

[0076] The processing procedures of the normal prediction motion vector mode derivation unit 301 on the encoding side and the normal prediction motion vector mode derivation unit 401 on the decoding side will be described using the flowcharts of Fig. 19 and Fig. 25. Fig. 19 is a flowchart showing the normal prediction motion vector mode derivation processing procedure by the normal motion vector mode derivation unit 301 on the encoding side, and Fig. 25 is a flowchart showing the normal prediction motion vector mode derivation processing procedure by the normal motion vector mode derivation unit 401 on the decoding side.

[0077] <Normal predicted motion vector mode derivation unit (normal AMVP): description on the encoding side> A normal predicted motion vector mode derivation processing procedure on the encoding side will be described with reference to Fig. 19. In the description of the processing procedure in Fig. 19, the word "normal" shown in Fig. 19 may be omitted.

[0078] First, the normal motion vector detection unit 326 detects a normal motion vector for each inter prediction mode and reference index (step S100 in FIG. 19).

[0079] Next, the spatial predicted motion vector candidate derivation unit 321, the temporal predicted motion vector candidate derivation unit 322, the history predicted motion vector candidate derivation unit 323, the predicted motion vector candidate supplementation unit 325, the predicted motion vector candidate selection unit 327, and the motion vector subtraction unit 328 calculate differential motion vectors for the motion vectors used in inter prediction in the normal predicted motion vector mode for each of L0 and L1 (steps S101 to S106 in FIG. 19 ). Specifically, when the prediction mode PredMode of the block to be processed is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), the predicted motion vector candidate list mvpListL0 of L0 is calculated, the predicted motion vector mvpL0 is selected, and the differential motion vector mvdL0 of the motion vector mvL0 of L0 is calculated. When the inter prediction mode of the block to be processed is L1 prediction (Pred_L1), an L1 predicted motion vector candidate list mvpListL1 is calculated, a predicted motion vector mvpL1 is selected, and a differential motion vector mvdL1 of the L1 motion vector mvL1 is calculated. When the inter prediction mode of the block to be processed is bi-predictive (Pred_BI), both L0 prediction and L1 prediction are performed, an L0 predicted motion vector candidate list mvpListL0 is calculated, an L0 predicted motion vector mvpL0 is selected, and a differential motion vector mvdL0 of the L0 motion vector mvL0 is calculated, and an L1 predicted motion vector candidate list mvpListL1 is calculated, an L1 predicted motion vector mvpL1 is calculated, and a differential motion vector mvdL1 of the L1 motion vector mvL1 is calculated.

[0080] Although differential motion vector calculation processing is performed for each of L0 and L1, the processing is common to both L0 and L1. Therefore, in the following description, L0 and L1 are represented as a common list LX. In the processing for calculating the differential motion vector for L0, X of LX is 0, and in the processing for calculating the differential motion vector for L1, X of LX is 1. Furthermore, when referencing information from another list instead of LX during the processing for calculating the differential motion vector for LX, the other list is represented as LY.

[0081] When the motion vector mvLX of LX is used (step S102: YES in FIG. 19 ), candidates for the predicted motion vector of LX are calculated, and a predicted motion vector candidate list mvpListLX for LX is constructed (step S103 in FIG. 19 ). The spatial predicted motion vector candidate derivation unit 321, the temporal predicted motion vector candidate derivation unit 322, the history predicted motion vector candidate derivation unit 323, and the predicted motion vector candidate supplementation unit 325 in the normal predicted motion vector mode derivation unit 301 derive multiple predicted motion vector candidates to construct the predicted motion vector candidate list mvpListLX. The detailed processing procedure of step S103 in FIG. 19 will be described later using the flowchart in FIG. 20 .

[0082] Next, the motion vector predictor candidate selection unit 327 selects a motion vector predictor mvpLX for LX from the motion vector predictor candidate list mvpListLX for LX (step S104 in FIG. 19 ). Here, one element (the i-th element counting from 0) in the motion vector predictor candidate list mvpListLX is represented as mvpListLX[i]. Each differential motion vector is calculated as the difference between the motion vector mvLX and each motion vector predictor candidate mvpListLX[i] stored in the motion vector predictor candidate list mvpListLX. The amount of code when these differential motion vectors are coded is calculated for each element (motion vector predictor candidate) in the motion vector predictor candidate list mvpListLX. Then, among the elements registered in the motion vector predictor candidate list mvpListLX, the motion vector predictor candidate mvpListLX[i] with the smallest amount of code for each motion vector predictor candidate is selected as the motion vector predictor mvpLX, and its index i is obtained. If there are multiple candidates for the predicted motion vector that result in the smallest amount of generated code in the predicted motion vector candidate list mvpListLX, the candidate predicted motion vector mvpListLX[i] represented by the smallest index i in the predicted motion vector candidate list mvpListLX is selected as the optimal predicted motion vector mvpLX, and its index i is obtained.

[0083] Next, the motion vector subtraction unit 328 subtracts the selected predicted motion vector mvpLX of LX from the motion vector mvLX of LX to calculate the differential motion vector mvdLX of LX as mvdLX=mvLX-mvpLX (step S105 in FIG. 19).

[0084] <Normal Predictor Motion Vector Mode Derivation Unit (Normal AMVP): Decoding Side> Next, the normal predictor motion vector mode processing procedure on the decoding side will be described with reference to Fig. 25. On the decoding side, the spatial predictor motion vector candidate derivation unit 421, the temporal predictor motion vector candidate derivation unit 422, the history predictor motion vector candidate derivation unit 423, and the predictor motion vector candidate supplementation unit 425 calculate motion vectors used in inter prediction in the normal predictor motion vector mode for each of L0 and L1 (steps S201 to S206 in Fig. 25). Specifically, when the prediction mode PredMode of the block to be processed is inter prediction (MODE_INTER) and the inter prediction mode of the block to be processed is L0 prediction (Pred_L0), the predictor motion vector candidate list mvpListL0 for L0 is calculated, the predictor motion vector mvpL0 is selected, and the motion vector mvL0 for L0 is calculated. When the inter prediction mode of the block to be processed is L1 prediction (Pred_L1), an L1 predicted motion vector candidate list mvpListL1 is calculated, a predicted motion vector mvpL1 is selected, and a motion vector mvL1 of L1 is calculated. When the inter prediction mode of the block to be processed is bi-predictive (Pred_BI), both L0 prediction and L1 prediction are performed, an L0 predicted motion vector candidate list mvpListL0 is calculated, an L0 predicted motion vector mvpL0 is selected, and a motion vector mvL0 of L0 is calculated, and an L1 predicted motion vector candidate list mvpListL1 is calculated, an L1 predicted motion vector mvpL1 is calculated, and a motion vector mvL1 of L1 is calculated.

[0085] As with the encoding side, the decoding side also performs motion vector calculation processing for each of L0 and L1, but the processing is common to both L0 and L1. Therefore, in the following description, L0 and L1 are represented as a common LX. LX represents the inter prediction mode used for inter prediction of the encoding block to be processed. In the processing to calculate the motion vector of L0, X is 0, and in the processing to calculate the motion vector of L1, X is 1. Furthermore, when information in another reference list is referenced during the processing to calculate the motion vector of LX, rather than the same reference list as the LX to be calculated, the other reference list is represented as LY.

[0086] When the motion vector mvLX of LX is used (step S202: YES in Fig. 25 ), a candidate predicted motion vector for LX is calculated, and a candidate predicted motion vector list mvpListLX for LX is constructed (step S203 in Fig. 25 ). A spatial predicted motion vector candidate derivation unit 421, a temporal predicted motion vector candidate derivation unit 422, a history predicted motion vector candidate derivation unit 423, and a predicted motion vector candidate supplementation unit 425 in the normal predicted motion vector mode derivation unit 401 calculate multiple candidate predicted motion vectors, and construct the candidate predicted motion vector list mvpListLX. The detailed processing procedure of step S203 in Fig. 25 will be described later using the flowchart in Fig. 20 .

[0087] Next, the predicted motion vector candidate selection unit 426 extracts the predicted motion vector candidate mvpListLX[mvpIdxLX] corresponding to the predicted motion vector index mvpIdxLX decoded and supplied by the bit string decoding unit 201 from the predicted motion vector candidate list mvpListLX as the selected predicted motion vector mvpLX (step S204 in Figure 25).

[0088] Next, the motion vector addition unit 427 adds the differential motion vector mvdLX of LX decoded and supplied by the bitstream decoding unit 201 and the predicted motion vector mvpLX of LX to calculate the motion vector mvLX of LX as mvLX = mvpLX + mvdLX (step S205 in Figure 25).

[0089] <Normal prediction motion vector mode derivation unit (normal AMVP): motion vector prediction method> Figure 20 is a flowchart showing the processing steps of a normal prediction motion vector mode derivation process having functions common to the normal prediction motion vector mode derivation unit 301 of an image encoding device and the normal prediction motion vector mode derivation unit 401 of an image decoding device according to an embodiment of the present invention.

[0090] The normal motion vector predictor mode derivation unit 301 and the normal motion vector predictor mode derivation unit 401 are provided with a motion vector predictor candidate list mvpListLX. The motion vector predictor candidate list mvpListLX has a list structure and is provided with a storage area for storing, as elements, a motion vector predictor index indicating a location within the motion vector predictor candidate list and a motion vector predictor candidate corresponding to the index. The numbers of the motion vector predictor index start from 0, and the motion vector predictor candidates are stored in the storage area of ​​the motion vector predictor candidate list mvpListLX. In this embodiment, the motion vector predictor candidate list mvpListLX is assumed to be capable of registering at least two motion vector predictor candidates (inter prediction information). Furthermore, a variable numCurrMvpCand indicating the number of motion vector predictor candidates registered in the motion vector predictor candidate list mvpListLX is set to 0.

[0091] The spatial motion vector predictor candidate derivation units 321 and 421 derive a candidate motion vector predictor from the adjacent block on the left. In this process, the spatial motion vector predictor candidate derivation units 321 and 421 derive a candidate motion vector predictor from the adjacent block on the left (A0 or A1 in FIG. 11 ) by referring to the inter prediction information of the adjacent block on the left (i.e., a flag indicating whether a candidate motion vector predictor is available, a motion vector, a reference index, etc.), and add the derived mvLXA to the motion vector predictor candidate list mvpListLX (step S301 in FIG. 20 ). Note that X is 0 for L0 prediction, and X is 1 for L1 prediction (the same applies below). Next, the spatial motion vector predictor candidate derivation units 321 and 421 derive a candidate motion vector predictor from the adjacent block above. In this process, a predicted motion vector mvLXB is derived by referring to the inter prediction information of the adjacent block above (B0, B1, or B2 in FIG. 11), i.e., a flag indicating whether a predicted motion vector candidate is available, a motion vector, a reference index, etc., and if the derived mvLXA and mvLXB are not equal, mvLXB is added to the predicted motion vector candidate list mvpListLX (step S302 in FIG. 20). The processes of steps S301 and S302 in FIG. 20 are common except for the positions and number of adjacent blocks to be referenced, and a flag availableFlagLXN indicating whether a predicted motion vector candidate of the coding block is available, a motion vector mvLXN, and a reference index refIdxN (N indicates A or B, the same applies below) are derived.

[0092] Next, the temporal motion vector predictor candidate derivation units 322 and 422 derive motion vector predictor candidate from a block in a picture whose time is different from that of the currently processed picture. In this process, a flag availableFlagLXCol indicating whether a motion vector predictor candidate of a coding block of a picture whose time is different is available, a motion vector mvLXCol, a reference index refIdxCol, and a reference list listCol are derived, and mvLXCol is added to the motion vector predictor candidate list mvpListLX (step S303 in FIG. 20 ).

[0093] It is assumed that the processing of the temporal motion vector predictor candidate derivation units 322 and 422 can be omitted in units of sequences (SPS), pictures (PPS), or slices.

[0094] Next, the history motion vector predictor candidate derivation units 323 and 423 add the history motion vector predictor candidates registered in the history motion vector predictor candidate list HmvpCandList to the motion vector predictor candidate list mvpListLX (step S304 in FIG. 20 ). Details of the registration process procedure in step S304 will be described later using the flowchart in FIG. 29 .

[0095] Next, the motion vector predictor candidate supplementing units 325 and 425 add motion vector predictor candidates of predetermined values, such as (0, 0), until the motion vector predictor candidate list mvpListLX is filled (S305 in FIG. 20).

[0096] <Normal merge mode derivation unit (normal merge)> The normal merge mode derivation unit 302 in Figure 18 includes a spatial merge candidate derivation unit 341, a temporal merge candidate derivation unit 342, an average merge candidate derivation unit 344, a history merge candidate derivation unit 345, a merge candidate supplementation unit 346, and a merge candidate selection unit 347.

[0097] The normal merge mode derivation unit 402 in FIG. 24 includes a spatial merge candidate derivation unit 441 , a temporal merge candidate derivation unit 442 , an average merge candidate derivation unit 444 , a history merge candidate derivation unit 445 , a merge candidate supplementation unit 446 , and a merge candidate selection unit 447 .

[0098] FIG. 21 is a flowchart illustrating the procedure of a normal merge mode derivation process having a function common to the normal merge mode derivation unit 302 of the image encoding device and the normal merge mode derivation unit 402 of the image decoding device according to an embodiment of the present invention.

[0099] The various processes will be described below in order. Note that, unless otherwise specified, the following description will be given assuming that the slice type "slice_type" is a B slice; however, the description can also be applied to a P slice. However, when the slice type "slice_type" is a P slice, only L0 prediction (Pred_L0) is available as an inter prediction mode, and L1 prediction (Pred_L1) and bi-prediction (Pred_BI) are not available, so processing related to L1 can be omitted.

[0100] The normal merge mode derivation unit 302 and the normal merge mode derivation unit 402 include a merge candidate list mergeCandList. The merge candidate list mergeCandList has a list structure and includes a merge index indicating a location within the merge candidate list and a storage area for storing merge candidates corresponding to the index as elements. The merge index numbers start from 0, and merge candidates are stored in the storage area of ​​the merge candidate list mergeCandList. In the following processing, the merge candidate with merge index i registered in the merge candidate list mergeCandList is represented by mergeCandList[i]. In this embodiment, the merge candidate list mergeCandList is capable of registering at least six merge candidates (inter-prediction information). Furthermore, a variable numCurrMergeCand indicating the number of merge candidates registered in the merge candidate list mergeCandList is set to 0.

[0101] The spatial merge candidate derivation unit 341 and the spatial merge candidate derivation unit 441 derive spatial merge candidates from the blocks adjacent to the target block (B1, A1, B0, A0, B2 in FIG. 11 ) in the order of B1, A1, B0, A0, B2 from the coding information stored in the coding information storage memory 111 of the image coding device or the coding information storage memory 205 of the image decoding device, and register the derived spatial merge candidates in a merge candidate list mergeCandList (step S401 in FIG. 21 ). Here, N is defined to indicate either B1, A1, B0, A0, B2, or a temporal merge candidate Col. The following are derived: a flag availableFlagN indicating whether the inter prediction information of block N can be used as a spatial merge candidate; an L0 reference index refIdxL0N and an L1 reference index refIdxL1N of spatial merge candidate N; an L0 prediction flag predFlagL0N indicating whether L0 prediction is performed; an L1 prediction flag predFlagL1N indicating whether L1 prediction is performed; an L0 motion vector mvL0N; and an L1 motion vector mvL1N. However, in this embodiment, merge candidates are derived without reference to the inter prediction information of blocks included in the coding block to be processed, and therefore spatial merge candidates using the inter prediction information of blocks included in the coding block to be processed are not derived.

[0102] Next, the temporal merge candidate derivation unit 342 and the temporal merge candidate derivation unit 442 derive temporal merge candidates from pictures of different times and register the derived temporal merge candidates in a merge candidate list mergeCandList (step S402 in FIG. 21 ). The temporal merge candidate derivation unit 342 and the temporal merge candidate derivation unit 442 derive a flag availableFlagCol indicating whether the temporal merge candidate is available, an L0 prediction flag predFlagL0Col indicating whether L0 prediction of the temporal merge candidate is performed, an L1 prediction flag predFlagL1Col indicating whether L1 prediction is performed, and an L0 motion vector mvL0Col and an L1 motion vector mvL1Col.

[0103] It is assumed that the processing of the temporal merge candidate derivation unit 342 and the temporal merge candidate derivation unit 442 can be omitted in units of sequences (SPS), pictures (PPS), or slices.

[0104] Next, the history merge candidate derivation unit 345 and the history merge candidate derivation unit 445 register the history motion vector predictor candidates registered in the history motion vector predictor candidate list HmvpCandList in the merge candidate list mergeCandList (step S403 in FIG. 21 ). Note that if the number of merge candidates registered in the merge candidate list mergeCandList, numCurrMergeCand, is smaller than the maximum number of merge candidates, MaxNumMergeCand, history merge candidates are derived with the number of merge candidates registered in the merge candidate list mergeCandList set to an upper limit of the maximum number of merge candidates, MaxNumMergeCand, and registered in the merge candidate list mergeCandList.

[0105] Next, the average merge candidate derivation unit 344 and the average merge candidate derivation unit 444 derive an average merge candidate from the merge candidate list mergeCandList and add the derived average merge candidate to the merge candidate list mergeCandList (step S404 in FIG. 21 ). Note that if the number of merge candidates registered in the merge candidate list mergeCandList (numCurrMergeCand) is smaller than the maximum number of merge candidates MaxNumMergeCand, the average merge candidate is derived with the number of merge candidates registered in the merge candidate list mergeCandList as an upper limit of the maximum number of merge candidates MaxNumMergeCand, and is registered in the merge candidate list mergeCandList. Here, the average merge candidate is a new merge candidate having a motion vector obtained by averaging the motion vectors of the first merge candidate and the second merge candidate registered in the merge candidate list mergeCandList for L0 prediction and L1 prediction.

[0106] Next, if the number of merge candidates registered in the merge candidate list mergeCandList (numCurrMergeCand) is smaller than the maximum number of merge candidates (MaxNumMergeCand), the merge candidate supplementation unit 346 and the merge candidate supplementation unit 446 derive additional merge candidates with the number of merge candidates registered in the merge candidate list mergeCandList set to the maximum number of merge candidates (MaxNumMergeCand) as an upper limit, and register the additional merge candidates in the merge candidate list mergeCandList (step S405 in FIG. 21 ). For P slices, merge candidates with a motion vector of (0,0) and a prediction mode of L0 prediction (Pred_L0) are added, with the maximum number of merge candidates (MaxNumMergeCand) as an upper limit. For B slices, merge candidates with a motion vector of (0,0) and a prediction mode of bi-prediction (Pred_BI) are added. The reference index used when adding a merge candidate is different from the reference index already added.

[0107] Next, the merge candidate selection unit 347 and the merge candidate selection unit 447 select a merge candidate from among the merge candidates registered in the merge candidate list mergeCandList. The merge candidate selection unit 347 on the encoding side selects a merge candidate by calculating the code amount and distortion amount, and supplies a merge index indicating the selected merge candidate and inter prediction information of the merge candidate to the motion compensation prediction unit 306 via the inter prediction mode determination unit 305. Meanwhile, the merge candidate selection unit 447 on the decoding side selects a merge candidate based on the decoded merge index, and supplies the selected merge candidate to the motion compensation prediction unit 406.

[0108] <Updating of Historical Motion Vector Predictor Candidate List> Next, a detailed description will be given of a method for initializing and updating the historical motion vector predictor candidate list HmvpCandList provided in the encoding-side coding information storage memory 111 and the decoding-side coding information storage memory 205. Fig. 26 is a flowchart illustrating the procedure for initializing and updating the historical motion vector predictor candidate list.

[0109] In this embodiment, the history prediction motion vector candidate list HmvpCandList is updated in the coding information storage memory 111 and the coding information storage memory 205. A history prediction motion vector candidate list update unit may be provided in the inter prediction unit 102 and the inter prediction unit 203 to update the history prediction motion vector candidate list HmvpCandList.

[0110] The historical prediction motion vector candidate list HmvpCandList is initially set at the beginning of the slice, and on the encoding side, the historical prediction motion vector candidate list HmvpCandList is updated when the prediction method determination unit 105 selects the normal prediction motion vector mode or the normal merge mode, and on the decoding side, the historical prediction motion vector candidate list HmvpCandList is updated when the prediction information decoded by the bitstream decoding unit 201 is the normal prediction motion vector mode or the normal merge mode.

[0111] Inter prediction information used when performing inter prediction in normal predicted motion vector mode or normal merge mode is registered as an inter prediction information candidate hMvpCand in a historical predicted motion vector candidate list HmvpCandList. The inter prediction information candidate hMvpCand includes an L0 reference index refIdxL0 and an L1 reference index refIdxL1, an L0 prediction flag predFlagL0 indicating whether L0 prediction is performed or not, an L1 prediction flag predFlagL1 indicating whether L1 prediction is performed or not, an L0 motion vector mvL0, and an L1 motion vector mvL1.

[0112] If inter-prediction information with the same value as the inter-prediction information candidate hMvpCand exists among the elements (i.e., inter-prediction information) registered in the history predicted motion vector candidate list HmvpCandList provided in the encoding-side encoding information storage memory 111 and the decoding-side encoding information storage memory 205, that element is deleted from the history predicted motion vector candidate list HmvpCandList. On the other hand, if inter-prediction information with the same value as the inter-prediction information candidate hMvpCand does not exist, the first element of the history predicted motion vector candidate list HmvpCandList is deleted, and the inter-prediction information candidate hMvpCand is added to the end of the history predicted motion vector candidate list HmvpCandList.

[0113] The number of elements in the historical motion vector predictor candidate list HmvpCandList provided in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side in the present invention is assumed to be six.

[0114] First, the historical motion vector predictor candidate list HmvpCandList is initialized for each slice (step S2101 in Fig. 26 ). All elements of the historical motion vector predictor candidate list HmvpCandList are cleared at the beginning of the slice, and the value of NumHmvpCand, which is the number of historical motion vector predictor candidates (current number of candidates) registered in the historical motion vector predictor candidate list HmvpCandList, is set to 0.

[0115] Note that although the initialization of the historical motion vector predictor candidate list HmvpCandList is performed in units of slices (first coding blocks of a slice), it may also be performed in units of pictures, tiles, or tree block rows.

[0116] Next, the following update process of the historical motion vector predictor candidate list HmvpCandList is repeatedly performed for each coding block in the slice (steps S2102 to S2107 in FIG. 26).

[0117] First, initial setting is performed for each coding block: A flag identicalCandExist indicating whether an identical candidate exists is set to FALSE, and a deletion target index removeIdx indicating a candidate to be deleted is set to 0 (step S2103 in FIG. 26).

[0118] It is determined whether or not there is an inter-prediction information candidate hMvpCand to be registered (step S2104 in FIG. 26 ). If the prediction method determination unit 105 on the encoding side determines the normal prediction motion vector mode or the normal merge mode, or if the bitstream decoding unit 201 on the decoding side decodes the normal prediction motion vector mode or the normal merge mode, the inter-prediction information is designated as the inter-prediction information candidate hMvpCand to be registered. If the prediction method determination unit 105 on the encoding side determines the intra prediction mode, the sub-block prediction motion vector mode, or the sub-block merge mode, or if the bitstream decoding unit 201 on the decoding side decodes the intra prediction mode, the sub-block prediction motion vector mode, or the sub-block merge mode, the history prediction motion vector candidate list HmvpCandList is not updated, and no inter-prediction information candidate hMvpCand to be registered exists. If there is no inter-prediction information candidate hMvpCand to be registered, steps S2105 to S2106 are skipped (step S2104 in FIG. 26 : NO). If there is an inter prediction information candidate hMvpCand to be registered, the processes in step S2105 and thereafter are performed (step S2104 in FIG. 26: YES).

[0119] Next, it is determined whether or not each element in the history motion vector candidate list HmvpCandList contains an element (inter-prediction information) with the same value as the inter-prediction information candidate hMvpCand to be registered, i.e., whether or not there is an identical element (step S2105 in FIG. 26 ). FIG. 27 is a flowchart of this identical element confirmation process. If the value of the number of history motion vector candidate predictors NumHmvpCand is 0 (step S2121: NO in FIG. 27 ), the history motion vector candidate predictor list HmvpCandList is empty and no identical candidate exists, so steps S2122 to S2125 in FIG. 27 are skipped, and this identical element confirmation process ends. If the value of the number of history motion vector candidate predictors NumHmvpCand is greater than 0 (YES in step S2121 in FIG. 27 ), the process of step S2123 is repeated for the history motion vector index hMvpIdx from 0 to NumHmvpCand-1 (steps S2122 to S2125 in FIG. 27 ). First, the hMvpIdx-th element HmvpCandList[hMvpIdx] counting from 0 in the history motion vector candidate list is compared to determine whether it is identical to the inter-prediction information candidate hMvpCand (step S2123 in FIG. 27). If they are identical (step S2123 in FIG. 27: YES), a flag identicalCandExist indicating whether an identical candidate exists is set to TRUE, and a deletion target index removeIdx indicating the position of the element to be deleted is set to the current value of the history motion vector index hMvpIdx, and this identical element confirmation process ends. If they are not identical (step S2123 in FIG. 27: NO), hMvpIdx is incremented by 1, and if the history motion vector index hMvpIdx is equal to or less than NumHmvpCand-1, the processes from step S2123 onwards are performed.

[0120] Returning to the flowchart of Fig. 26 again, a process of shifting and adding elements of the history motion vector predictor candidate list HmvpCandList is performed (step S2106 of Fig. 26 ). Fig. 28 is a flowchart of the process procedure of shifting / adding elements of the history motion vector predictor candidate list HmvpCandList in step S2106 of Fig. 26 . First, it is determined whether to remove elements stored in the history motion vector predictor candidate list HmvpCandList and then add a new element, or to add a new element without removing any elements. Specifically, it is compared whether the flag identicalCandExist indicating whether or not an identical candidate exists is TRUE (true) or whether NumHmvpCand is 6 (step S2141 of Fig. 28 ). If either the condition of the flag identicalCandExist indicating whether or not an identical candidate exists is TRUE (true) or the current number of candidates NumHmvpCand is 6 is met (step S2141 of Fig. 28 : YES), the elements stored in the history motion vector predictor candidate list HmvpCandList are removed and then a new element is added. The initial value of index i is set to the value of removeIdx + 1. The element shifting process of step S2143 is repeated from this initial value to NumHmvpCand (steps S2142 to S2144 in FIG. 28 ). The elements of HmvpCandList[i] are shifted forward by copying them to HmvpCandList[i - 1] (step S2143 in FIG. 28 ), and i is incremented by 1 (steps S2142 to S2144 in FIG. 28 ). Next, the inter-prediction information candidate hMvpCand is added to the (NumHmvpCand-1)-th HmvpCandList[NumHmvpCand-1], counting from 0 and corresponding to the end of the history motion vector predictor candidate list (step S2145 in FIG. 28 ), and the element shifting and addition process of this history motion vector predictor candidate list HmvpCandList is terminated.On the other hand, if neither of the conditions of the flag identicalCandExist indicating whether or not an identical candidate exists is TRUE (true) and NumHmvpCand is 6 is satisfied (step S2141 in FIG. 28 : NO), the inter-prediction information candidate hMvpCand is added to the end of the history motion vector predictor candidate list without removing the elements stored in the history motion vector predictor candidate list HmvpCandList (step S2146 in FIG. 28 ). Here, the end of the history motion vector predictor candidate list is the NumHmvpCand-th HmvpCandList[NumHmvpCand] counting from 0. Furthermore, NumHmvpCand is incremented by 1, and the element shift and addition process of this history motion vector predictor candidate list HmvpCandList is terminated.

[0121] Figure 31 is a diagram illustrating an example of a history predictor motion vector candidate list update process. When adding a new element to the registered history predictor motion vector candidate list HmvpCandList of six elements (inter-prediction information), compared with the new inter-prediction information in order from the front element of the history predictor motion vector candidate list HmvpCandList (Figure 31A), if the new element is the same value as the third element HMVP2 from the beginning of the history predictor motion vector candidate list HmvpCandList, delete the element HMVP2 from the history predictor motion vector candidate list HmvpCandList, and shift (copy) the rear elements HMVP3 to HMVP5 forward one by one, add a new element to the end of the history predictor motion vector candidate list HmvpCandList (Figure 31B), and complete the update of the history predictor motion vector candidate list HmvpCandList (Figure 31C).

[0122] <History predictor motion vector candidate derivation process> Next, a detailed description will be given of a method for deriving a history predictor motion vector candidate from the history predictor motion vector candidate list HmvpCandList, which is the processing procedure of step S304 in Fig. 20 , and which is common processing between the history predictor motion vector candidate derivation unit 323 of the normal predictor motion vector mode derivation unit 301 on the encoding side and the history predictor motion vector candidate derivation unit 423 of the normal predictor motion vector mode derivation unit 401 on the decoding side. Fig. 29 is a flowchart illustrating the history predictor motion vector candidate derivation process procedure.

[0123] If the number of current motion vector predictor candidates numCurrMvpCand is greater than or equal to the maximum number of elements (2 here) of the motion vector predictor candidate list mvpListLX or the number of history motion vector predictor candidates, the value of NumHmvpCand, is 0 (NO in step S2201 of Fig. 29), steps S2202 to S2209 of Fig. 29 are omitted, and the history motion vector predictor candidate derivation processing procedure is terminated. If the number of current motion vector predictor candidates numCurrMvpCand is smaller than 2, which is the maximum number of elements of the motion vector predictor candidate list mvpListLX, and the value of the number of history motion vector predictor candidates NumHmvpCand is greater than 0 (YES in step S2201 of Fig. 29), the processing of steps S2202 to S2209 of Fig. 29 is performed.

[0124] Next, the processes of steps S2203 to S2208 in Fig. 29 are repeated for index i from 1 to the smaller of 4 and the number of history motion vector predictor candidates, numCheckedHMVPCand (steps S2202 to S2209 in Fig. 29). If the current number of motion vector predictor candidates, numCurrMvpCand, is equal to or greater than 2, which is the maximum number of elements in the motion vector predictor candidate list, mvpListLX (step S2203: NO in Fig. 29), steps S2204 to S2209 in Fig. 29 are omitted, and the history motion vector predictor candidate derivation process procedure is terminated. If the current number of motion vector predictor candidates, numCurrMvpCand, is less than 2, which is the maximum number of elements in the motion vector predictor candidate list, mvpListLX (step S2203: YES in Fig. 29), the processes from step S2204 onwards in Fig. 29 are performed.

[0125] Next, the processes from steps S2205 to S2207 are performed for Y=0 and 1 (L0 and L1), respectively (steps S2204 to S2208 in FIG. 29 ). If the current number of motion vector predictor candidates numCurrMvpCand is equal to or greater than 2, which is the maximum number of elements in the motion vector predictor candidate list mvpListLX (step S2205: NO in FIG. 29 ), steps S2206 to S2209 in FIG. 29 are omitted, and the history motion vector predictor candidate derivation process procedure ends. If the current number of motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements in the motion vector predictor candidate list mvpListLX (step S2205: YES in FIG. 29 ), the processes from step S2206 onwards in FIG. 29 are performed.

[0126] Next, if the history predicted motion vector candidate list HmvpCandList contains an element with the same reference index as the reference index refIdxLX of the motion vector to be encoded / decoded, and is different from any element in the predicted motion vector list mvpListLX (step S2206: YES in Figure 29), the LY motion vector of the history predicted motion vector candidate HmvpCandList[NumHmvpCand - i] is added to the numCurrMvpCand-th element mvpListLX[numCurrMvpCand] counting from 0 in the predicted motion vector candidate list (step S2207 in Figure 29), and the number of current predicted motion vector candidates numCurrMvpCand is incremented by 1. If there is no element in the history predicted motion vector candidate list HmvpCandList that has the same reference index as the reference index refIdxLX of the motion vector to be encoded / decoded and that is different from any element in the predicted motion vector list mvpListLX (step S2206 in Figure 29: NO), the additional processing of step S2207 is skipped.

[0127] The above processing of steps S2205 to S2207 in Fig. 29 is performed for both L0 and L1 (steps S2204 to S2208 in Fig. 29). Index i is incremented by 1, and if index i is equal to or less than the smaller value of 4 or the number of historical motion vector predictor candidates NumHmvpCand, the processing from step S2203 onwards is performed again (steps S2202 to S2209 in Fig. 29).

[0128] <History Merge Candidate Derivation Process> Next, a detailed description will be given of a method for deriving history merge candidates from the history merge candidate list HmvpCandList, which is the processing procedure of step S404 in Fig. 21 and is common to the history merge candidate derivation unit 345 of the normal merge mode derivation unit 302 on the encoding side and the history merge candidate derivation unit 445 of the normal merge mode derivation unit 402 on the decoding side. Fig. 30 is a flowchart illustrating the history merge candidate derivation process procedure.

[0129] First, initialization is performed (step S2301 in FIG. 30). The value FALSE is set for each of the elements 0 to (numCurrMergeCand - 1) of isPruned[i], and the variable numOrigMergeCand is set to the number of elements currently registered in the merge candidate list, numCurrMergeCand.

[0130] Next, the initial value of the index hMvpIdx is set to 1, and the addition process from step S2303 to step S2310 in Figure 30 is repeated from this initial value to NumHmvpCand (steps S2302 to S2311 in Figure 30). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is not less than (the maximum number of merge candidates, MaxNumMergeCand-1), merge candidates have been added to all elements in the merge candidate list, and this history merge candidate derivation process is terminated (NO in step S2303 in Figure 30). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is less than (the maximum number of merge candidates, MaxNumMergeCand-1), processing from step S2304 onwards is performed. The value of sameMotion is set to FALSE (step S2304 in Figure 30). Next, the initial value of index i is set to 0, and steps S2306 and S2307 in Fig. 30 are performed from this initial value to numOrigMergeCand-1 (S2305 to S2308 in Fig. 30). A comparison is made to determine whether the (NumHmvpCand-hMvpIdx)th element HmvpCandList[NumHmvpCand-hMvpIdx], counting from 0 in the history motion vector prediction candidate list, has the same value as the i-th element mergeCandList[i], counting from 0 in the merge candidate list (step S2306 in Fig. 30).

[0131] Merge candidates are considered to have the same value if all of the components (inter prediction mode, reference index, motion vector) of the merge candidates have the same value. If the merge candidates have the same value and isPruned[i] is FALSE (YES in step S2306 of FIG. 30), both sameMotion and isPruned[i] are set to TRUE (step S2307 of FIG. 30). If the values ​​are not the same (NO in step S2306 of FIG. 30), the processing of step S2307 is skipped. When the repeated processing from step S2305 to step S2308 in Fig. 30 is completed, a comparison is made to see if sameMotion is FALSE (step S2309 in Fig. 30 ). If sameMotion is FALSE (YES in step S2309 in Fig. 30 ), that is, the (NumHmvpCand - hMvpIdx)-th element HmvpCandList[NumHmvpCand - hMvpIdx] counting from 0 in the history motion vector predictor candidate list is not present in mergeCandList, so the (NumHmvpCand - hMvpIdx)-th element HmvpCandList[NumHmvpCand - hMvpIdx] counting from 0 in the history motion vector predictor candidate list is added to the numCurrMergeCand-th mergeCandList[numCurrMergeCand] in the merge candidate list. hMvpIdx] and increment numCurrMergeCand by 1 (step S2310 in Figure 30). Index hMvpIdx is incremented by 1 (step S2302 in Figure 30), and steps S2302 to S2311 in Figure 30 are repeated. When confirmation of all elements in the history motion vector predictor candidate list has been completed, or when merge candidates have been added to all elements in the merge candidate list, this history merge candidate derivation process is completed.

[0132] <Average Merge Candidate Derivation Process> Next, a detailed description will be given of the average merge candidate derivation method, which is the processing procedure of step S403 in Fig. 21 , which is common to the average merge candidate derivation unit 344 of the normal merge mode derivation unit 302 on the encoding side and the average merge candidate derivation unit 444 of the normal merge mode derivation unit 402 on the decoding side. Fig. 38 is a flowchart illustrating the average merge candidate derivation processing procedure.

[0133] First, initialization is performed (step S1301 in FIG. 38). The variable numOrigMergeCand is set to the number of elements currently registered in the merge candidate list, numCurrMergeCand.

[0134] Next, the merge candidate list is scanned from the top to determine two pieces of motion information. The index i indicating the first piece of motion information is set to 0, and the index j indicating the second piece of motion information is set to 1 (steps S1302 to S1303 in FIG. 38). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is not less than (the maximum number of merge candidates, MaxNumMergeCand-1), merge candidates have been added to all elements in the merge candidate list, and this history merge candidate derivation process ends (step S1304 in FIG. 38). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is less than (the maximum number of merge candidates, MaxNumMergeCand-1), the process from step S1305 onwards is carried out.

[0135] It is determined whether the i-th motion information mergeCandList[i] in the merge candidate list and the j-th motion information mergeCandList[j] in the merge candidate list are both invalid (step S1305 in FIG. 38 ). If both are invalid, the average merge candidate for mergeCandList[i] and mergeCandList[j] is not derived, and the process moves to the next element. If mergeCandList[i] and mergeCandList[j] are not invalid, the following process is repeated with X set to 0 and 1 (steps S1306 to S1314 in FIG. 38 ).

[0136] It is determined whether the LX prediction of mergeCandList[i] is valid (step S1307 in FIG. 38). If the LX prediction of mergeCandList[i] is valid, it is determined whether the LX prediction of mergeCandList[j] is valid (step S1308 in FIG. 38). If the LX prediction of mergeCandList[j] is valid, that is, if both the LX predictions of mergeCandList[i] and mergeCandList[j] are valid, an average merge candidate for LX prediction having the reference index of the LX prediction of mergeCandList[i] and the motion vector of the LX prediction obtained by averaging the motion vector of the LX prediction of mergeCandList[i] and the motion vector of the LX prediction of mergeCandList[j] is derived and set as the LX prediction of averageCand, thereby validating the LX prediction of averageCand (step S1309 in FIG. 38). In step S1308 of Fig. 38, if the LX prediction of mergeCandList[j] is not valid, that is, if the LX prediction of mergeCandList[i] is valid and the LX prediction of mergeCandList[j] is invalid, an average merge candidate of the LX prediction having the motion vector and reference index of the LX prediction of mergeCandList[i] is derived and set as the LX prediction of averageCand, and the LX prediction of averageCand is made valid (step S1310 of Fig. 38). In step S1307 of Fig. 38, if the LX prediction of mergeCandList[i] is not valid, it is determined whether the LX prediction of mergeCandList[j] is valid (step S1311 of Fig. 38). If the LX prediction of mergeCandList[j] is valid, i.e., if the LX prediction of mergeCandList[i] is invalid and the LX prediction of mergeCandList[j] is valid, an average merge candidate for the LX prediction having the motion vector and reference index of the LX prediction of mergeCandList[j] is derived and set as the LX prediction of averageCand, and the LX prediction of averageCand is made valid (step S1312 of FIG. 38).In step S1311 of Figure 38, if the LX prediction of mergeCandList[j] is not valid, i.e., if the LX prediction of mergeCandList[i] and the LX prediction of mergeCandList[j] are both invalid, the LX prediction of averageCand is disabled (step S1312 of Figure 38).

[0137] The average merge candidate averageCand for L0 prediction, L1 prediction, or BI prediction generated as described above is added to the numCurrMergeCand-th mergeCandList[numCurrMergeCand] in the merge candidate list, and numCurrMergeCand is incremented by 1 (step S1315 in FIG. 38 ). This completes the process of deriving the average merge candidate.

[0138] The average merge candidate is calculated by averaging the horizontal component of the motion vector and the vertical component of the motion vector.

[0139] <Motion compensation prediction process> The motion compensation prediction unit 306 acquires the position and size of the block currently being predicted in encoding. The motion compensation prediction unit 306 also acquires inter prediction information from the inter prediction mode determination unit 305. A reference index and a motion vector are derived from the acquired inter prediction information, and a reference picture identified by the reference index in the decoded image memory 104 is acquired at a position shifted by the motion vector from the same position as the image signal of the prediction block, and then a prediction signal is generated.

[0140] When the inter prediction mode in inter prediction is prediction from a single reference picture, such as L0 prediction or L1 prediction, the prediction signal obtained from one reference picture is used as the motion-compensated prediction signal. When the inter prediction mode is prediction from two reference pictures, such as BI prediction, the prediction signals obtained from the two reference pictures are weighted and averaged to generate the motion-compensated prediction signal, which is then supplied to the prediction method determination unit 105. Here, the weighted average ratio for bi-prediction is set to 1:1, but other ratios may also be used for weighted averaging. For example, the closer the picture interval between the picture to be predicted and the reference picture, the larger the weighting ratio may be. Furthermore, the weighting ratio may be calculated using a correspondence table of combinations of picture intervals and weighting ratios.

[0141] The motion compensation prediction unit 406 has the same function as the motion compensation prediction unit 306 on the encoding side. The motion compensation prediction unit 406 obtains inter prediction information via a switch 408 from the normal prediction motion vector mode derivation unit 401, the normal merge mode derivation unit 402, the sub-block prediction motion vector mode derivation unit 403, and the sub-block merge mode derivation unit 404. The motion compensation prediction unit 406 supplies the obtained motion compensation prediction signal to the decoded image signal superimposition unit 207.

[0142] <Regarding inter prediction mode> A process of making a prediction from a single reference picture is defined as uni-prediction, and in the case of uni-prediction, prediction is made using one of two reference pictures registered in reference lists L0 and L1, called L0 prediction or L1 prediction.

[0143] Figure 32 shows a case where uni-prediction is performed and the L0 reference picture (RefL0Pic) is located at a time earlier than the current picture (CurPic). Figure 33 shows a case where uni-prediction is performed and the L0 prediction reference picture is located at a time later than the current picture. Similarly, uni-prediction can be performed by replacing the L0 prediction reference picture in Figures 32 and 33 with the L1 prediction reference picture (RefL1Pic).

[0144] A process of making predictions from two reference pictures is defined as bi-prediction, and in the case of bi-prediction, both L0 prediction and L1 prediction are used and expressed as BI prediction. Figure 34 shows a case in which the reference picture for L0 prediction is located at a time earlier than the current picture to be processed, and the reference picture for L1 prediction is located at a time later than the current picture to be processed. Figure 35 shows a case in which the reference picture for L0 prediction and the reference picture for L1 prediction are located at a time earlier than the current picture to be processed. Figure 36 shows a case in which the reference picture for L0 prediction and the reference picture for L1 prediction are located at a time later than the current picture to be processed.

[0145] In this way, the relationship between the L0 / L1 prediction type and time can be used without being limited to L0 being the past direction and L1 being the future direction. In the case of bi-prediction, the same reference picture may be used for both L0 prediction and L1 prediction. Whether motion compensation prediction is performed in uni-prediction or bi-prediction is determined based on, for example, information (e.g., a flag) indicating whether L0 prediction and L1 prediction are to be used.

[0146] <Reference Index> In an embodiment of the present invention, in order to improve the accuracy of motion compensation prediction, it is possible to select an optimal reference picture from multiple reference pictures in motion compensation prediction. To this end, the reference picture used in motion compensation prediction is used as a reference index, and the reference index is coded into the bitstream together with a differential motion vector.

[0147] <Motion compensation processing based on normal prediction motion vector mode> As also shown in the inter prediction unit 102 on the encoding side in Fig. 16 , when inter prediction information by the normal prediction motion vector mode derivation unit 301 is selected in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 acquires this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0148] 22 , when a switch 408 is connected to the normal prediction motion vector mode derivation unit 401 during the decoding process, the motion compensation prediction unit 406 acquires inter prediction information from the normal prediction motion vector mode derivation unit 401, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

[0149] <Motion compensation processing based on normal merge mode> As also shown in the inter prediction unit 102 on the encoding side in Fig. 16 , when inter prediction information by the normal merge mode derivation unit 302 is selected in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 acquires this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0150] 22 , when a switch 408 is connected to the normal merge mode derivation unit 402 during the decoding process, the motion compensation prediction unit 406 acquires inter prediction information from the normal merge mode derivation unit 402, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

[0151] <Motion compensation processing based on sub-block prediction motion vector mode> As also shown in the inter prediction unit 102 on the encoding side in Fig. 16 , when inter prediction information is selected by the sub-block prediction motion vector mode derivation unit 303 in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 acquires this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0152] 22 , when a switch 408 is connected to the sub-block prediction motion vector mode derivation unit 403 during the decoding process, the motion compensation prediction unit 406 acquires inter prediction information from the sub-block prediction motion vector mode derivation unit 403, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

[0153] <Motion compensation processing based on sub-block merge mode> As also shown in the inter prediction unit 102 on the encoding side in Fig. 16 , when inter prediction information is selected by the sub-block merge mode derivation unit 304 in the inter prediction mode determination unit 305, the motion compensation prediction unit 306 acquires this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0154] 22 , when a switch 408 is connected to the sub-block merging mode derivation unit 404 during the decoding process, the motion compensation prediction unit 406 acquires inter prediction information from the sub-block merging mode derivation unit 404, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

[0155] <Motion compensation process based on affine transformation prediction> In the normal prediction motion vector mode and normal merge mode, motion compensation using an affine model can be used based on the following flags. The following flags are reflected in the following flags based on the inter prediction conditions determined by the inter prediction mode determination unit 305 in the encoding process, and are encoded in the bitstream. In the decoding process, it is specified whether or not to perform motion compensation using an affine model based on the following flags in the bitstream.

[0156] sps_affine_enabled_flag indicates whether affine model motion compensation is available in inter prediction. If sps_affine_enabled_flag is 0, affine model motion compensation is suppressed on a sequence-by-sequence basis. Furthermore, inter_affine_flag and cu_affine_type_flag are not transmitted in the CU (coding block) syntax of the coded video sequence. If sps_affine_enabled_flag is 1, affine model motion compensation is available in the coded video sequence.

[0157] sps_affine_type_flag indicates whether motion compensation using a 6-parameter affine model is available in inter prediction. If sps_affine_type_flag is 0, motion compensation using a 6-parameter affine model is suppressed. Furthermore, cu_affine_type_flag is not transmitted in the CU syntax of the coded video sequence. If sps_affine_type_flag is 1, motion compensation using a 6-parameter affine model is available in the coded video sequence. If sps_affine_type_flag does not exist, it is assumed to be 0.

[0158] When decoding a P or B slice, if inter_affine_flag is 1 for the currently processed CU, motion compensation using an affine model is used to generate a motion compensation prediction signal for the currently processed CU. If inter_affine_flag is 0, the affine model is not used for the currently processed CU. If inter_affine_flag does not exist, it is assumed to be 0.

[0159] When decoding a P or B slice, if cu_affine_type_flag is 1 for the currently processed CU, motion compensation based on a 6-parameter affine model is used to generate a motion-compensated prediction signal for the currently processed CU. If cu_affine_type_flag is 0, motion compensation based on a 4-parameter affine model is used to generate a motion-compensated prediction signal for the currently processed CU.

[0160] In motion compensation using an affine model, reference indices and motion vectors are derived for each subblock, and therefore a motion compensation prediction signal is generated using the reference indices and motion vectors being processed for each subblock.

[0161] The four-parameter affine model is a mode in which a motion vector for a subblock is derived from four parameters, the horizontal and vertical components of the motion vectors of the two control points, and motion compensation is performed on a subblock basis.

[0162] <Intra Block Copy (IBC)> The valid reference area for intra block copy will be described with reference to Figure 39. Figure 39A shows an example of determining a valid reference area using a coding tree block unit as an intra block copy reference block. In Figure 39A, 500, 501, 502, 503, and 504 are coding tree blocks, and 504 is the coding tree block to be processed. 505 is the coding block to be processed. The coding tree blocks are processed in the order of 500, 501, 502, 503, and 504. In this case, the three coding tree blocks 501, 502, and 503 processed immediately before the coding tree block 504 including the coding block to be processed 505 are set as valid reference areas for the coding tree block to be processed 505. All coding tree blocks processed before the coding tree block 501 and areas included in the coding tree block 504 including the coding block to be processed 505 are set as invalid reference areas, regardless of whether processing has been completed before the coding tree block to be processed 505.

[0163] 39B shows an example of determining a valid reference area using quartered units of a coding tree block as intra block copy reference blocks. In FIG. 39B, 515 and 516 are coding tree blocks, and 516 is the coding tree block to be processed. The coding tree block 515 is divided into quarters 506, 507, 508, and 509, and 516 is divided into quarters 510, 511, 512, and 513. 514 is the coding block to be processed. The intra block copy reference blocks are processed in the order 506, 507, 508, 509, 510, 511, 512, and 513. In this case, the three intra block copy reference blocks 508, 509, and 510 processed immediately before the intra block copy reference block 511 containing the coding block to be processed 514 are used as the valid reference area for the coding block to be processed 514. Regardless of whether the coding tree blocks processed before the intra block copy reference block 508 and the processing target coding block 514 have been completed before the processing target coding block 514, all areas included in the intra block copy reference block 511 including the processing target coding block 514 are considered invalid reference areas.

[0164] <Predicted Intra Block Copy: Description on the Encoding Side> The procedure for predictive intra block copy processing on the encoding side will be described with reference to Fig. 44 . First, the block vector detection unit 375 detects a block vector mvL (step S4500 in Fig. 44 ). Next, the IBC spatial block vector candidate derivation unit 371, the IBC history prediction block vector candidate derivation unit 372, the IBC prediction block vector candidate supplementation unit 373, the IBC prediction block vector candidate selection unit 376, and the block vector subtraction unit 378 calculate a difference block vector of the block vector used in the prediction block vector mode (steps S4501 to S4503 in Fig. 44 ).

[0165] Prediction block vector candidates are calculated to construct a block vector candidate list mvpList (step S4501 in Fig. 44 ). The IBC spatial block vector candidate derivation unit 371, IBC history block vector candidate derivation unit 372, and IBC prediction block vector candidate supplementation unit 373 in the intra block copy prediction unit 352 derive multiple prediction block vector candidates to construct the prediction block vector candidate list mvpList. The detailed processing procedure of step S4501 in Fig. 44 will be described later using the flowchart in Fig. 47 .

[0166] Next, the IBC prediction block vector candidate selection unit 376 selects a prediction block vector mvpL from the prediction block vector candidate list mvpListL (step S4502 in FIG. 44 ). Each differential block vector, which is the difference between the block vector mvL and each prediction block vector candidate mvpListL[i] stored in the prediction block vector candidate list mvpListL, is calculated. The amount of code when these differential block vectors are coded is calculated for each element of the prediction block vector candidate list mvpListL. Then, among the elements registered in the prediction block vector candidate list mvpListL, the prediction block vector candidate mvpListL[i] that has the smallest amount of code for each prediction block vector candidate is selected as the prediction block vector mvpL, and its index i is obtained. If there are multiple candidates for the prediction block vector that will result in the smallest amount of generated code in the prediction block vector candidate list mvpListL, the prediction block vector candidate mvpListL[i] represented by the smallest index i in the prediction block vector candidate list mvpListL is selected as the optimal prediction block vector mvpL, and its index i is obtained.

[0167] Next, the block vector subtraction unit 378 subtracts the selected prediction block vector mvpL from the block vector mvL to calculate a difference block vector mvdL as mvdL=mvL-mvpL (step S4503 in FIG. 44).

[0168] <Predicted Intra Block Copy: Description on the Decoding Side> Next, the prediction block vector mode processing procedure on the decoding side will be described with reference to Fig. 45. On the decoding side, the IBC spatial prediction block vector candidate derivation unit 471, the IBC history block vector candidate derivation unit 472, and the IBC prediction block vector supplementation unit 473 calculate block vectors to be used in the prediction block vector mode (steps S4600 to S4602 in Fig. 45). Specifically, a prediction block vector candidate list mvpListL is calculated, a prediction block vector mvpL is selected, and a block vector mvL is calculated.

[0169] Prediction block vector candidates are calculated to construct a prediction block vector candidate list mvpListL (step S4601 in Fig. 45 ). The IBC spatial block vector candidate derivation unit 471, IBC history block vector candidate derivation unit 472, and IBC block vector supplementation unit 473 in the intra block copy prediction unit 362 calculate multiple prediction block vector candidates to construct the prediction block vector candidate list mvpListL. A detailed description of the processing procedure of step S4601 in Fig. 45 will be omitted. Next, the IBC prediction block vector candidate selection unit 476 extracts, from the prediction block vector candidate list mvpListL, the prediction block vector candidate mvpListL[mvpIdxL] corresponding to the prediction block vector index mvpIdxL decoded and supplied by the bitstream decoding unit 201, as the selected prediction block vector mvpL (step S4601 in Fig. 45 ). Next, the block vector adder 478 adds the difference block vector mvdL decoded and supplied by the bitstream decoder 201 to the predicted block vector mvpL to calculate the block vector mvL as mvL = mvpL + mvdL (step S4602 in FIG. 45).

[0170] <Prediction block vector mode: Block vector prediction method> Figure 47 is a flowchart showing the processing steps of a prediction intra block copy mode derivation process having functions common to the intra block copy prediction unit 352 of the video encoding device and the intra block copy prediction unit 362 of the video decoding device according to an embodiment of the present invention.

[0171] The intra block copy prediction unit 352 and the intra block copy prediction unit 362 each include a prediction block vector candidate list mvpListL. The prediction block vector candidate list mvpListL has a list structure and is provided with a storage area for storing, as elements, a prediction block vector index indicating a location within the prediction block vector candidate list and a prediction block vector candidate corresponding to the index. The prediction block vector index numbers start from 0, and prediction block vector candidates are stored in the storage area of ​​the prediction block vector candidate list mvpListL. In this embodiment, the prediction block vector candidate list mvpListL is assumed to be capable of registering three prediction block vector candidates. Furthermore, a variable numCurrMvpIbcCand indicating the number of prediction block vector candidates registered in the prediction block vector candidate list mvpListL is set to 0.

[0172] The IBC spatial block vector candidate derivation units 371 and 471 derive prediction block vector candidates from the adjacent block on the left (step S4801 in FIG. 47 ). In this process, a flag availableFlagLA indicating whether a prediction block vector candidate from the adjacent block on the left (A0 or A1) is available and a block vector mvLA are derived, and mvLA is added to the prediction block vector candidate list mvpListL. Next, the IBC spatial block vector candidate derivation units 371 and 471 derive prediction block vector candidates from the adjacent block above (B0, B1, or B2) (step S4802 in FIG. 47 ). In this process, a flag availableFlagLB indicating whether a prediction motion vector candidate from the adjacent block above is available and a block vector mvLB are derived, and if mvLA and mvLB are not equal, mvLB is added to the prediction block vector candidate list mvpListL. The processing of steps S4801 and S4802 in Figure 47 is common except for the positions and number of neighboring blocks to be referenced, and derives a flag availableFlagLN indicating whether a candidate prediction block vector for the coding block is available or not, and a motion vector mvLN (N is A or B, same below).

[0173] Next, the IBC history block vector candidate derivation units 372 and 472 add the history block vector candidates registered in the history block vector candidate list HmvpIbcCandList to the predicted block vector candidate list mvpListL (step S4803 in Fig. 47 ). Details of the registration procedure in step S4803 are omitted here because they are the same as the operation described in the flowchart of Fig. 29 when the motion vector is set to the block vector, the reference index list is set to L0, and the history predicted motion vector candidate list HmvpCandList is set to the history block vector candidate list HmvpIbcCandList.

[0174] Next, the IBC prediction block vector supplementing units 373 and 473 add block vectors of predetermined values, such as (0, 0), until the prediction block vector candidate list mvpListL is filled (S4804 in FIG. 47).

[0175] <Merge intra block copy mode derivation unit> The intra block copy prediction unit 352 in Figure 42 includes an IBC spatial block vector candidate derivation unit 371, an IBC history block vector candidate derivation unit 372, an IBC block vector supplementation unit 373, a reference position correction unit 380, a reference area boundary correction unit 381, an IBC merge candidate selection unit 374, and an IBC prediction mode determination unit 377.

[0176] The intra block copy prediction unit 362 in Figure 43 includes an IBC spatial block vector candidate derivation unit 471, an IBC history block vector candidate derivation unit 472, an IBC block vector supplementation unit 473, an IBC merge candidate selection unit 474, a reference position correction unit 480, a reference area boundary correction unit 481, and a block copy unit 477.

[0177] Figure 46 is a flowchart illustrating the procedure of a merge intra block copy mode derivation process having a function common to the intra block copy prediction unit 352 of the video encoding device and the intra block copy prediction unit 362 of the video decoding device according to an embodiment of the present invention.

[0178] The intra block copy prediction unit 352 and the intra block copy prediction unit 362 each include a merge intra block copy candidate list mergeIbcCandList. The merge intra block copy candidate list mergeIbcCandList has a list structure and includes a storage area for storing merge indices indicating the location within a merge intra block copy candidate and merge intra block copy candidates corresponding to the indices as elements. The merge index numbers start from 0, and merge intra block copy candidates are stored in the storage area of ​​the merge intra block copy candidate list mergeIbcCandList. In the following processing, the merge candidate with merge index i registered in the merge intra block copy candidate list mergeIbcCandList will be represented by mergeIbcCandList[i]. In this embodiment, the merge candidate list mergeCandList can register at least three merge intra block copy candidates. Furthermore, the variable numCurrMergeIbcCand, which indicates the number of merge intra block copy candidates registered in the merge intra block copy candidate list mergeIbcCandList, is set to 0.

[0179] The IBC spatial block vector candidate derivation unit 371 and the IBC spatial block vector candidate derivation unit 471 derive spatial merge candidates A and B from the blocks adjacent to the left and above the current block from the coding information stored in the coding information storage memory 111 of the video coding device or the coding information storage memory 205 of the video decoding device, and register the derived spatial merge candidates in a merge intra block copy candidate list mergeIbcCandList (step S4701 in FIG. 46 ). Here, N is defined as indicating either spatial merge candidate A or B. A flag availableFlagN indicating whether the intra block copy prediction information of block N can be used as spatial block vector merge candidate N and a block vector mvL are derived. However, in this embodiment, the block vector merge candidate is derived without reference to other coding blocks included in the block containing the current coding block, and therefore spatial block vector merge candidates included in the block containing the current coding block are not derived.

[0180] Next, the IBC history block vector candidate derivation unit 372 and the IBC history block vector candidate derivation unit 472 add the history prediction block vector candidate registered in the history prediction block vector candidate list HmvpIbcCandList to the merge intra block copy candidate list mergeIbcCandList (step S4702 in FIG. 46 ). In this embodiment, if the block vector already added to mergeIbcCandList and the block vector of the history prediction block vector candidate have the same value, the block vector is not added to mergeIbcCandList.

[0181] Next, if the number of merge candidates registered in the merge intra block copy candidate list mergeIbcCandList (numCurrMergeIbcCand) is smaller than the maximum number of intra block merge candidates (MaxNumMergeIbcCand), the IBC prediction block vector filling unit 373 and the IBC prediction block vector filling unit 473 derive additional intra block merge candidates such that the number of merge candidates registered in the merge intra block copy candidate list mergeIbcCandList is equal to or greater than the maximum number of merge candidates (MaxNumMergeIbcCand) and register the additional intra block merge candidates in the merge intra block copy candidate list mergeIbcCandList (step S4703 in FIG. 46 ). A block vector having a value of (0, 0) is added to the merge intra block copy candidate list mergeIbcCandList, with the maximum number of merge candidates (MaxNumMergeIbcCand) as the upper limit.

[0182] Next, the IBC merge candidate selection unit 374 and the IBC merge candidate selection unit 474 select one of the intra block merge candidates registered in the merge intra block copy candidate list mergeIbcCandList (step S4704 in FIG. 46 ). The IBC merge candidate selection unit 374 obtains the decoded image at the reference position from the decoded image memory 104, calculates the code amount and distortion amount, and selects a merge candidate, and supplies a merge index indicating the selected intra block merge candidate to the IBC prediction mode determination unit 377. The IBC prediction mode determination unit 377 calculates the code amount and distortion amount to determine whether or not to use the merge mode, and supplies the result to the prediction method determination unit 105. Meanwhile, the IBC merge candidate selection unit 474 on the decoding side selects an intra block merge candidate based on the decoded merge index, and supplies the selected intra block merge candidate to the reference position correction unit 480.

[0183] Next, the reference position correction unit 380 and the reference position correction unit 480 perform processing to correct the reference positions of the intra block merge candidates (step S4705 in FIG. 46). Details of the processing by the reference position correction unit 380 and the reference position correction unit 480 will be described later.

[0184] Next, the reference area boundary correction unit 381 and the reference area boundary correction unit 481 perform processing to correct the reference area boundary for the intra block merge candidate (step S4706 in FIG. 46). Details of the processing by the reference position correction unit 381 and the reference position correction unit 481 will be described later.

[0185] The block copy unit 477 obtains the decoded image at the reference position from the decoded image memory 208, and supplies it to the decoded image signal superimposing unit 207. Here, the block copy unit 477 copies the luminance component and the color difference component.

[0186] The above block vector mvL indicates the block vector of luminance. When the chrominance format is 420, the block vector mvC of chrominance is mvC = ((mvL >> (3 + 2)) * 32). Each of the x and y components of mvC is processed using the above formula.

[0187] 48 is a flowchart illustrating the processing of the reference position correction unit 380 and the reference position correction unit 480. Now, it is assumed that the unit of the intra block copy reference block is a coding tree block (CTU), and that the size is not 128x128 pixels.

[0188] First, the positions of the top left and bottom right of the reference block are calculated (S6001). The reference block refers to a block that the current coding block references using a block vector. If the top left of the reference block is (xRefTL, yRefTL) and the bottom right is (xRefBR, yRefBR), then (xRefTL, yRefTL) = (xCb + (mvL[0] >> 4), yCb + (mvL[1] >> 4)) (xRefBR, yRefBR) = (xRefTL + cbWidth - 1, yRefTL + cbHeight - 1). Here, the position of the current coding block is (xCb, yCb), the block vector is (mvL[0], mvL[1]), the width is cbWidth, and the height is cbHeight. Next, it is determined whether the size of the CTU is 128x128 pixels (S6002). Since the size is not 128x128 pixels (S6002: NO), the positions of the top left and bottom right of the accessible area are calculated (S6003). If the top left of the accessible area is (xAvlTL, yAvlTL) and the bottom right is (xAvlBR, yAvlBR), then NL = Min(1, 7 - CtbLog2SizeY) - (1 << ((7 - CtbLog2SizeY) << 1)) (xAvlTL, yAvlTL) = (((xCb >> CtbLog2SizeY) + NL) << CtbLog2SizeY, (yCb >> CtbLog2SizeY) << CtbLog2SizeY) (xAvlBR, yAvlBR) = (((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1, (((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) - 1). Here, the size of the CTU is CtbLog2SizeY.

[0189] Next, it is determined whether the reference position in the x direction of the reference block is smaller than the top left corner of the referenceable area (S6004). If the determination is false (S6004: NO), the process proceeds to the next step (S6006). On the other hand, if the determination is true (S6004: YES), the reference position in the x direction is corrected to match the top left corner of the referenceable area (S6005).

[0190] FIG. 49 is a diagram showing how the reference position is corrected. 6001 indicates the current coding tree block, 6002 indicates the current coding block, and 6003 indicates the referenceable area. If reference block r2 is located at 6011, the reference position in the x direction is smaller than the upper left corner of the referenceable area (S6004: YES). Therefore, xRefTL = xAvlTL, and the reference position is corrected to position 6012 (S6005). Here, since xRefBR = xRefTL + cbWidth - 1 as shown in S6001, correcting xRefTL also results in correcting xRefBR. In correcting the reference position, the block vector mvL[0] may also be corrected. That is, the correction is made so that mvL[0] = (xAvlTL - xCb) << 4. This results in xRefTL = xAvlTL, and the reference position can be corrected.

[0191] In this way, if a reference block is located outside the referenceable area, it becomes referenceable by correcting the reference position.

[0192] Now, suppose that some block vectors in the block vector candidate list constructed by the intra block copy prediction unit 352 are outside the referenceable area. If the reference positions are not corrected, those block vectors cannot be referenced, and therefore those block vectors cannot be used as candidates for the IBC merge mode. On the other hand, if the reference positions are corrected in the present invention, all block vectors in the constructed block vector candidate list will be inside the referenceable area. Therefore, all block vectors can be referenced, and all block vectors can be used as candidates for the IBC merge mode. Therefore, the IBC merge mode selection unit 374 can select the optimal prediction mode from the IBC merge mode candidates corresponding to all block vectors, thereby improving coding efficiency.

[0193] Suppose that some block vectors in the block vector candidate list constructed by the intra block copy prediction unit 362 are outside the referenceable area. If the reference positions are not corrected, references using those block vectors are impossible, and the IBC merge mode using those block vectors cannot be decoded. In encoding devices other than those of the present invention, merge indices indicating the IBC merge mode using those block vectors are treated as not being coded. However, due to an operational malfunction or other reasons, such merge indices may be coded to generate a bitstream. Alternatively, packet loss or other factors may cause a portion of the bitstream to be missing, resulting in the decoded result being such a merge index. When attempting to decode such an incomplete bitstream, there is a possibility that an attempt to reference outside the referenceable area will result in accessing the decoded image memory at an incorrect location. As a result, the decoding results may differ depending on the decoding device, or the decoding process may stop. On the other hand, when the reference positions are corrected in accordance with the present invention, all block vectors in the constructed block vector candidate list will be inside the referenceable area. Therefore, even when such an incomplete bitstream is decoded, the reference positions are corrected to within the referenceable area, making reference possible. Correcting the reference positions in this way guarantees the memory access range. As a result, the decoding results are the same depending on the decoding device, and the decoding process can be continued, thereby improving the robustness of the decoding device.

[0194] Furthermore, when correcting a block vector in the correction of the reference position, the target is the luminance block vector. Here, the chrominance block vector is calculated from the luminance block vector. In other words, correcting the luminance block vector also corrects the chrominance block vector. Therefore, there is no need to correct the reference position again for chrominance. This reduces the amount of processing compared to when it is necessary to determine whether reference is possible for both luminance and chrominance when the block vector is not corrected.

[0195] In addition, when a block vector is corrected in correcting the reference position, the corrected block vector is stored in the coding information storage memory 111 or the coding information storage memory 205 as the block vector of the coding block to be processed. In other words, the corrected reference position and the position indicated by the block vector are the same. Here, a deblocking filter process may be performed when storing the decoding result in the decoded image memory. In this filter process, the filter strength is controlled based on the difference between the block vectors of two blocks facing the block boundary. Compared to when the block vector is not corrected, in which case the corrected reference position and the position indicated by the block vector are different, a more appropriate filter strength is obtained, thereby improving coding efficiency.

[0196] Next, it is determined whether the reference position in the y direction of the reference block is smaller than the top left corner of the referenceable area (S6006). If the determination is false (S6006: NO), the process proceeds to the next step (S6008). On the other hand, if the determination is true (S6006: YES), the reference position in the y direction is corrected to match the top left corner of the referenceable area (S6007).

[0197] If reference block r4 is located at 6021, the reference position in the y direction is smaller than the upper left corner of the referenceable area (S6006: YES). Therefore, the reference position is corrected to 6022 by setting yRefTL = yAvlTL (S6007). Here, as in S6001, yRefBR = yRefTL + cbHeight - 1, so correcting yRefTL also results in correcting yRefBR. In correcting this reference position, the block vector mvL[1] may also be corrected. In other words, the correction is made so that mvL[1] = (yAvlTL - yCb) << 4. As a result, yRefTL = yAvlTL, and the reference position can be corrected.

[0198] Next, it is determined whether the reference position in the x direction of the reference block is greater than the bottom right corner of the referenceable area (S6008). If the determination is false (S6008: NO), the process proceeds to the next step (S6010). On the other hand, if the determination is true (S6008: YES), the reference position in the x direction is corrected to match the bottom right corner of the referenceable area (S6009).

[0199] If reference block r7 is located at 6031, the reference position in the x direction is larger than the bottom right corner of the referenceable area (S6008: YES). Therefore, xRefBR = xAvlBR, and the reference position is corrected to 6032 (S6009). Here, as in S6001, xRefBR = xRefTL + cbWidth-1, that is, xRefTL = xRefBR-(cbWidth-1), so correcting xRefBR also corrects xRefTL. In correcting this reference position, the block vector mvL[0] may also be corrected. In other words, the correction is made as follows: mvL[0] = (xAvlBR - (xCb + cbWidth - 1)) << 4. This results in xRefBR = xAvlBR, and the reference position can be corrected.

[0200] Next, it is determined whether the reference position in the y direction of the reference block is greater than the bottom right corner of the referenceable area (S6010). If the determination is false (S6010: NO), the process ends. On the other hand, if the determination is true (S6010: YES), the reference position in the y direction is corrected to match the bottom right corner of the referenceable area (S6011).

[0201] If reference block r5 is located at 6041, the reference position in the y direction is larger than the bottom right corner of the referenceable area (S6010: YES). Therefore, yRefBR = yAvlBR, and the reference position is corrected to 6042 (S6011). Here, as in S6001, yRefBR = yRefTL + cbHeight-1, that is, yRefTL = yRefBR-(cbHeight-1), so correcting yRefBR also corrects yRefTL. In correcting this reference position, the block vector mvL[1] may also be corrected. In other words, the correction is made as follows: mvL[1] = (yAvlBR - (yCb + cbHeight - 1)) << 4. This results in yRefBR = yAvlBR, and the reference position can be corrected.

[0202] Here, we will explain the case where reference block r1 is located at 6051. In this case, the reference position in the x direction is corrected, just as when the reference block is r2. Furthermore, the reference position in the y direction is corrected, just as when the reference block is r4. As a result, reference block r1 is located at 6052, which is inside the referenceable area.

[0203] If reference block r3 is located at 6061, if reference block r6 is located at 6062, or if reference block r8 is located at 6063, the reference positions in the x and y directions are corrected in the same manner as above. As a result, each reference block is located within the referenceable area.

[0204] This completes the process if the CTU size is not 128x128 pixels. On the other hand, if the CTU size is 128x128 pixels (S6002: YES), the upper left and lower right positions of the referenceable area are calculated when it is rectangular (S6012).

[0205] 50 is a diagram illustrating the positions of the upper left and lower right when the referenceable area is rectangular. In the case of FIG. 50A, the coding tree block 6101 to be processed is divided into four, and the coding block 6102 to be processed is located at the upper left of the division. In this case, the referenceable area has an inverted L shape, as shown by the hatched area in 6103. When the referenceable area is rectangular, its range is the rectangular range of 6103. If the referenceable area is rectangular, and the top left of the reference block is (xRefTL, yRefTL) and the bottom right is (xRefBR, yRefBR), then offset[4] = {0, 64, 128, 128} NL = -offset[3 - blk_idx], NR = offset[blk_idx] ( xAvlTL, yAvlTL ) = ( (xCb >> CtbLog2SizeY) << CtbLog2SizeY + NL, (yCb >> CtbLog2SizeY) << CtbLog2SizeY ) ( xAvlBR, yAvlBR ) = ( ((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1 + NR, (((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) - 1 ). Here, blk_idx is an index indicating the position of the coding block to be processed. When the coding tree block to be processed is divided into four, and the coding block to be processed is located in the upper left, blk_idx = 0. Similarly, when the coding block to be processed is located in the upper right, lower left, or lower right, blk_idx is set to 1, 2, or 3. Figure 50A is a diagram showing the case where blk_idx = 0. Similarly, Figures 50B to 50D are diagrams showing the cases where blk_idx = 1 to 3, respectively.

[0206] Next, the reference position of the non-rectangular referenceable area is corrected (S6013). Figure 51 is a flowchart explaining the process for correcting the reference position of the non-rectangular referenceable area. First, the position of the upper left corner of the referenceable area is calculated (S6021). Since the referenceable area is the shaded area in Figure 50, there are two upper left corner positions, 6111 and 6112, except when blk_idx=3. If these are (X1, Y1) and (X2, Y2), then: offset[4] = {64, 128, 64, 0}, NL = offset[blk_idx] (X1, Y1) = (xAvlTL, yAvlTL + 64) (X2, Y2) = (xAvlTL + NL, yAvlTL).

[0207] Next, it is determined whether or not to correct the reference position to match the top left corner of the referenceable area (S6022). In this determination, if blk_idx=3 is not true and the reference block is located in an area smaller than X2 and Y1, it is determined to be true (S6022: YES). If false (S6022: NO), proceed to the next process (S6026).

[0208] Next, it is determined whether the difference between the reference block and the referenceable area in the x direction is smaller than the difference between the reference block and the referenceable area in the y direction (S6023). If the determination is true (S6023: YES), the reference position in the x direction is corrected (S6024). On the other hand, if the determination is false (S6023: NO), the reference position in the y direction is corrected (S6025).

[0209] FIG. 52A shows how the reference position is corrected in S6024 and S6025. Currently, blk_idx = 0. If reference block r1 is located at 6201, blk_idx is not 3, and the upper left corner of the reference block is located in an area smaller than X2 (the x-direction of 6112) and Y1 (the y-direction of 6111) (S6022: YES). Furthermore, the difference between the reference block and the referenceable area in the x-direction is smaller than the difference between the reference block and the referenceable area in the y-direction (S6023: YES). Therefore, xRefTL = xAvlTL + NL, and the reference position in the x-direction is corrected to 6202 (S6024). Here, since xRefBR = xRefTL + cbWidth-1 as in S6001, xRefBR is also corrected in conjunction with the correction of xRefTL. In this correction of the reference position, the block vector mvL[0] may also be corrected. In other words, the correction is mvL[0] = (xAvlTL + NL - xCb) << 4. This makes xRefTL=xAvlTL+NL, so the reference position can be corrected.

[0210] On the other hand, if reference block r2 is located at 6203, blk_idx is not 3, and the upper left corner of the reference block is located in an area smaller than X2 (x direction of 6112) and Y1 (y direction of 6111) (S6022: YES). Furthermore, the difference between the reference block and the referenceable area in the x direction is not smaller than the difference between the reference block and the referenceable area in the y direction (S6023: NO). Therefore, the reference position in the y direction is corrected to 6204 by setting yRefTL = yAvlTL + 64 (S6025). Here, since yRefBR = yRefTL + cbHeight - 1 as in S6001, correcting yRefTL also results in correcting yRefBR. When correcting this reference position, the block vector mvL[0] may also be corrected. That is, the correction is made as follows: mvL[1] = (yAvlTL + 64 - yCb) << 4. As a result, yRefTL=yAvlTL+64 is obtained, and the reference position can be corrected.

[0211] Now, let's assume that reference block r3 is located at 6205. In this case, the difference between the reference block and the referenceable area in the x direction is smaller than the difference between the reference block and the referenceable area in the y direction (S6023: YES). Therefore, by correcting the reference position in the x direction, as with reference block r1, the reference block is positioned at 6206 (S6024). At this point, the reference block is outside the referenceable area. However, the reference position in the y direction is corrected by the processes of S6006 and S6007, which will be described later. Ultimately, the reference block is located inside the referenceable area.

[0212] Next, the position of the bottom right corner of the referenceable area is calculated (S6026). Since the referenceable area is the shaded area in Figure 50, there are two points 6113 and 6114 for the bottom right corner, except when blk_idx = 0. If these are (X3, Y3) and (X4, Y4), then offset[4] = {0, 64, 128, 64}, NR = offset[blk_idx] (X3, Y3) = (xAvlBR, yAvlBR - 64) (X4, Y4) = (xAvlBR - NR, yAvlBR).

[0213] Next, it is determined whether or not to correct the reference position to match the bottom right of the referenceable area (S6027). In this determination, if blk_idx=0 is not satisfied and the reference block is located in an area larger than X4 and Y3, it is determined to be true (S6027: YES). If false (S6027: NO), the process ends.

[0214] Next, it is determined whether the difference between the reference block and the referenceable area in the x direction is smaller than the difference between the reference block and the referenceable area in the y direction (S6028). If the determination is true (S6028: YES), the reference position in the x direction is corrected (S6029). On the other hand, if the determination is false (S6028: NO), the reference position in the y direction is corrected (S6030).

[0215] FIG. 52B shows how the reference position is corrected in S6029 and S6030. Currently, blk_idx = 3. If reference block r1 is located at 6211, blk_idx is not 0, and the bottom right corner of the reference block is located in an area greater than X4 (the x-direction of 6114) and Y3 (the y-direction of 6113) (S6027: YES). Furthermore, the difference between the reference block and the referenceable area in the x-direction is smaller than the difference between the reference block and the referenceable area in the y-direction (S6028: YES). Therefore, xRefBR = xAvlBR, and the reference position in the x-direction is corrected to 6212 (S6029). Here, as in S6001, xRefBR = xRefTL + cbWidth-1, i.e., xRefTL = xRefBR-(cbWidth-1), and therefore xRefTL is also corrected when xRefBR is corrected. When correcting this reference position, the block vector mvL[0] may be corrected. That is, the correction is made as follows: mvL[0] = (xAvlBR - NR - (xCb + cbWitdh - 1)) << 4. This makes xRefBR=xAvlBR, so the reference position can be corrected.

[0216] On the other hand, if reference block r2 is located at 6213, blk_idx is not 0, and the bottom right corner of the reference block is located in an area larger than X4 (the x-direction of 6114) and Y3 (the y-direction of 6113) (S6027: YES). Furthermore, the difference between the reference block and the referenceable area in the x-direction is not smaller than the difference between the reference block and the referenceable area in the y-direction (S6028: NO). Therefore, the reference position in the y-direction is corrected to 6214 by setting yRefBR = yAvlBR (S6030). Here, as in S6001, yRefBR = yRefTL + cbHeight-1, i.e., yRefTL = yRefBR - (cbHeight-1), so correcting yRefBR also results in correcting yRefTL. When correcting this reference position, the block vector mvL[1] may also be corrected. In other words, the correction is mvL[1] = (yAvlBR - 64 - (yCb + cbHeight - 1)) << 4. This makes yRefBR=yAvlBR, so the reference position can be corrected.

[0217] Now, let's assume that reference block r3 is located at 6215. In this case, the difference between the reference block and the referenceable area in the x direction is not smaller than the difference between the reference block and the referenceable area in the y direction (S6028: NO). Therefore, by correcting the reference position in the y direction, as with reference block r2, the reference block is positioned at 6216 (S6030). At this point, the reference block is outside the referenceable area. However, the reference position in the x direction is corrected by the processes of S6008 and S6009, which will be described later. Ultimately, the reference block is located inside the referenceable area.

[0218] 52, the process of correcting the reference position has been described using the example of blk_idx=0 and 3. When blk_idx=1 or 2, the process of correcting the reference position is performed in the same way as when blk_idx=0 and 3.

[0219] After the process of correcting the reference position of the non-rectangular referable area (S6013), the processes from S6004 to S6011 are performed. This completes the process when the CTU size is 128x128 pixels.

[0220] Now, suppose that in the process of correcting the reference position of a non-rectangular referenceable area (S6013), the reference position in the x direction is corrected to match the top left corner of the referenceable area (S6024). In this case, the reference position in the x direction of the reference block will never be smaller than the top left corner of the referenceable area, so the determination in S6004 will always be false (S6004: NO). Therefore, if S6024 is performed, S6004 and S6005 may be omitted. Similarly, if S6025 is performed, S6006 and S6007 may be omitted; if S6029 is performed, S6008 and S6009 may be omitted; and if S6030 is performed, S6010 and S6011 may be omitted.

[0221] 51, the comparison process of step S6023 may be omitted and step S6024 may always be executed, or step S6025 may always be executed. Similarly, the comparison process of step S6028 may be omitted and step S6029 may always be executed, or step S6030 may always be executed. In such a configuration, the reference position can be corrected by simple processing.

[0222] In Fig. 48, when the size of the CTU is 128 x 128 pixels, the reference position is corrected using the processes of S6012, S6013, and S6004 to S6011. Alternatively, as shown in Fig. 53, this can also be achieved by dividing the referenceable area into two and correcting the reference position of each (S6101).

[0223] FIG. 54 is a diagram illustrating how a referenceable area is decomposed into two. Unlike FIG. 50, in which the referenceable area is rectangular, in FIG. 54 the referenceable area is decomposed into two. If the coding tree block (6101) to be processed is divided into four and the coding block (6102) to be processed is located in the upper left, blk_idx=0. Similarly, if the coding block to be processed is located in the upper right, lower left, or lower right, respectively, blk_idx is set to 1, 2, or 3. FIG. 54A is a diagram illustrating the case where blk_idx=0. Similarly, FIGS. 54B to 54D are diagrams illustrating the cases where blk_idx=1 to 3, respectively. One referenceable area (6301) is referred to as referenceable area A, and the other referenceable area (6302) is referred to as referenceable area B.

[0224] Figure 55 is a flowchart explaining the process of dividing the referenceable area into two and correcting the reference position of each (S6101). In Figure 55, the same steps as in Figure 48 are given the same step numbers, and their explanations will be omitted. First, the positions of the top left and bottom right of the referenceable area A are calculated (S6111). If the top left of the accessible area A is (xAvlTL, yAvlTL) and the bottom right is (xAvlBR, yAvlBR), then xOffsetTL[4] = {-128, -128, -64, 0}, yOffsetTL[4] = {64, 64, 64, 0} xOffsetBR[4] = {0, 0, 0, 128}, yOffsetBR[4] = {128, 128, 128, 64} (xAvlTL, yAvlTL) = ((xCb >> CtbLog2SizeY) << CtbLog2SizeY + xOffsetTL[blk_idx], (yCb >> CtbLog2SizeY) << CtbLog2SizeY + yOffsetTL[blk_idx]) (xAvlBR, yAvlBR) = (((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1 + xOffsetBR[blk_idx], (((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) - 1 + yOffsetBR[blk_idx]).

[0225] Next, whether the reference block is outside the referenceable area A is calculated as follows (S6112): out_xRefTL=xRefTL<xAvlTL out_yRefTL=yRefTL<yAvlTL out_xRefBR=xRefBR>xAvlBR out_yRefBR=yRefBR>yAvlBR

[0226] Next, the top left and bottom right positions of the referable area B are calculated (S6113). If the top left of the accessible area B is (xAvlTL, yAvlTL) and the bottom right is (xAvlBR, yAvlBR), then xOffsetTL[4] = {-64, 0, 0, 0}, yOffsetTL[4] = {0, 0, 0, 0} xOffsetBR[4] = {0, 64, 128, 64}, yOffsetBR[4] = {128, 64, 64, 128} (xAvlTL, yAvlTL) = ((xCb >> CtbLog2SizeY) << CtbLog2SizeY + xOffsetTL[blk_idx], (yCb >> CtbLog2SizeY) << CtbLog2SizeY + yOffsetTL[blk_idx]) (xAvlBR, yAvlBR) = ( ((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1 + xOffsetBR[blk_idx], (((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) - 1 + yOffsetBR[blk_idx] ).

[0227] Next, it is determined whether the reference position in the x direction of the reference block is smaller than the top left corner of referenceable area A and smaller than the top left corner of referenceable area B (S6114). If the determination is false (S6114: NO), the process proceeds to the next step (S6116). On the other hand, if the determination is true (S6114: YES), the reference position in the x direction is corrected to match the top left corner of referenceable area B (S6005). The processing of S6005 has already been explained, so its explanation will be omitted.

[0228] Next, it is determined whether the reference position in the y direction of the reference block is smaller than the top left corner of referenceable area A and smaller than the top left corner of referenceable area B (S6116). If the determination is false (S6116: NO), the process proceeds to the next step (S6118). On the other hand, if the determination is true (S6116: YES), the reference position in the y direction is corrected to match the top left corner of referenceable area B (S6007). The process of S6007 has already been explained, so its explanation will be omitted.

[0229] Next, it is determined whether the reference position of the reference block in the x direction is greater than the bottom right corner of referenceable area A and greater than the bottom right corner of referenceable area B (S6118). If the determination is false (S6118: NO), the process proceeds to the next step (S6120). On the other hand, if the determination is true (S6118: YES), the reference position in the x direction is corrected to match the bottom right corner of referenceable area B (S6009). The processing of S6009 has already been explained, so its explanation will be omitted.

[0230] Next, it is determined whether the reference position in the y direction of the reference block is greater than the bottom right corner of referenceable area A and greater than the bottom right corner of referenceable area B (S6120). If the determination is false (S6120: NO), processing ends. On the other hand, if the determination is true (S6120: YES), the reference position in the y direction is corrected to match the bottom right corner of referenceable area B (S6011). The processing of S6011 has already been explained, so its explanation will be omitted.

[0231] As a result, when the size of the CTU is 128 x 128 pixels, even if the reference block is located outside the referenceable area, it can be referenced by correcting the reference position. Furthermore, by dividing the referenceable area into two and correcting the reference position of each, it is possible to simplify processing and reduce the amount of calculation. Here, one referenceable area (6301) is designated as referenceable area A, and the other referenceable area (6302) is designated as referenceable area B. Alternatively, it is also possible to swap referenceable area A and referenceable area B, and process one referenceable area (6301) as referenceable area B, and the other referenceable area (6302) as referenceable area A.

[0232] In this embodiment, it is determined whether the size of the CTU is 128x128 pixels (S6002) and the process is switched accordingly. This may be done by determining whether the intra block copy reference block is a unit obtained by dividing the coding tree block into four, or by determining whether the size of the CTU is larger than the maximum size of the coding block.

[0233] All of the above-described embodiments may be combined in multiple ways.

[0234] In all of the above-described embodiments, the bitstream output by the image coding device has a specific data format so that it can be decoded according to the coding method used in the embodiment. The bitstream may be provided by being recorded on a computer-readable recording medium such as an HDD, SSD, flash memory, or optical disk, or may be provided from a server via a wired or wireless network. Therefore, an image decoding device corresponding to this image coding device can decode a bitstream in this specific data format regardless of the providing means.

[0235] When a wired or wireless network is used to exchange bitstreams between an image encoding device and an image decoding device, the bitstreams may be converted into a data format suitable for the transmission mode of the communication channel before transmission. In this case, a transmitting device is provided that converts the bitstream output by the image encoding device into coded data in a data format suitable for the transmission mode of the communication channel and transmits the coded data to the network, and a receiving device is provided that receives the coded data from the network, restores the coded data to a bitstream, and supplies the bitstream to the image decoding device. The transmitting device includes a memory that buffers the bitstream output by the image encoding device, a packet processing unit that packetizes the bitstream, and a transmitting unit that transmits the packetized coded data via the network. The receiving device includes a receiving unit that receives the packetized coded data via the network, a memory that buffers the received coded data, and a packet processing unit that packetizes the coded data to generate a bitstream and provides it to the image decoding device.

[0236] When a wired or wireless network is used to exchange bitstreams between an image encoding device and an image decoding device, in addition to a transmitting device and a receiving device, a relay device may be provided that receives coded data transmitted by the transmitting device and supplies the coded data to the receiving device. The relay device includes a receiving unit that receives packetized coded data transmitted by the transmitting device, a memory that buffers the received coded data, and a transmitting unit that transmits the packetized coded data to the network. The relay device may further include a receiving packet processing unit that processes the packetized coded data into packets to generate a bitstream, a recording medium that stores the bitstream, and a transmitting packet processing unit that packetizes the bitstream.

[0237] Furthermore, a display unit for displaying images decoded by the image decoding device may be added to the configuration to form a display device. In this case, the display unit reads out the decoded image signal generated by the decoded image signal superimposition unit 207 and stored in the decoded image memory 208, and displays the decoded image signal on a screen.

[0238] Furthermore, an imaging unit may be added to the configuration and the captured image may be input to the image coding device, thereby forming an imaging device. In this case, the imaging unit inputs the captured image signal to the block division unit 101.

[0239] 37 shows an example of the hardware configuration of a coding / decoding device according to this embodiment. The coding / decoding device includes the configurations of the image coding device and image decoding device according to the embodiment of the present invention. The coding / decoding device 9000 includes a CPU 9001, a codec IC 9002, an I / O interface 9003, a memory 9004, an optical disk drive 9005, a network interface 9006, and a video interface 9009, and each unit is connected by a bus 9010.

[0240] The image encoding unit 9007 and the image decoding unit 9008 are typically implemented as a codec IC 9002. The image encoding process of the image encoding device according to the embodiment of the present invention is performed by the image encoding unit 9007, and the image decoding process of the image decoding device according to the embodiment of the present invention is performed by the image decoding unit 9008. The I / O interface 9003 is realized by, for example, a USB interface, and is connected to an external keyboard 9104, mouse 9105, etc. The CPU 9001 controls the encoding / decoding device 9000 to perform an operation desired by the user based on user operations input via the I / O interface 9003. User operations via the keyboard 9104, mouse 9105, etc. include selecting whether to execute encoding or decoding functions, setting encoding quality, bitstream input / output destinations, image input / output destinations, etc.

[0241] When a user desires to play back an image recorded on the disc recording medium 9100, the optical disc drive 9005 reads a bitstream from the inserted disc recording medium 9100 and sends the read bitstream to the image decoding unit 9008 of the codec IC 9002 via the bus 9010. The image decoding unit 9008 performs image decoding processing on the input bitstream in the image decoding device according to an embodiment of the present invention and sends the decoded image to an external monitor 9103 via a video interface 9009. The encoding / decoding device 9000 also has a network interface 9006 and is connectable to an external distribution server 9106 or a mobile terminal 9107 via a network 9101. When a user desires to play back an image recorded on the distribution server 9106 or the mobile terminal 9107 instead of the image recorded on the disc recording medium 9100, the network interface 9006 acquires the bitstream from the network 9101 instead of reading the bitstream from the input disc recording medium 9100. Furthermore, when the user wishes to play back the image recorded in memory 9004, an image decoding process is performed on the bitstream recorded in memory 9004 in an image decoding device according to an embodiment of the present invention.

[0242] When the user desires to encode an image captured by an external camera 9102 and record the encoded image in memory 9004, the video interface 9009 inputs the image from the camera 9102 and sends it to the image encoding unit 9007 of the codec IC 9002 via the bus 9010. The image encoding unit 9007 performs image encoding processing in the image encoding device according to an embodiment of the present invention on the image input via the video interface 9009 to create a bitstream. The bitstream is then sent to the memory 9004 via the bus 9010. When the user desires to record the bitstream on a disc recording medium 9100 instead of the memory 9004, the optical disc drive 9005 writes the bitstream to the inserted disc recording medium 9100.

[0243] It is also possible to realize a hardware configuration that has an image encoding device but not an image decoding device, or a hardware configuration that has an image decoding device but not an image encoding device. Such a hardware configuration can be realized, for example, by replacing the codec IC 9002 with an image encoding unit 9007 or an image decoding unit 9008, respectively.

[0244] The above encoding and decoding processes may be realized not only as a transmission, storage, and receiving device using hardware, but also as firmware stored in a ROM (read-only memory), flash memory, etc., or as software for a computer, etc. The firmware program or software program may be provided by recording it on a computer-readable recording medium, or may be provided from a server via a wired or wireless network, or may be provided as data broadcasting on terrestrial or satellite digital broadcasting.

[0245] The present invention has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of the components and treatment processes, and that such modifications are also within the scope of the present invention.

[0246] The present invention can be used in image encoding and decoding techniques in which an image is divided into blocks and prediction is performed.

[0247] 100 Image encoding device, 101 Block division unit, 102 Inter prediction unit, 103 Intra prediction unit, 104 Decoded image memory, 105 Prediction method determination unit, 106 Residual generation unit, 107 Orthogonal transform and quantization unit, 108 Bit string encoding unit, 109 Inverse quantization and inverse orthogonal transform unit, 110 Decoded image signal superimposition unit, 111 Encoding information storage memory, 200 Image decoding device, 201 Bit string decoding unit, 202 Block division unit, 203 Inter prediction unit, 204 Intra prediction unit, 205 Encoding information storage memory, 206 Inverse quantization and inverse orthogonal transform unit, 207 Decoded image signal superimposition unit, 208 Decoded image memory.

Claims

DEPCT6503 / 03 / 25651. Picture coding device comprising: a block vector selection unit constructed to convert the block vector selection of the target block in the target image from the encoding information stored in memory; a selector encoding information unit constructed to select the selected block vector from the block vector selections; and a reference position correction unit constructed to perform corrections to the referenced block references of the selected block vector, thus the reference position of the referenced block is corrected to refer to the inside of the referenced region where the decoded sample in the target image is retrieved from the decoded picture memory as the predicted value of the target block based on the reference position of the referenced block.2.The image encoding method includes: a vector block selection step, which transforms the target block in the target image from the encoding information stored in memory; a selection step, which selects the chosen vector block from the selection block vectors; and a reference position correction step, which corrects the reference position of the referenced block referenced by the selected vector block. Therefore, the reference position of the referenced block is corrected to refer to the inside of the referenced region where the decoded sample in the target image is retrieved from the decoded image memory, and the predicted value of the target block is determined based on the reference position of the referenced block.The image encoding program comprises: a vector block selection step, which transforms the vector block selection of the target block in the target image from the encoding information stored in memory; a selection step, which selects the chosen vector block from the selection block selections; and a reference position correction step, which performs corrections on the referenced block referenced by the selected vector block. Thus, the reference position of the referenced block is corrected to refer to the inside of the referenced region, where the decoded sample in the target image is retrieved from the decoded image memory, and the predicted value of the target block is determined based on the reference position of the referenced block.The image decoding device comprises: a vector block selection unit built to convert the vector block selection of the target block in the target image from the encoding information stored in memory; a selector encoding information unit structured to select the chosen vector block from the vector block selections; and a reference position correction unit built to perform corrections on the reference blocks referenced by the selected vector block. Thus, the reference position of the reference block is corrected to refer to the inside of the referenced region where the decoded sample in the target image is retrieved from the decoded image memory, based on the predicted value of the target block and the reference position of the reference block.The image decoding method comprises: a vector block selection step, which transforms the target block in the target image from the encoding information stored in memory; a selection step, which selects the chosen vector block from the selection block vectors; and a reference position correction step, which performs corrections on the referenced block referenced by the selected vector block. Thus, the reference position of the referenced block is corrected to refer to the inside of the reference region, where the decoded sample in the target image is retrieved from memory. The decoded image then yields the predicted value of the target block based on the reference position of the referenced block.The image decoding program comprises: a vector block selection step, which transforms the target block in the target image from the encoding information stored in memory; a selection step, which selects the chosen vector block from the selection block vectors; and a reference position correction step, which corrects the reference position of the referenced block referenced by the selected vector block. Thus, the reference position of the referenced block is corrected to refer to the inside of the referenced region, where the decoded sample in the target image is retrieved from the decoded image memory, and the predicted value of the target block is determined based on the reference position of the referenced block.-----------------------------------------------------------DEPCT651.The picture coding device comprises: a block vector selection unit built to convert the block vector selection of the target block in the target image from the encoding information stored in memory; a selector encoding information storage unit structured to select the selected block vector from the block vector selections; and a reference position correction unit built to perform corrections to the block references referenced by the selected block vector. Thus, the reference position of the reference block is corrected to refer to the inside of the referenced region where the decoded sample in the target image is retrieved from the decoded picture memory, based on the predicted value of the target block on the basis of the reference position of the reference block.The image encoding method includes: a vector block selection step, which transforms the target block in the target image from the encoding information stored in memory; a selection step, which selects the chosen vector block from the selection block vectors; and a reference position correction step, which corrects the reference position of the referenced block referenced by the selected vector block. Therefore, the reference position of the referenced block is corrected to refer to the inside of the referenced region where the decoded sample in the target image is retrieved from the decoded image memory, and the predicted value of the target block is determined based on the reference position of the referenced block.The image encoding program comprises: a vector block selection step, which transforms the vector block selection of the target block in the target image from the encoding information stored in memory; a selection step, which selects the chosen vector block from the selection block selections; and a reference position correction step, which performs corrections on the referenced block referenced by the selected vector block. Thus, the reference position of the referenced block is corrected to refer to the inside of the referenced region, where the decoded sample in the target image is retrieved from the decoded image memory, and the predicted value of the target block is determined based on the reference position of the referenced block.The image decoding device comprises: a vector block selection unit built to convert the vector block selection of the target block in the target image from the encoding information stored in memory; a selector encoding information unit structured to select the chosen vector block from the vector block selections; and a reference position correction unit built to perform corrections on the reference blocks referenced by the selected vector block. Thus, the reference position of the reference block is corrected to refer to the inside of the referenced region where the decoded sample in the target image is retrieved from the decoded image memory, based on the predicted value of the target block and the reference position of the reference block.The image decoding method comprises: a vector block selection step, which transforms the target block in the target image from the encoding information stored in memory; a selection step, which selects the chosen vector block from the selection block vectors; and a reference position correction step, which performs corrections on the referenced block referenced by the selected vector block. Thus, the reference position of the referenced block is corrected to refer to the inside of the reference region, where the decoded sample in the target image is retrieved from memory. The decoded image then yields the predicted value of the target block based on the reference position of the referenced block.The image decoding program comprises: a vector block selection step, which transforms the target block in the target image from the encoding information stored in memory; a selection step, which selects the chosen vector block from the selection block vectors; and a reference position correction step, which corrects the reference position of the referenced block referenced by the selected vector block. Thus, the reference position of the referenced block is corrected to refer to the inside of the referenced region, where the decoded sample in the target image is retrieved from the decoded image memory, and the predicted value of the target block is determined based on the reference position of the referenced block.