Moving image encoding apparatus and method, and moving image decoding apparatus and method

Abstract

The present invention relates to an image encoding apparatus and method, and an image decoding apparatus and method. The image encoding apparatus includes: a block vector candidate derivation unit that derives block vector candidates for a processing target block within a processing target picture from encoding information stored in an encoding information storage memory; a selection unit that selects a selected block vector from the block vector candidates; and a reference position correction unit that corrects a reference position of a reference block referenced by the selected block vector so as to be within an accessible region. The image encoding apparatus obtains, as a prediction value for the processing target block, a decoded pixel within the processing target picture from a decoded image memory based on the reference position of the reference block.

Description

[0001] This application is a divisional application of an invention titled "Moving Image Encoding Apparatus, Moving Image Encoding Method, Moving Image Encoding Program, Moving Image Decoding Apparatus, Moving Image Decoding Method, and Moving Image Decoding Program" with an application number of 202080031476.5, a filing date of June 19, 2020, and an applicant of JVC Kenwood Corporation. Technical Field

[0002] The present invention relates to image encoding and decoding techniques for dividing an image into blocks and performing prediction. Background Art

[0003] In image encoding and decoding, an image to be processed is divided into a predetermined number of pixel sets, i.e., blocks, and processing is performed on a block-by-block basis. By dividing into appropriate blocks and appropriately setting intra-picture prediction (intra-frame prediction) and inter-picture prediction (inter-frame prediction), the encoding efficiency is improved.

[0004] Patent Document 1 discloses an intra-frame prediction technique that uses decoded pixels adjacent to a block to be encoded / decoded to obtain a predicted image.

[0005] Prior Art Documents

[0006] Patent Documents

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2009-246975. Summary of the Invention

[0008] However, the technique of Patent Document 1 only uses decoded pixels adjacent to a block to be encoded / decoded for prediction, resulting in poor prediction efficiency.

[0009] In a certain aspect of the present invention for solving the above problems, it includes: a block vector candidate derivation unit that derives block vector candidates for a processing target block in a processing target picture from encoding information stored in an encoding information storage memory; a selection unit that selects a selected block vector from the block vector candidates; and a reference position correction unit that corrects the reference position of a reference block referred to by the selected block vector so as to be within a referenceable region, and obtains decoded pixels in the processing target picture in the decoded image memory as a predicted value for the processing target block based on the reference position of the reference block.

[0010] According to the present invention, high-efficiency image encoding / decoding processing can be achieved with low load. Brief Description of the Drawings

[0011] Figure 1 is a block diagram of an image encoding apparatus according to an embodiment of the present invention;

[0012] Figure 2It is a block diagram of an image decoding device according to an embodiment of the present invention;

[0013] Figure 3 It is a flowchart for explaining the operation of dividing a tree block;

[0014] Figure 4 It is a diagram showing a case where an input image is divided into tree blocks;

[0015] Figure 5 It is a diagram for explaining z-scan;

[0016] Figure 6A It is a diagram showing the division shape of a block;

[0017] Figure 6B It is a diagram showing the division shape of a block;

[0018] Figure 6C It is a diagram showing the division shape of a block;

[0019] Figure 6D It is a diagram showing the division shape of a block;

[0020] Figure 6E It is a diagram showing the division shape of a block;

[0021] Figure 7 It is a flowchart for explaining the operation of dividing a block into four parts;

[0022] Figure 8 It is a flowchart for explaining the operation of dividing a block into two or three parts;

[0023] Figure 9 It is a syntax for describing the shape of block division;

[0024] Figure 10A It is a diagram for explaining intra prediction;

[0025] Figure 10B It is a diagram for explaining intra prediction;

[0026] Figure 11 It is a diagram for explaining a reference block for inter prediction;

[0027] Figure 12A It is a syntax for describing the prediction mode of an encoded block;

[0028] Figure 12B It is a syntax for describing the prediction mode of an encoded block;

[0029] Figure 13 It is a diagram showing the correspondence between syntax elements related to inter prediction and modes;

[0030] Figure 14It is a diagram for explaining affine transformation motion compensation with two control points;

[0031] Figure 15 It is a diagram for explaining affine transformation motion compensation with three control points;

[0032] Figure 16 It is Figure 1 a block diagram of the detailed structure of the inter-frame prediction unit 102;

[0033] Figure 17 It is Figure 16 a block diagram of the detailed structure of the normal prediction motion vector mode derivation unit 301;

[0034] Figure 18 It is Figure 16 a block diagram of the detailed structure of the normal merge mode derivation unit 302;

[0035] Figure 19 It is used to explain Figure 16 the flowchart of the normal prediction motion vector mode derivation process of the normal prediction motion vector mode derivation unit 301;

[0036] Figure 20 It is a flowchart showing the processing steps of the normal prediction motion vector mode derivation process;

[0037] Figure 21 It is a flowchart explaining the processing steps of the normal merge mode derivation process;

[0038] Figure 22 It is Figure 2 a block diagram of the detailed structure of the inter-frame prediction unit 203;

[0039] Figure 23 It is Figure 22 a block diagram of the detailed structure of the normal prediction motion vector mode derivation unit 401;

[0040] Figure 24 It is Figure 22 a block diagram of the detailed structure of the normal merge mode derivation unit 402;

[0041] Figure 25 It is used to explain Figure 22 the normal prediction motion vector mode derivation process of the normal prediction motion vector mode derivation unit 401;

[0042] Figure 26 It is a diagram explaining the processing steps of the historical prediction motion vector candidate list initialization / update process;

[0043] Figure 27 It is a flowchart of the same element confirmation processing step in the historical prediction motion vector candidate list initialization / update processing step;

[0044] Figure 28 It is a flowchart of the element shift processing step in the historical prediction motion vector candidate list initialization / update processing step;

[0045] Figure 29 It is a flowchart for explaining the historical prediction motion vector candidate derivation processing step;

[0046] Figure 30 It is a flowchart for explaining the historical merge candidate derivation processing step;

[0047] Figure 31A It is a diagram for explaining an example of the historical prediction motion vector candidate list update processing;

[0048] Figure 31B It is a diagram for explaining an example of the historical prediction motion vector candidate list update processing;

[0049] Figure 31C It is a diagram for explaining an example of the historical prediction motion vector candidate list update processing;

[0050] Figure 32 It is a diagram for explaining motion compensation prediction in the case where the reference picture (RefL0Pic) of L0 is before the picture to be processed (CurPic) at the time of L0 prediction;

[0051] Figure 33 It is a diagram for explaining motion compensation prediction in the case where the reference picture of L0 prediction is after the picture to be processed at the time of L0 prediction;

[0052] Figure 34 It is a diagram for explaining the prediction direction of motion compensation prediction in the case where the reference picture of L0 prediction is before the picture to be processed and the reference picture of L1 prediction is after the picture to be processed at the time of bi-prediction;

[0053] Figure 35 It is a diagram for explaining the prediction direction of motion compensation prediction in the case where the reference pictures of L0 prediction and L1 prediction are before the picture to be processed at the time of bi-prediction;

[0054] Figure 36 It is a diagram for explaining the prediction direction of motion compensation prediction in the case where the reference pictures of L0 prediction and L1 prediction are after the picture to be processed at the time of bi-prediction;

[0055] Figure 37 It is a diagram for explaining an example of the hardware structure of the encoding / decoding device according to an embodiment of the present invention;

[0056] Figure 38is a flowchart illustrating the average merge candidate derivation processing steps;

[0057] Figure 39A is a diagram illustrating the valid reference area for intra block copy;

[0058] Figure 39B is a diagram illustrating the valid reference area for intra block copy;

[0059] Figure 40 is Figure 1 a block diagram of the detailed configuration of the intra prediction unit 103 in

[0060] Figure 41 is Figure 2 a block diagram of the detailed configuration of the intra prediction unit 204 in

[0061] Figure 42 is a block diagram of the intra block copy prediction unit 352;

[0062] Figure 43 is a block diagram of the intra block copy prediction unit 362;

[0063] Figure 44 is a flowchart for explaining the predicted intra block copy processing of the intra block copy prediction unit 352;

[0064] Figure 45 is a flowchart for explaining the predicted intra block copy processing of the intra block copy prediction unit 362;

[0065] Figure 46 is a flowchart for explaining the merged intra block copy processing;

[0066] Figure 47 is a flowchart showing the processing steps of the block vector mode derivation processing for predicted intra block copy;

[0067] Figure 48 is a diagram explaining the processing of the reference position correction unit 380 and the reference position correction unit 480;

[0068] Figure 49 is a diagram showing the case of correcting the reference position;

[0069] Figure 50A is a diagram explaining the upper left and lower right positions in the case where the referenceable area is rectangular;

[0070] Figure 50B is a diagram explaining the upper left and lower right positions in the case where the referenceable area is rectangular;

[0071] Figure 50C is a diagram explaining the upper left and lower right positions in the case where the referenceable area is rectangular;

[0072] Figure 50D A diagram for explaining the upper left and lower right positions in the case where the reference region is set to a rectangular shape;

[0073] Figure 51 A diagram for explaining the process of correcting the reference position of a part where the reference region is not rectangular;

[0074] Figure 52A A diagram showing the situation of correcting the reference position;

[0075] Figure 52B A diagram showing the situation of correcting the reference position;

[0076] Figure 53 A diagram for explaining the processing of the reference position correction unit 380 and the reference position correction unit 480;

[0077] Figure 54A A diagram for explaining the situation of decomposing the reference region into two;

[0078] Figure 54B A diagram for explaining the situation of decomposing the reference region into two;

[0079] Figure 54C A diagram for explaining the situation of decomposing the reference region into two;

[0080] Figure 54D A diagram for explaining the situation of decomposing the reference region into two;

[0081] Figure 55 A diagram for explaining the process of decomposing the reference region into two and correcting each reference position. Detailed implementation mode

[0082] Define the technologies and technical terms used in this implementation mode.

[0083] <Tree block>

[0084] In the implementation mode, the image to be processed for encoding / decoding is equally divided into a predetermined size. This unit is defined as a tree block. In Figure 4In this case, the size of the tree block is set to 128×128 pixels, but the size of the tree block is not limited to this, and any size can be set. The tree block as the processing object (corresponding to the encoding object in the encoding process and the decoding object in the decoding process) is switched in the raster scan order, that is, in the order from left to right and from top to bottom. Further recursive partitioning can be performed inside each tree block. The block that becomes the encoding / decoding object after recursive partitioning of the tree block is defined as the encoding block. In addition, the tree block and the encoding block are collectively defined as the block. By performing appropriate block partitioning, high-efficiency encoding can be achieved. The size of the tree block can be a fixed value set in advance in the encoding device and the decoding device, or a structure can be adopted in which the size of the tree block determined by the encoding device is transmitted to the decoding device. Here, the maximum size of the tree block is set to 128×128 pixels, and the minimum size of the tree block is set to 16×16 pixels. In addition, the maximum size of the encoding block is set to 64×64 pixels, and the minimum size of the encoding block is set to 4×4 pixels.

[0085] <Prediction Mode>

[0086] The intra-frame prediction (MODE_INTRA) and the inter-frame prediction (MODE_INTER) are switched in units of the processing object encoding block. The intra-frame prediction (MODE_INTRA) is predicted based on the processed image signal of the processing object image, and the inter-frame prediction (MODE_INTER) is predicted based on the image signal of the processed image.

[0087] The processed image is used for the image, image signal, tree block, block, encoding block, etc. obtained by decoding the signal that has been encoded in the encoding process, and for the image, image signal, tree block, block, encoding block, etc. that have been decoded in the decoding process.

[0088] The mode that identifies the intra-frame prediction (MODE_INTRA) and the inter-frame prediction (MODE_INTER) is defined as the prediction mode (PredMode). The prediction mode (PredMode) represents the intra-frame prediction (MODE_INTRA) or the inter-frame prediction (MODE_INTER) in the form of a value.

[0089] <Intra-Block Copy Prediction>

[0090] Intra Block Copy prediction is a process of encoding / decoding a processing object block by referring to decoded pixels in a processed object picture as prediction values. Moreover, the distance from the processing object block to the pixels to be referred to is represented by a block vector. Since the block vector refers to the processed object picture and the reference picture is uniquely determined, a reference index is not required. The difference between the block vector and the motion vector lies in whether the picture to be referred to is the processed object picture or the already processed picture. In addition, the block vector can use Adaptive Motion Vector Resolution (AMVR) to select 1-pixel accuracy or 4-pixel accuracy.

[0091] In Intra Block Copy, two modes, namely the predicted Intra Block Copy mode and the merged Intra Block Copy mode, can be selected.

[0092] The predicted Intra Block Copy mode is a mode that determines the block vector of the processing object block based on a predicted block vector and a differential block vector derived from processed information. The predicted block vector is derived from processed blocks adjacent to the processing object block and an index for determining the predicted block vector. The index for determining the predicted block vector and the differential block vector are transmitted in the bitstream.

[0093] The merged Intra Block Copy mode is a mode in which, instead of transmitting the differential motion vector, the Intra Block Copy prediction information of the processing object block is derived based on the Intra Block Copy prediction information of processed blocks adjacent to the processing object block.

[0094] <Inter-Frame Prediction>

[0095] In inter-frame prediction that predicts based on the image signal of a processed image, multiple processed images can be used as reference pictures. To manage multiple reference pictures, two reference lists, L0 (reference list 0) and L1 (reference list 1), are defined, and reference indexes are used to determine the reference pictures respectively. In a P-slice (Pslice), Pred_L0 can be used. In a B-slice (Bslice), Pred_L0, Pred_L1, and Pred_BI can be used. Pred_L0 is an inter-frame prediction that refers to the reference picture managed by L0, and Pred_L1 is an inter-frame prediction that refers to the reference picture managed by L1. Pred_BI is an inter-frame prediction that simultaneously performs Pred_L0 and Pred_L1 and refers to one reference picture managed by each of L0 and L1. The information for determining Pred_L0, Pred_L1, and Pred_BI is defined as the inter-frame prediction mode. Regarding constants and variables with subscript LX appended to the output in subsequent processing, it is assumed that processing is performed for each of L0 and L1.

[0096] <Predicted Motion Vector Mode>

[0097] The predictive motion vector mode is a mode for transmitting an index for determining a predictive motion vector, a differential motion vector, an inter-frame prediction mode, a reference index, and determining inter-frame prediction information for a block to be processed. The predictive motion vector is derived from a predictive motion vector candidate and an index for determining the predictive motion vector. The predictive motion vector candidate is derived from a processed block adjacent to the block to be processed or a block located at the same position as or near (proximate to) the block to be processed among blocks belonging to a processed image.

[0098] <Merge mode>

[0099] The merge mode is a mode in which a differential motion vector and a reference index are not transmitted, and inter-frame prediction information for the block to be processed is derived from inter-frame prediction information of a processed block adjacent to the block to be processed or a block located at the same position as or near (proximate to) the block to be processed among blocks belonging to a processed image.

[0100] A processed block adjacent to the block to be processed and the inter-frame prediction information of the processed block are defined as spatial merge candidates. A block located at the same position as or near (proximate to) the block to be processed among blocks belonging to a processed image and inter-frame prediction information derived from the inter-frame prediction information of the block are defined as temporal merge candidates. Each merge candidate is registered in a merge candidate list, and the merge candidate used in the prediction of the block to be processed is determined by a merge index.

[0101] <Adjacent block>

[0102] Figure 11 It is a diagram illustrating reference blocks referred to for deriving inter-frame prediction information in the predictive motion vector mode and the merge mode. A0, A1, A2, B0, B1, B2, B3 are processed blocks adjacent to the block to be processed. T0 is a block located at the same position as or near (proximate to) the block to be processed in the processed image block belonging to the processed image.

[0103] A1, A2 are blocks located on the left side of the block to be processed and adjacent to the block to be processed. B1, B3 are blocks located above the block to be processed and adjacent to the block to be processed. A0, B0, B2 are blocks located at the lower left, upper right, and upper left of the block to be processed, respectively.

[0104] Details of how to process adjacent blocks in the predictive motion vector mode and the merge mode are described later.

[0105] <Affine transform motion compensation>

[0106] Affine transform motion compensation divides a coded block into sub-blocks of a predetermined unit, and determines motion vectors for each of the divided sub-blocks separately to perform motion compensation. Motion vectors for each sub-block are derived based on more than one control point, and the more than one control point is derived from inter-frame prediction information of a processed block adjacent to the block to be processed or a block in a processed image that is at the same position as or near (adjacent to) the block to be processed. In the present embodiment, the size of the sub-block is set to 4×4 pixels, but the size of the sub-block is not limited thereto, and motion vectors may also be derived in units of pixels.

[0107] Figure 14 An example of affine transform motion compensation when there are two control points is shown. In this case, the two control points have two parameters, a horizontal direction component and a vertical direction component. Therefore, the affine transform when there are two control points is called a four-parameter affine transform. Figure 14 CP0 and CP1 of are control points.

[0108] Figure 15 An example of affine transform motion compensation when there are three control points is shown. In this case, the three control points have two parameters, a horizontal direction component and a vertical direction component. Therefore, the affine transform when there are three control points is called a six-parameter affine transform. Figure 15 CP0, CP1, and CP2 of are control points.

[0109] Affine transform motion compensation can be used in either the predicted motion vector mode or the merge mode. The mode in which affine transform motion compensation is applied in the predicted motion vector mode is defined as the sub-block predicted motion vector mode, and the mode in which affine transform motion compensation is applied in the merge mode is defined as the sub-block merge mode.

[0110] <Syntax of coded block>

[0111] Use Figure 12A , Figure 12B and Figure 13 to illustrate the syntax for describing the prediction mode of a coded block. Figure 12AThe pred_mode_flag is a flag indicating whether it is inter-frame prediction. If the pred_mode_flag is 0, it is inter-frame prediction; if the pred_mode_flag is 1, it is intra-frame prediction. In the case of intra-frame prediction, the pred_mode_ibc_flag, which is a flag indicating whether it is intra-block copy prediction, is sent. In the case of intra-block copy prediction (pred_mode_ibc_flag = 1), the merge_flag is sent. The merge_flag is a flag indicating whether it is the merged intra-block copy mode or the predicted intra-block copy mode. In the case of the merged intra-block copy mode (merge_flag = 1), the merge index merge_idx is sent. In the case where it is not intra-block copy prediction (pred_mode_ibc_flag = 0), as normal intra-frame prediction, the information intra_pred_mode of normal intra-frame prediction is sent.

[0112] In the case of inter-frame prediction, the merge_flag is sent. The merge_flag is a flag indicating whether it is the merge mode or the predicted motion vector mode. In the case of the predicted motion vector mode (merge_flag = 0), the inter_affine_flag, which is a flag indicating whether to apply the sub-block predicted motion vector mode, is sent. If the sub-block predicted motion vector mode is applied (inter_affine_flag = 1), the cu_affine_type_flag is sent. The cu_affine_type_flag is a flag used to determine the number of control points in the sub-block predicted motion vector mode.

[0113] On the other hand, in the case of the merge mode (merge_flag = 1), send Figure 12BThe merge_subblock_flag. The merge_subblock_flag is a flag indicating whether to apply the sub-block merge mode. In the case of the sub-block merge mode (merge_subblock_flag = 1), the merge index merge_subblock_idx is sent. On the other hand, in the case where the sub-block merge mode is not applied (merge_subblock_flag = 0), the flag merge_triangle_flag indicating whether to apply the triangle merge mode is sent. If the triangle merge mode is applied (merge_triangle_flag = 1), the merge triangle indices merge_triangle_idx0 and merge_triangle_idx1 are sent for the direction merge_triangle_split_dir of the split block and the two divided partitions. On the other hand, if the triangle merge mode is not applied (merge_triangle_flag = 0), the merge index merge_idx is sent.

[0114] Figure 13 Shows the values of the respective syntax elements of the inter-frame prediction and the corresponding prediction modes. merge_flag = 0, inter_affine_flag = 0 corresponds to the normal predicted motion vector mode (Inter Pred Mode, inter-frame prediction mode). merge_flag = 0, inter_affine_flag = 1 corresponds to the sub-block predicted motion vector mode (Inter Affine Mode, inter-frame affine mode). merge_flag = 1, merge_subblock_flag = 0, merge_trianlge_flag = 0 corresponds to the normal merge mode (Merge Mode, merge mode). merge_flag = 1, merge_subblock_flag = 0, merge_trianlge_flag = 1 corresponds to the triangle merge mode (Triangle Merge Mode). merge_flag = 1, merge_subblock_flag = 1 corresponds to the sub-block merge mode (Affine Merge Mode, affine merge mode).

[0115] <poc>

[0116] POC (Picture Order Count) is a variable associated with the picture to be encoded, and a value incremented by 1 corresponding to the output order of the picture is set. Based on the value of the POC, it is possible to identify whether pictures are the same, identify the front - back relationship between pictures in the output order, and derive the distance between pictures. For example, if two pictures have the same POC value, they can be determined to be the same picture. When the POCs of two pictures have different values, it can be determined that the picture with the smaller POC value is the picture output first, and the difference between the POCs of the two pictures represents the distance between the pictures in the time - axis direction.

[0117] (First Embodiment)

[0118] The image encoding device 100 and the image decoding device 200 according to the first embodiment of the present invention will be described.

[0119] Figure 1 It is a block diagram of the image encoding device 100 according to the first embodiment. The image encoding device 100 of the embodiment includes a block division unit 101, an inter - frame prediction unit 102, an intra - frame prediction unit 103, a decoded image memory 104, a prediction method determination unit 105, a residual generation unit 106, an orthogonal transformation / quantization unit 107, a bit - string encoding unit 108, an inverse quantization / inverse orthogonal transformation unit 109, a decoded image signal overlapping unit 110, and an encoded information storage memory 111.

[0120] The block division unit 101 recursively divides the input image to generate encoded blocks. The block division unit 101 includes a 4 - division unit and a 2 - 3 division unit. The 4 - division unit divides the block to be divided in the horizontal and vertical directions respectively, and the 2 - 3 division unit divides the block to be divided in either the horizontal or vertical direction. The block division unit 101 sets the generated encoded blocks as the processing target encoded blocks, and provides the image signals of the processing target encoded blocks to the inter - frame prediction unit 102, the intra - frame prediction unit 103, and the residual generation unit 106. In addition, the block division unit 101 provides the information indicating the determined recursive division structure to the bit - string encoding unit 108. The detailed operation of the block division unit 101 will be described later.

[0121] The inter-frame prediction unit 102 performs inter-frame prediction on the encoding block to be processed. The inter-frame prediction unit 102 derives candidates for a plurality of inter-frame prediction information based on the inter-frame prediction information stored in the encoding information storage memory 111 and the decoded image signal stored in the decoded image memory 104, selects an appropriate inter-frame prediction mode from the derived candidates, and provides the selected inter-frame prediction mode and the prediction image signal corresponding to the selected inter-frame prediction mode to the prediction method determination unit 105. The detailed structure and operation of the inter-frame prediction unit 102 will be described later.

[0122] The intra-frame prediction unit 103 performs intra-frame prediction on the encoding block to be processed. The intra-frame prediction unit 103 refers to the decoded image signal stored in the decoded image memory 104 as a reference pixel, and generates a prediction image signal by intra-frame prediction based on the encoding information such as the intra-frame prediction mode stored in the encoding information storage memory 111. In intra-frame prediction, the intra-frame prediction unit 103 selects an appropriate intra-frame prediction mode from a plurality of intra-frame prediction modes, and provides the selected intra-frame prediction mode and the prediction image signal corresponding to the selected intra-frame prediction mode to the prediction method determination unit 105. The detailed configuration and operation of the intra-frame prediction unit 103 will be described later.

[0123] The decoded image memory 104 stores the decoded image generated by the decoded image signal overlapping unit 110. The decoded image memory 104 provides the stored decoded image to the inter-frame prediction unit 102 and the intra-frame prediction unit 103.

[0124] The prediction method determination unit 105 evaluates each of the intra-frame prediction and the inter-frame prediction by using the encoding information, the code amount of the residual, the distortion amount between the prediction image signal and the image signal to be processed, etc., and determines the best prediction mode. In the case of intra-frame prediction, the prediction method determination unit 105 provides the intra-frame prediction information such as the intra-frame prediction mode as encoding information to the bit string encoding unit 108. In the case of the merge mode of inter-frame prediction, the prediction method determination unit 105 provides the inter-frame prediction information such as the merge index and the information indicating whether it is a sub-block merge mode (sub-block merge flag) as encoding information to the bit string encoding unit 108. In the case of the predicted motion vector mode of inter-frame prediction, the prediction method determination unit 105 provides the inter-frame prediction information such as the inter-frame prediction mode, the predicted motion vector index, the reference indices of L0 and L1, the differential motion vector, and the information indicating whether it is a sub-block predicted motion vector mode (sub-block predicted motion vector flag) as encoding information to the bit string encoding unit 108. In addition, the prediction method determination unit 105 provides the determined encoding information to the encoding information storage memory 111. The prediction method determination unit 105 provides the prediction image signal to the residual generation unit 106 and the decoded image signal overlapping unit 110.

[0125] The residual generation unit 106 generates a residual by subtracting the predicted image signal from the image signal to be processed, and supplies it to the orthogonal transform / quantization unit 107.

[0126] The orthogonal transform / quantization unit 107 performs orthogonal transform and quantization on the residual according to the quantization parameters to generate an orthogonally transformed / quantized residual, and supplies the generated residual to the bit string encoding unit 108 and the inverse quantization / inverse orthogonal transform unit 109.

[0127] The bit string encoding unit 108 encodes, for each coding block, the coding information corresponding to the prediction method determined by the prediction method determination unit 105, in addition to the information in units of sequence, picture, slice, and coding block. Specifically, the bit string encoding unit 108 encodes the prediction mode PredMode for each coding block. When the prediction mode is inter prediction (MODE_INTER), the bit string encoding unit 108 encodes coding information (inter prediction information) such as a flag for discriminating whether it is a merge mode, a sub-block merge flag, a merge index in the case of a merge mode, an inter prediction mode in the case of not a merge mode, a prediction motion vector index, information related to the differential motion vector, and a sub-block prediction motion vector flag according to a prescribed syntax (syntax rules of the bit string), and generates a first bit string. When the prediction mode is intra prediction (MODE_INTRA), the bit string encoding unit 108 encodes a flag for discriminating whether it is intra block copy according to a prescribed syntax. In the case of intra block copy, if it is a merge mode, the merge index is encoded according to a prescribed syntax, and if it is not a merge mode, coding information (intra prediction information) such as a prediction block vector index and a differential block vector is encoded according to a prescribed syntax. In the case of not intra block copy, coding information (intra prediction information) such as an intra prediction mode is encoded according to a prescribed syntax. Through the above encoding, a first bit string is generated. In addition, the bit string encoding unit 108 performs entropy encoding on the orthogonally transformed and quantized residual, generates a second bit string. The bit string encoding unit 108 multiplexes the first bit string and the second bit string according to a prescribed syntax, and outputs a bit stream.

[0128] The inverse quantization / inverse orthogonal transform unit 109 performs inverse quantization and inverse orthogonal transform on the orthogonally transformed / quantized residual supplied from the orthogonal transform / quantization unit 107 to calculate a residual, and supplies the calculated residual to the decoded image signal overlapping unit 110.

[0129] The decoded image signal overlapping section 110 overlaps the predicted image signal corresponding to the decision of the prediction method determination section 105 and the residual obtained by inverse quantization and inverse orthogonal transformation by the inverse quantization / inverse orthogonal transformation section 109 to generate a decoded image, which is stored in the decoded image memory 104. In addition, the decoded image signal overlapping section 110 may also perform a filtering process for reducing distortions such as block distortion caused by encoding on the decoded image and then store it in the decoded image memory 104.

[0130] The encoding information storage memory 111 stores encoding information such as the prediction mode (inter-frame prediction or intra-frame prediction) determined by the prediction method determination section 105. In the case of inter-frame prediction, the encoding information stored in the encoding information storage memory 111 includes the determined motion vector, reference indices of the reference lists L0 and L1, and inter-frame prediction information such as the historical predicted motion vector candidate list. Additionally, in the case of the merge mode of inter-frame prediction, the encoding information stored in the encoding information storage memory 111 includes, in addition to the above-mentioned information, inter-frame prediction information such as the merge index and information indicating whether it is a sub-block merge mode (sub-block merge flag). Further, in the case of the predicted motion vector mode of inter-frame prediction, the encoding information stored in the encoding information storage memory 111 includes, in addition to the above-mentioned information, inter-frame prediction information such as the inter-frame prediction mode, predicted motion vector index, differential motion vector, and information indicating whether it is a sub-block predicted motion vector mode (sub-block predicted motion vector flag). In the case of intra-frame prediction, the encoding information stored in the encoding information storage memory 111 includes intra-frame prediction information such as the determined intra-frame prediction mode.

[0131] Figure 2 It represents the Figure 1 block diagram of the structure of the image decoding apparatus according to an embodiment of the present invention corresponding to the image encoding apparatus. The image decoding apparatus of the embodiment includes a bit string decoding section 201, a block segmentation section 202, an inter-frame prediction section 203, an intra-frame prediction section 204, an encoding information storage memory 205, an inverse quantization / inverse orthogonal transformation section 206, a decoded image signal overlapping section 207, and a decoded image memory 208.

[0132] Figure 2 The decoding process of the Figure 1 image decoding apparatus corresponds to the decoding process provided inside the Figure 2 image encoding apparatus. Therefore, Figure 1 each structure of the encoding information storage memory 205, the inverse quantization / inverse orthogonal transformation section 206, the decoded image signal overlapping section 207, and the decoded image memory 208 of the

[0133] The bitstream provided to the bitstring decoding unit 201 is separated according to a prescribed syntax rule. The bitstring decoding unit 201 decodes the separated first bitstring to obtain information on sequences, pictures, slices, coding block units, and coding information of the coding block units. Specifically, the bitstring decoding unit 201 decodes the prediction mode PredMode in units of coding blocks, and the prediction mode PredMode discriminates between inter-frame prediction (MODE_INTER) and intra-frame prediction (MODE_INTRA). When the prediction mode is inter-frame prediction (MODE_INTER), the bitstring decoding unit 201 decodes the coding information (inter-frame prediction information) related to the flag for discriminating whether it is a merge mode, the merge index in the case of the merge mode, the sub-block merge flag, the inter-frame prediction mode in the case of the predicted motion vector mode, the predicted motion vector index, the differential motion vector, the sub-block predicted motion vector flag, etc. according to a prescribed syntax, and provides the coding information (inter-frame prediction information) to the coding information storage memory 205 via the inter-frame prediction unit 203 and the block splitting unit 202. When the prediction mode is intra-frame prediction (MODE_INTRA), the bitstring decoding unit 201 decodes the flag for discriminating whether it is intra-block copy. In the case of intra-block copy, if it is the merge mode, the merge index is decoded according to a prescribed syntax, and if it is not the merge mode, the coding information (intra-frame prediction information) such as the predicted block vector index and the differential block vector is decoded according to a prescribed syntax. In the case of not intra-block copy, the coding information (intra-frame prediction information) such as the intra-frame prediction mode is decoded according to a prescribed syntax. Through the above decoding, the coding information (intra-frame prediction information) is provided to the coding information storage memory 205 via the inter-frame prediction unit 203 or the intra-frame prediction unit 204, and the block splitting unit 202. The bitstring decoding unit 201 decodes the separated second bitstring, calculates the residual after orthogonal transformation / quantization, and provides the residual after orthogonal transformation / quantization to the inverse quantization / inverse orthogonal transformation unit 206.

[0134] When the prediction mode PredMode of the coding block of the object to be processed is a prediction motion vector mode in inter-frame prediction (MODE_INTER), the inter-frame prediction unit 203 uses the coding information of the decoded image signal stored in the coding information storage memory 205 to derive a plurality of candidate prediction motion vectors, and registers the derived plurality of candidate prediction motion vectors in the prediction motion vector candidate list described later. The inter-frame prediction unit 203 selects a prediction motion vector corresponding to the prediction motion vector index decoded by the bit string decoding unit 201 from the plurality of candidate prediction motion vectors registered in the prediction motion vector candidate list, calculates a motion vector based on the differential motion vector decoded by the bit string decoding unit 201 and the selected prediction motion vector, and stores the calculated motion vector together with other coding information in the coding information storage memory 205. Here, the coding information of the coding block to be provided / stored is the prediction mode PredMode, flags predFlagL0[xP][yP], predFlagL1[xP][yP] indicating whether to use L0 prediction and L1 prediction, reference indexes refIdxL0[xP][yP], refIdxL1[xP][yP] of L0 and L1, motion vectors mvL0[xP][yP], mvL1[xP][yP] of L0 and L1, etc. Here, xP and yP are indexes indicating the positions of the upper left pixels of the coding block within the picture. When the prediction mode PredMode is inter-frame prediction (MODE_INTER) and the inter-frame prediction mode is L0 prediction (Pred_L0), the flag predFlagL0 indicating whether to use L0 prediction is 1, and the flag predFlagL1 indicating whether to use L1 prediction is 0. When the inter-frame prediction mode is L1 prediction (Pred_L1), the flag predFlagL0 indicating whether to use L0 prediction is 0, and the flag predFlagL1 indicating whether to use L1 prediction is 1. When the inter-frame prediction mode is bi-prediction (Pred_BI), the flag predFlagL0 indicating whether to use L0 prediction and the flag predFlagL1 indicating whether to use L1 prediction are both 1. And when the prediction mode PredMode of the coding block of the object to be processed is the merge mode in inter-frame prediction (MODE_INTER), merge candidates are derived.Using the coded information of the decoded coded blocks stored in the coded information storage memory 205, a plurality of merge candidates are derived, registered in the merge candidate list described later, and a merge candidate corresponding to the merge index decoded by the bit string decoding unit 201 is selected from the plurality of merge candidates registered in the merge candidate list. Frame-inter prediction information such as the flags predFlagL0[xP][yP], predFlagL1[xP][yP], L0, L1 indicating whether to use the selected merge candidate for L0 prediction and L1 prediction, the reference indices refIdxL0[xP][yP], refIdxL1[xP][yP], L0, L1, and the motion vectors mvL0[xP][yP], mvL1[xP][yP] of L0, L1 are stored in the coded information storage memory 205. Here, xP and yP are indices indicating the position of the upper left pixel of the coded block within the picture. The detailed configuration and operation of the frame-inter prediction unit 203 described later.

[0135] When the prediction mode PredMode of the coded block to be processed is intra prediction (MODE_INTRA), the intra prediction unit 204 performs intra prediction. The coded information decoded by the bit string decoding unit 201 includes the intra prediction mode. The intra prediction unit 204 generates a prediction image signal by intra prediction based on the intra prediction mode included in the coded information decoded by the bit string decoding unit 201 and based on the decoded image signal stored in the decoded image memory 208, and supplies the generated prediction image signal to the decoded image signal overlapping unit 207. Since the intra prediction unit 204 corresponds to the intra prediction unit 103 of the image coding device 100, it performs the same processing as the intra prediction unit 103.

[0136] The inverse quantization / inverse orthogonal transformation unit 206 performs inverse orthogonal transformation and inverse quantization on the orthogonally transformed / quantized residual decoded by the bit string decoding unit 201 to obtain the inverse orthogonally transformed / inverse quantized residual.

[0137] The decoded image signal overlapping unit 207 decodes the decoded image signal by overlapping the prediction image signal obtained by frame-inter prediction by the frame-inter prediction unit 203 or the prediction image signal obtained by intra prediction by the intra prediction unit 204 and the inverse orthogonally transformed / inverse quantized residual by the inverse quantization / inverse orthogonal transformation unit 206, and stores the decoded decoded image signal in the decoded image memory 208. When storing in the decoded image memory 208, the decoded image signal overlapping unit 207 may also perform filtering processing such as reducing block distortion caused by coding on the decoded image and then store it in the decoded image memory 208.

[0138] Next, the operation of the block segmentation unit 101 in the image coding device 100 will be described. Figure 3 It is a flowchart showing the actions of dividing an image into tree blocks and further dividing each tree block. First, the input image is divided into tree blocks of a predetermined size (step S1001). For each tree block, scanning is performed in a predetermined order, i.e., the raster scan order (step S1002), and the interior of the tree block to be processed is divided (step S1003).

[0139] Figure 7 It is a flowchart showing the detailed actions of the division process in step S1003. First, it is judged whether to divide the block to be processed into four parts (step S1101).

[0140] In the case where it is judged that the block to be processed is divided into four parts, the block to be processed is divided into four parts (step S1102). For each block obtained by dividing the block to be processed, scanning is performed in the Z-scan order, i.e., the order of upper left, upper right, lower left, and lower right (step S1103). Figure 5 It is an example of the Z-scan order, Figure 6A 601 in it is an example after dividing the block to be processed into four parts. Figure 6A The numbers 0 to 3 of 601 represent the order of processing. Then, for each block obtained by dividing in step S1101, the Figure 7 division process is recursively executed (step S1104).

[0141] In the case where it is judged that the block to be processed is not divided into four parts, a 2-3 division is performed (step S1105).

[0142] Figure 8 It is a flowchart showing the detailed actions of the 2-3 division process in step S1105. First, it is judged whether to perform a 2-3 division on the block to be processed, that is, whether to perform either a 2-division or a 3-division (step S1201).

[0143] In the case where it is judged that the block to be processed is not subjected to a 2-3 division, that is, in the case where it is judged that no division is performed, the division ends (step S1211). That is, for the blocks obtained by recursive division processing, no further recursive division processing is performed.

[0144] In the case where it is judged that the block to be processed is subjected to a 2-3 division, it is judged whether to further divide the block to be processed into two parts (step S1202).

[0145] In the case where it is judged that the block to be processed is divided into two parts, it is judged whether to divide the block to be processed vertically (step S1203). Based on the result, the block to be processed is divided vertically (step S1204), or the block to be processed is divided horizontally (step S1205). As a result of step S1204, the block to be processed is as Figure 6B As shown by 602 in [reference], it is divided into two upper and lower (vertical) parts. As a result of step S1205, the processing target block is as Figure 6D shown by 604 in [reference], and is divided into two left and right (horizontal) parts.

[0146] In step S1202, when it is not determined that the processing target block is divided into two parts, that is, when it is determined that it is divided into three parts, it is determined whether to divide the processing target block into upper, middle, and lower (vertical) parts (step S1206). Based on this result, the processing target block is divided into three upper, middle, and lower (vertical) parts (step S1207), or the processing target block is divided into three left, middle, and right (horizontal) parts (step S1208). As a result of step S1207, the processing target block is as Figure 6C shown by 603 in [reference], and is divided into three upper, middle, and lower (vertical) parts. As a result of step S1208, the processing target block is as Figure 6E shown by 605 in [reference], and is divided into three left, middle, and right (horizontal) parts.

[0147] After any one of steps S1204, S1205, S1207, and S1208 is executed, each block obtained by dividing the processing target block is scanned in the order from left to right and from top to bottom (step S1209). Figures 6B to 6E The numbers 0 to 2 from 602 to 605 in [reference] indicate the order of processing. For each divided block, the 2-3 Figure 8 division processing (step S1210) is recursively executed.

[0148] The recursive block division described here can also limit whether division is required according to the number of divisions or the size of the processing target block, etc. The information for limiting whether division is required can be implemented in a structure where it is pre-agreed between the encoding device and the decoding device without information transfer, or can be implemented in a structure where the encoding device determines the information for limiting whether division is required and records it in the bit string to be transmitted to the decoding device.

[0149] When a certain block is divided, the block before division is called the mother block, and each divided block is called a child block.

[0150] Next, the operation of the block division unit 202 in the image decoding apparatus 200 will be described. The block division unit 202 divides the tree block according to the same processing steps as the block division unit 101 of the image encoding apparatus 100. However, the difference is that in the block division unit 101 of the image encoding apparatus 100, optimization methods such as estimation of the optimal shape based on image recognition or distortion rate optimization are applied to determine the optimal block division shape. In contrast, the block division unit 202 in the image decoding apparatus 200 determines the block division shape by decoding the block division information recorded in the bit string.

[0151] Figure 9 Shows the syntax related to the block division of the first embodiment (syntax rules of the bit string). coding_quadtree() represents the syntax related to the 4-division processing of the block. multi_type_tree() represents the syntax related to the 2-division or 3-division processing of the block. qt_split is a flag indicating whether to perform 4-division on the block. When performing 4-division on the block, set qt_split = 1. When not performing 4-division, set qt_split = 0. In the case of 4-division (qt_split = 1), the 4-division processing is recursively performed on each of the 4-divided blocks (coding_quadtree(0), coding_quadtree(1), coding_quadtree(2), coding_quadtree(3), and the independent variables 0 to 3 correspond to Figure 6A the numbers of 601). When not performing 4-division (qt_split = 0), the subsequent division is determined according to multi_type_tree(). mtt_split is a flag indicating whether to further perform division. And when performing division (mtt_split = 1), the flag mtt_split_vertical indicating whether to divide in the vertical direction or the horizontal direction and the flag mtt_split_binary determining whether to perform 2-division or 3-division are transmitted. mtt_split_vertical = 1 indicates division in the vertical direction, and mtt_split_vertical = 0 indicates division in the horizontal direction. mtt_split_binary = 1 indicates 2-division, and mtt_split_binary = 0 indicates 3-division. In the case of 2-division (mtt_split_binary = 1), the division processing is recursively performed on each of the 2-divided blocks (multi_type_tree(0), multi_type_tree(1), and the independent variables 0 to 1 correspond to Figure 6B , Figure 6D (the numbers of 602 or 604). In the case of 3-way splitting (mtt_split_binary = 0), the splitting process is recursively performed on each of the three split blocks (multi_type_tree(0), multi_type_tree(1), multi_type_tree(2), where 0 to 2 correspond to Figure 6C the 603 or Figure 6E (the numbers of 605). By recursively calling multi_type_tree, hierarchical block splitting is performed until mtt_split = 0.

[0152] <Intra Prediction>

[0153] The intra prediction method according to the embodiment is implemented in the intra prediction unit 103 of the image encoding apparatus 100 in Figure 1 and the intra prediction unit 204 of the image decoding apparatus 200 in Figure 2 .

[0154] The intra prediction method according to the embodiment will be described with reference to the drawings. The intra prediction method is implemented in any one of the processes of encoding and decoding in units of coding blocks.

[0155] <Explanation of the Intra Prediction Unit 103 on the Encoding Side>

[0156] Figure 40 is a diagram showing the detailed structure of the intra prediction unit 103 of the image encoding apparatus 100 in Figure 1 . Usually, the intra prediction unit 351 generates a predicted image signal from the decoded pixels adjacent to the coding block to be processed through normal intra prediction, selects a suitable intra prediction mode from a plurality of intra prediction modes, and provides the predicted image signal corresponding to the selected intra prediction mode and the selected intra prediction mode to the prediction method determination unit 105. Figure 10A and Figure 10B show examples of intra prediction. Figure 10A shows the correspondence between the prediction direction of normal intra prediction and the intra prediction mode number. For example, in intra prediction mode 50, an intra prediction image is generated by replicating pixels in the vertical direction. Intra prediction mode 1 is the DC mode, which is a mode in which the average value of all pixel values of the block to be processed is set as the reference pixel. Intra prediction mode 0 is the planar mode, and is a mode for creating a two-dimensional intra prediction image from the reference pixels in the vertical and horizontal directions. Figure 10B This is an example of generating an intra-predicted image in the case of intra-prediction mode 40. For each pixel of the processing target block, the value of the reference pixel in the direction indicated by the intra-prediction mode is copied. When the reference pixel of the intra-prediction mode is not at an integer position, the reference pixel value is determined by interpolation based on the reference pixel values at the surrounding integer positions.

[0157] The intra-block copy prediction unit 352 acquires the decoded region of the image signal that is the same as the encoded block to be processed from the decoded image memory 104, generates a prediction image signal through intra-block copy processing, and supplies the prediction image signal to the prediction method determination unit 105. The detailed structure and processing of the intra-block copy prediction unit 352 will be described later.

[0158] <Explanation of the intra-prediction unit 204 on the decoding side>

[0159] Figure 41 It represents Figure 2 a diagram showing the detailed configuration of the intra-prediction unit 204 of the image decoding apparatus 200.

[0160] The normal intra-prediction unit 361 generates a prediction image signal through normal intra-prediction based on the decoded pixels adjacent to the encoded block to be processed, selects a suitable intra-prediction mode from multiple intra-prediction modes, and obtains the selected intra-prediction mode and the prediction image signal corresponding to the selected intra-prediction mode. This prediction image signal is supplied to the decoded image signal overlapping unit 207 via the switch 364. Figure 41 The processing of the normal intra-prediction unit 361 in Figure 40 corresponds to the normal intra-prediction unit 351 in

[0161] so its detailed description is omitted.

[0162] <Inter-frame prediction>

[0163] The inter-frame prediction method of the embodiment is implemented in the inter-frame prediction unit 102 of the Figure 1 image encoding apparatus and the inter-frame prediction unit 203 of the Figure 2 image decoding apparatus.

[0164] The inter-frame prediction method according to the embodiment will be described with reference to the drawings. The inter-frame prediction method is implemented in either the encoding process or the decoding process in units of encoded blocks.

[0165] <Explanation of the Inter - frame Prediction Unit 102 on the Encoding Side>

[0166] Figure 16 It is a diagram showing Figure 1 the detailed structure of the inter - frame prediction unit 102 of an image encoding device. Usually, the prediction motion vector mode derivation unit 301 derives a plurality of usual prediction motion vector candidates to select a prediction motion vector, and calculates the differential motion vector between the selected prediction motion vector and the detected motion vector. The detected inter - frame prediction mode, reference index, motion vector, and the calculated differential motion vector are the inter - frame prediction information of the usual prediction motion vector mode. This inter - frame prediction information is provided to the inter - frame prediction mode determination unit 305. The detailed structure and processing of the usual prediction motion vector mode derivation unit 301 will be described later.

[0167] In the usual merge mode derivation unit 302, a plurality of usual merge candidates are derived, a usual merge candidate is selected, and the inter - frame prediction information of the usual merge mode is obtained. This inter - frame prediction information is provided to the inter - frame prediction mode determination unit 305. The detailed structure and processing of the usual merge mode derivation unit 302 will be described later.

[0168] In the sub - block prediction motion vector mode derivation unit 303, a plurality of sub - block prediction motion vector candidates are derived to select a sub - block prediction motion vector, and the differential motion vector between the selected sub - block prediction motion vector and the detected motion vector is calculated. The detected inter - frame prediction mode, reference index, motion vector, and the calculated differential motion vector are the inter - frame prediction information of the sub - block prediction motion vector mode. This inter - frame prediction information is provided to the inter - frame prediction mode determination unit 305.

[0169] In the sub - block merge mode derivation unit 304, a plurality of sub - block merge candidates are derived, a sub - block merge candidate is selected, and the inter - frame prediction information of the sub - block merge mode is obtained. This inter - frame prediction information is provided to the inter - frame prediction mode determination unit 305.

[0170] The inter - frame prediction mode determination unit 305 determines the inter - frame prediction information based on the inter - frame prediction information provided from the usual prediction motion vector mode derivation unit 301, the usual merge mode derivation unit 302, the sub - block prediction motion vector mode derivation unit 303, and the sub - block merge mode derivation unit 304. The inter - frame prediction information corresponding to the determination result is provided from the inter - frame prediction mode determination unit 305 to the motion compensation prediction unit 306.

[0171] The motion compensation prediction unit 306 performs inter - frame prediction on the reference image signal stored in the decoded image memory 104 based on the determined inter - frame prediction information. The detailed structure and processing of the motion compensation prediction unit 306 will be described later.

[0172] <Explanation of the Inter - frame Prediction Unit 203 on the Decoding Side>

[0173] Figure 22 is a diagram showing Figure 2 the detailed structure of the inter-frame prediction unit 203 of the image decoding device.

[0174] The normal prediction motion vector mode derivation unit 401 derives a plurality of normal prediction motion vector candidates to select a prediction motion vector, and calculates the sum value of the selected prediction motion vector and the decoded differential motion vector as the motion vector. The decoded inter-frame prediction mode, reference index, and motion vector are inter-frame prediction information in the normal prediction motion vector mode. This inter-frame prediction information is provided to the motion compensation prediction unit 406 via the switch 408. The detailed structure and processing of the normal prediction motion vector mode derivation unit 401 will be described later.

[0175] In the normal merge mode derivation unit 402, a plurality of normal merge candidates are derived to select a normal merge candidate, and the inter-frame prediction information in the normal merge mode is obtained. This inter-frame prediction information is provided to the motion compensation prediction unit 406 via the switch 408. The detailed structure and processing of the normal merge mode derivation unit 402 will be described later.

[0176] In the sub-block prediction motion vector mode derivation unit 403, a plurality of sub-block prediction motion vector candidates are derived to select a sub-block prediction motion vector, and the sum value of the selected sub-block prediction motion vector and the decoded differential motion vector is calculated as the motion vector. The decoded inter-frame prediction mode, reference index, and motion vector become the inter-frame prediction information in the sub-block prediction motion vector mode. This inter-frame prediction information is provided to the motion compensation prediction unit 406 via the switch 408.

[0177] In the sub-block merge mode derivation unit 404, a plurality of sub-block merge candidates are derived to select a sub-block merge candidate, and the inter-frame prediction information in the sub-block merge mode is obtained. This inter-frame prediction information is provided to the motion compensation prediction unit 406 via the switch 408.

[0178] In the motion compensation prediction unit 406, inter-frame prediction is performed on the reference image signal stored in the decoded image memory 208 based on the determined inter-frame prediction information. The detailed structure and processing of the motion compensation prediction unit 406 are the same as those of the motion compensation prediction unit 306 on the encoding side.

[0179] <Normal prediction motion vector mode derivation unit (Normal AMVP)>

[0180] Figure 17 The general predictive motion vector mode derivation unit 301 includes a spatial predictive motion vector candidate derivation unit 321, a temporal predictive motion vector candidate derivation unit 322, a history predictive motion vector candidate derivation unit 323, a predictive motion vector candidate supplement unit 325, a general motion vector detection unit 326, a predictive motion vector candidate selection unit 327, and a motion vector subtraction unit 328.

[0181] Figure 23 The general predictive motion vector mode derivation unit 401 on the decoding side includes a spatial predictive motion vector candidate derivation unit 421, a temporal predictive motion vector candidate derivation unit 422, a history predictive motion vector candidate derivation unit 423, a predictive motion vector candidate supplement unit 425, a predictive motion vector candidate selection unit 426, and a motion vector addition unit 427.

[0182] Respectively use Figure 19 、 Figure 25 The flowcharts of are used to illustrate the processing steps of the general predictive motion vector mode derivation unit 301 on the encoding side and the general predictive motion vector mode derivation unit 401 on the decoding side. Figure 19 is a flowchart showing the general predictive motion vector mode derivation processing steps based on the general motion vector mode derivation unit 301 on the encoding side, Figure 25 is a flowchart showing the general predictive motion vector mode derivation processing steps based on the general motion vector mode derivation unit 401 on the decoding side.

[0183] <General predictive motion vector mode derivation unit (general AMVP): Explanation on the encoding side>

[0184] Refer to Figure 19 to explain the general predictive motion vector mode derivation processing steps on the encoding side. In Figure 19 the explanation of the processing steps, sometimes the word "general" shown in Figure 19 is omitted.

[0185] First, the general motion vector detection unit 326 detects the general motion vector according to each inter-frame prediction mode and reference index ( Figure 19 step S100 of).

[0186] Next, the spatial predictive motion vector candidate derivation unit 321, the temporal predictive motion vector candidate derivation unit 322, the history predictive motion vector candidate derivation unit 323, the predictive motion vector candidate supplement unit 325, the predictive motion vector candidate selection unit 327, and the motion vector subtraction unit 328 calculate the differential motion vectors of the motion vectors used in the inter-frame prediction of the general predictive motion vector mode for each of L0 and L1 respectively ( Figure 19 Steps S101 to S106). Specifically, when the prediction mode PredMode of the processing object block is inter-frame prediction (MODE_INTER) and the inter-frame prediction mode is L0 prediction (Pred_L0), calculate the prediction motion vector candidate list mvpListL0 of L0, select the prediction motion vector mvpL0, and calculate the differential motion vector mvdL0 of the motion vector mvL0 of L0. When the inter-frame prediction mode of the processing object block is L1 prediction (Pred_L1), calculate the prediction motion vector candidate list mvpListL1 of L1, select the prediction motion vector mvpL1, and calculate the differential motion vector mvdL1 of the motion vector mvL1 of L1. When the inter-frame prediction mode of the processing object block is bi-prediction (Pred_BI), perform L0 prediction and L1 prediction simultaneously, calculate the prediction motion vector candidate list mvpListL0 of L0, select the prediction motion vector mvpL0 of L0, calculate the differential motion vector mvdL0 of the motion vector mvL0 of L0, calculate the prediction motion vector candidate list mvpListL1 of L1, calculate the prediction motion vector mvpL1 of L1, and calculate the differential motion vector mvdL1 of the motion vector mvL1 of L1 respectively.

[0187] Perform differential motion vector calculation processing on L0 and L1 respectively, but the processing of L0 and L1 is common. Therefore, in the following description, L0 and L1 are represented as a common LX. In the process of calculating the differential motion vector of L0, X of LX is 0, and in the process of calculating the differential motion vector of L1, X of LX is 1. In addition, in the process of calculating the differential motion vector of LX, when referring to the information of another list instead of LX, the other list is represented as LY.

[0188] When using the motion vector mvLX of LX ( Figure 19 Step S102: Yes), calculate the candidates of the prediction motion vector of LX, and construct the prediction motion vector candidate list mvpListLX of LX ( Figure 19 Step S103). Multiple candidates of the prediction motion vector are derived by the spatial prediction motion vector candidate derivation unit 321, the temporal prediction motion vector candidate derivation unit 322, the historical prediction motion vector candidate derivation unit 323, and the prediction motion vector candidate supplement unit 325 in the normal prediction motion vector mode derivation unit 301, and the prediction motion vector candidate list mvpListLX is constructed. Regarding Figure 19 The detailed processing steps of Step S103 are described later using Figure 20 The flowchart.

[0189] Next, the prediction motion vector selection unit 327 selects the prediction motion vector mvpLX of LX from the prediction motion vector candidate list mvpListLX of LX ( Figure 19 Step S104). Here, in the prediction motion vector candidate list mvpListLX, a certain element (the i-th element counted from 0) is represented as mvpListLX[i]. Calculate each differential motion vector that is the difference between the motion vector mvLX and the candidate mvpListLX[i] of each predicted motion vector stored in the prediction motion vector candidate list mvpListLX. For each element (predicted motion vector candidate) of the prediction motion vector candidate list mvpListLX, calculate the amount of code when encoding these differential motion vectors. Then, among the elements registered in the prediction motion vector candidate list mvpListLX, select the candidate mvpListLX[i] of the predicted motion vector with the smallest amount of code for each candidate of the predicted motion vector as the predicted motion vector mvpLX, and obtain the index i. When there are multiple candidates of the predicted motion vector that result in the smallest amount of generated code in the prediction motion vector candidate list mvpListLX, select the candidate mvpListLX[i] of the predicted motion vector represented by the smaller number of the index i in the prediction motion vector candidate list mvpListLX as the best predicted motion vector mvpLX, and obtain the index i.

[0190] Next, the motion vector subtraction unit 328 subtracts the selected predicted motion vector mvpLX of LX from the motion vector mvLX of LX, and sets mvdLX = mvLX - mvpLX to calculate the differential motion vector mvdLX of LX ( Figure 19 Step S105).

[0191] <Normal predicted motion vector mode derivation unit (normal AMVP): Explanation on the decoding side>

[0192] Next, refer to Figure 25 An explanation of the normal predicted motion vector mode processing steps on the decoding side will be given. On the decoding side, the spatial predicted motion vector candidate derivation unit 421, the temporal predicted motion vector candidate derivation unit 422, the history predicted motion vector candidate derivation unit 423, and the predicted motion vector candidate supplement unit 425 calculate the motion vectors used in the inter prediction of the normal predicted motion vector mode for each of L0 and L1 ( Figure 25 Steps S201 to S206). Specifically, when the prediction mode PredMode of the processing target block is inter prediction (MODE_INTER) and the inter prediction mode of the processing target block is L0 prediction (Pred_L0), calculate the prediction motion vector candidate list mvpListL0 of L0, select the prediction motion vector mvpL0, and calculate the motion vector mvL0 of L0. When the inter prediction mode of the processing target block is L1 prediction (Pred_L1), calculate the prediction motion vector candidate list mvpListL1 of L1, select the prediction motion vector mvpL1, and calculate the motion vector mvL1 of L1. When the inter prediction mode of the processing target block is bi-prediction (Pred_BI), perform L0 prediction and L1 prediction simultaneously, calculate the prediction motion vector candidate list mvpListL0 of L0, select the prediction motion vector mvpL0 of L0, calculate the motion vector mvL0 of L0, calculate the prediction motion vector candidate list mvpListL1 of L1, calculate the prediction motion vector mvpL1 of L1, and calculate the motion vector mvL1 of L1 respectively.

[0193] Similar to the encoding side, on the decoding side, motion vector calculation processing is also performed for L0 and L1 respectively, but the processing for L0 and L1 is the same. Therefore, in the following description, L0 and L1 are represented as a common LX. LX represents the inter prediction mode for inter prediction of the encoding block of the processing target. In the process of calculating the motion vector of L0, X is 0, and in the process of calculating the motion vector of L1, X is 1. In addition, in the process of calculating the motion vector of LX, when not referring to the same reference list as the LX being calculated but referring to the information of another reference list, the other reference list is represented as LY.

[0194] When using the motion vector mvLX of LX ( Figure 25 Step S202: Yes), calculate the candidates for the prediction motion vector of LX and construct the prediction motion vector candidate list mvpListLX ( Figure 25 Step S203). Multiple candidates for the prediction motion vector are calculated by the spatial prediction motion vector candidate derivation unit 421, the temporal prediction motion vector candidate derivation unit 422, the history prediction motion vector candidate derivation unit 423, and the prediction motion vector candidate supplement unit 425 in the normal prediction motion vector mode derivation unit 401, and the prediction motion vector candidate list mvpListLX is constructed. Regarding Figure 25 The detailed processing steps of Step S203 are described later using Figure 20 The flowchart.

[0195] Next, the prediction motion vector candidate selection unit 426 extracts the candidate mvpListLX[mvpIdxLX] of the prediction motion vector corresponding to the index mvpIdxLX of the prediction motion vector decoded and provided in the bit string decoding unit 201 from the prediction motion vector candidate list mvpListLX as the selected prediction motion vector mvpLX ( Figure 25 step S204).

[0196] Next, the motion vector addition unit 427 adds the differential motion vector mvdLX of LX and the prediction motion vector mvpLX of LX decoded and provided by the bit string decoding unit 201, sets mvLX = mvpLX + mvdLX, and calculates the motion vector mvLX of LX ( Figure 25 step S205).

[0197] <Normal prediction motion vector mode derivation unit (normal AMVP): Prediction method of motion vector>

[0198] Figure 20 It is a flowchart showing the processing steps of the normal prediction motion vector mode derivation process having a common function in the normal prediction motion vector mode derivation unit 301 of the image encoding device and the normal prediction motion vector mode derivation unit 401 of the image decoding device according to the embodiment of the present invention.

[0199] In the normal prediction motion vector mode derivation unit 301 and the normal prediction motion vector mode derivation unit 401, there is a prediction motion vector candidate list mvpListLX. The prediction motion vector candidate list mvpListLX constitutes a list structure, and a storage area is provided for storing the prediction motion vector index indicating the position inside the prediction motion vector candidate list and the prediction motion vector candidate corresponding to the index as elements. The number of the prediction motion vector index starts from 0, and the prediction motion vector candidate is stored in the storage area of the prediction motion vector candidate list mvpListLX. In this embodiment, it is assumed that the prediction motion vector candidate list mvpListLX can register at least two prediction motion vector candidates (inter-frame prediction information). In addition, a variable numCurrMvpCand indicating the number of prediction motion vector candidates registered in the prediction motion vector candidate list mvpListLX is set to 0.

[0200] The spatial prediction motion vector candidate derivation units 321 and 421 derive candidates for the prediction motion vector from the block adjacent to the left. In this process, the block adjacent to the left is referred to ( Figure 11 Inter-frame prediction information of A0 or A1), that is, a flag indicating whether a predicted motion vector candidate can be used, a motion vector, a reference index, etc., to derive a predicted motion vector mvLXA, and add the derived mvLXA to the predicted motion vector candidate list mvpListLX ( Figure 20 step S301). Additionally, X is 0 during L0 prediction and X is 1 during L1 prediction (the same applies hereinafter). Next, the spatial predicted motion vector candidate derivation units 321 and 421 derive candidates for the predicted motion vector from the block adjacent above. In this process, refer to the block adjacent above ( Figure 11 Inter-frame prediction information of B0, B1, or B2), that is, a flag indicating whether a predicted motion vector candidate can be used, and a motion vector, a reference index, etc., to derive a predicted motion vector mvLXB. If the respectively derived mvLXA and mvLXB are not equal, add mvLXB to the predicted motion vector candidate list mvpListLX ( Figure 20 step S302). Except for the difference in the position and number of adjacent blocks referred to, Figure 20 the processes of steps S301 and S302 are common, and derive a flag availableFlagLXN indicating whether a predicted motion vector candidate of the coded block can be used, a motion vector mvLXN, and a reference index refIdxN (N represents A or B, the same hereinafter).

[0201] Next, the temporal predicted motion vector candidate derivation units 322 and 422 derive candidates for the predicted motion vector from blocks in a picture whose time is different from the current picture being processed. In this process, derive a flag availableFlagLXCol indicating whether a predicted motion vector candidate of the coded block in a picture at a different time can be used, a motion vector mvLXCol, a reference index refIdxCol, a reference list listCol, and add mvLXCol to the predicted motion vector candidate list mvpListLX ( Figure 20 step S303).

[0202] In addition, it is assumed that the processes of the temporal predicted motion vector candidate derivation units 322 and 422 can be omitted in units of sequence (SPS), picture (PPS), or slice.

[0203] Next, the history predicted motion vector candidate derivation units 323 and 423 add the history predicted motion vector candidates registered in the history predicted motion vector candidate list HmvpCandList to the predicted motion vector candidate list mvpListLX ( Figure 20 step S304). Details of the registration processing steps for this step S304 are described later using Figure 29 a flowchart.

[0204] Next, the motion vector candidate supplementing units 325 and 425 add predicted motion vector candidates with predetermined values such as (0, 0) until the predicted motion vector candidate list mvpListLX is filled (S305 Figure 20 ).

[0205] <Normal merge mode derivation unit (normal merge)>

[0206] Figure 18 The normal merge mode derivation unit 302 of < > includes a spatial merge candidate derivation unit 341, a temporal merge candidate derivation unit 342, an average merge candidate derivation unit 344, a history merge candidate derivation unit 345, a merge candidate supplementing unit 346, and a merge candidate selection unit 347.

[0207] Figure 24 The normal merge mode derivation unit 402 of < > includes a spatial merge candidate derivation unit 441, a temporal merge candidate derivation unit 442, an average merge candidate derivation unit 444, a history merge candidate derivation unit 445, a merge candidate supplementing unit 446, and a merge candidate selection unit 447.

[0208] Figure 21 It is a flowchart showing the steps of the normal merge mode derivation process having common functions in the normal merge mode derivation unit 302 of the image encoding device and the normal merge mode derivation unit 402 of the image decoding device according to the embodiment of the present invention.

[0209] Hereinafter, each process will be described in turn. In addition, in the following description, unless otherwise specified, the case where the slice type slice_type is a B slice will be described, but it can also be applied to the case of a P slice. However, in the case where the slice type slice_type is a P slice, since only the L0 prediction (Pred_L0) exists as the inter-frame prediction mode, the L1 prediction (Pred_L1) and the bi-prediction (Pred_BI) do not exist. Therefore, the processing around L1 can be omitted.

[0210] In the normal merge mode export unit 302 and the normal merge mode export unit 402, there is a merge candidate list mergeCandList. The merge candidate list mergeCandList forms a list structure, and is provided with a merge index indicating the position inside the merge candidate list, and a storage area for storing the merge candidates corresponding to the index as elements. The numbers of the merge index start from 0, and the merge candidates are stored in the storage area of the merge candidate list mergeCandList. In subsequent processing, it is assumed that the merge candidate of the merge index i registered in the merge candidate list mergeCandList is represented by mergeCandList[i]. In the present embodiment, it is assumed that the merge candidate list mergeCandList can register at least six merge candidates (inter-frame prediction information). And, the variable numCurrMergeCand indicating the number of merge candidates registered in the merge candidate list mergeCandList is set to 0.

[0211] In the spatial merge candidate export unit 341 and the spatial merge candidate export unit 441, according to the coding information stored in the coding information storage memory 111 of the image coding device or the coding information storage memory 205 of the image decoding device, the spatial merge candidates from each block adjacent to the processing target block ( Figure 11 the B1, A1, B0, A0, B2) are exported in the order of B1, A1, B0, A0, B2), and the exported spatial merge candidates are registered in the merge candidate list mergeCandList ( Figure 21 step S401). Here, N representing any one of B1, A1, B0, A0, B2 or the temporal merge candidate Col is defined. The flag availableFlagN indicating whether the inter-frame prediction information of block N can be used as a spatial merge candidate, the reference index refIdxL0N of L0 of the spatial merge candidate N, the reference index refIdxL1N of L1, the L0 prediction flag predFlagL0N indicating whether L0 prediction is performed, the L1 prediction flag predFlagL1N indicating whether L1 prediction is performed, the motion vector mvL0N of L0, and the motion vector mvL1N of L1 are exported. However, in the present embodiment, since the merge candidates are exported without referring to the inter-frame prediction information of the blocks included in the coding block to be processed, the spatial merge candidates using the inter-frame prediction information of the blocks included in the coding block to be processed are not exported.

[0212] Next, in the temporal merge candidate export unit 342 and the temporal merge candidate export unit 442, the temporal merge candidates from pictures at different times are exported, and the exported temporal merge candidates are registered in the merge candidate list mergeCandList ( Figure 21 Step S402). Export the flag availableFlagCol indicating whether the temporal merge candidate can be used, the L0 prediction flag predFlagL0Col indicating whether to perform L0 prediction on the temporal merge candidate, the L1 prediction flag predFlagL1Col indicating whether to perform L1 prediction, the motion vector mvL0Col of L0, and the motion vector mvL1Col of L1.

[0213] In addition, the processing of the temporal merge candidate derivation units 342 and 442 for each sequence (SPS), picture (PPS), or slice can be omitted.

[0214] Next, in the historical merge candidate derivation units 345 and 445, register the historical predicted motion vector candidates registered in the historical predicted motion vector candidate list HmvpCandList in the merge candidate list mergeCandList ( Figure 21 Step S403).

[0215] In addition, when the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, derive the historical merge candidates with the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList capped by the maximum number of merge candidates MaxNumMergeCand, and register them in the merge candidate list mergeCandList.

[0216] Next, in the average merge candidate derivation units 344 and 444, derive the average merge candidates from the merge candidate list mergeCandList, and add the derived average merge candidates to the merge candidate list mergeCandList ( Figure 21 Step S404).

[0217] In addition, when the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, derive the average merge candidates with the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList capped by the maximum number of merge candidates MaxNumMergeCand, and register them in the merge candidate list mergeCandList.

[0218] Here, the average merge candidate is a new merge candidate that has a motion vector obtained by averaging the motion vectors of the first merge candidate and the second merge candidate registered in the merge candidate list mergeCandList for each of the L0 prediction and the L1 prediction.

[0219] Next, in the merge candidate supplementing unit 346 and the merge candidate supplementing unit 446, when the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList derives additional merge candidates capped at the maximum number of merge candidates MaxNumMergeCand and registers them in the merge candidate list mergeCandList ( Figure 21 step S405). Capped at the maximum number of merge candidates MaxNumMergeCand, in the P slice, a merge candidate with a motion vector having a value of (0, 0) and a prediction mode of L0 prediction (Pred_L0) is added. In the B slice, a merge candidate with a motion vector having a value of (0, 0) and a prediction mode of bi-prediction (Pred_BI) is added. The reference index at the time of adding the merge candidate is different from the already added reference index.

[0220] Next, in the merge candidate selection unit 347 and the merge candidate selection unit 447, a merge candidate is selected from the merge candidates registered in the merge candidate list mergeCandList. The merge candidate selection unit 347 on the encoding side selects a merge candidate by calculating the code amount and the distortion amount, and provides the merge index indicating the selected merge candidate and the inter-frame prediction information of the merge candidate to the motion compensation prediction unit 306 via the inter-frame prediction mode determination unit 305. On the other hand, in the merge candidate selection unit 447 on the decoding side, a merge candidate is selected based on the decoded merge index and provided to the motion compensation prediction unit 406.

[0221] <Update the historical predicted motion vector candidate list>

[0222] Next, a method for initializing and updating the historical predicted motion vector candidate list HmvpCandList included in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side will be described in detail. Figure 26 is a flowchart for explaining the steps of the historical predicted motion vector candidate list initialization / update process.

[0223] In the present embodiment, it is assumed that the update of the history predicted motion vector candidate list HmvpCandList is performed in the encoding information storage memory 111 and the encoding information storage memory 205. It is also possible to provide a history predicted motion vector candidate list update unit in the inter-frame prediction unit 102 and the inter-frame prediction unit 203 to perform the update of the history predicted motion vector candidate list HmvpCandList.

[0224] Initial setting of the history predicted motion vector candidate list HmvpCandList is performed at the start of a slice. On the encoding side, when the prediction method determination unit 105 selects the normal predicted motion vector mode or the normal merge mode, the history predicted motion vector candidate list HmvpCandList is updated. On the decoding side, when the prediction information decoded by the bit string decoding unit 201 is the normal predicted motion vector mode or the normal merge mode, the history predicted motion vector candidate list HmvpCandList is updated.

[0225] The inter-frame prediction information used for inter-frame prediction in the normal predicted motion vector mode or the normal merge mode is registered in the history predicted motion vector candidate list HmvpCandList as the inter-frame prediction information candidate hMvpCand. The inter-frame prediction information candidate hMvpCand includes the reference index refIdxL0 of L0, the reference index refIdxL1 of L1, the L0 prediction flag predFlagL0 indicating whether L0 prediction is performed, the L1 prediction flag predFlagL1 indicating whether L1 prediction is performed, the motion vector mvL0 of L0, and the motion vector mvL1 of L1.

[0226] In the elements (i.e., inter-frame prediction information) registered in the history predicted motion vector candidate list HmvpCandList in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side, if there is inter-frame prediction information having the same value as the inter-frame prediction information candidate hMvpCand, that element is deleted from the history predicted motion vector candidate list HmvpCandList. On the other hand, if there is no inter-frame prediction information having the same value as the inter-frame prediction information candidate hMvpCand, the element at the head of the history predicted motion vector candidate list HmvpCandList is deleted, and the inter-frame prediction information candidate hMvpCand is added to the end of the history predicted motion vector candidate list HmvpCandList.

[0227] The number of elements in the history predicted motion vector candidate list HmvpCandList provided in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side of the present invention is set to 6.

[0228] First, initialize the historical predicted motion vector candidate list HmvpCandList in units of strips (step S2101 of Figure 26 ). At the beginning of the strip, make all elements of the historical predicted motion vector candidate list HmvpCandList empty, and set the value of the number of historical predicted motion vector candidates (current candidate number) NumHmvpCand registered in the historical predicted motion vector candidate list HmvpCandList to 0.

[0229] In addition, although the initialization of the historical predicted motion vector candidate list HmvpCandList is performed in units of strips (the first coding block of the strip), it can also be performed in units of pictures, in units of rectangles (tiles), or in units of tree blocks.

[0230] Next, repeat the following update process of the historical predicted motion vector candidate list HmvpCandList for each coding block in the strip ( Figure 26 steps S2102 to S2107).

[0231] First, perform an initial setting in units of coding blocks. Set the value of the flag identicalCandExist indicating whether there is the same candidate to FALSE (false), and set the deletion object index removeIdx indicating the candidate to be deleted to 0 ( Figure 26 step S2103).

[0232] Determine whether there is a candidate hMvpCand of the inter-frame prediction information to be registered ( Figure 26 step S2104). When it is determined by the prediction method determination unit 105 on the encoding side as the normal predicted motion vector mode or the normal merge mode, or when it is decoded by the bit string decoding unit 201 on the decoding side as the normal predicted motion vector mode or the normal merge mode, set this inter-frame prediction information as the candidate hMvpCand of the inter-frame prediction information to be registered. When it is determined by the prediction method determination unit 105 on the encoding side as the intra-frame prediction mode, the sub-block predicted motion vector mode, or the sub-block merge mode, or when it is decoded by the bit string decoding unit 201 on the decoding side as the intra-frame prediction mode, the sub-block predicted motion vector mode, or the sub-block merge mode, do not perform the update process of the historical predicted motion vector candidate list HmvpCandList, and there is no candidate hMvpCand of the inter-frame prediction information to be registered. When there is no candidate hMvpCand of the inter-frame prediction information to be registered, skip steps S2105 to S2106 ( Figure 26 Step S2104: No). When there is an inter-frame prediction information candidate hMvpCand for the registration object, the processing after step S2105 is executed ( Figure 26 Step S2104: Yes).

[0233] Next, it is determined whether there is an element (inter-frame prediction information) with the same value as the inter-frame prediction information candidate hMvpCand for the registration object in each element of the historical prediction motion vector candidate list HmvpCandList, that is, whether there is an identical element ( Figure 26 Step S2105). Figure 27 This is the flowchart of the identical element confirmation processing step. When the value of the number of historical prediction motion vector candidates NumHmvpCand is 0 ( Figure 27 Step S2121: No), the historical prediction motion vector candidate list HmvpCandList is empty. Since there are no identical candidates, steps S2122 to S2125 are skipped, and this identical element confirmation processing step ends. When the value of the number of historical prediction motion vector candidates NumHmvpCand is greater than 0 ( Figure 27 Step S2121: Yes), the historical prediction motion vector index hMvpIdx ranges from 0 to NumHmvpCand - 1, and the processing of step S2123 is repeated ( Figure 27 Steps S2122 to S2125). First, it is compared whether the hMvpIdx-th element HmvpCandList[hMvpIdx] counted from 0 in the historical prediction motion vector candidate list is the same as the inter-frame prediction information candidate hMvpCand ( Figure 27 Step S2123). When they are the same ( Figure 27 Step S2123: Yes), the flag identicalCandExist indicating whether there is an identical candidate is set to TRUE (true), and the removal object index removeIdx indicating the position of the element to be deleted is set to the current value of the historical prediction motion vector index hMvpIdx, and this identical element confirmation processing ends. When they are not the same ( Figure 27 Step S2123: No), hMvpIdx is incremented by 1. If the historical prediction motion vector index hMvpIdx is less than or equal to NumHmvpCand - 1, the processing after step S2123 is performed. Figure 27 Step S2123: No), hMvpIdx is incremented by 1. If the historical prediction motion vector index hMvpIdx is less than or equal to NumHmvpCand - 1, the processing after step S2123 is performed.

[0234] Return again Figure 26 to the flowchart, and perform the shifting and adding processing of the elements in the historical prediction motion vector candidate list HmvpCandList ( Figure 26 Step S2106). Figure 28 Yes Figure 26 Flowchart of the element shift / addition processing step of the historical prediction motion vector candidate list HmvpCandList in step S2106. First, it is determined whether to add a new element after removing the elements stored in the historical prediction motion vector candidate list HmvpCandList, or to add a new element without removing the elements. Specifically, it is compared whether the flag identicalCandExist indicating the existence of the same candidate is TRUE (true) or NumHmvpCand is 6 ( Figure 28 step S2141). When either of the conditions that the flag identicalCandExist indicating the existence of the same candidate is TRUE (true) or the current number of candidates NumHmvpCand is 6 is satisfied ( Figure 28 step S2141: Yes), after removing the elements stored in the historical prediction motion vector candidate list HmvpCandList, a new element is added. The initial value of the index i is set to the value of removeIdx + 1. From this initial value to NumHmvpCand, the element shift processing of step S2143 is repeated. ( Figure 28 steps S2142 - S2144). By copying the element of HmvpCandList[i] to HmvpCandList[i - 1], the element is shifted forward ( Figure 28 step S2143), and i is incremented by 1 ( Figure 28 steps S2142 - S2144). Next, the inter-frame prediction information candidate hMvpCand is added to the (NumHmvpCand - 1)th HmvpCandList[NumHmvpCand - 1] starting from 0 corresponding to the end of the historical prediction motion vector candidate list ( Figure 28 step S2145), and the element shift / addition processing of this historical prediction motion vector candidate list HmvpCandList is ended. On the other hand, when neither of the conditions that the flag identicalCandExist indicating the existence of the same candidate is TRUE (true) and NumHmvpCand is 6 is satisfied ( Figure 28 step S2141: No), without removing the elements stored in the historical prediction motion vector candidate list HmvpCandList, the inter-frame prediction information candidate hMvpCand is added to the end of the historical prediction motion vector candidate list ( Figure 28 Step S2146). Here, the last element of the historical predicted motion vector candidate list is HmvpCandList[NumHmvpCand] which is the NumHmvpCand-th element starting from 0. Additionally, increment NumHmvpCand by 1 to end the process of shifting and adding elements to the historical predicted motion vector candidate list HmvpCandList.

[0235] Figures 31A to 31C This is a diagram for explaining an example of the update process of the historical predicted motion vector candidate list. When adding a new element to the historical predicted motion vector candidate list HmvpCandList that already has six registered elements (inter-frame prediction information), the elements from the front of the historical predicted motion vector candidate list HmvpCandList are sequentially compared with the new inter-frame prediction information ( Figure 31A ). If the new element has the same value as the third element HMVP2 from the start of the historical predicted motion vector candidate list HmvpCandList, then delete the element HMVP2 from the historical predicted motion vector candidate list HmvpCandList, and shift (copy) the subsequent elements HMVP3 - HMVP5 one by one to the front, and add the new element to the end of the historical predicted motion vector candidate list HmvpCandList ( Figure 31B ), thus completing the update of the historical predicted motion vector candidate list HmvpCandList ( Figure 31C ).

[0236] <Historical Predicted Motion Vector Candidate Derivation Process>

[0237] Next, a detailed description will be given of the method for deriving historical predicted motion vector candidates from the historical predicted motion vector candidate list HmvpCandList as the processing steps of Figure 20 Step S304. Figure 20 The processing steps of Step S304 are common to the historical predicted motion vector candidate derivation unit 323 of the normal predicted motion vector mode derivation unit 301 on the encoding side and the historical predicted motion vector candidate derivation unit 423 of the normal predicted motion vector mode derivation unit 401 on the decoding side. Figure 29 This is a flowchart for explaining the processing steps of the historical predicted motion vector candidate derivation process.

[0238] When the current number of predicted motion vector candidates numCurrMvpCand is greater than or equal to the maximum number of elements (here it is 2) of the predicted motion vector candidate list mvpListLX or the value of the historical predicted motion vector candidate number NumHmvpCand is 0 ( Figure 29 the No of Step S2201), skip Figure 29 The processing from step S2202 to S2209 ends the historical predicted motion vector candidate derivation processing step. When the current number of predicted motion vector candidates numCurrMvpCand is less than the maximum number of elements 2 of the predicted motion vector candidate list mvpListLX and the value of the historical predicted motion vector candidate number NumHmvpCand is greater than 0 ( Figure 29 in step S2201), execute Figure 29 the processing from step S2202 to S2209.

[0239] Next, repeat Figure 29 the processing from step S2203 to S2208 until either the index i from 1 to 4 or the smaller value of the historical predicted motion vector candidate number numCheckedHMVPCand ( Figure 29 steps S2202 to S2209). When the current number of predicted motion vector candidates numCurrMvpCand is greater than or equal to the maximum number of elements 2 of the predicted motion vector candidate list mvpListLX ( Figure 29 in step S2203: No), omit Figure 29 the processing from step S2204 to S2209 and end this historical predicted motion vector candidate derivation processing step. When the current number of predicted motion vector candidates numCurrMvpCand is less than the maximum number of elements 2 of the predicted motion vector candidate list mvpListLX ( Figure 29 in step S2203: Yes), execute Figure 29 the processing after step S2204.

[0240] Next, perform the processing from step S2205 to S2207 for Y being 0 and 1 (L0 and L1) respectively ( Figure 29 steps S2204 to S2208). When the current number of predicted motion vector candidates numCurrMvpCand is greater than or equal to the maximum number of elements 2 of the predicted motion vector candidate list mvpListLX ( Figure 29 in step S2205: No), omit Figure 29 the processing from step S2206 to S2209 and end this historical predicted motion vector candidate derivation processing step. When the current number of predicted motion vector candidates numCurrMvpCand is less than the maximum number of elements 2 of the predicted motion vector candidate list mvpListLX ( Figure 29 in step S2205: Yes), execute Figure 29 the processing after step S2206.

[0241] Next, in the case where there is an element in the historical predicted motion vector candidate list HmvpCandList that has the same reference index as the reference index refIdxLX of the coded / decoded object motion vector and is different from any element in the predicted motion vector list mvpListLX ( Figure 29 Step S2206: Yes), add the motion vector of LY of the historical predicted motion vector candidate HmvpCandList[NumHmvpCand - i] to the element mvpListLX[numCurrMvpCand] at the numCurrMvpCand-th position (starting from 0) of the predicted motion vector candidate list ( Figure 29 Step S2207), and increment the current number of predicted motion vector candidates numCurrMvpCand by 1. In the case where there is no element in the historical predicted motion vector candidate list HmvpCandList that has the same reference index as the reference index refIdxLX of the coded / decoded object motion vector and is different from any element in the predicted motion vector list mvpListLX ( Figure 29 Step S2206: No), skip the addition process in Step S2207.

[0242] Perform the above Figure 29 processing of Steps S2205 to S2207 in both L0 and L1 ( Figure 29 Steps S2204 to S2208). Increment the index i by 1. When the index i is less than or equal to the smaller value of 4 and the number of historical predicted motion vector candidates NumHmvpCand, perform the processing after Step S2203 again ( Figure 29 Steps S2202 to S2209).

[0243] <Historical merge candidate derivation process>

[0244] Next, a method for deriving historical merge candidates from the historical merge candidate list HmvpCandList, which is the processing step of Figure 21 Step S404, will be described in detail. The processing step of Figure 21 Step S404 is a common process in the historical merge candidate derivation unit 345 of the normal merge mode derivation unit 302 on the encoding side and the historical merge candidate derivation unit 445 of the normal merge mode derivation unit 402 on the decoding side. Figure 30 It is a flowchart showing the processing steps of the historical merge candidate derivation process.

[0245] First, perform initialization processing ( Figure 30 Step S2301). Set the value of each element of isPruned[i] from 0 to the (numCurrMergeCand - 1)-th element to FALSE (false), and set the variable numOrigMergeCand to the number of elements numCurrMergeCand registered in the current merge candidate list.

[0246] Next, set the initial value of the index hMvpIdx to 1, and repeat from this initial value to NumHmvpCand the addition process from Figure 30 Step S2303 to Step S2310 of ( Figure 30 Steps S2302 to S2311). If the number of elements numCurrMergeCand registered in the current merge candidate list is not less than (the maximum number of merge candidates MaxNumMergeCand - 1), then since merge candidates are added to all elements in the merge candidate list, end this historical merge candidate derivation process ( Figure 30 No in Step S2303). If the number of elements numCurrMergeCand registered in the current merge candidate list is less than (the maximum number of merge candidates MaxNumMergeCand - 1), then perform the processing after Step S2304. Set the value of sameMotion to FALSE (false) ( Figure 30 Step S2304). Next, set the initial value of the index i to 0, and perform Figure 30 the processing of Steps S2306 and S2307 from this initial value to numOrigMergeCand - 1 ( Figure 30 S2305 to S2308). Compare whether the (NumHmvpCand – hMvpIdx)-th element HmvpCandList[NumHmvpCand - hMvpIdx] counted from 0 in the historical motion vector prediction candidate list is the same value as the i-th element mergeCandList[i] counted from 0 in the merge candidate list ( Figure 30 Step S2306).

[0247] The same value of merge candidates means that the merge candidates are the same value when the values of all constituent elements (inter-frame prediction mode, reference index, and motion vector) of the merge candidates are the same. In the case where the merge candidates are the same value and isPruned[i] is FALSE (false) ( Figure 30 Yes in Step S2306), both sameMotion and isPruned[i] are set to TRUE (true) ( Figure 30 Step S2307). In the case where they are not the same value ( Figure 30 If the result of step S2306 is negative, skip the processing of step S2307. After Figure 30 After the repeated processing of steps S2305 to S2308 is completed, check whether sameMotion is FALSE ( Figure 30 step S2309). If sameMotion is FALSE ( Figure 30 step S2309 is affirmative), since the (NumHmvpCand - hMvpIdx)-th element HmvpCandList[NumHvpCand - hMvpIdx] of the historical predicted motion vector candidate list does not exist in mergeCandList, add the (NumHmvpCand - hMvpIdx)-th element HmvpCandList[NumHmvpCand - hMvpIdx] counted from 0 of the historical predicted motion vector candidate list to mergeCandList at the numCurrMergeCand-th position of the merge candidate list, and increment numCurrMergeCand by 1 ( Figure 30 step S2310). Increment the index hMvpIdx by 1 ( Figure 30 step S2302), and perform Figure 30 the repeated processing of steps S2302 to S2311.

[0248] After confirming all elements in the historical predicted motion vector candidate list or adding merge candidates to all elements in the merge candidate list, complete the export processing of this historical merge candidate.

[0249] <Average Merge Candidate Export Processing>

[0250] Next, the export method of the average merge candidate, which is the processing step of Figure 21 step S403, will be described in detail. Figure 21 The processing step of step S403 is a common process in the average merge candidate export section 344 of the normal merge mode export section 302 on the encoding side and the average merge candidate export section 444 of the normal merge mode export section 402 on the decoding side. Figure 38 It is a flowchart for explaining the average merge candidate export processing steps.

[0251] First, perform initialization processing ( Figure 38 step S1301). Set the variable numOrigMergeCand to the number of elements numCurrMergeCand registered in the current merge candidate list.

[0252] Next, starting from the beginning of the merge candidate list, scan sequentially to determine two pieces of motion information. Set the index i representing the first piece of motion information to 0, and the index j representing the second piece of motion information to 0. ( Figure 38 Steps S1302 to S1303 of Figure 38 ). If the number of elements numCurrMergeCand registered in the current merge candidate list is not less than (the maximum number of merge candidates MaxNumMergeCand - 1), then merge candidates have been added to all elements of the merge candidate list, so end this historical merge candidate derivation process (

[0253] Steps S1304 of Figure 38 ). If the number of elements numCurrMergeCand registered in the current merge candidate list is less than (the maximum number of merge candidates MaxNumMergeCand - 1), then proceed with the processing after step S1305. Figure 38 Steps S1306 to S1314 of

[0254] Determine whether both the i-th motion information mergeCandList[i] and the j-th motion information mergeCandList[j] in the merge candidate list are invalid ( Figure 38 Steps S1305 of Figure 38 ). If both of them are invalid, then do not derive the average merge candidate of mergeCandList[i] and mergeCandList[j], and transfer to the next element. If mergeCandList[i] and mergeCandList[j] are not both invalid, then set X to 0 and 1, and repeat the following processing ( Figure 38 Steps S1309 of Figure 38 In step S1308, if the LX prediction of mergeCandList[j] is not valid, that is, the LX prediction of mergeCandList[i] is valid and the LX prediction of mergeCandList[j] is invalid, then derive the average merge candidate of the LX prediction of the motion vector and reference index with the LX prediction of mergeCandList[i], and set it as the LX prediction of averageCand to make the LX prediction of averageCand valid ( Figure 38 step S1310). In Figure 38 step S1307, if the LX prediction of mergeCandList[i] is not valid, then determine whether the LX prediction of mergeCandList[j] is valid ( Figure 38 step S1311). If the LX prediction of mergeCandList[j] is valid, that is, if the LX prediction of mergeCandList[i] is invalid and the LX prediction of mergeCandList[j] is valid, then derive the average merge candidate of the LX prediction of the motion vector and reference index with the LX prediction of mergeCandList[j], and set it as the LX prediction of averageCand to make the LX prediction of averageCand valid ( Figure 38 step S1312). In Figure 38 step S1311, if the LX prediction of mergeCandList[j] is not valid, that is, if both the LX prediction of mergeCandList[i] and the LX prediction of mergeCandList[j] are invalid, then make the LX prediction of averageCand invalid ( Figure 38 step S1312).

[0255] Add the average merge candidate averageCand of the L0 prediction, L1 prediction, or BI prediction generated as described above to mergeCandList[numCurrMergeCand] at the numCurrMergeCand-th of the merge candidate list, and increment numCurrMergeCand by 1 ( Figure 38 step S1315). Above, the derivation process of the average merge candidate is completed.

[0256] In addition, the average merge candidate is averaged using the horizontal component of the motion vector and the vertical component of the motion vector respectively.

[0257] <Motion Compensation Prediction Processing>

[0258] The motion compensation prediction unit 306 acquires the position and size of the block that is the object of the current prediction process in encoding. In addition, the motion compensation prediction unit 306 acquires inter-frame prediction information from the inter-frame prediction mode determination unit 305. A reference index and a motion vector are derived from the acquired inter-frame prediction information. After acquiring an image signal at a position where the reference picture determined by the reference index in the decoded image memory 104 is moved by the amount of the motion vector from the same position as the image signal of the prediction block, a prediction signal is generated.

[0259] In the case where the inter-frame prediction mode in inter-frame prediction is a prediction from a single reference picture such as L0 prediction or L1 prediction, the prediction signal obtained from one reference picture is set as the motion compensation prediction signal. In the case where the inter-frame prediction mode is a prediction from two reference pictures such as BI prediction, the signal obtained by weighted averaging the prediction signals obtained from the two reference pictures is set as the motion compensation prediction signal, and the motion compensation prediction signal is provided to the prediction method determination unit 105. Here, the weighted average ratio of the dual prediction is set to 1:1, but other ratios can also be used for weighted averaging. For example, it can be set such that the closer the picture interval between the picture to be predicted and the reference picture is, the greater the weighting ratio. In addition, a correspondence table between the combination of picture intervals and the weighted ratio can also be used to calculate the weighted ratio.

[0260] The motion compensation prediction unit 406 has the same function as the motion compensation prediction unit 306 on the encoding side. The motion compensation prediction unit 406 acquires inter-frame prediction information from the normal prediction motion vector mode derivation unit 401, the normal merge mode derivation unit 402, the sub-block prediction motion vector mode derivation unit 403, and the sub-block merge mode derivation unit 404 through the switch 408. The motion compensation prediction unit 406 provides the obtained motion compensation prediction signal to the decoded image signal overlapping unit 207.

[0261] <Regarding the inter-frame prediction mode>

[0262] The process of performing prediction based on a single reference picture is defined as single prediction. In the case of single prediction, prediction such as L0 prediction or L1 prediction that uses either of the two reference pictures registered in the reference list L0 or L1 is performed.

[0263] Figure 32 The case where the reference picture (RefL0Pic) of L0 in single prediction is at a time before the picture to be processed (CurPic) is shown. Figure 33 The case where the reference picture of L0 prediction in single prediction is at a time after the picture to be processed is shown. Similarly, by Figure 32 and Figure 33 The reference picture for L0 prediction is replaced with the reference picture for L1 prediction (RefL1Pic) for single prediction.

[0264] The process of performing prediction based on two reference pictures is defined as dual prediction. In the case of dual prediction, both L0 prediction and L1 prediction are used and expressed as BI prediction. ​ The case where the reference picture for L0 prediction in dual prediction is at a time before the picture to be processed and the reference picture for L1 prediction is at a time after the picture to be processed is shown. ​ The case where the reference picture for L0 prediction and the reference picture for L1 prediction in dual prediction are at a time before the picture to be processed is shown. ​ The case where the reference picture for L0 prediction and the reference picture for L1 prediction in dual prediction are at a time after the picture to be processed is shown.

[0265] Thus, the relationship between the prediction categories and times of L0 / L1 can be used without L0 being limited to the past direction and L1 being limited to the future direction. Additionally, in the case of dual prediction, each of L0 prediction and L1 prediction can be performed using the same reference picture. Furthermore, based on information (e.g., a flag) indicating whether L0 prediction is used and whether L1 prediction is used, it is determined whether motion compensation prediction is performed by single prediction or dual prediction.

[0266] <Regarding the reference index>

[0267] In an embodiment of the present invention, in order to improve the accuracy of motion compensation prediction, the best reference picture can be selected from multiple reference pictures in motion compensation prediction. Therefore, the reference picture used in motion compensation prediction is used as a reference index, and the reference index is encoded into the bitstream together with the differential motion vector.

[0268] <Motion compensation processing based on the normal prediction motion vector mode>

[0269] As also shown in ​ the inter-frame prediction unit 102 on the encoding side, when the inter-frame prediction information based on the normal prediction motion vector derivation unit 301 is selected in the inter-frame prediction mode determination unit 305, the motion compensation prediction unit 306 obtains the inter-frame prediction information from the inter-frame prediction mode determination unit 305, derives the inter-frame prediction mode, reference index, and motion vector of the block that is the current processing object, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is provided to the prediction method determination unit 105.

[0270] Similarly, as in ​ As shown in the inter-frame prediction unit 203 on the decoding side, when the switch 408 is connected to the normal prediction motion vector mode derivation unit 401 during decoding, the motion compensation prediction unit 406 acquires the inter-frame prediction information based on the normal prediction motion vector mode derivation unit 401, derives the inter-frame prediction mode, reference index, and motion vector of the block that is the current processing target, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlapping unit 207.

[0271] <Motion Compensation Processing Based on Normal Merge Mode>

[0272] As in ​ As shown in the inter-frame prediction unit 102 on the encoding side, when the inter-frame prediction information based on the normal merge mode derivation unit 302 is selected in the inter-frame prediction mode determination unit 305, the motion compensation prediction unit 306 acquires the inter-frame prediction information from the inter-frame prediction mode determination unit 305, derives the inter-frame prediction mode, reference index, and motion vector of the block that is the current processing target, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0273] Similarly, as in ​ As shown in the inter-frame prediction unit 203 on the decoding side, when the switch 408 is connected to the normal merge mode derivation unit 402 during decoding, the motion compensation prediction unit 406 acquires the inter-frame prediction information based on the normal merge mode derivation unit 402, derives the inter-frame prediction mode, reference index, and motion vector of the block that is the current processing target, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlapping unit 207.

[0274] <Motion Compensation Processing Based on Sub-Block Prediction Motion Vector Mode>

[0275] As in ​ As shown in the inter-frame prediction unit 102 on the encoding side, when the inter-frame prediction information based on the sub-block prediction motion vector mode derivation unit 303 is selected in the inter-frame prediction mode determination unit 305, the motion compensation prediction unit 306 acquires the inter-frame prediction information from the inter-frame prediction mode determination unit 305, derives the inter-frame prediction mode, reference index, and motion vector of the block that is the current processing target, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

[0276] Similarly, as in ​ As shown in the inter-frame prediction unit 203 on the decoding side as well, when the switch 408 is connected to the sub-block prediction motion vector mode derivation unit 403 during decoding, the motion compensation prediction unit 406 obtains the inter-frame prediction information based on the sub-block prediction motion vector mode derivation unit 403, derives the inter-frame prediction mode, reference index, and motion vector of the block that is the current processing target, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is provided to the decoded image signal overlapping unit 207.

[0277] <Motion Compensation Processing Based on Sub-Block Merge Mode>

[0278] As also shown in ​ the inter-frame prediction unit 102 on the encoding side as well, when the inter-frame prediction information based on the sub-block merge mode derivation unit 304 is selected in the inter-frame prediction mode determination unit 305, the motion compensation prediction unit 306 obtains the inter-frame prediction information from the inter-frame prediction mode determination unit 305, derives the inter-frame prediction mode, reference index, and motion vector of the block that is the current processing target, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is provided to the prediction method determination unit 105.

[0279] Similarly, as also shown in ​ the inter-frame prediction unit 203 on the decoding side as well, when the switch 408 is connected to the sub-block merge mode derivation unit 404 during decoding, the motion compensation prediction unit 406 obtains the inter-frame prediction information based on the sub-block merge mode derivation unit 404, derives the inter-frame prediction mode, reference index, and motion vector of the block that is the current processing target, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is provided to the decoded image signal overlapping unit 207.

[0280] <Motion Compensation Processing Based on Affine Transformation Prediction>

[0281] In the normal prediction motion vector mode and the normal merge mode, motion compensation based on an affine model can be used based on the following flag. The following flag is reflected in the following flag based on the inter-frame prediction conditions determined by the inter-frame prediction mode determination unit 305 in the encoding process and is encoded into the bitstream. In the decoding process, it is determined whether to perform motion compensation based on the affine model based on the following flag in the bitstream.

[0282] The sps_affine_enabled_flag indicates whether motion compensation based on an affine model can be used in inter prediction. If the sps_affine_enabled_flag is 0, suppression is performed on a sequence basis such that motion compensation is not based on an affine model. Additionally, the inter_affine_flag and cu_affine_type_flag are not transmitted in the syntax of the CU (coded block) of the coded video sequence. If the sps_affine_enabled_flag is 1, motion compensation based on an affine model can be used in the coded video sequence.

[0283] The sps_affine_type_flag indicates whether motion compensation based on a six-parameter affine model can be used in inter prediction. If the sps_affine_type_flag is 0, it is suppressed to be not motion compensation based on a six-parameter affine model. Additionally, the cu_affine_type_flag is not transmitted in the syntax of the CU of the coded video sequence. If the sps_affine_type_flag is 1, motion compensation based on a six-parameter affine model can be used in the coded video sequence. In the absence of the sps_affine_type_flag, it is set to 0.

[0284] When decoding a P slice or a B slice, in the CU that is the current processing target, if the inter_affine_flag is 1, motion compensation based on an affine model is used to generate the motion compensation prediction signal for the CU that is the current processing target. If the inter_affine_flag is 0, the affine model is not used for the CU that is the current processing target. In the absence of the inter_affine_flag, it is set to 0.

[0285] When decoding a P slice or a B slice, in the CU that is the current processing target, if the cu_affine_type_flag is 1, motion compensation based on a six-parameter affine model is used to generate the motion compensation prediction signal for the CU that is the current processing target. If the cu_affine_type_flag is 0, motion compensation based on a four-parameter affine model is used to generate the motion compensation prediction signal for the CU that is the current processing target.

[0286] In motion compensation based on an affine model, since the reference index or motion vector is derived on a sub-block basis, the reference index or motion vector of the processing target is used on a sub-block basis to generate the motion compensation prediction signal.

[0287] The four-parameter affine model is the following pattern: The motion vector of a sub-block is derived from four parameters of the horizontal and vertical components of the respective motion vectors of two control points, and motion compensation is performed on a sub-block basis.

[0288] <Intra Block Copy (IBC)>

[0289] The effective reference area of intra block copy will be described with reference to FIG. 39. ​ This is an example in the case where the coding tree block unit is used as the intra block copy reference block to determine the effective reference area. ​ 500, 501, 502, 503, and 504 are coding tree blocks, and 504 is the coding tree block to be processed. 505 is the coding block to be processed. The processing order of the coding tree blocks is in the order of 500, 501, 502, 503, and 504. In this case, the three coding tree blocks 501, 502, and 503 processed immediately before the coding tree block 504 containing the coding block 505 to be processed are used as the effective reference area of the coding block 505 to be processed. Regardless of the coding tree blocks processed before the coding tree block 501 and whether the processing before the coding block 505 to be processed is completed, all the areas included in the coding tree block 504 containing the coding block 505 to be processed are set as invalid reference areas.

[0290] ​ This is an example in the case where the unit obtained by dividing a coding tree block into four parts is used as the intra block copy reference block to determine the effective reference area. ​ 515 and 516 are coding tree blocks, and 516 is the coding tree block to be processed. The coding tree block 515 is divided into 506, 507, 508, and 509, and 516 is divided into 510, 511, 512, and 513. 514 is the coding block to be processed. The processing order of the intra block copy reference blocks is 506, 507, 508, 509, 510, 511, 512, and 513. In this case, the three intra block copy reference blocks 508, 509, and 510 processed immediately before the intra block copy reference block 511 containing the coding block 514 to be processed are used as the effective reference area of the coding block 514 to be processed. Regardless of the coding tree blocks processed before the intra block copy reference block 508 and whether the processing before the coding block 514 to be processed is completed, all the areas included in the intra block copy reference block 511 containing the coding block 514 to be processed are set as invalid reference areas.

[0291] <Predicted Intra Block Copy: Explanation on the Encoding Side>

[0292] Reference ​ will be used to explain the processing steps of predicted intra block copy on the encoding side.

[0293] First, the block vector mvL is detected by the IBC block vector detection unit 375 ( ​ step S4500 in). Next, the differential block vector of the block vector used in the predicted block vector mode is calculated by the IBC spatial block vector candidate derivation unit 371, the IBC historical block vector candidate derivation unit 372, the IBC predicted block vector candidate supplementation unit 373, the IBC predicted block vector candidate selection unit 376, and the block vector subtraction unit 378 ( ​ steps S4501 to S4503 of).

[0294] The candidates for the predicted block vector are calculated, and the block vector candidate list mvpList is constructed ( ​ step S4501 of). Multiple candidates for the predicted block vector are derived by the IBC spatial block vector candidate derivation unit 371, the IBC historical block vector candidate derivation unit 372, and the IBC predicted block vector candidate supplementation unit 373 in the intra-frame block copy prediction unit 352, and the predicted block vector candidate list mvpList is constructed. The subsequent ​ flowchart description ​ of the detailed processing steps of step S4501.

[0295] Next, the predicted block vector mvpL is selected from the predicted block vector candidate list mvpListL by the IBC predicted block vector candidate selection unit 376 ( ​ step S4502 of). The difference between the block vector mvL and each candidate mvpListL[i] of the predicted block vector stored in the predicted block vector candidate list mvpListL, that is, each differential block vector, is calculated. The code amount when encoding these differential block vectors is calculated for each element of the predicted block vector candidate list mvpListL. Then, among the elements registered in the predicted block vector candidate list mvpListL, the candidate mvpListL[i] of the predicted block vector with the smallest code amount for each candidate of the predicted block vector is selected as the predicted block vector mvpL, and its index i is obtained. When there are multiple candidates of the predicted block vector that result in the smallest generated code amount in the predicted block vector candidate list mvpListL, the candidate mvpListL[i] of the predicted block vector represented by the smaller index number in the predicted block vector candidate list mvpListL is selected as the best predicted block vector mvpL, and the index i is obtained.

[0296] Next, in the block vector subtraction unit 378, the selected predicted block vector mvpL is subtracted from the block vector mvL as mvdL = mvL - mvpL to calculate the differential block vector mvdL ( ​ step S4503 of).

[0297] <Intra-frame block copy: Explanation on the decoding side>

[0298] Next, with reference to ​ the processing steps of the prediction block vector mode on the decoding side will be described. On the decoding side, the block vector used in the prediction block vector mode is calculated by the IBC spatial prediction block vector candidate derivation unit 471, the IBC history block vector candidate derivation unit 472, and the IBC prediction block vector supplement unit 473 ( ​ Steps S4600 to S4602). Specifically, the prediction block vector candidate list mvpListL is calculated, the prediction block vector mvpL is selected, and the block vector mvL is calculated.

[0299] Calculate the candidates of the prediction block vector and construct the prediction block vector candidate list mvpListL ( ​ Step S4601). A plurality of candidates of the prediction block vector are calculated by the IBC spatial block vector candidate derivation unit 471, the IBC history block vector candidate derivation unit 472, and the IBC block vector supplement unit 473 in the intra block copy prediction unit 362, and the prediction block vector candidate list mvpListL is constructed. The detailed processing steps of ​ Step S4601 are omitted. Next, the IBC prediction block vector candidate selection unit 476 takes out the candidate mvpListL[mvpIdxL] of the prediction block vector corresponding to the index mvpIdxL of the prediction block vector decoded and provided by the bit string decoding unit 201 from the prediction block vector candidate list mvpListL as the selected prediction block vector mvpL ( ​ Step S4601). Next, the block vector addition unit 478 adds the differential block vector mvdL decoded and provided by the bit string decoding unit 201 and the prediction block vector mvpL, and calculates the block vector mvL as mvL = mvpL + mvdL ( ​ Step S4602).

[0300] <Prediction Block Vector Mode: Prediction Method of Block Vector>

[0301] ​ is a flowchart showing the processing steps of the prediction intra block copy mode derivation process that represents the functions shared by the intra block copy prediction unit 352 of the moving image encoding device and the intra block copy prediction unit 362 of the moving image decoding device according to the embodiments of the present invention.

[0302] The intra block copy prediction units 352 and 362 include a predicted block vector candidate list mvpListL. The predicted block vector candidate list mvpListL forms a list structure, which is provided with a predicted block vector index indicating a position inside the predicted block vector candidate list and a storage area for storing predicted block vector candidates corresponding to the index as elements. The numbers of the predicted block vector index start from 0, and the predicted block vector candidates are stored in the storage area of the predicted block vector candidate list mvpListL. In the present embodiment, three predicted block vector candidates can be registered in the predicted block vector candidate list mvpListL. Also, a variable numCurrMvpIbcCand indicating the number of predicted block vector candidates registered in the predicted block vector candidate list mvpListL is set to 0.

[0303] The IBC spatial block vector candidate derivation units 371 and 471 derive candidates for the predicted block vector from the block adjacent on the left ([ ​ step S4801 in). In this process, a flag availableFlagLA indicating whether the predicted block vector candidate of the block adjacent on the left (A0 or A1) can be used and a block vector mvLA are derived, and mvLA is appended to the predicted block vector candidate list mvpListL. Subsequently, the IBC spatial block vector candidate derivation units 371 and 471 derive candidates for the predicted block vector from the block adjacent on the top (B0, B1, or B2) ([ ​ step S4802 in). In this process, a flag availableFlagLB indicating whether the predicted motion vector candidate of the block adjacent on the top can be used and a block vector mvLB are derived, and if mvLA and mvLB are not equal, mvLB is appended to the predicted block vector candidate list mvpListL. ​ The processes of steps S4801 and S4802 are common except for the positions and numbers of the adjacent blocks to be referred to, and a flag availableFlagLN indicating whether the predicted block vector candidate of the coded block can be used and a motion vector mvLN (N is A or B, the same hereinafter) are derived.

[0304] Next, the IBC history block vector candidate derivation units 372 and 472 append the history block vector candidates registered in the history block vector candidate list HmvpIbcCandList to the predicted block vector candidate list mvpListL. ([ ​ step S4803 in). For the detailed content of the registration processing steps of this step S4803, see ​ In the description of the operations shown in the flowchart, as long as they are the same as the operations in the case where the motion vector is set as the block vector, the list of reference indices is set as L0, and the historical predicted motion vector candidate list HmvpCandList is set as the historical block vector candidate list HmvpIbcCandList, the description is omitted.

[0305] Next, the IBC prediction block vector supplementing units 373 and 473 append block vectors with predetermined values such as (0, 0) until the prediction block vector candidate list mvpListL is filled ([ ​ of S4804).

[0306] <Intra Block Copy Mode Derivation Unit>

[0307] ​ The intra block copy prediction unit 352 in [

[0308] ​ includes an IBC spatial block vector candidate derivation unit 371, an IBC historical block vector candidate derivation unit 372, an IBC block vector supplementing unit 373, a reference position correction unit 380, a reference area boundary correction unit 381, an IBC merge candidate selection unit 374, and an IBC prediction mode determination unit 377.

[0309] ​ is a flowchart showing the steps of the intra block copy mode derivation process having a common function in the intra block copy prediction unit 352 of the moving image encoding device and the intra block copy prediction unit 362 of the moving image decoding device according to the embodiment of the present invention.

[0310] The intra block copy prediction unit 352 and the intra block copy prediction unit 362 include a merged intra block copy candidate list mergeIbcCandList. The merged intra block copy candidate list mergeIbcCandList forms a list structure, which is provided with a merge index indicating the position inside the merged intra block copy candidate and a storage area for storing the merged intra block copy candidate corresponding to the index as an element. The numbers of the merge index start from 0, and the merged intra block copy candidates are stored in the storage area of the merged intra block copy candidate list mergeIbcCandList. In subsequent processing, the merge candidate of the merge index i registered in the merged intra block copy candidate list mergeIbcCandList is represented as mergeIbcCandList[i]. In the present embodiment, it is assumed that at least 3 merged intra block copy candidates can be registered in the merge candidate list mergeCandList. And, a variable numCurrMergeIbcCand representing the number of the merged intra block copy candidates registered in the merged intra block copy candidate list mergeIbcCandList is set to 0.

[0311] In the IBC spatial block vector candidate derivation unit 371 and the IBC spatial block vector candidate derivation unit 471, spatial merge candidates A and B from blocks adjacent to the left side and the upper side of the processing target block are derived from the encoded information stored in the encoded information storage memory 111 of the moving image encoding device or the encoded information storage memory 205 of the moving image decoding device, and the derived spatial merge candidates are registered in the merged intra block copy candidate list mergeIbcCandList ( ​ step S4701). Here, N representing any one of the spatial merge candidates A and B is defined. A flag availableFlagN indicating whether the intra block copy prediction information of block N can be used as the spatial block vector merge candidate N and a block vector mvL are derived. However, in the present embodiment, since the block vector merge candidate is derived without referring to other encoded blocks included in the block including the encoded block to be processed, the spatial block vector merge candidate included in the block including the encoded block to be processed is not derived.

[0312] Next, in the IBC historical block vector candidate derivation unit 372 and the IBC historical block vector candidate derivation unit 472, the historical prediction block vector candidates registered in the historical prediction block vector candidate list HmvpIbcCandList are appended to the merged intra block copy candidate list mergeIbcCandList ( ​ Step S4702). In this embodiment, it is assumed that when the block vectors of the block vectors already added to the mergeIbcCandList and the historical prediction block vector candidates have the same value, no addition to the mergeIbcCandList is performed.

[0313] Next, when the number of merge candidates numCurrMergeIbcCand registered in the merge intra-block copy candidate list mergeIbcCandList is less than the maximum intra-block merge candidate number MaxNumMergeIbcCand, the IBC prediction block vector supplementing unit 373 and the IBC prediction block vector supplementing unit 473 derive additional intra-block merge candidates with the maximum merge candidate number MaxNumMergeIbcCand as the upper limit from the merge candidates registered in the merge intra-block copy candidate list mergeIbcCandList, and register them in the merge intra-block copy candidate list mergeIbcCandList ( ​ Step S4703). With the maximum merge candidate number MaxNumMergeIbcCand as the upper limit, a block vector with a value of (0, 0) is added to the merge intra-block copy candidate list mergeIbcCandList.

[0314] Next, in the IBC merge candidate selection unit 374 and the IBC merge candidate selection unit 474, one intra-block merge candidate is selected from the intra-block merge candidates registered in the merge intra-block copy candidate list mergeIbcCandList ( ​ Step S4704). The IBC merge candidate selection unit 374 acquires the decoded image at the reference position from the decoded image memory 104, calculates the code amount and the distortion amount to select a merge candidate, and provides a merge index indicating the selected intra-block merge candidate to the IBC prediction mode determination unit 377. The IBC prediction mode determination unit 377 selects whether it is a merge mode by calculating the code amount and the distortion amount, and provides the result to the prediction method determination unit 105. On the other hand, the IBC merge candidate selection unit 474 on the decoding side selects an intra-block merge candidate based on the decoded merge index, and provides the selected intra-block merge candidate to the reference position correction unit 480.

[0315] Next, the reference position correction unit 380 and the reference position correction unit 480 perform a process of correcting the reference position on the intra-block merge candidate ( ​ Step S4705). The detailed processes of the reference position correction unit 380 and the reference position correction unit 480 will be described later.

[0316] Next, the reference area boundary correction unit 381 and the reference area boundary correction unit 481 perform processing to correct the reference area boundary on the intra-block merge candidates (step S4706 in ​ ). The details of the processing of the reference position correction unit 381 and the reference position correction unit 481 will be described later.

[0317] The block copy unit 477 acquires the decoded image at the reference position from the decoded image memory 208 and provides it to the decoded image signal overlapping unit 207. Here, in the block copy unit 477, the luminance component and the chrominance components are copied.

[0318] The above block vector mvL represents the block vector of luminance. The block vector mvC of chrominance becomes as follows when the chrominance format is 420:

[0319] mvC = ((mvL>>(3 + 2)) * 32. Through the above formula, the x and y components of mvC are processed respectively.

[0320] <Reference Position Correction Unit>

[0321] ​ is a flowchart for explaining the processing of the reference position correction unit 380 and the reference position correction unit 480. Currently, it is assumed that the unit of the intra-block copy reference block is a coding tree block (CTU), and its size is not 128×128 pixels.

[0322] First, the positions of the upper left and lower right of the reference block are calculated (S6001). The reference block refers to the block that the processing target coding block references using the block vector. When the upper left of the reference block is set to (xRefTL, yRefTL) and the lower right is set to (xRefBR, yRefBR), it becomes:

[0323] (xRefTL, yRefTL) = (xCb + (mvL[0]>>4), yCb + (mvL[1]>>4))

[0324] (xRefBR, yRefBR) = (xRefTL + cbWidth - 1, yRefTL + cbHeight - 1). Here, the position of the processing target coding block is set to (xCb, yCb), the block vector is set to (mvL[0], mvL[1]), the width of the processing target coding block is set to cbWidth, and the height is set to cbHeight.

[0325] Next, it is determined whether the size of the CTU is 128×128 pixels (S6002). Currently, its size is not 128×128 pixels (S6002: No), so the positions of the upper left and lower right of the referenceable area are calculated (S6003). When the upper left of the referenceable area is set to (xAvlTL, yAvlTL) and the lower right is set to (xAvlBR, yAvlBR), it becomes:

[0326] NL = Min(1, 7 - CtbLog2SizeY) - (1 << ((7 - CtbLog2SizeY) << 1))

[0327] (xAvlTL, yAvlTL) = (((xCb >> CtbLog2SizeY) + NL) << CtbLog2SizeY,

[0328] (yCb >> CtbLog2SizeY) << CtbLog2SizeY)

[0329] (xAvlBR, yAvlBR) = (((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1,

[0330] ((((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) - 1). Here, the size of the CTU is CtbLog2SizeY.

[0331] Next, it is determined whether the reference position in the x direction of the reference block is smaller than the upper left of the referenceable area (S6004). If the determination is false (S6004: No), the next process is entered (S6006). On the other hand, if the determination is true (S6004: Yes), the reference position in the x direction is corrected to be the same as the upper left of the referenceable area (S6005).

[0332] ​ FIG. 6001 shows the case of correcting the reference position. 6001 represents the coding tree block to be processed, 6002 represents the coding block to be processed, and 6003 represents the referenceable area. Currently, if the reference block r2 is located at 6011, the reference position in the x direction is smaller than the upper left of the referenceable area (S6004: Yes). Therefore, xRefTL = xAvlTL is set, and the reference position is corrected to the position at 6012 (S6005). Here, as shown in S6001, since xRefBR = xRefTL + cbWidth - 1, as xRefTL is corrected, xRefBR is also corrected. In the correction of this reference position, the block vector mvL[0] can also be corrected. That is, it is corrected to:

[0333] mvL[0] = (xAvlTL - xCb) << 4. Thus, xRefTL = xAvlTL, and thus the reference position can be corrected.

[0334] In this way, when the reference block is outside the referenceable area, by correcting the reference position, referencing can be performed.

[0335] Currently, it is assumed that some of the block vectors in the block vector candidate list constructed in the intra block copy prediction unit 352 are outside the referenceable area. Without correcting the reference position, referencing based on these block vectors cannot be performed, and thus these block vectors cannot be used as candidates for the IBC merge mode. On the other hand, when the reference position is corrected in the present invention, all the block vectors in the constructed block vector candidate list are inside the referenceable area. Therefore, referencing based on all the block vectors can be performed, and all the block vectors can be set as candidates for the IBC merge mode. Therefore, the IBC merge mode selection unit 374 can select the best prediction mode from the candidates for each IBC merge mode corresponding to all the block vectors, thereby improving the coding efficiency.

[0336] Currently, it is assumed that some of the block vectors in the block vector candidate list constructed in the intra block copy prediction unit 362 are outside the referenceable area. Without correcting the reference position, referencing based on these block vectors cannot be performed, and thus the IBC merge mode using these block vectors cannot be decoded. In an encoding device not according to the present invention, the merge index indicating the IBC merge mode using these block vectors acts as an index not to be encoded. However, due to operation failures or the like, there is a possibility that such a merge index is encoded to generate a bitstream. Or, due to packet loss or the like, a part of the bitstream is missing, and the decoding result may be such a merge index. If such an incomplete bitstream is to be decoded, there is a possibility of accessing an incorrect position in the decoded image memory by referring to the outside of the referenceable area. As a result, depending on the decoding device, the decoding result is different, or the decoding process stops. On the other hand, when the reference position is corrected in the present invention, all the block vectors in the constructed block vector candidate list are inside the referenceable area. Therefore, even if such an incomplete bitstream is decoded, the reference position is corrected to the inside of the referenceable area and referencing can be performed. In this way, the memory access range is guaranteed by correcting the reference position. As a result, the decoding results of the decoding device are the same, and the decoding process can be continued, and thus the robustness of the decoding device can be improved.

[0337] In addition, when correcting the block vector in the correction of the reference position, the object is the block vector of luminance. Here, the block vector of the color difference is calculated based on the block vector of luminance. That is, if the block vector of luminance is corrected, the block vector of the color difference is also corrected. Therefore, in the case of the color difference, it is not necessary to correct the reference position again. Compared with the case where it is necessary to determine whether both luminance and color difference can be used as references without correcting the block vector, the processing amount can be reduced.

[0338] Furthermore, when correcting the block vector in the correction of the reference position, the corrected block vector is stored as the block vector of the coded block to be processed in the coded information storage memory 111 or the coded information storage memory 205. That is, the corrected reference position and the position indicated by the block vector are the same. Here, when the decoding result is stored in the decoded image memory, deblocking filtering processing may be performed. In this filtering processing, the strength of the filter is controlled according to the difference between the block vectors of the two blocks facing the block boundary. In the case of not correcting the block vector, compared with the case where the corrected reference position and the position indicated by the block vector are different, since it is a more appropriate filter strength, the coding efficiency can be improved.

[0339] Next, it is determined whether the reference position in the y direction of the reference block is smaller than the upper left of the referenceable area (S6006). If the determination is false (S6006: No), the process proceeds to the next process (S6008). On the other hand, if the determination is true (S6006: Yes), the reference position in the y direction is corrected to be consistent with the upper left of the referenceable area (S6007).

[0340] Currently, if the reference block r4 is located at 6021, the reference position in the y direction is smaller than the upper left of the referenceable area (S6006: Yes). Therefore, it is set that yRefTL = yAvlTL, and the reference position is corrected to the position of 6022 (S6007). Here, as shown in S6001, since yRefBR = yRefTL + cbHeight - 1, as yRefTL is corrected, yRefBR is also corrected. In this correction of the reference position, the block vector mvL[1] can also be corrected. That is, it is corrected to:

[0341] mvL[1] = (yAvlTL - yCb) << 4. Thus, since yRefTL = yAvlTL, the reference position can be corrected.

[0342] Next, it is determined whether the reference position in the x direction of the reference block is larger than the lower right of the referenceable area (S6008). If the determination is false (S6008: No), the process proceeds to the next process (S6010). On the other hand, if the determination is true (S6008: Yes), the reference position in the x direction is corrected to be consistent with the lower right of the referenceable area (S6009).

[0343] Currently, if the reference block r7 is at 6031, the reference position in the x direction is larger than the lower right of the referenceable area (S6008: Yes). Therefore, set xRefBR = xAvlBR, and correct the reference position to the position of 6032 (S6009). Here, as shown in S6001, since xRefBR = xRefTL + cbWidth - 1, that is, xRefTL = xRefBR - (cbWidth - 1), as xRefBR is corrected, xRefTL is also corrected. In the correction of this reference position, the block vector mvL[0] can also be corrected. That is, correct it to:

[0344] mvL[0] = (xAvlBR - (xCb + cbWidth - 1)) << 4. Thus, xRefBR = xAvlBR, so the reference position can be corrected.

[0345] Next, determine whether the reference position of the reference block in the y direction is larger than the lower right of the referenceable area (S6010). If the determination is false (S6010: No), the process ends. On the other hand, if the determination is true (S6010: Yes), correct the reference position in the y direction to be consistent with the lower right of the referenceable area (S6011).

[0346] Currently, if the reference block r5 is at 6041, the reference position in the y direction is larger than the lower right of the referenceable area (S6010: Yes). Therefore, set yRefBR = yAvlBR, and correct the reference position to the position of 6042 (S6011). Here, as shown in S6001, since yRefBR = yRefTL + cbHeight - 1, that is, yRefTL = yRefBR - (cbHeight - 1), as yRefBR is corrected, yRefTL is also corrected. In the correction of this reference position, the block vector mvL[1] can also be corrected. That is, correct it to:

[0347] mvL[1] = (yAvlBR - (yCb + cbHeitght - 1)) << 4. Thus, yRefBR = yAvlBR, so the reference position can be corrected.

[0348] Here, the case where the reference block r1 is at 6051 is described. In this case, similar to the case where the reference block is r2, correct the reference position in the x direction. And similar to the case where the reference block is r4, correct the reference position in the y direction. As a result, the reference block r1 is at 6052 which is inside the referenceable area.

[0349] When the reference block r3 is located at 6061, the reference block r6 is located at 6062, and the reference block r8 is located at 6063, the reference positions in the x and y directions are corrected in the same manner as described above. As a result, each reference block is located inside the referenceable area.

[0350] Thus, the processing ends when the size of the CTU is not 128×128 pixels. On the other hand, when the size of the CTU is 128×128 pixels (S6002: Yes), the upper left and lower right positions are calculated when the referenceable area is rectangular (S6012).

[0351] FIG. 50 is a diagram for explaining the upper left and lower right positions when the referenceable area is rectangular. In ​ this case, the coding tree block 6101 to be processed is divided into four, and the coding block 6102 to be processed is located at the upper left of the division. At this time, the referenceable area becomes an inverted L shape as shown by the diagonal lines in 6103. When the referenceable area is rectangular, its range is set to the rectangular range of 6103. When the referenceable area is rectangular, if the upper left of the reference block is set to (xRefTL, yRefTL) and the lower right is set to (xRefBR, yRefBR), then:

[0352] offset[4] = {0, 64, 128, 128}

[0353] NL = -offset[3 - blk_idx], NR = offset[blk_idx]

[0354] (xAvlTL, yAvlTL) = ((xCb >> CtbLog2SizeY) << CtbLog2SizeY + NL,

[0355] (yCb >> CtbLog2SizeY) << CtbLog2SizeY)

[0356] (xAvlBR, yAvlBR) = ((((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1 + NR,

[0357] ((((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) - 1)).

[0358] Here, blk_idx is an index indicating the position of the coded block of the object to be processed. After the coded tree block of the object to be processed is divided into four parts, when the coded block of the object to be processed is in the upper left, blk_idx is set to 0. Similarly, when the coded blocks of the object to be processed are in the upper right, lower left, and lower right respectively, blk_idx is set to 1, 2, and 3. ​ is a diagram showing the case where blk_idx = 0. Similarly, Figures 50B to 50D are diagrams showing the cases where blk_idx = 1 to 3 respectively.

[0359] Next, correct the reference position of the part where the reference area is not rectangular (S6013). Figure 51 is a flowchart explaining the process of correcting the reference position of the part where the reference area is not rectangular. First, calculate the position of the upper left of the reference area (S6021). The reference area is the slanted part in Fig. 50, so except for the case of blk_idx = 3, there are two points, 6111 and 6112, at the upper left position. If they are set as (X1, Y1) and (X2, Y2) respectively, it becomes:

[0360] offset[4] = {64, 128, 64, 0}, NL = offset[blk_idx]

[0361] (X1, Y1) = (xAvlTL, yAvlTL + 64)

[0362] (X2, Y2) = (xAvlTL + NL, yAvlTL).

[0363] Next, determine whether to correct the reference position in accordance with the upper left of the reference area (S6022). In this determination, it is determined to be true (S6022: yes) when it is not blk_idx = 3 and the reference block is in an area smaller than X2 and Y1. When it is false (S6022: no), proceed to the next process (S6026).

[0364] Next, determine whether the difference in the x direction between the reference block and the reference area is less than the difference in the y direction between the reference block and the reference area (S6023). When the determination is true (S6023: yes), correct the reference position in the x direction (S6024). On the other hand, when the determination is false (S6023: no), correct the reference position in the y direction (S6025).

[0365] Figure 52A This is a diagram showing the situation of correcting the reference position in S6024 and S6025. Currently, blk_idx = 0. If the reference block r1 is located at 6201, then it is not blk_idx = 3, and the upper left of the reference block is located in an area smaller than X2 (the x-direction of 6112) and Y1 (the y-direction of 6111) (S6022: Yes). Additionally, the difference in the x-direction between the reference block and the referenceable area is less than the difference in the y-direction between the reference block and the referenceable area (S6023: Yes). Therefore, set xRefTL = xAvlTL + NL, and correct the reference position in the x-direction to the position of 6202 (S6024). Here, as shown in S6001, since xRefBR = xRefTL + cbWidth - 1, as xRefTL is corrected, xRefBR is also corrected. In the correction of this reference position, the block vector mvL[0] can also be corrected. That is, correct it to:

[0366] mvL[0] = (xAvlTL + NL - xCb) << 4. Thus, xRefTL = xAvlTL + NL, so the reference position can be corrected.

[0367] On the other hand, if the reference block r2 is located at 6203, then it is not blk_idx = 3, and the upper left of the reference block is located in an area smaller than X2 (the x-direction of 6112) and Y1 (the y-direction of 6111) (S6022: Yes). Additionally, the difference in the x-direction between the reference block and the referenceable area is not less than the difference in the y-direction between the reference block and the referenceable area (S6023: No). Therefore, set yRefTL = yAvlTL + 64, and correct the reference position in the y-direction to the position of 6204 (S6025). Here, as shown in S6001, since yRefBR = yRefTL + cbHeight - 1, as yRefTL is corrected, yRefBR is also corrected. In the correction of this reference position, the block vector mvL[0] can also be corrected. That is, correct it to:

[0368] mvL[1] = (yAvlTL + 64 - yCb) << 4. Thus, yRefTL = yAvlTL + 64, so the reference position can be corrected.

[0369] Here, assume that the reference block r3 is located at 6205. In this case, the difference in the x-direction between the reference block and the referenceable area is less than the difference in the y-direction between the reference block and the referenceable area (S6023: Yes). Therefore, by correcting the reference position in the x-direction in the same way as the reference block r1, it is located at 6206 (S6024). At this moment, the reference block is outside the referenceable area. However, through the processing of S6006 and S6007 described later, the reference position in the y-direction is corrected. As a result, the reference block becomes inside the referenceable area.

[0370] Next, calculate the position of the lower right of the referenceable area (S6026). Since the referenceable area is the slanted part in Fig. 50, except for the case where blk_idx = 0, there are two points, 6113 and 6114, at the lower right. If they are set as (X3, Y3) and (X4, Y4) respectively, it becomes:

[0371] offset[4] = {0, 64, 128, 64}, NR = offset[blk_idx]

[0372] (X3, Y3) = (xAvlBR, yAvlBR - 64)

[0373] (X4, Y4) = (xAvlBR - NR, yAvlBR).

[0374] Next, determine whether to correct the reference position in accordance with the lower right of the referenceable area (S6027). In this determination, it is determined to be true when it is not blk_idx = 0 and the reference block is in an area larger than X4 and Y3 (S6027: Yes). In the case of false (S6027: No), the process ends.

[0375] Next, determine whether the difference in the x - direction between the reference block and the referenceable area is less than the difference in the y - direction between the reference block and the referenceable area (S6028). In the case of a true determination (S6028: Yes), correct the reference position in the x - direction (S6029). On the other hand, in the case of a false determination (S6028: No), correct the reference position in the y - direction (S6030).

[0376] Figure 52B Fig. shows the situation of correcting the reference position in S6029 and S6030. Currently, blk_idx = 3. If the reference block r1 is at 6211, then it is not blk_idx = 0, and the lower right of the reference block is in an area larger than X4 (the x - direction of 6114) and Y3 (the y - direction of 6113) (S6027: Yes). In addition, the difference in the x - direction between the reference block and the referenceable area is less than the difference in the y - direction between the reference block and the referenceable area (S6028: Yes). Therefore, set xRefBR = xAvlBR, and correct the reference position in the x - direction to the position of 6212 (S6029). Here, as shown in S6001, since xRefBR = xRefTL + cbWidth - 1, that is, xRefTL = xRefBR - (cbWidth - 1), as xRefBR is corrected, xRefTL is also corrected. In the correction of this reference position, the block vector mvL[0] can also be corrected. That is, correct it to:

[0377] mvL[0] = (xAvlBR - NR - (xCb + cbWidth - 1)) << 4. Thus, xRefBR = xAvlBR, so the reference position can be corrected.

[0378] On the other hand, if the reference block r2 is located at 6213, then blk_idx ≠ 0, and the lower right of the reference block is in a region larger than X4 (in the x - direction of 6114) and Y3 (in the y - direction of 6113) (S6027: Yes). Also, the difference in the x - direction between the reference block and the referenceable region is not less than the difference in the y - direction between the reference block and the referenceable region (S6028: No). Therefore, let yRefBR = yAvlBR, and correct the reference position in the y - direction to the position of 6214 (S6030). Here, as shown in S6001, since yRefBR = yRefTL + cbHeight - 1, that is, yRefTL = yRefBR - (cbHeight - 1), so as yRefBR is corrected, yRefTL is also corrected. In the correction of this reference position, the block vector mvL[1] can also be corrected. That is, it is corrected to:

[0379] mvL[1] = (yAvlBR - 64 - (yCb + cbHeight - 1)) << 4. Thus, yRefBR = yAvlBR, so the reference position can be corrected.

[0380] Here, assume the reference block r3 is located at 6215. In this case, the difference in the x - direction between the reference block and the referenceable region is not smaller than the difference in the y - direction between the reference block and the referenceable region (S6028: No). Therefore, by correcting the reference position in the y - direction in the same way as the reference block r2, it is located at 6216 (S6030). At this moment, the reference block is outside the referenceable region. However, through the processing of S6008 and S6009 described later, the reference position in the x - direction is corrected. As a result, the reference block becomes inside the referenceable region.

[0381] In Fig. 52, the process of correcting the reference position is described taking the cases of blk_idx = 0 and 3 as examples. In the case of blk_idx = 1 or 2, the process of correcting the reference position is carried out in the same way as in the cases of blk_idx = 0 and 3.

[0382] After the process of correcting the reference position for the part where the referenceable region is not rectangular (S6013), the processes of S6004 to S6011 are carried out. Thus, the processing for the case where the size of the CTU is 128×128 pixels is completed.

[0383] Currently, it is assumed that in the process of correcting the reference position of the non-rectangular part of the referenceable area (S6013), the process of correcting the reference position in the x direction is performed in accordance with the upper left of the referenceable area (S6024). Then, the reference position of the reference block in the x direction will not become smaller than the upper left of the referenceable area, so the determination in S6004 is always false (S6004: No). Therefore, when the process of S6024 is performed, the processes of S6004 and S6005 may not be performed. Similarly, when the process of S6025 is performed, the processes of S6006 and S6007 may not be performed. When the process of S6029 is performed, the processes of S6008 and S6009 may not be performed. When the process of S6030 is performed, the processes of S6010 and S6011 may not be performed.

[0384] In addition, in Figure 51 the flowchart, the comparison process of step S6023 can be omitted, and a structure that always executes step S6024 can be adopted, or a structure that always executes step S6025 can be adopted. Similarly, the comparison process of step S6028 can be omitted, and a structure that always executes step S6029 can be adopted, or a structure that always executes step S6030 can be adopted. In such a structure, the reference position can be corrected by a simple process.

[0385] In Figure 48 , when the size of the CTU is 128×128 pixels, the reference position is corrected by using the processes of S6012, S6013, and S6004 to S6011. Instead, as Figure 53 shown, it can also be achieved by the process of decomposing the referenceable area into two and correcting each reference position (S6101).

[0386] FIG. 54 is a diagram illustrating the case where the referenceable area is decomposed into two. Different from the case where the referenceable area is set to a rectangular shape in FIG. 50, the referenceable area is decomposed into two in FIG. 54. After the coding tree block (6101) to be processed is divided into four, when the coding block (6102) to be processed is located in the upper left, blk_idx = 0 is set. Similarly, when the coding blocks to be processed are located in the upper right, lower left, and lower right respectively, blk_idx is set to 1, 2, and 3. Figure 54A is a diagram showing the case of blk_idx = 0. Similarly, Figures 54B to 54D are diagrams showing the cases of blk_idx = 1 to 3 respectively. In addition, one referenceable area (6301) is set as referenceable area A, and the other referenceable area (6302) is set as referenceable area B.

[0387] Figure 55 This is a flowchart illustrating the process (S6101) of decomposing the referenceable region into two and correcting each reference position. In Figure 55 For the same processing as Figure 48 the same step numbers are assigned, and the description is omitted. First, the positions of the upper left and lower right of the referenceable region A are calculated (S6111). When the upper left of the referenceable region A is set as (xAvlTL, yAvlTL) and the lower right is set as (xAvlBR, yAvlBR), it becomes:

[0388] xOffsetTL[4] = {-128, -128, -64, 0}, yOffsetTL[4] = {64, 64, 64, 0}

[0389] xOffsetBR[4] = {0, 0, 0, 128}, yOffsetBR[4] = {128, 128, 128, 64}

[0390] (xAvlTL, yAvlTL) = ((xCb >> CtbLog2SizeY) << CtbLog2SizeY + xOffsetTL[blk_idx],

[0391] (yCb >> CtbLog2SizeY) << CtbLog2SizeY + yOffsetTL[blk_idx])

[0392] (xAvlBR, yAvlBR) = (((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1

[0393] + xOffsetBR[blk_idx],

[0394] (((yCb >> CtbLog2SizeY + 1) << CtbLog2SizeY) - 1 + yOffsetBR[blk_idx].

[0395] Next, regarding whether the reference block is outside the referenceable region A, the following calculations are made (S6112).

[0396] out_xRefTL = xRefTL < xAvlTL

[0397] out_yRefTL = yRefTL < yAvlTL

[0398] out_xRefBR = xRefBR > xAvlBR

[0399] out_yRefBR = yRefBR > yAvlBR

[0400] Next, calculate the positions of the upper left and lower right of the referenceable region B (S6113). When the upper left of the referenceable region B is set to (xAvlTL, yAvlTL) and the lower right is set to (xAvlBR, yAvlBR), it becomes:

[0401] xOffsetTL[4] = {-64, 0, 0, 0}, yOffsetTL[4] = {0, 0, 0, 0}

[0402] xOffsetBR[4] = {0, 64, 128, 64}, yOffsetBR[4] = {128, 64, 64, 128}

[0403] (xAvlTL, yAvlTL) = ((xCb >> CtbLog2SizeY) << CtbLog2SizeY + xOffsetTL[blk_idx],

[0404] (yCb >> CtbLog2SizeY) << CtbLog2SizeY + yOffsetTL[blk_idx])

[0405] (xAvlBR, yAvlBR) = ((((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1

[0406] + xOffsetBR[blk_idx],

[0407] ((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) - 1 + yOffsetBR[blk_idx]).

[0408] Next, determine whether the reference position in the x direction of the reference block is smaller than the upper left of the referenceable region A and whether the reference position in the x direction of the reference block is smaller than the upper left of the referenceable region B (S6114). If the determination is false (S6114: No), then proceed to the next process (S6116). On the other hand, if the determination is true (S6114: Yes), then correct the reference position in the x direction to be consistent with the upper left of the referenceable region B (S6005). The process of S6005 has been described, so the description is omitted.

[0409] Subsequently, it is determined whether the reference position in the y direction of the reference block is smaller than the upper left of the referenceable area A and whether the reference position in the y direction of the reference block is smaller than the upper left of the referenceable area B (S6116). If the determination is false (S6116: No), the process proceeds to the next process (S6118). On the other hand, if the determination is true (S6116: Yes), the reference position in the y direction is corrected to be consistent with the upper left of the referenceable area B (S6007). The process of S6007 has been described, so the description is omitted.

[0410] Next, it is determined whether the reference position in the x direction of the reference block is larger than the lower right of the referenceable area A and whether the reference position in the x direction of the reference block is larger than the lower right of the referenceable area B (S6118). If the determination is false (S6118: No), the process proceeds to the next process (S6120). On the other hand, if the determination is true (S6118: Yes), the reference position in the x direction is corrected to be consistent with the lower right of the referenceable area B (S6009). Since the process of S6009 has been described, the description is omitted.

[0411] Next, it is determined whether the reference position in the y direction of the reference block is larger than the lower right of the referenceable area A and whether the reference position in the y direction of the reference block is larger than the lower right of the referenceable area B (S6120). If the determination is false (S6120: No), the process ends (S6120). On the other hand, if the determination is true (S6120: Yes), the reference position in the y direction is corrected to be consistent with the lower right of the referenceable area B (S6011). Since the process of S6011 has been described, the description is omitted.

[0412] Thus, when the size of the CTU is 128×128 pixels, even if the reference block is outside the referenceable area, the reference position can be corrected for reference. In addition, by decomposing the referenceable area into two and correcting each reference position, the process can be simplified and the amount of computation can be reduced. Here, one referenceable area (6301) is set as the referenceable area A, and the other referenceable area (6302) is set as the referenceable area B. Instead, the referenceable area A and the referenceable area B can be swapped, and one referenceable area (6301) can be set as the referenceable area B, and the other referenceable area (6302) can be set as the referenceable area A for processing.

[0413] In this embodiment, it is determined whether the size of the CTU is 128×128 pixels (S6002), and the process is switched. This can determine whether the intra-block copy reference block is the unit obtained by quartering the coding tree block, or can determine whether the size of the CTU is larger than the maximum size of the coding block.

[0414] All of the above-described embodiments can be combined in multiple ways.

[0415] In all of the above-described embodiments, the bitstream output by the image encoding device has a specific data format so as to be decodable according to the encoding method used in the embodiment. The bitstream can be provided by being recorded on a recording medium such as an HDD, SSD, flash memory, or optical disc that can be read by a computer or the like, or can be provided from a server via a wired or wireless network. Accordingly, an image decoding device corresponding to the image encoding device can decode the bitstream in the specific data format without using the providing means.

[0416] In the case where a wired or wireless network is used to exchange the bitstream between the image encoding device and the image decoding device, the bitstream can be converted into a data format suitable for the transmission form of the communication line and transmitted. In this case, a transmitting device that converts the bitstream output from the image encoding device into encoded data in a data format suitable for the transmission form of the communication line and transmits the encoded data to the network, and a receiving device that receives the encoded data from the network and restores the encoded data to a bitstream and provides it to the image decoding device are provided. The transmitting device includes: a memory that buffers the bitstream output from the image encoding device; a packet processing unit that packets the bitstream; and a transmitting unit that transmits the packeted encoded data via the network. The receiving device includes: a receiving unit that receives the packeted encoded data via the network; a memory that buffers the received encoded data; and a packet processing unit that performs packet processing on the encoded data to generate a bitstream and provides the bitstream to the image decoding device.

[0417] In order to exchange the bitstream between the image encoding device and the image decoding device, in the case of using a wired or wireless network, in addition to the transmitting device and the receiving device, a relay device that receives the encoded data transmitted by the transmitting device and provides it to the receiving device can also be provided. The relay device includes: a receiving unit that receives the packeted encoded data transmitted by the transmitting device; a memory that buffers the received encoded data; and a transmitting unit that transmits the packeted encoded data to the network. Further, the relay device can also include: a receiving packet processing unit that performs packet processing on the packeted encoded data to generate a bitstream; a recording medium that stores the bitstream; and a transmitting packet processing unit that packets the bitstream.

[0418] Further, a display unit that displays the image decoded by the image decoding device can be added to the configuration as a display device. In this case, the display unit reads out the decoded image signal generated by the decoded image signal overlapping unit 207 and stored in the decoded image memory 208, and displays it on the screen.

[0419] Alternatively, a photographing unit may be added to the configuration, and the photographed image may be input into the image encoding device, thereby serving as a photographing device. In this case, the photographing unit inputs the photographed image signal into the block segmentation unit 101.

[0420] Figure 37 FIG. 1 shows an example of the hardware configuration of the encoding / decoding device according to the present embodiment. The encoding / decoding device includes the configurations of the image encoding device and the image decoding device according to the embodiments of the present invention. The encoding / decoding device 9000 includes a CPU 9001, a codec IC 9002, an I / O interface 9003, a memory 9004, an optical disk drive 9005, a network interface 9006, and a video interface 9009, and each part is connected via a bus 9010.

[0421] The image encoding unit 9007 and the image decoding unit 9008 are typically installed as the codec IC 9002. The image encoding process of the image encoding device according to the embodiments of the present invention is executed by the image encoding unit 9007, and the image decoding process in the image decoding device according to the embodiments of the present invention is executed by the image decoding unit 9008. The I / O interface 9003 is implemented by, for example, a USB interface and is connected to an external keyboard 9104, a mouse 9105, etc. The CPU 9001 controls the encoding / decoding device 9000 based on user operations input via the I / O interface 9003 to perform actions desired by the user. Examples of operations performed by the user via the keyboard 9104, the mouse 9105, etc. include selection of which of encoding and decoding to perform, setting of encoding quality, input / output destinations of bitstreams, input / output destinations of images, etc.

[0422] When the user desires to reproduce an image recorded on the disc recording medium 9100, the optical disk drive 9005 reads the bitstream from the inserted disc recording medium 9100 and transmits the read bitstream to the image decoding unit 9008 of the codec IC 9002 via the bus 9010. The image decoding unit 9008 performs the image decoding process in the image decoding device according to the embodiments of the present invention on the input bitstream and transmits the decoded image to an external monitor 9103 via the video interface 9009. In addition, the encoding / decoding device 9000 has a network interface 9006 and can be connected to an external distribution server 9106 and a portable terminal 9107 via a network 9101. When the user desires to reproduce an image recorded on the distribution server 9106 or the mobile terminal 9107 rather than an image recorded on the disc recording medium 9100, the network interface 9006 obtains the bitstream from the network 9101 instead of reading the bitstream from the inserted disc recording medium 9100. In addition, when the user desires to reproduce an image recorded in the memory 9004, the bitstream recorded in the memory 9004 is subjected to the image decoding process in the image decoding device according to the embodiments of the present invention.

[0423] In the case where the user wishes to encode an image captured by an external camera 9102 and record it in the memory 9004, the video interface 9009 inputs the image from the camera 9102 and sends it via the bus 9010 to the image encoding unit 9007 of the codec IC 9002. The image encoding unit 9007 performs image encoding processing in the image encoding apparatus according to an embodiment of the present invention on the image input via the video interface 9009, and generates a bitstream. Then, the bitstream is sent to the memory 9004 via the bus 9010. When the user wishes to record the bitstream on the disc recording medium 9100 instead of in the memory 9004, the optical disc drive 9005 reads and writes the bitstream for the inserted disc recording medium 9100.

[0424] It is also possible to implement a hardware structure having an image encoding apparatus without an image decoding apparatus, or a hardware structure having an image decoding apparatus without an image encoding apparatus. Such a hardware structure is implemented, for example, by replacing the codec IC 9002 with the image encoding unit 9007 or the image decoding unit 9008, respectively.

[0425] The processes related to the above encoding and decoding can of course be implemented as transfer, storage, and reception apparatuses using hardware, and can also be implemented by firmware stored in a ROM (read-only memory), flash memory, etc., or software of a computer, etc. The firmware program and software program can be provided by being recorded on a computer-readable recording medium, can also be provided from a server via a wired or wireless network, and can also be provided as data broadcasting of terrestrial wave or satellite digital broadcasting.

[0426] As described above, the present invention has been described based on the embodiments. The embodiments are illustrative, and various modification examples are possible for the combination of these respective constituent elements and respective processing steps, and such modification examples are also within the scope of the present invention, which can be understood by those skilled in the art.

[0427] Industrial Applicability

[0428] The present invention can be applied to image encoding and decoding techniques for predicting an image by dividing it into blocks.

[0429] Symbol Explanation

[0430] 100 Image encoding device, 101 Block division unit, 102 Inter-frame prediction unit, 103 Intra-frame prediction unit, 104 Decoded image memory, 105 Prediction method determination unit, 106 Residual generation unit, 107 Orthogonal transformation / Quantization unit, 108 Bit string encoding unit, 109 Inverse quantization / Inverse orthogonal transformation unit, 110 Decoded image signal overlapping unit, 111 Encoding information storage memory, 200 Image decoding device, 201 Bit string decoding unit, 202 Block division unit, 203 Inter-frame prediction unit, 204 Intra-frame prediction unit, 205 Encoding information storage memory, 206 Inverse quantization / Inverse orthogonal transformation unit, 207 Decoded image signal overlapping unit, 208 Decoded image memory.< / poc>

Claims

1. An image encoding device that encodes in units of intra-block copy reference blocks, characterized in that, it includes: a block vector candidate derivation unit that derives block vector candidates for a processing target block in a processing target picture from encoding information stored in an encoding information storage memory; a predicted block vector supplementation unit that adds (0, 0) to the block vector candidates; a selection unit that selects a selected block vector from the block vector candidates; a reference position correction unit that corrects the reference position of a reference block referenced by the selected block vector so that it refers to the inside of a referenceable area; and a prediction unit that obtains decoded pixels in the processing target picture from a decoded image memory as prediction values for the processing target block based on the reference position of the reference block, wherein the reference position correction unit uses a fixed number of intra-block copy reference blocks that were encoded immediately before the intra-block copy reference block containing the processing target block as the referenceable area, and sets the intra-block copy reference blocks before the referenceable area and the intra-block copy reference block containing the processing target block as invalid reference areas regardless of whether the encoding process is completed.

2. An image encoding method that encodes in units of intra-block copy reference blocks, characterized in that, it includes: a block vector candidate derivation step of deriving block vector candidates for a processing target block in a processing target picture from encoding information stored in an encoding information storage memory; a predicted block vector supplementation step of adding (0, 0) to the block vector candidates; a selection step of selecting a selected block vector from the block vector candidates; a reference position correction step of correcting the reference position of a reference block referenced by the selected block vector so that it refers to the inside of a referenceable area; and a prediction step of obtaining decoded pixels in the processing target picture from a decoded image memory as prediction values for the processing target block based on the reference position of the reference block, wherein in the reference position correction step, a fixed number of intra-block copy reference blocks that were encoded immediately before the intra-block copy reference block containing the processing target block are used as the referenceable area, and the intra-block copy reference blocks before the referenceable area and the intra-block copy reference block containing the processing target block are set as invalid reference areas regardless of whether the encoding process is completed.

3. An image decoding device that decodes in units of intra-block copy reference blocks, characterized in that, it includes: a block vector candidate derivation unit that derives block vector candidates for a processing target block in a processing target picture from encoding information stored in an encoding information storage memory; a predicted block vector supplementation unit that adds (0, 0) to the block vector candidates; a selection unit that selects a selected block vector from the block vector candidates; a reference position correction unit that corrects the reference position of a reference block referenced by the selected block vector so that it refers to the inside of a referenceable area; and a prediction unit that obtains decoded pixels in the processing target picture from a decoded image memory as prediction values for the processing target block based on the reference position of the reference block, The reference position correction unit uses a fixed number of in-frame block copy reference blocks that were decoded immediately before the in-frame block copy reference block containing the processing target block as the referenceable area, and sets the in-frame block copy reference blocks before the referenceable area and the in-frame block copy reference block containing the processing target block as invalid reference areas regardless of whether the decoding process is completed.

4. An image decoding method that decodes in units of in-frame block copy reference blocks, characterized in that it includes: a block vector candidate derivation step of deriving block vector candidates for a processing target block in a processing target picture from encoding information stored in an encoding information storage memory; a predicted block vector supplementation step of adding (0, 0) to the block vector candidates; a selection step of selecting a selected block vector from the block vector candidates; a reference position correction step of correcting the reference position of a reference block referenced by the selected block vector so that it is inside the referenceable area; and a prediction step of obtaining decoded pixels in the processing target picture from a decoded image memory based on the reference position of the reference block as a predicted value for the processing target block, In the reference position correction step, a fixed number of in-frame block copy reference blocks that were decoded immediately before the in-frame block copy reference block containing the processing target block are used as the referenceable area, and the in-frame block copy reference blocks before the referenceable area and the in-frame block copy reference block containing the processing target block are set as invalid reference areas regardless of whether the decoding process is completed.

5. A storage method of storing a bitstream generated by the image encoding method according to claim 2 in a recording medium.

6. A transmission method of transmitting a bitstream generated by the image encoding method according to claim 2.

Images