Image decoding apparatus and method, and image encoding apparatus and method

Abstract

The present invention relates to an image decoding apparatus and method, and an image encoding apparatus and method. A merge candidate list is generated, a merge candidate is selected from the merge candidate list as a selected merge candidate, a motion vector is decoded from an encoded stream and a correction vector is derived, the correction vector is added to a first predicted motion vector of the selected merge candidate without scaling the correction vector, the correction vector is subtracted from a second predicted motion vector of the selected merge candidate without scaling the correction vector, and a corrected merge candidate is derived.

Description

[0001] This application is a divisional application of the invention patent application with application number 201980050605.2, application date December 13, 2019, entitled "Image Decoding Apparatus, Image Decoding Method and Image Decoding Program". Technical Field

[0002] This invention relates to image decoding technology. Background Technology

[0003] Existing image coding technologies include HEVC (High Efficiency Video Coding) (H.265). In HEVC, a merging mode is used as an inter-frame prediction mode.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent document 1: Japanese Patent Application Publication No. 10-276439. Summary of the Invention

[0007] In HEVC, there are two inter-frame prediction modes: merge mode and differential motion vector mode. The inventors recognized that by correcting the motion vector of the merge mode, there is room to improve coding efficiency.

[0008] This invention was made in view of such circumstances, and its purpose is to correct the motion vector of the merging mode and provide a new inter-frame prediction mode with higher efficiency.

[0009] To address the aforementioned problems, an image decoding apparatus according to a certain aspect of the present invention includes: a merging candidate list generation unit that generates a merging candidate list, the merging candidate list including motion information of multiple blocks adjacent to the predicted object block as merging candidates; a merging candidate selection unit that selects merging candidates from the merging candidate list as selected merging candidates; a code string decoding unit that decodes a code string from an encoded stream and derives a correction vector; and a merging candidate correction unit that adds the correction vector to a first predicted motion vector of the selected merging candidate without scaling the correction vector, subtracts the correction vector from a second predicted motion vector of the selected merging candidate without scaling the correction vector, and derives a corrected merging candidate.

[0010] Furthermore, any combination of the above structural elements, or variations in the form of the invention in methods, apparatus, recording media, computer programs, etc., are also effective ways to implement the invention.

[0011] According to the present invention, a new inter-frame prediction mode with higher efficiency can be provided. Attached Figure Description

[0012] Figure 1 (a) is a diagram illustrating the structure of the image encoding apparatus 100 according to the first embodiment. Figure 1 (b) is a diagram illustrating the structure of the image decoding apparatus 200 according to the first embodiment;

[0013] Figure 2 This is a diagram illustrating an example of an input image being segmented into blocks based on block size;

[0014] Figure 3 This is an explanation Figure 1 (a) A diagram showing the structure of the inter-frame prediction unit of an image coding apparatus;

[0015] Figure 4 This is a flowchart illustrating the actions of the merging mode in the first embodiment;

[0016] Figure 5 This is an explanation Figure 1 (a) A diagram showing the structure of the merged candidate list generation unit of the image encoding apparatus;

[0017] Figure 6 This is a diagram illustrating the blocks adjacent to the block being processed;

[0018] Figure 7 It is a diagram illustrating blocks on the decoded image that are located in the same position as the block being processed and in its surrounding area;

[0019] Figure 8 This is a diagram illustrating a portion of the syntax of a block in the merging mode of the first embodiment;

[0020] Figure 9 This is a diagram illustrating the syntax of the differential motion vector in the first embodiment;

[0021] Figure 10 This is a diagram illustrating the syntax of the differential motion vector in a variant of the first embodiment;

[0022] Figure 11 This is a diagram illustrating a portion of the syntax of a block in a merging pattern of other variations of the first embodiment;

[0023] Figure 12 This is a diagram illustrating a portion of the syntax of a block in a merging pattern of yet another variation of the first embodiment;

[0024] Figure 13 This is a flowchart illustrating the operation of the merging mode in the second embodiment;

[0025] Figure 14 This is a diagram illustrating a portion of the syntax of a block in the merging mode of the second embodiment;

[0026] Figure 15 This is a diagram illustrating the syntax of the differential motion vector in the second embodiment;

[0027] Figure 16 This is a diagram illustrating the effect of variation 8 of the first embodiment;

[0028] Figure 17 This is a diagram illustrating the effect when the image spacing of Modification 8 of the first embodiment is not equal.

[0029] Figure 18 This is a diagram illustrating an example of the hardware structure of the encoding / decoding apparatus according to the first embodiment. Detailed Implementation

[0030] (First Implementation)

[0031] Hereinafter, with reference to the accompanying drawings, we will describe in detail the image encoding apparatus, image encoding method and image encoding program, as well as the image decoding apparatus, image decoding method and image decoding program involved in the first embodiment of the present invention.

[0032] Figure 1 (a) is a diagram illustrating the structure of the image encoding apparatus 100 according to the first embodiment. Figure 1 (b) is a diagram illustrating the structure of the image decoding apparatus 200 according to the first embodiment.

[0033] The image encoding apparatus 100 of this embodiment includes a block size determination unit 110, an inter-frame prediction unit 120, a transform unit 130, a code string generation unit 140, a local decoding unit 150, and a frame memory 160. The image encoding apparatus 100 accepts input images, performs intra-frame prediction and inter-frame prediction, and outputs an encoded stream. Hereinafter, "image" and "picture" will be used interchangeably.

[0034] The image decoding apparatus 200 includes a code stream decoding unit 210, an inter-frame prediction unit 220, an inverse transform unit 230, and a frame memory 240. The image decoding apparatus 200 receives input from the encoded stream output from the image encoding apparatus 100, performs intra-frame prediction and inter-frame prediction, and outputs a decoded image.

[0035] The image encoding device 100 and the image decoding device 200 are implemented by hardware such as information processing devices including CPU (Central Processing Unit), memory, etc.

[0036] First, the functions and operations of each part of the image coding apparatus 100 will be explained. Assuming that intra-frame prediction is performed in the same way as HEVC, inter-frame prediction will be explained below.

[0037] The block size determination unit 110 determines the block size for inter-frame prediction based on the input image, and provides the determined block size, block position, and input pixels (input values) that match the block size to the inter-frame prediction unit 120. Regarding the method for determining the block size, the RDO (rate-distortion optimization) method used in the reference software for HEVC is employed, etc.

[0038] Here, the block size is explained. Figure 2 This example illustrates how a portion of an image input to the image encoding device 100 is divided into blocks based on a block size determined by the block size determination unit 110. Block sizes include 4×4, 8×4, 4×8, 8×8, 16×8, 8×16, 32×32, ..., 128×64, 64×128, and 128×128. The input image is divided into blocks of one of these block sizes in a manner where each block is unique.

[0039] The inter-frame prediction unit 120 determines inter-frame prediction parameters for inter-frame prediction using information input from the block size determination unit 110 and a reference image input from the frame memory 160. Based on these parameters, the inter-frame prediction unit 120 performs inter-frame prediction and derives predicted values, providing the block size, block position, input values, inter-frame prediction parameters, and predicted values ​​to the transform unit 130. The method for determining the inter-frame prediction parameters includes the RDO (Rate-Distortion Optimization) method used in HEVC reference software. Details regarding the inter-frame prediction parameters and the operation of the inter-frame prediction unit 120 will be described later.

[0040] The transformation unit 130 calculates the difference value by subtracting the prediction value from the input value, performs orthogonal transformation and quantization on the calculated difference value to calculate the prediction error data, and provides the block size, block position, inter-frame prediction parameters and prediction error data to the code string generation unit 140 and the local decoding unit 150.

[0041] The code string generation unit 140 encodes the SPS (Sequence Parameter Set), PPS (Picture Parameter Set), and other information as needed. It also encodes the code string provided by the transformation unit 130 for determining the block size, encodes the inter-frame prediction parameters into a code string, encodes the prediction error data into a code string, and outputs the encoded stream. Details of the encoding of the inter-frame prediction parameters will be described later.

[0042] The local decoding unit 150 performs inverse orthogonal transformation and inverse quantization on the prediction error data to restore the difference value, performs addition operation on the difference value and the prediction value to generate a decoded image, and provides the decoded image and inter-frame prediction parameters to the frame memory 160.

[0043] The frame memory 160 stores the decoded images and inter-frame prediction parameters of multiple images, and provides the decoded images and inter-frame prediction parameters to the inter-frame prediction unit 120.

[0044] Next, the functions and operations of each part of the image decoding apparatus 200 will be explained. Assuming that intra-frame prediction is performed in the same way as HEVC, inter-frame prediction will be explained below.

[0045] The code stream decoding unit 210 decodes the SPS, PPS header and other information from the encoded stream as needed, decodes the block size, block position, inter-frame prediction parameters and prediction error data from the encoded stream, and provides the block size, block position, inter-frame prediction parameters and prediction error data to the inter-frame prediction unit 220.

[0046] The inter-frame prediction unit 220 uses the information input from the code string decoding unit 210 and the reference image input from the frame memory 240 to perform inter-frame prediction and derive the predicted value. The inter-frame prediction unit 220 provides the block size, block position, inter-frame prediction parameters, prediction error data, and predicted value to the inverse transform unit 230.

[0047] The inverse transform unit 230 performs inverse orthogonal transform and inverse quantization on the prediction error data provided by the inter-frame prediction unit 220 and calculates the difference value. It adds the difference value to the prediction value to generate a decoded image, provides the decoded image and inter-frame prediction parameters to the frame memory 240, and outputs the decoded image.

[0048] The frame memory 240 stores the decoded images and inter-frame prediction parameters of multiple images, and provides the decoded images and inter-frame prediction parameters to the inter-frame prediction unit 220.

[0049] Furthermore, the inter-frame prediction performed by the inter-frame prediction unit 120 and the inter-frame prediction unit 220 is the same operation, and the decoded images and inter-frame prediction parameters stored in the frame memory 160 and the frame memory 240 are also the same.

[0050] Next, the inter-frame prediction parameters are explained. These parameters include a merge flag, a merge index, a valid flag for predicted LX, a motion vector for predicted LX, a reference image index for predicted LX, a merge correction flag, and a differential motion vector for predicted LX. LX consists of L0 and L1. The merge flag indicates whether to use the merge mode or the differential motion vector mode as the inter-frame prediction mode. If the merge flag is 1, the merge mode is used; if it is 0, the differential motion vector mode is used. The merge index indicates the index of the selected merge candidate within the merge candidate list. The valid flag for predicted LX indicates whether the predicted LX is valid or invalid. If both L0 and L1 predictions are valid, it is a double prediction; if L0 prediction is valid and L1 prediction is invalid, it is an L0 prediction; if L1 prediction is valid and L0 prediction is invalid, it is an L1 prediction. The merge correction flag indicates whether to correct the motion information of the merge candidates. If the merge correction flag is 1, the merge candidates are corrected; if it is 0, the merge candidates are not corrected. Here, the code string generation unit 140 does not encode the valid flags of the predicted LX as a code string into the code stream. Furthermore, the code string decoding unit 210 does not decode the valid flags of the predicted LX from the code stream into a code string. The reference image index is used to determine the index of the decoded image in the frame memory 160. Additionally, the combination of the valid flags of the L0 prediction, the valid flags of the L1 prediction, the motion vectors of the L0 prediction, the motion vectors of the L1 prediction, the reference image index of the L0 prediction, and the reference image index of the L1 prediction is defined as motion information.

[0051] In addition, if the block is an intra-frame coded block, a block outside the image region, etc., then both the valid flags for L0 prediction and L1 prediction are set to invalid.

[0052] The following explanation uses B-images, which utilize all L0 predictions (one-way prediction), L1 predictions (one-way prediction), and bi-directional prediction, as the image type. However, the image type can also be P-images, which only utilize one-way prediction. In the case of P-images, inter-frame prediction parameters are processed only for L0 predictions, without L1 predictions. When coding efficiency is typically improved using B-images, the ratio of the L0 prediction reference image to the prediction target image is a past image, and the ratio of the L1 prediction reference image to the prediction target image is a future image. This is because, when the L0 and L1 prediction reference images are viewed from the prediction target image in opposite directions, coding efficiency is improved through interpolation prediction. Whether the L0 and L1 prediction reference images are viewed from the prediction target image in opposite directions can be determined by comparing the POC (Picture Order Count) of the reference images. The following explanation assumes that the L0 and L1 prediction reference images are in opposite directions in time relative to a certain prediction target image of the prediction target block.

[0053] Next, the details of the inter-frame prediction unit 120 will be explained. Unless otherwise specified, the inter-frame prediction unit 120 of the image encoding device 100 has the same structure and operation as the inter-frame prediction unit 220 of the image decoding device 200.

[0054] Figure 3 This diagram illustrates the structure of the inter-frame prediction unit 120. The inter-frame prediction unit 120 includes a merging mode determination unit 121, a merging candidate list generation unit 122, a merging candidate selection unit 123, a merging candidate correction determination unit 124, a merging candidate correction unit 125, a differential motion vector mode implementation unit 126, and a prediction value derivation unit 127.

[0055] As an inter-frame prediction mode, the inter-frame prediction unit 120 switches between the merging mode and the differential motion vector mode for each block. Assuming that the differential motion vector mode implemented by the differential motion vector mode implementation unit 126 is implemented in the same way as HEVC, the merging mode will be mainly described below.

[0056] The merging mode determination unit 121 determines whether to use the merging mode as the inter-frame prediction mode for each block. If the merging flag is 1, the merging mode is used; if the merging flag is 0, the differential motion vector mode is used.

[0057] In the inter-frame prediction unit 120, the determination of whether to set the merging flag to 1 uses the RDO (rate-distortion optimization) method, etc., of the reference software for HEVC. In the inter-frame prediction unit 220, the code stream decoding unit 210 obtains the merging flag decoded from the encoded stream based on the syntax. Details of the syntax will be described later.

[0058] When the merging flag is 0, the differential motion vector mode is implemented by the differential motion vector mode implementation unit 126, and the inter-frame prediction parameters of the differential motion vector mode are provided to the prediction value derivation unit 127.

[0059] When the merging flag is 1, the merging candidate list generation unit 122, the merging candidate selection unit 123, the merging candidate correction determination unit 124, and the merging candidate correction unit 125 implement the merging mode, and the inter-frame prediction parameters of the merging mode are provided to the prediction value derivation unit 127.

[0060] The following details the processing when the merge flag is 1.

[0061] Figure 4 This is a flowchart illustrating the actions of the merge mode. The following uses... Figure 3 and Figure 4 The details of the merger model are explained.

[0062] First, the merge candidate list generation unit 122 generates a merge candidate list based on the motion information of blocks adjacent to the processing target block and the motion information of blocks in the decoded image (S100), and provides the generated merge candidate list to the merge candidate selection unit 123. Hereinafter, the processing target block and the prediction target block are used with the same meaning.

[0063] Here, the generation of the candidate list for merging is explained. Figure 5 This is a diagram illustrating the structure of the candidate list generation unit 122. The candidate list generation unit 122 includes a spatial candidate generation unit 201, a temporal candidate generation unit 202, and a candidate supplementation unit 203.

[0064] Figure 6 This is a diagram illustrating the blocks adjacent to the block being processed. Here, the blocks adjacent to the block being processed are designated as block A, block B, block C, block D, block E, block F, and block G. However, it is not limited to using multiple blocks adjacent to the block being processed.

[0065] Figure 7 This diagram illustrates blocks located in the same position as and around the block being processed. Here, the blocks in the decoded image located in the same position as and around the block being processed are designated as blocks CO1, CO2, and CO3. However, multiple blocks in the decoded image located in the same position as and around the block being processed can be used; this is not a limitation. Hereafter, blocks CO1, CO2, and CO3 will be referred to as "same-position blocks," and the decoded image containing the same-position blocks will be referred to as "same-position image."

[0066] The following uses Figure 5 , Figure 6 and Figure 7 The details of generating the candidate list for merging are explained.

[0067] First, the spatial merging candidate generation unit 201 sequentially checks blocks A, B, C, D, E, F, and G. If either or both of the valid flags predicted by L0 and L1 are valid, the motion information of that block is added sequentially to the merging candidate list as a merging candidate. The merging candidates generated by the spatial merging candidate generation unit 201 are called spatial merging candidates.

[0068] Next, the time merging candidate generation unit 202 sequentially checks blocks C01, CO2, and CO3. First, it performs scaling and other processing on the motion information of blocks for which either or both of the valid flags predicted by L0 and L1 are valid, and then adds them sequentially to the merging candidate list as merging candidates. The merging candidates generated by the time merging candidate generation unit 202 are called time merging candidates.

[0069] Here, we explain the scaling of the temporal merge candidate. The scaling of the temporal merge candidate is the same as that of HEVC. The motion vector of the temporal merge candidate is based on the distance between the image at the same location and the reference image of the block at the same location. The motion vector of the block at the same location is scaled and exported according to the distance between a certain image of the predicted object block and the image referenced by the temporal merge candidate.

[0070] Furthermore, the reference image for the temporal merging candidate is the reference image with reference image index 0, which is the same as both the L0 and L1 predictions. Additionally, which same-location block is derived from the L0 and L1 predictions is determined by encoding (decoding) the same-location derivation flag. As described above, the temporal merging candidate scales the motion vector of either the L0 or L1 prediction for the same-location block in the L0 and L1 predictions, deriving new motion vectors for the L0 and L1 predictions, which are then set as the motion vectors of the temporal merging candidate's L0 and L1 predictions.

[0071] Next, if the candidate list contains multiple identical motion information, one motion information is kept and the others are removed.

[0072] Next, if the number of merge candidates included in the merge candidate list is less than the maximum number of merge candidates, before the number of merge candidates included in the merge candidate list reaches the maximum number of merge candidates, the merge candidate supplementation unit 203 adds supplementary merge candidates to the merge candidate list, setting the number of merge candidates included in the merge candidate list to the maximum number of merge candidates. Here, the supplementary merge candidates are motion information where the motion vectors of both the L0 and L1 predictions are (0,0) and the reference image indices of both the L0 and L1 predictions are 0.

[0073] Here, the maximum number of candidate merges is set to 6, but any number above 1 is acceptable.

[0074] Next, the merge candidate selection unit 123 selects a merge candidate from the merge candidate list (S101), and provides the selected merge candidate (called "selected merge candidate") and merge index to the merge candidate correction determination unit 124, setting the selected merge candidate as the motion information of the processing target block. In the inter-frame prediction unit 120 of the image coding apparatus 100, a merge candidate is selected from the merge candidates included in the merge candidate list using the RDO (rate-distortion optimization) method or the like for HEVC reference software, and the merge index is determined. In the inter-frame prediction unit 220 of the image decoding apparatus 200, the code stream decoding unit 210 obtains the merge index decoded from the encoded stream, and selects a merge candidate from the merge candidates included in the merge candidate list as the selected merge candidate based on the merge index.

[0075] Next, the merge candidate correction determination unit 124 checks whether the width of the processing object block is greater than or equal to a predetermined width, whether the height of the processing object block is greater than or equal to a predetermined height, and whether at least one of the L0 prediction and L1 prediction selected as the merge candidate is valid (S102). If the conditions that the width of the processing object block is greater than or equal to a predetermined width, whether the height of the processing object block is greater than or equal to a predetermined height, and whether at least one of the L0 prediction and L1 prediction selected as the merge candidate is valid are not met (S102 no), the merge candidate is not corrected and is treated as motion information of the processing object block, and the process proceeds to step S111. Here, since the merge candidate list must include at least one of the L0 prediction and L1 prediction as a valid merge candidate, it is self-evident that at least one of the L0 prediction and L1 prediction selected as the merge candidate is valid. Therefore, the "whether at least one of the L0 prediction and L1 prediction selected as the merge candidate is valid" in S102 is omitted, and S102 is set to check whether the width of the processing object block is greater than or equal to a predetermined width and whether the height of the processing object block is greater than or equal to a predetermined height.

[0076] If the width of the processing object block is greater than or equal to a predetermined width, and the height of the processing object block is greater than or equal to a predetermined height, and at least one of the L0 prediction and L1 prediction selected as merging candidates is valid (as in S102), the merging candidate correction determination unit 124 sets a merging correction flag (S103) and provides the merging correction flag to the merging candidate correction unit 125. In the inter-frame prediction unit 120 of the image encoding apparatus 100, if the prediction error when inter-frame prediction is performed in the merging candidates is greater than or equal to a predetermined prediction error, the merging correction flag is set to 1; if the prediction error when inter-frame prediction is performed in the selected merging candidates is not greater than or equal to a predetermined prediction error, the merging correction flag is set to 0. In the inter-frame prediction unit 220 of the image decoding apparatus 200, the code string decoding unit 210 obtains the decoded merging correction flag from the encoded stream based on the syntax.

[0077] Next, the candidate merging correction unit 125 checks whether the merging correction flag is 1 (S104). If the merging correction flag is not 1 (No in S104), the selected candidate merging is not corrected but is used as motion information of the processing target block, and the process proceeds to step S111.

[0078] If the merge correction flag is 1 (Yes in S104), then check if the L0 prediction of the selected merge candidate is valid (S105). If the L0 prediction of the selected merge candidate is not valid (No in S105), proceed to step S108. If the L0 prediction of the selected merge candidate is valid (Yes in S105), then determine the differential motion vector of the L0 prediction (S106). As described above, if the merge correction flag is 1, then the motion information of the selected merge candidate is corrected; if the merge correction flag is 0, then the motion information of the selected merge candidate is not corrected.

[0079] In the inter-frame prediction unit 120 of the image coding apparatus 100, the differential motion vector predicted by L0 is obtained by motion vector search. Here, the search range of motion vector is set to ±16 in both the horizontal and vertical directions, but it can also be a multiple of 2 such as ±64. In the inter-frame prediction unit 220 of the image decoding apparatus 200, the code string decoding unit 210 obtains the decoded differential motion vector predicted by L0 from the encoded stream based on the syntax.

[0080] Next, the candidate correction unit 125 calculates the L0 predicted correction motion vector and sets the L0 predicted correction motion vector as the L0 predicted motion vector of the block motion information of the object being processed (S107).

[0081] Here, the relationship between the corrected motion vector (mvL0) predicted by L0, the motion vector of L0 predicted from the selected merged candidates (mmvL0), and the differential motion vector (mvdL0) predicted by L0 is explained. The corrected motion vector (mvL0) predicted by L0 is obtained by adding the motion vector of L0 predicted from the selected merged candidates (mmvL0) to the differential motion vector (mvdL0) predicted by L0, as shown in the following formula. Furthermore, "0" represents the horizontal component of the motion vector, and "1" represents the vertical component of the motion vector.

[0082] mvL0[0]=mmvL0[0]+mvdL0[0]

[0083] mvL0[1]=mmvL0[1]+mvdL0[1]

[0084] Next, check if the selected candidate L1 predictions are valid (S108). If the selected candidate L1 predictions are not valid (No in S108), proceed to step S111. If the selected candidate L1 predictions are valid (Yes in S108), determine the differential motion vector of the L1 predictions (S109).

[0085] In the inter-frame prediction unit 120 of the image coding apparatus 100, the differential motion vector predicted by L1 is obtained through motion vector search. Here, the search range of the motion vector is set to ±16 in both the horizontal and vertical directions, but it can also be set to an exponent of 2 such as ±64. In the inter-frame prediction unit 220 of the image decoding apparatus 200, the code string decoding unit 210 obtains the decoded differential motion vector predicted by L1 from the coded stream based on the syntax.

[0086] Next, the candidate correction unit 125 calculates the L1 predicted correction motion vector and sets the L1 predicted correction motion vector as the L1 predicted motion vector for processing the motion information of the object block (S110).

[0087] Here, the relationship between the L1 predicted corrected motion vector (mvL1), the L1 predicted motion vector selected for merging (mmvL1), and the L1 predicted differential motion vector (mvdL1) is explained. The L1 predicted corrected motion vector (mvL1) is obtained by adding the L1 predicted motion vector selected for merging (mmvL1) to the L1 predicted differential motion vector (mvdL1), as shown in the following formula. Furthermore, "0" represents the horizontal component of the motion vector, and "1" represents the vertical component of the motion vector.

[0088] mvL1[0]=mmvL1[0]+mvdL1[0]

[0089] mvL1[1]=mmvL1[1]+mvdL1[1]

[0090] Next, the prediction value derivation unit 127 performs an inter-frame prediction of L0 prediction, L1 prediction, or dual prediction based on the motion information of the processing target block, and derives the prediction value (S111). As described above, if the merge correction flag is 1, the motion vector of the selected merge candidate is corrected; if the merge correction flag is 0, the motion vector of the selected merge candidate is not corrected.

[0091] The details of the encoding of inter-frame prediction parameters are explained. Figure 8 This is a graph showing a portion of the syntax of a block in merge mode. Table 1 shows the relationship between inter-frame prediction parameters and syntax. Figure 8 `cbWidth` is the width of the processed object block, and `cbHeight` is the height of the processed object block. Both the predetermined width and height are set to 8. By setting the predetermined width and height, no correction is made for merge candidates in small block units, thereby reducing processing overhead. Here, `cu_skip_flag` is 1 if the block is in skip mode, and 0 otherwise. The syntax for skip mode is the same as for merge mode. `merge_idx` is the merge index used to select a merge candidate from the merge candidate list.

[0092] Compared to merge_mod_flag, merge_idx is encoded (decoded) first to determine the merge index, and then the encoding (decoding) of merge_mod_flag is determined. merge_idx shares the merge index with the merge mode, thus suppressing the complexity of syntax and the increase of context, and improving encoding efficiency.

[0093] (Table 1)

[0094]

[0095]

[0096] Figure 9 This is a diagram illustrating the syntax of the differential motion vector. Figure 9 The `mvd_coding(N)` is the same syntax used in the differential motion vector model. Here, N is either 0 or 1. N = 0 indicates L0 prediction, and N = 1 indicates L1 prediction.

[0097] The syntax of the differential motion vector includes flags indicating whether the components of the differential motion vector are greater than 0, namely abs_mvd_greater0_flag[d]; flags indicating whether the components of the differential motion vector are greater than 1, namely abs_mvd_greater1_flag[d]; mvd_sign_flag[d] indicating the sign (±) of the components of the differential motion vector; and abs_mvd_minus2[d] indicating the absolute value of the vector obtained by subtracting 2 from the components of the differential motion vector. Here, d is either 0 or 1. d = 0 indicates the horizontal component, and d = 1 indicates the vertical component.

[0098] In HEVC, there are two inter-frame prediction modes: merge mode and differential motion vector mode. In merge mode, motion information can be recovered using a merge flag, making it a highly efficient coding mode. However, since the motion information in merge mode depends on the already processed blocks, the potential for increased prediction efficiency is limited, and further improvements in efficiency utilization are needed.

[0099] On the other hand, in the differential motion vector mode, L0 and L1 predictions are prepared separately as syntax. For each prediction type (L0 prediction, L1 prediction, or dual prediction) and for each L0 and L1 prediction, a motion vector flag, a differential motion vector, and a reference image index need to be predicted. Therefore, compared to the merging mode, the differential motion vector mode has poor coding efficiency. However, it is a stable and highly efficient mode for motions that cannot be derived in the merging mode, i.e., bursty motions with low correlation between spatially or temporally adjacent blocks.

[0100] In this embodiment, the motion vector of the merging mode can be corrected by maintaining the prediction type and reference image index of the fixed merging mode, thereby improving coding efficiency compared with the differential motion vector mode and improving usage efficiency compared with the merging mode.

[0101] In addition, by setting the differential motion vector to be the difference from the motion vector of the selected merging candidate, the size of the differential motion vector can be kept small, and coding efficiency can be suppressed.

[0102] Furthermore, by setting the syntax of the differential motion vector in the merge mode to be the same as that in the differential motion vector mode, the structural changes can be reduced even when adding differential motion vectors in the merge mode.

[0103] Furthermore, by defining a predetermined width and a predetermined height, if the conditions are not met—that the prediction block width is greater than the predetermined width, the prediction block height is greater than the predetermined height, and either or at least one of the selected L0 and L1 predictions for merging are valid—then the processing amount of the motion vector for the correction merging mode can be suppressed by omitting the processing of the motion vector for the correction merging mode. Moreover, when it is not necessary to suppress the processing amount of the correction motion vector, the correction of the motion vector does not need to be limited based on the predetermined width and predetermined height.

[0104] Hereinafter, variations of this embodiment will be described. Unless otherwise specified, the variations can be combined with each other.

[0105] (Variation Example 1)

[0106] In this embodiment, differential motion vectors are used as the syntax for blocks in the merging mode. In this variation, differential motion vectors are defined to encode (or decode) as differential unit motion vectors. A differential unit motion vector is the motion vector when the image interval is the minimum interval. In HEVC and similar protocols, the minimum image interval is encoded as a code string in the encoded stream.

[0107] The differential unit motion vector is scaled based on the interval between the encoded object image and the reference image of the merged pattern to serve as the differential motion vector. When the POC (Picture Order Count) of the encoded object image is set to POC(Cur), the POC of the reference image predicted by L0 of the merged pattern is set to POC(L0), and the POC of the reference image predicted by L1 of the merged pattern is set to POC(L1), the motion vector is calculated as follows: umvdL0 represents the differential unit motion vector predicted by L0, and umvdL1 represents the differential unit motion vector predicted by L1.

[0108] mvL0[0]=mmvL0[0]+umvdL0[0]*(POC(Cur)-POC(L0))

[0109] mvL0[1]=mmvL0[1]+umvdL0[1]*(POC(Cur)-POC(L0))

[0110] mvL1[0]=mmvL1[0]+umvdL1[0]*(POC(Cur)-POC(L1))

[0111] mvL1[1]=mmvL1[1]+umvdL1[1]*(POC(Cur)-POC(L1))

[0112] As shown above, in this variation, by using the differential unit motion vector as the differential motion vector, the coding amount of the differential motion vector is reduced, thereby improving coding efficiency. Furthermore, coding efficiency is particularly improved when the differential motion vector is large and the distance between the predicted object image and the reference image increases. Additionally, prediction and coding efficiency can also be improved when the interval between the processed object image and the reference image is proportional to the speed of the moving object within the frame.

[0113] In addition, unlike time-merging candidates, scaling and exporting based on the distance between images is not required. Since scaling can be performed using only multipliers in the decoding device, dividers are not needed, which reduces circuit size and processing load.

[0114] (Variation Example 2)

[0115] In this embodiment, the components of the differential motion vector can be encoded (or decoded) for 0; for example, it is assumed that only the L0 prediction can be changed. In this variant, the components of the differential motion vector cannot be encoded (or decoded) for 0.

[0116] Figure 10 This is a diagram illustrating the syntax of the differential motion vector in Variation Example 2. The syntax of the differential motion vector includes a flag indicating whether the component of the differential motion vector is greater than 1, namely abs_mvd_greater1_flag[d]; a flag indicating whether the component of the differential motion vector is greater than 2, namely abs_mvd_greater2_flag[d]; abs_mvd_minus3[d] indicating the absolute value of the vector obtained by subtracting 3 from the component of the differential motion vector; and mvd_sign_flag[d] indicating the sign (±) of the component of the differential motion vector.

[0117] As shown above, the components of the differential motion vector are set to not be encoded (or decoded) for 0, which can improve the encoding efficiency when the components of the differential motion vector are 1 or higher.

[0118] (Variation Example 3)

[0119] In this embodiment, the components of the differential motion vector are set to integers, while in Modification 2 they are set to integers other than 0. In this Modification, the components other than the ± sign of the differential motion vector are limited to exponents of 2.

[0120] Instead of the syntax of this implementation, namely abs_mvd_minus2[d], abs_mvd_pow_plus1[d] is used. The differential motion vector mvd[d] is calculated based on mvd_sign_flag[d] and abs_mvd_pow_plus1[d] as follows.

[0121] mvd[d]=mvd_sign_flag[d]*2^(abs_mvd_pow_plus1[d]+1)

[0122] Alternatively, instead of the syntax of Variation Example 2, namely abs_mvd_pow_plus2[d], abs_mvd_pow_plus2[d] is used. The differential motion vector mvd[d] is calculated based on mvd_sign_flag[d] and abs_mvd_pow_plus2[d] as follows.

[0123] mvd[d]=mvd_sign_flag[d]*2^(abs_mvd_pow_plus2[d]+2)

[0124] By limiting the components of the differential motion vector to an exponent of 2, prediction efficiency can be improved with larger motion vectors while significantly reducing the processing load of the encoding device.

[0125] (Variation Example 4)

[0126] In this embodiment, it is assumed that mvd_coding(N) contains differential motion vectors, but in this variant, it is assumed that mvd_coding(N) contains motion vector magnification.

[0127] In the syntax of mvd_coding(N) in this variant, abs_mvd_greater0_flag[d], abs_mvd_greater1_flag[d], and mvd_sign_flag[d] do not exist. Instead, they are set as a structure containing abs_mvr_plus2[d] and mvr_sign_flag[d].

[0128] The corrected motion vector (mvLN) predicted by LN is obtained by multiplying the motion vector (mmvLN) of the selected candidate LN predictions by the motion vector multiplier (mvrLN), and is calculated by the following formula.

[0129] mvLN[d]=mmvLN[d]*mvrLN[d]

[0130] By limiting the components of the differential motion vector to an exponent of 2, prediction efficiency can be improved with larger motion vectors while significantly reducing the processing load of the encoding device.

[0131] Furthermore, this modified example cannot be combined with modified example 1, modified example 2, modified example 3, and modified example 6.

[0132] (Variation Example 5)

[0133] In this embodiment Figure 8 In the syntax, when cu_skip_flag is 1 (it is skip mode), it is set to have the possibility of merge_mod_flag existing, but when it is skip mode, it can be set to not exist.

[0134] Thus, by omitting merge_mod_flag, the coding efficiency of skip mode can be improved and the determination of skip mode can be simplified.

[0135] (Variation Example 6)

[0136] In this embodiment, the validity of the LN predictions selected as merging candidates (N=0 or 1) is checked. If the LN predictions selected as merging candidates are not valid, the differential motion vector is not set to valid. However, it is also possible to not check the validity of the LN predictions selected as merging candidates and set the differential motion vector to valid regardless of whether the LN predictions selected as merging candidates are valid. In this case, if the LN predictions selected as merging candidates are invalid, the motion vector of the LN predictions selected as merging candidates is set to (0, 0), and the reference image index of the LN predictions selected as merging candidates is set to 0.

[0137] Thus, in this variant, by making the differential motion vector valid regardless of whether the LN predictions for selecting merging candidates are valid, the opportunity to utilize bidirectional prediction can be increased, thereby improving coding efficiency.

[0138] (Variation Example 7)

[0139] In this embodiment, the determination of whether the L0 prediction and L1 prediction selected as merging candidates are valid controls whether to encode (or decode) the differential motion vector. If both the L0 prediction and L1 prediction selected as merging candidates are valid, the differential motion vector is encoded (or decoded). If neither the L0 prediction nor the L1 prediction selected as merging candidates is valid, the differential motion vector can be set not to be encoded (or decoded). In this modified example, step S102 is as follows.

[0140] The merge candidate correction determination unit 124 checks whether the width of the processing object block is greater than or equal to a predetermined width, whether the height of the processing object block is greater than or equal to a predetermined height, and whether both the L0 prediction and L1 prediction selected as merge candidates are valid (S102).

[0141] Furthermore, steps S105 and S108 are not required in this variant.

[0142] Figure 11 This is a diagram showing a portion of the syntax of a block in the merging pattern of Variation Example 7. The syntax differs from that related to steps S102, S105, and S108.

[0143] Thus, in this variant, by making the differential motion vector valid when both the L0 and L1 predictions of the selected merge candidates are valid, and by correcting the motion vector of the selected merge candidates of the bidirectional predictions that are used more frequently, the prediction efficiency can be improved efficiently.

[0144] (Variation Example 8)

[0145] In this embodiment, as the syntax of the block in the merging mode, two differential motion vectors are used: the differential motion vector predicted by L0 and the differential motion vector predicted by L1. In this variation, only one differential motion vector is encoded (or decoded), and the L0-predicted correction motion vector and the L1-predicted correction motion vector share the same differential motion vector. The corrected merging candidate motion vector mvLN (N=0,1) is calculated based on the selected merging candidate motion vector mmvLN (N=0,1) and the differential motion vector mvd, as shown in the following formula.

[0146] If the L0 prediction selected for merging is valid, the motion vector of the L0 prediction is calculated according to the following formula.

[0147] mvL0[0]=mmvL0[0]+mvd[0]

[0148] mvL0[1]=mmvL0[1]+mvd[1]

[0149] If the L1 predictions selected for merging are valid, the motion vector of the L1 prediction is calculated according to the following formula: The difference motion vectors, which are opposite to and in the reverse direction of the L0 predictions, are added together. The difference motion vector can be subtracted from the motion vector of the L1 predictions selected for merging.

[0150] mvL1[0]=mmvL1[0]+mvd[0]*-1

[0151] mvL1[1]=mmvL1[1]+mvd[1]*-1

[0152] Figure 12 This is a diagram illustrating a portion of the syntax of a block in the merging mode of Variation Example 8. It differs from this embodiment in that the checks on the validity of L0 and L1 predictions are removed, and mvd_coding(1) is absent. mvd_coding(0) corresponds to a differential motion vector.

[0153] Thus, in this variant, by defining only one differential motion vector for L0 prediction and L1 prediction, the number of differential motion vectors is halved in the case of dual prediction, and they are shared under L0 prediction and L1 prediction, thereby suppressing the decrease in prediction efficiency and improving coding efficiency.

[0154] Furthermore, when the reference images to be referenced in the L0 prediction and the reference images to be referenced in the L1 prediction are in opposite directions (not in the same direction) relative to the predicted object image, the coding efficiency for motion in a fixed direction can be improved by adding the differential motion vectors in opposite directions.

[0155] The effects of this variation will be explained in detail. Figure 16 This is a diagram illustrating the effect of variation example 8. Figure 16 This is an image depicting a sphere (the area marked with diagonal lines) moving horizontally within a moving rectangular region (the region enclosed by dashed lines). In this case, the motion of the sphere relative to the image is the sum of the motion of the rectangular region and the motion of the sphere moving horizontally. Assume image B is the predicted object image, image A is the reference image for L0 prediction, and image C is the reference image for L1 prediction. Images A and C are reference images viewed from the predicted object image in opposite directions.

[0156] When a sphere moves at a fixed speed in a fixed direction, if images A, B, and C are equally spaced, the sphere's motion can be accurately reproduced by adding the amount of sphere movement that cannot be obtained from adjacent blocks to the L0 prediction and subtracting the movement of the sphere that cannot be obtained from adjacent blocks from the L1 prediction.

[0157] When the sphere moves in a fixed direction at a speed that is not constant, images A, B, and C are not equally spaced. However, if the amount of movement corresponding to the rectangular area of ​​the sphere is equally spaced, the movement of the sphere can be accurately reproduced by adding the amount of movement of the sphere that cannot be obtained from the adjacent block to the L0 prediction and subtracting the amount of movement of the sphere that cannot be obtained from the adjacent block from the L1 prediction.

[0158] In addition, when the sphere moves at a fixed speed in a fixed direction during a fixed period, images A, B, and C are not at equal intervals, but there are cases where the amount of movement corresponding to the rectangular area of ​​the sphere is at equal intervals. Figure 17 This diagram illustrates the effect of the non-equal spacing in variation example 8. Regarding this example, [the diagram uses...]. Figure 17 Please provide detailed information. Figure 17 Images F0, F1, ..., F8 represent images at fixed intervals. Assume the sphere is stationary from image F0 to F4, and moves at a fixed speed in a fixed direction after image F5. When images F0 and F6 are reference images, and image F5 is the prediction target image, images F0, F5, and F6 are not equally spaced, but the movement amounts corresponding to the rectangular regions of the sphere are equally spaced. When image F5 is the prediction target image, image F4, which is closer, is usually chosen as the reference image. However, image F0 is chosen instead of F4 as the reference image because it is a higher-quality image with less distortion compared to F4. Reference images are typically managed in a FIFO (First-In First-Out) manner within the reference image cache. This serves as a mechanism to retain high-quality images with less distortion as reference images in the cache for extended periods, resulting in long-term reference images. Long-term reference images are not managed as FIFOs within the reference image buffer. Instead, their status as long-term reference images is managed by reference image list control information encoded into the stripe header. Thus, this variation improves prediction and coding efficiency when either or both of L0 or L1 predictions are long-term reference images. Furthermore, this variation also improves prediction and coding efficiency when either or both of L0 or L1 predictions are intra-frame images.

[0159] Furthermore, similar to the temporal merging candidate, circuit size and power consumption can be reduced by not scaling the differential motion vectors based on the distance between images. For example, even when scaling the differential motion vectors, when selecting a merging candidate, both the scaling of the temporal merging candidate and the scaling of the differential motion vectors are necessary. In the scaling of the temporal merging candidate and the scaling of the differential motion vectors, since the motion vectors used as the scaling references are different, the scaling of both cannot be performed in a combined manner and must be performed separately.

[0160] Furthermore, as shown in this embodiment, when the merge candidate list includes temporal merge candidates, the temporal merge candidates are scaled. If the differential motion vector is small compared to the motion vector of the temporal merge candidates, coding efficiency can be improved even without scaling the differential motion vector. Additionally, when the differential motion vector is large, selecting a differential motion vector mode can suppress the decrease in coding efficiency.

[0161] (Variation Example 9)

[0162] In this embodiment, the maximum number of merge candidates is set to be the same for both the merge correction flag being 0 and 1. In this variation, the maximum number of merge candidates when the merge correction flag is 1 is less than the maximum number of merge candidates when the merge correction flag is 0. For example, the maximum number of merge candidates when the merge correction flag is 1 is set to 2. Here, the maximum number of merge candidates when the merge correction flag is 1 is set as the maximum corrected merge candidate number. Furthermore, when the merge index is less than the maximum corrected merge candidate number, the merge correction flag is encoded (decoded); when the merge index is greater than or equal to the maximum corrected merge candidate number, the merge correction flag is not encoded (decoded). Here, the maximum number of merge candidates when the merge correction flag is 0 and the maximum corrected merge candidate number can be predetermined values ​​or obtained by encoding (decoding) the SPS and PPS in the encoded stream.

[0163] Thus, in this modified example, by making the maximum number of merge candidates when the merge correction flag is 1 less than the maximum number of merge candidates when the merge correction flag is 0, the determination of whether to correct the merge candidate is only made for the merge candidate with the higher selection probability. This reduces the processing of the encoding device and suppresses the decrease in encoding efficiency. In addition, when the merge index is greater than or equal to the maximum number of corrected merge candidates, the encoding efficiency is improved because there is no need to encode (decode) the merge correction flag.

[0164] (Second Implementation)

[0165] The structure of the image encoding device 100 and image decoding device 200 in the second embodiment is the same as that of the image encoding device 100 and image decoding device 200 in the first embodiment. This embodiment differs from the first embodiment in the operation and syntax of the merging mode. The differences between this embodiment and the first embodiment will be explained below.

[0166] Figure 13 This is a flowchart illustrating the operation of the merging mode in the second embodiment. Figure 14 This is a diagram illustrating a portion of the syntax of a block in the merging mode of the second embodiment. Figure 15 This is a diagram illustrating the syntax of the differential motion vector in the second embodiment.

[0167] The following uses Figure 13 , Figure 14 , Figure 15 The differences from the first embodiment will be explained. Figure 13 and Figure 4 The steps from S205 to S207 and from S209 to S211 are different.

[0168] If the merge correction flag is 1 (Yes in S104), then check if the L0 prediction selected as a merge candidate is invalid (S205). If the L0 prediction selected as a merge candidate is not invalid (No in S205), proceed to step S208. If the L0 prediction selected as a merge candidate is invalid (Yes in S205), then determine the corrected motion vector of the L0 prediction (S206).

[0169] In the inter-frame prediction unit 120 of the image coding apparatus 100, the corrected motion vector predicted by L0 is obtained through motion vector search. Here, the search range of the motion vector is set to ±1 in both the horizontal and vertical directions. In the inter-frame prediction unit 220 of the image decoding apparatus 200, the corrected motion vector predicted by L0 is obtained from the coded stream.

[0170] Next, the reference image index for L0 prediction is determined (S207). Here, the reference image index for L0 prediction is set to 0.

[0171] Next, check if the strip type is B and if the selected merge candidate L1 prediction is invalid (S208). If the strip type is not B or the selected merge candidate L1 prediction is not invalid (S208, No), proceed to S111. If the strip type is B and the selected merge candidate L1 prediction is invalid (S208, Yes), determine the corrected motion vector of the L1 prediction (S209).

[0172] In the inter-frame prediction unit 120 of the image coding apparatus 100, the L1-predicted corrected motion vector is obtained through motion vector search. Here, the search range for the motion vector is set to ±1 in both the horizontal and vertical directions. In the inter-frame prediction unit 220 of the image decoding apparatus 200, the L1-predicted corrected motion vector is obtained from the coded stream.

[0173] Next, the reference image index for L1 prediction is determined (S110). Here, the reference image index for L1 prediction is set to 0.

[0174] As described above, in this embodiment, when the slice type is a slice type that allows dual prediction (i.e., slice type B), the merging candidates for L0 or L1 predictions are transformed into merging candidates for dual predictions. By setting them as merging candidates for dual predictions, an improvement in prediction efficiency can be expected through the filtering effect. Furthermore, by setting the adjacent decoded image as the reference image, the search range of motion vectors can be minimized.

[0175] (Modified Example)

[0176] In this embodiment, the reference image index in steps S207 and S210 is set to 0. In this variation, if the L0 prediction of the selected merging candidate is invalid, the reference image index of the L0 prediction is set to the reference image index of the L1 prediction; if the L1 prediction of the selected merging candidate is invalid, the reference image index of the L1 prediction is set to the reference image index of the L0 prediction.

[0177] Thus, by filtering the motion vectors based on the selection and merging of candidate L0 or L1 predictions with slight offsets, minute motions can be reproduced, thereby improving prediction efficiency.

[0178] In all the embodiments described above, the encoded bitstream output by the image encoding device has a specific data format, enabling decoding according to the encoding method used in the embodiment. The encoded bitstream can be provided by recording on a recording medium readable by a computer, such as an HDD, SSD, flash memory, or optical disc, or it can be provided from a server via a wired or wireless network. Therefore, the image decoding device corresponding to this image encoding device can decode the encoded bitstream of this specific data format regardless of the providing unit.

[0179] To exchange encoded bitstreams between an image encoding device and an image decoding device, the encoded bitstreams can be transmitted in data format for communication path transmission when using a wired or wireless network. In this case, a transmitting device is provided that converts the encoded bitstream output by the image encoding device into encoded data in data format for communication path transmission and transmits it over the network, and a receiving device receives encoded data from the network, restores it to an encoded bitstream, and provides it to the image decoding device. The transmitting device includes a memory for buffering the encoded bitstream output by the image encoding device, a packet processing unit for packetizing the encoded bitstream, and a transmitting unit for transmitting the packetized encoded data via the network. The receiving device includes a receiving unit for packetized encoded data via the network, a memory for buffering the received encoded data, and a packet processing unit for packetizing the encoded data to generate an encoded bitstream and providing it to the image decoding device.

[0180] To exchange encoded bitstreams between an image encoding device and an image decoding device, in the case of using a wired or wireless network, a relay device may be provided in addition to the transmitting and receiving devices. The relay device receives encoded data transmitted by the transmitting device and provides it to the receiving device. The relay device includes a receiving unit that receives packetized encoded data transmitted by the transmitting device, a memory that buffers the received encoded data, and a transmitting unit that transmits the packetized encoded data to the network. Furthermore, the relay device may include a receiving packet processing unit that performs packet processing on the packetized encoded data to generate an encoded bitstream, a recording medium that stores the encoded bitstream, and a transmitting packet processing unit that performs packetization on the encoded bitstream.

[0181] Alternatively, by adding a display unit to the structure to display the image decoded by the image decoding device, it can also be configured as a display device. Alternatively, by adding a capturing unit to the structure and inputting the captured image to the image encoding device, it can be configured as a capturing device.

[0182] Figure 18 An example of the hardware structure of the encoding / decoding apparatus of this application is shown. The encoding / decoding apparatus includes the structure of the image encoding apparatus and the image decoding apparatus according to embodiments of the present invention. The encoding / decoding apparatus 9000 includes a CPU 9001, an encoding / decoding IC 9002, an I / O interface 9003, a memory 9004, an optical disc drive 9005, a network interface 9006, and a video interface 9009, all of which are connected via a bus 9010.

[0183] Image encoding unit 9007 and image decoding unit 9008 are typically mounted as codec IC 9002. Image encoding processing in the image encoding apparatus according to embodiments of the present invention is performed by image encoding unit 9007, and image decoding processing in the image decoding apparatus according to embodiments of the present invention is performed by image encoding unit 9007. I / O interface 9003 is implemented, for example, via USB interface, and is connected to external keyboard 9104, mouse 9105, etc. CPU 9001 controls codec device 9000 based on user operations input via I / O interface 9003 to realize the user's desired action. User operations performed by keyboard 9104, mouse 9105, etc., include selecting which function to perform encoding or decoding, setting encoding quality, input / output destination of the encoded stream, input / output destination of the image, etc.

[0184] When a user wishes to reproduce an image recorded on the disc recording medium 9100, the optical disc drive 9005 reads the encoded bitstream from the inserted disc recording medium 9100 and sends the read encoded stream to the image decoding unit 9008 of the codec IC 9002 via the bus 9010. The image decoding unit 9008 performs image decoding processing in the image decoding apparatus according to the embodiments of the present invention on the input encoded bitstream and sends the decoded image to an external monitor 9103 via the video interface 9009. Furthermore, the codec apparatus 9000 has a network interface 9006 and can be connected to an external distribution server 9106 or a portable terminal 9107 via a network 9101. When a user wishes to reproduce an image recorded on the distribution server 9106 or the portable terminal 9107 instead of the image recorded on the disc recording medium 9100, the network interface 9006 obtains the encoded stream from the network 9101 instead of reading the encoded bitstream from the input disc recording medium 9100. Furthermore, when a user wishes to reproduce an image recorded in memory 9004, the image decoding process in the image decoding apparatus according to the embodiments of the present invention is performed on the encoded stream recorded in memory 9004.

[0185] When a user wishes to encode and record an image captured by an external camera 9102 into the memory 9004, the video interface 9009 inputs the image from the camera 9102 and sends it to the image encoding unit 9007 of the codec IC 9002 via the bus 9010. The image encoding unit 9007 performs image encoding processing in the image encoding apparatus according to the embodiments of the present invention on the image input via the video interface 9009 and generates an encoded bitstream. Then, the encoded bitstream is sent to the memory 9004 via the bus 9010. When a user wishes to record the encoded stream on the disc recording medium 9100 instead of the memory 9004, the optical disc drive 9005 writes the encoded stream to the inserted disc recording medium 9100.

[0186] It is also possible to implement a hardware structure that has an image encoding device but no image decoding device, or a hardware structure that has an image decoding device but no image encoding device. Such a hardware structure is, for example, the codec IC9002 is implemented by replacing it with an image encoding unit 9007 or an image decoding unit 9008, respectively.

[0187] The encoding and decoding processes described above can certainly be implemented using hardware such as ASICs for transmission, storage, and reception. However, they can also be implemented using firmware stored in ROM (Read-Only Memory) or flash memory, or computer software such as CPUs or SoCs (System on a Chip). The firmware or software program can also be provided by recording it on a computer-readable medium, by providing it from a server via wired or wireless networks, or by broadcasting it as data via terrestrial radio waves or satellite digital broadcasting.

[0188] The present invention has been described above according to embodiments. These embodiments are illustrative, and various modifications are possible in the combination of these structural elements or processing steps. Furthermore, these modifications are also within the scope of the present invention, as will be understood by those skilled in the art.

[0189] Industrial applicability

[0190] This invention can be applied to image decoding technology.

[0191] Symbol Explanation

[0192] 100 Image encoding device, 110 Block size determination unit, 120 Inter-frame prediction unit, 121 Merging mode determination unit, 122 Merging candidate list generation unit, 123 Merging candidate selection unit, 124 Merging candidate correction determination unit, 125 Merging candidate correction unit, 126 Differential motion vector mode implementation unit, 127 Prediction value derivation unit, 130 Transform unit, 140 Code string generation unit, 150 Local decoding unit, 160 Frame memory, 200 Image decoding device, 201 Spatial merging candidate generation unit, 202 Temporal merging candidate generation unit, 203 Merging candidate supplementation unit, 210 Code string decoding unit, 220 Inter-frame prediction unit, 230 Inverse transform unit, 240 Frame memory.

Claims

1. An image decoding device, characterized in that, include: A merge candidate list generation unit generates a merge candidate list, which includes motion information of multiple blocks adjacent to the predicted object block, and motion information of motion vectors derived by scaling the motion vectors of blocks on the decoded image at the same position as the predicted object block as merge candidates. The merge candidate selection unit selects a merge candidate from the merge candidate list as the selected merge candidate; The code string decoding unit decodes the code string from the encoded stream and derives the differential motion vector and the merge correction flag indicating whether to correct the merge candidate; as well as In the candidate merging correction unit, when the merging correction flag indicates that a merging candidate is to be corrected, if either or both of the first prediction reference image or the second prediction reference image of the selected merging candidate are long-term reference images, and the first prediction reference image and the second prediction reference image of the selected merging candidate are in opposite directions relative to the decoded object image containing the predicted object block, the unit directly adds the differential motion vector to the motion vector of the first prediction of the selected merging candidate without scaling the differential motion vector, and directly subtracts the differential motion vector from the motion vector of the second prediction of the selected merging candidate without scaling the differential motion vector, thereby deriving the corrected merging candidate for dual prediction. Compared to when the merge correction flag does not indicate a corrected merge candidate, the merge correction flag indicates that the number of selectable merge candidates is small when correcting merge candidates.

2. An image decoding method, characterized in that, include: The merge candidate list generation step generates a merge candidate list, which includes motion information of multiple blocks adjacent to the predicted object block, and motion information of motion vectors derived by scaling the motion vectors of blocks located in the same position as the predicted object block as merge candidates. The merge candidate selection step involves selecting a merge candidate from the merge candidate list as the selected merge candidate. The code string decoding step involves decoding the code string from the encoded stream and deriving the differential motion vector and the merge correction flag indicating whether to correct the merge candidate. as well as In the candidate merging correction step, when the merging correction flag indicates that a merging candidate is to be corrected, if either or both of the first prediction reference image or the second prediction reference image of the selected merging candidate are long-term reference images, and the first prediction reference image and the second prediction reference image of the selected merging candidate are in opposite directions relative to the decoded object image containing the predicted object block, the differential motion vector is directly added to the motion vector of the first prediction of the selected merging candidate without scaling the differential motion vector, and the differential motion vector is directly subtracted from the motion vector of the second prediction of the selected merging candidate without scaling the differential motion vector, thus deriving the corrected merging candidate for dual prediction. Compared to when the merge correction flag does not indicate a corrected merge candidate, the merge correction flag indicates that the number of selectable merge candidates is small when correcting merge candidates.

3. An image encoding device, characterized in that, include: A merge candidate list generation unit generates a merge candidate list, which includes motion information of multiple blocks adjacent to the predicted object block, and motion information of motion vectors derived by scaling the motion vectors of blocks located in the encoded image at the same position as the predicted object block as merge candidates. The merge candidate selection unit selects a merge candidate from the merge candidate list as the selected merge candidate; The merge correction determination unit sets a merge correction flag to indicate whether to correct merge candidates; In the candidate merging correction unit, when the merging correction flag indicates that a merging candidate is to be corrected, if either or both of the first prediction reference image or the second prediction reference image of the selected merging candidate are long-term reference images, and the first prediction reference image and the second prediction reference image of the selected merging candidate are in opposite directions relative to the encoded object image containing the predicted object block, the unit directly adds the differential motion vector to the motion vector of the first prediction of the selected merging candidate without scaling the differential motion vector, and directly subtracts the differential motion vector from the motion vector of the second prediction of the selected merging candidate without scaling the differential motion vector, and derives the corrected merging candidate for dual prediction. as well as The code string encoding unit encodes the merging correction flag and the differential motion vector into a code stream. Compared to when the merge correction flag does not indicate a corrected merge candidate, the merge correction flag indicates that the number of selectable merge candidates is small when correcting merge candidates.

4. An image encoding method, characterized in that, include: The merge candidate list generation step generates a merge candidate list, which includes motion information of multiple blocks adjacent to the predicted object block, and motion information of motion vectors derived by scaling the motion vectors of blocks on the encoded image at the same position as the predicted object block as merge candidates. The merge candidate selection step involves selecting a merge candidate from the merge candidate list as the selected merge candidate. The merge correction determination step sets a merge correction flag to indicate whether to correct merge candidates; In the candidate merging correction step, when the merging correction flag indicates that a merging candidate is to be corrected, if either or both of the first prediction reference image or the second prediction reference image of the selected merging candidate are long-term reference images, and the first prediction reference image and the second prediction reference image of the selected merging candidate are in opposite directions relative to the encoded object image containing the predicted object block, the differential motion vector is directly added to the motion vector of the first prediction of the selected merging candidate without scaling the differential motion vector, and the differential motion vector is directly subtracted from the motion vector of the second prediction of the selected merging candidate without scaling the differential motion vector, and the corrected merging candidate for dual prediction is derived. as well as The code string encoding step encodes the merging correction flag and the differential motion vector into a code stream. Compared to when the merge correction flag does not indicate a corrected merge candidate, the merge correction flag indicates that the number of selectable merge candidates is small when correcting merge candidates.

5. A method for storing a bit stream, characterized in that, The image encoding method of claim 4 is used to generate a bitstream; and the bitstream is stored.

6. A method for transmitting a bit stream, characterized in that, The image encoding method of claim 4 is used to generate a bit stream; and the bit stream is transmitted.

Images