Inter prediction device, image encoding device, image decoding device, and program

Abstract

To improve a coding efficiency in the case where an inter-prediction is performed.SOLUTION: An inter-prediction device that performs an inter-prediction of a coding object block obtained by dividing an original image, comprises: an adjacent vector acquisition part that acquires one or more first motion vectors adopted to an adjacent decoded block group that is adjacent to the coding object block; a motion vector correction part that generates a correction motion vector by correcting a second motion vector adopted to the coding object block by using the first motion vector; and a prediction-image generation part that generates a prediction image of the coding object block by performing the inter-prediction by using the correction motion vector.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an inter prediction device, an image encoding device, an image decoding device, and a program. [Background technology]

[0002] In conventional video coding methods such as HEVC described in Non-Patent Document 1, an image coding device is configured to divide an original image into blocks, perform prediction by switching between inter-prediction and intra-prediction for each block, and output a stream by performing orthogonal transformation, quantization, and entropy coding on the prediction residual, which represents the error of the predicted image obtained by the prediction.

[0003] In inter-prediction in such a video coding method, if a motion vector applied to a block (Coding Unit: CU) to be coded is different from a motion vector applied to an adjacent CU adjacent to the block to be coded, the predicted image of the CU to be coded and the predicted image of the adjacent CU will have significantly different characteristics.

[0004] In the example of the conventional video coding method shown in FIG. 1, object #1 moves to the upper left, and object #2 moves to the upper right. In inter prediction for adjacent CU #1 located to the left of the current CU containing only object #1, a motion vector (MV #3) similar to the motion of object #1 is likely to be applied. The current CU is located on an object boundary containing both object #1 and object #2, but since object #1 occupies a large area, a motion vector (MV #1) similar to the motion vector (MV #2) of adjacent CU #1 adjacent to the left is likely to be applied to the current CU.

[0005] When the motion vectors of adjacent CUs are similar, the continuity of the motion of the boundary area of ​​the CU is high, and the prediction accuracy is likely to be high. On the other hand, since CU#2 adjacent to the upper side of the current CU in FIG. 1 contains a large amount of object#2, it is likely that a motion vector (MV#3) similar to the motion of object#2 is applied. In other words, it is likely that the motion vector (MV#1) applied to the current CU and the motion vector (MV#3) applied to the adjacent CU#1 are significantly different. When the motion vectors of such adjacent CUs are significantly different, it is highly likely that at least one of these CUs contains an object boundary. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Recommendation ITU-T H.265,(12 / 2016), "High efficiency video coding", International Telecommunication Union Summary of the Invention [Problem to be solved by the invention]

[0007] The discontinuity of motion vectors at object boundaries leads to a decrease in prediction accuracy, which leads to a decrease in coding efficiency. In order to solve this problem, HEVC introduces a method in which a loop filter controls whether or not to apply a deblocking filter that removes block distortion, depending on whether or not the difference between a motion vector applied to a CU to be coded and a motion vector applied to an adjacent block is greater than a predetermined value.

[0008] However, in prediction (inter prediction), the same process is applied regardless of the discontinuity of the motion vectors described above, so the amount of information required to transmit the orthogonal transform coefficients increases in blocks near object boundaries, resulting in a problem of reduced coding efficiency.

[0009] Therefore, an object of the present invention is to provide an inter prediction device, an image encoding device, an image decoding device, and a program capable of improving the encoding efficiency when performing inter prediction. [Means for solving the problem]

[0010] An inter prediction device according to a first aspect performs inter prediction on a current block to be coded obtained by dividing an original image. The inter prediction device includes an adjacent vector acquisition unit that acquires one or more first motion vectors applied to a group of adjacent decoded blocks adjacent to the current block to be coded, a motion vector correction unit that generates a corrected motion vector by correcting a second motion vector applied to the current block to be coded using the first motion vector, and a predicted image generation unit that generates a predicted image of the current block to be coded by performing the inter prediction using the corrected motion vector.

[0011] An image encoding device according to a second aspect includes the inter prediction device according to the first aspect.

[0012] An image decoding device according to a third aspect includes the inter prediction device according to the first aspect.

[0013] A program according to a fourth aspect causes a computer to function as the inter prediction device according to the first aspect. Effect of the Invention

[0014] According to the present invention, it is possible to provide an inter prediction device, an image encoding device, an image decoding device, and a program capable of improving the encoding efficiency when performing inter prediction. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram for explaining the background art. [Diagram 2]FIG. 1 is a diagram showing a configuration of an image encoding device according to first and second embodiments. [Diagram 3] 1 is a diagram showing a configuration of an inter prediction unit (inter prediction device) according to the first embodiment. [Figure 4] 6 is a diagram illustrating an example of the operation of an adjacent vector acquisition unit according to the first and second embodiments. FIG. [Diagram 5] 5 is a diagram illustrating an example of the operation of a motion vector correction unit according to the first embodiment. FIG. [Figure 6] FIG. 1 is a diagram showing a configuration of an image decoding device according to first and second embodiments. [Figure 7] FIG. 4 is a diagram showing the operation of an inter prediction unit according to the first embodiment. [Figure 8] FIG. 13 is a diagram showing a modification of the inter prediction unit according to the first embodiment. [Figure 9] FIG. 11 is a diagram showing a configuration of an inter prediction unit (inter prediction device) according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a small region according to the second embodiment. [Figure 11] 13 is a diagram illustrating an example of the operation of a motion vector correction unit according to the second embodiment. FIG. [Figure 12] FIG. 11 is a diagram showing the operation of an inter prediction unit according to the second embodiment. [Figure 13] FIG. 11 is a diagram illustrating a modification of the motion vector correction unit according to the second embodiment. [Figure 14] FIG. 11 is a diagram showing the operation of a modified example of a motion vector correction unit according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] An image encoding device and an image decoding device according to an embodiment will be described with reference to the drawings. The image encoding device and the image decoding device according to the embodiment respectively encode and decode moving images as represented by MPEG. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0017] <1. First embodiment> <Image encoding device> First, an image coding device according to this embodiment will be described. Fig. 2 is a diagram showing the configuration of an image coding device 1 according to this embodiment.

[0018] As shown in FIG. 2, the image encoding device 1 includes a block division unit 100, a subtraction unit 110, a transformation and quantization unit 120, an entropy encoding unit 130, an inverse quantization and inverse transform unit 140, a synthesis unit 150, a memory 160, and a prediction unit 170.

[0019] The block division unit 100 divides an input image of a frame (or picture) constituting a moving image into a plurality of blocks, and outputs the blocks obtained by division to the subtraction unit 110. The size of the block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the block is not limited to a square, and may be a rectangle. A block is a unit for encoding by the image encoding device 1 and a unit for decoding by the image decoding device. Hereinafter, such a block is called a CU (Coding Unit). Moreover, a CU processed by the image encoding device 1 is called a CU to be encoded, and a CU processed by the image decoding device is called a CU to be decoded.

[0020] The subtraction unit 110 calculates a prediction residual that represents a difference (error) between the current CU to be coded input from the block division unit 100 and a predicted image obtained by predicting the current CU to be coded by the prediction unit 170. Specifically, the subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the predicted image from each pixel value of the CU, and outputs the calculated prediction residual to the transformation and quantization unit 120.

[0021] The transform / quantization unit 120 performs orthogonal transform processing and quantization processing for each CU. The transform / quantization unit 120 includes a transform unit 121 and a quantization unit 122.

[0022] The transform unit 121 performs an orthogonal transform process on the prediction residual input from the subtraction unit 110 to calculate an orthogonal transform coefficient, and outputs the calculated orthogonal transform coefficient to the quantization unit 122. The orthogonal transform refers to, for example, a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), or the like.

[0023] The quantization unit 122 quantizes the orthogonal transform coefficients input from the transform unit 121 using a quantization parameter (Qp) and a quantization matrix, and outputs the quantized orthogonal transform coefficients to the entropy coding unit 130 and the inverse quantization / inverse transform unit 140. Note that the quantization parameter (Qp) is a parameter commonly applied to each orthogonal transform coefficient in a CU, and is a parameter that determines the coarseness of quantization. The quantization matrix is ​​a matrix having quantization values ​​for quantizing each orthogonal transform coefficient as elements.

[0024] The entropy coding unit 130 performs entropy coding on the orthogonal transform coefficients input from the quantization unit 122, performs data compression to generate a coded stream (bit stream), and outputs the coded stream to the outside of the image coding device 1. For the entropy coding, Huffman codes, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. can be used. Note that the entropy coding unit 130 receives control information related to prediction from the prediction unit 170, and also performs entropy coding on the input control information.

[0025] The inverse quantization and inverse transform unit 140 performs inverse quantization processing and inverse orthogonal transform processing on a CU basis. The inverse quantization and inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142.

[0026] The inverse quantization unit 141 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the orthogonal transform coefficients by inverse quantizing the orthogonal transform coefficients input from the quantization unit 122 using a quantization parameter (Qp) and a quantization matrix, and outputs the restored orthogonal transform coefficients to the inverse transform unit 142.

[0027] The inverse transform unit 142 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121. For example, when the transform unit 121 performs discrete cosine transform, the inverse transform unit 142 performs inverse discrete cosine transform. The inverse transform unit 142 performs inverse orthogonal transform processing on the orthogonal transform coefficients input from the inverse quantization unit 141 to restore prediction residuals, and outputs restored prediction residuals, which are the restored prediction residuals, to the synthesis unit 150.

[0028] The synthesis unit 150 synthesizes, in pixel units, the reconstructed prediction residual input from the inverse transform unit 142 and the predicted image input from the prediction unit 170. The synthesis unit 150 adds each pixel value of the reconstructed prediction residual to each pixel value of the predicted image to reconstruct (decode) the CU to be coded, and outputs the decoded image in units of CU to the memory 160. Such a decoded image may be referred to as a reconstructed image.

[0029] The memory 160 stores the decoded image input from the synthesis unit 150. The memory 160 stores the decoded image on a frame-by-frame basis. The memory 160 outputs the stored decoded image to the prediction unit 170. Furthermore, the memory 160 stores the motion vector calculated by the inter prediction unit 172 for each CU. Note that the memory 160 may be configured with a plurality of memories. Also, a loop filter may be provided between the synthesis unit 150 and the memory 160.

[0030] The prediction unit 170 performs prediction on a CU basis. The prediction unit 170 includes an intra prediction unit 171, an inter prediction unit 172, and a switching unit 173.

[0031] The intra prediction unit 171 generates an intra prediction image by referring to decoded pixel values ​​around the current CU to be coded among the decoded images stored in the memory 160, and outputs the generated intra prediction image to the switching unit 173. The intra prediction unit 171 selects an optimal intra prediction mode to be applied to the current CU to be coded from among multiple intra prediction modes, and performs intra prediction using the selected intra prediction mode. The intra prediction unit 171 outputs control information related to the selected intra prediction mode to the entropy coding unit 130.

[0032] The inter prediction unit 172 uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector by a method such as block matching, predicts the current CU to be coded, generates an inter prediction image, and outputs the generated inter prediction image to the switching unit 173. In addition, the inter prediction unit 172 outputs the motion vector calculated for each CU to the memory 160, and causes the memory 160 to store the motion vector.

[0033] The inter prediction unit 172 selects an optimal inter prediction method from among inter prediction using multiple reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 172 outputs control information (such as a motion vector) related to the inter prediction to the entropy coding unit 130.

[0034] The switching unit 173 switches between the intra-predicted image input from the intra-prediction unit 171 and the inter-predicted image input from the inter-prediction unit 172, and outputs either of the predicted images to the subtraction unit 110 and the synthesis unit 150.

[0035] Next, the inter prediction unit 172 of the image encoding device 1 according to this embodiment will be described. Fig. 3 is a diagram showing the configuration of the inter prediction unit 172 of the image encoding device 1 according to this embodiment.

[0036] As shown in FIG. 3, the inter prediction unit 172 according to this embodiment has an adjacent vector acquisition unit 172a, a motion vector calculation unit 172c, a motion vector correction unit 172d, and a predicted image generation unit 172e.

[0037] The adjacent vector acquisition unit 172a acquires, from the memory 160, motion vectors Vp applied to inter prediction of adjacent decoded CUs located around (above or to the left of) the current CU to be coded, and outputs a list of the acquired motion vectors Vp to the motion vector correction unit 172d. Note that the motion vector may include the values ​​of the horizontal (x-axis direction) and vertical (y-axis direction) vectors of the motion vector, as well as the temporal position of the picture to be referenced (reference image) (for example, POC: Picture Order Count or a reference index in a reference list).

[0038] 4, when intra prediction is applied to some of the adjacent decoded CUs located above or to the left of the encoding target CU, the adjacent vector acquisition unit 172a may interpolate the motion vector by substituting the motion vector applied to the CU close to the part of the adjacent decoded CU, may calculate and interpolate the motion vector by a weighted average of the surrounding available motion vectors, or may set a motion vector that is predefined by the encoding method. Interpolation is performed in the same manner when the encoding target CU is at the edge of the screen.

[0039] The motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector V by a method such as block matching, and outputs the calculated motion vector V to the motion vector correction unit 172d.

[0040] The motion vector correction unit 172d generates a corrected motion vector V' by correcting the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a, and outputs the corrected motion vector V' to the predicted image generation unit 172e.

[0041] Specifically, the motion vector correction unit 172d corrects the motion vector V as shown in Fig. 5. The motion vector correction unit 172d corrects the motion vector V by weighted averaging using the adjacent motion vector Vp acquired by the adjacent vector acquisition unit 172a and the motion vector V of the current block to be coded.

[0042] First, the motion vector correction unit 172d calculates the adjacent motion vector Vp by subtracting the adjacent motion vector V L and the adjacent motion vector V T These are classified as shown in the following formula (1).

[0043] V L = [Vp[-1,0], Vp[-1,1], Vp[-1,2], …, Vp[-1,N-1]] V T = [Vp[0,-1], Vp[1,-1], Vp[2,-1], …, Vp[M-1,-1]] (1)

[0044] Secondly, the motion vector correction unit 172d calculates a weighting factor W L , W T , W V The motion vector correction unit 172d determines the weighting factor W by, for example, the following equation (2): L , W T , W V Calculate.

[0045] W L = 32 >> log2(height) W T = 32 >> log2(width) W V = 32 - W L - W T (2)

[0046] Here, width and height represent the width and height of the CU to be coded, respectively. L , W TIt is sufficient that the value is determined according to the width or height of the block, or the ratio thereof, and is not limited to the calculation method of formula (2).

[0047] Third, the motion vector correction unit 172d calculates the motion vector V[x, y] before correction using a weighting factor W L , W T and the adjacent motion vector V L and V T Then, the corrected motion vector V' is calculated, for example, by the following equation (3).

[0048] V' = W L ·median(V L ) + W T ·median(V T ) + W V ·V) (3)

[0049] The predicted image generation unit 172e generates a predicted image of the CU to be encoded by performing inter prediction of the CU to be encoded using the corrected motion vector V' generated by the motion vector correction unit 172d, and outputs the generated predicted image (inter predicted image) to the switching unit 173.

[0050] The predicted image is input to the subtraction unit 110 via the switching unit 173, and the subtraction unit 110 outputs a prediction residual representing the difference between the current CU to be coded and the predicted image to the transform / quantization unit 120. The transform / quantization unit 120 generates quantized orthogonal transform coefficients from the prediction residuals, and outputs the generated orthogonal transform coefficients to the entropy coding unit 130.

[0051] <Image Decoding Device> Next, an image decoding device according to this embodiment will be described. However, duplicated explanations will be omitted for operations similar to those of the above-mentioned image encoding device 1. Fig. 6 is a diagram showing the configuration of an image decoding device 2 according to this embodiment.

[0052] As shown in FIG. 6, the image decoding device 2 includes an entropy decoding unit 200 , an inverse quantization and inverse transform unit 210 , a synthesis unit 220 , a memory 230 , and a prediction unit 240 .

[0053] The entropy decoding unit 200 decodes the coded stream generated by the image coding device 1, and outputs the quantized orthogonal transform coefficients to the inverse quantization and inverse transform unit 210. In addition, the entropy decoding unit 200 obtains control information related to prediction (intra prediction and inter prediction), and outputs the obtained control information to the prediction unit 240.

[0054] The inverse quantization and inverse transform unit 210 performs inverse quantization processing and inverse orthogonal transform processing on a CU basis. The inverse quantization and inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212.

[0055] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the image encoding device 1. The inverse quantization unit 211 reconstructs the orthogonal transform coefficients of the decoding target CU by inverse quantizing the quantized orthogonal transform coefficients input from the entropy decoding unit 200, using a quantization parameter (Qp) and a quantization matrix, and outputs the reconstructed orthogonal transform coefficients to the inverse transform unit 212.

[0056] The inverse transform unit 212 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121 of the image encoding device 1. The inverse transform unit 212 performs inverse orthogonal transform processing on the orthogonal transform coefficients input from the inverse quantization unit 211 to restore prediction residuals, and outputs the restored prediction residuals (restored prediction residuals) to the synthesis unit 220.

[0057] The synthesis unit 220 reconstructs (decodes) the original CU by synthesizing the prediction residual input from the inverse transform unit 212 and the prediction image input from the prediction unit 240 on a pixel-by-pixel basis, and outputs the decoded image on a CU-by-CU basis to the memory 230.

[0058] The memory 230 stores the decoded image input from the synthesis unit 220. The memory 230 stores the decoded image on a frame-by-frame basis. Furthermore, the memory 230 stores the motion vector calculated by the inter prediction unit 242 for each CU. The memory 230 outputs the decoded image on a frame-by-frame basis to the outside of the image decoding device 2. Note that the memory 230 may be configured with a plurality of memories. A loop filter may be provided between the synthesis unit 220 and the memory 230.

[0059] The prediction unit 240 performs prediction on a CU basis. The prediction unit 240 includes an intra prediction unit 241, an inter prediction unit 242, and a switching unit 243.

[0060] The intra prediction unit 241 refers to the decoded image stored in the memory 230, and generates an intra prediction image by predicting the CU to be decoded using intra prediction in accordance with the control information input from the entropy decoding unit 200, and outputs the generated intra prediction image to the switching unit 243.

[0061] The inter prediction unit 242 predicts the current CU to be decoded by inter prediction, using the decoded image stored in the memory 230 as a reference image. The inter prediction unit 242 generates an inter prediction image by performing inter prediction according to the control information input from the entropy decoding unit 200, and outputs the generated inter prediction image to the switching unit 243. In this embodiment, the inter prediction unit 242 has the same configuration as the inter prediction unit 172 shown in FIG. 3.

[0062] The switching unit 243 switches between the intra-predicted image input from the intra-prediction unit 241 and the inter-predicted image input from the inter-prediction unit 242 , and outputs either of the predicted images to the synthesis unit 220 .

[0063] <Inter prediction section operation> 7 is a diagram showing the operation of the inter prediction unit 172 of the image encoding device 1 according to this embodiment. The inter prediction unit 242 of the image decoding device 2 operates in the same manner as the inter prediction unit 172 of the image encoding device 1. However, in the operation of the inter prediction unit 242, the "CU to be encoded" in the operation of the inter prediction unit 172 is read as the "CU to be decoded."

[0064] As shown in FIG. 7, in step S1, the adjacent vector acquisition unit 172a acquires, from the memory 160, a motion vector Vp applied to inter prediction of each adjacent decoded CU located around (above or to the left of) the current CU to be coded.

[0065] In step S2, the motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector V by a technique such as block matching.

[0066] In step S3, the motion vector correction unit 172d corrects the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a to generate a corrected motion vector V'.

[0067] In step S4, the predicted image generation unit 172e performs inter prediction using the corrected motion vector V', and generates a predicted image of the current CU for encoding.

[0068] In this way, according to the inter prediction unit 172 of this embodiment, the motion vector corrected using the adjacent motion vector Vp can be used to reduce discontinuity in the motion vector, thereby preventing the prediction residual from increasing in blocks near object boundaries and improving coding efficiency.

[0069] <Example of changes to the inter prediction section> FIG. 8 is a diagram showing a modification of the inter predictor 172 according to the first embodiment.

[0070] As shown in FIG. 8, the inter prediction unit 172 has a continuity evaluation unit 172f and a correction adjacent vector determination unit 172g in addition to the configuration according to the embodiment described above.

[0071] The continuity evaluation unit 172f evaluates the continuity of motion in the boundary area between each decoded block included in the group of adjacent decoded blocks and the current CU by comparing the motion vector V applied to the current CU with the adjacent motion vector Vp.

[0072] The continuity evaluation unit 172f acquires motion vectors Vp applied to one or more decoded adjacent CUs adjacent to the current CU to be coded from the memory 160. Then, the continuity evaluation unit 172f evaluates the motion continuity between the current CU to be coded and the adjacent decoded CUs by comparing the motion vector V applied to the current CU to the motion vectors Vp applied to the adjacent decoded CUs, and outputs the evaluation result to the correction adjacent vector determination unit 172g.

[0073] For example, the continuity evaluation unit 172f evaluates that the continuity is high when the L1 norm, which indicates the difference (distance) between the motion vector V applied to the encoding target CU and the motion vector Vp of the decoded CU, is less than a predefined threshold. On the other hand, when the L1 norm is equal to or greater than the threshold, the continuity is evaluated as low. However, the index value of the continuity is not limited to the L1 norm, and other index values ​​may be used.

[0074] In addition, the motion vector includes a flag indicating a decoded frame to be referenced (e.g., inter_pred_idc, ref_idx in HEVC) in addition to vector values ​​in the X-axis and Y-axis directions. When evaluating the continuity of the motion vector with an adjacent block, the continuity evaluation unit 172f may evaluate the continuity after scaling the vector value of the motion vector applied to the adjacent decoded CU according to the temporal distance (POC distance) between a frame (current frame) including the coding target CU and a decoded frame referenced by the coding target CU, and the POC distance between the current frame and a decoded frame referenced by the adjacent decoded CU.

[0075] The correction adjacent vector determination unit 172g determines a motion vector to be used for correcting the motion vector V based on the motion continuity evaluated by the continuity evaluation unit 172f, based on the motion vector Vp.

[0076] For example, the correction adjacent vector determination unit 172g V L = {Vp[-1,0], Vp[-1,1], Vp[-1,2], …, Vp[-1,N-1]} V T = {Vp[0,-1], Vp[1,-1], Vp[2,-1], …, Vp[M-1,-1]} When V' L ={ v L ∈V L | || v L - V ||2< Th} V' T ={ v T ∈V T | || v T - V ||2< Th} In this manner, the motion vector to be used for correcting the motion vector V is determined.

[0077] That is, V' L is a set of adjacent vectors Vp on the left side of the current CU to be coded, whose L2 norm with respect to the motion vector V is equal to or smaller than a threshold Th, and V' T is a set of vectors Vp adjacent to the upper side of the current CU to be coded, the L2 norm of which with respect to the motion vector V is equal to or smaller than a threshold value Th.

[0078] The motion vector correction unit 172d calculates these V' L and V' T Using the (third motion vector), a corrected motion vector V' is calculated according to the following equation (4).

[0079] V' = W L median(V' L ) + WT median(V' T ) + W V ·V) (4)

[0080] <2. Second embodiment> The second embodiment will be described mainly focusing on the differences from the first embodiment.

[0081] FIG. 9 is a diagram showing the configuration of the inter prediction unit 172 of the image encoding device 1 according to the second embodiment.

[0082] As shown in FIG. 9, the inter prediction unit 172 according to the second embodiment has a small region division unit 172b in addition to the configuration described in the first embodiment (the adjacent vector acquisition unit 172a, the motion vector calculation unit 172c, the motion vector correction unit 172d, and the predicted image generation unit 172e).

[0083] The small region division unit 172b divides the current CU into a plurality of small regions of a predetermined size, and outputs information on each of the small regions obtained by the division to the motion vector correction unit 172d. Such small regions may be called sub-blocks.

[0084] Fig. 10 is a diagram showing an example of a small region. As shown in Fig. 10(a), the size of each small region may be the same, for example, each small region may be a region of 4 x 4 pixels. Alternatively, if a common process is defined in advance for the image decoding device and the image decoding device, the size of each small region may be different, as shown in Fig. 10(b). For example, the CU to be coded may be divided into small regions by a method in which the vicinity of the boundary is finely divided and the vicinity of the center is roughly divided.

[0085] The motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector V by a method such as block matching, and outputs the calculated motion vector V to the motion vector correction unit 172d. In this embodiment, the motion vector calculation unit 172c may calculate one motion vector V for the current CU to be coded, or may calculate a motion vector V for each small region obtained by the small region division unit 172b dividing the current CU to be coded.

[0086] The motion vector correction unit 172d generates a corrected motion vector V' by correcting the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a, and outputs the corrected motion vector V' to the predicted image generation unit 172e.

[0087] In this embodiment, the motion vector correction unit 172d corrects the motion vector V for each small region, as shown in Fig. 11. The motion vector correction unit 172d weights the motion vector Vp of the adjacent decoded CU corresponding to the position of the small region by using a weight determined according to the position of the small region, and corrects the motion vector V to be applied to the small region by using the weighted motion vector Vp.

[0088] For example, the motion vector correction unit 172d substitutes the motion vector V before correction into the motion vector array V[x,y] for each small region of the encoding target CU. However, if the motion vector V before correction is a motion vector that differs for each small block in the encoding target CU, the motion vector that differs for each small region is substituted into the motion vector array V[x,y] based on the motion vector set for each small block. If the size of the small block is different from that of the small region, the substitution is performed by a method specified in advance. For example, if the size of the small block is larger than the small region, the motion vector of the small block is substituted into the motion vector array V[x,y] of all small regions included in the small block. On the other hand, if the size of the small block is smaller than the small region, a vector calculated by the average or median of the motion vectors of all small blocks included in the small region is substituted into the motion vector V[x,y] of the small region.

[0089] First, the motion vector correction unit 172d calculates a weighting factor W L , W T , W LT , W V The motion vector correction unit 172d determines the weighting factor W by, for example, the following equation (5): L , W T , W LT , W V Calculate.

[0090]

number

[0091] Here, x and y represent the position of the small region, and width and height represent the width and height of the CU to be coded, respectively. L , W T , W LT , W V It is sufficient that the calculation method is determined according to the position of the small region and the width and height of the CU, and is not limited to the calculation method of formula (5).

[0092] Secondly, the motion vector correction unit 172d corrects the motion vector V[x, y] before correction based on the weighting factors WL, WT, WLT, and WV and the adjacent motion vector Vp, for example, using the following formula: (6) The corrected motion vector V'[x, y] is calculated.

[0093]

number

[0094] The predicted image generation unit 172e generates a predicted image of the CU to be encoded by performing inter prediction for each small area of ​​the CU to be encoded using the corrected motion vector V'[x, y] generated by the motion vector correction unit 172d, and outputs the generated predicted image (inter predicted image) to the switching unit 173.

[0095] Specifically, the predicted image generation unit 172e generates a predicted image for each small region by performing inter prediction for each small region using the motion vector V'[x, y] for each small region, and generates a predicted image for the CU to be encoded by synthesizing the predicted images for each small region.

[0096] Note that the predicted image is input to the subtraction unit 110 via the switching unit 173, and the subtraction unit 110 outputs a prediction residual representing the difference between the current CU to be coded and the predicted image to the transformation and quantization unit 120. The transformation and quantization unit 120 generates a quantized orthogonal transform coefficient from the prediction residual, and outputs the generated orthogonal transform coefficient to the entropy coding unit 130. Note that the entropy coding unit 130 need only code one motion vector for each current CU to be coded, without coding (signaling) a motion vector for each small region.

[0097] In addition, in this embodiment, the inter prediction unit 242 of the image decoding device 2 has the same configuration as the inter prediction unit 172 shown in FIG.

[0098] <Inter prediction section operation> FIG. 12 is a diagram showing the operation of the inter prediction unit 172 of the image encoding device 1 according to this embodiment. The inter prediction unit 242 of the image decoding device 2 also operates in the same manner as the inter prediction unit 172 of the image encoding device 1. However, in the operation of the inter prediction unit 242, The "CU to be encoded" in the operation of the prediction unit 172 is replaced with the "CU to be decoded."

[0099] As shown in FIG. 12, in step S11, the adjacent vector acquisition unit 172a acquires, from the memory 160, the motion vector Vp applied to the inter prediction of each adjacent decoded CU located around (above or to the left of) the current CU to be coded.

[0100] In step S12, the small region dividing unit 172b divides the current CU into a plurality of small regions of a predetermined size.

[0101] In step S13, the motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector V by a technique such as block matching.

[0102] In step S14, the motion vector correction unit 172d corrects the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a to generate a corrected motion vector V'.

[0103] In step S15, the predicted image generation unit 172e performs inter prediction for each small region using the corrected motion vector V', and generates a predicted image of the current CU for encoding.

[0104] In this way, according to the inter prediction unit 172 of this embodiment, fine-grained inter prediction can be performed for each small region in the CU to be decoded using a motion vector corrected using the adjacent motion vector Vp, thereby preventing the prediction residual from increasing in blocks near object boundaries and improving coding efficiency.

[0105] <Example of changes to the inter prediction section> FIG. 13 is a diagram showing a modification of the inter predictor 172 according to the second embodiment.

[0106] As shown in FIG. 13, the inter prediction unit 172 according to this modified example has a continuity evaluation unit 172f and a corrected small region determination unit 172h in addition to the configuration according to the above-described embodiment.

[0107] The continuity evaluation unit 172f evaluates the continuity of motion in the boundary area between each decoded block included in the group of adjacent decoded blocks and the current CU by comparing the motion vector V applied to the current CU with the adjacent motion vector Vp.

[0108] The continuity evaluation unit 172f acquires motion vectors Vp applied to one or more decoded adjacent CUs adjacent to the current CU from the memory 160. Then, the continuity evaluation unit 172f evaluates the motion continuity between the current CU and the adjacent decoded CUs by comparing the motion vector V applied to the current CU with the motion vectors Vp applied to the adjacent decoded CUs, and outputs the evaluation result to the correction small region determination unit 172h.

[0109] For example, as shown in Fig. 14(a), the continuity evaluation unit 172f evaluates that the continuity is high when the L1 norm, which indicates the difference (distance) between the motion vector V applied to the encoding target CU and the motion vector Vp of the decoded CU, is less than a predefined threshold. On the other hand, when the L1 norm is equal to or greater than the threshold, the continuity is evaluated as low. However, the index value of the continuity is not limited to the L1 norm, and other index values ​​may be used.

[0110] In addition, the motion vector includes a flag indicating a decoded frame to be referenced (e.g., inter_pred_idc, ref_idx in HEVC) in addition to vector values ​​in the X-axis and Y-axis directions. When evaluating the continuity of the motion vector with an adjacent block, the continuity evaluation unit 172f may evaluate the continuity after scaling the vector value of the motion vector applied to the adjacent decoded CU according to the temporal distance (POC distance) between a frame (current frame) including the coding target CU and a decoded frame referenced by the coding target CU, and the POC distance between the current frame and a decoded frame referenced by the adjacent decoded CU.

[0111] The correction small region determination unit 172h determines at least one small region in which the motion vector V should be corrected, based on the motion continuity evaluated by the continuity evaluation unit 172f. For example, as shown in Fig. 14(b), the correction small region determination unit 172h determines only small regions corresponding to regions with low motion continuity as small regions in which the motion vector V should be corrected. However, the correction small region determination unit 172h may determine all small regions of the current CU to be coded as small regions in which the motion vector V should be corrected.

[0112] In this modification, the motion vector correction unit 172d generates a corrected motion vector V' for the small region determined by the corrected small region determination unit 172h. The motion vector correction unit 172d may also determine a weighting factor according to the position of the small region and the continuity of the motion evaluated by the continuity evaluation unit 172f, and weight the adjacent motion vector Vp corresponding to the position of the small region using the determined weighting factor.

[0113] The motion vector correction unit 172d may set the weight of the adjacent motion vector Vp corresponding to an area with low motion continuity to be large. For example, in the formula (5), a value c corresponding to the motion continuity is x , c y Then, the weighting factor W is calculated by the following equation (7). L , W T , W LT , W V Calculate.

[0114]

number

[0115] In this manner, according to this modified example, the coding efficiency can be further improved when performing inter prediction by correcting the motion vector V of the CU (small region) to be coded taking into account the continuity of motion in the block boundary region.

[0116] <3. Other embodiments> The image encoding device 1 may be provided by a program that causes a computer to execute each process performed by the image decoding device 2, and the image encoding device 1 may be provided by a program that causes a computer to execute each process performed by the image decoding device 2. The program may be recorded in a computer-readable medium. The computer-readable medium can be used to install the program in a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0117] Furthermore, circuits for executing the processes performed by the image encoding device 1 may be integrated, and the image encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC). Similarly, circuits for executing the processes performed by the image decoding device 2 may be integrated, and the image decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC).

[0118] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes, etc. are possible without departing from the spirit of the invention. [Explanation of symbols]

[0119] 1: Image encoding device 2: Image decoding device 100: Block division section 110: Subtraction section 120: Transformation and quantization section 121: Conversion section 122: Quantization section 130: Entropy coding unit 140: Inverse quantization and inverse transformation unit 141: Inverse quantization section 142: Inverse conversion section 150: Composite section 160:Memory 170: Prediction Department 171: Intra prediction unit 172: Inter prediction section 172a: Adjacent vector acquisition unit 172b:Small area division part 172c: Motion vector calculation unit 172d: Motion vector correction unit 172e: predicted image generation unit 172f: Continuity evaluation section 172g: Correction adjacent vector determination unit 172h: Correction small area determination section 173: Switching section 200: Entropy decoding unit 210: Inverse quantization and inverse transformation unit 211: Inverse quantization section 212: Inverse conversion unit 220: Composite section 230: Memory 240: Prediction Department 241: Intra prediction unit 242: Inter prediction unit 243: Switching section

Claims

1. An inter prediction device that performs inter prediction on a block to be coded obtained by dividing an original image, comprising: an adjacent vector acquisition unit that acquires a first motion vector applied to a block adjacent to the encoding target block; a motion vector generation unit that generates a new motion vector based on a second motion vector to be applied to the encoding target block, the first motion vector, and parameters determined according to a width and a height of the encoding target block; and a predicted image generating unit configured to generate a predicted image of the block to be coded by performing the inter prediction using the newly generated motion vector.

2. An image encoding device comprising the inter prediction device according to claim 1.

3. An image decoding device comprising the inter prediction device according to claim 1.

4. A program for causing a computer to function as the inter prediction device according to claim 1.

Images