Search area for motion vector refinement

Abstract

The invention relates to the construction of a search space for determining a motion vector of a current block of a picture in a video sequence. The search space construction is divided into two main stages, wherein a first part search space and a second part search space are determined, respectively. First, a first search space is constructed based on an initial estimate of a motion vector. A first candidate motion vector and a second candidate motion vector in the first search space are identified according to a cost function. A second search space is constructed based on the first and second candidate motion vectors. The motion vector of the current block is selected from the candidate motion vectors in the first search space and the second search space.

Description

[0001] This application is a divisional application of the original application with the application number 201780092336.7 and the original filing date of 30 June 2017, the whole content of the original application is incorporated herein by reference.

[0002] The present invention relates to the construction of a search space for determining motion vectors of image blocks in a video sequence. BACKGROUND

[0003] Current hybrid video codecs employ predictive coding. The images of a video sequence are subdivided into blocks of pixels, which are then coded. Instead of coding the blocks pixel by pixel, the whole block is predicted from coded pixels that are spatially or temporally close to the block. The encoder only further processes the difference between the block and its prediction. The further processing usually comprises transforming the block pixels into coefficients in a transform domain. The coefficients can then be further compressed, e.g. by quantization, and further compressed, e.g. by entropy coding, to form a bitstream. The bitstream can also comprise any signaling information that enables a decoder to decode the coded video. For example, the signaling can comprise settings similar to the encoder settings, e.g. the size of the input images, the frame rate, quantization step indications, the prediction applied to the blocks of images, etc.

[0004] The difference between a block and its prediction is called the residual of the block. More specifically, each pixel of a block has a residual, i.e. the difference between the intensity level of the pixel and the intensity level of its prediction. The intensity level of a pixel is called the pixel value or the value of the pixel. The residuals of all pixels of a block are collectively called the residual of the block. In other words, a block has a residual, which is a set or matrix composed of the residuals of all pixels of the block.

[0005] Temporal prediction exploits the temporal correlation between the images, also called frames, of a video. Therefore, a block to be decoded, also called current block, is predicted from one or more previously decoded images, also called reference images. The one or more reference images are not necessarily the images that precede the current image in which the current block is located in the display order of the video sequence. An encoder can code the images in a coding order that is different from the display order. As a prediction block for the current block, a collocated block in one of the reference images can be determined, called predictor. The collocated block can be located in the same position in the reference image as the current block in the current image. This prediction is accurate for stationary image regions, i.e. regions that do not move from one image to another.

[0006] In the encoder, in order to obtain a prediction value that takes motion into account, i.e. a motion-compensated prediction value, motion estimation is typically employed. The current block is predicted by a block located at a position in a reference picture indicated by a motion vector. The motion vector points from the position of the co-located block to the position of the current block (or vice versa, depending on the sign convention). In order to enable the decoder to determine the same prediction block for the current block as the encoder, the motion vector can be transmitted in the bitstream. In order to further reduce the signaling overhead due to transmitting a motion vector for each block, the motion vector itself can be estimated. The motion vector estimation can be based on motion vectors of blocks that are neighbors of the current block in the spatial and / or temporal domain.

[0007] The prediction block for the current block can be calculated using one reference picture or by weighting prediction blocks obtained from two or more reference pictures. The reference picture can be a neighboring picture, i.e. a picture that immediately precedes or follows the current picture in display order, as neighboring pictures are most likely to be similar to the current picture. However, the reference picture can generally be any picture that precedes the current picture in display order and any picture that precedes the current picture in the bitstream (in decoding order). This can provide an advantage, e.g. in the presence of occlusions and / or non-linear motion in the video content. The reference picture can be transmitted in the bitstream.

[0008] A special mode of inter prediction is so-called bi-prediction, in which two reference pictures are used to generate a prediction block for the current block. In particular, two prediction blocks determined in the respective two reference pictures are combined into a prediction signal for the current block. Bi-prediction can enable a more accurate prediction of the current block compared to uni-prediction, i.e. a prediction using only a single reference picture. A more accurate prediction results in a smaller difference between the pixels of the current block and the prediction block, i.e. smaller residual, which can be more efficiently encoded, i.e. compressed into a shorter bitstream.

[0009] In order to provide a more accurate motion estimation, the resolution of the reference picture can be enhanced, e.g. by interpolating samples between pixels. Fractional pixel interpolation can be performed by a weighted average of the nearest pixels. For example, in case of half-pixel resolution, bilinear interpolation can be employed. Other fractional pixels can be calculated as an average of the nearest pixels, e.g. weighted by the inverse of the distance between the respective nearest pixel and the pixel to be predicted.

[0010] For example, a motion vector can be estimated by computing a similarity between the current block and a corresponding prediction block in a reference picture to which the candidate motion vector points. This can be a complex computational task. To reduce the complexity, the number of candidate motion vectors can be reduced by restricting the candidate motion vectors within a certain search space. The search space can for example be defined by a number and / or position of pixels around a position in the reference picture that corresponds to the position of the current block in the current picture. Alternatively, the candidate motion vectors can be defined by a list of candidate motion vectors that consists of motion vectors of neighboring blocks.

[0011] Motion vectors are typically determined at least partly at the encoder side and signaled in the encoded bitstream to the decoder. However, motion vectors can also be derived at the decoder. In this case, the current block is not available at the decoder and cannot be used to compute a similarity between the current block and any block in a reference picture to which a candidate motion vector points. Therefore, instead of the current block, a template can be used that can be constructed from pixels of already decoded blocks. For example, already decoded pixels neighboring the current block can be used. Such a motion estimation provides the advantage of reduced signaling: the motion vector is derived in the same way at the encoder and the decoder and therefore does not need to be signaled. On the other hand, the accuracy of such a motion estimation can be lower.

[0012] To trade off accuracy and signaling overhead, motion vector estimation can be split into two steps: motion vector derivation and motion vector refinement. For example, motion vector derivation can comprise selecting a motion vector from a list of candidate motion vectors. The selected motion vector can be further refined, for example, by performing a search within a search space. The search in the search space is based on computing a cost function for each candidate position of the block to which each candidate motion vector points.

[0013] Document JVET-D0029: Decoder-side motion vector refinement based on bilateral template matching, X. Chen, J. An, J. Zheng (available at: http: / / phenix.it-sudparis.eu / jvet / site) shows a motion vector refinement where a first motion vector at integer pixel resolution is found and further refined by performing a search in a search space around the first motion vector with half-pixel resolution. Here, the pixel resolution (e.g. integer or half-integer pixel resolution) describes the resolution of the search space, i.e. the displacement of the search points relative to the non-refined motion vector input to the process. As a result, the search coordinates of the refinement stage do not necessarily coincide with actual pixel coordinates on the image plane. SUMMARY

[0014] Starting from the above described methods, it is an object of the present application to further improve the efficiency of motion vector estimation in order to improve the coding efficiency and / or to reduce the complexity.

[0015] To this end, a scheme for constructing a search space for motion vector refinement is provided, which involves a first search space and a second search space. The number and / or the positions in the second search space are determined based on two positions in the first search space, which are derived based on a cost function.

[0016] In particular, according to a first aspect, a device for determining a motion vector is provided, the motion vector to be used for inter prediction of a current block of a video frame. The device comprises a search space determination unit configured to obtain an estimate of the motion vector, to determine a first search space comprising a plurality of candidate motion vectors based on the estimate, to select a first candidate motion vector and a second candidate motion vector in the first search space according to a cost function, and to determine a second search space comprising one or more candidate motion vectors based on the first candidate motion vector and the second candidate motion vector. The device further comprises a motion vector selection unit configured to select the motion vector for the current block from the candidate motion vectors in the first search space and the second search space.

[0017] As an advantage, the number of candidate motion vectors to be tested in the motion vector refinement process at the decoder side can be reduced while maintaining a high quality encoding performance in terms of image quality and bit rate.

[0018] Advantageously, the cost function is based on a predetermined template and indicates for each candidate motion vector a similarity between the predetermined template and a prediction value pointed to by the respective candidate motion vector. Thus, the search space determination unit is configured to select two candidate motion vectors in the first search space as the first candidate motion vector and the second candidate motion vector, which point to the prediction values of the current block that are most similar to the predetermined template.

[0019] According to a first embodiment, the search space determination unit is further configured to determine the size and / or the positions of the second search space based on a direction of a connecting line of the positions pointed to by the first candidate motion vector and the second candidate motion vector, i.e. based on a direction of a difference vector of the first candidate motion vector and the second candidate motion vector. The difference vector can be defined as the first candidate motion vector subtracted from the second candidate motion vector and vice versa.

[0020] For example, the search space determination unit is further configured to set the first search space to have an integer pixel resolution. The search space determination unit is configured to include in the second search space one or more candidate motion vectors pointing to positions located in a direction of a connecting line of positions pointed to by the first and second candidate motion vectors, the second search space having a fractional pixel resolution. The direction of the line is the direction of a difference vector of the first and second candidate motion vectors.

[0021] As an example, at least one of the candidate motion vectors in the second search space points to a position between the positions pointed to by the first and second candidate motion vectors.

[0022] According to a second embodiment, the search space determination unit is configured to determine a position of a region comprising at least two positions pointed to by at least two candidate motion vectors, respectively, the at least two positions being adjacent in a pixel resolution of the second search space, and to determine the second search space as the positions in the region not belonging to the first search space.

[0023] For example, the search space determination unit can be configured to determine a first search space comprising an estimate of the motion vector and a candidate motion vector pointing to a position adjacent to the position pointed to by the estimate of the motion vector in a pixel resolution of the first search space.

[0024] According to a third embodiment, the search space determination unit is further configured to determine one candidate motion vector as the first candidate motion vector in the second search space, wherein the candidate motion vector points to a position adjacent to the positions pointed to by the identified first and second candidate motion vectors in a pixel resolution of the first search space and different from the position pointed to by the estimate of the motion vector.

[0025] For example, the search space determination unit is further configured to select one candidate motion vector as the other candidate motion vectors in the second search space, wherein the candidate motion vector points to a position in a resolution of the second search space, the position being approximately located on a connecting line of the estimate of the motion vector and the position pointed to by the candidate motion vector of the second search space, the pixel resolution of the second search space being higher than the pixel resolution of the first search space.

[0026] As an example, the other candidate motion vectors of the second search space point to positions between the first candidate motion vector of the second search space and the position pointed to by the estimate of the motion vector.

[0027] As one example of all embodiments of the first aspect, the second search space has a higher resolution than the first search space.

[0028] As yet another example, the apparatus further comprises a motion vector determination unit configured to determine the estimate of the motion vector from a motion vector list comprising motion vectors of at least one block neighboring the current block.

[0029] A video decoder for decoding a plurality of pictures from a bitstream is also provided. The video decoder comprises a bitstream parser configured to obtain an indication of an estimate of a motion vector from the bitstream; an inter prediction unit comprising the apparatus of any of the embodiments and examples of the first aspect, further configured to determine a prediction block to which a motion vector of a current block points; and a reconstruction unit configured to reconstruct the current block based on the prediction block.

[0030] A video encoder for encoding a plurality of pictures into a bitstream is also provided. The video encoder comprises an inter prediction unit comprising the apparatus of any of the embodiments and examples of the first aspect, further configured to determine a prediction block to which a motion vector of a current block points; a bitstream former configured to include an indication of an estimate of the motion vector in the bitstream; and a reconstruction unit configured to reconstruct the current block based on the prediction block and store the reconstructed block in a memory.

[0031] According to a second aspect, a method for determining a motion vector to be used for inter prediction of a current block is provided. The method comprises the steps of obtaining an estimate of the motion vector; determining a first search space comprising a plurality of candidate motion vectors based on the estimate; selecting a first candidate motion vector and a second candidate motion vector from the first search space according to a cost function; determining a second search space comprising one or more candidate motion vectors based on the first candidate motion vector and the second candidate motion vector; and selecting the motion vector of the current block from the candidate motion vectors in the first and second search spaces.

[0032] Advantageously, the cost function is based on a predetermined template and indicates, for each candidate motion vector, a similarity between the predetermined template and a prediction value to which the respective candidate motion vector points. Thus, selecting the first candidate motion vector and the second candidate motion vector comprises selecting two candidate motion vectors in the first search space which point to prediction values of the current block that are most similar to the predetermined template.

[0033] In a first example embodiment, in the step of determining the second search space, the size and / or position of the second search space is determined based on a direction of a connecting line of positions to which the first candidate motion vector and the second candidate motion vector point.

[0034] As an example, in the step of determining the first search space comprising a plurality of candidate motion vectors, the first search space has an integer pixel resolution. In the step of determining the second search space, the second search space is determined as one or more candidate motion vectors pointing to a position located in a direction of a connecting line of positions pointed to by the first candidate motion vector and the second candidate motion vector, the second search space having a fractional pixel resolution.

[0035] For example, at least one of the candidate motion vectors in the second search space points to a position between the positions pointed to by the first candidate motion vector and the second candidate motion vector.

[0036] According to a second exemplary embodiment, in the step of determining the second search space, a position of a region comprising at least two positions pointed to by at least two candidate motion vectors, respectively, is determined, the at least two positions being adjacent in a pixel resolution of the second search space, and the second search space is determined as positions in the region not belonging to the first search space.

[0037] As an example, in the step of determining the first search space, the first search space comprises an estimated value of the motion vector and a candidate motion vector pointing to a position adjacent to a position pointed to by the estimated value of the motion vector in a pixel resolution of the first search space.

[0038] According to a third exemplary embodiment, in the step of determining the second search space, one candidate motion vector is determined as a first candidate motion vector in the second search space, wherein the candidate motion vector points to a position adjacent to positions pointed to by the first candidate motion vector and the second candidate motion vector identified in the first search space in a pixel resolution of the first search space and different from a position pointed to by the estimated value of the motion vector.

[0039] For example, one candidate motion vector is determined as the other at least one candidate motion vector in the second search space, wherein the candidate motion vector points to a position in a resolution of the second search space, the position being located approximately on a connecting line of a position pointed to by the estimated value of the motion vector and a position pointed to by the candidate motion vector of the second search space. Wherein the pixel resolution of the second search space is higher than the pixel resolution of the first search space.

[0040] As an example, the other candidate motion vector of the second search space points to a position located between the first candidate motion vector of the second search space and the position pointed to by the estimated value of the motion vector.

[0041] As one example of all embodiments of the second aspect, the second search space has a higher resolution than the first search space.

[0042] As yet another example, the step of obtaining an estimate of the motion vector is performed by determining the estimate of the motion vector from a motion vector list comprising motion vectors of at least one block neighboring the current block. BRIEF DESCRIPTION OF DRAWINGS

[0043] Example embodiments will be described in detail below with reference to the following drawings:

[0044] Figure 1 is a block diagram of an exemplary structure of a video encoder;

[0045] Figure 2 is a block diagram of an exemplary structure of a video decoder;

[0046] Figure 3 is a block diagram of an exemplary structure of an apparatus for determining a motion vector;

[0047] Figure 4 is a schematic diagram of a current block and an exemplary search space configuration;

[0048] Figures 5 to 8 is a schematic diagram of a search space configuration according to a first exemplary embodiment;

[0049] Figure 9 is a schematic diagram of determining a second search space according to a first exemplary embodiment;

[0050] Figure 10 and 11 is a schematic diagram of further search space configurations according to a first embodiment;

[0051] Figure 12 and Figure 13 is a schematic diagram of a current block and an exemplary search space configuration according to a second embodiment;

[0052] Figure 14 is a schematic diagram of determining a search space according to a third embodiment;

[0053] Figure 15 is a block diagram of a search space determination unit according to a third embodiment;

[0054] Figure 16 is a schematic diagram of a search space configuration by combining different embodiments;

[0055] Figure 17 is a schematic diagram of exemplary determining a search space by combining different embodiments;

[0056] Figure 18is a flowchart of a method for motion vector refinement;

[0057] Figure 19 is a flowchart of a method for computing the cost of a search space position. DETAILED DESCRIPTION

[0058] The present invention relates to efficient determination of a search space for motion compensation, in particular for motion vector refinement. The determination of the search space can be used for motion estimation applied during encoding and decoding of a video. Exemplary encoders and decoders are described below, which can implement motion estimation using the search space construction of the present invention.

[0059] Figure 1 An encoder 100 is shown, which comprises an input for receiving an input block of a frame or image of a video stream and an output for providing an encoded video bitstream. The term "frame" is used as a synonym for image in the present invention. However, it is noted that the present invention is also applicable in the field of application of interlacing. Typically, an image comprises m x n pixels. These pixels correspond to image samples and each pixel can comprise one or more color components. For simplicity, in the following description, a pixel represents a luminance sample. However, it is noted that the motion vector search of the present invention can be applied to any color component comprising chrominance or components of a color space such as RGB. On the other hand, it can be beneficial to perform motion vector estimation for only one component and apply the determined motion vector to multiple (or all) components.

[0060] The size of the input blocks to be encoded is not necessarily the same. One image can comprise blocks of different sizes and different image block raster can also be different.

[0061] The encoder 100 is configured to perform prediction, transform, quantization and entropy coding on the video stream. The transform, quantization and entropy coding are performed by a transform unit 101, a quantization unit 102 and an entropy coding unit 103, respectively, to generate an encoded video bitstream as output.

[0062] The video stream can comprise a plurality of frames. Each frame is divided into intra- or inter-coded blocks. For example, the blocks of the first frame of the video stream are intra-coded by the intra-prediction unit 109. Intra-coded frames are coded using only information from that frame so that they can be decoded independently from other frames. Thus, intra-coded frames can provide entry points in the bitstream, e.g. for random access. The blocks of other frames of the video stream are inter-coded by the inter-prediction unit 110: each block of an inter-coded frame is predicted from a block in another frame (reference frame), e.g. a previously coded frame. The mode selection unit 108 is used to select whether the blocks of a certain frame are intra- or inter-predicted, i.e. whether the blocks of that frame are processed by the intra-prediction unit 109 or by the inter-prediction unit 110. The mode selection unit 108 also controls the parameters of the intra- or inter-prediction. In order to be able to refresh the image information, inter-coded frames can comprise not only inter-coded blocks but also one or more intra-coded blocks. In contrast, intra-coded frames contain only intra-coded blocks without inter-coded blocks. Intra-coded frames can be inserted into the video sequence (e.g. periodically, i.e. after a certain number of inter-coded frames are inserted) to provide entry points for decoding, i.e. points at which the decoder can start decoding without using information from previous frames.

[0063] The intra-prediction unit 109 is a block prediction unit. The coded blocks can also be processed by the inverse quantization unit 104 and the inverse transform unit 105 for spatial or temporal prediction. After the blocks are reconstructed, the loop filter unit 106 can be applied to further improve the quality of the decoded image. The filtered blocks then form the reference frames which are subsequently stored in the frame buffer 107. This decoding loop (decoder) on the encoder side provides the advantage of producing the same reference frames as the reference images reconstructed on the decoder side. Thus, the encoder and decoder sides work in a corresponding manner. The term "reconstruction" here refers to adding the decoded residual block to the prediction block to obtain the reconstructed block.

[0064] The inter-prediction unit 110 receives as input the current frame or image block to be inter-coded and one or several reference frames or images from the frame buffer 107. Motion estimation and motion compensation are performed by the inter-prediction unit 110. Motion estimation is used to obtain (e.g. based on a cost function) a motion vector and a reference frame. Motion compensation then describes a current block of the current frame in terms of a reference block of the reference frame, i.e. by the motion vector. The inter-prediction unit 110 selects a prediction block (i.e. predictor) for the current block from a set of candidate blocks (i.e. candidate predictors) in one or several reference frames such that the prediction block minimizes the cost function. In other words, the candidate block with the minimum cost function will be used as the prediction block for the current block.

[0065] For example, the cost function can be a measure of the difference between the current block and the candidate block, i.e. a measure of the residual of the current block with respect to the candidate block. For example, the cost function can be the sum of absolute difference (SAD) between all pixels (samples) of the current block and all pixels of the candidate block in the candidate reference picture. However, generally any similarity measure can be employed, e.g. mean square error (MSE) or a structural similarity metric (SSIM).

[0066] However, the cost function can also be the number of bits necessary for the inter- block coding and / or the distortion caused by such coding. Thus, a rate-distortion optimization process can be used to decide on the motion vector selection and / or generally on the coding parameters, e.g. whether to use inter- or intra-prediction for a certain block and which setting to use.

[0067] The intra-prediction unit 109 receives as input the current block of the current frame or picture to be intra-coded and one or several reference samples from the already constructed area of the current frame. Then, the intra-prediction describes the pixels of the current block of the current frame as a function of the reference samples of the current frame. The intra-prediction unit 109 outputs a predicted block of the current block, which advantageously minimizes the difference between the current block to be coded and its predicted block, i.e. minimizes the residual block. The minimization of the residual block can be based on a rate-distortion optimization process, for example. In particular, the predicted block is obtained as a directional interpolation of the reference samples. The direction can be determined by a rate-distortion optimization and / or by computing a similarity measure related to the intra-prediction as described above.

[0068] The difference between the current block and its prediction block, i.e. the residual block, is then transformed by a transform unit 101. The transform coefficients are quantized by a quantization unit 102 and entropy coded by an entropy coding unit 103. The resulting coded video bitstream comprises intra coded blocks and inter coded blocks and corresponding signaling (e.g. mode indication, motion vector indication and / or intra prediction direction). The transform unit 101 can apply a linear transform, e.g. a discrete Fourier transformation (DFT) or a discrete cosine transformation (DCT). Such a transform into the spatial frequency domain offers the advantage that the resulting coefficients usually have higher values in lower frequencies. Thus, after performing an efficient coefficient scan (e.g. a zigzag scan) and quantization, the resulting value sequence usually has some larger values at the beginning and ends with zeros. This enables a further efficient coding. The quantization unit 102 performs a lossy compression by reducing the resolution of the coefficient values. Then, the entropy coding unit 103 assigns binary code words to the coefficient values. The code words are written into a bitstream, which is called coded bitstream. The entropy coder also encodes signaling information (not shown in Figure 1

[0069] Figure 2 An example of a video decoder 200 is shown. The video decoder 200 comprises, inter alia, a reference picture buffer 207 and an intra prediction unit 209, which is a block prediction unit. The reference picture buffer 207 is used to store at least one reference frame reconstructed from the coded video bitstream. The intra prediction unit 209 is used to generate a prediction block, which is an estimate of the to-be-decoded block. The intra prediction unit 209 is used to generate this prediction block based on reference samples taken from the reference picture buffer 207.

[0070] The decoder 200 is used to decode the coded video bitstream generated by the video encoder 100, preferably the decoder 200 and the encoder 100 generate the same prediction block for the respective block to be encoded / decoded. The features of the reference picture buffer 207 and the intra prediction unit 209 are similar to the features of the reference picture buffer 107 and the intra prediction unit 109 in the Figure 1

[0071] The video decoder 200 comprises further units also present in the video encoder 100, e.g. an inverse quantization unit 204, an inverse transform unit 205 and a loop filter unit 206, which correspond to the inverse quantization unit 104, the inverse transform unit 105 and the loop filter unit 106 in the video encoder 100, respectively.

[0072] ​​The entropy decoding unit 203 is configured to decode the received encoded video bitstream to obtain quantized residual transform coefficients and signaling information. The quantized residual transform coefficients are fed to the inverse quantization unit 204 and the inverse transform unit 205 to produce a residual block. The residual block is added to a prediction block and the resulting sum is fed to the loop filter unit 206 to obtain a decoded video block. Frames of the decoded video can be stored in the reference picture frame buffer 207 and used as reference frames for inter prediction.

[0073] Generally, Figure 1 and Figure 2 The intra prediction units 109 and 209 in the encoders and decoders 100 and 200, respectively, can use reference samples from the already coded region to generate a prediction signal for a block that needs to be coded or decoded.

[0074] The entropy decoding unit 203 receives the encoded bitstream as its input. The bitstream can first be parsed, i.e. the signaling parameters and the residuals are extracted from the bitstream. The syntax and semantics of the bitstream can be defined by a standard so that the encoder and the decoder can work in an interoperable manner. As mentioned in the background section above, the encoded bitstream comprises further information than the prediction residuals. In case of motion-compensated prediction, the motion vector indication is also encoded in the bitstream and parsed from the bitstream at the decoder. The motion vector indication can be given by providing a reference picture for the motion vector and by motion vector coordinates. In the present example, it is assumed that the coordinate (0, 0) is the position in the reference frame that corresponds to the position of the current block being processed in the current frame, and the motion vector coordinates are the x- and y-coordinates within the reference picture that define the point at which the motion vector is displayed. However, the motion vector indication does not necessarily transmit this coordinate directly. Typically, any identification of the motion vector is applicable, e.g. a pointer (index) to a list of candidate motion vectors or any other identification that enables to identify the inter prediction of a block.

[0075] To efficiently encode the reference pictures, the H.265 codec (ITU-T, H.265, Series H: Audiovisual and Multimedia Systems: High efficiency video coding) provides a list of reference pictures. Each entry of the list indicates a specific reference frame. In other words, each index (i.e. entry number) of the list is assigned a respective reference frame. For each inter-coded frame, the bitstream includes a respective list index, thereby identifying a certain reference frame for reconstructing the inter-coded frame. The list can be defined in the standard or transmitted at the beginning of a video or a set of many frames. It is noted that in H.265, two lists of reference pictures are defined, called L0 and L1. Then, a reference picture is signaled in the bitstream by indicating the list (L0 or L1) and the index in this list associated with the desired reference picture. Providing two or more lists can have the advantage of better compression. For example, L0 can be used for both uni-directional inter prediction slices and bi-directional inter prediction slices, while L1 can only be used for bi-directional inter prediction slices. However, in general, the present application is not limited to any content of the L0 and L1 lists.

[0076] The motion vector can be directly signaled by the coordinates of the block it points to (in the reference picture). Alternatively, according to the specification in H.265, a list of candidate motion vectors can be constructed and the index of the list associated with a particular motion vector can be transmitted.

[0077] The motion vector of a current block is usually related to the motion vectors of neighboring blocks in the current picture or in an earlier coded picture. This is because neighboring blocks are likely to correspond to the same moving object with similar motion, and the motion of the object is unlikely to change suddenly over time. Therefore, using one or more motion vectors of spatial or temporal neighboring blocks to define a predicted value of the motion vector of the current block, called a motion vector predictor (MVP), reduces the transmitted motion vector difference. The MVP can be derived from the decoded motion vectors of spatial or temporal neighboring blocks in the collocated picture. In H.264 / AVC, this is done by taking the median of the components of three spatial neighboring motion vectors. Using this approach, there is no need to transmit the predicted value. Temporal MVPs from collocated pictures are only considered in the so-called H.264 / AVC temporal direct mode. H.264 / AVC direct modes are also used to derive motion data other than motion vectors. Therefore, they are more related to the block merge concept in HEVC. In HEVC, the approach of implicitly deriving the MVP has been replaced by a technique called motion vector candidate, which explicitly indicates which MVP is selected from a list of MVPs for motion vector derivation. The variable coding quadtree block structure in HEVC enables a block to have several neighboring blocks with motion vectors as potential MVP candidates. Taking the left neighbor as an example, in the worst case, when a 64x64 luma prediction block is no longer split and the luma prediction block to the left is split to the maximum depth, the 64x64 luma prediction block can have 16 4x4 luma prediction blocks to the left.

[0078] Advanced Motion Vector Prediction (AMVP) was introduced to modify the motion vector competition to account for this flexible block structure. During the development of HEVC, the original AMVP design was greatly simplified to strike a good balance between coding efficiency and implementation-friendly design. The original design of AMVP included five MVPs from three different types of predictors: three motion vectors from spatial neighboring blocks, the median of the three spatial predictors, and a scaled motion vector from a collocated temporal neighboring block. In addition, the predictor list was modified by reordering to put the most likely motion predictor first and removing redundant candidate motion vectors to ensure minimal signaling overhead. The final design of AMVP candidate list construction included the following two MVP candidates: a) up to two spatial candidate MVPs derived from five spatial neighboring blocks; b) one temporal candidate MVP derived from each of two temporal collocated blocks when the two spatial candidate MVPs are not available or are identical; c) a zero motion vector when the spatial, temporal, or both candidates are not available. Details on motion vector determination can be found in the book by V. Sze et al. (eds.), High-Efficiency Video Coding (HEVC): Algorithms and Architectures, Springer, 2014, incorporated by reference herein in its entirety, in particular in Chapter 5.

[0079] As will be described in detail below, the motion vectors derived at the encoder side and provided in the bitstream can be further refined. Thus, the motion vector estimation can be improved without further increasing the signaling overhead. The motion vector refinement can be performed at the decoder without assistance from the encoder. The decoder loop in the encoder can employ the same refinement to obtain the corresponding reference picture. The refinement can be performed by determining a template, determining a search space, and finding the position of the reference picture portion that best matches the template in the search space. The position of the best matching portion determines the best motion vector, which is then used to obtain the prediction for the current block, i.e., the current block that is being reconstructed.

[0080] In operation, the motion estimation (see Figure 1 ) is performed by circuitry of an inter prediction unit 110 that can be implemented in a video encoder 100 in Figure 3 . Similar prediction can also be performed by an inter prediction unit 210 of a video decoder 200 in Figure 2 , and Figure 3 the appended description also apply to this similar prediction.

[0081] An initial motion vector MV0, which can be considered as an estimate or approximation of the precise motion vector, is obtained by the inter prediction unit 110. For example, MV0 can be selected from a list of candidate motion vectors. The list can include motion vectors of at least one block neighboring the current block. Alternatively, MV0 can be obtained by block matching at the encoder side and transmitted to the decoder side within the bitstream. Accordingly, at the decoder side, the inter prediction unit 210 can obtain the initial motion vector MV0 from the bitstream. For example, an index of the candidate list is extracted from the bitstream and the motion vector candidate identified by the index is provided as the initial motion vector MV0 to the inter prediction unit. Alternatively, the coordinates of MV0 are directly extracted from the bitstream. However, the application is not limited to any particular way of obtaining the initial motion vector MV0. In general, MV0 can be determined in any way, for example, by template matching at the encoder and the decoder in the same way. Alternatively, the motion vector can be predicted from a function of motion vectors of neighboring blocks of the current block in spatial or temporal domain.

[0082] The initial motion vector MV0 is an initial estimate of the final motion vector MV0" used for inter prediction of the current block. It constitutes an input of a refinement process, which outputs the final motion vector MV0" at the end of the refinement process. The refinement process comprises determining a search space and selecting the final motion vector from the search space.

[0083] In general, the search space construction (e.g., performed by a search space determination unit 310 of the inter prediction unit 110 or 210) comprises two phases, in each of which a part of the search space is constructed. Then, a motion vector selection unit 340 (also part of the inter prediction unit 110 and / or 210) selects the motion vector MV0" (corresponding to the coordinates of a position of the search space) according to the matching cost. It is noted that for some candidate motion vectors of the search space, the cost can already be calculated as part of and during the search space construction for all candidate motion vectors of the respective part of the search space determined in each phase.

[0084] The search space construction comprises a first phase 301 of constructing a first (part) search space. In the positions of the first search space determined in the first phase 301, at least two candidate positions are selected 302 for determining candidate positions to be examined in a second phase 303.

[0085] In other words, based on the initial motion vector MV0, a first search space comprising a plurality of candidate motion vectors is determined 301. In the first search space, a first candidate motion vector and a second candidate motion vector are identified 302 according to a cost function. Based on the first candidate motion vector and the second candidate motion vector, a second search space comprising one or more candidate motion vectors is determined 303. From the candidate motion vectors in the first search space and the second search space, a motion vector selection unit 340 selects a motion vector MV0" for the current block. In particular, after having evaluated the second search space, the candidate motion vector that minimizes the cost function is found, which is selected as the final motion vector MV0" to be applied in inter prediction. The first search space corresponds to a first subset of positions in the reference image, i.e. a subset of positions to which the candidate motion vectors in the first search space point. Similarly, the second search space corresponds to a second subset of positions in the reference image, i.e. a subset of positions to which the candidate motion vectors in the second search space point.

[0086] The motion vector refinement is performed in a search space, which is a subset of positions in the reference image and comprises positions in said first search space and said second search space. The positions are the positions to which the respective candidate motion vectors point, i.e. the positions to be evaluated for matching the template. Integer or fractional resolution can be used for the reference image. Irrespective of the reference image resolution, the search space or parts thereof can have a lower or higher resolution of its own than the reference image. Higher resolution can be achieved by performing fractional pixel interpolation to obtain fractional pixel positions.

[0087] For example, the initial motion vector MV0 can point to an integer pixel position, also referred to as a full pixel position. Alternatively, MV0 can point to a fractional pixel position, e.g. a half pixel position or a quarter pixel position. Here and in the description in the remainder, a "half pixel position" (or "quarter pixel position") refers to a point on a line between two adjacent full pixel positions (i.e. adjacent pixels having full pixel resolution), the fractional pixel position being at a distance from the next full pixel position which is half (or quarter) of the distance between the two adjacent full pixel positions.

[0088] In different embodiments of the invention, the first search space and the second search space can have the same resolution or different resolutions, irrespective of whether MV0 points to a full pixel or a half pixel position. For example, the second search space can have a higher resolution than the first search space. Thus, the second search space can be regarded as a refinement of the first search space.

[0089] Advantageously, the first search space has full-pel resolution. Then, if the resolution of the second search space is different from the resolution of the first search space, the resolution of the second search space can be a fractional-pel resolution, e.g. a half-pel resolution. Note that the resolution of the search space can be different from the resolution of the reference picture. For example, the initial motion vector can point to a half-pel position in the reference picture. However, the first search space can only include positions that are at least one integer-pel distance apart from each other. However, these positions can all lie on sub-pel positions of the reference picture.

[0090] In Figure 4 And in the remaining figures showing different configurations of search spaces according to various embodiments of the application, full-pel positions are indicated by shaded (filled) dots, while fractional-pel positions are shown as non-shaded (empty) dots. The pixels of a picture in a coded or decoded video can be arranged in a square pattern. However, in general they can have a general rectangular pixel pattern that is not necessarily a square pattern. The present application is not generally limited to any particular pixel pattern. The pixels can also be arranged in a non-rectangular pattern.

[0091] In one implementation, the candidate motion vector for the current block points from the top-left pixel of the current block in the current picture (assumed to have coordinates (0,0)) to the corresponding top-left pixel of the candidate prediction block in the reference picture (as shown in Figure 4 Thus, the top-left pixel of the candidate prediction block represents the search space in the reference picture. In this implementation, the top-left pixel of a block is used as the position of the block. However, any other pixel of a block can be used as the position of the block. It should be understood that the same position convention applies to all blocks. For example, a motion vector can equivalently be defined as extending from the center pixel of the current block to the center pixel of the corresponding candidate block.

[0092] As an example (again referring to Figure 4 ), the first search space includes 9 candidate motion vectors, namely the initial estimate of the motion vector MV0 and the four closest neighboring positions and the four second closest neighboring positions of the full-pel resolution of the position to which MV0 points correspond. Figure 4 The constellation of the first search space in

[0093] As mentioned above, the motion vector refinement scheme can be performed in the same way at the encoder and the decoder without the need for additional control signaling. For example, this can be achieved by providing a template at the encoder side as well as at the decoder side. For example, the template can be determined from the already encoded / decoded pixels (e.g. using one of the techniques described in the above-mentioned document JVET-D0029). An example of such a template can be the pixels of one or more blocks that are decoded before the current block and that are adjacent to the current block. More specifically, assuming that the decoding of the blocks is done from left to right and from top to bottom as usual, the refinement template for the motion vector of the current block can be determined as N pixels of the adjacent block at the left boundary and M pixels of the adjacent block at the top boundary. M and N are integers greater than 1. However, the template can be determined in different ways and in addition to the pixels of the adjacent blocks that are directly adjacent to the boundaries of the current block, the template also includes other pixels of the adjacent blocks and / or the entire boundaries of one or more adjacent blocks.

[0094] In practice, the motion vector refinement is most relevant for the decoder. Since no information about the refinement of a particular motion vector is encoded in the bitstream, the refinement is only applied at the decoder side in the decoding loop in order to generate the reference pictures taking into account the refined motion vectors.

[0095] The similarity can be measured by a cost function, which for example can be the sum of absolute differences between the template and the reference picture region corresponding to the template in the position pointed to by the motion vector candidate. After computing the sum of absolute differences (SAD) for all candidate motion vectors, the candidate motion vector with the smallest SAD is selected. However, it is to be noted that the SAD is merely an example and any other similarity measure can be applied, such as the sum of squared differences or the correlation, etc.

[0096] The first and second candidate motion vectors refer to the positions of the prediction values of the current block that are most similar (and second most similar) to a predetermined template. The template can be generated in a previous step, for example using one of the techniques described in JVET-D0029.

[0097] Reference is made below to Figure 18A method of refining a motion vector is described. The method starts in S1801. In S1802, an initial estimate MV0 of the motion vector is obtained, and a first search space is set based on the initial estimate of the motion vector. The first search space comprises candidate motion vectors pointing to positions around the position associated with MV0. In S1803, costs associated with the candidate motion vectors in the first search space are computed, and based on the computed costs, a first candidate motion vector and a second candidate motion vector P1 and P2 are selected. In step S1804, a second search space comprising one or more candidate motion vectors is set based on P1 and P2. Since the second search space is set based on two most promising points, the second search space can be rather small (and thus can be searched quickly). In particular, by considering two (or more) positions, a direction of a trend in which the cost (i.e. the value of the cost function) decreases (or is likely to decrease) can be determined, and the second search space can be set in this direction of the trend, which can have a smaller size compared to setting the second search space based on the initial motion vector or based on a single best point, for example. It is also noted that, in general, the present invention is not limited to considering two best candidate motion vectors (and the respective positions they point to). In general, the trend of the cost function can be determined more precisely by considering more than two best positions. In this case, the direction in which the cost function decreases is determined based on the two or more positions among the positions in the first search space that are considered which have the lowest costs. Then, the second search space is set at a position in this direction of the trend. Thus, the number of positions in the search space, in particular the second search space, can be kept small while still checking the most promising positions.

[0098] In S1805, costs associated with the candidate motion vectors in the second search space are computed. From the candidate motion vectors in the first search space and the second search space, the best candidate motion vector, i.e. the candidate motion vector associated with the lowest cost, is selected (in S1806). After the best candidate motion vector is selected, the motion vector refinement ends (in S1807).

[0099] Different embodiments of the present invention can differ in the way the first search space and the second search space are determined, as described below.

[0100] First exemplary embodiment

[0101] According to the first exemplary embodiment (see Figure 4 ), a cost function is evaluated for each of the candidate motion vectors in the first search space. In other words, for each of these motion vectors, the respective cost, i.e. the value of the cost function, is computed or estimated or otherwise determined. Based on this evaluation, the candidate motion vector with the lowest cost and the candidate motion vector with the second lowest cost are selected. In Figure 4In the example of FIG. 4, the initial motion vector MV0 points to position 405. The position 409 in the first search space has the lowest cost and is therefore selected as the first candidate motion vector MV0'. The right neighbor 406 of MV0 has the second lowest cost and is therefore selected as the second candidate motion vector MV0'secondBest. MV0' and MV0'secondBest are used to construct the second search space. In this example, the second search space includes two additional candidate motion vectors that point to two half-pixel positions 411 and 412 (empty circles in the figure) that lie on the connecting line of positions 409 and 406 (i.e., MV0' and MV0'secondBest). In this example, the half-pixel positions 411 and 412 are the half-pixel positions above and below MV0', respectively. From the candidate motion vectors in the first search space and the candidate motion vectors in the second search space, the candidate motion vector with the lowest cost is selected, in this example, position 412, as the final motion vector MV0".

[0102] Figure 4 The example of FIG. 4 illustrates the motion vector refinement for a current block and one reference image, namely the reference image assigned with index 0 in the reference picture list L0. The drawing of the current block is only schematic and shows the positions of the search space points corresponding to the positions of the search template given by the top-left corner of the template. The present application is applicable to any size and form of the template. Advantageously, the template is a block of the size of the current block and the search for the two best candidate motion vectors is performed by template (block) matching in the first search space and the second search space. Since the current block is typically not available at the decoder, the template is constructed from the already decoded image parts. For example, in the case of bi-prediction, there are two initial motion vectors MV0 and MV1 associated with two corresponding reference images RefPict0 and RefPict1. The template block for the motion vector refinement can then be constructed by a weighted average of the two blocks pointed to by MV0 in RefPict0 and MV1 in RefPict1, respectively. Other template constructions are possible based on decoded pixels from the current image or the corresponding reference images or the already decoded nearest images.

[0103] According to the direction of the connecting line of the top end (i.e. the end point) of the first candidate motion vector and the top end of the second candidate motion vector MV0’secondBest, the size (i.e. the number of candidate motion vectors) and / or the position (i.e. the position to which the candidate motion vectors in the second search space are pointing) of the second search space is determined. In particular, the first candidate motion vector and the direction (line) given by connecting the top ends of the first and second candidate motion vectors are used to determine the number and / or the coordinates of the candidate motion vectors used in the second step. The size of the second search space can be determined according to the position to which the first candidate motion vector MV0’ is pointing. However, it is to be noted that the present application is not limited to determining the size and position of the second search space based on two best points. For example, the size (in terms of the number of positions) of the second search space can be fixed and only the position of the second search space can be determined based on the two best positions.

[0104] Figures 5 to 8 Exemplarily, the configuration of the search spaces according to the first embodiment of the present application is shown. In these examples, the size of the second search space is always 2, but its position is given by the two best points in the first search space. As can be seen from the figures, the first search space with the first (e.g. integer) pixel resolution has the “square” constellation already shown in Figure 4

[0105] Advantageously, according to the first embodiment, the first search space comprising a plurality of candidate motion vectors has an integer pixel resolution. Thus, first, the first candidate motion vector MV0’ pointing to the position with the smallest cost function after the first step and the second candidate motion vector MV0’secondBest having the second lowest cost function value are determined using an integer pixel search resolution.

[0106] Furthermore, the second search space has a fractional pixel resolution with respect to the resolution of the first search space, comprising one or more candidate motion vectors pointing to positions located in the direction given by the first candidate motion vector and the second candidate motion vector comprised in the first search space. Thus, in the second stage (more precisely, in the third stage), the candidate motion vectors of the second search space are searched using a fractional pixel search resolution. Figure 3 ​In "stage 2" 303 of the present invention), one or more, for example two half-pixel positions in the second search space (i.e., a new search window) are determined. This means that since the best candidate motion vector MV0' in the first search space is known, it is only necessary to further compare the cost of MV0' with the costs of other points in the second search space until a candidate motion vector with a lower cost than MV0' is found to finally perform motion vector selection. In this case, the second search step involving the second search space has a higher accuracy than the first search step. In other words, it may be advantageous if the second search space has a higher resolution than the first search space (i.e., a smaller distance between search space positions). In this way, the search space is refined at each stage of its construction and may include more than two such stages. For example, based on the two best points of the combined first and second search spaces, a third search space with a higher resolution than the first and second search spaces may be constructed.

[0107] exist Figures 4 to 8 In the example of , one or more half-pixel positions in the second search space are selected according to the direction of the line connecting the positions of MV0' and MV0'secondBest corresponding to the difference vector MV0'diff = (MV0' - MV0'secondBest). Therefore, the second search space is determined according to the angle between MV0'diff and the image boundary (or the horizontal pixel row in the reference image). At the end of the second search step, Figure 3 The final motion vector MV0 is determined in stage 304.

[0108] Furthermore, advantageously, at least one of the candidate motion vectors in the second search space points to a position between positions pointed to by the first candidate motion vector and the second candidate motion vector included in the first search space. It should be noted that the second search space may include a single candidate motion vector that points to a position between the first candidate motion vector and the second candidate motion vector.

[0109] The following will refer to Figures 5 to 8 、 Figure 10 and Figure 11 Other exemplary search space constellations according to the first embodiment of the present invention are described.

[0110] exist Figures 5 to 8, MV0' points to one of the nearest or second-nearest positions of the initial motion vector MV0 (i.e., the position of the center of the first search space), that is, it points to one of the positions immediately adjacent to the position of MV0. The second search space is determined to include two candidate motion vectors, the two candidate motion vectors pointing to positions located on both sides of MV0', and the two candidate motion vectors both approximately point to positions on the connecting line of the positions pointed to by MV0' and MV0'secondBest. In other words, the second search space includes a first position between MV0' and MV0'secondBest and a second position located in the connecting direction of MV0' and MV0'secondBest, the second position being located on the other side of the first position.

[0111] Here, and in the remainder of this application, a "nearest" neighbor or adjacent position refers to a position that is adjacent to a given position in the resolution of the (part of) search space under consideration. For example, even if the reference image has a fractional 1 / 4 pixel resolution, if the first search space has an integer resolution, then adjacent positions in the first search space will also have an integer pixel distance from each other. This applies even though the first search space position may be located at a fractional pixel position in the reference image.

[0112] Furthermore, the term "second closest position" refers to a position adjacent to two nearest neighbors (diagonal neighbors in a rectangularly arranged search space). However, in a general case not shown in any figure, the distance to the adjacent position in one direction (e.g., vertical direction) may be different from the distance in another direction (e.g., horizontal direction). In this case, regardless of the possible difference in distance, the term "nearest neighbor" as used herein applies to adjacent positions in both directions.

[0113] exist Figure 5 and Figure 6 In the first search space, MV0' and MV0'secondBest are the nearest neighbors of each other in the resolution (i.e., full pixel resolution). In the figure, the line connecting the positions pointed to by MV0' and MV0'secondBest is a vertical line. Figure 5 The search space constellation shown is similar to Figure 4 The search space constellation shown is the same. If MV0'secondBest is located to the left or right of MV0', rather than above or below MV0', the line connecting the positions pointed to by the first candidate motion vector and the second candidate motion vector corresponds to a horizontal line. Although not shown in the figure, the first embodiment is also applicable to the case where the first candidate motion vector and the second candidate motion vector are connected by a horizontal line.

[0114] exist Figure 7In the example shown, the second candidate motion vector MV0' secondBest points to the second nearest neighbor of the first candidate motion vector MV0'. In this case, the connecting line of the positions to which the first and second candidate vectors point is diagonal.

[0115] It is generally desirable that the similarity between the prediction of the current block and the template block increases monotonously in one direction. Therefore, as Figures 5 to 8 shown, in the resolution of the first search space, the candidate vectors MV0' and MV0' secondBest should be the nearest or second nearest neighbors. However, it can happen that there is a third candidate motion vector between the search space of MV0' and MV0' secondBest for which the value of the cost function is higher than for each of the candidate motion vectors MV0' and MV0' secondBest, as Figure 8 shown. This can happen, for example, due to noise in the encoded / decoded video. In this case, the two fractional pixel positions on the connecting line of the positions to which MV0' and MV0' secondBest point can be selected to form the second search space, which are closer to MV0' than to MV0' secondBest, but still lie in the direction given by the two best points of the first search space. However, the present invention is not limited to this procedure. For example, in order to keep the complexity low, if the cost function trend based on the two or more positions of lowest cost is not monotonic, a default second search space can be set, for example, assuming a horizontal direction. Due to the panning of the camera and the movement of typical objects in natural video, the horizontal direction can be considered as the more likely direction in natural video sequences. In other words, if there is no clear cost function trend based on the first and second best motion vector candidates in the first search space, some points around the first best candidate motion vector are preferably set as the second search space. In order to reduce the second search space, a default direction can be assumed and the corresponding default second search space can be set.

[0116] Figure 9 The procedure of determining the second search space in the second stage according to the first embodiment is shown. In particular, the pixel positions to which the candidate motion vectors in the second search space point are determined from the components of the difference vector MV0' diff = (MV0' - MV0' secondBest).

[0117] If MV0' diff has only a horizontal, i.e. non-zero, component, determining the second search space comprises two positions left and right of MV0', each at a distance to MV0' corresponding to the resolution of the second search space (e.g. half-pel resolution), as Figure 9(a) shown. In addition, if MV0'diff has only a vertical component, the second search space is determined to include two positions above and below MV0', each at a distance from MV0' corresponding to the resolution of the second search space (see Figure 9 (b)).

[0118] If MV0'diff has both a horizontal and a vertical component (with non-zero values) as shown in parts (c) and (d), the second search space is selected to be the second nearest (diagonal) neighbor position relative to the position associated with MV0' in the resolution of the second search space. If both the horizontal and the vertical component are positive or both are negative, the second nearest neighbor position is selected to be located in the upper left and lower right corner relative to MV0' as shown in Figure 9 (c). If one component is positive and the other component is negative, the second nearest neighbor position is selected to be located in the lower left and upper right corner relative to MV0' as shown in Figure 9 (d). In addition, if MV0'diff cannot be determined (e.g. due to the characteristics of the cost function, e.g. the costs of all candidate motion vectors in the first search space are identical), MV0'diff can be set to (0, 0) and can be chosen arbitrarily, e.g. in Figure 9 (a) to Figure 9 (d), the default second search space can be set. However, in this case, due to the statistical properties of typical video sequences (horizontal objects or camera movements are more likely than vertical objects or camera movements, as typically the region of interest is located in the horizontal direction), search point configuration 9(a) is preferred (over (b), (c) and (d)).

[0119] It should be noted that in Figure 9 and in the rest of this application considering coordinates, the positive direction of the horizontal axis ("x-axis") points to the right end, while the positive direction of the vertical axis ("y-axis") points to the lower end (opposite to the Cartesian convention, but commonly used in image processing).

[0120] In all search space constellations shown in Figures 5 to 8 , the first candidate motion vector MV0' points to a pixel position at the edge of the first search space. In particular, one candidate motion vector points to a position between two candidate motion vectors in the first search space. The other candidate motion vector in the second search space points to a position outside the first search space, i.e. a position not enclosed by the candidate motion vectors in the first search space.

[0121] However, as mentioned above, both the position and the size of the candidate motion vectors in the second search space, i.e. the number of candidate motion vectors, are determined depending on the direction of the connecting line of the positions to which the candidate motion vectors MV0' and MV0'secondBest point. More specifically, if the first candidate motion vector MV0' points to a position in the center of the first search space, then one candidate vector suffices for the second search space. In particular, the single candidate motion vector in the second search space then points to a position in the resolution of the second search space between the two candidate motion vectors in the first search space. However, with Figures 5 to 8 In contrast to the search space constellation shown in Fig. 1, the second candidate motion vector outside the first search space is omitted. Thus the procedure shown in Fig. 1 is modified so that only one candidate motion vector in the second search space, i.e. the search window of the second search step, is determined. Figure 9

[0122] Figure 10 and Figure 11 Examples of a second search space comprising only one motion vector candidate are shown in Figs. 2 and 3. In Fig. 2, MV0' and MV0'secondBest are the nearest neighbors (horizontally adjacent). In Fig. 3, MV0' and MV0'secondBest are the second nearest neighbors (diagonally adjacent). As shown in both figures, MV0' points to a position within the first search space. In other words, there are candidate motion vectors pointing to all pixel positions adjacent to MV0'. One of these candidate motion vectors pointing to an adjacent position is MV0'secondBest. Figure 10 Figure 11 In other words, if the second search space comprises a second position in the first search space adjacent to MV0' and located in the direction given by connecting MV0' and MV0'secondBest belonging to the first search space, then the second search space comprises only one candidate motion vector pointing to a fractional pixel position between the first candidate motion vector and the second candidate motion vector. In this case, the likelihood of finding a candidate motion vector with lower cost in this direction is very low since the cost of the second position has already been calculated and the cost of the second position is higher than the cost of MV0' and MV0'secondBest. In general, the number of positions in the search space can also depend on the likelihood of finding a candidate motion vector better (in terms of cost) than MV0'. This likelihood can be estimated by the cost function calculated for the positions of the first search space by interpolation and / or extrapolation.

[0123]

[0124] ​​​It should be noted that in the above example, the first search space has been shown to have 9 adjacent positions arranged in a square grid. However, the present invention is not limited to a specific size of the search space or a specific arrangement of pixels. Similarly, the first embodiment can be applied to a first search space and a second search space ( Figures 5 to 8 The second search space may also have more than two positions.

[0125] According to the first exemplary embodiment, Figure 18 The sub-step S1803 of calculating the cost of the candidate motion vectors in the first search space in the refinement method is as follows: Figure 19 As shown. Variables i, P1, and P2 are initialized, where i is an index variable that subsequently indicates each of the corresponding candidate motion vectors in the search space (e.g., the first search space). Variables P1 and P2 represent the corresponding candidate motion vectors with the lowest cost and the second lowest cost (i.e., the position in the search space and the cost value associated with the position). At the beginning, P1 and P2 can be initialized to values ​​that are not associated with any position, and the corresponding costs associated with P1 and P2 can be initialized to values ​​that are higher than any value that may be obtained in the cost calculation, i.e., the maximum cost representable by the variables. In the loop iteration of i, in S1902, the cost of the i-th candidate motion vector is calculated. In S1903, the cost of the i-th candidate motion vector is compared with the cost of the currently stored motion vector with the lowest cost, P1. In S1904, if the cost of the i-th candidate motion vector is lower than the stored cost of P1, P1 is set as the i-th candidate motion vector and stored. In S1905, if the cost of the i-th candidate motion vector is not lower than the cost of P1, the cost of the i-th candidate motion vector is compared with the cost of P2. In S1906, if the cost of the i-th candidate motion vector is lower than the cost of P2, P2 is set as the i-th candidate motion vector and stored. After the two comparison steps S1903 and S1905 and possibly one of the storage steps S1904 and S1906, i is incremented. If in S1908 i has not yet reached the maximum value i representing the number of candidate motion vectors in the first search space, max , the method returns to the cost calculation step S1902. If i reaches i in S1908 max , then terminate the cost calculation in S1909, Figure 18 The refinement continues.

[0126] The sub-step S1805 of calculating the cost of the candidate motion vectors in the second search space can be combined with Figure 19The steps described above are performed similarly. However, step S1905 of comparing the cost of the i-th candidate motion vector with the cost of P2 and step S1906 of storing the second candidate motion vector P2 can be omitted. This is because the result of the second search space search is the best motion vector in the first search space and the second search space. If the second search space is not further expanded, the second best motion vector is no longer used.

[0127] Second Exemplary Embodiment

[0128] According to the second exemplary embodiment, the first and second candidate motion vectors used in determining the second search space are candidate motion vectors included in the first search space, which are associated with the lowest and second lowest matching costs, respectively (as in the first exemplary embodiment).

[0129] Further, according to the second exemplary embodiment, Figure 3 The search space determination unit 310 in the operation determines the position of the second search space, which is a region. Here, the term "region" refers to a space including at least two positions pointed to by the corresponding at least two candidate motion vectors. In particular, the at least two positions are adjacent in the pixel resolution of the second search space. Figure 12 and Figure 13 As shown, the second search space may have the same resolution as the first search space. However, the resolutions of the search spaces may also be different.

[0130] Advantageously, in Figure 3 The search space determined in the first stage 301 of the search space construction includes an initial estimate MV0 of the motion vector and candidate motion vectors pointing to positions adjacent to the position pointed to by MV0 in the pixel resolution of the first search space, i.e., the nearest neighbors of the initial estimate of the motion vector. In other words, in contrast to the first embodiment in which a first search space with a "square" geometry (shape) is constructed in the first stage 301 of the search space construction, the first search space has a "cross" geometry. However, it should be noted that the first search space can have any shape as long as the encoder and decoder use the same search space. If the search space has a predetermined form, such as a cross or square geometry or any other arrangement, and the position of the initial vector MV0 only determines the position of the first search space, it is beneficial for simplification of implementation. On the other hand, the present invention is also applicable to first search spaces of different sizes (in terms of the positions pointed to by the candidate MVs) and / or shapes.

[0131] A first candidate motion vector MV0′ having the lowest cost function value and a second candidate MV0′ secondBest having the second lowest value are calculated and selected 302 .

[0132] Based on the coordinates defining the position pointed to by MV0' and the direction given by the connecting line of the positions pointed to by the first and second best candidate motion vectors MV0' and MV0' secondBest, a certain region is selected to perform the construction of the second search space in a second stage 303. Figure 3 The construction of the second search space is performed in a second stage 303.

[0133] More specifically, one candidate motion vector included in the second search space points to the position on the connecting line of the positions pointed to by the first and second best candidate motion vectors MV0' and MV0' secondBest which is nearest to the position of said MV0' in the resolution of said second search space, which candidate motion vector is not included in said first search space. One or more additional candidate motion vectors are included in the second search space which point to positions adjacent in the pixel resolution of the second search space and which are not included in the first search space.

[0134] Figure 12 An example of a search space configuration according to this second embodiment is shown in Figure 13 As an example, the pixel resolution of the second search space is the same as the pixel resolution of the first search space. As can be seen from the figure, the position pointed to by the initial candidate motion vector is surrounded by the four pixel positions which are adjacent to MV0 in the respective pixel resolution, i.e. the four nearest neighbors. MV0 and the candidate motion vectors pointing to the four positions adjacent to MV0 are included in the first search space. The first and second best candidate motion vectors MV0' and MV0' secondBest in the first search space are determined according to the cost function. The position pointed to by MV0' and the direction of the connecting line of the positions pointed to by MV0' and MV0' secondBest given by the difference vector MV0'diff are used to determine the second search space in a second stage 303. Figure 3 Here, the definition of MV0'diff is the same as described for the first exemplary embodiment. In Figure 12 Figure 13 The second search space includes candidate motion vectors pointing to positions on the connecting line of the positions pointed to by MV0' and MV0' secondBest given by (MV0' + MV0'diff) and positions adjacent to this position, i.e. nearest neighbors, which are not pointed to by the candidate motion vectors of the first search space.

[0135] In Figure 12 MV0' and MV0' secondBest are not adjacent in the resolution of the first search space, they are the second nearest neighbors. The second search space includes vectors, i.e. vectors pointing to said position on said line and the four nearest neighbors of said position. ​

[0136] In Figure 13 case, the position in the second search space on the connecting line of the positions pointed to by (MV0' + MV0'diff) and MV0'secondBest is the nearest neighbor of MV0'. The second search space then comprises the vector pointing to the position corresponding to (MV0' + MV0'diff) and the three nearest neighbors of (MV0' + MV0'diff) not equal to MV0'. Thus, the second search space comprises four candidate motion vectors.

[0137] However, if MV0' and MV0'secondBest are neither the nearest neighbor nor the second nearest neighbor in the pixel resolution of the first search space, i.e. if there is one pixel position in the first search space between the pixel positions pointed to by MV0' and MV0'secondBest, respectively, the same second search space / window can be determined as in Figure 13 the case shown.

[0138] If the search coordinates indicated by the second search space are already included in the first search space, the second search operation is not performed (terminated). This can be particularly true if the matching template and / or the cost function used in the first stage and the second stage are identical. As a further alternative, the second search operation can be performed if the matching template and / or the cost function used for the first search step and the second search step are different. It is noted that the present application is concerned with reducing the search space and, in particular, achieving this reduction by setting the second search space based on features developed from the cost function. Any template is suitable for the present application and the template can be the same or different for the respective partial search spaces, e.g. the first search space and the second search space or other search spaces, if the search space determination is cascaded in more than two stages.

[0139] According to a second exemplary embodiment, the sub-step S1803 of calculating the cost of the candidate motion vectors in the first search space (sub-step S1805 of calculating the cost of the candidate motion vectors in the second search space) in the motion vector refinement shown in Figure 18 may be performed analogously to the calculation according to the first embodiment described above in connection with Figure 19 .

[0140] A third exemplary embodiment

[0141] In the embodiments described so far, Figure 3The illustrated search space determination unit 310 identifies a first candidate motion vector and a second candidate motion vector from the first search space, which are the candidate motion vectors with the lowest and second lowest matching costs in the first search space.

[0142] According to a third exemplary embodiment of the present invention, for selection 302 of the first candidate motion vector and the second candidate motion vector, the template matching cost is calculated for four candidate motion vectors pointing to positions around the starting position pointed to by the initial estimated value MV0 of the motion vector. In particular, in order to determine the second search space, the matching cost of the pixel positions adjacent to the position pointed to by the estimated value MV0 of the motion vector in the pixel resolution of the first search space is evaluated. It is determined that a certain pixel position is pointed to by the first candidate motion vector of the second search space, and the pixel position is adjacent to the position pointed to by the first candidate motion vector and the second candidate motion vector in the pixel resolution of the first search space, and is different from the position pointed to by the estimated value of the motion vector. The first candidate motion vector points to a quadrant, and the matching cost of the quadrant is expected to be reduced, such as Figure 14 shown.

[0143] exist Figure 15 , a search space determination unit 1510 is shown, which is Figure 3 The more general modified form of the search space determination unit 310 is shown. Based on the initial estimate value MV0 of the motion vector, Figure 15 In the search space construction 1501 in stage 1, it is determined that Figure 14 The first search space, exemplarily shown in (a), has coordinates of an initial estimate of a motion vector denoted as MV0_x and MV0_y. The first search space includes MV0 and candidate motion vectors pointing to pixel locations surrounding the location corresponding to MV0, such as the nearest neighbors of MV0 at the pixel resolution of the first search space. Matching costs are calculated for the candidate motion vectors in the first search space. By selecting 1501 the first candidate motion vector and the second candidate motion vector, two directions along preferred orthogonal directions (e.g., vertical and horizontal) where matching costs are expected to be reduced are calculated.

[0144] exist Figure 14 As well as in the remainder of this application regarding coordinates, the positive direction of the horizontal axis ("x-axis") points to the right end, while the positive direction of the vertical axis ("y-axis") points to the bottom end.

[0145] More precisely, two comparisons 15021 and 15022 are made for which the points neighboring the pixel position corresponding to MV0 are divided into two groups. The matching costs of two pixel positions neighboring the pixel position pointed by the initial candidate motion vector in the pixel resolution of the first search space and having the same vertical component as the initial candidate motion vector are compared. Among the two comparison vectors evaluated in this first comparison 15021, the vector with the lower matching cost is selected as the first candidate motion vector in the first search space.

[0146] Additionally, the matching costs of two pixel positions neighboring the pixel position pointed by the initial candidate motion vector in the pixel resolution of the first search space and having the same horizontal component as the initial candidate motion vector are compared. Among the two comparison vectors evaluated in this second comparison 15022, the vector with the lower matching cost is selected as the second candidate motion vector in the first search space.

[0147] As a result of these two comparisons, one pixel position is determined to be pointed by the first candidate motion vector in the second search space, this pixel position having the same vertical component as the first candidate motion vector and the same horizontal component as the second candidate motion vector. The first candidate motion vector and the second candidate motion vector define respectively a positive or a negative half-plane in the vertical and horizontal directions. Their overlapping quadrant is chosen as the area where the matching cost is expected to decrease and defines the second space. In Figure 14 In (b), the second search space comprises only one point.

[0148] The first search space can comprise the initial estimate of the motion vector MV0 and its nearest neighbors, i.e. the candidate motion vectors pointing to the pixel positions neighboring MV0 in the resolution of the first search space. Figure 14 Such a search space configuration with a "cross" geometry is illustrated in (a) and is also described for the second embodiment. The matching costs of these five candidate motion vectors of the search space are computed according to the cost function used.

[0149] In the following, without loss of generality, it is assumed that the pixel position pointed by the initial estimate of the motion vector point MV0 has coordinates (0, 0). The two comparisons described above for determining the coordinates (abscissa, ordinate) of the first motion vector in the second search space can then be performed for example according to the following procedure.

[0150] Ordinate = -1, Abscissa = -1;

[0151] If (cost of candidate motion vector (0, 1) < cost of candidate motion vector (0, -1))

[0152] Ordinate = 1;

[0153] if (cost of candidate motion vector (1, 0) < cost of candidate motion vector (-1, 0))

[0154] x-coordinate = 1;

[0155] By determining the motion vector (x-coordinate, y-coordinate) based on the procedure given above or a similar procedure, a quadrant can be selected in which the matching cost is expected to decrease.

[0156] Therefore, the quadrant to be used in determining the second search space (303) in Figure 3 includes candidate motion vectors having coordinates (x-coordinate * x, y-coordinate * y), where x, y > 0, and the "x-coordinate" and "y-coordinate" have the values determined by the procedure described above. The determination of this quadrant is exemplarily shown in (b). In the example shown, the motion vector (1, -1) is determined to define the selected quadrant (i.e. the upper right quadrant). The potential motion vectors in the other three quadrants that would not be included in the second search space in this specific example are shown as smaller dots. Figure 14

[0157] The present application is not limited to the explicit definition of the procedure described above. For example, instead of (-1, -1), (1, 1) can be used as initial values, or instead of setting the initial coordinates, the "else" clause (comparison if-else clauses 15021 and 15022 in (c)) can be used, and the order of the "if" conditions can be exchanged. Figure 15

[0158] When the first candidate motion vector in the second search space has been determined as described above, its matching cost is calculated. As a specific case, the second search space can include only one said first candidate motion vector. At this time, among the candidate motion vectors under examination (in the example described, five candidate motion vectors in the first search space and one candidate motion vector in the second search space), the candidate motion vector with the lowest matching cost is selected as the motion vector to be used for the current block.

[0159] However, one candidate motion vector can be determined as the other at least one candidate motion vector in the second search space, which points to a position in the resolution of the second search space. The at least one other candidate motion vector points to a position in the resolution of the second search space, which lies approximately on a connecting line of the estimated value of the motion vector and the position to which said candidate motion vector of the second search space points. The pixel resolution of the second search space can be higher than the pixel resolution of the first search space. Said other candidate motion vector of the second search space can point to a position that lies between said first candidate motion vector of the second search space and the position to which the estimated value of the motion vector points.

[0160] ​​In particular, after determining the first candidate motion vector, the second search space 1503 can be determined in a second stage of determining the second search space, and then the second search space comprises at least one additional candidate motion vector pointing to a position in the quadrant defined by the first candidate motion vector of the second search space. For example, among the candidate motion vectors examined so far, the two candidate motion vectors with the smallest and second smallest matching costs can be identified and used to calculate the direction for determining the other point to which the candidate motion vector of the second search space points.

[0161] An exemplary procedure for calculating the direction based on the candidate motion vectors with the lowest and second lowest matching costs will be given below, where the coordinates of the candidate motion vectors with the lowest and second lowest matching costs are denoted as (P_min_x, P_min_y) and (P_second_x, P_second_y), and the variables "direction_vertical" and "direction_horizontal" represent the components of the vector defining the direction.

[0162] direction_vertical = 0, direction_horizontal = 0;

[0163] if (P_min_x!= P_second_x)

[0164] direction_horizontal = 1;

[0165] if (P_min_y!= P_second_y)

[0166] direction_vertical = 1;

[0167] As Figure 16As shown, the new candidate motion vector of the second search space can be selected based on the direction given by the vector (direction_vertical, direction_horizontal) and the coordinates of the candidate motion vector with the lowest matching cost (P_min_x, P_min_y) as 0.5x(P_min_x+direction_vertical, P_min_y-direction_horizontal) and 0.5x(P_min_x- vertical, P_min_y+horizontal) or one of these two points, depending on the pixel positions pointed to by the first and second candidate motion vectors in the first search space. The matching cost of the new candidate motion vector of the second search space is calculated and the candidate motion vector with the lowest matching cost is selected from the candidate motion vectors in the first search space and the second search space as the output of the motion vector refinement process, i.e. Figure 3 MV0” in

[0168] In the example shown in Figure 16 , the second search space has a fractional pixel resolution, in particular a half-pixel resolution (according to the coefficient 0.5 of the vector specifying the direction for the position of the additional candidate motion vector of the second search space). Alternative pixel resolutions such as a quarter-pixel resolution can be used and instead of one or two motion vector candidates, two or four candidate motion vectors can be used which approximate to point to the line given by the candidate motion vector (P_min_x, P_min_y) and the direction (direction_vertical, direction_horizontal).

[0169] In the example search space configuration shown in Figure 16 , the first motion vector of the second search space coincides with the candidate motion vector (P_min_x, P_min_y) with the lowest matching cost, wherein the calculation of the additional motion vectors 0.5x(P_min_x+direction_vertical, P_min_y-direction_horizontal) and 0.5x(P_min_x- vertical, P_min_y+horizontal) of the second search space is based on this candidate motion vector (P_min_x, P_min_y).

[0170] Combinations of embodiments

[0171] According to each of the exemplary embodiments described above, the second search space is selected based on the output of the first step, wherein in the first step, the first search space is determined 301 and the first candidate motion vector and the second candidate motion vector are selected 302 from the first search space. However, the entire search process can be divided into more steps than determining the first search space and the second search space and selecting one or two candidate motion vectors from the respective search spaces. In each step or sub-step, a new search space can be determined according to one of the exemplary embodiments described. For example, the above combined Figure 16 The search space configuration described is an example, in which determining the second search space means subsequently applying the third exemplary embodiment and the first exemplary embodiment. This will be explained below.

[0172] exist Figure 16 In the exemplary search space configuration shown, the second search space includes a first candidate motion vector in the second search space and two additional candidate motion vectors. The two additional candidate motion vectors in the second search space approximately point to pixel positions on a line, where the line is defined by the pixel position candidate motion vector with the lowest matching cost among the five candidate motion vectors in the first search space and the second search space and a line with a direction given by a vector, where the coordinates of the direction are direction_horizontal and direction_vertical calculated according to the process given above. This process is an example of calculating two candidate motion vectors according to the direction given by the line connecting the positions pointed to by the two candidate motion vectors. This process can also be used for the process already referred to above Figures 5 to 11 The calculation of the second search space of the first embodiment is described.

[0173] In other words, the optional second stage of determining the second search space according to the third embodiment corresponds to the determination of the second search space according to the first embodiment. In other words, in the above-described example of determining a second search space having more candidate motion vectors than a second search space having only the first candidate motion vector, additional motion vectors of the search space have been obtained by combining the third embodiment of the present invention with the first embodiment.

[0174] In the reference Figure 16 In the example of the third exemplary embodiment described, the third embodiment and the first embodiment are combined when determining the second search space. However, the present invention is not limited to this specific example of the combination of different embodiments.

[0175] Furthermore, the present invention is not limited to combining two different embodiments. Alternatively, the second search space can be determined by subsequently applying the second stage 303 according to the first embodiment and thereby determining a nested search space with different pixel resolutions, wherein the nested search space with different pixel resolutions is, for example, a first search space with half-pixel resolution and a second search space with quarter-pixel resolution.

[0176] Figure 17 Another example of combining different exemplary embodiments is shown. Figure 17 As can be seen from (a), according to the second exemplary embodiment, a first search space including five candidate motion vectors is determined, wherein the five candidate motion vectors are the initial estimated motion vector and four candidate motion vectors adjacent to the initial estimated motion vector at the pixel resolution (e.g., full pixel resolution) of the first search space. The candidate motion vectors with the lowest matching cost and the second lowest matching cost are used to determine a second search space, which is an area according to the second embodiment, which includes Figure 17 (b) shows the other five candidate motion vectors. On these other five candidate motion vectors, the method of the third embodiment is applied, that is, by applying Figure 15 1502, determining the selection of the first candidate motion vector and the second candidate motion vector MV0'first and MV0'second Figure 17 (c) shows the additional candidate motion vectors. Again according to the first embodiment, the two candidate motion vectors with the lowest and second lowest matching costs are determined (denoted as MV0'c and MV0'secondBest_c in the figure). Figure 17 As shown in (d), two additional candidate motion vectors are added pointing to pixel positions at a higher pixel resolution than used so far (e.g., half-pixel resolution), which roughly point to positions on the connecting line corresponding to the positions corresponding to MV0'c and MV0'secondBest_c.

[0177] An advantage of combining different embodiments is that the number of candidate motion vectors can be kept small while maintaining similar accuracy in increasing areas of the reference image. Figure 17 As shown, the cascade of three stages corresponding to the three embodiments allows providing the location of the prediction value in exact half-pixel resolution for an area corresponding to a square of 7×7 full pixels.

[0178] Motion vector determination including motion vector refinement as described above can be implemented as part of encoding and / or decoding of video signals (moving pictures). However, motion vector determination can also be used for other purposes in image processing, such as motion detection, motion analysis, etc.

[0179] The motion vector determination can be implemented as an apparatus. Such an apparatus can be a combination of software and hardware. For example, the motion vector determination can be performed by a general purpose processor, a digital signal processor (DSP), a field programmable gate array (FPGA), etc. However, the present application is not limited to an implementation on programmable hardware. It can also be implemented on an application-specific integrated circuit (ASIC) or by a combination of the above mentioned hardware components.

[0180] The motion vector determination can also be implemented by program instructions stored on a computer readable medium. The program, when executed, causes a computer to perform the following steps: obtaining an estimate of the motion vector; determining a first search space comprising a plurality of candidate motion vectors based on the estimate; selecting a first candidate motion vector and a second candidate motion vector in the first search space according to a cost function; determining a second search space comprising one or more candidate motion vectors based on the first candidate motion vector and the second candidate motion vector; selecting the motion vector of the current block from the candidate motion vectors in the first and second search spaces. The computer readable medium can be any medium storing the program, such as a DVD, a CD, a USB (flash) drive, a hard disk, a server storage available over a network, etc.

[0181] The encoder and / or decoder can be implemented in various devices, including a television set, a set-top box, a PC, a tablet, a smartphone, etc. It can be a software, an application implementing the above described method steps.

[0182] In summary, the present application relates to the construction of a search space for determining a motion vector of a current block of a picture in a video sequence. The search space construction is divided into two main stages, wherein a first part search space and a second part search space are determined, respectively. The first search space is constructed based on an initial estimate of the motion vector. A first candidate motion vector and a second candidate motion vector in the first search space are identified according to a cost function. The second search space is constructed based on the first and second candidate motion vectors. The motion vector of the current block is selected from the candidate motion vectors in the first and second search spaces.

[0183] Although the present application is mainly described above in the framework of motion image video coding, the proposed techniques can also be applied to the coding (i.e. encoding or decoding) of any image set comprising two or more images. The image set can comprise for example a set of still images obtained in a medical imaging application, e.g. a sequence of computed tomography (CT) scan images. Thus, in the appended claims, the term "video" can denote a sequence of motion images or any other image set comprising two or more images.

Claims

1. An apparatus for determining a motion vector, the apparatus comprising: The motion vector is to be used for inter prediction of a current block of a video frame, and the apparatus comprises: a search space determination unit configured to obtain an estimated value of the motion vector, determine a first search space comprising a plurality of candidate motion vectors based on the estimated value, the first search space having integer pixel resolution; the search space determination unit is further configured to calculate a cost value for each of the plurality of candidate motion vectors based on a cost function, select a first candidate motion vector (MV0') and a second candidate motion vector (MV0' secondBest) in the first search space according to the cost function, the first candidate motion vector being the candidate motion vector with the minimum cost value among the plurality of candidate motion vectors, the second candidate motion vector being the candidate motion vector with the second minimum cost value among the plurality of candidate motion vectors; the search space determination unit is also configured to determine a second search space comprising one or more candidate motion vectors based on the first candidate motion vector and the second candidate motion vector, the second search space having fractional pixel resolution; wherein the determining the second search space comprising one or more candidate motion vectors based on the first candidate motion vector and the second candidate motion vector comprises determining a size and / or a position of the second search space according to a direction of a connecting line of positions pointed to by the first candidate motion vector and the second candidate motion vector; a motion vector selection unit configured to select, from the at least one candidate motion vector contained in the second search space and the plurality of candidate motion vectors contained in the first search space, the candidate motion vector with the minimum cost value as the motion vector of the current block; wherein the cost value of each candidate motion vector contained in the second search space is calculated based on the cost function.

2. The apparatus of claim 1, wherein the search space determination unit determines the second search space to comprise: one or more candidate motion vectors contained in the second search space pointing to positions located in a direction of a connecting line of positions pointed to by the first candidate motion vector and the second candidate motion vector (MV0', MV0' secondBest).

3. The apparatus of claim 2, wherein, at least one of the candidate motion vectors in the second search space points to a position between the positions pointed to by the first candidate motion vector and the second candidate motion vector (MV0', MV0' secondBest).

4. The apparatus of claim 1, wherein, the search space determination unit determines the first search space to comprise: the estimated value of the motion vector and a candidate motion vector pointing to a position adjacent to the position pointed to by the estimated value of the motion vector in the pixel resolution of the first search space.

5. The apparatus of claim 4, wherein, the search space determination unit is configured to: selecting a candidate motion vector as a first candidate motion vector in a second search space, wherein the first candidate motion vector in the second search space points to a position adjacent to positions pointed to by the identified first candidate motion vector and the second candidate motion vector in the first search space and different from a position pointed to by the estimated value of the motion vector in a pixel resolution of the first search space.

6. The device of any one of claims 1 to 5, wherein, Further comprising: a motion vector determination unit configured to determine the estimated value of the motion vector from a motion vector list comprising motion vectors of at least one block adjacent to the current block.

7. A method for determining a motion vector, characterized by, The motion vector is to be used for inter prediction of a current block of a video frame, the method comprising: obtaining an estimated value of the motion vector; determining a first search space comprising a plurality of candidate motion vectors based on the estimated value, the first search space having an integer pixel resolution; calculating a cost value for each candidate motion vector of the plurality of candidate motion vectors based on a cost function; selecting a first candidate motion vector (MV0’) and a second candidate motion vector (MV0’secondBest) in the first search space according to the cost function; the first candidate motion vector being a candidate motion vector of the plurality of candidate motion vectors having a minimum cost value, the second candidate motion vector being a candidate motion vector of the plurality of candidate motion vectors having a second minimum cost value; determining a second search space comprising one or more candidate motion vectors based on the first candidate motion vector and the second candidate motion vector, the second search space having a fractional pixel resolution; wherein the determining the second search space comprising one or more candidate motion vectors based on the first candidate motion vector and the second candidate motion vector comprises determining a size and / or a position of the second search space according to a direction of a connecting line of positions pointed to by the first candidate motion vector and the second candidate motion vector; selecting, as the motion vector of the current block, a candidate motion vector having a minimum cost value from at least one candidate motion vector contained in the second search space and the plurality of candidate motion vectors contained in the first search space; wherein the cost value of each candidate motion vector contained in the second search space is calculated based on the cost function.

Images