Memory access window limited for elaboration of motion vector

By performing motion vector refinement on-chip with a limited memory access window, the method addresses memory access and computational complexity issues in video encoding and decoding, improving processing speed and efficiency.

JP2024161026AInactive Publication Date: 2024-11-15HUAWEI TECH CO LTD

Patent Information

Application Number
JP2024135732
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2017-10-09
Filing Date
2024-08-15
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Current video encoding and decoding methods face challenges in reducing memory access and computational complexity during motion vector refinement, particularly due to the need for external memory access and increased processing time.

Method used

Perform motion vector refinement on-chip by limiting memory access through a defined window that restricts the number of samples required for template matching, using interpolation filtering to obtain fractional positions without additional integer samples.

Benefits of technology

Reduces the number of external memory accesses and computational complexity, enhancing processing speed and efficiency in video encoding and decoding.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024161026000001_ABST
    Figure 2024161026000001_ABST
Patent Text Reader

Abstract

To provide an invention that is related to elaboration of a motion vector.SOLUTION: In a first step, an initial motion vector is acquired. Sequentially, elaboration of the initial motion vector is determined by a bilateral matching in a searching space. The searching space is positioned at a position applied by the initial motion vector, and contains one or more fraction sample positions. Each of fraction sample positions belonged to the searching space can be obtained by an interpolation filter using a filter of a tap size that only evaluates an integer number sample in a window and is previously defined. The window is formed by the integer sample that can be accessed for the bilateral matching in the searching space.SELECTED DRAWING: Figure 14
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a method for determining motion vectors that may be used during video encoding and decoding. Concerning elaboration. [Background technology]

[0002] Current hybrid video codecs use predictive coding. The picture is subdivided into blocks of pixels, which are then encoded. Instead of coding the block pixel by pixel, the entire block is coded in the spatial The prediction is made using previously coded pixels that are either local or temporally nearby. Only the differences between a block and its prediction are further processed. This involves converting the block pixels into coefficients in the transform domain. The coefficients are further compressed by quantization and then entropy coded to form a matrix. The bitstream may be further compacted by the decoder. It further includes any signaling information that allows the user to decode the video. Nulling is done based on the input picture size, frame rate and quantization step indication. , and may include settings related to encoder settings such as prediction applied to blocks of a picture. stomach.

[0003] Temporal prediction exploits the temporal correlation between pictures, also called frames, of a video. Temporal prediction, also called inter-prediction, is a method to predict the time between different video frames. This is because it is a prediction that uses dependencies on the current block. Therefore, it is also called the current block. The block being coded is based on one or more previously coded pictures, called reference pictures. The reference picture is the picture that is closest to the current block in display order of the video sequence. The picture that is the current picture is not necessarily a picture that is ahead of the current picture. A picture may be coded in a coding order different from that of the reference block as a prediction of the current block. A co-located block in the reference picture may be determined. It is a block in a picture that is at the same position as the current block in the current picture. Such prediction is performed in the region of a picture without motion, i.e., the region of a picture without motion is predicted. It is accurate in terms of area.

[0004] To obtain a predictor that takes motion into account, i.e., a motion-compensated predictor, ,When determining the prediction of the current block, motion estimation is generally used. The current block is located at a distance given by the motion vector from the position of the co-located block. The decoder uses the same To enable prediction to be determined, motion vectors are signaled in the bitstream. By signaling the motion vector for each of the blocks, To further reduce the signaling overhead caused by the motion vector The motion vector estimation may be performed in the spatial and / or temporal domain. may be performed based on the motion vectors of neighboring blocks in

[0005] The prediction of the current block is made by weighting predictions obtained from two or more reference pictures. The reference picture may be calculated using one reference picture by The picture, i.e. the picture immediately before and / or after the current picture in display order. Since the picture is most likely similar to the current picture, it is an adjacent picture. However, in general, a reference picture may be before or after the current picture in display order. and any other picture before the current picture in the bitstream (decoding order). This may be done, for example, to account for occlusions and / or non-linear motion in the video content. Therefore, the identification of the reference picture may also be stored in the bitstream. It may be signaled within the stream.

[0006] A special mode of inter prediction is where two reference pictures generate the prediction of the current block. In particular, the bidirectional prediction is used to predict the difference between the two reference pictures. The two predictions determined for each block are combined into a prediction signal for the current block. The prediction is more accurate in terms of the current block than unidirectional prediction, i.e., prediction that uses only a single reference picture. A more accurate prediction may result in a more accurate prediction of the current block and the prediction This leads to smaller differences (also called "residuals") between the pixels of It can be efficiently coded, i.e. compressed into a shorter bitstream. Therefore, more than two reference pictures are used to predict the current block. It may be used to find many reference blocks, i.e., multi-reference inter prediction Therefore, the term multi-reference prediction includes bi-directional prediction and more than two The prediction includes prediction using the reference picture.

[0007] To provide more accurate motion estimation, the resolution of the reference picture is adjusted based on the sampling The pixel resolution may be enhanced by interpolating the pixel resolution of the nearest pixel. This can be done by weighted averaging. For half pixel resolution, for example, it is roughly bilinear Interpolation is used. Other fractional pixels are predicted with their nearest neighbors. as the average of the nearest pixels weighted by the inverse of the distance between the pixel It is calculated.

[0008] The motion vector estimation is performed by using a candidate motion vector in the current block and the reference picture. The similarity between the corresponding predicted block and the corresponding block is calculated, which is a computationally complex task. In general, the search area contains M×M samples of the image, and the samples among the M×M candidate locations are Each of the positions in the search area is tested. The test is performed using the N × N reference block C and the test area The method involves the computation of a measure of similarity between the block R at the candidate location to be The sum of absolute differences (SAD) is a measure frequently used for this purpose.

number

[0009] In the above equation, x and y define candidate positions within the search region, while index i and j represent samples in the reference block C and the candidate block R. The candidate positions are often If the displacement or offset of a block is larger than the displacement of the block, the block matching is performed. Shift the reference block in the search area to make the area between the reference block C and the overlapping part of the search area To reduce complexity, we usually use The number of candidate motion vectors is reduced by restricting the candidate motion vectors to a specific search space. The search space is then reduced by, for example, a reference block corresponding to the position of the current block in the current image. It may be defined by the number and / or location of pixels around the location in the picture. After computing the SAD for all ×M candidate locations x and y, the best matching block R is The block whose position yields the lowest SAD corresponding to the highest similarity with the reference block C. On the other hand, the candidate motion vector is a candidate formed by the motion vectors of the neighboring blocks. It may be defined by a list of complementary motion vectors.

[0010] Typically, motion vectors are at least partially determined at the encoder side and are then used to generate the encoded video. However, the motion vectors are signaled to the decoder in the bit stream. In such a case, the current block may be derived in the decoder. The motion vector is not available in the reference picture, and the similarity between the block and the candidate motion vector is calculated. Therefore, instead of the current block, the already restored block cannot be used to A template constructed from the pixels of the coded block is used. For example, The already decoded pixels adjacent to the block may be used. This has the advantage of reducing signaling, i.e. the motion vectors are encoded The eigenvalues ​​are derived in the same way in both the decoder and the decoder, so signaling is However, it is not required that such motion estimation be less accurate.

[0011] To achieve a compromise between accuracy and signaling overhead, The estimation of the motion vectors consists of two steps: motion vector derivation and motion vector refinement. For example, the derivation of a motion vector may be divided into Such a selected motion vector may include, for example, selecting a motion vector within the search space. The search within the search area may be further refined by searching for each candidate motion vector. For each candidate position of the block pointed to by the candidate motion vector, the cost function It is based on calculating

[0012] Document JVET-D0029: Decoder-Side Motion Vector Refinement Based on Bilateral Templ ate Matching, X. Chen, J. An, J. Zheng (reference available at http: / / phenix.it-sudparis.eu / jvet / site) is generated by the first motion vector of integer pixel resolution. The first motion vector is found by searching with half-pixel resolution in a search space around the first motion vector. 4 shows the refinement of the motion vectors, which are further refined by

[0013] In order to perform motion vector refinement, at least the current block must be The samples required to perform the search are the samples corresponding to the search space and the samples in the search space. It stores samples in memory that can be accessed when template matching is performed. It is necessary to remember.

[0014] External memory access is dependent on current hardware architecture and / or software This is an important design parameter for the implementation of the software. This is due to the fact that the internal memory on the chip is slower than the For example, chip size implementations are limiting. Summary of the Invention [Means for solving the problem]

[0015] This disclosure provides an on-chip approach to motion vector refinement when implemented in combination with fractional interpolation. This may require a further increase in the size of the underlying memory or even external memory access. This is based on the observation that either option is potentially undesirable.

[0016] In view of the above problems, the present disclosure provides a method for reducing the number of accesses to an external memory and the number of coding blocks. The motion vectors for the blocks need to be accessible for the refinement of the motion vectors. The present invention provides a prediction of the motion vectors that allows a sufficient number of samples to be taken into account.

[0017] This reduces the number of samples required for template matching to integer samples. The number of samples is limited to a certain number of samples, and the interpolation filter can be used without requiring additional integer samples. This is achieved by only enabling fractional positions that are available via

[0018] According to an aspect of the present invention, there is provided an apparatus for determining a motion vector for a prediction block. , to obtain an initial motion vector and template for the prediction block, and The initial motion vector is refined by template matching with the template. The apparatus includes a processing circuit configured to determine the search space. The search space is given by a vector, contains one or more fractional sample positions, and Each fractional sample position between the first and second samples evaluates only integer samples in the window. The filter is obtained by interpolation filtering using a filter with a defined tap size. A window is an integer number accessible for template matching in the search space. Formed by the sample.

[0019] One advantage of such motion vector determination is that it does not require extensive reference to external memory. Simultaneously limit the number of accesses to memory / storage / cache that stores the entire picture The prediction block can be accessed to perform motion vector refinement on the prediction block while A limited number of samples that need to be collected.

[0020] In the example, the window is N integer samples away from the initial motion vector of the prediction block. is defined as M integer sample rows and N columns, where N and M are nonzero integer values. Such a definition determines which samples are taken for the purpose of determining and / or refining motion vectors. It may also provide a simple means for specifying what should be , may be easily configurable within the bitstream or standard.

[0021] In one embodiment, the processing circuitry selects more of the search space in the most recent iteration. The search space is iteratively expanded in a direction given by one of the best matching positions. Refining the initial motion vector by template matching with the template in The window is defined by a predefined maximum number of iterations. will be done.

[0022] The search space is accessed for the interpolation filtering of each fractional sample in the subwindow. All integer samples to be processed are pre-processed with respect to an interpolation filter with a predefined tap size. Within the window may be any rectangular subwindow of the window.

[0023] The search space may include a rectangular search sub-window of the window, and the initial motion vectors The refinement of the algorithm is performed by interpolating the fractional samples in the search subwindow for filtering. The integer samples accessed are then compared to the above-mentioned filter with respect to an interpolation filter of a predefined tap size. A search subwindow of a rectangle such as that in the template window is It is determined by rate matching.

[0024] In one implementation, the processing circuitry selects more of the most recent iterations of the search space. In the search space that is iteratively expanded in the direction given by one of the matching positions The refinement of the initial motion vector is determined by template matching with the template The iteration may be configured to determine at least one of the search spaces of the most recent iteration. The search is terminated when the sample is outside the search subwindow.

[0025] In particular, as a specific example, the interpolation filter may be configured to: When the pixel is on a horizontal or vertical line, K pixels are It is a one-dimensional filter that also evaluates integer samples of

[0026] Furthermore, for example, the search space is - adjacent to the top or bottom of the subwindow and on a horizontal line of integer samples or - On a vertical line of integer samples adjacent to the left or right side of the subwindow Is it in , and further includes any fractional positions that are outside the subwindow.

[0027] According to another aspect of the present invention, a video image divided into prediction blocks is fed to a bitstream. A coding device for coding the above-mentioned prediction block is provided, the coding device being and a device for determining a motion vector for a prediction block based on the determined motion vector. In order to encode the difference between the predictor given by the predicted block at the position and generating a bitstream including the coded difference and the initial motion vector. and a decoding circuit.

[0028] According to another aspect of the present invention, a video image divided into prediction blocks is fed to a bitstream. A decoding device is provided for decoding from an initial motion vector, a prediction The block is given by the prediction block at the position specified by the refined motion vector. The coded difference between the obtained predictor and the bitstream is analyzed. an analysis unit for determining a refined motion vector for said predicted block; The device, the analyzed difference, and the predicted block are placed at the position specified by the refined motion vector. A reconstruction method for reconstructing a predicted block as a function of a predictor given by the block. The function may be or include a summation. The function may be a cutout, a circle, The method may further include scaling, subtraction, or further operations.

[0029] According to another aspect of the invention, a method for determining a motion vector for a predictive block is provided. obtaining an initial motion vector and a template for a prediction block; In the search space, an initial motion vector is obtained by template matching with the template. determining a refinement of the motion vector, the search space being given by an initial motion vector; The fractional sample locations are located at positions that can be obtained by the search, and contain one or more fractional sample locations. Each tap position has a predefined tap size that evaluates only integer samples within the window. the window is obtained by interpolation filtering using a filter of the same size, Formed by integer samples accessible for template matching in space A method is provided in which

[0030] For example, the window may be a sequence of N integer samples relative to the initial motion vector of the predicted block. and M integer sample rows, where N and M are non-zero integer values.

[0031] In an embodiment, the refinement of the initial motion vector is performed by Iteratively extend in the direction given by one of the more best matching positions determined by template matching with said template in a search space; A window is defined by a predefined maximum number of iterations.

[0032] In an exemplary implementation, the search space is determined by the interpolation filter for each fractional sample in the subwindow. All integer samples accessed for tapping are of a predefined tap size. A rectangular subwindow of the window that is within said window with respect to the interpolation filter. Includes.

[0033] The search space may include a rectangular search sub-window of the window, and the initial motion vectors The refinement of the algorithm is performed by interpolating the fractional samples in the search subwindow for filtering. The integer samples accessed are then compared to the above-mentioned filter with respect to an interpolation filter of a predefined tap size. A search subwindow of a rectangle such as that in the template window is It is determined by rate matching.

[0034] In one implementation, the refinement of the initial motion vectors is performed by iterating over the search space in the most recent iteration. is iteratively extended in the direction given by one of the more best matching positions of The template may be determined by template matching with the template in a search space. Often, an iteration is performed such that at least one sample in the search space of the most recent iteration is a search subspace. It is terminated when the program is outside the window.

[0035] Further, for example, the interpolation filter may be configured to interpolate each horizontal sample whose fractional position is an integer sample. When on a line or vertical line, K horizontal or vertical integers It is a one-dimensional filter that evaluates several samples.

[0036] Advantageously, the search space is contiguous above or below the subwindow and is an integer number of samples long. On a horizontal line or adjacent to the left or right side of the subwindow, A fractional position that is either on a vertical line of samples or outside the subwindow. Further includes:

[0037] According to another aspect of the present invention, a video image divided into prediction blocks is fed to a bitstream. An encoding method for encoding a signal to a signal, the encoding method being related to any of the above-mentioned methods. determining a motion vector for a predictive block to be processed; The difference between the predictor given by the prediction block and the position based on the motion vector is A process for encoding the signal to generate a bitstream containing the encoded difference and the initial motion vector. This includes the step.

[0038] According to another aspect of the present invention, a video image divided into prediction blocks is fed to a bitstream. A decoding method is provided for decoding from a prediction The block is given by the prediction block at the position specified by the refined motion vector. A step of analyzing the coded difference between the bit stream and the obtained predictor. and determining a refined motion vector for the prediction block according to any of the above methods. determining the location specified by the analyzed difference and the refined motion vector; Reconstruct the predicted block as the sum of the predictor given by the predicted block The method includes the steps of:

[0039] According to an aspect of the present invention, the above-mentioned aspects or or any combination thereof. A non-transitory computer-readable storage medium is provided.

[0040] In the following, exemplary embodiments are explained in more detail with reference to the accompanying figures and drawings. [Brief description of the drawings]

[0041] [Figure 1] FIG. 2 is a block diagram illustrating an example structure of an encoder in which motion vector derivation and refinement may be used. [Diagram 2] 1 is a block diagram illustrating an example structure of a decoder in which motion vector derivation and refinement may be used. [Diagram 3] FIG. 2 is a schematic diagram illustrating an exemplary template matching suitable for bi-prediction. [Figure 4] FIG. 2 is a schematic diagram illustrating an exemplary template matching suitable for uni- and bi-prediction. [Diagram 5] FIG. 2 is a block diagram illustrating a stage of motion vector derivation that operates without providing an initial motion vector that is refined in the bitstream. [Figure 6] FIG. 2 is a block diagram illustrating exemplary hardware for implementing an embodiment of the present invention. [Figure 7] FIG. 2 is a schematic diagram illustrating an example window with samples that should be available to be accessed for a coding block; [Figure 8] FIG. 1 is a schematic diagram illustrating an iterative search space. [Figure 9] FIG. 1 is a schematic diagram illustrating the expansion of a memory access window in the horizontal direction by interpolation filtering. [Figure 10] FIG. 13 is a schematic diagram illustrating sub-window definition with respect to fractional sample positions. [Figure 11] FIG. 2 is a schematic diagram illustrating an exemplary definition of a memory access window. [Figure 12] FIG. 1 is a schematic diagram illustrating example positions, including fractional positions, that are allowed to form positions in a search space for motion vector refinement. [Figure 13] FIG. 10 is a schematic diagram illustrating example fractional positions that are not allowed to form positions of a search space for motion vector refinement. [Figure 14] 1 is a flow diagram illustrating a method for determining which locations in a search space are allowed to be tested by template matching for motion vector refinement. [Figure 15] FIG. 1 is a schematic diagram illustrating example positions, including fractional positions, that are allowed to form positions in a search space for motion vector refinement. [Figure 16] 1 is a flow diagram illustrating an iterative refinement process on a memory access window. [Figure 17] 13 is a flow diagram illustrating the iterative refinement process in a search subwindow. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0042] This disclosure provides a method for refining and supplementing motion vectors to obtain fractional positions within a reference picture. This relates to adjusting the number of samples that should be accessible to perform the interval.

[0043] As mentioned above, external memory access is a fundamental requirement of today's hardware and software architectures. It is one of the most important design considerations in architecture. In the case of tor refinement, especially when template matching is involved, the estimation of motion vectors is It may also be used in conjunction with interpolation filtering to obtain fractional positions in the search space. The use of interpolation filtering reduces the number of samples that need to be accessed in memory. However, this requires expensive on-chip memory or external memory. This may lead either to an increase in the number of memory accesses, which on the one hand slows down the implementation. Especially on the decoder side, these problems can lead to undesirably more expensive or slower application This could lead to

[0044] To prevent such a situation, the present disclosure imposes restrictions on external memory access. According to this embodiment, the number of samples that should be accessible for the refinement of the motion vectors is The window is determined by the unrefined motion vector, i.e., the initial motion vector. The window is defined around the position pointed to by the It defines the maximum number of samples that need to be accessed in memory to perform Therefore, the samples that should be accessible are the search areas where template matching should be performed. The sample of positions in the search space is matched with the template for all positions in the search space. The latter generally exceeds the search space. Therefore, the window of memory accesses is limited to the number of coding blocks (for which motion vectors should be found). may be defined as an extension around the left edge of the current block. and R samples from the right boundary and R samples from the top and bottom boundaries define the window. In other words, when the current block has a size of N×N samples, The access window has a size of (R+N+R)×(R+N+R), i.e. (N+2R)×(N+2R) samples. For example, R may be equal to 4. However, the current block may have The size N of the vertical direction may be different from the size N of the horizontal direction, and the vertical direction may be up, down, left, and right. The number of directional dilation samples may also be different.

[0045] According to the present disclosure, in order to limit the memory access window, the necessary sub-threshold for interpolation is Memory accesses for the refinement of motion vectors where the sample is defined for integer samples. The fractional pixel coordinates are refined by the motion vector only if they are within the window is accessed.

[0046] FIG. 1 shows a block diagram of a conventional digital video input device for receiving input image samples of a frame or picture of a video stream. and an output for generating an encoded video bitstream. In this disclosure, the term "frame" is used as a synonym for picture. However, this disclosure is also applicable to fields where interlacing is applied. It is noted that, in general, a picture contains m by n pixels. For brevity, the following description refers to the luminance However, the motion vector search of the present invention refers to a pixel, which means a sample of the pixel. It can be applied to any color component, including chrominance or components of a search space such as RGB. On the other hand, it is possible to perform motion vector estimation for only one component and determine It may be beneficial to apply the resulting motion vector to further (or all) components. .

[0047] The input blocks to be coded are not necessarily of the same size. A picture can consist of different The block rasters of different pictures may contain blocks of different sizes, and the block rasters of different pictures may also be different. stomach.

[0048] In an illustrative implementation, the encoder 100 performs prediction, transformation, and quantization on the video stream. The encoded video bits are then subjected to a coding process, and an entropy coding process. Transformation, quantization, and entropy coding are performed to produce a digital image stream as output. , by the transform unit 106, the quantization unit 108, and the entropy coding unit 170. are executed respectively.

[0049] The video stream may include multiple frames, each frame being intra-coded. The image is divided into blocks of a certain size, either encoded or inter-coded. For example, a block of a first frame of a video stream is predicted by intra prediction unit 1. 54. Intra frames use only information within the same frame. Therefore, intra frames can be decoded independently. and providing an entry point within the bitstream for random access. Blocks of other frames of the video stream are inter-prediction units. The frame may be inter-coded by the frame 144, i.e., the frame may be inter-coded by the frame 144 after it has already been coded (reference frame). Information from the inter-frame is used to reduce temporal redundancy, and thus Each block in the coded frame is predicted from blocks in a reference frame. The selection unit 160 is configured to select the blocks of the frame to be processed by the intra prediction unit 154. 10 or 114 to be processed by the inter-prediction unit 144; The mode selection unit 160 also selects the parameters for intra or inter prediction. To allow refreshing of image information, the intra-coded blocks are The frame may be provided in an inter-coded frame. An intraframe containing only the blocks that have been decoded is the entry point for decoding, i.e. , the decoder can start decoding without having information from already encoded frames. These may be inserted periodically into the video sequence to provide points of interest.

[0050] The intra estimation unit 152 and the intra prediction unit 154 perform intra prediction. In particular, the intra estimation unit 152 predicts the The intra prediction unit 154 may derive the corresponding mode for differential encoding. A predictor that provides the predicted samples using the selected prediction mode. To perform spatial or temporal prediction, the coded block is subjected to an inverse quantization unit. The blocks may be further processed by the inverse transform unit 110 and the inverse transform unit 112. Later, the loop filtering unit 120 filters the decoded image to further improve the quality of the decoded image. The filtered blocks then form a reference frame, The reference frame is then stored in the decoded picture buffer 130. The decoding loop (decoder) is the same as the reference picture reconstructed on the decoder side. This provides the advantage of generating a reference frame. The second side operates in a corresponding manner. It refers to obtaining a reconstructed block by adding a predicted block to a block.

[0051] The inter estimation unit 142 estimates the inter-coded current frame or picture. It takes as input a block and one or more reference frames from the decoded picture buffer 130. The motion estimation is performed by the inter estimation unit 142, while the motion compensation is applied by the inter prediction unit 144. Motion estimation is, for example, The original image is also used to obtain the motion vectors and reference frames based on a certain cost function. For example, the motion estimation unit 142 may be used to provide an initial motion vector estimate. The initial motion vectors may then be sent either directly in vector form to the encoder and The motion vectors in the list of candidates are constructed in the same way in the decoder according to a given rule. Alternatively, the vector may be signaled in the bitstream as an index to a candidate vector. Then, motion compensation is performed by using the reference block of the same position as the current block in the reference frame. The motion vector is used to transform the current block to a reference block in the reference frame. The inter prediction unit 144 derives a predictor for the current block. A prediction block is output, which minimizes a cost function. The estimation function is the difference between the current block being coded and the prediction block of that current block. That is, the cost function minimizes the residual block. The reduction may be, for example, the reduction of all the candidate blocks in the current block and the candidate reference picture. It is based on calculating the sum of absolute differences (SAD) between pixels (samples). But generally , Mean Squared Error (MSE) or any other similarity metric such as Structural Similarity Metric (SSIM) Standards may be used.

[0052] However, the cost function also depends on the number of bits required to code such an inter block. The number of bits and / or distortion resulting from such encoding may be Thus, the rate-distortion optimization procedure determines the selection of motion vectors and / or the block size in general. Whether inter prediction or intra prediction should be used for the frame and which It may be used to determine encoding parameters such as which settings to use.

[0053] The intra estimation unit 152 and the intra prediction unit 154 are blocks of the current frame or picture and already reconstructed areas of the current frame Then, the intra prediction is performed by using the current Record the pixels of the current block of the current frame as a function of the reference sample of the frame. The intra prediction unit 154 outputs a prediction block for the current block; Said prediction block is advantageously a block corresponding to the current block to be coded and the The difference between the residual block and the predicted block is minimized, i.e., the predicted block is the residual block that is the least significant. The minimization of the residual block may be based, for example, on a rate-distortion optimization procedure. In particular, the prediction block is obtained as a directional interpolation of the reference samples. Similar to the above mentioned method with respect to rate-distortion optimization and / or inter prediction. It may be determined by calculating a measure.

[0054] The inter estimation unit 142 estimates the inter-coded current frame or picture. A block or more general form of image sample and two or more already decoded pictures Then, the inter prediction is performed by using the reference image sample of the reference picture. The current image sample of the current frame is described by a motion vector to the corresponding pixel. The prediction unit 142 outputs one or more motion vectors for the current image sample and The reference image samples pointed to by the vectors are advantageously the current image samples to be coded. Minimize the difference between the current image sample and the reference image sample for that image sample, i.e. , the reference image samples are minimized by the residual image samples. Then, A predictor for the difference is provided by the inter prediction unit 144 for differential encoding. do.

[0055] The difference between the current block and the prediction of the current block is then calculated, i.e., the residual block. The blocks 105 are transformed by the transform unit 106. The transform coefficients 107 are quantized by the quantization unit 108. and entropy coded by entropy coding unit 170. The encoded picture data 171 thus generated, i.e., the encoded video data The bit stream consists of intra-coded and inter-coded blocks. and (mode indication, motion vector indication, and / or The transform unit 106 includes corresponding signaling (e.g., Fourier or A linear transformation such as the discrete cosine transform (DFT / FFT or DCT) may be applied. Such a conversion to the 107 frequency domain results in the resulting coefficients 107 generally being more concentrated at lower frequencies and This gives the advantage that the effective coefficients (such as zigzag) have higher values. After scanning and quantization, the resulting sequence of values ​​is generally a series of zeros. has some larger values ​​at the beginning and end of The quantization unit 108 reduces the resolution of the coefficient values ​​to reduce the actual Then, the entropy coding unit 170 encodes the coefficient values ​​using a binary codeword. The entropy coding unit 170 also allocates the bit code to generate a bit stream. It encodes signaling information (not shown in FIG. 1).

[0056] FIG. 2 shows a video decoder 200. The video decoder 200 includes, among other things, a decoded picture buffer. The block prediction unit 240 includes a block prediction buffer 230, an inter prediction unit 244, and an intra prediction unit. The decoded picture buffer 230 includes a unit 254 for storing the encoded video bitstream. At least one (for unidirectional prediction) or at least two (for bidirectional prediction) a reference frame for directional prediction, the reference frame being This differs from the current frame (the currently decoded frame) of the encoded video bitstream. The intra prediction unit 254 generates a prediction block, which is an estimate of the block being decoded. The intra prediction unit 254 is configured to generate a reconstructed block 215 or The prediction is generated based on a reference sample obtained from the buffer 216. do.

[0057] The decoder 200 receives the encoded video bitstream generated by the video encoder 100. The decoder 200 and the encoder 100 are configured to decode the encoded stream. It is preferable to generate the same prediction for each block that is reproduced / decoded. The coded picture buffer 230, the reconstructed block 215, the buffer 216, and the intra The prediction unit 254 is characterized by the decoded picture buffer 130 of FIG. 15, buffer 116, and intra prediction unit 154.

[0058] The video decoder 200 may include, for example, the inverse quantization unit 110 of the video encoder 100, the inverse transform unit 112 of the video encoder 100, and the inverse transform unit 114 of the video encoder 100. The inverse quantization unit 112 corresponds to the conversion unit 112, and the loop filtering unit 120 corresponds to the inverse quantization unit 120. Such as a bidirectional transform unit 210, an inverse transform unit 212, and a loop filtering unit 220. It includes further units that are also present in videoencoder 100.

[0059] The entropy decoding unit 204 decodes the received encoded video bitstream. 209 and correspondingly obtain the quantized residual transform coefficients 209 and the signaling information. The quantized residual transform coefficients 209 are inversely transformed to generate a residual block. The residual block is fed to a quantization unit 210 and an inverse transform unit 212. The loop filtering unit is added to the H.265 signal to obtain the decoded video. The decoded frames of video are fed to the decoded picture buffer 230. and can serve as the decoded picture 231 for inter prediction.

[0060] Generally, the intra prediction units 154 and 254 in FIG. 1 and FIG. 2 need to be coded. To generate a prediction signal for a block that needs to be encoded or decoded, A reference sample from a region that has been determined to be non-uniform can be used.

[0061] The entropy decoding unit 204 receives as its input the encoded bitstream 171. In general, the bitstream is first parsed, i.e. the signaling parameters are The data and the residual are extracted from the bitstream. The syntax and semantics are designed to allow encoders and decoders to interoperate. As explained in the Background section above, In addition, the encoded bitstream does not only contain the prediction residual. In this case, an indication of the motion vector is also coded into the bitstream and is The motion vector indication is parsed from the bitstream. The coordinates of the motion vectors and the reference pictures to which the vectors are attached may be given. Up to this point, coding the complete motion vector has been considered. ,Only the difference between the current motion vector and the previous motion vector in the bitstream is This technique exploits the redundancy between the motion vectors of adjacent blocks. This makes it possible.

[0062] To efficiently encode reference pictures, the H.265 codec (ITU-T, H265, Series H: Audiovisual and multimedia systems: High Efficient Video Coding) Provide a list of reference pictures that assigns reference frames to list indices. In,the reference frame, the corresponding assigned list index is,added to the bitstream. Such a list is signaled in the bitstream by including It is defined in a case or signaled at the beginning of a video or a set of several frames. In H.265, there are two defined reference pictures called L0 and L1. It is noted that there are two lists. The reference picture then indicates the list (L0 or L1). and assigning a bit to the list by indicating the index in that list that corresponds to the desired reference picture. Providing more than one list is signaled in the stream. For example, L0 is a unidirectionally inter-predicted slice and L1 may be used for both inter-predicted slices in the bidirectional In general, however, this may be used only for slices that are inter-predicted in the The disclosure is not limited to any content of the L0 and L1 lists.

[0063] Lists L0 and L1 may be defined and fixed in the standard. Greater flexibility in encoding / decoding by signaling at the beginning of the video sequence Thus, the encoder may determine whether a particular reference picture is an index. The lists L0 and L1 may be constructed such that they are ordered according to The lists may have the same fixed size. In general, there may be more than two lists. The motion vectors may be signaled directly by their coordinates in the reference picture. Alternatively, as specified in H.265, a list of candidate motion vectors is constructed. Typically, an index within the list that is associated with a particular motion vector may be transmitted.

[0064] Usually, the motion vector of the current block is derived from the motion vector of the current picture or a previously coded picture. The motion vectors of adjacent blocks in the same frame are correlated. They are likely to correspond to the same moving object with similar motion, and the motion of the object changes over time. Therefore, the motion vectors in the adjacent blocks are unlikely to change suddenly. Using as a predictor reduces the size of the signaled motion vector difference. Typically, the motion vector predictor (MVP) uses spatially adjacent block or from temporally adjacent or co-located blocks In H.264 / AVC, this is done by dividing the three spatial This is done by taking the median of each component of the motion vectors adjacent to the Then no predictor signaling is required. The temporal MVP from the lock is considered only in the so-called temporal direct mode of H.264 / AVC. The H.264 / AVC Direct Mode derives motion data other than motion vectors. Therefore, these modes are also used for HEVC block merging concepts. In HEVC, the method to implicitly derive an MVP is to select which MVP from a list of MVPs. A motion vector that explicitly signals whether It has been replaced by a competing technology known as the variable coding quadtree block structure of HEVC. is a vector of 1 with several neighboring blocks with motion vectors as potential MVP candidates. Taking the left adjacent block as an example, in the worst case , the 64x64 luma coding tree block is not further divided, and the left block is expanded to the maximum depth. When the 64x64 luma prediction block is divided by 16 4x4 luma prediction blocks to the left, It is possible.

[0065] Advanced Motion Vector Prediction (AMVP) is a technique to improve the motion vector prediction to take such flexible block structures into account. It was introduced to fix vector contention. During the development of HEVC, the initial AMVP design It has been significantly simplified to provide a good compromise between efficiency and ease of implementation. The initial design of VP consisted of five MVPs from three different classes of predictors: spatial, The three motion vectors from the neighboring blocks, the median of the three spatial predictors, and and scaled motion vectors from co-located temporally adjacent blocks. In addition, the list of predictors is organized in a way that ranks the most promising move predictors in the first position. By reordering the order of the packets, and with minimal signaling overhead. This was fixed by removing redundant candidates to ensure consistency. The final design of the project was based on two MVP candidates: a) a block of five spatially adjacent b) at most two spatial candidate MVPs derived from or identical, one temporal block derived from two temporally co-located blocks. and c) no spatial candidates, no temporal candidates, or no both candidates are available. For more information regarding determining motion vectors, see the specification below, which is incorporated herein by reference. The book High Efficiency Video Coding (HEVC): Algorithm 14, edited by V. Sze et al., is incorporated in this volume. These can be found in Orientations and Architectures, Springer, 2014, especially chapter 5.

[0066] Further improving motion vector estimation without further increasing signaling overhead To achieve this, motion vectors are derived at the encoder side and provided in the bitstream. Further refinement may be beneficial. Motion vector refinement is done by the encoder. The encoder may be implemented in the decoder loop without the assistance of the decoder. The same refinement may be used to obtain the corresponding motion vector within The refinement is performed in a search space that includes integer and fractional pixel locations of the reference picture. For example, fractional pixel positions are performed at half pixel positions or quarter pixel positions. The fractional pixel positions may be further fractional positions. The fractional pixel positions may be calculated by interpolation, such as bilinear interpolation. may be obtained from integer (full pixel) positions.

[0067] In the bidirectional prediction of the current block, the first motion vector and The two predicted blocks obtained using the first motion vector in list L1 and the second motion vector in list L2 are combined into a single prediction block. signal, which provides better adaptation to the original signal than unidirectional prediction. This results in less residual information and possibly more efficient compression. Drop.

[0068] At the decoder, the current block is available because it has been decoded. Therefore, for the purpose of refining the motion vector, it is an estimate of the current block and is already processed. The part of the image that is encoded (i.e., coded at the encoder side and decoded at the decoder side) A template that is built on the basis of is used.

[0069] First, a first motion vector estimate MV0 and a second motion vector estimate MV1 are stored in the decoder 20. 0. At the encoder side 100, the motion vector estimate MV0 and MV1 is determined by block matching and / or (within the same picture or The motion vectors of the blocks adjacent to the current block (in the adjacent picture) The MV0 may be obtained by searching a list of candidates (such as a merge list) that have been created. and MV1 are advantageously signaled to the decoder side in the bitstream. However, Generally, the first decision stage of the encoder also reduces the signaling overhead. It is noted that this can be performed by template matching, which offers the advantage that:

[0070] At the decoder side 200, the motion vectors MV0 and MV1 are advantageously stored in the bitstream. MV0 and MV1 are either directly signaled or Either differentially signaled and / or a list of motion vectors ( However, the present disclosure is not limited to the bit stream. The present invention is not limited to signaling motion vectors within a frame. Rather, the motion vectors may be The first step is already determined by template matching according to the encoder operation. The template matching in the first stage (deriving the motion vector) may be set as a second refinement. The refinement step may be performed based on a search space different from the search space of the refinement step. , performed in a search space with higher resolution (i.e., shorter distances between search positions) This may be done.

[0071] Indicators of the two reference pictures RefPic0 and RefPic1 that MV0 and MV1 point to respectively. The reference picture is also provided to the decoder. As a result of each encoding and decoding, the decoded picture buffers on the encoder and decoder sides are generated. One of these reference pictures is selected by the search to find the motion vector The reference picture selection unit of the device for determining the motion vectors is The frame selects the first reference picture pointed to by MV0 and the second reference picture pointed to by MV1. Following the selection, the reference picture selection unit is configured to refine the motion vector. Whether the first reference picture or the second reference picture is used to perform the refinement To perform the motion vector refinement, the first reference picture is used to determine which of the following two reference pictures is used: A search area of ​​is defined around the candidate position pointed to by the motion vector MV0. Candidate search space locations are then matched by performing template matching within the search space and sum of absolute differences (SAD ) to find the blocks that are most similar to the template block. The search space is analyzed to find the lock. The location of the search space is determined by the top left corner of the template. As mentioned above, the upper left corner is just a convention and generally represents the position where the Any point in the search space, such as a point in can be used to represent a matching position.

[0072] According to the above-mentioned document JVET-D0029, the decoder-side motion vector refinement (DMVR) is a method of Let MV0 and MV1 be the initial motion vectors pointing to the respective reference pictures RefPict0 and RefPict1. These initial motion vectors are then taken as inputs to RefPict0 and RefPict1. The motion vectors MV0 and MV1 are then used to determine the search space for each of the Then, for each block (of samples) pointed to by MV0 and MV1, A template is constructed based on A and B. Template = function ((Block A, Block B))

[0073] The function may be a sample cropping operation combined with a sample-wise weighted sum. The template is then assigned to MV0 and MV1 in reference pictures 0 and 1, respectively. The template matching is then used to perform template matching within a search space determined based on the The cost function for determining the best template match in each search space is SAD( Template, Block candA', where block candA' is the The candidate coding blocks are pointed to by the candidate MVs in the search space. Denote the determination of the matching block A' and the resulting refined motion vector MV0'. Correspondingly, the same template is used to find the best matching block B ' and the corresponding motion vector MV1' that points to block B'. In other words, the template is aligned to the block pointed to by the initial motion vectors MV0 and MV1. After being constructed based on blocks A and B, the refined motion vectors MV0' and MV1' are It is found by searching on RefPic0 and RefPic1 using the template.

[0074] The motion vector derivation technology is also known as Frame Rate Up Conversion (FRUC). In general, the initial motion vectors MV0 and MV1 are used by the encoder and decoder to A bitstream is used to ensure that the same initial point may be used for vector refinement. Alternatively, the initial motion vector may include one or more initial candidates. This may be obtained by providing a list of initial candidates. Then, a refined motion vector is determined, and finally, the refined motion vector is calculated by minimizing the cost function. The motion vector that is selected is selected.

[0075] The present invention is not limited to the template matching described above with reference to FIG. It is further noted that FIG. 4 illustrates an alternative template mapping scheme that is also applicable to unidirectional prediction. For more details, see document JVET-A1001, in particular http: / / phenix.it-sudparis.eu / jvet / . "Algorithm Description of Joint Exploratory" by Jianle Chen et al., available at Section "2.4.6. Pattern matched motion ation Test Model 1" in document JVET-A1001 The template for this template matching method can be found in "Template Matching Vector Derivation". The rate is determined as the samples adjacent to the current block in the current frame. As shown in Figure 1, the already reconstructed samples adjacent to the top and left boundaries of the current block are A pull may be taken, called an "L-shaped template."

[0076] Figure 5 illustrates the derivation of another type of motion vector that may also be used. The input to the derivation process is a flag indicating whether motion vector derivation is applied or not. Implicitly, another input to the derivation process is a neighboring (temporally or spatially) already coded The motion vector of the reconstructed / encoded block is the motion vector of several adjacent blocks. The vectors are used as candidates for the initial search step of derivation of the motion vector. The output of the process is MV0' (and possibly MV1' if bi-prediction is used), respectively. and the corresponding reference picture indexes refPict0 and possibly refPict1. The motion vector refinement stage then involves template matching, as described above. One (unidirectional prediction) or multiple (bidirectional prediction / multiframe prediction) motion vectors are After finding it, the predictor for the current block is If not, it is indicated by the refined MV. (by referencing the sample).

[0077] The present invention is not limited to the above two template matching methods. A third template matching method called bilateral matching (also described in JVET-A1001) is A switching method can also be used for the refinement of the motion vectors, and the present invention , the same is true. According to bilateral matching, The best match between the two blocks along the motion trajectory of the current block is searched for. Under the assumption of continuous motion trajectories, we define motion vectors MV0 and MV1 pointing to two reference blocks. MV1 is the temporal distance between the current picture and two reference pictures, i.e., TD0 and T D1. In bilateral matching, SAD(Block cand0', Block cand1') may be used, where block cand0' is pointed to by MV0, Block cand1' is pointed to by MV1.

[0078] According to an embodiment of the present invention, an apparatus for determining a motion vector for a prediction block is provided, The apparatus includes a processing circuit that calculates an initial motion vector for a prediction block. and obtain a template, and perform template matching with the template in the search space. The method is configured to determine a refinement of the initial motion vector by iterative filtering. at a location given by an initial motion vector and including one or more fractional sample locations; Each fractional sample position in the search space evaluates only integer samples within the window. and the resulting signal is obtained by interpolation filtering using a filter with a predefined tap size, The window is accessible for template matching in the search space. It is formed by a set of integer samples.

[0079] A processing circuit 600 is shown in FIG. 6. The processing circuit may include any hardware configuration. The creation can be done by any kind of programming or hardware design, a combination of both. For example, the processing circuitry may be implemented with corresponding software that implements the steps described above. It may be formed by a single processor, such as a general purpose processor with associated software. On the other hand, the processing circuit is ASIC (Application Specific Integrated Circuit) or DSP (Digital Signal Processor) F Implemented by special hardware such as PGA (Field Programmable Gate Array) This may also be the case.

[0080] The processing circuitry includes the above-mentioned hardware components interconnected to perform the above-mentioned derivation of the motion vectors. The processing circuit 600 may include one or more of the following hardware components: This results in one initial motion vector (or multiple initial motion vectors if bi- / multi-prediction is used). The method further includes obtaining 610 the motion vectors and templates, and refining 620 the motion vectors. These two functions may be implemented on the same hardware. or the initial motion vector and template determination unit 610 and the motion vector refinement unit 620. This may be performed by a separate unit of hardware, such as the elaboration unit 620. The circuit 600 is in communication with an external memory 650 in which the reconstructed reference picture samples are stored. In addition, the processing circuit 600 may be connected to transfer data from an external memory and to process the data simultaneously. The number of samples in the window used to determine the motion vector for the block being processed is The processing circuit may further include an internal memory 640 for buffering the data. may be implemented on a single chip.

[0081] The processing circuit may be an encoder and / or decoder as described with reference to Figs. It is noted that further features may be implemented. The internal memory may be used as a cache or The chip memory may be an on-chip memory, such as a chip memory. The chip memory can be used to increase the speed of calculations. It is advantageous to implement it on the encoder / decoder chip to improve the performance. On-chip memories are usually small because of the limited size of the device. The memory can be very large in size, but access to external memory is much faster. It consumes a lot of energy and access is much slower. Usually, all the necessary information is , are fetched from external memory to on-chip memory before the computation is performed. External memory access (or memory bus) must be provisioned when designing The bandwidth (bandwidth) is the bandwidth required to transfer data between the external memory and the chip while decoding a frame or coding unit. This represents the largest possible amount of memory transfer between the In other words, generally speaking, only a single pixel can be accessed. Instead, the smallest unit (e.g. 8x8) is accessed. Large on-chip memories increase costs, so the size of the on-chip memories must be reduced. Size is also an important design consideration.

[0082] In other words, the above-mentioned device includes an internal memory embedded in an integrated circuit and the window Memory access to fetch integer samples in from external memory to internal memory The integrated circuit may further include a unit (interface).

[0083] The term "predicted block" as used above refers to the current block to be predicted. A block is a block that hierarchically divides an image (e.g., a coding tree unit (CTU)) into smaller units. Subdividing a block into equal or unequal sized blocks A square or a rectangle is a block in an image that may be obtained by Since this is a typical shape also used in encoders / decoders, the blocks are However, the present disclosure does not apply to any size / shape of blocks. Not limited.

[0084] The device including the processing circuitry may be an encoder, or a decoder, or even such an encoder. Apparatus that includes a coder or decoder, e.g. a recording device and / or a playback device It may also be a chair.

[0085] The fractional sample positions are used to reconstruct the reference picture coded as shown in FIG. The fractional position is between the actual picture sample positions given by The position shall be obtained by interpolation based on the nearest integer position. For details of the exemplary interpolation filtering used, see the book High Efficiency Video Coding (HEVC), Springer, 2014, section "5.3 Fractional Sample Interpolation" It can be found.

[0086] Typically, the interpolation filtering is done using different Apply the filter. As an example, in the H.265 video compression standard, the following 1D separable A filter is applied to generate quarter-pel and half-pel positions. [Table 1]

[0087] As can be seen from the table above, the interpolation filtering is done by dividing the filter taps (the number of coefficients in the table) by To generate a half-pel position, we need several samples around the corresponding fractional-pel position. Using the example filter above to filter out 4 integer samples from the left / top and right / bottom is required. The length of the interpolation filter is 1 / 4 pel sample positions (which is 7 taps). Note that the length for half-pel sample positions (which is 8 taps) differs from that for

[0088] In some embodiments of the present invention, the interpolation filter with predefined tap sizes is The integer samples available for template matching in the search space are We evaluate only integer samples within a given window. The window is the time it takes for a particular prediction block to be processed. may include many more samples than are actually used in the lock calculation. This is because the refinement search operation usually determines which part of the sample is the incremental part of the search operation. Fast search (as opposed to brute force search methods) according to whether or not it is evaluated as it goes along This is due to the fact that the template matching is performed using the The number of iterations of the search and the samples used for the refinement search operation are calculated for each prediction block. Locks may vary.

[0089] This disclosure takes into account that the interpolation filtering needs to be applied to fractional sample positions. to set an upper bound (boundary of the region) on the integer samples that may be used during the refinement search operation. This corresponds to the term "integer samples accessible for template matching". Which sample is actually accessed depends on the search space, as illustrated below: It depends on the method of forming the gap.

[0090] FIG. 7 shows a coding block (prediction block) and corresponding samples of a window. The samples shown in FIG. 7 are reference picture samples, where the coding block is In practice, the current time in the current frame from which a motion vector is to be derived in the reference picture It is noted that the block corresponds in size and position to the block in . ,The coding block of FIG. 7 is actually a block in the same position as the block in which the predictor is searched. However, for simplicity reasons, this block is referred to below as the "coding block". It is called.

[0091] In this example, the unrefined motion vector MV0 points to an integer sample position. (The initial motion vector can point to a fractional sample position, just for ease of illustration.) (Integer sample positions are chosen to make the motion vector refinement search granularity 1 Since the starting point is an integer sample, only the integer sample points are The search is performed in this example in a gradually evolving search space. This means that the search space responds to the best direction in terms of the cost function with respect to the positions already tested. This means that the search can be improved at each iteration by adding new search positions as needed.

[0092] Such an approach is shown in simplified form in Figure 8. In Figure 8, the initial motion vector is The search space is gradually built up around the location of the initial motion vector. In the first step, the upper and lower parts of the position 810 pointed to by the initial motion vector are , the four immediately adjacent positions to the left and right, as well as the initial motion vector. The location 810 that yields the lowest cost function among the five points tested is Based on the direction, further positions to be tested are added to the search space. ,The lowest cost function can be found at the right point, and thus the search The space was expanded horizontally to the right by three more points in the second step. In this step, the lowest cost function is (relative to the lowest cost point in the first step) It is possible to search for three points in the horizontal direction to the right. In the third step, the lowest cost function is again selected as the step The lowest cost point in top 2 is observed at the right point, and three more points are observed horizontally to the right. This results in expanding the search space by 1 point. According to the example in Figure 8, three further steps are In the example, the search points are A diamond-shaped pattern is used for each iteration, and the search space is expanded at each step. is expanded to fill in the missing search points.

[0093] At each iteration of determining the search space, the search space is enlarged by one or more integer sample positions. Now, let us return to FIG. 7 for an example in which the maximum number of search iterations is 4. Since iterations are possible, if the gradual evolution of the search space is to the left, the subplot drawn on the left All of the samples must be retrieved from memory to perform the search operation. An extension of 4 samples above M is required. Thus, the search space is expanded to the refined M Since V can move in either direction, it is expanded in both directions (left and right and up and down) and The hardware implementation must ensure that all of the samples that may be required are removed before the refinement search is applied. If the search space evolves downwards or to the right, In this case, template matching with a template corresponding to the size of the coding block (prediction block) is performed. Because the switch needs access to some of those samples, four additional samples are Furthermore, hardware implementations are generally not suitable for handling irregular shapes. Since it is not possible to fetch the sample from the corners (rectangle access is more feasible), For example, the top right corner) must also be fetched from memory.

[0094] The iterative search space evolution described above is merely illustrative; it is possible to expand the search space at each iteration. The rules for extending the line and the number of points may be different, i.e., specified in different ways. is noted.

[0095] Figure 8 also illustrates scenarios that may occur due to the external memory access rules described above. The number of samples fetched from external memory is determined by the number of samples actually used in the computation step. The template here is for simplicity's sake, it is a one-sample Assuming that the size of the sample is the same as that of the external memory, the white circles represent the samples and It represents the actual filled samples that will be used, but not until the current block begins to be processed. Since the actual required samples are not yet known when the Such redundancy is necessary if the number of accesses is to be kept low.

[0096] The search space can also be expanded in different ways, for example by changing the position pointed to by the initial motion vector. It is noted that a shape may be defined as a stable shape in a given position. The shape may be a square, a rectangle, It may be of any shape, such as a diamond.

[0097] Figure 9 shows an example where the search space may also include fractional samples. The search for the access vector is performed on integer samples, resulting in an access window This resulted in the position indicated by the larger solid point in the When performed on samples with pel resolution (the smaller solid points), the interpolation filter Assuming the filter is symmetric and has 8 taps, it produces the fractional samples shown on the right. To complete the calculation, three additional columns of samples must also be retrieved from memory. Furthermore, the search operation is symmetric so that fractional pixels may be on the left side of the window ( The same applies to the left side due to the fact that the (an expansion of 3 columns of pixels).

[0098] As a result, the number of samples that need to be taken from memory due to the interpolation filtering is The number of circles has been further increased and now includes dotted circles that represent positions added due to fractional interpolation. Similarly, if we allow half-pel positions in the vertical direction to be searched as well, The window of samples accessed by the memory is also extended vertically above and below. (not shown in the example of Figure 9).

[0099] The window of memory access is determined by the motion vectors for the prediction block (coding block). contains all of the samples that need to be retrieved from memory to perform the search for The window of memory access is defined as a rectangle around the number of actual samples required. It not only contains the remaining sub-segments that may be accessed during a motion vector search operation, but also In the example of FIG. 9, the motion vector search has been shifted to the right. However, it can also be moved to the left, which is not known in advance. Therefore, in order to avoid accessing the external memory multiple times, a window of memory access (also The access window is the total number of samples that are accessible by each process. Includes

[0100] Figure 10 shows the window of memory accesses for motion vector refinement. Center point 10 10 is the unrefined input motion vector (either from the bitstream or as described above) obtained either by template matching already performed or by testing of candidates. The fractional position is the position pointed to by the initial motion vector (specified by the To avoid further increase in window size due to the The refinement is performed according to the following rules: A) The refinement memory access window is the same as the unrefined initial motion vector. The coordinates of the vector are defined around the coordinate of the current vector (i.e., the position pointed to by the initial motion vector). The window refines the motion vector by template matching in the search space. Specifies the maximum number of pixel samples that must be accessed from memory to perform do. 1. In this example, for simplicity, we use the size of the current block (encoding block size) is 1x1 sample, but the size of the current block can be larger. Yes, and generally larger. 2. The memory access window is 4 samples from the left / right and above / below as shown in the figure. It is defined as an extension around a coding block, such as 1 to 4 samples. B) Fractional pixel coordinates are the window of memory accesses for samples needed for interpolation. It is accessed for MV refinement only if it is within

[0101] Requirement B is satisfied by the number of samples required for the refinement of the motion vector to integer samples. This rule ensures that the access window defined by the The actual number of fractional samples accessible through the interpolation filter is given by the size and shape of the interpolation filter. Therefore, in FIG. 10, assuming a 6-tap interpolation filter, the dotted line is However, as shown with reference to FIG. It is noted that other fractional pixel locations may be acceptable. Vertical filtering only or horizontal filtering that does not require expansion beyond the Fractional positions that require only one digit may still be used. Limiting the sample window to the fraction shown may be too restrictive for some applications. There may be a limit.

[0102] In other words, according to an embodiment, the memory access window is It includes all samples that are accessible by the motion vector refinement performed by the , the samples that are not accessible due to the refinement of the motion vectors performed on integer samples. Therefore, fractional samples are used for the refinement of the motion vectors. If so, then those fractional samples are obtained in a manner that does not require additional samples.

[0103] In the example of FIG. 10, this is done in terms of the shape and size of the predefined interpolation filters. By allowing only fractional samples, which does not require samples outside the access window The dotted fractional sample window extends within the access window. If is the number of taps of the interpolation filter, then the boundaries of the window of fractional samples are The sample is taken from the boundary of the access window and is within a distance of 1020 integer samples, which is floor(T / 2)-1. In particular, in this example, T=6, T / 2=3, and the access window The distance from the boundary samples of the fractional window to the samples is T / 2-1=2 integer samples. be.

[0104] However, it is noted that this determination of the fractional window is merely an example. The windows may have different shapes and sizes. Vertical and horizontal interpolation is , may be performed by filters of different sizes. Furthermore, some fractional positions are generally Both vertical and horizontal filters are required, which may or may not be separable. It is also possible to use the following.

[0105] Alternatively, the interpolation filter may be modified for fractional positions outside the dotted window of FIG. It is possible (e.g., the number of taps in at least one direction is reduced). However, for implementation purposes For reasons of time and quality of the interpolation, such a solution may be relatively unattractive.

[0106] The window for memory access may be defined in a variety of ways. The access window is set to the position of the coding block given by the initial motion vector. The example shows that the extension EXT is defined as the left / right or top / bottom extension EXT of the corresponding coding block. The amount of may depend on the size and shape of the coding or prediction block. The extension is 4 samples long in each direction (up, down, left, right). However, EXT is 1 sample long (vertical The block size (which may have different sizes in the vertical and horizontal directions) and / or the search space Note that the values ​​may vary for different directions depending on the shape and size of the It is intended.

[0107] For example, according to the example, the window is N integer numbers with respect to the initial motion vector of the prediction block. is defined as M integer sample columns and M integer sample rows, where at least one of N and M is non-negative. The integer value of zero (both are integers, but one of N and M can be zero). In the example, N and M are shown to have the same size. As mentioned above, N and M can be different. N and M are integers, and at least one of them may be , nonzero. To obtain the parameters N and M and the shape and size of the template, In particular, if the template has T1 rows and T2 If the memory has N+T2+N rows and M+T1+M columns, then the size of the memory access window is (N+T2+N) rows and (M+T1+M) columns. This means that the search advances N samples to the left or right, and 2N samples horizontally. This is because it can cause a pull, go M samples up or down, and 2M samples vertically. .

[0108] On the other hand, certain techniques for constructing the search space, such as those described with reference to Figs. 7 and 8, Regarding the memory access window, it is later converted into the maximum amount of left, right, up, and down displacement. The maximum number of refinement iterations (iterations of constructing the search space) that can be performed and the maximum number of iterations that can be achieved at each iteration. The iteration step size (in terms of the maximum possible sample distance) may be defined as the iteration step size. Therefore, the memory access window is defined as the maximum displacement in each direction. For example, four iterations, each of which may advance the search space by up to one integer sample position, are called EXT=4. It results.

[0109] In other words, according to this example, the processing circuitry is Iteratively expand in the direction given by one (or more) of many best matching positions The initial motion is obtained by template matching with the template in the expanded search space. The window is configured to determine the refinement of the vector, the window being a predefined minimum of iterations. It is defined by the large number of times.

[0110] In general, this disclosure is not limited to any particular shape or form of the search space or type of decision. In another example, the search space is determined by dividing each fraction in the subwindow by All integer samples accessed for sample interpolation filtering are predefined The tap size of the interpolation filter is determined by the tap size of the window. A similar example has already been discussed above with reference to FIG. In this case, the search space is 9x9 integer samples, and the initial motion vector position is the center of the sample. and the fractional samples in the region formed by the 5 × 5 integer samples in can be done.

[0111] The definition of the memory access window probably requires the corresponding signaling parameters to be For inclusion in the bitstream (for example, parameter EXT or parameters N and M) However, the size of the memory access window is defined by the standard. or other factors (such as template size, prediction block size, image resolution, etc.) It may be derivable based on the coding parameters.

[0112] FIG. 12 shows an example with different fractional pixel positions. For this example, The size is (for simplicity) 1x1 sample, and a 6-tap interpolation filter is applied at each half-pel position. In the example, the search coordinates to be searched are assumed to be A number indicating the order in which the targets are examined, i.e., the order in which template matching searches are performed. Positions 1 and 2 are half-pel positions (i.e., they are , halfway between two integer sample locations, and pel is short for pixel, The term is used interchangeably with the term sample in this application. Positions 1 and 2 show that the necessary expansion for the interpolation filtering is within the window of memory access. Since it is in the position 1, it can be checked (the 3 integer samples diagonally to the upper left and the 3 integer samples diagonally to the lower right, (3 integer samples to the right and 3 integer samples to the left of position 2). Fractional sample point 1 is horizontally and vertically Note that both the vertical and horizontal extensions are required and that both are within the window. Position 2 only requires expansion to the right and left.

[0113] Positions 3, 4, and 5 are integer sample (integer pel) positions. They are the positions of the interpolated frame. No extension for filtering is required, so it can be explored. Fractional (half-pel) samples 6 can also be used for vertical Only vertical expansion (up and down by 3 integer positions respectively) is required, and that is still the win Since the data is inside the window, it can be accessed. No horizontal expansion is required. Again, the above implementation In this case, fractional sample points are used when the extent of the required interpolation is within the memory access window. is accessed only by

[0114] In other words, according to the example, the interpolation filter is When on a horizontal or vertical line, there are K horizontal or vertical It is a one-dimensional filter that is evaluated every integer sample.

[0115] Such a point on a line between two horizontally or two vertically adjacent integer positions Such one-dimensional fractional positions (e.g., positions 2 and 6 in Figure 12) can be scaled horizontally or vertically. Only the horizontal or vertical interpolation requires dilation. The fractional positions should be filtered only by the filter. In addition to the fractional positions allowed in the example of FIG. 10, the following fractional positions can be used: It may be advantageous to add further one-dimensional positions, such as position 6, which is added.

[0116] In other words, the search space is - adjacent to the top or bottom of the subwindow and on a horizontal line of integer samples or - On a vertical line of integer samples adjacent to the left or right side of the subwindow Is it in Any fractional position that is outside the fractional sub-window (see the dotted window in Figure 10) Including etc.

[0117] Some fractional samples are integer samples further in a given direction, horizontally or vertically. It is noted that it may be necessary to use Different cases may apply to generate the position in each direction.

[0118] FIG. 13 shows an example of fractional half-pel locations 1 and 2 that cannot be accessed. The location is outside the subwindow shown in Figure 10. For this example, a 6-tap interpolation It is assumed that a filter is used for half-pel location. Half-pel search points 1 and 2 are water If horizontal or vertical interpolation filtering requires samples that are outside the window, Therefore, it is not allowed to be searched. is required by the vertical filter to filter position 2. The integer sample positions required by As such, the dashed circles correspond to integer locations that are not within the memory access window.

[0119] In the above example, the memory access window is ( Ensure that samples outside the window are not accessed (even for interpolation filtering). In other words, the memory access window is the time required for the refinement and The smallest window that contains the samples that may need to be accessed for interpolation. In addition, the memory access window is based on integer sample positions and is a motion vector. The sample was then designed according to the required sample size for the refinement of the vector. Only fractional positions that do not require further extension of such an access window by will be done.

[0120] It is noted that the above examples are given in terms of half-pel interpolation. However, the present disclosure is not limited to such. Generally, any fractional position such as 1 / 4, 1 / 8, etc. may be used, i.e., It may be interpolated using a corresponding interpolation filter.

[0121] The processing circuitry described in relation to FIG. 6 may be used to implement the encoder and It may be used in the decoder and / or the processor.

[0122] In particular, a coder for encoding a video image divided into predictive blocks into a bitstream. A coding device may be provided, the coding device including a processing circuit for processing the above-mentioned prediction block. A device for determining a motion vector for a prediction block and a method for determining a motion vector for a prediction block The difference between the predictor given by the prediction block is coded at the position specified by and generating a bitstream containing the coded difference and the initial motion vectors. and an encoding circuit for generating the encoded data.

[0123] The further units and functions of the encoder described above with reference to FIG. 1 also include a processing circuit The present invention may be provided or implemented in a

[0124] Correspondingly, the video image divided into prediction blocks is decoded from the bitstream. A decoding device for decoding a predicted block may be provided, the decoding device being configured to generate an initial motion vector and a predicted block. The motion vector is then refined and the predicted block is assigned to the location specified by the motion vector. A solution for parsing the coded difference between the predictor and the predicted one from the bitstream. A motion vector analysis unit and a processing circuit for the refined motion vector for the prediction block. and a device for determining the location of the motion vector based on the analyzed difference and the refined motion vector. to reconstruct the predicted block as a sum with a predictor given by the block For example, the predictor may include a decoder circuit for determining whether a motion vector is generated based on the location of the refined motion vector. However, it may be possible to use the The motion vector of the current prediction block may be further modified. There may be further processing steps.

[0125] The further units and functions of the decoder described above with reference to FIG. 2 are also included in the processing circuit. may be provided or implemented.

[0126] Additionally, embodiments of the present invention may include processing circuitry for performing motion vector refinement. However, the present disclosure is not limited thereto and may be modified in any manner in which the processing circuitry described above is implemented. A corresponding method is also provided that includes processing steps corresponding to the processing steps configured for the row. do.

[0127] In particular, obtaining an initial motion vector and a template for the prediction block. In the search space, an initial motion vector is obtained by template matching with the template. determining a refinement of the motion vector, the search space being given by an initial motion vector; The fractional sample locations are located at positions that can be obtained by the search, and contain one or more fractional sample locations. Each tap position has a predefined tap size that evaluates only integer samples within the window. the window is obtained by interpolation filtering using a filter of the same size, Formed by integer samples accessible for template matching in space In accordance with the present invention, a method is provided for determining a motion vector for a predictive block.

[0128] The taps correspond to the filter coefficients. The tap size corresponds to the filter order. where the filter is assumed to be a linear filter. In some examples, the filter is However, the present disclosure does not apply to symmetric filters. The present invention is not limited to a linear filter, or a filter of any kind. The number position may be obtained in any manner based on adjacent samples.

[0129] Further, a method for encoding the video image divided into prediction blocks into a bitstream is provided. A coding method is provided, the coding method comprising: determining a motion vector for the prediction block; and The difference between the predictor given by the predicted block and the predicted block is coded. and generating a bitstream including the estimated difference and the initial motion vector.

[0130] The encoding method may further include the steps described with reference to the functions of the blocks in FIG. good.

[0131] Further, a method for decoding the video image divided into prediction blocks from a bitstream is provided. A decoding method is provided, the decoding method being refined with an initial motion vector and a prediction block. The predicted value given by the prediction block is set to the position specified by the motion vector. and analyzing the coded difference between the bit stream and the data. determining a refined motion vector for any of the predictive blocks; The difference is analyzed and the predicted block is inserted at the location specified by the refined motion vector. and reconstructing the prediction block as a sum of the predictor given by:

[0132] The decoding method may further include steps described with reference to the functions of the blocks in FIG. stomach.

[0133] However, it is noted that Figures 1 and 2 should not limit the present disclosure. To give non-limiting examples of implementations of the present invention within existing encoders and / or decoders, Not too much.

[0134] FIG. 14 illustrates an exemplary implementation of the method according to the embodiment. C) is the additional horizontal and vertical scaling required to apply the interpolation filtering. Returns the number of samples. The number of samples required is - Whether the search coordinates are integer pel positions, half pel positions, or quarter pel positions. Is there one? - A horizontal interpolation filter must be applied to generate the search coordinate samples. Specifies whether a vertical interpolation filter should be applied, or whether both interpolation filters should be applied. Can the data be applied? It depends on the.

[0135] The method begins at step 1430. In particular, the position of an initial motion vector is tested. The function InterpolationFilterLength(C) is the position C(x,y) of the first search space to be The number of horizontal and vertical samples to apply the interpolation filtering, Returns the sum of C(x,y) and InterpolationFilterLength(C(x,y)) multiplied by MAX(max_x, max_y). If the access window size defined by the search is exceeded, then the location is a part of the search space. Instead, the next search coordinate C(x,y) is used as the step to be tested. In step 1440 (e.g., x or y or the like depending on the order in which the searches are performed), (Both of these are incremented.) The location tested in step 1430 is If no window exceedance is required, then in step 1410, the motion vector refinement is performed. Template matching as part of the refinement is performed on the location C(x,y). and whether there are any more search space positions left for template matching. This is tested in step 1420. If not, the refinement is terminated. If so, the next coordinate is selected in step 1440 and the condition of step 1430 is satisfied for that new coordinate. The position is evaluated for a new position. These steps are repeated.

[0136] As already explained above, (possibly with further fractional samples as shown in FIG. 12) Alternatively possible ways of defining the allowed fractional positions (such as the window in Figure 10 extended by Based on these, we can determine whether the position C(x,y) belongs to the allowed window. A simpler condition to just evaluate may be formulated in step 1430. For example, ,The search is performed over a memory access window such as the ,memory access window shown in Fig. 10. The fractional search window is limited to integer search points and fractional points within a fractional search window encapsulated in

[0137] The search space may include a rectangular search sub-window of the window, and the initial motion vectors The refinement of the algorithm is performed by interpolating the fractional samples in the search subwindow for filtering. The integer samples accessed are then compared to the above-mentioned filter with respect to an interpolation filter of a predefined tap size. A search subwindow of a rectangle such as that in the template window is This may be determined by rate matching.

[0138] More precisely, the allowed integer and fractional positions are the unrefined initial motion vectors In particular, the distance of the search point from the point associated with the and the unrefined initial motion vector In particular, the memory access window is It is defined for a point in a reference picture pointed to by an initial motion vector that is not yet encoded. This is illustrated in FIG. 15, where the center point of the memory access window is , which is a half-pel point between two integer points along the horizontal axis. Dough is defined as the extent in the x direction (N) and the extent in the y direction (M), where N and M are integers. , at least one of which is non-zero. That is, the subwindow (or search window) for the search point is the memory access window. All objects defined in the window are allowed to be searched by template matching. Define the x-direction extension (P) and y-direction extension (R) as the integral-pel and fractional-pel search points of In FIG. 15, the numbers P and R are equal to 1 for illustrative purposes. If the distance of the search point to the point exceeds P or R in the x and y directions, respectively, is not included in the memory access window. In this particular example, fractional search points are added. Since the memory access required is N and M, The cess window encapsulates or contains the secondary window. P and R are horizontal It can be a real number indicating the distance in units of the distance between two integer sample points in the direction and perpendicular to the As an example, if P and R are defined as P=1.5 and R=1.5, then the initial center A search point (integer or fractional) at a distance of 1.5 in the x direction to the point is used in the motion vector refinement process. In addition, in FIG. 15, the left and right expansion and the up and down expansion are allowed. The extensions of the directions are defined to be equal, but in general they do not have to be equal. In general, all four extensions to the left, top, right, and bottom can be defined independently.

[0139] Figure 16 shows a possible implementation of the iterative refinement process on memory access windows. According to Fig. 16, the refinement search is applied iteratively, i.e., only up to K search points are are searched in each iteration. First, K search points are searched around the initial starting point or The best search point selected as a result of the iteration is determined (1610). Second, the K search points are If all of the points are within the memory access window (1620), the refinement search operation However, if any one of the K points is in the memory access window, If it is outside, then the search iterations are terminated. Third, the condition for the maximum number of search iterations is checked (1630) and if the current iteration exceeds the maximum number of allowed iterations of the search, Finally, a refinement search is applied to the K search points (1640), The best point among 1640 and 1650 is selected according to a matching cost function (1650). The application increments the number of iterations by 1. After 1650, the process returns the best point, which is the output of 1650, to 161. 0 as input to iterate. According to Figure 16, the search points searched during one iteration are The number K, which determines the number of search points to be searched for each iteration, defines the maximum number of search points for each iteration. The number of points K and the pattern may vary depending on the search strategy used. According to the explicit search strategy, K=5 search points (center, left, right, bottom, and top) are selected for the first iteration. while K=3 points (center, bottom right, bottom left) may be searched in the second iteration. The number K can be less than the total number of search points in the memory access window. be.

[0140] According to the specific implementation shown in FIG. 16, a single search point among the K search points of an iteration is If it is outside the memory access window, the search iteration is terminated. There may be other points among the K search points that are within the window, but the iterations are terminated. Therefore, those points within the memory access window are also not explored. The goal is to reduce the number of search iterations and at the same time reduce the number of searches that fall outside the memory access window. The aim is to ensure that the point is not searched.

[0141] 1620 allows you to check whether the search point is within the memory access window. , can be performed by examining the distance of the search point to the initial starting point. Thus, the distance If the x component of is greater than N, or the y component of the distance is greater than M, the search point is Generally, N and M are integers, and At least one of them is greater than 0.

[0142] The refinement of the initial motion vectors is performed by finding more of the best matches in the search space in the latest iteration. The search space is iteratively expanded in a direction given by one of the positions The iteration may be determined by template matching with a previously stored template, and the iteration may be determined by When at least one sample in the search space of the new iteration is outside the search subwindow will be terminated.

[0143] FIG. 17 illustrates possible iterative schemes that can be applied to the search subwindows as shown in FIG. The flowchart of FIG. 17 is the same as the flowchart shown in FIG. 16 except for step 1720. According to step 1720 of FIG. 17, a refinement search is performed on K search points. The decision is to check whether all of the K points are within the subwindow for the search point. In other words, as shown in Figure 15, the initial unrefined The distance of any one of the K search points to the center point pointed to by the motion vector If is greater than P or R, then the condition shown in 1720 evaluates to false and the iteration is terminated. It will be completed.

[0144] As described above, the determination of the motion vectors with the memory window restriction is However, the determination of the motion vectors may be performed as part of the encoding and / or decoding of the The present invention is not limited to being used for encoding / decoding, but may also be used for motion detection, motion analysis, etc. It may also be used for other purposes in image processing, such as:

[0145] The determination of the motion vectors may be implemented as a device. Such a device may be implemented as a software For example, the determination of the motion vectors may be performed by a general purpose processor. processor, or digital signal processor (DSP), or field programmable The present invention may be implemented by a chip such as a FPGA. The present invention is not limited to implementation in programmable hardware. (ASIC), or by a combination of the hardware components mentioned above.

[0146] The motion vector determination may also be performed by program instructions stored on a computer readable medium. The program, when executed, causes a computer to execute the steps of the above-described method. The computer readable medium may be a DVD, CD, USB (flash) drive, hard disk, hard disk, server storage available via a network, etc. The information may be stored on any medium.

[0147] The encoder and / or decoder may be installed in a television receiver, a set-top box, a PC, Various devices, including tablets, smartphones, etc., can record, encode, The encoder and / or decoder may be implemented in a transcoding, decoding, or playback device. The reader implements the steps of the method and is stored in an electronic device such as the electronic device described above. It is software or an application that runs on a processor contained in an electronic device. This is also fine.

[0148] In summary, the present disclosure relates to the refinement of motion vectors. As a first step, a prediction An initial motion vector and template for the block are obtained. The refinement of the vector is performed by template matching with the template in the search space. The search space is determined by the position given by the initial motion vector, The search space includes one or more fractional sample positions, each of which is a win Interpolation using a filter with a predefined tap size that evaluates only integer samples in the dot. the window is obtained by vector filtering, the window being a template in the search space; The sample is formed by integer samples accessible for matching. [Explanation of symbols]

[0149] 100 Encoder 105 Residual Blocks 106 Conversion Units 107 Conversion Factors 108 quantization units 110 Inverse Quantization Unit 112 Reverse conversion unit 115 Reconstructed Blocks 116 Buffers 120 Loop Filtering Unit 130 Decoded Picture Buffer 142 Inter-Estimation Units 144 Inter Prediction Units 152 Intra Estimation Units 154 intra prediction units 160 Mode Selection Unit 170 Entropy Coding Unit 171 Encoded picture data, encoded bitstream 200 Decoder 204 Entropy Decoding Unit 209 Quantized Residual Transform Coefficients 210 Inverse Quantization Unit 212 Reverse conversion unit 215 reconstructed blocks 216 Buffers 220 Loop Filtering Unit 230 Decoded Picture Buffer 231 Decoded Pictures 244 Inter Prediction Units 254 intra prediction units 265 predicted blocks 600 Processing Circuit 610 Initial Motion Vector and Template Determination Unit 620 Motion Vector Refinement Unit 640 Internal Memory 650 External Memory 810 center point, position 1010 center point 1020 distance

Claims

1. 1. An apparatus for determining a motion vector for a prediction block, comprising: obtaining an initial motion vector and template for the prediction block; The initial motion is calculated by template matching with the template in the search space. a processing circuit configured to determine a refinement of the vector; The search space is located at a position given by the initial motion vector and includes one or more wherein each of the fractional sample positions belonging to the search space is a window. Interpolation using a filter with a predefined tap size that evaluates only integer samples within the The window is obtained by filtering the template in the search space. The apparatus is formed by integer samples accessible for edge matching.

2. The window is a sequence of N integer samples and a time period for the initial motion vector of the prediction block. and M integer sample rows, where at least one of N and M is a nonzero integer value. The device of claim 1.

3. The processing circuitry selects a number of best matching positions in a search space in a most recent iteration. The template in the search space is iteratively expanded in a direction given by one of the determining the refinement of the initial motion vector by template matching with a rate; It is configured to 10. The method of claim 1, wherein the window is defined by a predefined maximum number of the iterations.

3. The apparatus according to claim 2.

4. The search space is then adjusted for interpolation filtering of each fractional sample in the subwindow. All integer samples accessed are interpolated using the interpolation filter of the predefined tap size. The subwindow of the rectangle of the window that is within the window with respect to the data is 4. An apparatus according to any one of claims 1 to 3, comprising:

5. The search space includes a rectangular search sub-window of the window, and the initial motion vector The refinement of the vector comprises the interpolation filtering of each fractional sample in the search subwindow. The integer samples accessed for the sampling are the complement of the predefined tap size. In the rectangular search subwindow that is within the window with respect to the intermediate filter 3. The method according to claim 1 or 2, wherein the template is determined by template matching with the template. The equipment.

6. The processing circuitry selects a number of best matching positions in a search space in a most recent iteration. The template in the search space is iteratively expanded in a direction given by one of the determining the refinement of the initial motion vector by template matching with a rate; and the iteration is configured to:

6. The apparatus of claim 5, wherein the search is terminated when at least one sample falls outside the search subwindow. 。

7. The interpolation filter interpolates each horizontal line or is on a vertical line, K integer samples either horizontally or vertically.

5. The apparatus of claim 4, wherein the filter is a one-dimensional filter that also evaluates

8. The search space is: - on a horizontal line of integer samples adjacent to the top or bottom of said subwindow in or - a vertical line of integer samples adjacent to the left or right side of the subwindow Is it on the 6. The apparatus of claim 5, further comprising any of the fractional positions outside the subwindow.

9. Coding device for coding a video image divided into predictive blocks into a bitstream - Patents.com The position is For determining a motion vector for a prediction block according to any one of claims 1 to 8 The apparatus, The predicted block is provided at a position based on the motion vector determined as the predicted block. and encoding the difference between the encoded difference and the initial predictor. and an encoding circuit for generating a bitstream including the final motion vector. Place.

10. Decoding device for decoding a video image divided into predictive blocks from a bitstream - Patents.com And, The initial motion vector, the predicted block, and the refined motion vector are specified. The coded difference between the predictor given by the prediction block and the position of the a parsing unit for parsing from the bitstream; The refined motion for the predicted block according to any one of claims 1 to 8. A device for determining a vector; The analyzed difference and the predicted motion vector are then applied to the location specified by the refined motion vector. Reconstruct the predicted block as a sum of the predictor given by the measured block and a decoding circuit for decoding the encoded data.

11. 1. A method for determining a motion vector for a predictive block, comprising the steps of: obtaining an initial motion vector and a template for the prediction block; The initial motion is calculated by template matching with the template in the search space. determining a refinement of the vector; The search space is located at a position given by the initial motion vector and includes one or more wherein each of the fractional sample positions belonging to the search space is a window. Interpolation using a filter with a predefined tap size that evaluates only integer samples within the The window is obtained by filtering the template in the search space. The method according to the present invention, wherein the signal is formed by integer samples accessible for signal matching.

12. The window is a sequence of N integer samples and a time period for the initial motion vector of the prediction block. and M integer sample rows, where N and M are non-zero integer values. Method of posting.

13. The refinement of the initial motion vectors expands more of the search space in the most recent iteration. The search space is iteratively expanded in a direction given by one of the best matching positions. by template matching with the template between 11. The method according to claim 10, wherein the window is defined by a predefined maximum number of said iterations.

13. The method according to claim 12.

14. The search space is then adjusted for interpolation filtering of each fractional sample in the subwindow. All integer samples accessed are interpolated using the interpolation filter of the predefined tap size. The subwindow of the rectangle of the window that is within the window with respect to the data is 14. The method according to any one of claims 11 to 13, comprising:

15. The search space includes a rectangular search sub-window of the window, and the initial motion vector The refinement of the vector comprises the interpolation filtering of each fractional sample in the search subwindow. The integer samples accessed for the sampling are the complement of the predefined tap size. In the rectangular search subwindow that is within the window with respect to the intermediate filter 13. The method according to claim 11 or 12, wherein the template is determined by template matching with the template The method described.

16. The refinement of the initial motion vectors expands more of the search space in the most recent iteration. The search space is iteratively expanded in a direction given by one of the best matching positions. and the iterations are determined by template matching with the template between the At least one sample in the search space of the most recent iteration is 16. The method of claim 15, wherein the method is terminated when the user is outside of a window.

17. The interpolation filter interpolates each horizontal line or is on a vertical line, K integer samples either horizontally or vertically. The method of claim 14, wherein the filter is a one-dimensional filter that evaluates:

18. The search space is: - on a horizontal line of integer samples adjacent to the top or bottom of said subwindow in or - a vertical line of integer samples adjacent to the left or right side of the subwindow Is it on the 20. The method of claim 17, further comprising any of the fractional positions outside the subwindow.

19. Coding method for coding a video image divided into predictive blocks into a bitstream - Patents.com It is a law, Determining a motion vector for a prediction block according to any one of claims 11 to 18 Steps and The predicted block is provided at a position based on the motion vector determined as the predicted block. and encoding the difference between the predicted motion vector and the initial motion vector. and generating a bitstream including the bitstream.

20. Decoding method for decoding a video image divided into predictive blocks from a bitstream - Patents.com And, The initial motion vector, the predicted block, and the refined motion vector are specified. The coded difference between the predictor given by the prediction block and the position of the parsing from the bitstream; The refined motion vector for the prediction block according to any one of claims 11 to 18. determining a vector; The analyzed difference and the predicted motion vector are then applied to the location specified by the refined motion vector. Reconstruct the predicted block as a sum of the predictor given by the measured block and

21. When executed on a processor, the processor is caused to A computer readable medium storing instructions for performing the method described herein.

Citation Information

Patent Citations

  • Motion information derivation mode determination in video coding

    US20160286230A1

  • Low memory access motion vector derivation

    WO2012125178A1

  • Method and apparatus of decoder side motion derivation for video coding

    WO2017036414A1

  • Image encoding / decoding method and device, and recording medium having bitstream stored thereon

    WO2018097693A2

Cited By

  • Display device

    CN121712215A